Author Kevin Evans, Ph.D., evaluated the USPSTF's report methodology and found that it did not meet established standards for systematic reviews (JDMS, January/February 2010, Vol. 26:1, pp. 19-23). Evans is chair of the radiologic sciences division in the School of Allied Medical Professions at Ohio State University in Columbus.
Evans used two resources to evaluate the USPSTF's report: the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), a 27-point checklist, and the Assessment of Multiple Systematic Reviews (AMSTAR), an 11-point checklist.
The task force's report scored 7 out of 27 on the PRISMA checklist and 1 out of 11 on the AMSTAR list. These low methodological scores put in question the rigor used in developing the report, limiting it to a review of literature instead of a formal systematic review and reducing its overall scientific impact to a much lower level in the hierarchy of evidence, according to Evans.
"I picked two of the most well-known methods to evaluate systematic reviews and applied them to the report," he told AuntMinnie.com. "It's possible that USPSTF met these standards but failed to provide their methodology in the report. This becomes problematic in reading their guidelines."
The USPSTF's intention was to update its 2002 report by using other systematic reviews, meta-analyses, recently published literature, and data from the Breast Cancer Surveillance Consortium from 2000 to 2005. In its guidelines, it proposed the following, according to Evans:
- Routine screening mammography in women ages 40 to 49 years should not be conducted; rather, this process should be biennial at ages 50 to 74 years.
- A lack of published evidence currently exists to provide a guideline for screening mammography for women older than 75 years of age.
- A lack of evidence exists for assessing the benefits and harms of using clinical breast examinations for women 40 years and older.
- Self breast examination is not recommended to be taught to women by clinicians, as it is not a sensitive technique and raises a woman's level of anxiety.
- A lack of published evidence currently exists to provide a guideline about benefits and harms associated with digital mammography or MRI instead of film-screen mammography.
USPSTF used data from film-screen mammography in its report, rather than taking into consideration that digital mammography was developed to address film-screen's limitations and is in widespread use, according to Evans. In fact, one of the puzzling things about the USPSTF report is its lack of any reference to the American College of Radiology Imaging Network (ACRIN) Digital Mammographic Imaging Screening Trial (DMIST), conducted in 2005.
"USPSTF didn't make specific mention of DMIST," he said. "And yet they claim that more evidence is needed to provide a guideline about benefits and harms associated with digital mammography, instead of film-screen mammography."
"Other U.S. Preventive Services Task Force reports are routinely high quality," Evans said. "It's possible that the breast cancer screening task force did a good job but didn't spell out their methods. In any case, their report has created confusion for everyone, as well as our government officials."
If the U.S. Department of Health and Human Services had addressed the guidelines point by point, this confusion might have been put to rest earlier, Evans said.
"[After the guidelines were released], the Department of Health and Human Services responded by telling the public not to pay attention," he said. "They should have asked the USPSTF to provide an addendum with additional details on their review."
In the aftermath of the report's publication, USPSTF should take several steps to clear up the confusion, Evans wrote: It needs to provide an addendum that details its methodology, and if a systematic review as outlined by PRISMA or AMSTAR has not been conducted, it needs to be done.
"A revised set of guidelines is needed to assist patients in making the best decision about participating in screening breast examinations," he concluded.
