“Really pissed off”: Cochrane receives backlash from advocates and experts after abandoning ME/CFS review

For decades, Cochrane — formerly called the Cochrane Collaboration — has been known internationally for its systematic reviews of medical treatments and interventions. Now the U.K.-based charity is under fire for its clumsy handling of a thorny issue: the status of a flawed 2019 review of exercise therapy for myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS).

In recommending exercise as a treatment, the 2019 review angered many people with ME/CFS, who routinely experience relapses after excess activity — a phenomenon known as post-exertional malaise (PEM). (This story uses “ME/CFS” because those involved in the update have used the term and because it is currently the most commonly used term for what is believed to be a cluster of related illnesses.) 

In December, Cochrane blindsided people with ME/CFS — as well as those with Long COVID, many of whom also experience PEM — by abruptly abandoning a commitment to develop an updated version of the 2019 review. In addition, Cochrane republished the old review with a 2024 date, creating the false impression that it had, in fact, been updated. 

Many patients understandably viewed these actions as a betrayal of the promises Cochrane made when it published the 2019 version. At that time, Cochrane’s editor in chief, Dr. Karla Soares-Weiser, acknowledged in an editorial statement that the review was “based on a research question and a set of methods from 2002, and reflects evidence from studies that applied definitions of ME/CFS from the 1990s.” In other words, Cochrane committed to the update because its leadership was aware that the published review was inadequate for present purposes.

“I think patients are disappointed but not surprised, because they’re pretty used to being given the run-around by the authorities,” said Todd Davenport, a professor of physical therapy at the University of the Pacific in Stockton, California, and a member of the team appointed to write the updated review, including a new protocol.

“The net result of all this is that Cochrane has flash-incinerated their credibility, or what was left of it,” Davenport said.

The update project was already way behind schedule. Davenport and the rest of the writing team, which included patient representatives as well as researchers and Cochrane staff, had invested considerable time in drafting a new protocol. They submitted a final version to Cochrane in February 2023, after having received feedback on previous drafts. They heard nothing further until last month, when Cochrane sent out a curt message informing them that the project was disbanded.

Davenport was particularly disappointed because of his past respect for Cochrane, which was founded in 1993 and named after Archie Cochrane, a Scottish physician and advocate for randomized controlled trials and systematic reviews. Working with thousands of volunteer experts around the world, the organization has produced many hundreds of systematic reviews. These reviews can be highly influential in clinical medicine, although in 2023 Cochrane received significant backlash over its inept handling of a controversial review of masks and viral illness. 

“Ever since I was a student, Cochrane has been held up as this trustworthy, reliable source of information,” said Davenport.“If you had a clinical question, Cochrane was the place where you went to find an answer. This experience has really showed me the extent of the political games behind this supposedly objective process of trying to synthesize studies.”

Mary Dimmock, a patient advocate and another member of the writing team, said she was “really pissed off” at Cochrane’s actions. “We put a lot of work into doing this in good faith, and they just pulled it with no consideration for the patients,” she said.

We put a lot of work into doing this in good faith, and they just pulled it with no consideration for the patients.

Mary Dimmock

Sharp criticism from a Cochrane insider

Last month, Hilda Bastian, a Cochrane co-founder and longtime insider who was involved in the review-update effort, launched a new round of attention to the matter with a blistering critique of the organization’s actions. “After all those months and years of delays, there was something particularly cruel about dumping all this on a patient community in the week before Christmas,” wrote Bastian, an Australian consumer health advocate, on her blog, Absolutely Maybe, on the PLOS network.

The events have triggered some negative media attention for Cochrane, with recent articles about the kerfuffle posted by Retraction Watch, a popular science site, and The BMJ. A German journalism cooperative, RiffReporter, published a critical piece headlined (via Google Translate) “ME/CFS: New dispute over potentially harmful activation therapy shakes Cochrane network.” 

Meanwhile, an online petition seeking retraction of the 2019 review picked up new signers, with the total now more than 15,000.

(I have criticized the 2019 review and called for its retraction; I have also criticized Cochrane’s actions throughout this episode. The online petition was organized by the Science for ME forum; I am a member of the forum but was not involved in the petition.) 

Cochrane had tapped Bastian to lead an independent advisory group (IAG) of stakeholders to help guide development of the updated exercise review. The IAG includes patient representatives, healthcare practitioners, and researchers, including some with long experience with Cochrane systematic reviews. In Bastian’s blog post, she expressed distress that the organization she helped create had behaved in such a callous and dismissive manner toward a vulnerable patient population, especially one so prone to setbacks in the form of PEM.

 “As serial bad news came from Cochrane across that week [in December], the distress and emergency emails and calls and meetings had predictable results for some: They crashed,” she wrote in her post.

“I don’t know how to put my sorrow and anger about this into words,” she added.

The Cochrane exercise review is about ME/CFS, not Long COVID. However, a significant percentage of people with Long COVID experience PEM and meet diagnostic criteria for ME or ME/CFS. The investigators and clinicians who promote treatments like graded exercise therapy (GET) and cognitive behavior therapy (CBT) for ME/CFS, and for Long COVID as well, routinely cite the 2019 review as evidence to back their claims. According to Cochrane, the review has been cited 119 times, including in World Health Organization guidelines on “clinical management of COVID-19.”

The GET/CBT approach loses ground

The GET/CBT approach to ME/CFS was considered the standard of care for decades. GET was designed to target the purpoted physical deconditioning said to be generating the disabling symptoms, CBT to address patients’ “unhelpful” beliefs of having an ongoing illness, which were said to prevent them from engaging in activity. The largest study testing these treatments, the PACE trial, published its main results in The Lancet in 2011 and was hailed as a success by GET/CBT proponents. It was also the largest trial included in the 2019 Cochrane review.

In recent years, however, the GET/CBT defenders have lost significant ground. The PACE trial has been widely rejected for its many methodological and ethical lapses. In 2021, the UK’s National Institute for Clinical Excellence published its own new guidelines for ME/CFS, which rescinded the agency’s prior recommendations for GET and for CBT as curative treatments. Similarly, the U.S. Centers for Disease Control and Prevention no longer recommends the two interventions for ME/CFS. 

The dispute over the 2019 exercise review is part of a larger concern about Cochrane’s overall approach to ME/CFS. From the start, the illness — previously called CFS or CFS/ME — was under the purview of Cochrane’s “common mental disorders” group. (That group is no longer active.) Cochrane’s output in this field, which also includes a review of CBT for “chronic fatigue syndrome,” has invariably reflected these biases, as have many or most of the trials included in the reviews. 

In the case of the 2019 exercise review, the included trials were all unblinded and relied on subjective outcomes — a study design that is far more likely to generate bias than trials that are either blinded or rely on objective outcomes. The trials also did not generally require the presence of PEM, making it hard to determine whether the samples accurately reflected the ME/CFS population. Some of the trials, among them PACE, included objective outcomes with poor results that were left out of the Cochrane review. 

The update process was expected to take about two years. The pandemic inevitably hampered the process, as did funding and related issues at Cochrane. Nonetheless, the reasons offered by Cochrane for its decision to abandon the entire effort didn’t add up.

In the editorial statement announcing the end of the project, Cochrane stated, “due to insufficient new research in the field and a lack of resources to oversee this work, the update will not be proceeding.”

However, the commitment to revise the review had nothing to do with the availability of new evidence. Moreover, the statement about “insufficient new research” overlooks a major pediatric study of GET that reported poor results last year. Incorporating those findings into a new review would have undermined the claims of potential benefits from exercise. (To be fair, the 2019 review was not designed to include pediatric studies.)  As far as resources, it is unclear why Cochrane would have suddenly decided it did not have the bandwidth to handle the review after sitting on a draft protocol for almost two years.

The pressure on Cochrane is likely to continue. In addition to Bastian’s blog post and a letter from the IAG charging Cochrane with violating its own policies and expressing “dismay and concern,” Davenport and Dimmock are planning to send their own letter and are seeking an opportunity to publish their draft protocol. On social media and patient forums, speculation has swirled about which powerful supporters of the GET/CBT paradigm might have forced Cochrane’s hand.

In its public responses, Cochrane has remained steadfast. As a spokesperson told Retraction Watch, “We have no plans to revisit this decision.” So it appears that Cochrane’s actions are, from its perspective, irreversible. But unless it acts quickly to clean up this mess, the long-term damage to its reputation could be irreversible as well.  

It appears that Cochrane’s actions are, from its perspective, irreversible. But unless it acts quickly to clean up this mess, the long-term damage to its reputation could be irreversible as well.

---

Readers can sign the petition calling on Cochrane to withdraw the 2019 review at Change.org.

David Tuller is a senior fellow in public health and journalism at the Center for Global Public Health at the University of California, Berkeley. His academic position is supported by crowdfunded donations to UC Berkeley, largely from people with ME/CFS and, more recently, Long COVID.

All articles by The Sick Times are available for other outlets to republish free of charge. We request that you credit us and link back to our website.