In February, the National Institutes of Health (NIH) published a controversial paper that aimed to characterize myalgic encephalomyelitis (ME), also known as chronic fatigue syndrome (CFS). The study, published in Nature Communications, took eight years and cost $8 million. Participants flew in from all over the country and went through intense testing on NIH’s Bethesda, Maryland campus.
The paper highlights results from one test in particular called the “Effort Expenditure for Rewards Task” (EEfRT). Authors claimed that a “defining feature” of ME/CFS “was an alteration of effort preference,” and not fatigue from the muscles or the nervous system.
Like many others, I was taken aback when I read this. It struck me as another way to psychologize a physical disease — and it didn’t match my experience or any other of the hundreds of patient stories I’ve followed.
I used to ride 100-mile mountain bike races, and know what it feels like to be fatigued and want to quit but just keep pushing. I’d read the stories from former athletes, who try everything to get back to exercise but are almost always shut down by post exertional malaise (PEM), sometimes called post-exertional symptom exacerbation (PESE), at each attempt. Instead of exercise making their bodies stronger, it now makes them weaker and less functional. In these reports and my own experience, effort is never impaired. It is the ability to recover that is broken.
I’ll outline the reasons the conclusion is not justified and argue that the “effort preference” finding is deeply flawed and worthy of retraction. A misinterpreted finding from a 15-minute computer game, developed for depressive disorders, on a tiny sample of 15 sick participants should not have made it past Nature Communications’ reviewers as a key finding of this paper.
The EEfRT is a game in which researchers give participants a choice between an easy task and a hard task. In the hard task, the participant would attempt to press a button 98 times in 21 seconds with their non-dominant hand’s little finger; in the easy task, they press a button 30 times in seven seconds with the dominant hand’s index finger. Participants make this choice and do the tasks repeatedly over 15 minutes. Participants were told that they would receive a very small financial incentive for taking part in this specific section of the study.
According to the study authors, the 17 “healthy volunteers” who participated in the test “chose more hard tasks” than the 15 people with ME/CFS who participated. This finding is insignificant and should be retracted. Researchers should not spend any further resources on tests like this in ME/CFS or Long COVID.
When The Sick Times editor Betsy Ladyzhets asked NIH Director Dr. Monica Bertagnolli about criticisms of this study, including the “effort preference” finding, in an interview last month, Bertagnolli said the agency was taking concerns “very seriously.” The Director’s Office later confirmed that the NIH is “reviewing the study to determine if the methods justified the conclusion.”
1. Validated methods to measure maximal effort in ME/CFS already exist.
The EEfRT test was designed to explore “effort-based decision-making” in depression and was validated in college students. However, the test has not been validated for ME/CFS. Instead, people with ME/CFS can be tested for effort with Cardiopulmonary Exercise Tests (CPETs).
In a CPET, a participant rides an exercise bike; scientists measure their heart and lung function as biking intensity increases, until the participant can no longer turn the pedals. CPETs can objectively determine when a person is maximally exerting with a metric called respiratory exchange ratio (RER), an indication of hard physical effort and anaerobic respiration.
“We don’t need to gamify [testing effort],” said Todd Davenport, a professor of physical therapy at the University of the Pacific, who studies ME/CFS. In other words, the researchers should have used well-established CPET tests rather than a different test never previously used for ME/CFS.
In fact, the NIH study did look at this measure in eight people with ME/CFS — and found seven out of those eight people reached peak RER. If an objective test shows participants are maximally exerting, then there can be no “effort preference,” or a “mismatch between what someone thinks they can achieve and what their bodies perform,” as study author Brian Wallit has claimed.
Additionally, the NIH researchers only conducted the CPET on one day, rather than doing it over two days as is standard for ME/CFS research. Two-day CPETs are important in ME/CFS research because they are the only objective marker that shows reduced power and oxygen extraction on the second day due to PEM. NIH has still not responded by including the two-day CPET in similar work they are doing on Long COVID. The lack of a second test allowed the NIH to claim that physical and central fatigue were not features of ME/CFS.
2. The EEfRT finding fails to take into account that people with ME/CFS are sick and have a lower physical ability than the healthy controls.
People with ME/CFS successfully completed the hard trials they attempted in the EEfRT test at a much lower rate than the healthy controls. Controls completed 96% of the easy trials and 99% of the hard trials, while people with ME/CFS completed 98% of the easy trials but only 65% of the hard trials. Exactly as one might expect, the ME/CFS group, suffering due to weeks of intense testing, wasn’t able to complete the hard tasks reliably.
The EEfRT is explicitly not supposed to be used in these circumstances. It is designed to measure differences in people’s motivations to complete tasks, not differences in ability or fatigue, the test’s creator, psychologist Michael Treadway, and colleagues wrote in a paper introducing it.
In an email, Treadway told me it is “important to confirm that both groups were able to complete the easy and hard tasks at equally high rates.” The ME/CFS participants’ lower completion rate for hard tasks could suggest those participants made a “rational decision” based on their ability, rather than a difference in “effort preference,” he said. Furthermore, he stressed the importance of individually calibrating button pressing speed to distinguish preference from ability, which NIH failed to do.
3. The EEfRT, by design, is a problematic test that can be ‘gamed’ even when administered appropriately.
One of the healthy controls “gamed” the test by intentionally failing easy, low-value tasks and instead focused on high-value, hard tasks. The NIH researchers excluded this participant’s data from the analysis, claiming they “did not follow instructions.” Yet if these results are added back in, there is no significant difference between the ME/CFS and control participants.
If one outlier, who technically had the lowest “effort preference” but actually made the most money by playing the optimal strategy, can reduce your finding’s significance to zero — then the “findings” were never there. This is an example of “p-hacking,” or scientists manipulating data (such as excluding participants) until it produces statistically significant results.
4. If “an alteration of effort preference” is “one defining feature of PI-ME/CFS,” then why is there a large overlap of healthy controls and ME/CFS participants?
This chart shows the proportion of “hard tasks” chosen by the participants compared to “easy tasks.” The broad overlap between the healthy controls and ME/CFS participants demonstrates that participants scored similarly across the two groups. How can altered “effort preference” be a “defining feature” of ME/CFS if over a third of the ME/CFS cohort doesn’t have it and over a third of the controls do?
5. “Effort preference” flies in the face of clinical reality for people with ME/CFS.
At a ME/CFS clinic I was told, “Your best days are your worst days” from a nurse who had worked there for 20 years. She referred to a trend in which, on their best days, many patients feel better — and exert too much and go right back into a PEM state. Patients know that strict pacing is often harder than staying still.
Dianna Cowern, a popular YouTuber who runs the Physics Girl channel, tried to hike a long-distance trail in Scotland in the early stages of Long COVID before becoming bedbound. Oonagh Cousins, who got Long COVID in the first wave, was an Olympic-bound rower. After rest and some degree of recovery, she went back to full-time training only to relapse. It led to her giving up her slot on the British rowing team.
These stories, my own, and many others all have similar elements of a large exertion or immune challenge followed by temporary or permanent deterioration which “effort preference” cannot explain. There is no subconscious decision not to exert, but the common theme is that recovery from exertion is impaired.
There is no subconscious decision not to exert, but the common theme is that recovery from exertion is impaired.
Calling on the NIH for retraction
Given these problems, the NIH must retract everything related to the “effort preference” portion in the paper. The patient community needs closure so we can focus on moving science forward.
I’d rather be spending my time advocating for money for forward-looking research and clinical trials. But when paper-thin findings, from one of the largest research organizations in the world, in a tiny sample, are misrepresented and used to broadly define millions of people, those patients have a right to speak up.
Furthermore, NIH should stop any further research into “effort preference.” To make up for this insult, the NIH should fund a large extramural research study led by the ME/CFS Research Roadmap authors that includes meaningful tests, like the two-day CPET, the nanoneedle, extensive autonomic testing, MicroRNAs, metabolomics, WASF3 protein, and markers for Mast Cell Activation Syndrome.
People with ME/CFS have had to fight battles all over the world for decades to stop precious research funding from being used for psychosomatic research. NIH should instead be signaling to the world that this type of psychosomatic research is unacceptable.
John Bolecek is a father of two young boys, a husband and a former bicycle and pedestrian transportation planner who has been disabled by Long COVID since early 2022. When he has energy he advocates for Long COVID and ME/CFS research and writes articles.
Acknowledgements: Thanks to the members of the Science for ME forum for the chart, the extensive analysis, and discussion of the EEfRT, and also to Jeannette Burmeister, attorney and longtime ME advocate, who wrote a four-part series titled “The NIH Intramural ME Study: “Lies, Damn Lies, and Statistics” where many of these issues and more are discussed in detail. To join Jeannette’s call to action for the paper’s retraction visit the bottom of part four. This article would not be possible without their work.
All articles by The Sick Times are available for other outlets to republish free of charge. We request that you credit us and link back to our website.
Leave a Reply