Questions to ask when evaluating evidence from clinical trials

By Simon Spichak

This resource is intended to help readers interpret results from and communications about clinical trials for Long COVID and related diseases. It accompanies Simon Spichak’s July 2026 story, “How to interpret clinical trial results for Long COVID.”

Table of contents

What stage is the treatment tested in?

Study type or trial phaseDescriptionLimitations
Preclinical Preclinical studies use animal models (in vivo) or cells in a dish (in vitro) to test potential treatments. Even if the results at this stage are overwhelmingly positive, there is no guarantee that the effects in cells, rodents, or other animal models will replicate in humans.  
Case reports and observational studiesThese studies might provide a signal that a drug or treatment is worth exploring. While scientists may use statistical methods to see how sociodemographic factors and other health conditions impact health alongside a treatment, these studies aren’t randomized and don’t have a control group. That means even treatments that look very promising often don’t pan out.
Phase 1 or Phase 1/2Early stage trials that look at safety and sometimes dosingThese studies do not determine effectiveness. Most treatments that succeed in Phase 1 don’t end up working.
Phase 2Larger trials that look for indications that a drug might work.Phase 2 trials cannot prove the effectiveness. Large studies are still required for confirmation. Many times evaluate safety of a treatment.
Phase 3Large studies conducted across multiple hospitals or research centers that pits a new drug or medical device against a placebo. They’re designed to prove effectiveness and provide enough data for regulatory approval.
Even if a study is positive, it isn’t always clear whether the benefits of a treatment are meaningful. 
Phase 4Large studies conducted to test whether supplements or approved treatments could be repurposed.Even if a study is positive, it isn’t always clear whether the benefits of a treatment are meaningful.

Trial glossary

Look for these terms in the methods section to understand the trial design.

Term DefinitionImplication
RandomizedThe researchers randomly divided up the participants into different treatment arms.Help evenly distribute outside factors like demographic differences between the two groups, eliminating bias that could affect study results. 
Placebo or sham controlOne group of participants receives a treatment while the other group receives something that looks and feels like a treatment but is biologically inert.Many people may show improvement on some scales simply because they take part in a trial. Placebo and sham arms help control for these biases. 
Open-labelThe participants know that they’re receiving treatment. This introduces biases that make it challenging to draw causal conclusions about the treatment’s effects, especially in Long COVID where symptoms and severity may fluctuate over time.  
Single-blindedParticipants are unaware if they are receiving a treatment or a placebo/sham.The experimenters who are assessing the outcomes are privy to this information, introducing bias to the results.
Double-blindedNeither the participants nor the experimenters know which participant is receiving a certain treatment.Considered a gold-standard for reducing bias in trial outcomes.

Trial reporting

Was the study registered on ClinicalTrials.gov? 

Check that the Study Registration Date occurs before the first trial participant is recruited. For example, the registration page for a clinical trial of oxaloacetate for fatigue in Long COVID includes that the study was registered in April 2023 and began recruiting in June 2023.

Is the primary outcome, as outlined in the trial registration, statistically significant?

The primary outcome is the main metric that the researchers have chosen to determine if the treatment is helping participants. Studies may also include secondary and exploratory outcomes, which provide more information about the treatment’s effects.

If not, then even if secondary or exploratory outcomes are statistically significant, there isn’t enough evidence to say there might be an effect in the trial. 

Is the study reporting on an outcome within the treatment group or is it comparing between the treatment and the control group? 

A study needs to show that participants receiving a new treatment not only do better over time, but show more improvements than a control group. Sometimes, studies will say that a treatment led to improvements because the people in the treatment group improved. But that’s an invalid comparison if it’s not weighed against a control group. This is important in Long COVID where symptoms may fluctuate and a small proportion of individuals experience spontaneous recovery.

If the result is statistically significant, does it meet the threshold for a minimally clinically important difference?

Any effects below this threshold for a specific outcome measure might not be noticeable to an individual or their clinician.

Outcome measures and subgroups

How is Long COVID defined? Are there any pre-set subgroups listed in the Trial Registration?

This information lets you know who is included in the study. People with Long COVID who only have organ damage may be different from those who may only report post-exertional malaise and different from those with immune-related deficits and post-exertional malaise. Understanding who is included also allows you to compare this study to other studies that measured a similar treatment. 

Subgroups of patients should be predefined, to reduce the changes of false positives and subgrouping participants after receiving the data to try and find a positive result. 

Are the outcome measures validated for Long COVID or a specific pathology?

Some outcome measures, like many of the fatigue or quality of life scales, are general surveys that aren’t very informative. Some patient derived surveys like FUNCAP, and other measurements designed to see whether a drug changes underlying pathology, are more helpful to show a drug works. For example, an anticoagulant should show it reduces the levels of coagulation in the blood or improves blood flow alongside symptomatic improvement.

Who is funding the study and how does that funding impact the trial? 

Look at the bottom of the study to find a list of funders. Studies funded by government initiatives may use survey-based outcome measures that aren’t always reliable. Take studies funded heavily by pharmaceutical, supplement, or medical device companies with an extra grain of salt. 

How do the results compare to other trials testing the same treatment?

Even though some studies are weaker, it’s important to look at all of the evidence rather than cherry-picking studies. Take the results of new research in the context of previous work to evaluate how well a treatment may or may not work. 

Is the mechanism of the treatment scientifically plausible in the first place?Some products marketed directly to people with chronic illnesses don’t have any validity. For example, vagus nerve stimulator devices that are placed over the skin on the neck purport they activate the nerve to exert immunomodulatory effects. Scientists are highly skeptical that non-medical grade devices work as advertised, and manufacturers don’t provide any mechanistic proof that the electrical stimulation is enough to activate the vagus nerve.     

Do the results hold up?

Take a look at PubPeer, news coverage and trials that interview scientific experts that aren’t involved in the research, and forums like Science for ME to get more perspectives on how the results hold up and whether parts of the trial are problematic.