GRADE: Assessing Evidence and Recommendations

1. What is GRADE?

The GRADE (Grading of Recommendation, Assessment, Development and Evaluation) approach was developed to create a single, unified system for rating the quality of evidence and the strength of recommendations. Its key features are transparency and a systematic process. Major health organizations, including the World Health Organization (WHO), use GRADE to develop clinical practice guidelines.

To get a clear overview of the basic concepts, let's start with a short video.

As the video explained, the GRADE process has two distinct but related parts:

1. Assessing the Certainty of the Evidence: How confident are we in the estimate of an intervention's effect?
2. Determining the Strength of the Recommendation: Based on the evidence and other factors, should we recommend for or against the intervention?

Let's break down each part.

2. Part 1: Assessing the Certainty of Evidence

Certainty of evidence is categorized into four levels: High, Moderate, Low, and Very Low.

The process for determining this rating involves a clear starting point and a set of factors that can either lower or raise our confidence.

Step 1: The Starting Point

The initial certainty level is based on the study design of the evidence being evaluated:

• A body of evidence from Randomized Controlled Trials (RCTs) starts as High certainty.
• A body of evidence from Observational Studies starts as Low certainty.

This starting point is just a preliminary grade. We must then consider several factors that can change this initial rating.

Step 2: Factors that Downgrade Certainty

There are five key domains that can lower our confidence in the evidence. If a body of evidence has serious problems in one or more of these domains, we "downgrade" its certainty level (e.g., from High to Moderate, or Moderate to Low).

To understand these five factors in detail, please watch the following video. It provides clear explanations and visual examples for each domain.

Let's summarize those five critical downgrading factors, drawing from the video and other key resources.

Here is a breakdown of the five downgrading domains from the resources you just reviewed:

1. Risk of Bias: The studies have methodological flaws (e.g., no blinding, poor randomization). You learned how to assess this in our lesson on appraising RCTs. If most studies in a review have a high risk of bias, our confidence in the overall result decreases.

2. Inconsistency (or Heterogeneity): The results vary significantly from one study to another without a clear explanation. For example, if some studies show an Ayurvedic herb helps reduce pain, while others show it has no effect or even worsens it, the evidence is inconsistent.

3. Indirectness: The evidence doesn't directly answer your clinical question. This can happen in several ways:

   • Population: The studies were done on young adults, but your patient is elderly.
   • Intervention: The studies used a standardized extract, but in your practice, you use a whole-herb powder (churna).
   • Comparison: The studies compared the herb to a placebo, but you want to know if it's better than the standard conventional treatment.
   • Outcome: The studies measured a biomarker (e.g., C-reactive protein), but you are interested in a clinical outcome (e.g., patient-reported pain). This is using a "surrogate outcome."

4. Imprecision: The results are not precise, usually due to a small number of patients or events. This is reflected by a wide confidence interval (CI). For example, if a study reports that a treatment reduces symptoms by 15% (95% CI: 2% to 28%), the true effect could be very small (2%) or quite large (28%). This uncertainty reduces our confidence.

5. Publication Bias: As we discussed in the last lesson, studies with "positive" results are more likely to be published. If a systematic review is missing unpublished "negative" studies, its results will be overly optimistic. This is often suspected if a funnel plot is asymmetrical.

Step 3: Factors that Upgrade Certainty

For evidence from observational studies, which starts as "low" certainty, it's possible to upgrade the rating if certain conditions are met:

1. Large Magnitude of Effect: The observed effect is very large and consistent. For example, if an observational study shows that a particular diet reduces the risk of a disease by 80-90%, it's less likely that this large effect is due to confounding alone.
2. Dose-Response Gradient: There is evidence that a larger dose or longer duration of the intervention leads to a greater effect. For example, if studies show that taking 1 gram of an herb has a small effect, while 3 grams has a large effect, this increases our confidence.
3. Effect of Plausible Confounding: If all likely confounding factors would have biased the results towards finding no effect, but the study still found an effect, this can increase our confidence.

3. Part 2: From Evidence to Recommendations

Assessing the certainty of evidence is only the first step. To make a clinical recommendation, we must consider other factors. GRADE uses the Evidence to Decision (EtD) framework to do this transparently.

A guideline panel making a recommendation will consider:

• The certainty of the evidence for benefits and harms.
• The balance between those desirable and undesirable effects.
• Patients' values and preferences. Do patients care more about the potential benefit or the potential side effects and cost?
• Resource use (costs).
• Equity, acceptability, and feasibility.

This is particularly relevant for Ayurvedic medicine. A treatment might have low-certainty evidence, but if it is also very safe, inexpensive, and aligns with a patient's cultural and personal values, a panel might still make a weak recommendation for its use.

4. Strength of Recommendation

After weighing all the EtD criteria, the panel makes a recommendation, which can be either Strong or Weak.

• Strong Recommendation: ("We recommend..."). Confidence is high that the desirable effects outweigh the undesirable ones. For most patients, this is the right course of action.
• Weak Recommendation: ("We suggest..."). There is uncertainty in the balance of effects, or the best choice depends heavily on the patient's situation, values, or preferences. This calls for shared decision-making between the practitioner and the patient.

It is crucial to understand that the strength of the recommendation is not the same as the certainty of the evidence. You can have high-certainty evidence of a very small benefit, leading to a weak recommendation because the benefit might not be worth the cost or side effects.

5. Applying GRADE to Traditional Medicine: An Example

The article "GRADE Methods in traditional medicine" provides a perfect example of how these concepts are applied to a non-pharmacological therapy.

Let's break down that table (Table 2 from the article):

• Outcomes: The panel chose three critical/important outcomes: Pain, Function, and Hematoma (an adverse event).
• Study Design: The evidence for all three comes from RCTs, so it starts as High certainty.
• Quality Assessment: For the Pain and Function outcomes, the evidence from 2 RCTs (with 520 patients) was not downgraded. The authors judged there were no serious issues with limitations (bias), inconsistency, indirectness, imprecision, or publication bias.
• Quality (Certainty): Therefore, the final certainty rating remains ⊕⊕⊕⊕ HIGH.
• Summary of Findings: We see the absolute effect. For pain, the effect size was a standardized mean difference (SMD) of -0.52. This is generally considered a moderate effect.
• Conclusion: Based on this table, a guideline panel would see high-certainty evidence that acupuncture provides a moderate improvement in pain and function with a slightly increased risk of minor hematoma. This would likely lead to a strong recommendation for using acupuncture for knee osteoarthritis.

This example shows the power of GRADE to transparently summarize a large amount of information into a clear assessment of confidence, which can then guide clinical decisions. The paper also highlights a key challenge you will face: for many traditional therapies, the evidence base is weak, meaning many outcomes will be rated as low or very low certainty. This doesn't mean the therapy is ineffective, but it does mean more high-quality research is needed.

Conclusion

You have now learned to apply the GRADE framework, one of the most important tools in modern evidence-based practice. This moves you from being a consumer of research to a sophisticated evaluator of it. You can now not only appraise a single study but also assess an entire body of evidence and understand the rationale behind clinical recommendations.

Key Takeaways:

• GRADE is a transparent framework for rating the certainty of evidence (High, Moderate, Low, Very Low) and the strength of recommendations (Strong, Weak).
• The certainty rating begins with the study design (RCTs start high, observational studies start low) and is then modified by five downgrading factors (risk of bias, inconsistency, indirectness, imprecision, publication bias) and three upgrading factors.
• Recommendations are based not only on evidence certainty but also on the balance of benefits/harms, patient values, and resource use, as outlined in the Evidence to Decision (EtD) framework.
• Applying GRADE to Ayurvedic research will be essential for identifying both promising therapies and areas where evidence is currently weak and more research is desperately needed.

Preview of the Next Lesson:
This lesson concludes Module 7. You have now journeyed from appraising single trials to synthesizing evidence and judging its overall quality. Our next lesson begins Module 8, "Translating Ayurvedic Concepts into Scientific Frameworks," where we will start by exploring how to correlate the concept of Agni with modern understandings of digestion and cellular metabolism. You will see how the research skills you've just acquired are essential for building these bridges.