Bradley C. Johnston on understanding GRADE methodology to reliably inform nutrition practice and policy\” />
Photo by Vitalii Pavlyshynets on Unsplash
This article is Sponsored Content by Micah Williams
In an era where nutrition advice seems to often shift with each new study, Bradley C. Johnston and fellow researchers at EvidenceBasedNutrition.org are working to bring consistency and rigor to how nutrition evidence is evaluated and applied in clinical and public health practice. Their recent work addresses the need for standardized approaches to assessing research quality—a challenge that affects both healthcare providers, patients and members of the public seeking reliable dietary guidance.
The problem with inconsistent evidence assessment
The nutrition field faces a predicament related to shifting public and clinical advice that Bradley C. Johnston and his colleagues promulgate as an easy fix, a fix related to the certainty of evidence and strength of recommendation. Perhaps the best example people can relate to is the broad public health advice to lower fat intake beginning in the US in the 1960s through to 2015, when the US Dietary Guidelines moved away from a cap on fat intake (e.g. 20% to 35% of daily calories). This advice, further promulgated by guidelines around the world, did not initially differentiate saturated from unsaturated fat. Moreover, to this day, the advice has not differentiated the certainty of evidence (i.e. very low, low, moderate, high) or the strength of recommendations, either strong or conditional (weak). If guideline advice had come clean on the low certainty of evidence, and made a conditional or weak recommendation using Grading of Recommendations Assessment, Development and Evaluation (GRADE) methods, there would have been less controversy, no need for the 2015 walkback, and industry would have been far less likely to develop a plethora of low fat or no fat products, replacing satiating fat with refined carbohydrates including added sugars. Similar and ongoing nutrition debates continue to exist, including on the optimal amount of protein intake per kilogram of body weight.
Related to translating evidence to care, the recent literature review by Johnston and colleagues found no studies among dietitians that have studied or reported on their competencies in interpreting the absolute magnitude (size) of effect for health outcomes, the certainty of evidence in effect or the strength of recommendations, competencies essential for optimising informed clinical and public health decision-making.
This gap becomes concerning when considering the broader landscape of nutrition practice and policy. Studies have shown that skills in evidence-based practice (EBP) competencies have resulted in advice that is more stable over time and improved patient outcomes, yet their systematic review revealed no published evidence for important EBP competencies.
The research team’s systematic review, published in The Journal of Nutrition, examined evidence-based practice competencies among nutrition professionals and students based on available evidence from 12 studies published in just six countries. In addition to finding no studies reporting on competencies in interpreting effect size, certainty of evidence or strength of dietary recommendation, the overall quality of study reports was poor, and the questionnaires on competencies were predominantly self-perceived, as opposed to objective assessments.
GRADE as a solution
Bradley C. Johnston has become a strong advocate for training students and health professionals in evidence-based practice, and for implementing the GRADE methodology when summarizing all the available nutrition research on a given clinical or public health question. This approach, which has been adopted by over a hundred organizations worldwide (e.g. American Academy of Nutrition and Dietetics, Cochrane, WHO), offers a standardized framework for evaluating evidence certainty and making recommendations.
The rationale for this standardization is straightforward. As Johnston and his co-authors explain in their introduction to the Nutrition Users’ Guides, if one wants to compare the effectiveness of various dietary changes, versus various physical activity interventions, versus statin drugs or some combination of these interventions for cardiovascular risk reduction, one can only do so optimally if the evidence for the different interventions is brought together and summarized using using a common standard, the common standard being GRADE methods. Otherwise, you are comparing apples with oranges, and you cannot make fully informed decisions. For example, along with over 50 interdisciplinary colleagues including researchers, clinicians, surgeons and those with lived disease experience, Johnston recently led the application of GRADE methods to make evidence-based clinical guideline recommendations for nutrition, physical activity, drugs like semaglutide and bariatric surgery for managing pediatric obesity.
The unique challenges of nutrition research
Johnston acknowledges that nutrition research presents distinct challenges that require specialized consideration within the GRADE framework. Nutrition evidence is unique in that randomized controlled trials (RCTs) may be absent for many clinical or public health nutrition questions, many nutrition RCTs lack genuine placebos and while data analysts and data collectors can be blinded, study participants cannot, RCTs of dietary interventions may be challenging to adhere to and may not be long enough to capture impacts on hard endpoints (e.g. mortality, stroke), and, unlike pharmaceuticals, many interventions aim to modify existing variables that participants are already exposed to. When there are low quality RCTs or RCTs are lacking, observational studies such as cohort studies are used to make recommendations, which when based on systematic reviews using GRADE methods start at low certainty.
These challenges have led some to argue for different evidentiary standards in nutrition research. However, Johnston and colleagues maintain that GRADE methodology can accommodate these unique aspects while maintaining standardized scientific rigor, especially if research methodology and nutrition experts are working together. While often not recognized, the GRADE method includes provisions that authors of systematic reviews based on observational studies may “rate up” the certainty of evidence in certain situations, resulting in low certainty evidence being upgraded to moderate or even high certainty evidence, when justified, particularly when there is a large effect (e.g. relative risk 2.0 to 5.0) or there is a valid dose-response gradient. For example, among adolescents with severe obesity there are many observational studies of bariatric surgery, most indicating large effects on weight loss and related cardiometabolic outcomes. A recent systematic review used to inform obesity management guidelines has rated the certainty of evidence up to moderate.
Educational initiatives and practical applications
Beyond identifying problems, as with obesity management guidelines Johnston has contributed to developing solutions. The Nutrition Users’ Guides work represents a collaborative effort across countries and among experts with specific expertise (e.g. study methodology, nutritional biochemistry, behavioral change) to provide structured guidance for evaluating the nutrition literature. Each article, accompanied by worksheets, addresses three key components when assessing different study designs used to assess nutrition interventions or exposures, including (1) assessing the methodological quality of the study, (2) interpreting study results (magnitude and precision of treatment or exposure effects for outcomes of benefit and harm) and (3) applying (i.e. generalizing) the results to unique patient or population scenarios.
With respect to studies that are systematic literature reviews or practice guidelines, the practical implications of this work extend directly to clinical practice and policy. During a recent interview, Johnston emphasized the importance of transparency in literature reviews and guidelines. He explained that GRADE methods when applied to literature reviews, if used correctly, are intended to make everything transparent. When we rate the certainty of evidence or strength of recommendation, there are footnotes and details about how we came to conclude that, for example, there was high, moderate, low or very low certainty evidence for the potential benefits and harms of an intervention. With transparent and publicly available notes on the determinants of the certainty, others are then free to see the logic and agree, quibble or disagree. Good science is about reliable and transparent methods, curiosity, conversation with colleagues and following the evolving science.
Evidence-based practice competencies
While the systematic review on EBP competencies did not assess training among nutrition students and professionals, Johnston, based on his observations in the nutrition field for the last 7 years has suggested the need for standardized training, which in his view would be advantageous in the field. While there is EBP training, according to the guidance from varying governing bodies in various countries, the training requirements vary a lot, and requirements are often vague allowing schools and programs to differ greatly in how competencies are taught and tested.
The implications extend beyond individual practitioners to the profession as a whole. According to Johnston, competency training needs to be more specific and standardized, including more questions on national board exams. This would help uplift the profession by allowing registered dietitians, for example, to be leaders among their allied health professionals, allowing them to better communicate the evidence to their clients, while better promoting informed and shared decision-making between clients, dietitians and allied medical colleagues.
A vision for establishing new standards for practice and policy
Through research, educational initiatives, and advocacy for methodological rigor, Johnston and colleagues aim to change how nutrition professionals are trained to interact with research evidence and how they communicate the evidence to clients, patients and to the public. The Nutrition Users’ Guides series of articles, education on GRADE methods and work to improve guideline methods will empower clinicians, health service workers, and health policymakers to better understand the validity, interpretability, and applicability of the nutrition literature, while also helping practitioners and their clients make shared evidence-based decisions.
