Review of Clinical Guideline Methodology

In November 1989, Congress amended the Public Health Service Act to create the Agency for Health Care Policy and Research (AHCPR). In turn, AHCPR requested advice from the Institute of Medicine (IOM) about how the agency might approach their new and challenging responsibilities for practice guidelines. Currently, the AHCPR has transformed into the Agency for Healthcare Research and Quality (AHRQ), which maintains guidelines online, including links to the National Guideline Clearhinghouse (NGC) and to other guideline agencies and resources, including an annotated bibliography on guideline developement methodology.

One of the early results of the AHCPR effort was the report “Clinical Practice Guidelines: Directions for a New Program” (Field and Lohr, 1990). A major conclusion of this report was that clinical guidelines should be founded on eight key principles.

• Validity — Practice guidelines are valid if, when followed, they lead to the health and cost outcomes projected for them, other things being equal. A prospective assessment of validity will consider the projected health outcomes and costs of alternative courses of action, the relationship between the evidence and recommendations, the substance and quality of the scientific and clinical evidence cited, and the means used to evaluate the evidence.
• Reliability/Reproducibility — Practice guidelines are reliable and reproducible (1) if—given the same evidence and methods for guidelines development—another set of experts would produce essentially the same statements and (2) if—given the same circumstances—the guidelines are interpreted and applied consistently by practitioners or other appropriate parties. A prospective assessment of reliability may consider the results of independent external reviews and pretests of the guidelines.
• Clinical Applicability — Practice guidelines should be as inclusive of appropriately defined patient populations as scientific and clinical evidence and expert judgment permit, and they should explicitly state the populations to which statements apply.
• Clinical Flexibility — Practice guidelines should identify the specifically known or generally expected exceptions to their recommendations.
• Clarity — Practice guidelines should use unambiguous language, define terms precisely, and use logical, easy-to-follow modes of presentation.
• Multidisciplinary Process — Practice guidelines should be developed by a process that includes participation by representatives of key affected groups. Participation may include serving on panels that develop guidelines, providing evidence and viewpoints to the panels, and reviewing draft guidelines.
• Scheduled Review — Practice guidelines should include statements about when they should be reviewed to determine whether revisions are warranted, given new clinical evidence or changing professional consensus.
• Documentation — The procedures followed in developing guidelines, the participants involved, the evidence used, the assumptions and rationales accepted, and the analytic methods employed should be meticulously documented and described.

Occurring in parallel with government agency efforts on clinical guidelines, other efforts have been directed toward defining outcome-based competencies for healthcare. These efforts have resulted in a number of publications and guides. Recognizing that medicine is taking on a global face, the Institute for International Medical Education (IIME) has developed a body of work on Global Minimum Essential Requirements. This work is summarized in GMER Core Committee and Wojtczak and Schwarz (2000).

In an apparently coordinated effort with the Association of Medical Education in Europe AMEE, a number of papers have been published both in the journal Medical Teacher and as AMEE Guides. In particular, Harden (1999) introduces concepts of outcome-based education and Shumway and Harden (2003) introduce assessment of outcomes.

Also during this period, the Accreditation Council for Graduate Medical Education (ACGME) has created the Outcome Project, addressing outcome-based competency for physicians in the 21st century. A summary of the ACGME efforts on defining competency was published by Epstein and Hundert (2002) with an editorial comment by Leach (2002). The ACGME noted that standards for professional competence delineate key technical, cognitive, and emotional aspects of practice, including those that may not be measurable. They defined six areas of competence and some means of assessing them:

• Patient care (including clinical reasoning),
• Medical knowledge,
• Practice-based learning and improvement (including information management),
• Interpersonal and communication skills,
• Professionalism, and
• Systems-based practice (including health economics and teamwork).

Several other agencies provide extensive guidance on the guidelines development process, with a considerable level of inter-knowledge existing between efforts. The Austrialian National Health and Medical Research Council (NHMRC) has developed a protocol to guide the creation and implementation of clinical guidelines (NHMRC, 1999). They have derived nine basic principles for developing guidelines.

• Processes for developing and evaluating clinical practice guidelines should focus on outcomes. Outcome measures can range from survival rates to quality-of-life attributes.
• Clinical practice guidelines should be based on the best available evidence and should include a statement about the strength of their recommendations. Evidence can be graded according to its level, quality, relevance and strength. The ‘level’ of evidence refers to the study design used to minimise bias: the highest level involves a systematic review of randomised controlled clinical trials. ‘Quality’ refers to the methods used to minimise bias in the design and conduct of a study. ‘Relevance’ refers to the extent to which research findings can be applied in other settings. The ‘strength’ of evidence relates to the magnitude and reliability of the treatment effect seen in clinical studies: strong effects are more likely to be real and more likely to be clinically important. Ideally, recommendations would be based on the highest level of evidence, but this may be difficult to achieve in public health and social science interventions.
• The method used to synthesise the available evidence should be the strongest applicable. Taking the evidence—of whatever level, quality, relevance or strength—and turning it into a clinically useful recommendation depends on the judgment, experience and good sense of the group developing the guidelines. The fact of having evidence from a high-level study does not automatically result in a good clinical recommendation.
• The process of guideline development should be multidisciplinary and should include consumers. If guidelines are to be relevant, those who are expected to use them or to benefit from their use should play a part in their conception and development. Involving a range of generalist and specialist clinicians, allied health professionals, experts in methodology, and consumers will improve the quality and continuity of care and will make it more likely that the guidelines will be adopted.
• Guidelines should be flexible and adaptable to varying local conditions. They should include evidence relevant to different target populations and geographic and clinical settings, take into account costs and constraints, and make provision for accommodating the different values and preferences of patients.
• Guidelines should be developed with resource constraints in mind. They should incorporate an economic appraisal, which may be helpful for choosing between treatment options.
• Guidelines are developed to be disseminated and implemented taking into account their target audiences. They should also be disseminated in such a way that practitioners and consumers become aware of them and use them.
• The implementation and impact of guidelines should be evaluated.
• Guidelines should be revised regularly.

Recently, NHMRC (2005), the NHMRC has released a document to inform external persons and bodies of the procedures to be followed in developing guidelines which are intended for submission to the NHMRC for approval. This document includes flow-charts summarizing the process. In the UK, the National Institute for Health and Clinical Excellence has developed a manual on the guidelines development process (NICE, 2006). The AGREE Collaboration (Appraisal of Guidelines Research and Evaluation) has created the AGREE Instrument ((AGREE, 2003) to help unify the approach to guideline creation and implementation across Europe. The Scottish Intercollegiate Guidelines Network (SIGN) also provides guidelines and, as one of their guidelines, maintains the "Guidelines Developer's Handbook" (SIGN, 2004). The introduction the the SIGN handbook contains concise flow-chart summaries of the guideline development process and life-cycle.

There are a significant number of individual papers from different domains of health practices. Shekelle et al. (1999) discuss guideline development to ensure desired patient outcomes, based on a combination of the literature about guideline development and the results of their combined experience in guideline development in North America and Britain. Their paper considers the five steps in the initial development of an evidence based guideline. Thomas (1999) describes the development, implementation, and appraisal of clinical practice guidelines for nursing. She notes that while, "most of the development and evaluation of clinical guidelines has occurred in the field of medicine, nurses are becoming more interested in the use of guidelines as one means of facilitating evidence-based practice". Eccles and Grimshaw (2004) provide a succinct review of criteria for guideline selection and presentation. Cognitive factors affecting group consensus in creating guidelines were investigated by Raine et al. (2004). They note that "the nominal group technique is a method of eliciting and aggregating judgments in a transparent and structured way. It can provide important information on levels of agreement between experts. However, conclusions can be at odds with the published literature. If they are, reasons need to be explicit". Grilli et al. (2000) raise concerns about the quality, reliability, and independence of practice guidelines created by specialty societies. They note that such guidelines often fail to follow accepted methodological criteria.

Writing from a chiropractic perspective, Mootz and Hansen (1999) have discussed the use and understanding of what they term clinical algorithms. We note that their use of algorithm differs from general scientific usage in which an algorithm is taken to be a procedural recipe that includes allowance for repetition and branches; in short, any definite procedure for solving a problem or performing a task, independent of its specific form of presentation. Mootz and Hansen's use of algorithm as a graphical format would more generally be taken as the presentation of an algorithm as a graphical flowchart. The more widely used terminology will be used here, to facilitate interdisciplinary understanding. This generalizes Mootz and Hansen's statement that 'algorithms' offer providers and health care administers a "sense of predictability and a systematic organization to case management" to be information dependent rather than specific to a graphical format. As a specific example, it would include capturing guidelines within the Guidelines Element Model (GEM). It generalizes the decision tree from a representation as a flowchart to a more general concept of a 'graph' as an abstract structure of interconnections.

Eisenberg et al. (2002) examined a number of issues related to credentialing of CAM providers, including massage providers. Several short articles by Parkman (2004a, b, c, d) discussed credentialing issues. All of the above are general considerations, lacking discussion of any specific outcomes for massage.

Studdert et al. (1998) presented insurance claim data for massage, chiropractic, and medicine. For the period of 1993-1996 reported for massage, the Studdert et al. data indicate average massage claims of 1.8 per 1000 insured per year and 0.79 per 1000 claims paid, with an average payment of about $6400. The breakdown of claims were 61% minor injuries, 6% injuries above minor, 14% sexual misconduct, and 19% as 'nonphysical' or 'other'. As a raw comparison, claims for chiropractic and medicine averaged 26 and 94 per 1000 per year, respectively. The rates reported for massage by Studdert et al. are supported by two separate review papers. Ernst (2003) and Grant (2003) looked at the extremely rare reports of harm from massage in the medical literature. The conclusions of both reviews are well characterized by Ernst's conclusion that while "massage is not entirely risk free, serious adverse events are probably true rarities." Batavia (2004) examined the state of massage contraindications, noting lack of agreement between sources and a general lack of ties to any original basis or paper.

Cohen and Eisenberg (2002) address malpractice liability issues associated with CAM provision. They base their discussion on a four-cell table based on independent determination of safety and efficacy. In this framework, the papers by Studdert et al. (1998), Ernst (2003), and Grant (2003) would indicate that efficacy for specific conditions and contexts will be the larger consideration for massage therapy, coupled with practitioner abilities for communication and working as part of a healthcare team. Cohen (2004) and Ernst et al. (2004) have looked at ethical issues arising within CAM provision.

A number of papers have examined the balance and paradox of combining objective evidence with other considerations of practice. Hundert and Epstein (2002) include communication skills and team skills within their definition of competence. Egnew (2005) explores, from a medical perspective, the issues of healing that transend the application of technique. These issues have also been explored by cardiologist Bernard Lown in a recent book. Weissman (2000) comments on two papers that he feels embody both sides of the medical coin. Welch (2000) discusses the teaching of evidence-based medicine. Branch (2000) discusses medical education and the ethics of caring. Hewitt-Taylor (2003) and Hewitt-Taylor (2004) looks at the definition and balance of evidence related to nursing care. In the conclusions of the latter paper she specifically notes this balance.

References