Dr. Roy Poses, a well-respected physician who has done a lot of work studying physician probability judgment (one example of which is mentioned in Chapter 7 of Medical Decision Making) has posted a rebuttal on the Health Care Renewal blog. Dr. Poses demolishes the misrepresentation of EBM that appears in the original article (as well as asking some on-point questions about the author’s interests in the matter), and does it quite effectively, so I won’t repeat his criticism here. Instead, I’ll focus on some other misunderstandings in the original commentary: that cutting health care costs is at odds with the medical imperative to improve care, that patients are so biologically unique that studies of patient groups has little value, and that EBM reduces physician autonomy.

Are considerations of costs at odds with the “medical imperative”?

The commentary invites us to “Consider an overweight man who is forced to take a cheaper, less effective anti-cholesterol drug. If he ends up in the emergency room because of undertreated cardiovascular disease, this could end up costing the health-care system significantly more money.” Setting aside that the author now seems concerned with health care costs in this situation (not only with effectiveness), we could equally consider a situation in which comparative effectiveness evidence helps ensure that the patient is prescribed one of a set of anti-cholesterol drugs that evidence suggests are equally effective and safe. The physician, knowing this, chooses which of these drugs to prescribe first on the basis of judgments about the needs of the patient, which might include which of the drugs the patient can afford, which offers the most convenient dosing, etc. A patient prescribed an equally effective but more costly drug who can not afford to fill the prescription may wind up in that same emergency room, when an equally effective and more affordable generic drug might have kept him healthy.

Here’s an example of a comparative effectiveness review from January 2007 by the US Agency for Healthcare Research and Quality (AHRQ): Comparative effectiveness of second generation antidepressants in the pharmacologic treatment of adult depression. Readers will note that this report makes no references to cost-effectiveness (it barely mentions cost at all), and focuses on reviewing the available evidence that compares the medical effectiveness of different second generation antidepressants on a variety of factors and for a variety of subgroups of patients. It points out the strength of evidence to answer different questions (which ranges from none to high), and, where evidence exists, what the evidence says (e.g. there is no difference in effectiveness between these drugs for major depressive disorder).

Is each patient biologically unique?

There is no question that individual human beings have unique genetic makeups that lead to unique biology. The question is whether this matters for medical treatment and medical research.

All medical research is based on the idea that biological similarity is as important as biological difference. When a new patient presents to a physician with the first strep throat of their lives, the physician considers prescribing an antibiotic, suggests the patient take an analgesic, but doesn’t offer an anticonvulsant. Why? Because the physicians knows the pathophysiology of strep throat, and understands that the infection can be eradicated and pain can be reduced with those medications. The physician knows this because we have studied enough people to enable us to generalize. Research on groups of people is fundamental to modern medicine.

Not every drug works as well for every person, but through well-designed research we can reduce the uncertainty and increase our confidence in how likely a drug is to work on average. We can, and do, also learn about how variable the drug’s effect is around that average, and for which patients it may perform better or worse.

As we learn more about genetic medicine, we may one day seek to tailor therapies to the specific genetic makeup of a patient. Of course, it will require extensive research on the application of these processes applied to large groups of patients before we can be confident of our ability to do this. If our knowledge is great enough that tailored therapies can complete reduce uncertainty in outcomes, the profession of medicine will experience fundamental changes. But people need health care now, too.

Does EBM reduce physician autonomy?

Physicians need the freedom to pursue effective treatments for a patient based on their knowledge — which should include available, credible research evidence — and judgment — which should include contextual factors that are individual to a patient (see, for example, this excellent article by my colleague, Dr. Saul Weiner). EBM enhances this freedom, by improving the knowledge base on which the physician relies.

Physician autonomy does not extend to prescribing ineffective cures when there is valid evidence of an effective cure. The same applies to diagnosis. No physician I know would prefer to measure fever by a hand on the forehead when a thermometer is available.

Physicians need evidence. Synthesizing evidence to review conclusions about comparative effectiveness allows the physician to reduce a major source of uncertainty, and to better inform his/her judgment in selecting a therapy. It also helps shield the physician and patient from spurious claims of effectiveness made by those with a vested interest in selling more profitable interventions — a financial imperative that can be much more at odds with the medical imperative than social imperatives to manage health care spending for the benefit of society as a whole.

The proverb captures important psychological nuances of choice. The same product - a bottle of wine or a risky medical procedure - may be perceived differently depending on its context, and it is often possible to arrange the context to influence a choice while still maintaining the decision maker’s autonomy.

The practice of structuring choices is called “choice architecture” in a brilliant and important new book, Nudge: Improving Decisions About Health, Wealth, and Happiness, by University of Chicago Distinguished Professors Richard Thaler (Business) and Cass Sunstein (Law). Nudge lays out the groundwork for the science of choice architecture in investing, insurance, health care delivery, and other areas, and argues for a “libertarian paternalism” in which choices are structured to make it more likely that a decision maker will select what is considered the most beneficial option, without impairing the ability to decision makers to select other options. For example, making enrollment in 401(k) plans automatic for new employees, with a form for opting out, is likely to result in greater retirement savings than an opt-in system, without limiting anyone’s freedom to choose.

Thaler and Sunstein apply the principles of choice architecture to a few problems in health care (How could Medicare part D be improved? How can organ donation rates be increased? Why shouldn’t patients be allowed to waive their right to sue for medical negligence in return for cheaper health care?) But the concepts in the book go beyond their specific examples and could prove very useful to practicing clinicians, who, they note, are often in the position of being choice architects for their patients.

Their principles of choice architecture (paraphrased by me and focused on physicians helping patients make decisions) are:

• Make sure incentives are aligned with desired outcomes
• Help patients map outcomes of different alternatives into formats they can understand (a major focus of Medical Decision Making as well)
• Arrange default options to favor better health. Pediatricians have done a good job of making vaccination a default option.
• Provide timely and relevant feedback about choices and outcomes. A patient seeking to lose weight needs to experience feedback in the form of measurable progress soon enough that they are not discouraged.
• Expect error and develop systems to prevent, detect, and minimize it. For example, pill cases and inhalers with dosage counters are simple and valuable ways to reduce the frequent errors people make in remembering medication. Psychological research provides direction as to what kinds of errors are to be expected when people are making decisions.
• Structure complex choices to reduce the difficulty of making good decisions. In many ways, that’s what medical decision making — and Medical Decision Making — is about.

I highly recommend Nudge. It’s a great read, and has the potential to change the way you think about clinical practice. Here’s a link to the Nudge web site and blog for more information.

The link is here.

The first is journalistic. National Public Radio’s program Talk of the Nation did a segment on a new blood test that can diagnose early stages of Alzheimer’s diease.

The second is literary, as the science fiction podcast Escape Pod presented the story Results by Kristine Kathryn Rusch, originally written in 2000.

Very different formats, very similar ideas about patient-focused decision making. Each is well worth a listen.

As a result of a $430 million settlement between drug manufacturer Warner-Lambert and the U.S. government, several projects have been funded to study and address issues in the marketing of pharmaceuticals. One such project, Formulary Leveraged Improved Prescribing (FLIP) is centered across the street from my office, and is a joint effort between the University of Illinois at Chicago and the John H. Stroger, Jr., Hospital of Cook County.

As its name suggests, FLIP is directed at members of formulary committees of hospitals and health plans and is intended to encourage them to take a critical look at their drugs choices for the formulary and to become an important source of provider education. Both patients and physicians may be particularly interested in FLIP’s 24 Principles for Rational Prescribing, a set of guidelines for increasing the overall safety of prescribing while lowering costs. Many of these, not surprising, involve using non-drug therapies and favoring older, well-established drugs over newer.

This is the meeting to attend if you’re interested in medical decision science; presentations typically focus on clinical applications, methodological advances in decision and cost-effectiveness analysis, psychology of medical decision making, and other key topics in clinical and health policy decision science. There are also excellent short courses in the days before the meeting, providing education on MDM topics at a variety of levels (including a course on problem solving for medical educators by George, Frank Kee, and I).

Helpful tip for newcomers to SMDM: You may well find your first meeting overwhelming. The presentations tend to be at a high level and can intimidate people who are just starting to explore medical decision making. Expect this and don’t worry about it. You’ll get a lot of interesting new ideas from your first meeting, and you’ll find that a lot will “click” if you return the following year.

While decision scientists have traditionally emphasized utilitarian approaches to decision making, bioethicists have questioned whether one should focus on the consequences of decisions or their intrinsic features (as in deontological approaches to ethics). They have also asked such questions as whether morality is a feature of actions or a set of virtues, how moral claims should be justified, how to balance contextually-specific decisions with universal principles, and how health care ethics related to ethics in other endeavors (Khushf 2004).

Like decision scientists, ethicists, particularly those that recognize multiple ethical principles, virtues, or rights, regularly consider situations in which all of their goals can not be fully met. Decision science approaches this problem through prioritization and trade-off of goals. Ethics is more likely to use a process of specification, in which each principle is made more contextual and concrete in order to provide guidance for the particular decision in question (Beauchamp and DeGrazia 2004). Indeed, some philosophers advocate case-based approaches to bioethics, just as the judicial system in most Western nations interprets the law through the analysis of cases and the development of precedence (Boyle 2004).

Although it would be natural to expect these two perspectives to converge and these two groups of theorists to share ideas, historically these questions have been answered through entirely separate academic endeavors, and have resulted in separate communities of scholarship (Stiggelbout, Elstein et al. 2006).

Two recent studies have compared the thinking of decision scientists and bioethicists. In a survey by Ubel, et al. (1996), 568 prospective jurors, 74 members of the American Association of Bioethics and 73 members of the Society for Medical Decision Making were asked to choose between providing a more effective screening program to half a population (resulting in a greater number of deaths prevented overall) or a less effective program to the complete population (resulting in fewer deaths prevented overall). Decision scientists were less likely than others to favor the less effective (but arguably more equitable) program.[i]

A more extensive study by Stiggelbout, et al. (2006) surveyed 327 members of the American Society for Bioethics and Humanities and 77 members of the Society for Medical Decision Making, presenting each participant with two clinical cases, one concerning refusal of a beneficial treatment, and one concerning surrogate decision making about futile care at the end-of-life. The authors found substantial similarities in the opinions of the two groups, although decision scientists were more likely to emphasize outcomes and trade-offs and ethicists were more likely to emphasize patient autonomy. Notably, both groups called for greater exploration of patient goals and values, and ongoing theme in this book.

As the Stiggelbout study suggests, the concerns of bioethics and decision science are often in alignment; the ethical principle of autonomy, for example, is often naturally reflected in the importance of properly eliciting and considering patient preferences in decision analysis. Bioethical principles can also be naturally incorporated into decision making as personal or societal constraints on possible choice alternatives: in a society which espouses the natural right of people to be free from becoming unwilling subjects of medical procedures, a decision alternative which results in forcing a family member to undergo genetic screening might be simply unacceptable.

Suppose that we want to know if someone has an active H. pylori infection without forcing him to undergo invasive endoscopic testing. Instead, we perform a ¹³C-urea breath test (UBT) that measures change in the ratio of ¹³CO­₂ to ¹²CO₂ (denoted by δ) exhaled by the patient 30 minutes after ingestion of ¹³C-urea compared with the ratio before ingestion. Because H. pylori hydrolyzes ¹³C-urea to ¹³CO₂, the resulting change value, Δδ, tends to be higher in patients with H. pylori infection than the average uninfected patient, but there’s natural variation both groups, and it’s not impossible that a healthy patient could have a high Δδ. The figure below illustrates the situation.

Figure: Distribution of Δδ for infected (bell curve on the right of the figure) and uninfected (bell curve on the left of the figure) patients. Both infected and uninfected patients may have Δδ as low as 3 or as high as 15. Adapted from Figure 1 of Herold and Becker (2002), BMC Gastroenterology 2:12, with permission of the BioMed Central Open Access license agreement (http://www.biomedcentral.com/info/authors/license).

The UBT is thus an imperfect test, because there is the potential for error. A test which defines the disease (in the way that, for example, bacterial growth on a culture defines bacterial infection) and thus has, in principle, perfect discrimination, is often referred to as a reference standard or gold standard test.

Test thresholds

As the Δδ gets higher, however, the person is more likely to be infected, and as it gets lower, they’re more likely to be healthy. And our goal is to treat the sick differently than the healthy; for example, to prescribe a proton pump inhibitor and antibiotics if we are sufficiently convinced of the likelihood that the patient is, in fact, suffering from H. pylori infection. This implies that we need a criterion, or threshold Δδ, above which we will act in one way (e.g., start drug therapy), and below which we will act in a different way (e.g., watch and wait). A threshold would be graphically represented by drawing a vertical line at the Δδ threshold score; patients whose Δδ is higher than the threshold are treated as infected, and those whose count is lower are treated as healthy.

Because of the overlap between the distributions of Δδ (that is, because of the imperfect discriminative power of Δδ in this example), no threshold can accurately classify every patient. Whatever criterion we set for calling someone infected or healthy based on this test, there will be some people rightly classified as infected or healthy, and some people wrongly classified as sick or healthy.

The threshold determines the kind of error we are likely to make. The higher the Δδ threshold we require in order to call someone infected (graphically, the farther to the right we draw the vertical line), the more we’ll wrongly classify infected people as healthy; these errors are called false negatives. For example, if we set the threshold at Δδ=18, we will almost never wrongly classify a healthy person, but about half of those infected will be misclassified as false negatives (and potentially remain untreated for their infection).

Conversely, the lower the Δδ we require, the more we’ll wrongly classify healthy people as infected; these are false positives. For example, if we set the threshold at Δδ=2, we will almost never wrongly classify a infected person, but almost half of those not infected will be misclassified as false positives (and potentially undergo unnecessary treatment).

Changing the threshold will always either increase false positives and decrease false negatives or vice versa. Only improving the discriminative power can lower both false positives and false negatives.

Choosing thresholds

Unfortunately, we generally can’t improve the discriminative power of a given test; we have to develop new and more discriminative tests (or variations of tests). But the choice of the threshold is arbitrary. A threshold may be recommended by the test developer or by guidelines on the use of the test. These thresholds should be chosen on the basis of the purpose of the test, and the consequences of false positives and false negatives.

For example, consider a rapid strep antibody test for strep throat. A false positive on this test results in a patient receiving an unnecessary dose of antibiotics for a few days; a false negative results in a patient with an untreated bacterial infection for a few days (until the results of the throat culture, a gold standard test, are available). The general consensus among physicians has been that a few days of unnecessary antibiotics is generally preferable to missing a bacterial infection for a few days, but not so preferable that antibiotics should be routinely started in all patients. Accordingly, the kits are developed to have a relatively low, but not very low, threshold for positive results. When the cost of a false negative is much greater, tests may have a very low threshold. A 17-year-old with unknown vaccination history who presents to the emergency department with high fever and possible neck stiffness is very likely to receive immediate presumptive treatment for meningitis. Although the probability of bacterial meningitis is quite low, the consequences of missing a case are so high that a marginally positive finding on a test with low discrimination (neck stiffness) is sufficient to warrant the relatively benign treatment.

In general, when noninvasive and inexpensive tests are used to screen a population for a serious condition, the goal of testing is to broadly identify individuals who may be at higher risk for the condition and refer them for confirmatory testing or other evaluation. Screening tests, therefore, are usually designed to have very few false negatives, and are willing to accept a larger number of false positives in order to assure that high-risk cases are not missed.

On the other hand, when the treatment is invasive and the cost of the disease is low or when the primary aim of the test is to provide reassurance that a patient does not have a serious condition, false positives may be a much greater concern than false negatives. A high threshold is required of a test for carpal tunnel syndrome if the treatment contemplated is open carpal tunnel release surgery.