From 1989-2002, annual screening mammography was recommended for women at low risk of breast cancer starting at age 50 in the US. In 2002, USPSTF changed that recommendation to recommend mammography every 1-2 years for women starting at age 40. This change was scientifically contentious – there were questions about the data – but endorsed by cancer organizations. This month, USPSTF changed the recommendation back to starting at age 50. You can read the published recommendation here. This change is less scientifically contentious, but has been even more upsetting to cancer organizations and, in many cases, to women.

I’m going to tackle just a few of the key decision making questions here to try to clarify what’s going on. For example, I’m going to set aside questions of cost and of insurance coverage, which are significant issues, to focus only on the health questions, and only on women at low risk for breast cancer.

Is there a threshold for screening?

Assumption: A false positive mammogram is harmful.

Why? A positive mammogram creates a decision: ignore the finding or confirm the finding? Ignoring the finding may mean ongoing anxiety, which can range from mild concern to debilitating worry. Confirming the finding requires a breast biopsy, which is uncomfortable, and, depending on the technique required, can involve small risks of breast damage, infection, anesthesia, etc. A positive biospy (which may also be false positive in rare cases) generally leads to treatment for breast cancer (lumpectomy, mastectomy, lymph node biopsy, radiation, chemotherapy, and drug therapies). Notably, some breast cancers can be very slow-growing, so that treating them at all is unnecessary (you are more likely to die with these cancers than of them), but, again, ignoring them may mean ongoing anxiety.

If you read that description and you don’t think that the possibilities of anxiety, unnecessary biopsy, and potentially unnecessary treatment of indolent cancer are harmful, then guidelines for starting ages for mammography are unnecessary — mammography should begin after puberty. After all, any chance of detecting cancer, however small, is worthwhile if there’s no harm in looking.

Most people, however, and USPSTF in particular, don’t discount those harms. Therefore, the question is: When do the benefits of screening mammography exceed the harms? As women get older, they are more likely to have breast cancer, so positive mammographies are less likely to be false positives. On the other hand, cancers found in younger women may be more beneficial to treat. (And as women get much older, they’re more likely to die of other causes, so it’s sometimes more harmful to find and treat breast cancer than to stop looking, which is why all these guidelines refer to women 75 and younger).

Put like this, the question isn’t “Is mammography a good thing?” Overall, it undoubtedly is. The question is “Knowing that there’s *some age* under which mammography is not worthwhile for the average woman, what age is that?” And, “How sure are we?”

It’s entirely possible, for example, that that age could be 45. But by convention, patients are usually studied in 10-year age ranges, so if the true “best age” for the average patient is 45, some studies will look like the right answer is 40 and some will look like the right answer is 50.

How sure are we?

So, how sure are we? In 2002, the USPSTF gave their recommendation (40-75) a certainty grade of “B”, which means (from their web site): “The USPSTF recommends the service. There is high certainty that the net benefit is moderate or there is moderate certainty that the net benefit is moderate to substantial.”

What is high certainty to USPSTF? “The available evidence usually includes consistent results from well-designed, well-conducted studies in representative primary care populations. These studies assess the effects of the preventive service on health outcomes. This conclusion is therefore unlikely to be strongly affected by the results of future studies.”

What’s moderate certainty? “The available evidence is sufficient to determine the effects of the preventive service on health outcomes, but confidence in the estimate is constrained [by concerns about the quality of the evidence]…As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the conclusion.”

Today, in 2009, USPSTF is effectively saying that they were overconfident in 2002. Or, if you like, they were moderately certain in 2002, and more information has become available that was large enough to alter the conclusion. The new 2009 recommendation (50-75) has a certainty grade of “C”:

“The USPSTF recommends against routinely providing the service. There may be considerations that support providing the service in an individual patient. There is at least moderate certainty that the net benefit is small.”

The new scientific evidence since 2002 hasn’t made us hugely more certain. But it has tipped USPSTF’s beliefs about the net benefit of screening mammography for 40-49 year old women far enough to cross back over their threshold for recommendations.

Why are people so upset about it?

New evidence changes recommendations all the time, and not just in health care. In my childhood, dinosaurs were depicted as brown and scaly. Evidence now suggests many were feathered, probably colorful, and the recent discovery of fossilized melanin-producing cells makes it likely that we will know what some of those colors were. This bothers no one. What’s different about mammography?

First, of course, health evidence affects the quality and length of our lives in a way that dinosaur colors don’t.

Second, because mammograms have been urged on women for years, they are considered a valuable health good. Extending a good (as in the 2002 recommendations) seems like a gain. Removing a good (as in the 2009 recommendations) seems like a loss, and losses are psychologically more painful than equivalent gains. Of course, the debate itself is over whether mammography really is a good in younger women, not about whether we should provide goods or not.

Third, there is evidence that people may actually think well of false positives. In a 2002 study of prostate cancer screening (which may have even less benefit than mammography for women 40-49), Cantor and colleagues found that several patients were willing to endure the anxiety of a hypothetical false positive PSA test and the pain of a biospy to be reassured that they did not have prostate cancer. On the other hand, Katz and colleagues surveyed patients who had actually had a (false) positive PSA test and negative biopsy, and found increased worry and decreased sexual function among these patients as compared to those with a negative PSA test. It may be that we underestimate the downside of a false positive. (Hat tip to Rob Hamm and Scott Cantor for the references.)

Fourth, the recommendations have changed and changed back within a short enough period that the USPSTF appears indecisive and inconsistent, and this leads to distrust of their recommendations. It is easier to see the recommendations change than the evidence behind them, and it’s certainly a lot harder for USPSTF to communicate the changing evidence, but we need to develop better strategies for doing so.

Finally, USPSTF recommendations are for the population — what I’ve called the average woman. There are women, however, who had true positive mammographies in their 40’s, and on the basis of their experience are vocal supporters of earlier screening. On the other hand, the many women who had (retrospectively) unnecessary mammographies from 40-49 and didn’t experience a false positive are not a loud voice in the discussion. This returns us to the threshold question — if a single 25-year old woman turns up to say that she had a mammogram that led to discovery and treatment of a dangerous cancer that saved her life, is that sufficient reason to begin screening mammography for women in their 20’s?

With respect to individual women in their 40’s, Diana Petitti, the Vice Chair of USPSTF, says “You should talk to your doctor and make an informed decision about whether a mammography is right for you based on your family history, general health, and personal values.” That, at least, is a recommendation that has always been true.

Four times a year I lead a group third-year medical students through afternoon seminar on using test results in the diagnostic process. Although one could make this topic very broad, the focus of the seminar is essentially Bayes Theorem. There is plenty of evidence that this is a challenging topic for students in the health sciences (and practicing physicians). I certainly found it challenging when it was introduced to me in medical school. However, once mastered I had to wonder why students could not see how obvious Bayes Theorem is; it is only a simple mathematic transformation.  When teaching Bayes Theorem it always seemed to take me multiple attempts at the computation and providing explanations until a few of the group grasped the concept of probability revision. Most would leave bewildered.

In frustration, I searched for a better approach- I thought my students should be able to experience the wonder of probability revision and not the pain of elementary mathematics. The search led to a wonderful report about simplifying bayesian inference by making it visual. (S Krauss, L Martignon, U Hoffrage. Simplifying Bayesian Inference. Conference on Model-Based Reasoning in Scientific Discovery, 1998. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.4.9233) Students still are required to perform simple mathematical computations but the visual presentation of Bayes Theorem allows the students to see where they are in the process. Once completed students can easily go back and review the steps they took.  This simple visual approach has turned afternoons of student frustration into afternoons of discovery where they come to “see” the importance of pre-test probability in interpreting a test result and “see” the importance of not only sensitivity but also specificity.

If you are facing a similar challenge in your teaching, I highly recommend that you take a look at this approach.

Session Description

Will decision aids have a vital role in health care reform? Patient decision support tools are well-validated and effective, but under used. The symposium will address this problem by examining the keys to success of decision support tools used in the US outside of health care.  Investigators will present results of an in-depth investigation of four successful tools used to present evidence and promote informed choices by consumers.

Consumer Reports’s annual car buying guide, the FDA Nutrition Facts Panel, eBay, and US News’s America’s Best Colleges all provide decision support, some mandated by government, some entrepreneurial. Representatives from health care patient decision support producers will comment. The symposium will present key variables for success, and debate the implications for consumer and patient decision involvement in health care decisions.

Speakers

• Margaret Holmes-Rovner, PhD, Professor – Health Services Research, Michigan State University, East Lansing, MI
• Dale Shaller, MPA, Principal, Shaller Consulting, Stillwater, MN
• Dorothy Jeffress, M.B.A., M.S.W., M.A., Executive Director, The Center for Advancing Health, Washington, DC
• Richard Wexler, MD, Director, Patient Support Strategies, The Foundation for Informed Medical Decision Making, Boston, MA
• Representative, Consumer Reports Health Rating (Invited)

Registration

Information about registration for the Society for Medical Decision Making meeting, including pre-meeting symposia and short courses, will soon be available at http://www.smdm.org/2009meeting.shtml

He focuses on the difference between the value of an identified life and a statistical life, as well as the impact of uncontrollability on risk perception (part of the constellation of factors Paul Slovic has referred to as being associated with “dread risk”).

The availability heuristic is also relevant here – there is very little reporting of deaths associated with seasonal flu and a lot of reporting about potential new strains. This leads to an underestimate of the risk of seasonal flu, even in the young and elderly.

The article does a fine job of reviewing and highlighting key research in patient risk communication. It offers clinically-relevant guidance for physicians seeking to educate their patients that is very much in the MDM:APG spirit. It also links to Dr. Chris Cates’s Visual Rx web site, which provides a form that can generate pictograms for representing number needed to treat or number needed to harm data (Dr. Cates, if you’re reading this, I’d love to see version that can combine both efficacy and adverse event data and produce a unified plot of NNT/NNH – something I’ve been meaning to do with my own (nonvisual) NNT/NNH calculator).

It’s great to see this kind of discussion appearing in publications like these that reach a large community of practice.

The study of risk communication is an active one. Determinants of whether or not lay people achieve adequate comprehension of risk information are multifaceted, and include how the risk communicator understands the risk information, his or her mental model of what the lay person knows, the format and context of the risk communication, the lay person’s own numeracy and health literacy, the human cognitive processes involved in judgment and decision making, and cultural expectations around the discussion of medical and environmental risk.

Due to the growing requirements to communicate numerical information to lay individuals, there has recently been a significant push to conduct research that can provide direct guidance regarding how to best present numerical information in order to increase the likelihood of informed decisions. Accuracy is one key goal; evaluability (the ability to make a meaningful interpretation of risk so that it can be used in judgments and decisions- cf. Hsee C, The Evaluability Hypothesis: An Explanation for Preference Reversals between Joint and Separate Evaluations of Alternatives. Organ Behav Hum Decis Process, 1996;67:3) is a conceptually distinct goal, and both are important. Look for a lot more research on these topics in the next few years.

On the other hand, nearly everyone pays the full cost of their veterinary care, and taking your pet to the vet is a good time to think about how medical decisions should get made.

This came up for me recently. I have an 11-year-old Labrador retriever.  My wife took her to our usual vet practice because she had a strange swelling on her flew (upper lip). On this particular day, she saw a new young vet who we hadn’t seen before. He made three findings:

1. The swelling was an infection, and antibiotics were indicated. He prescribed Simplicef, which the manufacturer describes as “the first and only once-daily oral cephalosporin that’s FDA approved for veterinary use in the treatment of canine skin infections. It’s designed to be convenient and a good value for your large dog.” (from simplicef.com)
   
2. Her weight was 57 pounds, about 10 pounds lower than a year ago. He suspected a metabolic disorder (and probably also considered cancer, but didn’t say that), and ordered a panel of blood tests.
3. He heard a heart murmur, left-side, grade 2-3/6, which had been documented in her chart for several years. He recommended an ultrasound, chest x-ray, and ekg.

I like veterinarians. I think they’re good people with the interests of animals at heart. That said, in this particular consultation, several avoidable errors were made that illustrate some key principles of medical decision making (for humans as well as dogs):

1. The dog improved on the first 14-day course of Simplicef, as we all expected, but these infections typically require multiple courses. Readers of this blog will not be surprised to hear that while once-daily dosing may be more convenient than twice-daily dosing, a 14-day course of Simplicef was about $50; the equivalent course of generic cephalexin is about $8. There is no evidence of any difference in effectiveness, and cephalexin has been around a long time, and our dog has taken it before with no ill effects. We pointed this out to the vet, who was surprisingly reluctant to switch for the second course, but agreed. The infection continued to improve on cephalexin. Attention to principles of rational prescribing would have been beneficial here.
2. The blood tests were all normal, as I had predicted, because I knew something that the vet didn’t — but should have. About 10 months earlier, we changed to a prescription dog food (our dog has arthritis, the combination of glucosamine and omega-3’s we were adding to her regular dog food seemed to be helping, and the new dog food had that all built-in and was cheaper that the cost of the supplements). When we changed food, we continued to feed at the same volume, without considering differences in the caloric content of the food, which turned out to be lower. She was losing weight because she was simply not getting enough calories. I had the vet calculate the appropriate feeding amount and she was gaining weight within two weeks (although that requires titration too — her joints are less painful when she’s lighter, so we don’t want her to gain it all back). The key decision point here – one echoed by many expert human diagnosticians -  is that lab tests can’t take the place of a good history. Had this vet asked “have you changed her food?”, we would have embarked on this trial of increasing her food without the costly blood tests — which, given that she had no clinical fatigue or other symptoms, could certainly have been postponed a few weeks anyway.
3. Everyone likes to know what’s going on. Naturally, the vet would like to know the cause of the heart murmur. As we point out in the book, however, the main purpose of gathering information is to drive a decision (allaying worry might be a secondary purpose, but that should be the patient’s — or owner’s — worry, not the doctor’s). So I asked the vet what he expected to learn, and how that would affect subsequent treatment. He indicated that they would learn if her murmur was amenable to medical treatment. I pressed for exactly which drugs he was referring to, and he mentioned diuretics and ACE inhibitors. I asked if there was any evidence of benefit for an asymptomatic dog, and he assured me there were several studies. I asked where to find them; he couldn’t specifically cite any, but suggested I search Medline for canine heart murmur studies. Which I did, finding two medium-to-large trials, one showing no benefit and one showing ambiguous benefit (and with several methodological flaws). Now, the veterinary literature is considerably smaller than the human medicine literature, and many fewer well-designed large-scale randomized trials are conducted in dogs. I haven’t yet brought these studies with me to see if (a) I didn’t find some important study that the vet had in mind, (b) he was thinking of these studies and interprets the findings differently, (c) he was thinking of studies in dogs with documented heart disease, in which ACE inhibitors do appear to prolong life by several months, or (d) he was passing along a vague memory of reading these or being told that evidence supported the drugs without checking for himself. Moreover, his recommendation did not consider values in the decision beyond the value of gaining knowledge. Cardiac ultrasound requires sedation, which is a procedure that is not without risk in an older dog. Moreover, medications can have side effects that can diminish quality of life, and these may not be worth extending our dog’s life by a few months. These are crucial considerations in this decision, and this particular vet, on this particular day, missed them.

My wife is a very smart person, but, like most people, not a decision scientist – and she shouldn’t have to be. A veterinarian, like a physician, has far greater expertise and experience in health care than their clients do. When they make a recommendation, it will usually be followed – often without question. That’s one reason why veterinarians (and physicians) also need to become experts in decision making, and why owners (and patients) need to ask good questions and seek out providers who demonstrate a commitment to good decision making.

Videorecordings of the symposia are now available. You can find links at http://www.sjdm.org/content/video-recordings-2008-sjdmsmdm-symposia

After thinking about it for a while, though, I realized that medical decision science (at least as I practice it) is primary about helping physicians and patients with to do three key things:

• know your Goals
• understand Uncertainty
• pay attention to Trade-offs

Which conveniently folds up into the acronym G-U-T: goals, uncertainty, trade-offs.

Normally, we think of making gut decisions as an intuitive, impulsive, “hot” process. Now, though, when I think of asking people what their gut is telling them, I’ll be thinking of their GUT: What are your goals for life, and how does that factor into this decision? What is uncertain in this decision, how much does it matter, and how can you know more or worry less? What trade-offs will you face between outcomes that you care about, and how will you make them?

(And should I add “S” for “society” and make it “GUTS?”)
Readers, what are your three words for making medical decisions?

My response to her (revised for this blog) reviewed some evidence, much of which also appears in the Salon article:

Among doctors, there is really no question. The one published study that claimed a relationship between MMR vaccine and autism has been renounced by 10 of the 13 authors and retracted by the journal (for some relevant links see http://blog.openmedicine.ca/node/50).

With respect to thimerisol, it’s out of all childhood vaccines in the US right now, except for some flu shots. See http://www.cdc.gov/nip/vacsafe/concerns/thimerosal/faqs-thimerosal.htm. The one study in this area that claimed a link was also authored by a pair of docs who make their money as expert witnesses and was seriously flawed.

The purported link is, at base, no more than a scam to make profits off of people who have the misfortune to already be dealing with how to raise their autistic child and are accustomed to (rightly) standing up to recognized authority in other matters related to their children (e.g. education).

Since 2001, thimersol has been removed from vaccines, and autism rates have increased. And now measles is making a comeback.

What is known is that vaccines have made many childhood diseases
nearly extinct in the US, and saved a ton of lives and health.
E.g. http://www.cdc.gov/mmwR/preview/mmwrhtml/00056803.htm

Readers of this blog will know that I think scientific evidence is a good thing in medical decision making. But looking over my response, I wonder if I really addressed her key uncertainty: “My confidence in institutions is at an all time low”

Unfortunately, I think there are many people who are concerned about the trustworthiness of public health research and recommendations (and perhaps too few people who are concerned about the trustworthiness of pharmaceutical company research, but that’s a different story). Some of this concern is undoubtedly justified, as I wrote, because people who are raising autistic kids or living with other disabilities do have to be critical of the institutions around them, and often must make their voices heard loudly in order to secure equitable and beneficial treatment of their children and themselves in non-medical arenas. It is hard to avoid skepticism about medical science if you’re African American and familiar with the Tuskegee studies. IUD use is very low in the U.S., despite its safety and effectiveness, in part because of memories of problems with the Dalkon Shield. And sometimes things that look good, like thalidomide, do turn out to be bad.

As we wrote in the book, uncertainty is a fundamental feature of medicine; on the other hand, decisions have to be made. For vaccination, the default option (the one proposed by nearly all pediatricians) is to vaccinate, and that’s a good thing, but once a treatment has entered the public consciousness as uncertain, people will rightly want to make a more considered decision. A key question for physicians and patients alike is how much evidence, and what kind, should we expect in order to make these decisions, and how can we be sure we’re hearing reliable evidence and not biased evidence? This is a particular problem because, much of the time, good science speaks softly; bias shouts.

There are some time-honored heuristics that serve well here. Prefer evidence that you seek yourself to evidence that someone else thrusts at you. Prefer evidence that can clearly be replicated, that has received open scrutiny from other scientists, and especially findings that have been replicated repeatedly. Ask “cui bono?” Understand statistical reasoning. Recognize that in a nation of 300 million people, a one-in-a-million chance will happen about 300 times. Compare medical risks to other risks that you’re willing to undertake in daily life for similar benefit. Remember to consider the risks incurred by choosing not to choose.

Subjective confidence can be measured in several ways. People are regularly asked to state their confidence verbally or numerically: “Just how sure are you of that?” Confidence in an outcome is also measured, perhaps more subtly, by asking how willing someone is to take a bet that the outcome they predict will actually occur. A person who is willing to take a bet that pays $10 if they are correct, but costs them $50 if they are wrong must be quite confident that they are correct!

An important measure of the accuracy of someone’s confidence judgments is their calibration. Calibration refers to the relationship between the numerical confidence than an event will occur and the actual frequency with which the event occurs. For example, if we followed a group of 100 patients presenting to an ER with chest pain for whom the attending physician had estimated a 30% chance (each) of a myocardial infarction, and 30 of those patients actually were diagnosed with an MI, the attending physician would be said to be perfectly calibrated in his judgments. That is, when he gives a 30% confidence of MI, it accurately reflects a 30% chance of MI.

Few people Рphysicians or patients Рare well-calibrated in their confidence judgments around medical uncertainty. Overconfidence, in particularly, has been repeatedly demonstrated. Clinically promising approaches to improving calibration include discussion and searching for conflicting evidence. Koriat, et al. (1980) reduced overconfidence by asking people to consider conflicting evidence that would weigh against their initial belief. Arkes, et al. (1987) showed that group discussion with peers could improve calibration as well. Both of these techniques encourage deeper consideration of choice alternatives, and allow for the combination of multiple viewpoints (Weber, B̦ckenholt et al. 2000; Armstrong 2001); moreover, collegial discussion of cases is a venerable tradition in medicine, and occurs naturally in most group practices.

Although we most often think of confidence as a statement about belief in the accuracy of a judgment, Weber et al. (2000) provided a convincing demonstration that confidence may instead reflect a lack of conflict about the decision. In their study, they asked 84 physicians to generate most-likely and second-most-likely diagnoses for cases, and to give their confidence in each diagnosis and in the proposition that the correct diagnosis was somewhere in their set. Confidence in the set should always be higher than confidence in any single diagnosis (and it was); moreover, if confidence reflects belief in accuracy of judgment, the confidence in the set should be higher when the top two diagnoses are both likely than when one is likely and the other considerably less likely. Instead, confidence was reduced when the top two diagnoses were both judged to be quite likely.

The authors conclude that expressions of confidence may actually be expressions of a lack of decision conflict. When there is only one likely diagnosis, the confidence in the set is high, because there is little conflict about which diagnosis is correct. When there are two likely diagnoses, however, there is much more conflict about which is the most likely diagnosis, and this is reflected in a lower overall confidence for the set. In support of this theory, physicians who mentioned a rival hypothesis when discussing their reasoning for selecting their most-likely diagnoses also tended to have lower confidence in that diagnosis (as well as in the set of diagnoses).