In a 4,000-word cover story in the Jan./Feb. 2013 issue of the Columbia Journalism Review, David H. Freedman, a contributing editor at The Atlantic, offers us a comprehensive critique of what he calls "personal-health journalism"–what most of us would call medical writing. "Personal-health journalists have fallen into a trap," he writes, producing stories that "grossly mislead the public, often in ways that can lead to poor health decisions with catastrophic consequences."
The problem is not "the sloppiness of poorly trained science writers looking for sensational headlines," he writes. "Many of these articles were written by celebrated health-science journalists and published in respected magazines and newspapers; their arguments were backed up with what appears to be solid, balanced reporting and the careful citing of published scientific findings."
Freedman's criticism is breathtaking in its expanse and its failure to admit exceptions. The logical problem here is that Freedman's story is a piece of personal-health journalism, and the criticism he applies to others dooms his own piece. His story collapses like a liar's paradox: All health journalists are wrong, he writes. But it can't be true. If they are, then he is right.
I'm joking here, but I have a point: If Freedman is going to indict stories that are based on solid reporting and careful citation of scientific findings, what is he going to rely on to make his case? If not solid reporting and published findings, what?
Here is a summary of what Freedman has to say.
He leads with a dissection of a Tara Parker-Pope story, "The Fat Trap," that appeared in the Times magazine in December, 2011. The article discusses research that shows that most people who lose weight gain it back, and it includes studies that show why that might be the case–and why the body's systems designed to conserve energy as fat are so difficult to defeat.
Freedman writes that the article is "a well-reported, well-written, highly readable, and convincing piece of personal-health-science journalism that is careful to pin its claims to published research." It's hard to imagine how he could criticize it after such expansive praise, but criticize it he does. "There’s really just one problem with Parker-Pope’s piece: Many, if not most, researchers and experts who work closely with the overweight and obese would pronounce its main thesis—that sustaining weight loss is nearly impossible—dead wrong, and misleading in a way that could seriously, if indirectly, damage the health of millions of people."
Again, we find ourselves in a logical knot. If the article was "well reported," how did it miss the "many experts" who would pronounce its thesis dead wrong?
Freedman bases his claim that she was wrong on "many readers–including a number of physicians, nutritionists, and mental-health professionals," none of whom he names; and two doctors whom he quotes. He does not review published studies on this point. This casual survey of critics might be enough if Freedman were arguing that Parker-Pope's reporting had missed something–but it falls far short of what he needs to call her "dead wrong."
Freedman does make some useful points about medical reporting, but they are not new. "The problem isn’t unique to the Times, or to the subject of weight loss," he writes. "In all areas of personal health, we see prominent media reports that directly oppose well-established knowledge in the field, or that make it sound as if scientifically unresolved questions have been resolved." He's right; we do see stories that challenge widely held notions about obesity, and some of them are wrong. And many stories do make scientific questions sound settled when they are not. These criticisms are worth making, but the "celebrated" medical journalists that Freedman is condemning are aware of these concerns and try to avoid these mistakes. Newcomers might find this discussion helpful.
After dispatching Parker-Pope, Freedman broadens his critique. "Indeed, most major Times articles on obesity contradict one another, and they all gainsay the longstanding consensus of the field." I'm not sure what longstanding consensus he is referring to, but Freedman has twisted himself into another knot. The stories cannot all contradict one another and contradict the consensus at the same time. If one story contradicts Freedman's unspecified consensus, the story that contradicts that story must agree with the consensus. Freedman seems to be operating in some multidimensional space where one line can be perpendicular to dozens of others, and they are all perpendicular to each other. That doesn't happen in the three-dimensional medical writing that we're accustomed to. Again, I have a point: Freedman can't dismiss all Times stories. Some of them, even if by happenstance, must turn out to be what he considers correct.
As an example of bad medical writing, Freedman cites "innumerable articles," including Parker-Pope's, that suggest that obesity is largely genetically determined. And then he engages in a little medical writing of his own:
But study after study has shown that obesity tends to correlate to environment, not personal genome, as per the fact that people who emigrate from countries with traditionally low obesity rates, such as China, tend to hew to the obesity rates of their adopted countries. What’s more, global obesity rates are rapidly rising year by year, including in China, whereas the human genome barely changes over thousands of years. And studies clearly show that “obesity genes” are essentially neutralized by healthy behaviors such as exercise.
This could all be true, but these two paragraphs are nowhere near enough to dismiss every story that comes to a different conclusion. And suddenly Freedman is citing published studies, which he has already argued cannot provide the basis for legitimate conclusions. Do we sense an agenda here? Freedman writes that "study after study has shown that obesity tends to correlate to environment, not personal genome." He expresses no doubt on that point. So we wonder: Is it all medical writing that Freedman condemns, or just the stories that emphasize genes over environment?
Freedman then cites the noted critic and scientist John Ioannidis of Stanford, who has convincingly argued that there are many, many problems with the conclusions reported in medical journals. Freedman explains at some length why randomized clinical trials "are plagued with inaccurate findings," but his analysis will not be news to the people who do these studies or those who write about them. Indeed, Freedman is recycling some of what he wrote in a story that he published in The Atlantic in November, 2010. Ioannidis is correct: These studies are not perfect. But what is the alternative?
Freedman then builds to his close, where we expect him to explain what the alternatives are. How can medical writers do better? "Too many health journalists tend to simply pass along what scientists hand them—or worse, what the scientists’ PR departments hand them," Freedman writes. Of course they do. But not Parker-Pope. How do we solve the Parker-Pope problem: Eradicating the "catastrophic consequences" in the work of "celebrated" medical journalists?
Because published medical findings are "more often wrong than right" (a conclusion drawn from Ioannidis), a reporter who quotes studies "is probably transmitting the wrong findings," Freedman writes. "And because the media tend to pick the most exciting findings from journals to pass on to the public, they are in essence picking the worst of the worst. Health journalism, then, is largely based on a principle of survival of the wrongest."
What should reporters do? Freedman:
Readers ought to be alerted, as a matter of course, to the fact that wrongness is embedded in the entire research system, and that few medical research findings ought to be considered completely reliable, regardless of the type of study, who conducted it, where it was published, or who says it’s a good study.
That sounds like a prescription for refusing to report any medical news, and more–for actively working to shield readers from new medical findings. It is not true that "wrongness is embedded in the entire research system," and anyone who has been cured of cancer, protected from polio, or treated for pneumonia knows it isn't true.
Health journalists make the situation worse, Freedman writes, by writing about "the exciting, controversial idea that their editors are counting on." For example (and this is my example), a reporter probably couldn't get far with his editors if he pitched a story saying some medical reporters are lousy at what they do. But the "exciting, controversial" idea that even the best medical reporters mislead and threaten the lives of their readers might be what his editors are counting on.
In his closing paragraph, he seems to broaden his critique to all science writers, not just medical writers. This comes as a surprise. He quotes Dennis Overbye of the Times, who does not cover medicine, as saying that scientists' values are honesty, doubt, respect for evidence, and so forth. Freedman's response: "But given what we know about the problems with scientific studies, anyone who wants to assert that science is being carried out by an army of Abraham Lincolns has a lot of explaining to do." Apparently Freedman thinks his criticism applies to the cosmologists and astronomers that Overbye covers, although it's difficult to know how a cosmology story based on overwhelmingly misleading studies would threaten readers' lives, which is where the argument began.
In the end, Freedman offers us no remedy for what he thinks is wrong with medical journalism. He doesn't cite any examples in which he thinks medical reporters got it right; apparently, in his view, there are none. Freedman has succeeded in writing a provocative piece, and it is sure to get more attention than a story that says what is actually the case: Some medical researchers and medical writers are very good at what they do; some are terrible; and most are somewhere in the middle.
Freedman is correct: Few medical research findings should be considered completely reliable. But many medical research findings, and many of the stories the best medical reporters write, have enriched our lives and, to use Freedman's metric, saved readers' lives in some cases, I'm sure. I commend Freedman for reminding medical reporters that they are not beyond criticism. But he offers them no way out. According to his website, Freedman has written mostly about technology, and only occasionally about medicine. Maybe that's what his criticism ultimately comes down to: Medical writers would do best to write about something else.
-Paul Raeburn
Leave a Reply