Should we stop believing Malcolm Gladwell?

One of the premises of Malcolm Gladwell's new book, David and Goliath, is that "the act of facing overwhelming odds produces greatness and beauty."

And here's one of his examples: A paper published in 2007 reports that two groups of people were asked to solve three mathematical reasoning problems. One group read the problems in a normal typeface; the other read the problems in a light-gray italic print that was difficult to read. The group that had trouble reading the problems scored 29% higher. The counter-intuitive conclusion is that making things harder improves performance.

I haven't read the book; I'm taking this example from a review in The Wall Street Journal by Christopher F. Chabris, a psychology professor at Union College in Schenectady, N.Y. Gladwell's example is impressive, not only because of its fascinating and unexpected result, but also because unlike so many scientific studies, it has immediate, practical implications: We can try this trick ourselves.

But before you try that, you might want to read Chabris's deconstruction:

First, the study involved just 40 people, or 20 per typeface—a fact Mr. Gladwell fails to mention. That's a very small sample on which to hang a big argument. Second, they were all Princeton University students, an elite group of problem-solvers. Such matters wouldn't matter if the experiment had been repeated with larger samples that are more representative of the general public and had yielded the same results. But Mr. Gladwell doesn't tell readers that when other researchers tried just that, testing nearly 300 people at a Canadian public university, they could not replicate the original effect.

This is not an isolated incident, Chabris writes. "This flaw permeates Mr. Gladwell's writings: He excels at telling just-so stories and cherry-picking science to back them."

Chabris goes on along these lines for 2,500 words in the Journal and in a long post on his own blog, entitled "Why Malcolm Gladwell Matters (And Why That's Unfortunate)." If Chabris's charges are correct–and he's not the only one to make such accusations–then I'd take this a step further:

Should we stop believing Malcolm Gladwell?

Chabris links to a 2009 New York Times review by the Harvard psychologist and best-selling popular writer Steven Pinker of Gladwell's What the Dog Saw–And Other Adventures. Pinker writes:

Gladwell frequently holds forth about statistics and psychology, and his lack of technical grounding in these subjects can be jarring. He provides misleading definitions of “homology,” “sagittal plane” and “power law” and quotes an expert speaking about an “igon value” (that’s eigenvalue, a basic concept in linear algebra). In the spirit of Gladwell, who likes to give portentous names to his aperçus, I will call this the Igon Value Problem: when a writer’s education on a topic consists in interviewing an expert, he is apt to offer generalizations that are banal, obtuse or flat wrong.

Pinker also takes a slap at Gladwell's book Outliers, writing, "The reasoning in “Outliers,” which consists of cherry-picked anecdotes, post-hoc sophistry and false dichotomies, had me gnawing on my Kindle." Ouch.

Both Chabris and Pinker confess to admiring Gladwell's story-telling, but in this context, that's not necessarily a compliment. The charge is that everything–facts, analysis, and fairness–is sacrificed on the altar of storytelling

Chabris sticks to criticism of David and Goliath in The Wall Street Journal, but on his blog he takes a look at some of the things Gladwell has said in interviews. Incredibly, Gladwell seems to be telling us that accuracy is not among his priorities. And many reviewers who should know better seem to think that's just fine.

Chabris quotes Dave Berri at the Freakonomics blog who writes:

…critics have primarily focused on whether the argument they think Gladwell is making is valid. I am going to argue that this approach misses the fact that the stories Gladwell tells are simply well worth reading.

Isn't this upending the whole rationale for nonfiction writing? If stories are worth telling, Berri is saying, who cares if they're built on cockeyed reasoning? Chabris quotes another reviewer who writes, "I read (and write about) most pop science as science fiction." Well, that certainly lets science writers off the hook. Thought-provoking is good enough! I wish I'd known that when I started my last book, and I'll keep it in mind for the next.

But it's Gladwell's own comments that are most disturbing. "If my books appear to a reader to be oversimplified, then you shouldn't read them: you're not the audience." This is an interesting twist on the more common authors' response that if the book is too complicated for you, try something else. What is Gladwell saying–that he's aiming for the lowest common denominator? Anyone who's read his work knows that is not the case. Excerpts from is books fit very nicely into the pages of The New Yorker, which publishes some of the most intelligent and literate writing in America.

Here are further comments from Gladwell, as collected by Chabris:

I am a story-teller, and I look to academic research … for ways of augmenting story-telling.

…we’re obsessed with things like coherence, consistency, neatness of argument. Readers are indifferent to those things.

Gladwell, it seems to me is admitting, in the first comment, to cherry-picking research that fits his thesis (which is based on…what?), and in the second he is saying that readers will happily settle for sloppiness and faulty argument. It's not true. Coherence and neatness of argument are precisely the values that Gladwell adds to the research he writes about. The art of nonfiction storytelling is framing a neat, consistent tale out of a jumbled pile of factual building blocks. (And notice that Gladwell doesn't say anything about accuracy.)

I have long been an admirer of Gladwell's; I wish I could put stories together the way he does. But I'm now afraid to read him. My work, my intellectual life, and even my social and emotional experiences with my family are based on knowing what's really going on–not Gladwell's made-up ideas of how things should be. I don't want to base my reading, or my life, on Gladwell's currency: things that might or might not be true, but which make possible masterful storytelling.

-Paul Raeburn