John Jarcho, on What It Takes to Make It Into the World’s Oldest Medical Journal

Jarcho, Deputy Editor of the New England Journal of Medicine, spoke to Knight Science Journalism fellows about the journal’s famously rigorous review process. 

“Most medical journals don’t have the resources or the luxury of having full-time editorial staff,” said John Jarcho, deputy editor of the New England Journal of Medicine (NEJM), during an event hosted by the Knight Science Journalism program on November 21^st.

But NEJM is not most medical journals. Started in 1812, it is the oldest continuously published medical journal in existence. For scientists across the globe, it has become one of the most coveted venues for publishing groundbreaking medical research.

Part of what sets NEJM apart, Jarcho explained, is its thorough editorial and peer review process. The journal employs editors from six major areas of medicine who review papers corresponding to their backgrounds. Jarcho handles cardiovascular medicine, his specialty at Harvard Medical School and at Brigham and Women’s Hospital in Boston. There, he said, his clinical interests in cardiomyopathy, congestive heart failure, and heart transplantation are informed by the cardiology research manuscripts he fields as deputy editor for NEJM.

Manuscripts submitted to NEJM face a gauntlet of six review stages, a detail-oriented process that draws together a series of experts to collaborate and make reports better. Roughly 10 percent of the submissions NEJM receives get rejected right away; half are tossed out by their handling editors; another quarter are rejected during peer review; and still others are turned away during final editorial meetings and statistical review.

Jarcho shared five key criteria by which NEJM editors gauge submissions: Originality, importance, quality, the possibility of error — which Jarcho said is “the thing that keeps editors up at night” — and ethics. Roughly five percent of the submissions that NEJM receives pass muster and make it into the journal. Everything else meets rejection somewhere along the way. Papers based only on animal research or studies based on a small number of subjects are likely to be rejected summarily; Jarcho said these wouldn’t “likely be of high interest” to NEJM readers.

Throughout the discussion, Jarcho shared insights on best practices for reviewing new medical research. He stressed that it was critical for journalists to scrutinize between clinical trials and observational studies, cautioning that the latter method isn’t randomized and often exhibits confounding factors that can muddy conclusions. To illustrate his point, he walked attendees through two cases — studies on cancer risks and on coronary disease in post-menopausal women — that showed how critical randomization is to determining the significance of a study’s results.

“[Randomization] makes it possible to truly isolate the effect of the treatment,” Jarcho said. “But the reality is that a very large number of the studies that you read are observational studies.”

Jarcho thinks the next big issue confronting journalists and editors who review medical research will be knowing when to trust big data in medicine, particularly when it’s applied to observational studies. “The thing that’s important to know is that most of the problems with observational data I have mentioned are not solved by making the numbers bigger,” Jarcho said.

With its rigorous review process, NEJM is prepared to scrutinize all manner of new studies — from flashy reports driven by big data to medical advances emerging from more established techniques. “We have the ability to think a little bit harder about the work that we’re evaluating, because we’re not doing it on the fly,” Jarcho said.