A lot of people are reading scientific papers for the first time these days, hoping to make sense of the coronavirus pandemic. If you’re one of them, be advised the scientific paper is a peculiar literary genre that can take some getting used to. And also bear in mind that these are not typical times for scientific publishing.
It is hard to think of another moment in history when so many scientists turned their attention to one subject with such speed. In mid-January, scientific papers began trickling out with the first details about the new coronavirus. By the end of the month, the journal Nature marveled that over 50 papers had been published. That number has swelled over the past few months at an exponential rate, fitting for a pandemic.
The National Library of Medicine’s database at the start of June contains over 17,000 published papers about the new coronavirus. A website called bioRxiv, which hosts studies that have yet to go through peer review, contains over 4,000 papers.
In earlier times, few people aside from scientists would have laid eyes on these papers. Months or years after they were written, they’d wind up in printed journals tucked away on a library shelf. But now the world can surf the rising tide of research on the new coronavirus. The vast majority of papers about it can be read for free online.
But just because scientific papers are easier to get hold of doesn’t mean that they are easy to make sense of. Reading them can be a challenge for the layperson, even one with some science education. It’s not just the jargon that scientists use to compress a lot of results into a small space. Just like sonnets, sagas and short stories, scientific papers are a genre with its own unwritten rules, rules that have developed over generations.
The first scientific papers read more like letters among friends, recounting hobbies and oddities. The first issue of the Philosophical Transactions of the Royal Society, published on May 30, 1667, included brief dispatches with titles such as “An account of the improvement of optick glasses,” and “An account of a very odd monstrous calf.”
When natural philosophers sent their letters to 17th-century journals, the editors decided whether they were worth publishing or not. But after 200 years of scientific advances, Victorian scientists could no longer be experts on everything. Journal editors sent papers to outside specialists who understood the details of a particular branch of research better than most scientists.
By the mid-1900s, this practice evolved into a practice known as peer review. A journal would publish a paper only after a panel of outside experts decided it was acceptable. Sometimes the reviewers rejected the paper outright; other times they required the fixing of weak points — either by revising the paper or doing additional research.
Along the way, scientific papers also developed a distinctive narrative arc. A paper published in Philosophical Transactions today is no longer a gossipy letter, but a four-part story. Papers typically open with some history, giving a justification for the new research they contain. The authors then lay out the methods they used to carry out that research — how they eavesdropped on lions, how they measured chemicals in Martian dust. Then the papers present results, followed by a discussion of what those results mean. Scientists will typically point out the shortcomings in their own research and offer ideas for new studies to see if their interpretations hold water.
As a science writer, I’ve been reading scientific papers for 30 years. I’d guess that I’ve read tens of thousands of them, in search of new advances to write about, or to do background research for stories. While I’m not a scientist myself, I’ve gotten pretty comfortable navigating around them.
One lesson I’ve learned is that it can take work to piece together the story underlying a paper. If I call scientists and simply ask them to tell me about what they’ve done, they can offer me a riveting narrative of intellectual exploration. But on the page, we readers have to assemble the story for ourselves.
Part of the problem may be that many scientists don’t get much training in writing. As a result, it can be hard to figure out precisely what question a paper is tackling, how the results answer it and why any of it really matters.
The demands of peer review — satisfying the demands of several different experts — can also make papers even more of a chore to read. Journals can make matters worse by requiring scientists to chop up their papers in chunks, some of which are exiled into a supplementary file. Reading a paper can be like reading a novel and realizing only at the end that Chapters 14, 30, and 41 were published separately.
The coronavirus pandemic now presents an extra challenge: There are far more papers than anyone could ever read. If you use a tool like Google Scholar, you may be able to zero in on some of the papers that are already getting cited by other scientists. They can provide the outlines of the past few months of scientific history — the isolation of the coronavirus, for example, the sequencing of its genome, the discovery that it spreads quickly from person to person even before symptoms emerge. Papers like these will be cited by generations of scientists yet to be born.
Most won’t, though. When you read through a scientific paper, it’s important to maintain a healthy skepticism. The ongoing flood of papers that have yet to be peer-reviewed — known as preprints — includes a lot of weak research and misleading claims. Some are withdrawn by the authors. Many will never make it into a journal. But some of them are earning sensational headlines before burning out in obscurity.
In April, for example, a team of Stanford researchers published a preprint in which they asserted that the fatality rate of Covid-19 was far lower than other experts estimated. When Andrew Gelman, a Columbia University statistician, read their preprint, he was so angry he publicly demanded an apology.
“We wasted time and effort discussing this paper whose main selling point was some numbers that were essentially the product of a statistical error,” he wrote on his blog.
But just because a paper passes peer review doesn’t mean it’s above scrutiny. In April, when French researchers published a study suggesting that hydroxychloroquine might be effective against Covid-19, other scientists pointed out that it was small and not rigorously designed. In May, a much bigger paper was published in the Lancet suggesting that the drug could increase the risk of death. A hundred leading scientists published an open letter questioning the authenticity of the database on which the study relied.
When you read a scientific paper, try to think about it the way other scientists do. Ask some basic questions to judge its merit. Is it based on a few patients or thousands? Is it mixing up correlation and causation? Do the authors actually present the evidence required to come to their conclusions?
One shortcut that can sometimes help you learn how to read a paper like a scientist is by making judicious use of social media. Leading epidemiologists and virologists have been posting thoughtful threads on Twitter, for example, laying out why they think new papers are good or bad. But always make sure you’re following people with deep expertise, and not bots or agents of disinformation peddling conspiracy nonsense.
Science has always traveled down a bumpy road. Now it is in an extraordinary rush, with the world looking for every new preprint and peer-reviewed paper in the hope that some clue will emerge that helps save millions of lives.
Yet our current plight does not change the nature of the scientific paper. It’s never a revelation of absolute truth. At best, it’s a status report on our best understanding of nature’s mysteries.
