Early on Feb. 1, John Inglis picked up his phone and checked Twitter, as he does most mornings. He was shocked at what fresh hell awaited.
Since 2013, Dr. Inglis, executive director of Cold Spring Harbor Laboratory Press in New York, has been helping manage a website called bioRxiv, pronounced “bio archive.” The site’s goal: improve communication between scientists by allowing them to share promising findings months before their research has gone through protracted peer review and official publication.
But the mess he was seeing on Twitter suggested a downside of the service provided by the site, known as a preprint server, during the emerging coronavirus pandemic. The social media platform was awash with conspiracy theories positing that the new coronavirus had been engineered by the Chinese government for population control. And the theorists’ latest evidence was a freshly submitted paper on bioRxiv from a team of Indian researchers that suggested an “uncanny similarity” between proteins in H.I.V. and the new virus.
Traditionally, the Indian researchers would have submitted a paper to a peer-reviewed journal, and their manuscript would be scrutinized by other scientists. But that process takes months, if not more than a year. BioRxiv, medRxiv — another site co-founded by Dr. Inglis — and other preprint servers function as temporary homes that freely disseminate new findings. For scientists on the front lines of the coronavirus response, early glimpses at others’ research helps with study of the virus. But there is a growing audience for these papers that are not yet fully baked, and those readers may not understand the studies’ limitations.
Views and downloads on medRxiv, for instance, have increased more than 100-fold since December, Dr. Inglis says. People with little scientific training, or none at all, are desperate for new knowledge to better inform their day-to-day decisions. The news media wants to keep readers and viewers updated with the latest developments. And agents of disinformation seek to fuel conspiratorial narratives.
“Anyone who reads a preprint will embrace it almost in a blind fashion,” and they might cherry pick information that fits their worldview, said Eric Topol, director of the Scripps Research Translational Institute in San Diego and a member of bioRxiv’s advisory board.
The use and misuse of what’s posted on preprint servers is challenging the normal operations of these sites, and raising questions about how these and other forms of scientific publishing should function during a pandemic.
“Science is a conversation,” said Dr. Ivan Oransky, a physician and co-founder of Retraction Watch, a blog that reports on retractions of scientific papers. “Unfortunately people in times of crisis forget that science is a proposition and a conversation and an argument. I know everybody’s desperate for absolute truth, but any scientist will say that’s not what we’re dealing with.”
Science’s first draft
In November 2013, Cold Spring Harbor Laboratories — a 130-year-old research institution — launched bioRxiv. Inspired in part by arXiv, a preprint server focusing on the physical sciences that started in 1991, Dr. Inglis and Richard Sever, a colleague at Cold Spring, hoped that the rapid dissemination of new biological research findings could help other researchers around the world validate or use that data. Interest in bioRxiv then drove the launch of medRxiv, which focuses on health sciences, in July 2019.
To submit a paper to these two servers, researchers simply need to get all co-authors on board. MedRxiv authors need to declare that their study had ethical approval and participant consent, state any conflicts of interest and if the study was a clinical trial.
Once a paper gets uploaded, it’s subject to a screening process: Was it plagiarized? Is it science? Manuscripts that make a health claim on bioRxiv — for instance, if they say doing something could increase the risk of developing a disease or condition — are not permitted. MedRxiv won’t accept individual case reports or manuscripts with small sample sizes. Both servers decline manuscripts with findings that may cause harm by changing human behavior.
“We are trying to be responsible,” Dr. Inglis said, “to not create expectations that are overblown, or are even potentially risky or dangerous.”
Manuscripts that pass the screening are generally posted on bioRxiv within 48 hours, while medRxiv screening is more complex and takes longer. According to Dr. Inglis, at least 70 percent of preprints on bioRxiv are published in peer-reviewed journals within two years; medRxiv is less than a year old, so its rate so far is much lower.
Authors can withdraw their manuscript if they no longer stand behind the work. Among the Covid-19 papers that have been uploaded to both servers — 1,558 and growing — two have been withdrawn from bioRxiv and two from medRxiv.
Although preprints can rapidly add to important scientific discourse — a necessity during a pandemic — they often read like first drafts, and may contain language that risks misleading people who lack scientific expertise, says Samantha Yammine, a science communicator in Toronto. She says this creates problems when media outlets pick up on these studies.
Raising the bar
On Feb. 2, the day after Dr. Inglis discovered the swarm on Twitter around the study comparing H.I.V. and coronavirus, the Indian researchers withdrew their paper after other scientists knocked down its findings.
Faced with the public misuse of the Indian team’s findings, Dr. Inglis and Dr. Sever decided to add a more prominent notice to readers than was already on the site for those who might not be familiar with preprints.
Now, a yellow banner on every manuscript at bioRxiv warns readers that coronavirus papers on the site are “preliminary reports that have not been peer-reviewed. They should not be regarded as conclusive, guide clinical practice/health-related behavior, or be reported in news media as established information.”
But in subsequent weeks, there have been more challenges.
One manuscript to medRxiv uploaded on March 10 said that transmission of the new coronavirus by respiratory secretions in the form of droplets or aerosols “appears to be likely.” A few days later, the authors uploaded a second version that took out that language. The following week, the New England Journal of Medicine published a peer-reviewed version of the paper. But by then, numerous news articles had been written based on versions of the paper that had not been scrutinized by other scientists.
Dr. Inglis and his colleagues at bioRxiv and medRxiv have placed more limits on coronavirus submissions. On bioRxiv, scientists with expertise in outbreaks are taking a look at those papers. Since mid-February, they are rejecting manuscripts that propose possible coronavirus treatments solely based on computer modeling.
Some authors denied publication on the servers are understandably disappointed. “We might have been more willing to take this kind of work in the past,” Dr. Inglis said, “but now people are so desperate for things to work, I think it’s entirely OK for us to raise the bar to show more evidence.”
Such problems are not confined to preprint servers. Peer-reviewed journals are also receiving a greater volume of submissions about the novel coronavirus, and reviewers are working through them at a breakneck pace. “All the top journals have gotten lightning fast,” Dr. Topol said.
While some of these peer-reviewed findings have helped other scientists, others have been exaggerated on social media and by traditional news outlets. One example was a study about the potential of combining anti-malarial and antibiotic drugs to treat Covid-19. President Trump touted it as “one of the biggest game changers in the history of medicine.” The paper’s publisher is now investigating its findings. To date, there is no conclusive data that suggests these drugs work.
Many prominent journals publishing coronavirus research have also seen a rise in online readership. “We have seen a huge surge in traffic to our website since the pandemic started, and we think it’s safe to assume that a portion of those visits came from a non-physician audience,” said Jennifer Zeis, the media relations director for the NEJM.
Some of these journals, such as Science, Nature and the NEJM have also made their coronavirus research free to the public.
Dr. Yammine thinks that preprint servers and peer-reviewed journals should “encourage scientists to lead the fight against misinformation by making it mandatory to include a plain-language summary outlining the scope and limitations of their work prior to publication,” she said.
“Every drug ad has to end with: ‘talk to your doctor,’” Dr. Oransky said. “Maybe every preprint header or primer should end with, ‘talk to your neighborhood statistician,’” to determine how legitimate a paper is — or is not.
