Days after a mysterious new illness was declared a pandemic in March of last year, a prominent scientist in France announced that he had already found a cure.
Based on a small clinical trial, microbiologist Didier Raoult claimed that hydroxychloroquine, a decades-old antimalarial drug, was part of a 100% effective treatment against COVID-19. Then–US president Donald Trump promptly proclaimed that the finding could be “one of the biggest game changers in the history of medicine.”
But the study seemed off to Elisabeth Bik, a scientist turned science detective living in Silicon Valley. Bik has a sharp eye for spotting errors buried in arcane scientific papers, particularly when it comes to duplicated images. And much about Raoult’s paper looked fishy, as she later noted on her blog. Unfavorable data was left out, and the trial’s timeline was mathematically impossible. “Something does not seem quite right,” she wrote.
Before long, Bik would learn the price of raising such concerns. Raoult and a coauthor went on to call her a “witch hunter,” a “mercenary,” and a “crazy woman” on Twitter and in the press. Then, in April 2021, Raoult’s collaborator announced that they had filed a criminal complaint against Bik and a spokesperson for PubPeer, a website where she and others post scientific criticism, accusing them of blackmail, extortion, and harassment. He tweeted out a screenshot of the complaint, revealing her home address to the world.
These were the most direct threats Bik had ever received for identifying problems in scientific research — an activity she sees as integral to science. Alarmed, she tweeted a plea: “I could use some legal help.”
Tens of thousands of discoveries about the coronavirus have been made over the last two years, launching countless debates about policy and behavior. How deadly is the virus? Who should wear masks and where? How well do the vaccines fend off infections? But to find the right answers, studies must be accurate, verifiable, and responsibly done. Do a paper’s numbers add up? Are the images real? Did the scientists do the experiment they describe doing, follow ethical standards, minimize bias, and properly analyze their results?
The answer to all these questions, even before the pandemic, was: not as often as you might think. And COVID has made science’s frequent inability to police itself a clear problem with incredibly high stakes.
Because as vital as error detection is to keeping the whole enterprise honest, those who do it say there is no individual upside. No one pays them to comb through papers for mistakes. On the other hand, it’s a great way to make enemies fast. “It pisses people off,” said Nick Brown, a fellow data sleuth who cut his teeth exposing sloppy food-marketing research in 2017.
Bik’s efforts to clean up science are immense: Since 2014, she’s contributed to the retractions of at least 594 papers and 474 corrections. But Raoult is a daunting adversary. He’s authored thousands of papers and heads a leading infectious disease research institute in France. And during the pandemic, he has become one of the world’s biggest champions of hydroxychloroquine. His Twitter following has swelled to over 850,000, more than twice that of France’s health minister. His institute’s YouTube videos, many of which feature him, have been viewed 96 million times.
The legal threat against Bik came at a highly vulnerable time for her. Two years ago, she quit her biotech industry job to be a full-time scientific misconduct investigator, piecing together a living from consulting, speaking fees, and Patreon donations. Within the scientific community, where fact-checking almost universally happens on one’s own time and dime, Raoult’s move to press charges was a clear warning.
“We support the work needed to investigate potential errors and possible misconduct and believe the scientific community can do more to protect whistleblowers against harassment and threats,” said a letter in support of Bik signed by more than 2,000 researchers and 30 scholastic organizations in May. They aren’t wrong to worry: More recently, other scientists have also sent legal threats Bik’s way.
Science watchdogs have always worked alone on the periphery of the research enterprise. The pandemic is laying bare how vulnerable — and vital — they are.
“I’m convinced there is a chilling effect,” Bik told BuzzFeed News. “I’m feeling the cold, too.”
Bik has always had a discerning eye. She swears that she is merely average at puzzles and slow to recognize faces, but patterns — like in tiles and floor panels — leap out at her. “I guess most people don’t see that,” she said over a Zoom call.
Growing up in Gouda, the Netherlands, Bik was an avid bird-watcher who dreamed of being an ornithologist. Later she traded in her binoculars for a microscope, earning a PhD in microbiology at the University of Utrecht. Her first job out of school, on staff at a hospital, involved scanning for infectious disease microbes in patients’ samples.
In the early 2000s, she moved with her husband to Northern California. For over a decade, she worked on early efforts at Stanford University to map and analyze the microbiome, the thriving communities of bacteria inside our bodies.
Bik’s first foray into scientific misconduct began with the accidental discovery that she was a victim of it. Around 2013, she was reading an academic article about plagiarism and, on a whim, plugged a random sentence from one of her papers into Google Scholar. It popped up, verbatim, in another author’s text. It was a turning point. If she had just chosen another sentence, she said, “my whole career might not have changed at that moment.”
Another lightbulb moment came when she was reading a graduate student’s PhD thesis on inflammation and cancer and laid eyes on a particular Western blot photograph. In these images, proteins show up as dark splotches, like grayscale Mark Rothko paintings. Bik realized that the same photo appeared in two different chapters, ostensibly for different experiments, and that research articles based on the thesis repeated the errors. She reported the duplicates to journal editors in 2014. Following a university investigation, the papers were retracted.
Her discoveries coincided with a burgeoning movement to ferret out bad science. In the early 2010s, some of psychology’s most high-profile findings began falling apart, whether because they were false positives generated from cherry-picking, could not be replicated by other labs, or, in rare instances, were outright fakes. Economics, artificial intelligence, and cancer research have also reckoned with their own crises.
Science is often mistakenly referred to as self-correcting. But peer reviewers — outside experts who review studies before they’re published in journals — are neither paid nor always qualified to assess the papers they’re assigned. Months or years can pass before journals correct or retract papers, if they ever do. And universities have little incentive to investigate or punish professors over questionable work. Nudging any of these entities into taking action tends to require behind-the-scenes work — and sometimes public pressure.
Enter the website PubPeer. Founded in 2012 by a scientist, a graduate student, and a web developer, it’s now a widely used forum where commenters can weigh in on any paper and study authors can respond. Posters can be anonymous. But PubPeer is not simply Reddit for research trolls: Critiques must be based on publicly verifiable information. As its FAQ states, “You can’t say, ‘My friend used to work in the lab and said their glassware is dirty.’”
Boris Barbour, one of PubPeer’s co-organizers, acknowledged that the site is “an experiment, sometimes an uncomfortable one — there’s not a safety net for some of what we do.” But he added that “it is a maybe necessary and certainly practical approach to making something happen, to correcting some of the literature.”
Bik single-handedly drives much of the discussion on PubPeer, where she’s flagged or weighed in on more than 5,500 papers. In 2016, she put her powers to the test. She looked up 20,621 papers that contained Western blots and manually scanned them for duplicates. Two microbiologists agreed with 90% of her picks. Together, they reported that 4% of the studies, which had appeared in 40 journals over nearly two decades, contained copied images, a “disturbingly common” phenomenon. In a follow-up, Bik found duplicated images in 6% of 960 papers from a single journal over seven years. Extrapolating out to the millions of biomedical papers published over the same period, that means that as many as 35,000 studies could be worthy of retraction, she estimated.
“She’s the Liam Neeson of scientific integrity,” said Brian Nosek, executive director of the Center for Open Science, a nonprofit that promotes reproducibility in science. “She has a remarkable eye for detection … it has a magician-like quality in some cases.”
When Bik, 55, sits down to work, she puts on her tortoiseshell reading glasses and zooms in on images on her curved 34-inch computer screen. Hundreds of tiny turtle figurines line her home office, a collection she tracks in a detailed spreadsheet. Hung above her workstation is an illustration of a peacock, flashing its eye-spotted feathers in all their colorful, patterned glory.
Only in the last year or so has Bik started using software to help scan for uncanny similarities. Otherwise, her process is manual, akin to close-reading clouds in the sky or bloodstains at a crime scene. When observing cells in an image, “I see it looks like a dog or fish or two cells squashed together,” she said. “I look for those same groups of cells in the other panel. It’s almost like there’s a little ping in my brain if I see them.”
Toward the end of March 2020, as cities and states shut down, Bik suddenly had even more time to put her scanning abilities to the test. And Raoult’s hydroxychloroquine study was making headlines worldwide.
After the SARS outbreak of 2002, Raoult had hypothesized that, based on lab studies, hydroxychloroquine and a related drug, chloroquine, could be “an interesting weapon” to fight future outbreaks. When early studies out of China identified chloroquine as a promising agent against SARS-CoV-2, Raoult promoted them — and then set out to test the idea himself.
In his study, 14 COVID patients admitted to hospitals in southern France in early March 2020 were treated with hydroxychloroquine, and six more also received azithromycin, an antibiotic. On the sixth day, most of the people who received no treatment were still COVID-positive. But he reported that about half of the patients on hydroxychloroquine alone, and all of the ones taking it with the antibiotic, were testing negative.
Bik had known of Raoult, a fellow microbiologist, and had seen Trump’s tweets raving about his latest discovery. Unlike most papers she scrutinizes, his didn’t have worrisome images. But other irregularities caught her eye.
Why, she wondered, did Raoult’s team leave out a number of patients who dropped out of the trial, including those who transferred to intensive care or died? Without these negative outcomes included, the results looked more promising. If the study received ethics approval on March 6, and the patients were tracked for 14 days, how did the authors submit their paper to the International Journal of Antimicrobial Agents on the 16th? And how was it accepted for publication less than 24 hours later? Impossible to ignore was the fact that one of the study’s authors, Jean-Marc Rolain, was the editor-in-chief of the journal.
“This would be the equivalent of allowing a student to grade their own paper,” Bik wrote on her blog, Science Integrity Digest, on March 24. “Low [sic] and behold, the student got an A+!”
Days later, the scientific society overseeing the journal said that an editor besides Rolain had been involved in reviewing the manuscript but admitted that the study was below its standards. It commissioned outside experts to take a closer look at whether concerns such as Bik’s had merit.
But by then, Raoult’s narrative that the drug was a miracle cure had assumed a life of its own. Emmanuel Macron, France’s president, traveled to Marseille to meet Raoult. Trump’s endorsement of the research, and later his claim that he was taking hydroxychloroquine himself, sent sales soaring and dried up supplies for patients who depend on it to treat lupus and rheumatoid arthritis. Then, in an abrupt move that shocked many scientists, the FDA authorized the drug for emergency use against COVID. Nearly 1 in 4 COVID-19 clinical trials launched that spring were studying hydroxychloroquine or chloroquine.
In April 2020, when Bik first raised alarms about Raoult’s study, the scientist was displeased. “The witchhunter @MicrobiomDigest is not attentive to details when she judges that a study is useful to her paranoiac fights!” he tweeted. “Fake news.”
By the end of the year, large clinical trials of hydroxychloroquine would find no effect against the coronavirus, and the FDA would revoke its authorization, citing the risk of severe heart complications.
Raoult’s was among the first of many COVID-19 studies to fall under the scrutiny of dedicated watchdogs like Bik. Researchers, students, journalists, and others have also spotted, sometimes by accident, things that don’t add up.
One of the biggest examples, ironically, drew a conclusion that was the opposite of Raoult’s: that hydroxychloroquine wasn’t just ineffective against COVID, it was also likely to kill you. In May 2020, that news led at least two major clinical trials to grind to a halt. But the basis for the explosive finding — a database compiled by a startup named Surgisphere — collapsed when outside researchers pointed out inconsistencies. Three of the paper’s authors admitted that their collaborator, Surgisphere’s founder, had refused to share the data with them. They retracted that paper from the Lancet and a second from the New England Journal of Medicine. (Surgisphere’s founder defended his company and claimed it was not responsible for any issues with the data.)
Allegedly fraudulent data had slipped past two of science’s most exclusive journals. But with preprints — essentially first drafts, uploaded straight to the internet — there aren’t even gatekeepers to blame. Being able to immediately share cutting-edge science is useful, especially in a pandemic. It also means no peer reviewer or journal editor is checking for oversights and methodological problems.
One widely publicized preprint reported that hospitalized coronavirus patients were 90% less likely to die when given ivermectin, an antiparasitic drug that proponents have touted as a cure-all. But a trio of sleuths found big problems in the data, including entries from dead patients. The preprint was taken down in July over “ethical concerns.” (Its lead author has defended the study and said he was not consulted before it was removed.)
“We need some minimum level of quality control. We’re churning out millions of papers.”
In the prepandemic era, you would put your preprint “on the table of the coffee break room and say, ‘Please, anybody, read it,’” said Nosek of the Center for Open Science. During the Zika outbreak of 2015 to 2016, 78 preprints were posted on one server, BioRxiv. In contrast, upward of 19,000 SARS-CoV-2 preprints have been uploaded to BioRxiv and a new server, MedRxiv, since the pandemic started.
Some say the deluge demands more oversight. “We need some minimum level of quality control,” Brown said. “We’re churning out millions of papers.”
But to Nosek, the issues raised by preprints predate preprints themselves. “The interesting thing of the moment is almost all of the events are entirely ordinary — not in terms of [being] acceptable, but ordinary,” he said. “Yes, this is what’s happening in research practice all the time.”
Now, however, the stakes of getting things wrong are unbelievably high. In June, a group of scientists wrote in JAMA Pediatrics — another prestigious journal — that children in face coverings were inhaling “unacceptable” levels of carbon dioxide. Jay Bhattacharya, a Stanford University professor of medicine, praised it on Fox News and called mask-wearing “child abuse.” Soon after, Florida Gov. Ron DeSantis, whom Bhattacharya has advised, blocked schools from requiring masks in the classroom, claiming in an executive order that “forcing children to wear masks could inhibit breathing.”
That study was retracted by the journal after scientists complained about its methodological problems. (The authors have said they stand by their findings and that their critics were not qualified to judge them.)
One of the study’s most outspoken detractors was James Heathers, a longtime data detective. He believes that many are taking advantage of the pandemic to build their personal brands. “There are people in science who think basically any crisis is an opportunity, anything that becomes a topic du jour is something they should chase,” he said, adding that he wasn’t referring to anyone in particular. “A lot of COVID work is an extension of that same mentality” — that is, “maximally flashy and minimally insightful.”
Until spring 2020, Raoult was best known as an eminent microbiologist who founded and heads the research hospital Institut Hospitalo-Universitaire Méditerranée Infection, or IHU. He has discovered or codiscovered dozens of new bacteria — a group of them are named Raoultella — as well as giant viruses. By many accounts, his extensive reach in the scientific community is matched by his temper: In 2012, Science magazine described him as “imaginative, rebellious, and often disdainful.” “He can make life hard for you,” one researcher said.
A handful of Raoult’s thousands of publications have also fallen under scrutiny. In 2006, the American Society for Microbiology banned him and four coauthors from its journals for a year over a “misrepresentation of data” after a reviewer spotted figures that were identical, but shouldn’t have been, across two versions of a submitted manuscript. (Raoult objected to the ban, saying he wasn’t at fault.) And some researchers noticed that Raoult was on one-third of all papers to ever appear in a single journal, which was staffed by some of his collaborators.
Last year, Raoult’s team issued a correction to a 2018 study, and another from 2013 was retracted altogether (the journal said that Raoult could not be reached when it was making its decision). Both contained apparently duplicated or otherwise suspect images, first spotted by Bik, who has flagged more than 60 other studies of his on PubPeer for potential issues.
And by July of last year, his most infamous study had been looked over by even more outside experts commissioned by the journal’s publishers. The scientists did not hold back. “Gross methodological shortcomings,” “non-informative,” and “fully irresponsible,” one said. Another group said it “raised a lot of attention and contributed to a demand for the drug without the appropriate evidence.”
Despite acknowledging these flaws, the leaders of the International Society of Antimicrobial Chemotherapy, which publishes the journal along with Elsevier, opted not to retract the study. “We believe, in addition to the importance of sharing observational data at the height of a pandemic, a robust public scientific debate about the paper’s findings in an open and transparent fashion should be made available,” they said. Around the same time, a group of 500 French infectious disease experts filed a complaint with local health officials, accusing Raoult of spreading misinformation about hydroxychloroquine.
Raoult defended his “seminal work,” arguing that the call for a retraction had “no justification other than the opinion of people who were fiercely hostile to” hydroxychloroquine. At a French Senate hearing that September, he once again downplayed criticisms of his research. Bik had “managed to find five errors in a total of 3,500 articles,” he said, while acknowledging that there were potentially a small number of other errors as well. He denied ever committing fraud.
At the Senate hearing, Raoult called Bik a term that translates to “head hunter,” a “girl” who had been “stalking” him since he was “famous.” And around Thanksgiving, biologist Eric Chabrière, a frequent collaborator of Raoult’s and a coauthor of the hydroxychloroquine study, tweeted that Bik “harasses” and “tries to denigrate” Raoult.
He invoked her past employment at uBiome, a microbiome-testing startup that the FBI raided in 2019. (Bik, who was scientific editorial director there until the end of 2018, has said that she was never questioned and was not involved in the founders’ alleged scheme to defraud insurers and investors.) Chabrière also accused her of being paid by the pharmaceutical industry.
“I am not sponsored by any company, but you can sponsor me at @Patreon,” Bik tweeted back, linking to her account. As she explained to Chabrière, she is also a consultant to universities and publishers who want suspicious papers investigated.
“Happy to investigate any papers of your institute, too, as long as you pay me :-),” she added.
Over the following months, Chabrière would call her “a real dung beetle,” “a mercenary who only obeys money,” and a person “paid to attack and discredit certain targets.” His supporters piled on, sometimes with vague threats. Meanwhile, Raoult called her a “crazy woman” and a “failed researcher” of “medium intelligence.”
Then, on April 30 of this year, Chabrière tweeted a screenshot of a legal complaint allegedly filed with a public prosecutor in France. It accused her and Barbour, the PubPeer co-organizer, of “moral harassment,” “attempted blackmail,” and “attempted extortion.” Her home address was listed. The tweet was later deleted.
“There’s something unhelpful in the way we think about science as a self-correcting process. It makes you think that it’s just going to correct itself on its own.”
According to the French newspaper Le Monde, the basis of the blackmail allegation was her tweet offering to investigate papers for a fee. The complaint also noted that a total of 240 papers by Raoult and nearly 30 by Chabrière were flagged on PubPeer, mostly by anonymous commenters. “As long as we stick to scientific criticism, this is beneficial to science. But there, it goes beyond the limits and prevents my clients from working,” a lawyer for Raoult and Chabrière told the newspaper.
Bik stands by her critiques and denies ever blackmailing or harassing anyone. And as of October, she said she had not seen the full complaint or been contacted by any attorneys or authorities. Raoult, Chabrière, and their lawyer did not return multiple requests for comment from BuzzFeed News.
The episode highlighted the divisive rise of public peer review, where hundreds of people can instantly weigh in on a finding. Young and internet-fluent scientists tend to look favorably on this shift toward transparency. But others argue that “cancel culture campaigns in social media,” as one oft-criticized researcher has put it, taint the scientific process.
That unease was apparent in a statement on Raoult’s legal filing from the French National Centre for Scientific Research, where Barbour, the PubPeer co-organizer, is a neuroscientist. While calling critiques “indispensable when they are constructive and backed by cogent arguments,” the institution admitted that it had “serious reservations” about the fact that PubPeer critics do not have to share their real names. This, it wrote, contributes to “the excesses of certain social networks for which anonymous insults and accusations are commonplace.” (Barbour declined to comment on the complaint.)
But some data sleuths point out that threats like Raoult’s are a good reason to stay anonymous. And while scientific discourse is traditionally polite, deliberate, and conducted behind closed doors, they say that doesn’t work during a pandemic.
After Hampton Gaddy, an undergraduate student at the University of Oxford, inquired about 26 fishy COVID studies by a single researcher and made his complaints public, all of them were withdrawn. The author did not dispute the retractions.
“There’s something unhelpful in the way we think about science as a self-correcting process,” Gaddy said. “It makes you think that it’s just going to correct itself on its own.”
Not long after Raoult’s criminal complaint was announced, attorneys came after Bik over different critiques. These involved a professor in China who claimed that he could kill cancer cells in a petri dish by “emitting external Qi,” the life force believed in traditional Chinese medicine to exist in everything. He repeated this procedure in more than a half-dozen studies, often with Harvard-affiliated researchers.
In 2019, Bik accused the studies of failing to describe the process in sufficient detail. But in a pair of cease-and-desist letters in May, lawyers for the scientists argued that they had properly described their methods, accusing her of publishing false and defamatory statements and mocking Chinese medicine.
Bik deleted her tweets but refused to retract her blog post or PubPeer comments. “This is a scientific discussion,” she wrote back to one attorney.
She also found it curious that it took two years for these lawyers to come knocking. “I think they thought I was being threatened by Didier Raoult and then decided, ‘Maybe she’s in a vulnerable position, let’s slap on another threat,’” Bik said. (The attorneys did not return requests for comment.)
While Bik accepts that blowback comes with the territory, she has less of an appetite for needless conflict these days. She regrets joking with Chabrière as she did and has toned down the sarcasm on Twitter, where 111,000 people now follow her every word. “I feel more watched,” she said. “I think about what I tweet and how that could look in a courtroom.” That said, as one of the few women widely known for being a science watchdog, Bik has always been conscious of how she comes across and is used to constantly being questioned by men. “It’s a very thin line as a woman that we have to make between saying what we think is right and not coming across as very aggressive,” she said.
A degree of paranoia also colors her offline life. Upon trying to enter the Netherlands on a recent trip, she went to scan her passport and the machine informed her there was an error. As an employee walked over, the first thought that went through her head was Oh my god, I’m going to be arrested right now. (It was just a glitch.)
Brushes with the law may still be rare for scientific fact-checkers, but being on the receiving end of antagonism isn’t.
“People hate you,” said Gideon Meyerowitz-Katz, an epidemiology graduate student at the University of Wollongong in Australia who has dug through some of the pandemic’s most flawed studies. “Even people who are not involved with the study think you are a nasty, grubby troll sitting in a basement finding mistakes in others’ work.” Having ruffled all the feathers he’s ruffled, he feels unsure over what his post-PhD future holds.
That’s why data sleuths don’t usually rely on fact-checking to pay the bills. They support themselves through any number of other ways — attending graduate school (Meyerowitz-Katz is working at a public health agency while finishing his degree), working at a company (Heathers), or being retired (Brown). That makes their “job” inaccessible to most people, they said.
“If you are someone in that precarious position or someone who is a person of color from a disadvantaged background, doesn’t have financial resources, and can’t afford to ever be sued or even [face] the threat of a lawsuit, they’re just driven away from it,” Meyerowitz-Katz said.
Is there a future where watchdogs have proper careers, funded by the institutions they’re trying to fix? Nosek, a psychology professor at the University of Virginia, thinks that they have a place in the system. Funders could back fellowships for data sleuths “so they can dedicate time rather than having it be marginalized work,” he said.
But Brown believes that he and his colleagues are most effective on the margins, where they are beholden to no one but themselves. “The instant you have somebody funding you to do this kind of thing,” he said, “it’s like, ‘Why did you fund Nick Brown?’”
“The fact you can do everything she’s done and still be in a position where the system hasn’t directly rewarded you speaks very poorly of that system.”
As someone who makes a living exposing bad science, Bik is exceptional in more ways than one, her peers say.
“She should be receiving awards and prizes. Journals should be asking her to check stuff,” Heathers said. “The fact you can do everything she’s done and still be in a position where the system hasn’t directly rewarded you speaks very poorly of that system.”
Last month, the dispute between Bik and Raoult seemed to be winding down. The founding members of the IHU Méditerranée Infection announced that Raoult will be replaced as the head of the institution next September. The head of Marseille’s hospital system cited the need to “turn a page.” The decision, which Raoult protested, came amid reports that some of his studies are under investigation for alleged ethics violations.
In a recent interview, Bik said she felt optimistic that this one particular feud appeared to be quieting down. There are so many other fights to focus on: more dodgy images, more suspect papers, more scientists and journals and universities needing to clean up their acts. It’s become the pattern of her life.
“I’ll probably be doing this for a while, until all science misconduct has been resolved and all science is completely honest and clear,” she said with a laugh. “And then I can retire, I guess.”
But Raoult, it seems, is not quite ready to move on. Just last week, he said in a YouTube video that the people who made “attempts to blacklist us on scientific journals … will have to be arrested … including Madame Bik,” according to a translation that Bik shared on Twitter. She quickly locked her account to, she said, “prevent the next wave of insults, jail threats, and death wishes from reaching me.” Retirement would have to wait another day. ●
Oct. 18, 2021, at 22:35 PM
Correction: PubPeer was founded by a scientist, a graduate student, and a web developer. An earlier version of this story misidentified the founders.