COVID-19: Why is the science so confusing?

The fatality ratio is 5%, no it’s 0.5%, no it’s 0.01% but only in France. Masks will not protect you, but wear them because they will. Don’t touch the surfaces because COVID-19 likes to rest on them, but it actually disappears almost as soon as it settles on the surface. Why is COVID-19 science so hard to understand? There are two main reasons: 1) the science happens in front of our eyes and 2) there are multiple layers of noise and sources of error. 

The scientific method and the scientific process are our best tools to study and understand the world out of all tools currently available. Overall it is a fairly robust process. Imagine an entomologist working as a professor at a university. This entomologist has spent anywhere between 4 years required to complete a PhD to many decades studying insects and learning from other researchers with as many or more years of experience. She observes something about killer bees. She then formulates a hypothesis and reviews the existing scientific studies on killer bees to see if her hypothesis was tested by someone else. If it wasn’t or the existing data is not satisfactory in her opinion, she designs and executes a study, that tests her hypothesis or improves on existing studies. If in the process she finds her method to be flawed, which happens quite often, she revises it and starts again. Most frequently the researcher doesn’t work alone, but with collaborators. When she has initial data, she is likely to present her findings at a scientific conference on killer bees. There she would chat with many other researchers in the filed that would come by her poster, or listen to her talk. They will poke holes in her methods and findings, and ask poignant questions. They will also suggest improvements and further questions. Once she is back in the lab, the entomologist will finish her experiments and write a paper on it. In this paper she will have specify every minute detail of her study and the results. She would then submit it to a scientific journal. She will try for the best journal she thinks she can get the paper into, one that is well respected in the field. The paper will be reviewed by two or more other scientists that specialize in killer bees. They will scrutinize every detail of her paper, ask poignant questions and make suggestions for improvements. They might reject the paper right away and then she will have to improve it and submit to another journal. Or they might invite her to review in accordance to their comments and resubmit to the same journal. She might have to repeat a part of her study or add a new experiment. Usually the review-and-resubmit happens multiple times. Finally, the paper might be accepted for publication. If her findings on killer bees are new and unexpected or important to the field, other researchers in the field are likely to try to replicate them and extend them. When other researchers can’t replicate her results, a long data-based debate ensues until either the hypothesis is completely disproved or more data shows that it can be replicated after all or researchers find other explanations to the inconsistencies. All this usually takes years and some phenomena and hypotheses are studied for decades. So how does that translate to COVID-19 times?

(1) The science happens in front of our eyes: With COVID-19, this process is happening with the whole world watching and drumming their fingers impatiently. There is no time for the usual years-long scientific process to unfold. It is accelerated and curtailed and all the tribulations of honing in on the methods and the right data and the right hypotheses is very public instead of happening behind the ivory tower doors. For the vast majority that are not personally familiar with the academic process this looks like a terrible mess.

(2) There are multiple layers of noise and sources of error. The scientific method is not perfect. There is human error involved at all stages starting from hypothesis forming to subsequent replication of the results. Expediting this process increases the probability of error. Right now many COVID-19 papers are published in pre-print awaiting peer review and are immediately picked up by the media.  That is error/noise layer #1. The next layer is introduced by the media that takes the science and interprets it for the public. Sometimes news reporters misrepresent the data both because they don’t fully understand all the nuances and because they want a clear, juicy message that they can offer the public. This is error/noise layer #2. Then there is the layer of politics that goes on top of that, that is applied both by reporters and politicians – error/noise layer #3. Finally, there is the reader or the listener, that might not fully read and understand the news and the data and that also apply their own biases – error/noise layer #4.

So what we do about this? First, we have to go with what is currently available. The science might not be perfectly accurate, but it is the best we can do at the moment. The more time passes by, the more accurate the results will be. Second, try to find and read the sources of the scientific findings, if you can. Read papers from several countries, based on several different data sets to try and cut through the noise. Third, you can find people who can reliably cut through the noise for you and who are actively participating in the debate.