Volume 5, 2022
|Number of page(s)||2|
|Section||Life Sciences - Medicine|
|Published online||12 August 2022|
Meta-analysis in a time of pandemic
Sylvia Lawry Centre for Multiple Sclerosis Research e.V. – The Human Motion Institute, Munich, Germany
2 Theodor-Billroth-Akademie®, Germany, USA
3 INCORE, International Consortium of Research Excellence of the Theodor-Billroth-Academy®, Germany, USA
4 Precision Medicine Unit, Lausanne University Hospital, Chemin des Roches 1a/1b CH-1010 Lausanne, Switzerland
5 Risk-Based Decisions Inc., Sacramento, CA, USA
6 Department of Surgery, Carl-Thiem-Klinikum, Cottbus, Germany
* Corresponding author: email@example.com
Accepted: 12 July 2022
Key words: Contact tracing / Coronavirus / COVID-19 / meta-analysis / Modeling / Pandemic / Reproduction rate / SARS-CoV-2 / Statistics / Virus
There is a perception that meta-analyses can provide neat and concise conclusions which can then be used for clinical treatment guidelines or in public health as in the on-going Coronavirus disease 2019 (COVID-19) pandemic. The widespread popularity of meta-analyses also presents a risk in that inconclusive results in the original individual studies, evaluated under the rubric of a meta-analysis, become accepted based on flawed methodologies and thus be used inappropriately for decision-making, especially for easy to implement, but potentially incorrect, patient treatments or public health measures. Recently a meta-analysis was published in 4open as a means to obtain a better estimate of the serial interval (SI) for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) during the COVID-19 pandemic . The author aimed to reduce the uncertainty around the SI for different COVID-19 variants by performing a meta-analysis of publications between 1st December 2019 and 15 February 2022 and concluded: “The meta-analysis was unable to provide a suitable estimate of serial intervals for Covid-19 modeling purposes although its uncertainty was reduced. Furthermore, serial intervals estimate for alpha variant was close to earlier reports and lower than previous publications, respectively. Another limitation is, that meta-analysis of COVID pandemic studies in principle contains and produces itself a significant source of heterogeneity.”
The author addresses the heterogeneity of SI estimates using a random effects model. However, one should distinguish between two sources of heterogeneity: Firstly, the studies underlying the meta-analysis used different methods to estimate SI, e.g. selection of infector–infectee pairs. For this source of variability, the use of a random effects model seems appropriate although one may question if these pairs can be considered as typical, as other plausible transmission modes, e.g. overcrowded living conditions or public transportation, do not show up in the data. Secondly, the source of heterogeneity includes studies that address different stages of the pandemic which could result in different SI estimates.
The basic reproduction number R0 (expected number of infections generated by one case in a fully susceptible population) and the generation interval (average time interval between primary and secondary infection, approximated) are key parameters in the modeling of the spread of COVID-19. For obvious reasons, the public discussion on control measures was focused on R0, which determines if the epidemic will spread or subside. The generation interval, approximated by the SI (time between symptom onset between infector and infected) is the time for the infection to be observed, which is necessary in order to estimate R0 from incidence data. Since all control measures were aiming to decrease the contact rate and hence R0, it would be absurd to perform a meta-analysis in order to estimate a universal R0. The SI, however, is linked to the infectious period and, in fact, if control measures would randomly filter out a proportion of infectious contacts, the SI would be unaffected. However, control measures, such as early isolation, can have a considerable effect on the SI (decreased by a factor of 3) . Therefore, the term “effective serial interval” was introduced to distinguish it from the biological period of infectiousness of a subject.
Jusot  included  in his meta-analysis and referred to this change of the SI estimate. He also noted that control measures in the studies underlying the meta-analysis were not sufficiently described in the underlying literature to include this factor into his meta-analysis. However, he did not draw the conclusion that estimating a universal SI is as meaningless as estimating a universal R0, but instead investigated the underlying publication bias. The rationale for the use of the Egger’s test is that studies showing no statistically significant effect are more likely to remain unpublished. This can result in a bias towards larger effects for small studies (skewness and asymmetry in funnel plots). But would there be a reason for a publisher to be biased towards small SI estimates since they are likely to have a small standard error?
Concerning decision making, forecasting of the time course of the COVID-19 epidemic is desirable and since the beginning attempts have been made to make long-term forecasts of various COVID-19 consequences to better plan intensive care beds and to decide on control measures (lockdowns). Increasing intensive care capacity however requires a time scale which is far beyond the forecasting horizon. When comparing forecasts for the second wave over a four week time frame from 19th Oct until 19th Nov 2020 ( and see also ), report forecasts ranging from a return to the lower incidence of previous weeks to exponential growth. No validated model was available at that time, and the model that performed best for the 19th Oct 2020 forecast did not necessarily perform well for 19th Nov 2020 forecast. Notably, in regard to planning of intensive care unit requirements, statistical modeling is not helpful beyond worst-case scenarios.
With regard to short-term forecasting for decisions on control measures, various phenomenological models have been proposed which are not based on SI or R0. For example,  presented a model that gave reliable short-term estimates for bed occupancy in various hospitals, although it was applied during the highly variable dynamics of the first wave in 2020. Here, the hospital admission rate (arrival rate as term in accordance to the queuing theory) was ad hoc modeled by fitting a Richard’s curve based on data available at the hospital. Models like this are exactly what are needed for short-term decisions on the need for lockdowns. The only disadvantage is the lack of interpretable parameters like R0 for public communication of decisions.
In summary, the author correctly questioned and illustrated what can be gleaned from his meta-analysis . The dynamics of the pandemic makes it complicated as the historical SI values are no longer valid given the continuous change in control measures, vaccination status, and vaccination efficacy.
The manuscript was intellectually supported by the Theodor-Billroth-Academy® (TBA®) and INCORE, (International Consortium of Research Excellence) of the (TBA®). We highly acknowledge the peer-review by Marjan Slak Rupnik, Senior Editorial Board member in Life Sciences-Medicine of 4open by EDP Sciences.
Conflict of interest
The author reports the following conflict of interest: Björn LDM Brücher is Editor-in-Chief in Life Sciences-Medicine of 4open by EDP Sciences. Ijaz S Jamall is Deputy Editor-in-Chief in Life Sciences-Medicine of 4open by EDP Sciences. Martin Daumer and Romain-Daniel Gosselin are Senior Editorial Board members. The authors alone are responsible for the content and writing of this Editorial. This manuscript contains original material that has not previously been published. Each author contributed equally to its contents and approved the manuscript.
An independent peer-review before submission was performed by Professor Dr. Marjan Slak Rupnik, Center for Physiology and Pharmacology, Medical University of Vienna, Vienna, Austria, and Senior Editorial Board member in Life Sciences-Medicine of 4open by EDP Sciences.
Nomenclature of abbreviation
- Jusot JF (2022), An update of serial interval estimate for COVID-19: a meta-analysis. 4open 5, 16. [Google Scholar]
- Ali ST, Wang L, Lau EHY, Xu XK, Du Z, Wu Y, Leung GM, Cowling BJ (2020), Serial interval of SARS-CoV-2 was shortened over time by nonpharmaceutical interventions. Science 369(6507), 1106–1109. https://doi.org/10.1126/science.abc9004. [CrossRef] [PubMed] [Google Scholar]
- Bracher J, Wolffram D, Deuschel J, Görgen K, Ketterer JL, Ullrich A, Abbott S, Barbarossa MV, Bertsimas D, Bhatia S, Bodych M, Bosse NI, Burgard JP, Castro L, Fairchild G, Fuhrmann J, Funk S, Gogolewski K, Gu Q, Heyder S, Hotz T, Kheifetz Y, Kirsten H, Krueger T, Krymova E, Li ML, Meinke JH, Michaud IJ, Niedzielewski K, Ożański T, Rakowski F, Scholz M, Soni S, Srivastava A, Zieliński J, Zou D, Gneiting T, Schienle M, List of Contributors by Team (2021), A pre-registered short-term forecasting study of COVID-19 in Germany and Poland during the second wave. Nat Commun 12(1), 5173. https://doi.org/10.1038/s41467-021-25207-0. [CrossRef] [PubMed] [Google Scholar]
- Ioannidis JPA, Cripps S, Tanner MA (2020), Forecasting for COVID-19 has failed. Int J Forecast 38(2), 423–438. https://doi.org/10.1016/j.ijforecast.2020.08.004. [Google Scholar]
- Baas S, Dijkstra S, Braaksma A, van Rooij P, Snijders FJ, Tiemessen L, Boucherie RJ (2021), Real-time forecasting of COVID-19 bed occupancy in wards and Intensive Care Units. Health Care Manag Sci 24(2), 402–419. https://doi.org/10.1007/s10729-021-09553-5. [CrossRef] [PubMed] [Google Scholar]
Cite this article as: Lederer C, Daumer M, Gosselin R-D, Jamall IS & Brücher BLDM, et al. 2022. Meta-analysis in a time of pandemic. 4open, 5, E5.
© C. Lederer et al., Published by EDP Sciences, 2022
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Current usage metrics show cumulative count of Article Views (full-text article views including HTML views, PDF and ePub downloads, according to the available data) and Abstracts Views on Vision4Press platform.
Data correspond to usage on the plateform after 2015. The current usage metrics is available 48-96 hours after online publication and is updated daily on week days.
Initial download of the metrics may take a while.