Types of studies

The criteria for study selection have evolved. Changes in protocols are detailed in Appendix 1. .

We will include randomized controlled trials, whatever the trial design, including cluster‐randomized trials and crossover trials.

We will exclude early‐phase clinical trials, single‐arm trials, non‐randomized studies or modelling studies of interventions for COVID‐19.

We will exclude studies about prognosis, systematic reviews and meta‐analyses and diagnostic test accuracy studies.

We have no restriction on language.

Types of participants

For each research question, we will consider different participants, as detailed in each specific sub‐protocol.

Types of interventions

For each research question, we will consider different interventions. Briefly, eligible interventions will include the following.

• Preventive interventions

  • Prophylactic pharmacologic treatments for preventing SARS‐CoV‐2 transmission

  • Vaccination

  • Non‐pharmacologic interventions for preventing SARS‐CoV‐2 transmission (e.g. personal protective equipment)

• Interventions for treating COVID‐19 (anti‐infectious agents, specific and non‐specific immunomodulators, supportive treatments for hospitalized patients, general treatments for viral infection)

• Post‐acute care interventions

  • Rehabilitation

  • Any other treatment for long‐term effects of COVID‐19

The treatments and preventive interventions considered in this systematic review will likely expand over time to take into account new emerging management options and combination regimens. We will update this list according to the results of the mapping of all RCTs registered on ICTRP, which is updated weekly Nguyen 2020. 

Interventions will be included in the same NMA only when we anticipate that any patient who meets the pre‐defined inclusion criteria would, in principle, be equally likely to be randomized to any of the interventions within a network.

Types of outcome measures

Our outcome selection will be based on:

• the CORE outcome sets identified through the COMET initiative, where available (www.comet-initiative.org/Studies/Details/1538);

• the results of the research mapping that will describe the outcomes assessed in ongoing trials; and

• the clinical outcomes relevant to patients (i.e. patient‐important outcomes).

The outcomes considered will evolve over time to take into account the new CORE outcome sets being developed by the COMET initiative, and any other important outcomes that may arise over time.

We will describe outcomes as critical (primary) and important (secondary). We will define outcomes in the specific sub‐protocols for each research question.

Electronic searches

From 4 September 2020, we will search only the following secondary sources. 

• The L·OVE platform (app.iloveevidence.com/covid19). Details of this platform are available in Appendix 3.

• The Cochrane COVID‐19 Study register (covid-19.cochrane.org).

We will no longer search PubMed, CNKI, MedRχiv and Chinaxiv, which we had previously been searching.

We will regularly contact investigators of ongoing studies to update the status of their study and obtain results.

We will also search the Retraction Watch Database for retracted studies (retractionwatch.com/retracted-coronavirus-covid-19-papers).

We recognize that preprint articles are not peer‐reviewed, and are living documents that can be updated or published. In collaboration with a research team from the French National Centre for Scientific Research (Centre National de la Recherche Scientifique; CNRS), we developed a preprint tracker that systematically informs us when a preprint is updated or published. As soon as an update is identified, we record the data and run the analysis if needed.

Searching other resources

We will search several resources for unpublished and ongoing studies.

We will search the WHO ICTRP (www.who.int/ictrp) to identify ongoing and completed clinical trials on COVID‐19. We will use the "List By Health Topic: 2019‐nCoV" COVID‐19 filter and retrieve all studies identified. 

We will regularly contact investigators of ongoing studies to update study status and obtain results.

We will also search the European Medicines Agency (EMA) clinical data website (clinicaldata.ema.europa.eu/web/cdp/home) to identify trials submitted to the EMA, and will retrieve the Clinical Study Reports (CSR) of eligible studies. We will also search the website of the United States Food and Drug Administration (FDA) (www.fda.gov) to identify FDA approval documents.

Selection of studies

We will search and screen the citations retrieved on a daily basis. We will use an Excel spreadsheet to document search dates and numbers of hits identified. Two review authors working independently will screen all records retrieved in searches, and will call upon a third review author to resolve disagreements.

Data extraction and management

Two review authors working independently will extract all data and will call upon a third review author to resolve disagreements. Two review authors will independently read each preprint, publication, protocol, or other study reports available, evaluate the completeness of the data availability, and assess the risk of bias. We will design and use a specific structured online data extraction form to ensure consistency of information. All discrepancies automatically identified by the online tool will be discussed by the two review authors and their consensus will be recorded. As soon the consensus is validated, data related to the characteristics of the study and 'Risk of bias' assessment will be made available online.

Information extracted will include study characteristics (such as first author, publication year and journal), number of participants randomized, patient characteristics, intervention details, outcome measures, and 'Risk of bias' assessment. 

For dichotomous outcomes, we will extract the number of events and number of total participants in each study arm. For continuous outcomes, we will extract means, standard deviations (SDs) and number of total participants per study arm. When SDs are not available but standard errors (SE), Tau statistics or P values are reported, we will extract these and transform to SDs when possible. For time‐to‐event outcomes, we will extract hazard ratios (HR) and SEs. When these are not provided, we will attempt to obtain them using the tools provided in Tierney 2007. When several analyses are reported, we will extract results obtained from the intention‐to‐treat (ITT) sample whenever these are available. If ITT results are not available, the modified ITT analysis will be our second choice. When both adjusted and unadjusted analyses are reported, we will extract the latter (to avoid introducing additional methodological heterogeneity across studies) unless the two types of analyses yield substantially different results. If they are different, we will examine each study on a case by case basis.

We will systematically contact authors and ask them to supply 1) information that could not be retrieved from the available study reports and 2) individual‐participant data (IPD). These data will be requested by a personalized email sent by the WHO. These data will be curated and stored. In the presence of IPD, we will re‐analyze the outcomes. Furthermore, if possible, we will conduct IPD NMAs. If acquiring IPD for some of the studies will be deemed feasible, a specific protocol describing the methods to perform IPD meta‐analyses and NMA will be prepared.

Study and participant characteristics, risk of bias data as well as outcome data will be made publicly available on a dedicated website as soon as they are extracted. 

Every week, all the complementary data obtained from authors as well as all the updates or publications of preprint will be recorded by one review author and systematically checked by a second review author. The data available online will be updated accordingly.

Once per week, all the data for new studies, or updates of studies previously identified, will be sent to the statistical analysis team, who will perform the relevant analyses and update the forest plots available online.

Assessment of risk of bias in included studies

We will assess each study with the Cochrane 'Risk of bias 2' (RoB 2) tool for randomized controlled trials (Sterne 2019).

We will assess the risk of bias for critical and important outcomes recorded at all time points. We will record the judgement using the online data extraction tool we developed, but not the answer to each signalling question. Researchers with a master's degree in epidemiology, or members of Cochrane's rapid response team, will assess the risk of bias. The members of the team have all previously been trained in clinical epidemiology and systematic reviews. They have also all participated in a training phase where they read the training material and performed data extraction and RoB 2 assessment with a team of experienced researchers. Members of the Cochrane Bias Methods group will regularly check the quality of a random sample of the data extracted.

The Cochrane RoB 2 tool is structured into five domains: 1) risk of bias arising from the randomization process; 2) risk of bias due to deviations from intended interventions; 3) risk of bias due to missing outcome data; 4) risk of bias in measurement of the outcome; and 5) risk of bias in selection of the reported result. Within each domain, a series of signalling questions elicit information relevant to 'Risk of bias' assessment. The response options to the signalling questions are 'yes,' 'probably yes,' 'probably no,' 'no,' and 'no information.' A 'Risk of bias' judgement arising from each domain is generated by an algorithm, based on answers to the signalling questions. Judgements can be 'low risk of bias,' ‘some concerns’ or 'high risk of bias.' We will consider the overall risk of bias to be 'low risk of bias' if all domains are at ‘low risk’; 'some concerns' if at least one domain is of ‘some concern’ and no domain is at ‘high risk of bias’; and 'high risk of bias' if there is at least one domain considered to be at ‘high risk,' or several domains with ‘some concerns.' In the context of this protocol, we are interested in quantifying the effect of assignment to the interventions at baseline, regardless of whether the interventions are received as intended (i.e. under the ITT principle).

We will define in each sub‐protocol the co‐interventions that could differ between intervention groups and potentially affect study outcomes.

In assessing the certainty of the evidence with GRADE, we will take our 'Risk of bias' assessment into account.

Measures of treatment effect

For dichotomous outcomes, we will use as measure of effect the risk ratio (RR) accompanied by the 95% CI. For continuous and time‐to‐event outcomes we will use the standardized mean difference (SMD), or mean difference (MD), if all studies use the same scale; and the HR, respectively.

Unit of analysis issues

In the pairwise meta‐analysis, different comparisons from multi‐arm trials will be analyzed separately. In the network meta‐analyses, we will properly account for the inherent correlation in multi‐arm trials.

We do not expect to identify any cross‐over trials. Cluster‐randomized trials might appear. If we identify eligible cluster‐randomized trials, we will extract results that properly account for the cluster design (such as based on a multilevel model or on generalized estimating equations). If such an analysis is not reported, we will try to obtain an estimate of the intraclass correlation coefficient.

Dealing with missing data

For missing outcome data, we will extract the number of participants who dropped out before the completion of the study and describe how missing outcome data were handled by the study authors. We will assess the appropriateness of any imputation methods used to account for early dropouts in our 'Risk of bias' assessments. To assess the potential impact of missing outcome data on the results, we will conduct sensitivity analyses, making different assumptions.

Assessment of heterogeneity

At each update, we will first generate descriptive statistics for study and population characteristics to show the available comparisons, the amount of information and the distribution of important clinical and methodological variables (such as age, disease severity, comorbidities, location etc.). We will present the data by pairwise comparison and network diagrams with nodes representing the interventions being compared and lines representing the available direct comparisons in the studies. We will additionally use colours to represent the risk of bias of the studies in each direct comparison (Chaimani 2013). Using a contribution matrix (Papakonstantinou 2018), we will show the effect of each piece of evidence in the full body of evidence and how new evidence affects the existing results.

We will use visual inspection of forest plots, prediction intervals (the interval within which the effect of a future study is expected to lie) (Riley 2011) and comparison of Tau² with appropriate empirical distributions (Rhodes 2015, Turner 2012) to assess the presence of important statistical heterogeneity.

Transitivity is the fundamental assumption of NMA and needs careful examination to reassure that results will be valid (Salanti 2012). We will investigate the distribution of clinical and methodological characteristics that may act as effect modifiers across treatment comparisons. Such characteristics include age, severity status, comorbidity status, and country where care is delivered. To avoid intransitive networks, we will evaluate the similarity of studies comparing different sets of interventions and only synthesize them when important clinical and methodological characteristics are sufficiently similar. Similarity will be judged qualitatively using boxplots (for continuous characteristics) and histograms (for categorical characteristics). We will also investigate whether different studies similarly define the interventions forming the nodes of the networks.

Assessment of reporting biases

We will assess the selective non‐reporting or under‐reporting of results in the studies identified according to the framework proposed in Chapter 13 of the Cochrane Handbook for Systematic Reviews of Interventions (Higgins 2019).

We will use the comparison‐adjusted funnel plot (Chaimani 2013) (a modified funnel plot appropriate for NMA) and appropriate network meta‐regression models (Chaimani 2012) to assess the potential for small‐study effects in each NMA. If asymmetry is found, we will explore possible reasons for the apparent association between study size and study effect. If publication bias is suspected, we will apply selection models that make assumptions about the probability of publication based on the study results (Mavridis 2014).

Data synthesis

The main analysis will include RCTs only. All eligible RCTs will be included in the primary analysis, whatever the RoB 2 assessment

For each direct comparison with at least two studies providing data, we will present effect estimates with 95% CIs. We will use the random‐effects model to incorporate the anticipated clinical and methodological heterogeneity across studies. We will use two assumptions for the between‐study variance (Tau²): 1) a separate Tau² for every comparison between two interventions; and 2) a common Tau² for studies comparing the same types of interventions.

For the sets of studies for which transitivity is plausible, we will perform random‐effects NMAs to compare the different interventions or combination regimens and potentially obtain their ranking. Each outcome will constitute a different network of interventions. We will assume a common heterogeneity parameter (Tau²) for every network of interventions. We will present the results in terms of effect sizes and 95% CIs in league tables and will use colours to represent the confidence in the evidence for every comparison. We will assess the impact of heterogeneity on the results by using prediction intervals. To rank the interventions, in the absence of excessive uncertainty in the relative effects, we will use the surface under the cumulative ranking curve (SUCRA) (Salanti 2011). We will run analyses and produce graphical displays using R (netmeta package (Rucker 2013) and Stata network (White 2015) and network graphs packages (Chaimani 2015). Network meta‐regressions will be run in a Bayesian environment using r2jags with self‐written models (R2jags).

We will supplement the conceptual evaluation of transitivity with a statistical evaluation of the assumption coherence, which refers to the agreement between direct and indirect evidence. We will use both local and global methods. Local approaches assess coherence in parts of the network but global approaches in the entire network jointly. Specifically, we will use the loop‐specific approach (Bucher 1997), the side‐splitting method (Dias 2010) and the design‐by‐treatment interaction model (Higgins 2012). Tests for incoherence are known to have low power, so we will interpret the results of the tests with caution.

Subgroup analysis and investigation of heterogeneity

We will detail pre‐specified subgroup analyses in each sub‐protocol. If we find substantial heterogeneity or incoherence, we will use subgroup analyses and meta‐regressions to explore the impact of specific characteristics on the results. The characteristics explored will evolve as we consider new knowledge on COVID‐19.

Sensitivity analysis

We will perform sensitivity analyses by excluding studies at high risk of bias as well as study reported as preprint. We will also run the analyses using the number of participants analysed, instead of those randomized, and by incorporating uncertainty in our missing outcome data assumptions (Chaimani 2018; Mavridis 2015; White 2008).

Summary of findings and assessment of the certainty of the evidence

To evaluate the confidence in the results of the pairwise comparisons for the critical outcomes, we will rely on the GRADE approach (Schünemann 2019). We will prepare 'Summary of findings' tables to present estimated relative and absolute risks. Two review authors will independently grade the overall certainty of the evidence for each outcome using the GRADE classifications (GRADEpro GDT). We will include the critical outcomes in the 'Summary of findings' tables.

To evaluate the confidence in the NMA for the critical outcomes, we will use the CINeMA tool that considers the following domains: within‐study bias, across‐studies bias, indirectness, imprecision, heterogeneity and incoherence (CiNeMA 2017, Nikolakopoulou 2019). For within‐study bias and indirectness, CINeMA calculates the contribution of each study in each network estimate and combines these contributions with the study‐specific evaluations (low, moderate, high) to rate the relative effect for each comparison in the network. The domains of imprecision, heterogeneity and incoherence use a pre‐specified clinically important size of effect to specify the margin of clinical equivalence between two interventions.

Given that knowledge in the field is still limited, pre‐defining a clinically important size of effect is not straightforward. Therefore, we will follow a conservative approach and make our judgements based on the statistical difference between two interventions. However, as more evidence becomes available we will attempt to also use clinically important effect sizes to assess the robustness of our judgements.