IJPDS International Journal of Population Data Science IJPDS 2399-4908 Swansea University 10.23889/ijpds.v5i4.1710 5:4:1710 S2399490821017109 Population Data Science Estimating surge in COVID-19 cases, hospital resources and PPE demand with the interactive and locally-informed COVID-19 Health System Capacity Planning Tool KrylovaOlga1* KazmiOmar1 WangHui1 LamKelvin1 Logar-HendersonChloe1 GapanenkoKaterina1 Advanced Analytics, Canadian Institute for Health Information, Ottawa, Ont Corresponding author: Olga Krylova okrylova@cihi.ca

None to be declared.

 06042022 2020 5 4 1710 This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. This article is available from the IJPDS website at: https://ijpds.org/article/view/1710 Abstract Introduction

The COVID-19 pandemic revealed an urgent need for analytic tools to help health system leaders plan for surges in hospital capacity. Our objective was to develop a practical and locally informed Tool to help explore the effects of public health interventions on SARS-CoV-2 transmission and create scenarios to project potential surges in hospital admissions and resource demand.

 Methods

Our Excel-based Tool uses a modified S(usceptible)-E(xposed)-I(nfected)-R(emoved) model with vaccination to simulate the potential spread of COVID-19 cases in the community and subsequent demand for hospitalizations, intensive care unit beds, ventilators, health care workers, and personal protective equipment. With over 40+ customizable parameters, planners can adapt the Tool to their jurisdiction and changes in the pandemic.

 Results

We showcase the Tool using data for Ontario, Canada. Using healthcare utilization data to fit hospitalizations and ICU cases, we illustrate how public health interventions influenced the COVID-19 reproduction number and case counts. We also demonstrate the Tool’s ability to project a potential epidemic trajectory and subsequent demand for hospital resources. Using local data, we built three planning scenarios for Ontario for a 3-month period. Our worst-case scenario accurately projected the surge in critical care demand that overwhelmed hospital capacity in Ontario during Spring 2021.

 Conclusions

Our Tool can help different levels of health authorities plan their response to the pandemic. The main differentiators between this Tool and other existing tools include its ease of use, ability to build scenarios, and that it provides immediate outcomes that are ready to share with executive decision makers. The Tool is used by provincial health ministries, public health departments, and hospitals to make operational decisions and communicate possible scenarios to the public. The Tool provides educational value for the healthcare community and can be adapted for existing and emerging diseases.

 epidemiology health policy infectious diseases public health statistics and research methods COVID-19 interactive tool health system capacity predictive modeling hospital bed demand PPE demand This project was supported and funded by the Canadian Institute for Health Information. Introduction

The Coronavirus Disease 2019 (COVID-19) pandemic has put unprecedented pressure on healthcare systems in Canada [1] and around the world [2]. Health leaders have sought models to provide short- and long-term forecasts of COVID-19 cases [3–5], demand for hospital resources [5–7], and the potential impact of public health interventions [5, 8, 9]. Many analytical models have been developed [5, 6, 10–12] to help decision makers plan for the pandemic. However, they are predominantly based on sophisticated software packages and require advanced knowledge and technical skills to use. To meet the planning needs of hospital managers, public health units and provincial health ministries in March 2020, we started developing the COVID-19 Health System Capacity Planning Tool (referred to as the Tool) with flexible and user-friendly features. By simulating the impact of various public health interventions, the Tool models SARS-CoV-2 transmission in the population and provides scenario-based projections for acute and critical care beds, ventilators, health care worker (HCW) staffing, and personal protective equipment (PPE) needed to care for patients with COVID-19. Designed for users with different levels of knowledge, the Tool uses the same underlying mathematical models as most other tools, but is Excel-based with parameter inputs, data outputs, and visualizations that are simple to understand, modify, and adapt to local contexts.

The impact of COVID-19 has been felt across Canada, but provinces and territories have experienced differences in disease burden, timing, and number of waves, and in the implementation of policies to prevent the spread of the virus [13, 14]. Our Tool contains over 40 customizable parameters, enabling decision-makers at all levels to create local scenarios and plan for surges in health resource demands and supply shortfalls associated with the COVID-19 pandemic in their region. The Tool has been used successfully by provincial health ministries, public health departments and hospitals across Canada to make operational decisions and inform the public about potential scenarios. The Tool continues to evolve to reflect changes in our understanding of SARS-CoV-2 transmission as new variants of concern (VOCs) emerge and vaccination rollouts offer hope in defeating this virus.

In the following sections, we present our underlying methodology, data sources and parameters, and results by applying the Tool to data from Ontario, Canada. Ontario is Canada’s largest province, has the country’s largest number of confirmed COVID-19 cases [13], and has easily accessible COVID-19 data.

 Methods Tool development and framework

During the early stage of the pandemic, our team engaged and exchanged information with the Public Health Agency of Canada, ministries of health, modelling experts in the academic community, hospitals, regional health authorities, and public health units to develop and shape this Tool. For modelling transparency, simplicity and ease of use, we implemented the Tool in Microsoft Excel. It is available to potential users upon request by contacting the Canadian Institute for Health Information (help@cihi.ca).

The COVID-19 Health System Capacity Planning Tool covers the major aspects of COVID-19 health capacity planning in one model, from forecasting epidemic trajectories to hospital bed supply and demand to PPE and staffing needs. Our Tool has three interconnected modules: the Epidemiological (Epi) module, Capacity module and HCW/PPE module (Figure 1, detailed description provided in Appendix A1). The Epi module simulates the potential spread of COVID-19 cases, considering local public health measures, transmissibility, and vaccination rates. The Capacity and HCW/PPE modules estimate hospital resources and the number of healthcare workers and PPE required to treat COVID-19 patients, respectively.

Abbreviations: HCW; health care worker, PPE; personal protective equipment, RT; respiratory therapist. The Tool includes three modules: the Epidemiological (Epi) module (teal), the Capacity module (orange) and the HCW/PPE module (purple). Each module consists of multiple compartments (boxes) that represent COVID-19 disease states in the Epi and Capacity modules, and supply categories in HCW/PPE module. Arrows with solid lines show the flow of individuals in the population between compartments in the Epi and Capacity modules. The dotted line for the Reported compartment indicates that not every case is reported and that there is a delay between the time of exposure and the case being reported. The dashed arrows between the Capacity and HCW/PPE modules represent the flow of information rather than individuals. Model Epi module

The Epi module simulates the potential spread of COVID-19 cases in a previously unexposed population. This module requires users to input locally derived data such as cumulative COVID-19 cases, population size, dates of public health measures, and vaccination rates.

The Epi module was built using a modified version of the Susceptible-Exposed-Infected-Removed-type deterministic compartmental model with vaccination [15–18]. To implement this model in Excel we used a discrete-time version of the model [19]. The Epi module in Figure 1 (teal box) illustrates how individuals in the population move across compartments according to the stages of COVID-19 progression. For example, someone could be susceptible to infection, an asymptomatic infectious case, recovered, etc. In addition to S-E-I-R classes, we included a Reported compartment to capture the reporting delay [20] between an individual being exposed to the virus and reported as a case. By customizing the reporting delay and proportion of COVID-19 cases reported, planners can reflect local testing guidelines. We also created a Vaccination compartment to examine the impact of vaccination rollouts [21]. The vaccination component is implemented as a step function where the daily vaccination rate, vaccine efficacy, and the delay for the vaccine to be effective can be modified for different time periods. Table 1 contains a list of key epidemiological parameters and values (full list is provided in Appendix Table A2.2, model equations can be found in Appendix Table A2.3).

Parameter Definition Value Reference
Latent period, 1/σ The period between the point of infection and the onset of infectiousness (based on 5.1 days incubation period, i.e., from the point of infection to symptoms onset and a 1-day pre-symptomatic infectious period) 4.1 days Lauer et al. (2020) [22], He et al. (2020) [23]
Infectious period for asymptomatic cases, 1/ƔA The period between onset of infectiousness and recovery for someone who is asymptomatic 6 days Hu et al. (2020) [24]
Infectious period for mildly symptomatic cases, 1/ƔSM The period between the onset of infectiousness and self-isolation for someone who develops mild symptoms 4 days Li et al. (2020) [25]
Infectious period for severely symptomatic cases, 1/ƔSS The period of infectivity for someone who develops severe symptoms 4 days Assumed to be the same as the infectious period for the mild cases
Percentage of asymptomatic cases, PA Percentage of persons who are infected with SARS-CoV-2 but never show symptoms of the disease. 33% Oran et al. (2021) [26]

Note: Parameters can be adjusted by the user to reflect local information, if available, as well as the latest scientific evidence.

Disease transmission varies between regions and over the course of the pandemic because of factors such as the initial rate of disease spread, local contact rate, demographics, population density, and public health interventions, among others [27]. The changing COVID-19 transmission is captured in our model with a dynamic effective reproduction number, Reff(t), using a time-varying piecewise function. Reff(t) is estimated in the Tool by fitting the modeled reported cases to the local observed cumulative cases (Figure 2). By creating the timeline of interventions and finding the appropriate model fit, the planner can study the effects of various public health strategies on the changes in Reff(t) and community transmission as shown in the example in the Results section.

The top panel shows the reported cumulative cases (teal circles) versus the modelled cases (solid line) for Ontario, Canada from 1st February 2020 to 31st January 2021. Model fit: MAPE = 0.8%. The estimated time-dependent reproduction number, Reff(t), is plotted as a dashed line. The bottom panel shows a timeline of public health measures that coincide with changes in Reff(t). Capacity module

The Capacity module estimates the hospital resources required to treat COVID-19 patients. The model is an extension of the Epi module and is split into three compartments representing the hospital environment: Ward accounts for non-critical care inpatient beds; Intensive Care Unit (ICU) represents critical care beds without ventilation; and Ventilator signifies ICU beds with invasive mechanical ventilation (Figure 1). Hospital resource capacity is also included as an input to the model to estimate the potential date when resources may be depleted and to quantify the gap between bed capacity and demand.

This module requires users to input locally derived clinical administrative data such as statistics on health care usage and deaths (Table 2), which have changed dramatically across Canadian jurisdictions over the course of the pandemic [29]. The proportion of cases hospitalized and their LOS have changed throughout the pandemic due to the circulation of new variants. Our model allows planners to input multiple sets of health system usage parameter values, making this component highly customizable to each jurisdiction where the timing of changes in health care utilization might differ.

Parameter 1st February 2020–31st March 2020 1st April 2020–31st May 2020 1st June 2020–31st July 2020 1st August 2020–30th September 2020 1st October 2020–30th November 2020 1st December 2020–31st January 2021
Percentage of hospitalizations (calibrated), %┴ 19.0 11.6 2.0 2.7 3.4 4.1
Percentage of hospitalizations (baseline), %┴┴ 11.3 12.5 10.5 3.8 3.5 4.0
Percentage of ICU admissions (among hospitalized cases), %° 33.9 22.9 21.8 24.3 32.5┴ 30.5┴
Percentage of ICU admissions with mechanical ventilation, %° 75.7 64.7 51.1 45.1 33.6┴ 42.5┴
Average LOS in ward (non-fatal cases), days° 22.86 23.07 14.27 12.66 12.66* 12.66*
Average LOS in ward (fatal cases), days° 32.70 13.74 24.48 21.98 21.98* 21.98*
Average LOS in ICU w/o ventilation (non-fatal cases), days° 5.30 5.00 6.20 5.50 5.50* 5.50*
Average LOS in ICU w/o ventilation (fatal cases), days° 4.80 3.40 5.50 5.80 5.80* 5.80*
Average LOS in ICU with ventilation (non-fatal cases), days° 25.10 26.40 30.20 18.60 18.60* 18.60*
Average LOS in ICU with ventilation (fatal cases), days° 14.00 16.90 17.70 22.10 22.10* 22.10*
Percentage of deaths among non-critical cases (ward only), %° 15.2 23.4 10.6 12.7 12.7* 12.7*
Percentage of deaths among cases admitted to ICU w/o ventilation, %° 19.5 23.9 16.5 11.5 11.5* 11.5%*
Percentage of deaths among cases admitted to ICU with ventilation, %° 35.5 46.5 44.2 51.7 51.7* 51.7*

Abbreviations: ICU, intensive care unit; LOS, length of stay; w/o, without.

┴These values were calibrated from the Ontario baseline values based on the model best fit (MAPE for cumulative reported cases = 0.8%, MAPE for hospitalizations = 8.5%, MAPE for ICU admissions = 12.8%, MAPE for ICU admissions with ventilation = 18.4%, see also Figure).

┴┴The baseline values for the percentage of cases hospitalized were estimated as the percentage of “total confirmed cases” “ever hospitalized” for the reported period from the daily epidemiologic summaries from Public Health Ontario [28]. Detailed calculations are presented in Appendix Table A2.4. Note that the percentage of cases hospitalized cannot be accurately determined from the existing data as there is no linkage between the testing data and the hospitalization data.

°Based on the detailed information on acute care hospitalizations for patients with a diagnosis of COVID-19 in CIHI’s COVID-19 Hospitalization and Emergency Department Statistics, 2019–2020 and 2020–2021 [29]. Detailed COVID-19 episode of care breakdowns for modelling, by recipient province/territory and admission month, DAD, January to November 2020” Table.

*The values for LOS and fatality rates for the 1st October 2020–30th November 2020 and 1st December 2020–31st January 2021 time periods were assumed to be the same as for the 1st August 2020–30th September 2020 period. While the parameters from the DAD data were available up to 30th November 2020, the last period couldn’t be included in the analysis, as a significant proportion of cases hospitalized from 1st October 2020–30th November 2020 are discharged after 30th November 2020 and will not be included in the reported data.

 HCW/PPE module

The HCW/PPE module estimates the number of health care workers (i.e., physicians, nurses and other clinical and hospital support staff) and PPE required to care for COVID-19 patients in inpatient acute-care settings (Figure 1). The formulas to calculate healthcare worker demand are based on the average number of active daily COVID-19 patients in each acute-care setting and information on staff-patient interactions including contact frequency, the number of contacts per patient per shift, shift length, and the staff-to-patient ratio in that setting.

PPE usage is calculated for each type of healthcare worker based on the number of each type of equipment required and whether this is per shift (as in the case of eye protection or gowns) or per patient contact (as in the case of disposable gloves) [30]. The default values for PPE usage were obtained from the World Health Organization [31] and from interviews with clinicians in Ontario and Newfoundland (see Appendix Tables A2.5 and A2.6 for default values). Users can alter these values to reflect the reality of their jurisdiction or organization.

 Model validation

We validated the Tool results against two benchmarks. First, an internal validation was done to ensure that outputs produced by the model aligned with historical case, hospitalization, ICU admissions, and ICU admissions with ventilation data (Figure 2 and Figure 3, more details can be found in Appendix Tables A3.3.1 and A3.3.2). Second, we compared outputs from the model with case and hospitalization estimates generated by other major models (the McMaster Pandemic model [31], the online Epidemic Calculator [32], University of Pennsylvania’s CHIME PPE Calculator Excel application [33], and COVID-19 Modeling Collaborative’s PPE Resource estimator [34]) and obtained very similar outputs, with no statistical differences across the models (see Appendix A3 for details).

The three scenarios were produced for February to April 2021 (solid, dashed and dash-dotted black lines). The orange circles show the observed data from February to April 2021.

For quality assurance of the Excel implementation, formulas and results were verified with a Python version of the model.

In the Tool, model fit is evaluated by the root mean square error (RMSE) [35]. The dynamic effective reproduction number (described below) values that minimize RMSE are obtained by utilizing Excel’s Solver add-in [36] with nonlinear optimization using the Generalized Reduced Gradient method [37]. In this paper, we report the Median Absolute Percentage Error (MAPE), which, despite some limitations, is scale-independent and a more easily interpretable measure of the prediction accuracy of a projection [38, 39].

 Results

For illustrative examples presented in this section, we use publicly available data for Ontario, Canada on daily cumulative cases [40], daily [40], and cumulative hospitalizations [28], ICU cases, ICU cases with mechanical ventilations [40], and population estimates [41]. For the dates of public health measures, we utilized the COVID-19 Intervention Scan Tool [42] and public announcements. Healthcare usage data were obtained from recent COVID-19 hospitalization statistics published by the Canadian Institute for Health Information [29].

In our first example, we used Ontario data from 1st February 2020 to 31st January 2021 to analyse the effects of various intervention policies on the reproduction number. In the second example, we illustrate how this Tool can be used to build scenarios to project the potential demand for hospital, HCW, and PPE resources using historically available data.

Observed impact of public health interventions

We created a timeline of major public health measures implemented in Ontario from 1st February 2020 to 31st January 2021 (Figure 2(B) and Appendix Table A2.7). We then estimated changes in Reff(t) for this period by fitting our model to the reported cumulative cases. We also validated these estimates by comparing them with the Reff(t) reported by Ontario (See Appendix Figure A3.4).

The number of reported cases in Ontario grew from 2 cases on 1st February 2020 to 268,211 cases on 31st January 2021. During the same period, Reff(t) fluctuated, as shown in the top panel of Figure 2. At the beginning of the pandemic when no preventative measures were in place, Reff(t) was estimated to be 2.8. It started to decline to ~0.8 after various escalation measures were implemented including a declaration of a state of emergency, school and business closures, social distancing rules. The re-opening in May 2020 and the lifting of restrictions seemed to be associated with a slow growth of the reproduction number (and therefore transmission), which reached a value of 1.4 in September 2020. The recommendation of wearing face masks announced on 20th May, 2020 [43] seemed to have been associated with a reduction in the Reff. New restrictions were implemented in some regions at the beginning of October 2020 to slow transmission, resulting in Reff that fluctuated around 1 in October-November 2020. In December 2020, the Reff increased again to about 1.3, which coincided with increased social interaction before and during the Christmas holidays. Following a second province-wide lockdown and stay-at-home orders, transmission decreased, and Reff was estimated to be 0.7 at the end of January 2021.

 Projected COVID-19 patterns and need for hospital resources

In our second example, we created three scenarios to project demand for healthcare resources for a three-month period, from 1st February 2021 to 30th April 2021 (Table 3, Figure 3) based on historical data presented in the previous example and reopening plans announced in February by the Ontario government [44]. The Tool can be applied in a similar manner to build projections for upcoming months with more recent historical data and current public health measures.

Scenario 1 Scenario 2 Scenario 3
Potential changes in Reff(t)
Public health announcements Date (2021) Reff(t) Reff(t) Reff(t)
3 regions to reopen [44] 10th February 0.9 1.0 1.1
Additional 27 regions to reopen [53] 16th February 0.95 1.1 1.2
Additional 3 regions to reopen [54] 8th March 1.0 1.1 1.3
Further lifting of restrictions [55] 21st March 1.05 1.2 1.4
Potential further lifting of restrictions 1st April 1.1 1.2 1.5
Potential changes in hospital usage parameters
Percentage of cases requiring hospitalization, % 1st February–30th April 4.1 5 5
Percentage admitted to ICU among those hospitalized, % 1st February 1–30th April 30.5 32.5 34.5
Percentage of ICU admissions with mechanical ventilation, % 1st February–30th April 42.5 44.5 46.5
Potential vaccine rollouts [56]
Daily vaccination rate* January 2,200 2,200 2,200
Daily vaccination rate* February 6,900 6,900 6,900
Daily vaccination rate** March 45,000 45,000 45,000
Daily vaccination rate** April 90,000 90,000 90,000
Capacity available for patients with COVID-19
Hospital beds 1st February–30th April 4,514 4,514 4,514
ICU beds 1st February–30th April 514 514 514
Additional 500 “surge beds” February 1,014 1,014 1,014
ICU beds with ventilator 1st February–30th April 328 328 328

*Number of daily fully vaccinated individuals (i.e., 2 doses with 95% efficacy after 12 days)

**Number of daily vaccinated individuals with a single dose (i.e., 1 dose with 70% efficacy after 12 days)

When building our scenarios, we focused on adjusting five parameters: Reff(t), which reflects changes in contact rate and viral transmissibility; hospitalization, ICU and ventilation rates, which vary depending on the virulence of circulating VOCs; and the daily vaccination rate, which changes over time and impacts the proportion of the population susceptible to infection and therefore the epidemic curve.

We proposed three scenarios. First, we projected the expected outcomes in a situation where the announced measures of reopening lead to a slight increase in population mobility and person-to-person contact. As a result, in this most optimistic scenario, Scenario 1, Reff(t) was assumed to increase very slightly. In contrast, the two more plausible scenarios represent situations where Reff(t) is assumed to increase by a magnitude like that during the reopening phase of the second wave of the pandemic in Ontario (Scenario 2) or even faster (Scenario 3) due to the higher transmissibility of emerging VOCs [45, 46]. When limited information about emerging VOCs exists, we can only make assumptions on how contagious or virulent they are in comparison with the dominant variant. Since there was early evidence of the Alpha variant (B1.1.7 VOC) being more transmissible and causing more severe illness than earlier variants, we assumed higher transmissibility, hospitalization, ICU and ventilation rates [47, 48] in Scenarios 2 and 3.

For all scenarios, the vaccination rates for January to February 2021 represent the number of fully vaccinated individuals in Ontario [49] with 95% efficacy [21]. For the period of March to April 2021, we assumed that only a single dose vaccine was administered with 70% efficacy [49]. This is reflective of the changes in Canada’s national vaccination strategy in Spring 2021, which aimed to maximize single dose immunizations and extend the interval for the second dose by up to four months [49].

Our worst case, Scenario 3, projected almost nine times the number of daily cases projected by Scenario 1 (5,997 vs. 690 on 1st May 2021). In Scenario 3, the projected ICU bed and ventilator demand surpassed the numbers that were available for COVID patients (Tables 3 and 4). However, the additional 500 ICU beds announced by the Ontario Government on 19th January 2021 [50] could provide adequate resources to care for this potential increase in hospitalized cases. As highlighted in Tables 5 and 6, estimated demand for PPE and health care staffing needs were also significantly higher for Scenario 3.

Projected values for 1st May 2021 Scenario 1 Scenario 2 Scenario 3
Daily reported cases 690 1,764 5,997
Hospitalized cases 379 1,046 2,963
In ICU 105 307 930
In ICU with ventilator 78 223 650

Abbreviations: ICU, intensive care unit.
Week end date COVID-19 patients hospitalized Gloves (pair) Eye protection Surgical masks N95 masks Gowns
Scenario 1 3 1 3 1 3 1 3 1 3 1 3
3/07/2021 588 815 44,660 63,441 8,777 12,576 5,699 7,544 6,840 10,578 8,777 12,576
3/14/2021 525 853 39,868 66,541 7,836 13,188 5,074 7,875 6,122 11,135 7,836 13,188
3/21/2021 476 928 36,198 72,554 7,111 14,365 4,605 8,573 5,554 12,131 7,111 14,365
3/28/2021 440 1,050 33,467 82,083 6,570 16,234 4,263 9,703 5,120 13,686 6,570 16,234
4/04/2021 414 1,226 31,526 95,937 6,182 18,951 4,023 11,352 4,803 15,942 6,182 18,951
4/11/2021 398 1,474 30,291 115,411 5,934 22,774 3,875 13,671 4,591 19,115 5,934 22,774
4/18/2021 391 1,821 29,728 142,663 5,816 28,126 3,813 16,918 4,481 23,562 5,816 28,126
4/25/2021 390 2,270 29,609 177,749 5,787 35,030 3,809 21,115 4,438 29,287 5,787 35,030
5/02/2021 391 2,802 29,725 219,163 5,805 43,192 3,835 26,088 4,433 36,047 5,805 43,192
Month COVID-19 patients Nurses (Ward) Nurses (ICU) Physicians (Ward) Physicians (ICU) Respiratory therapists Cleaners Porters
Scenario 1 3 1 3 1 3 1 3 1 3 1 3 1 3 1 3
2021–02 891 961 216 226 490 566 43 45 49 49 1 1 244 279 3 4
2021–03 499 936 119 213 285 597 24 43 28 28 1 2 141 282 2 6
2021–04 395 1,922 95 438 222 1,213 19 88 22 22 0 4 105 547 2 14

We then compared our projected scenarios with the cases and hospitalizations observed from February to April in Ontario (Figure 3, orange dots). As shown in our worst-case projections (Scenario 3), without enhanced public health measures and with several Ontario hospitals already at or over capacity in February [50], the rise in COVID hospitalizations reached the ICU bed capacity in April 2021 [51, 52]. As illustrated by this example, the scenario comparison can be a useful tool to help policy makers estimate risks and take actions to reduce new infections and ultimately, hospitalizations and fatalities.

 Discussion

We developed a tool that can assist various levels of health care planners in making short- and long-term decisions about healthcare capacity, PPE, and healthcare workforce needs. Used alongside other models and information (e.g., financial, ethical, etc.) it could aid planners when weighing decisions related to COVID-19. Different jurisdictions and regions experience differences in intensity and timing of outbreaks [57], presence of VOCs [58] and severity of cases [29], implementation of public health measures [42] and vaccination strategies [59]. These local specifics can be simulated by our Tool with its highly customizable parameters that rely on local data inputs.

Our examples from Ontario, Canada illustrate how a planner could apply the Tool to a particular context given the current state of events, local case, hospital, healthcare workforce, supply chain data, and evolving information about emerging VOCs. The coronavirus changes very rapidly due to the high number of mutations [60]. As we witnessed with the current VOC, Omicron, viral transmissibility and disease severity can drastically change from one variant to the next [61]. These factors can be reflected by adjusting parameters in the Tool, making it highly adaptable to new VOCs.

Many local [5, 62–67] and national [7, 9, 68, 69] Canadian models and tools [11, 12, 70, 71] were developed to assist jurisdictional authorities and hospital planners with pandemic preparedness. Often these models require a high level of understanding of the underlying mathematical modelling techniques, proficiency in a specific software program and a team of modellers to build projections, maintain and update the model [6, 10–12]. In contrast, existing online tools [70, 71] are relatively simple to use but cannot easily reflect the rapidly changing situation due to the use of static parameters and cannot fit the model to the observed data. Our Tool was designed with our end-users in mind; the Excel implementation, user-friendly outputs, and visualizations make the Tool easy to apply by planners with different levels of knowledge. It also includes several time-varying parameters such as Reff(t), which enables the simulation of multiple waves, and hospital usage parameters, which changed significantly over time. We also periodically update the Tool and parameters with the most recent information and user feedback.

The impact of the Tool has been significant. Federal and provincial public health agencies and health ministries, public health units, academics, and hospitals across the country have used it for a variety of purposes including updates to executive teams and government officials, validation of projections produced by other models, and within local dashboards for operational planning. The Government of Newfoundland and Labrador [72] and Toronto Public Health [73] have used the Tool to generate scenarios used in public releases and highlight the importance of public health measures to curb viral transmission. Our Tool continues to evolve and inform health care planning. We continue to engage stakeholders in methodological and policy discussions to understand their health resource planning and pandemic preparedness needs and help them apply the Tool to create realistic local scenarios.

Limitations

Our model, as with any deterministic SEIR-type model, has several limitations [74]. The main assumption of the model, a homogeneous well-mixed population, ignores the fact that most contacts with COVID-19 cases occur not at random but within groups of people in social or geographical proximity [75]. The model works well when simulating outbreaks in large, well-mixed populations rather than in small populations (e.g., small town or long-term care facility), where stochastic effects are much more profound and agent-based type models are more suitable [76]. Our model does not account for differences in susceptibility to COVID-19 with respect to age, comorbidities, and sociodemographic factors [77]. One of the implications is that vaccinating priority populations [78] could not be addressed. We also assumed that vaccinated and recovered individuals develop immunity and are not susceptible to re-infection, which may not be the case [51, 79]. We want to emphasize that this scenario-based Tool does not forecast a future but projects a range of outcomes based on the observed data and model assumptions. These projections are best considered as helpful guides, not definitive outcomes.

 Conclusions

Our interactive Tool can help local governments and hospital managers plan their response to the evolving pandemic. The Tool also creates educational value for the healthcare community and can be an important addition to planners’ arsenal of models. Our Tool continues to evolve to reflect our changing understanding of SARS-CoV-2 transmission and its impact on hospital resources, and answer critical questions posed by health system planners.

The main difference between this Tool and other existing tools is its ease of use, ability to build scenarios, and ability to provide immediate outcomes that are ready to share with executive decision-makers to help them understand the evolution of the disease and make appropriate decisions.

The Tool is readily adaptable to future emerging infectious diseases and can therefore be an important addition to planners’ arsenal of models for the future.

The COVID-19 Health System Capacity Planning Tool is currently available by request from CIHI (help@cihi.ca).

 Supplementary Files Acknowledgements

The authors gratefully acknowledge early advice and input from members of the Public Health Agency of Canada’s External Modelling Experts Group on model assumptions and parameters. The authors thank members of public health units, hospitals, and provincial ministries for helping to test and validate the Tool, and their thoughtful feedback on the Tool’s usability and improvements.

The model conceptualization and technical development of the Epidemiological and Capacity modules were led by Olga Krylova, and the HCW/PPE module development was led by Omar Kazmi. Hui Wang led the Microsoft Excel implementation of the tool. Omar Kazmi performed the Python verification of the model. Kelvin Lam led the estimation of the healthcare usage parameters and client support. Olga Krylova and Chloe Logar-Henderson performed the formal analysis of the Ontario data and drafting of the manuscript. Overall, the project was led by Katerina Gapanenko. All of the authors revised the manuscript for important intellectual content, gave final approval of the version to be published, and agreed to be accountable for all aspects of the work.

 Data sharing

All data used for parameterization of this model is in the public domain and can be accessed through references cited in the manuscript and technical appendix. The COVID-19 Health System Capacity Planning Tool is currently available by request from CIHI (help@cihi.ca).

 Ethics statement

Our research did not require ethical approval as our research relied exclusively on publicly available and accessible data, and there is no reasonable expectation of privacy.

 References Yu A, Prasad S, Akande A, Murariu A, Yuan S, Kathirkamanathan S et al. COVID-19 in Canada: A self-assessment and review of preparedness and response. J Glob Health. 2020 Dec;10(2):203104. 10.7189%2Fjogh.10.0203104 Lal A, Erondu NA, Heymann DL, Gitahi G, Yates R. Fragmented health systems in COVID-19: rectifying the misalignment between global health security and universal health coverage. The Lancet. 2021 Jan;397(10268):61–7. 10.1016/S0140-6736(20)32228-5 Chen L-P, Zhang Q, Yi GY, He W. Model-based forecasting for Canadian COVID-19 data. PLoS One. 2021 Jan;16(1):e0244536. 10.1371/journal.pone.0244536 Nikolopoulos K, Punia S, Schäfers A, Tsinopoulos C, Vasilakis C. Forecasting and planning during a pandemic: COVID-19 growth rates, supply chain disruptions, and governmental decisions. Eur J Oper Res. 2021 Apr;290(1):99–115. 10.1016/j.ejor.2020.08.001 Tuite AR, Fisman DN, Greer AL. Mathematical modelling of COVID-19 transmission and mitigation strategies in the population of Ontario, Canada. CMAJ. 2020 May;192(19):E497–E505. 10.1503/cmaj.200476 Barrett K, Khan Y, Mac S, Ximenes R, Naimark D, Sander B. Estimation of COVID-19-induced depletion of hospital resources in Ontario, Canada. CMAJ. 2020 Jun;192(24):E640–E646. 10.1503/cmaj.200715 Shoukat A, Wells CR, Langley JM, Singer BH, Galvani AP, Moghadas SM. Projecting demand for critical care beds during COVID-19 outbreaks in Canada. CMAJ. 2020 May;192(19):E489–E496. https://www.cmaj.ca/content/192/19/E489 Ludwig A, Berthiaume P, Orpana H, Nadeau C, Diasparra M, Barnes J et al. Assessing the impact of varying levels of case detection and contact tracing on COVID-19 transmission in Canada during lifting of restrictive closures using a dynamic compartmental model. Can Commun Dis Rep. 2020 Nov;46(11/12):409–21. 10.14745/ccdr.v46i1112a08 Ng V, Fazil A, Waddell LA, Bancej C, Turgeon P, Otten A et al. Projected effects of nonpharmaceutical public health interventions to prevent resurgence of SARS-CoV-2 transmission in Canada. CMAJ. 2020 Sept;192(37):E1053–E1064. 10.1503/cmaj.200990 National Collaborating Centre for Infectious Diseases [Internet]. PHAC models on COVID-19. NCCID [cited 2021 Mar 29]. Available from: https://nccid.ca/phac-modelling/. Github – seananderson/covidseir. Bayesian SEIR modelling for multivariate COVID-19 case data [cited 2021 Feb 17]. Available from: https://github.com/seananderson/covidseir Github – mac-theobio/McMasterPandemic. McMaster Pandemic [cited 2021 Feb 17]. Available from: https://github.com/bbolker/McMasterPandemic Public Health Agency of Canada [Internet]. Coronavirus disease 2019 (COVID-19): Epidemiology update. PHAC; 2021 [cited 2021 Feb 22]. Available from: https://health-infobase.canada.ca/covid-19/epidemiological-summary-covid-19-cases.html?stat=num&measure=total&map=pt#a2 Canadian Institute for Health Information [Internet]. COVID-19 Intervention Timeline in Canada. Ottawa, ON: CIHI; 2021 [cited 2021 Mar 1]. Available from: https://www.cihi.ca/en/covid-19-intervention-timeline-in-canada Hethcote HW. The mathematics of infectious diseases [Internet]. SIAM Rev. 2000;42(4):599–653. 10.1137/S0036144500371907 Keeling MJ, Rohani P. Modeling infectious diseases in humans and animals. Princeton University Press; 2008. Anderson RM, May RM. Infectious diseases of humans: dynamics and control. Oxford: Oxford University Press; 1991. Radulescu A, Williams C, Cavanagh K. Management strategies in a SEIR-type model of COVID 19 community spread. Scientific Reports. 2020 Dec;10(1):21256. 10.1038/s41598-020-77628-4 Carcione JM, Santos JE, Bagaini C, Ba J. A simulation of a COVID-19 epidemic based on a deterministic SEIR model. Front Public Health. 2020; 8:230. 10.3389/fpubh.2020.00230 Ping Y. Some preliminary trend analysis of COVID-19 in five selected provinces based on data reported. Presentation at the 2019–2020 COVID-19 Math Modelling Seminar. The Fields Institute for Research in Mathematical Sciences; 2020 May 19; Online. Government of Ontario [Internet]. COVID-19 vaccines for Ontario. Ontario: Queen’s Printer for Ontario; 2021 [cited 2021 August]. Available from: https://covid-19.ontario.ca/covid-19-vaccines-ontario Lauer SA, Grantz KH, Bi Q, Jones FK, Zheng Q, Meredith HR et al. The incubation period of Coronavirus Disease 2019 (COVID-19) from publicly reported confirmed cases: estimation and application. Ann Intern Med [Internet]. 2020 March [cited 2021 Mar 1];172(9):577–82. 10.7326%2FM20-0504 He X, Lau EHY, Wu P, Deng X, Wang J, Hao X et al. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nature Medicine. 2020 Apr;26(5):672–5. 10.1038/s41591-020-1016-z Hu Z, Song C, Xu C, Jin G, Chen Y, Xu X et al. Clinical characteristics of 24 asymptomatic infections with COVID-19 screened among close contacts in Nanjing, China. Sci China Life Sci 2020 May;63(5):706–11. 10.1007/s11427-020-1661-4 Li R, Pei S, Chen B, Song Y, Zhang T, Yang W et al. Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2). Science [Internet]. 2020 Mar [cited 2021 Mar 1];368(6490):489–93. 10.1126/science.abb3221 Oran DP, Topol EJ. The proportion of SARS-CoV-2 infections that are asymptomatic: a systematic review. Ann Intern Med [Internet]. 2021 Jan [cited 2021 Mar 1];174(5)655–62. 10.7326/m20-6976 Delamater PL, Street EJ, Leslie TF, Yang YT, Jacobsen KH. Complexity of the basic reproduction number (R(0)). Emerg Infect Dis. 2019 Jan;25(1):1–4. 10.1371/journal.pone.0248731 Public Health Ontario. COVID-19: Epidemiologic summaries from Public Health Ontario [Internet]. PHO [cited 2021 Mar 4]. Available from: https://covid-19.ontario.ca/covid-19-epidemiologic-summaries-public-health-ontario Canadian Institute for Health Information. COVID-19 hospitalization and emergency department statistics, 2019–2020 and 2020–2021. Ottawa, ON: CIHI; 2021 Mar [cited 2021 Mar 2]. Available from: https://www.cihi.ca/en/covid-19-hospitalization-and-emergency-department-statistics World Health Organization. WHO COVID-19 Essential supplies forecasting tool (COVID-ESFT), version 2. WHO; 2020 [cited 2021 Mar 2]. Available from: https://apps.who.int/iris/handle/10665/333984 World Health Organization. Rational use of personal protective equipment for COVID-19 and considerations during severe shortages: Interim guidance. Geneva: WHO; 2020 Apr 6 [cited 2021 Mar 2]. Available from: https://apps.who.int/iris/bitstream/handle/10665/331695/WHO-2019-nCov-IPC_PPE_use-2020.3-eng.pdf GitHub. Epidemic Calculator [cited 2021 Feb 17]. Available from: http://gabgoh.github.io/COVID/index.html University of Pennsylvania. PPE Forecasting Calculator [cited 2021 Mar 10]. Available from: https://penn-chime.phl.io Barrett K, Nakamachi Y, Ierasts T, Khan YA, St. Mac, Naimark D et al. A model to estimate demand for personal protective equipment for Ontario acute care hospitals during the COVID-19 pandemic. medRxiv 2020. 10.1101/2020.04.29.20085142 Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014 Jun;7(3):1247–50. 10.5194/gmd-7-1247-2014 Microsoft [Internet]. Load the Solver add-in in Excel. [cited 2021 Mar 15]. Available from: https://support.microsoft.com/en-us/office/load-the-solver-add-in-in-excel-612926fc-d53b-46b4-872c-e24772f078ca Engineer Excel [Internet]. Excel Solver: which solving method should I choose? [cited 2021 Mar 25]. Available from: https://engineerexcel.com/excel-solver-solving-method-choose/. Hyndman RJ, Athanasopoulos G. Forecasting: principles and practice. OTexts; 2018. Kim, Heeyoung. A new metric of absolute percentage error for intermittent demand forecasts. Int J Forecasting. 2016 Sept;32(3):669–79. 10.1016/j.ijforecast.2015.12.003 Government of Ontario. Data Catalogue: status of COVID-19 cases in Ontario; 2021 [cited 2021 Feb 11]. Available from: https://data.ontario.ca/dataset/status-of-covid-19-cases-in-ontario Statistics Canada. Population estimates, quarterly [Internet]. [cited 2021 Feb 10]. Table 17-10-0009-01, Population estimates, quarterly. 10.25318/1710000901-eng Canadian Institute for Health Information. COVID-19 Intervention Scan - Data Tables. CIHI; 2020 [cited 2021 Feb 10]. Available from: https://www.cihi.ca/en/covid-19-intervention-scan Rachel Gilmore. It’s now recommended that Canadians wear face masks. CTV News [Internet]. 2020 May 20 [cited 2021 Feb 24]. Available from: https://www.ctvnews.ca/politics/it-s-now-recommended-that-canadians-wear-face-masks-1.4946752 Government of Ontario [Internet]. Ontario extending stay-at-home order across most of the province to save lives: public health units to gradually return to the COVID-19 Response Framework; 2021 Feb 8 [cited 2021 Feb 23]. Available from: https://news.ontario.ca/en/release/60261/ontario-extending-stay-at-home-order-across-most-of-the-province-to-save-lives Leung K, Shum MH, Leung GM, Lam TT, Wu JT. Early transmissibility assessment of the N501Y mutant strains of SARS-CoV-2 in the United Kingdom, October to November 2020. Euro Surveill. 2021 Jan;26(1). 10.2807/1560-7917.es.2020.26.1.2002106 Volz E, Mishra S, Chand M, Barrett JC, Johnson R, Geidelberg L et al. Transmission of SARS-CoV-2 lineage B.1.1.7 in England: insights from linking epidemiological and genetic data. medRxiv 2021. 10.1101/2020.12.30.20249034 Peter Horby, Catherine Huntley, Nick Davies, John Edmunds, Neil Ferguson. NERVTAG paper on COVID-19 variant of concern B.1.1.7 [Internet]. Department of Health and Social Care and Scientific Advisory Group for Emergencies; 2021 Jan 22 [cited 2021 Mar 5]. 9 p. Available from: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/961037/NERVTAG_note_on_B.1.1.7_severity_for_SAGE_77__1_.pdf Centers for Disease Control and Prevention [Internet]. SARS-CoV-2 variant classifications and definitions. CDC; 2021 [cited 2021 Mar 31]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/cases-updates/variant-surveillance/variant-info.html#Concern Government of Canada. Archived 5: NACI rapid response: extended dose intervals for COVID-19 vaccines to optimize early vaccine rollout and population protection in Canada [2021-03-03]; 2021 [cited 2021 Apr 1]. Available from: https://www.canada.ca/en/public-health/services/immunization/national-advisory-committee-on-immunization-naci/rapid-response-extended-dose-intervals-covid-19-vaccines-early-rollout-population-protection.html Government of Ontario [Internet]. Ontario adding over 500 hospital beds to expand critical care capacity: Mackenzie Health’s new Cortellucci Vaughan Hospital to support short-term pandemic response; 2021 [cited 2021 Mar 15]. Available from: https://news.ontario.ca/en/release/59982/ontario-adding-over-500-hospital-beds-to-expand-critical-care-capacity The Canadian Press [Internet]. Ontario frantically adding ICU beds to hospitals as occupancy due to COVID-19 hits record high; 2021 [cited 2021 Oct 14]. Available from: https://www.cp24.com/news/ontario-frantically-adding-icu-beds-to-hospitals-as-occupancy-due-to-covid-19-hits-record-high-1.5385128?cache= Government of Ontario [Internet]. Ontario maximizing critical care capacity to support hospitals: New emergency measure ensures patients receive high-quality care in the right setting. Ontario: Queen’s Printer for Ontario; 2021 [cited 2021 Apr 28]. Available from: https://news.ontario.ca/en/release/1000020/ontario-maximizing-critical-care-capacity-to-support-hospitals. Sean Davidson. Ontario moves 27 regions from stay-at-home order to reopening framework on Feb. 16. CTV News [Internet]. 2021 Feb 12 [cited 2021 Mar 10]. Available from: https://toronto.ctvnews.ca/ontario-moves-27-regions-from-stay-at-home-order-to-reopening-framework-on-feb-16-1.5306272 Sean Davidson. Here’s what can reopen as Toronto and Peel Region enter the grey zone. CTV News [Internet]. 2021 Mar 5 [cited 2021 Mar 10]. Available from: https://toronto.ctvnews.ca/here-s-what-can-reopen-as-toronto-and-peel-region-enter-the-grey-zone-1.5334937 Sean Davidson. Ontario extends all emergency orders under the Reopening Ontario Act until April 20. CTV News [Internet]. 2021 Mar 19 [cited 2021 Mar 25]. Available from: https://toronto.ctvnews.ca/ontario-extends-all-emergency-orders-under-the-reopening-ontario-act-until-april-20-1.5354733 Government of Ontario. COVID-19 vaccine data: Government of Ontario; 2021 [cited 2021 Mar 15]. Available from: https://data.ontario.ca/dataset/752ce2b7-c15a-4965-a3dc-397bf405e7cc/resource/8a89caa9-511c-4568-af89-7f2174b4378c/download/vaccine_doses.csv Detsky AS, Bogoch II. COVID-19 in Canada: experience and response. JAMA. 2020 Aug; 324(8):743–4. 10.1001/jama.2020.14033 Rachael D’Amore. How prevalent are variants? A closer look at what — and where — they are in Canada. Global News [Internet]. 2021 Feb 17 [cited 2021 Mar 29]. Available from: https://globalnews.ca/news/7645252/covid-variants-how-many-canada/. Government of Canada [Internet]. Canada’s COVID-19 immunization plan: saving lives and livelihoods. 2020 [cited 2021 Mar 25]. Available from: https://www.canada.ca/en/public-health/services/diseases/2019-novel-coronavirus-infection/canadas-reponse/canadas-covid-19-immunization-plan.html#a8 Callaway E. Beyond Omicron: what’s next for COVID’s viral evolution. Nature. 2021 Dec; 600(7888):204–7. 10.1038/d41586-021-03619-8 Wolter N, Jassat W, Walaza S, Welch R, Moultrie H, Groome M, et al. Early assessment of the clinical severity of the SARS-CoV-2 omicron variant in South Africa: a data linkage study. The Lancet. 2022 Jan;399(10323):437–46. 10.1016/S0140-6736(22)00017-4 Linka K, Rahman P, Goriely A, Kuhl E. Is it safe to lift COVID-19 travel bans? The Newfoundland story. medRxiv 2020. 10.1101/2020.07.16.20155614 Government of British Columbia. COVID-19: critical care and acute care hospitalization modelling [Technical briefing on the Internet]. British Columbia, Canada; 2020 Mar 27 [cited 2021 Mar 29]. 34 p. Available from: https://news.gov.bc.ca/files/COVID19_TechnicalBriefing_Mar27_2020.pdf Mishra S, Wang L, Ma H, Yiu KCY, Paterson JM, Kim E et al. Estimated surge in hospital and intensive care admission because of the coronavirus disease 2019 pandemic in the Greater Toronto Area, Canada: a mathematical modelling study. CMAJ Open. 2020 Sept; 8(3):E593–E604. 10.9778/cmajo.20200093 Aleman DM, Tham BZ, Wagner SJ, Semelhago J, Mohammadi A, Price P et al. How effective was Newfoundland and Labrador’s travel ban to prevent the spread of COVID-19? An agent-based analysis. medRxiv 2021:2021.02.05.21251157. 10.1101/2021.02.05.21251157 Champredon D, Najafi M, Laskowski M, Chit A, Moghadas SM. Individual movements and contact patterns in a Canadian long-term care facility. AIMS Public Health. 2018 May;5(2):111–21. 10.3934%2Fpublichealth.2018.2.111 City of Ottawa [Internet]. COVID-19 [cited 2021 Mar 30]. Available from: https://613covid.ca/model/. Ogden NH, Fazil A, Arino J, Berthiaume P, Fisman DN, Greer AL et al. Modelling scenarios of the epidemic of COVID-19 in Canada. Can Commun Dis Rep 2020 Jun;46(8):198–204. 10.14745/ccdr.v46i06a08 National Collaborating Centre for Infectious Diseases [Internet]. PHAC SEIR model on COVID-19. NCCID [cited 2021 Mar 29]. Available from: https://nccid.ca/phac-seir-model-on-covid-19/. Tuite A, Fisman D. Reporting, epidemic growth, and reproduction numbers for the 2019-nCoV epidemic: understanding control: Dalla Lana School of Public Health, University of Toronto [cited 2021 Mar 29]. Available from: https://art-bd.shinyapps.io/nCov_control/. Barrett K, Nakamachi Y, Ierasts T, Khan YA, Mac S, Naimark D et al. Forecasting demand for personal protective equipment for Ontario acute care hospitals during the COVID-19 pandemic: COVID-19-ModCollab; 2020 [cited 2021 Mar 30]. Available from: https://www.covid-19-mc.ca/interactive-models/ppe-estimator-policy-scenarios CBC News. N.L. could run out of intensive-care beds, new model predicts. Newfoundland and Labrador. CBC News [Internet]; 2020 Apr 8 [cited 2020 Feb 26]. Available from: https://www.cbc.ca/news/canada/newfoundland-labrador/newfoundland-labrador-covid-projection-1.5525160 CBC News. COVID-19 infections in Toronto this month could exceed peak reached in April, data suggests. CBC News [Internet]. 2020 Oct 7 [cited 2021 Feb 26]. Available from: https://www.cbc.ca/news/canada/toronto/new-modelling-data-toronto-public-health-infections-covid-19-eileen-de-villa-1.5754041 Bjørnstad ON, Shea K, Krzywinski M, Altman N. The SEIRS model for infectious disease dynamics. Nat Methods. 2020 Jun;17(6):557–8. 10.1038/s41592-020-0856-2 Tsori Y, Granek R. Epidemiological model for the inhomogeneous spatial spreading of COVID-19 and other diseases. PLoS One. 2021 Feb;16(2):e0246056. 10.1371/journal.pone.0246056 Huppert A, Katriel G. Mathematical modelling and prediction in infectious disease epidemiology. Clin Microbiol Infect 2013;19(11):999–1005. 10.1111/1469-0691.12308 Centers for Disease Control and Prevention [Internet]. COVID-19: People with certain medical conditions. CDC; 2021 [cited 2021 Apr 1]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/need-extra-precautions/people-with-medical-conditions.html Government of Ontario. COVID-19 vaccine distribution plan For deployment of Pfizer and Moderna vaccines [cited 2021 Feb 23]. Available from: https://files.ontario.ca/moh-covid-19-vaccine-distribution-plan-en-2021-02-19.pdf Kim DS, Rowland-Jones S, Gea-MallorquÍ E. Will SARS-CoV-2 infection elicit long-lasting protective or sterilising immunity? Implications for vaccine strategies (2020). Front Immunol. 2020 Dec;11:3190. 10.3389/fimmu.2020.571481