Background and Aims
The most restrictive nonpharmaceutical interventions (NPIs) for controlling the spread of COVID-19 are mandatory stay-at-home and business closures. Given the consequences of these policies, it is important to assess their effects. We evaluate the effects on epidemic case growth of more restrictive NPIs (mrNPIs), above and beyond those of less-restrictive NPIs (lrNPIs).
We first estimate COVID-19 case growth in relation to any NPI implementation in subnational regions of 10 countries: England, France, Germany, Iran, Italy, Netherlands, Spain, South Korea, Sweden and the United States. Using first-difference models with fixed effects, we isolate the effects of mrNPIs by subtracting the combined effects of lrNPIs and epidemic dynamics from all NPIs. We use case growth in Sweden and South Korea, 2 countries that did not implement mandatory stay-at-home and business closures, as comparison countries for the other 8 countries (16 total comparisons).
Implementing any NPIs was associated with significant reductions in case growth in 9 out of 10 study countries, including South Korea and Sweden that implemented only lrNPIs (Spain had a nonsignificant effect). After subtracting the epidemic and lrNPI effects, we find no clear, significant beneficial effect of mrNPIs on case growth in any country. In France, for example, the effect of mrNPIs was +7% (95% CI: −5%-19%) when compared with Sweden and + 13% (−12%-38%) when compared with South Korea (positive means pro-contagion). The 95% confidence intervals excluded 30% declines in all 16 comparisons and 15% declines in 11/16 comparisons.
While small benefits cannot be excluded, we do not find significant benefits on case growth of more restrictive NPIs. Similar reductions in case growth may be achievable with less-restrictive interventions.
The spread of COVID-19 has led to multiple policy responses that aim to reduce the transmission of the SARS-CoV-2. The principal goal of these so-called nonpharmaceutical interventions (NPI) is to reduce transmission in the absence of pharmaceutical options in order to reduce resultant death, disease and health system overload. Some of the most restrictive NPI policies include mandatory stay-at-home and business closure orders (‘lockdowns’). The early adoption of these more restrictive nonpharmaceutical interventions (mrNPIs) in early 2020 was justified because of the rapid spread of the disease, overwhelmed health systems in some hard-hit places and substantial uncertainty about the virus’ morbidity and mortality.1
Because of the potential harmful health effects of mrNPI—including hunger,2 opioid-related overdoses,3 missed vaccinations,4, 5 increase in non-COVID diseases from missed health services,6–9 domestic abuse,10 mental health and suicidality,11, 12 and a host of economic consequences with health implications13, 14—it is increasingly recognized that their postulated benefits deserve careful study. One approach to evaluating NPI benefits uses disease modelling approaches. One prominent modelling analysis estimated that, across Europe, mrNPIs accounted for 81% of the reduction in the effective reproduction number (), a measure of disease transmission.15 However, in the absence of empirical assessment of the policies, their effects on reduced transmission are assumed rather than assessed.16, 17 That analysis attributes nearly all the reduction in transmission to the last intervention, whichever intervention happened to be last, complete lockdowns in France or banning of public events in Sweden.16
Another, more empirically grounded approach to assessing NPI effects uses statistical regression models and exploits variation in the location and timing of NPI implementations to identify changes in epidemic spread following various policies.18 These empirical studies find large reductions in the growth rate of new cases that are attributable to NPIs. An important challenge with these analyses is that they use pre-policy growth rates to determine the ‘counterfactual’ trajectory of new cases—the expected case growth rate in the absence of NPIs. This is problematic because it is widely recognized that epidemic dynamics are time-varying, and brakes on disease transmission occur without any interventions (through resolution of infections), as well as from behaviour changes unrelated to the NPIs.19, 20 These epidemic dynamics are demonstrated by an analysis showing that slowing of COVID-19 epidemic growth was similar in many contexts, in a way that is more consistent with natural dynamics than policy prescriptions.21
These challenges suggest that assessing the impact of mrNPIs is important, yet difficult. We propose an approach that balances the strengths of empirical analyses while taking into consideration underlying epidemic dynamics. We compare epidemic spread in places that implemented mrNPIs to counterfactuals that implemented only less-restrictive NPIs (lrNPIs). In this way, it may be possible to isolate the role of mrNPIs, net of lrNPIs and epidemic dynamics.
Here, we use Sweden and South Korea as the counterfactuals to isolate the effects of mrNPIs in countries that implemented mrNPIs and lrNPIs. Unlike most of its neighbours that implemented mandatory stay-at-home and business closures, Sweden’s approach in the early stages of the pandemic relied entirely on lrNPIs, including social distancing guidelines, discouraging of international and domestic travel, and a ban on large gatherings.22, 23 South Korea also did not implement mrNPIs. Its strategy relied on intensive investments in testing, contact tracing and isolation of infected cases and close contacts.24, 25
We isolate the effect of more restrictive NPIs (mrNPIs) by comparing the combined effect size of all NPIs in 8 countries that implemented more restrictive policies (England, France, Germany, Iran, Italy, the Netherlands, Spain and the United States) with the effect size of all NPIs in the 2 countries that only implemented less-restrictive NPIs (lrNPIs). In effect, we follow the general scheme:
We analyse only these countries because the analysis depends on subnational data, which were only available for those countries, as explained further below.
The conceptual model underlying this approach is that, prior to meaningful population immunity, individual behaviour is the primary driver of reductions in transmission rate, and that any NPI may provide a nudge towards individual behaviour change, with response rates that vary between individuals and over time. lrNPIs could have large anti-contagion effects if individual behavioural response is large, in which case additional, more restrictive NPIs may not provide much additional benefit. On the other hand, if lrNPIs provide relatively small nudges to individual behaviour, then mrNPIs may result in large behavioural effects at the margin, and large reductions in the growth of new cases. However, because underlying epidemic dynamics are imprecisely characterized and are important for estimating the policy effects, our models test the extent to which mrNPIs had additional effect on reducing transmission by differencing the sum of NPI effects and epidemic dynamics in countries that did not enact mrNPIs from the sum of NPI effects and epidemic dynamics in countries that did.
We estimate the unique effects of mrNPIs on case growth rate during the Northern Hemisphere spring of 2020 in England, France, Germany, Iran, Italy, the Netherlands, Spain and the United States by comparing the effect of NPIs in these countries to those in Sweden and South Korea (separately). The data we use build on an analysis of NPI effects and consist of daily case numbers in subnational administrative regions of each country (eg regions in France, provinces in Iran, states in the United States and counties in Sweden), merged with the type and timing of policies in each administrative region.18, 26 We use data from a COVID-19 policy databank and previous analyses of policy impacts to determine the timing and location of each NPI.18, 27 Each observation in the data, then, is identified by the subnational administrative region and the date, with data on the number of cases on that date and indicators characterizing the presence of each policy. We include indicators for changes in case definitions or testing technologies to capture abrupt changes in case counts that are not the result of the underlying epidemic (these are mostly single-day indicators), as suggested in a previous analysis.18
We define the dependent variable as the daily difference in the natural log of the number of confirmed cases, which approximates the daily growth rate of infections (). We then estimate the following linear models:
The model terms are indexed by country (), subnational unit (), day () and NPI indicator (. is a series of fixed effects for the subnational unit, and is country-specific day-of-week fixed effects. The parameters of interest are , which identify the effect of each policy on the growth rate in cases. The parameter is a single-day indicator that models changes in case definitions that result in short discontinuities in case counts that are not due to underlying epidemic changes.
We estimate these models separately for each pair of countries (one with mrNPIs, one without), for a total of 16 models. We then add the coefficients of all the policies for the country with mrNPIs (yielding the combined effects of all NPIs in the mrNPI country) and subtract the combined effects of all NPIs in the comparator country without mrNPI. As noted above, the difference isolates the effect of mrNPIs on case growth rates. We estimate robust standard errors throughout, with clustering at the day-of-week level to account for serial correlation.
It is important to note that because the true number of infections is not visible in any country, it is impossible to assess the impact of national policies on transmission or new infections.28 Instead, we follow other studies evaluating the effects of NPIs that use case numbers, implicitly assuming that their observed dynamics may represent a consistent shadow of the underlying infection dynamics.18
The code for the data preparation, analysis and visualization is provided along with the article (Supplementary Material).
The growth rate in new cases prior to implementation of any NPIs was positive in all study countries (Figure 1). The figure shows that, across all subnational units in all ten countries, the average growth rate prior to NPIs ranged from 0.23 in Spain (23% daily growth; 95% CI: 0.13 to 0.34) to 0.47 (95% CI: 0.39 to 0.55) in the Netherlands. The average across all 10 countries was 0.32, and in South Korea and Sweden, the 2 countries without mrNPIs, the pre-NPI growth rates were 0.25 and 0.33, respectively. The variation of pre-policy growth rates in cases may reflect epidemic intensity, testing coverage (higher growth may be a reflection of expanding testing capacity and of more people wishing to be tested) and pre-policy behaviour changes that led to increased or decreased transmission.