Optimal Vaccination Policies for COVID-19 Considering Vaccine Doses Delays

ABSTRACT In this work we introduce a methodology to determine an optimal vaccination strategy for the COVID-19 disease with limited vaccination capacity, considering the delays between vaccine doses in order to minimize the number of deaths. We start proposing a compartmental model in order to study the evolution of the pandemic before the existence of a vaccine, in the city of Tandil, province of Buenos Aires, Argentina. We calibrate the parameters involved according to official data. Based on that model, we design an age-structured optimal control problem to determine the best way of administrate the available vaccines, taking into account their characteristics and the vaccination capacity. Finally, we compare optimal solutions with other feasible strategies, considering or not strict isolation measures.


INTRODUCTION
The COVID-19, known as the coronavirus 2019 disease, is an infectious disease caused by the SARS-CoV-2 virus.The first cases appeared in Wuhan, China, in December 2019, the virus quickly spread globally and the disease was characterized as a pandemic by the World Health Organization (WHO) on March 11, 2020 [18].To prevent the spread of the virus, governments have imposed travel restrictions, quarantines, social isolation and prophylaxis measures.Nevertheless, the global cumulative number of confirmed cases and deaths continued to grow rapidly, 122 COVID-19 VACCINATION POLICIES reaching more than a million cases and 50 thousand deaths on April 4, 2020 [19], 10 million cases and near 500 thousand deaths on June 29, 2020 [20], and over 100 million cases and 2,2 million deaths on January 31, 2021 [21].
Facing this situation, a lot of research was carried out from different areas with the aim of analyzing the evolution of the disease and proposing mitigation measures.In particular, it has been published a large number of works studying the COVID-19 dynamics through compartmental models (see for instance [1,2,3,5,8,12,14,15,16]).These are some of the most used models in epidemiology, whose study goes back to the work of Kermack and McKendrick that in 1927 introduced the well-known SIR model [11].In that early work the total population is divided into three compartments (Susceptible, Infected, and Removed) and it is assumed that the recovered patients do not get reinfected.Since some diseases have a certain incubation period, the introduction of a new compartment for the Exposed (but not infectious) population leads to the SEIR models (Susceptible, Exposed, Infected, and Removed), see for example [10].
The mentioned works have adapted compartmental models to COVID-19, according to the purpose of each study and taking into account various disease features and different health policies.SIR models considering finite-time quarantines as control are presented in [3] and [12].A SIRD model (where the Removed compartment is split into Recovered and Dead) is considered in [15] to perform a comparative analysis of the virus spread in the USA, Brazil and Colombia.Large compartmental models of SEIR type, explicitly considering asymptomatic patients, isolation, quarantine and/or hospitalization can be found in [2,8,14] and [1].An age-structured SIR model with confinement as a control variable is introduced in [5], following the lines of [6] Finally, there are some models that take into account the vaccination, such as [1].In this latest work, as in much of the epidemiology literature that considers models with vaccination, it is assumed that the vaccine is a single dose, see for instance [4] and [13].
In contrast, in this work, it is assumed that there are two vaccine doses to be applied and the objective is to determine which doses should be administered each day in order to minimize the number of fatalities, considering the delays between doses on a certain time range.Hence, the study differs from those presented in [1,4,13].The aim is to compute optimal vaccination strategies in a context of limited vaccination capacity (due to the low availability of vaccines, logistic issues, not enough health agents, etc.).
The analysis is carried out for the case of Tandil city, Buenos Aires province, Argentina, where reliable official data are available.
The main contribution of this work is the design of an age-structured control problem that represents the evolution of the pandemic when a two doses vaccine is available.This control problem will be based on some compartmental model that describes the evolution of the pandemic without vaccination.For the purpose of this work, a simplified version of the one introduced in [14] is considered, removing the compartments that explicitly consider asymptomatic population.Of course, the existence of asymptomatic population contribute to the dynamics of the evolution of the pandemic, but determining the percentage of them has been a difficult task in the absence of massive testing.Indeed, in [9] it is summarized the results of several studies, leading to very different conclusions about this proportion, which reinforces the difficulty in this topic.It is important to remark that the maintenance of these compartments (or even others as in [2]) does not represent a mathematical challenge, but it could increases considerably the scale of the computing problem.In addition, the hypothesis that most of the cases are detected in Tandil is not so unrealistic since it is a mid-size city of Argentina with a considerably good health system.Thus, for the sake of simplicity and according to the available data, the model is restricted to seven compartments: Susceptible, Exposed, Infected, Detected, Hospitalized, Recovered and Dead.
The article is organized as follows.In section 2, the methodology to compute the optimal vaccionation strategies is presented.In subsection 2.1 a detailed description of the model without vaccination is given, together with an explanation of the calibration issues.In subsection 2.2, based on the previous model, an age-structured optimal control problem is established taking into account the vaccination characteristics.Section 3 is devoted to simulating the evolution of the pandemic for different scenarios, taking into account the possible containment measures and comparing the optimal policy obtained in section 2 with other feasible strategies.Lastly, section 4 presents some final conclusions.

OPTIMAL VACCINATION STRATEGIES
As we mentioned above, one of the aim of this paper is to present a methodology to obtain optimal vaccination strategies.This methodology consist in two main steps.The first one is to design a compartmental model that fit well the evolution of the pandemic before the application of pharmaceutical measures, in particular vaccination.Once this model has been calibrated, we can modify it by introducing new variables and parameters that describe the new evolution of the pandemic, taking vaccination into account.The second step consist of solving an optimal control problem, where the dynamics is given by the modified original problem, the control represents the way of administering the available doses of vaccines, and the objective is to reduce the number of deaths.
We want to point out that several models can be considered for the first step, including those that take into account the existence of asymptomatic patients.Based on the model presented in the first step, we design an age-structured optimal control problem that allows us to consider a two-doses vaccine.As we mentioned before, this is one of the main characteristics of our work.Finally, to obtain the optimal solution we use an optimal control solver, so the computational burden will depend on the complexity of the model and the dimension of the age-structure control problem.In the next two subsection we present the models (with and without vaccination) that we consider in this work for the case of Tandil city.

Modelization and calibration without vaccination
In this section, we introduce the model without vaccination considered throughout this work.It is an SEIR compartmental type model sketched in Figure 1.It has 7 compartments where the infected population compartment is divided into three new ones: Infected (the people who are infected and can infect others), Detected (infected people who are detected and isolated) and Hospitalized.The removed population compartment is split into Recovered and Dead population, where this last compartment is denoted by F (for Fatalities) in order to distinguish it from Detected.In Table 1, we explain in detail the meaning of the variables (compartments) and coefficients.In this model, we consider that an exposed person, after approximately a period of time equal to T EI , begins to be infectious, that is, they are capable of transmitting the virus to other people with whom they have contact.After a period T ID , those infected are detected and from that moment we assume that they are isolated so that they stop contributing to the spread of the virus.We assume that once detected, patients can go through the disease in a mild way until they recover or, they can present severe symptoms and be hospitalized.Lastly, we assume that only people Trends Comput.Appl.Math., 24, N. 1 (2023) who were hospitalized are those who decease.Taking all this into account, the dynamics of our model are as follows: T HF H(t). (2.1) In order to anticipate the evolution of the disease using this model, it is necessary to know the values of the parameters (coefficients and initial conditions).Some coefficients can be found in the recent bibliography (see for instance [17]) and others can be obtained from data provided by official agencies (https://www.tandil.gov.ar/,https://www.argentina.gob.ar/).However, to obtain the rest of the parameters and the initial conditions it is necessary to calibrate the model using the available information.
The calibration is carried out with a one day Euler time-discretization of system (2.1) by looking for the minimum of the squared error between the observed data x and the model trajectories x.
In our case, we have the following objective function where x(t) = (S(t), E(t), I(t), D(t), H(t), R(t), F(t)) is the solution of the discrete system associated with (2.1) with initial conditions ξ and I corresponds to the set of observable variables {D, H, R, F}.
We consider the available data until March 23, 2021, and we use the previous 15 days to calibrate the model 1 .The values are presented in Table 2.
We simulate our model using the calibrated parameters and compare it with the data of the following 15 days as shown in Figure 2.

An optimal control problem with vaccination
Based on the compartmental model presented in the above section, we propose an age-structured control model to analyze the best way of administering vaccine doses, taking into account the main features of the available vaccines in Argentina at the beginning of 2021 and the vaccination capacity per day.The aim of this section is to design an optimal control problem where the control is the vaccination policy.We consider the situation in the city of Tandil at the beginning of 2021 when the first vaccines arrived in Argentina.At that moment the most available vaccine in Argentina was the Sputnik V vaccine (see https://sputnikvaccine.com/).Taking into account some of the characteristics of this vaccine, we formulate our assumptions for the model.Like most vaccines available so far, two doses should be given and a minimum and maximum delay between doses is suggested.It is also known that after the application of the first dose, it takes a few weeks before obtaining a significant level of immunity.Taking all these features into consideration, we formulate a vaccination-age model, rather than one with infection-age as in [5].
Considering the compartmental model presented in (2.1), we introduced two controls u 1 and u 2 that represent the proportion of first and second doses applied per day.We add the susceptible vaccinated population, which depends on the vaccination age, that is the number of days since they received the first dose.Since the development of the disease changes in people who have already received a dose of the vaccine, we introduce the compartments of exposed vaccinated population E V , infected vaccinated population I V , and detected infected population D V .Once Trends Comput.Appl.Math., 24, N. 1 (2023) they arrive at the hospital, we assume that the behavior of the disease no longer depends on being or not vaccinated.We summarize this information in Table 3.As we mention before, based on the characteristics of the Sputnik V vaccine, we make the following assumptions: the second dose can be applied between 21 and 84 days after the first dose; after 21 days of the first dose application a 70% of efficacy is attained and a reduction of 80% of severe cases; after the application of the second dose, full immunity is acquired.We also suppose that people with one dose of the vaccine who become infected do not receive the second dose, since they are already immunized.Furthermore, we assume that 84 days after the first dose is applied, the application of the second dose is mandatory, so we consider the variables v a , with a = 0, . . ., 83, that represent people who received the first dose of the vaccine a days ago.
The involved coefficients are the ones described in Table 1 with the addition of β V (the transmission coefficient under the effect of vaccine) and C V H (proportion from detected to hospitalized under the effect of vaccine).Taking into account the aforementioned considerations, we have Trends Comput.Appl.Math., 24, N. 1 (2023) 128 COVID-19 VACCINATION POLICIES Now we present our discrete time model with vaccination, where the time step is one day, k = 0, . . .N − 1: (2. 3) The initial conditions for this system are those obtained in Section 2.1.Finally, due to the availability of vaccines or the capacity of the health system, there is a maximum number of vaccines M > 0 that can be administered per day (M = 410 in our case).Therefore, the total number of first doses, second doses before 84 days from the first dose and second mandatory doses for having reached the 84 th day of vaccination, must be less than or equal to this maximum capacity.We have the following constraints: The objective of the optimal control problem is to minimize the number of deaths during a certain period of time N: min F N .Now, using the available data of Tandil until March 24, 2021, and the calibration of the parameters obtained in Section 2.1, we solve our optimal control problem with the solver Bocop2 (http://www.bocop.org,see [7]).Our objective is to solve the problem with a horizon of 250 days, but since there exists a delay between getting infected and dying, we work with a horizon of 300 days in Bocop.
In the following Figure we show the number of doses to be applied per day, corresponding to the optimal solution obtained with Bocop.In the picture on the left we can see the number of the first doses applied per day (u k 1 S k ).In the picture on the right we see in orange the number of second doses applied for having reached the vaccination-age of 84 days (v k 83 ) and in green the second doses applied to people with vaccination-age lower than 84 days (∑ 82 a=22 u k 2 v k a ).This strategy suggests starting by applying only first doses during the first 84 days, we observe that in the first picture every day the maximum vaccination capacity is reached (410 vaccines per day).In the following period of 84 days second mandatory doses are applied (for having reached the vaccination-age of 84 days) that complete the vaccination schedule and some first doses, because people with a single dose became infected and do not receive a second dose.The solution obtained suggests delaying the second dose until the maximum allowed interval during the first 210 days and then applying the second dose before the interval of 84 days.Notice again that since the vaccination of the last days has no impact on the cost, the behavior of the solution during the last period is not so regular.

NUMERICAL RESULTS
As we mentioned in the introduction, throughout this pandemic period, governments have taken different measures to mitigate the effects of COVID-19 on the population.Before the appearance of effective vaccines, non-pharmaceutical measures were implemented.Some of them, such as the use of masks and the hygiene of hands and surfaces, can be assumed to be maintained throughout the entire period.However, as they were not enough to stop the waves of contagion, strict confinement measures were implemented for certain periods of time and with different intensities.In model (2.1), these measures have impact in the transmission coefficient β , as will be explained later in subsection 3.2.Since, isolation measures are not known very far in advance, in section 2 we assumed that this coefficient remains constant.Nevertheless, it is to be expected that if the number of cases starts to climb rapidly, some containment measures will be taken.
In this section we study different possible scenarios of the evolution of the pandemic in Tandil, taking into account vaccination policies and confinement measures.In all the tests presented in this section, we consider the parameters calibrated in subsection 2.1 and a horizon of 250 days, starting from March 24, 2021.We simulate the trajectories in Python, with a one day Euler time-discretization of the corresponding dynamical system.

Vaccination strategies
We start by considering only vaccination strategies, without taking into account possible confinement measures.We present here five different tests.As we can see in Figure 4, there is an exponential growing until day 100, that is followed by a fast decreasing, which is the usual behaviour of this kind of models.On day 250, the population within the Exposed and Infected compartments is near zero so the pandemic is almost finished, with a large number of deaths.
In each of the following tests we show five different figures.The first two present the vaccination strategy, corresponding to the total number of vaccines, applied per day, of first and second doses.In the case of second doses, we differentiate between the second doses applied before the vaccination-age of 84 days and the second mandatory doses applied to people that have reached the vaccination-age of 84 days.The last three images show the evolution of the different variables of our model.

Test 2:
We simulate the evolution of the system, following the dynamics (2.3), using the strategy obtained with Bocop until day 250.In Figure 5 we show the vaccination strategy and the evolution of the variables involved in the model.Notice that the peak of the dynamics is reached a few days earlier and the values are considerably lower than those obtained without vaccination in Test 1.In particular, as expected, there is an important reduction of the death toll.Now, we compare the results obtained in Test 2 with other feasible and perhaps easier to implement vaccination policies.The following three tests show the evolution of the controlled system (2.3) for vaccination strategies with a fixed delay between doses.We consider delays between doses of 84, 42 and 28 days.
Test 3: We consider a fixed delay of 84 days between doses.Figure 6 shows the vaccination strategy and the dynamics.The obtained dynamics are very similar to those given by the optimal policy in Test 2.  Test 4: We consider a fixed delay of 42 days between doses, see Figure 7.In this case, we can see that the number of fatalities is larger than those obtained in Tests 2 and 3, but it is quite less than the one given by Test 1.
Trends Comput.Appl.Math., 24, N. 1 (2023) Test 5: We consider a fixed delay of 28 days between doses, see Figure 8.Here, the death toll is even higher than that obtained in Test 4, but still lower than that in Test 1.
We summarize the final values of the variables (after 250 days) in Table 4, where we can observe that using the optimal policy of Test 2 and the strategy of a fixed delay of 84 days of Test 3 we obtain almost the same value of the objective function, that is, the same number of deaths.Furthermore, in all cases, we obtain that the number of Exposed and Infected is almost zero, which indicates that after 250 days the pandemic would be over.
This result does not seem to be the closest to reality, but this is because in the previous tests we assumed that all the coefficients remain constant.In particular, the calibration of these parameters was carried out with the data available until March, in Tandil, when a significant increase in cases was beginning.So, we can conclude that the transmission coefficient β obtained in Section 2.1 is really high.Due to the increase in cases at the beginning of 2021, some containment measures were taken in Argentina to reduce this transmission factor.In the next section we consider these changes along with the different vaccination policies.

Confinement measures
In the above section, we analyze several scenarios of different vaccination policies assuming that the coefficients of the model remain constant throughout the entire period of time considered.
In particular, the transmission coefficient β , which is related to the number of contacts among people and the probability of infection when contact occurs, is not expected to remain constant if, for example, some confinement measures are taken.In fact, given that in April 2021 there was a large increase in the number of confirmed cases in Argentina, lockdown measures were taken during the months of May and June.
Taking that into account, we incorporate a new control to our model that can be interpreted as confinement measures that governments can take.This control w appears as a multiplicative  factor that reduces the transmission coefficients β and β V .We only rewrite the first equations of the system (2.3)where this control, w k ∈ (0, 1], is involved.
In the different scenarios analyzed in the previous section, considering the assumptions made about the vaccine, we realize that one of the best strategies is to delay the second dose until the maximum allowed interval.Now we study the same strategies analyzed before but adding some confinement measures to our model.
Considering the lockdown measures taken in Argentina, we define the following control: In Figure 9 we show the results obtained using the strategy of Test 2, where in the last two images with vertical dashed lines we separate the different periods of confinements.
Finally, we present in Table 5 the final values of the variables, using the control w and the vaccination policies of the tests of the previous section.We can conclude that even with these confinement measures, the best strategy is still to delay the second dose to the maximum allowed.

CONCLUSIONS
The objective of this work was to optimize the use of vaccines to reduce the number of deaths.We started by proposing a compartmental model that described well the behavior of the pandemic in Tandil, before the existence of vaccines.We calibrated the model with the data available up to March 2021 and then based on that model we design an optimal control problem that includes the administration of vaccines as a control.We solved this problem with an optimal control solver, and then compared the optimal strategy obtained with different scenarios, of no vaccination and of fixed delays between doses.
Based on the numerical results, we can conclude that delaying the second dose to the maximum allowed seems to be one of the best strategies.Of course, these results are related to the assumptions made about the vaccine.In the case of the vaccine considered here, a few weeks after the application of the first dose, a high level of immunity is achieved.If this level of immunity were lower, the optimal strategy would probably not be the same.
The optimal control problem in section 2 was posed assuming that the coefficients were constants, in particular that means that the transmission coefficient remains constant.As explained before, this is not the case if some lockdown measure are taken.So, in section 3 we explained how to modify the control model presented in section 2 to take into consideration this kind of confinement measures.Using this formulation for a particular containment measure, we simulated the evolution of the pandemic for the different vaccination policies studied in subsection 3.1.We observe that also in this case, as the delay between doses decreases, the number of deaths increases.We want to point out that in this case with confinement measures, after 250 days the number of the infected population is greater than zero, and maybe this situation is closer to the current situation of the pandemic, at least in Tandil.To end this section of conclusions, we want to mention that after the first submission of this paper, new variant of COVID-19 have appeared, such as the famous Omicron, and access to different kind of vaccine was gradually becoming possible.Therefore, it is difficult to compare the real situation in Tandil with the scenarios presented in this work.However, during most of 2021 the policy of delaying the second dose to the maximum allowed interval was one of the strategies adopted in Argentina.This decision, together with confinement measures, could have been one of the reasons for the delay of the first wave of Omicron compared to other countries.The most critical situation of the pandemic, in terms of the number of confirmed cases, but without saturation of the health system, was at the beginning of 2022.

Figure 2 :
Figure 2: Comparison between the data and the calibrated model.

Figure 3 :
Figure 3: Administration of first and second doses, following the optimal vaccination strategy obtained with Bocop.

Figure 6 :
Figure 6: Vaccination policy respecting a fixed of 84 days between doses.

Figure 7 :
Figure 7: Vaccination policy respecting a fixed delay of 42 days between doses.

Figure 8 :
Figure 8: Vaccination policy respecting a fixed delay of 28 days between doses.

Table 1 :
Variables and coefficients involved in the model.

EI average time of incubation I infected T ID average time to detection. D detected T DR average recovery time for detected H hospitalized T DH average time between detection and hospitalization R recovered T HR average hospitalization time for recovering
patients F dead C H proportion of detected patients to be hospitalized C F proportion of hospitalized patients who die

Table 2 :
Parameters and initial conditions on March 24, 2021.

Table 3 :
Control model variables.

Table 4 :
Comparison of vaccination policies without confinement after 250 days.

Table 5 :
Comparison of vaccination policies with confinement after 250 days.