Skip to contents

Overview

We make use of Gaussian Processes in several places in EpiNow2. For example, the default model for estimate_infections() uses a Gaussian Process to model the 1st order difference on the log scale of the reproduction number. This vignette describes the implementation details of the approximate Gaussian Process used in EpiNow2.

Definition

The one-dimensional Gaussian Processes (GPt\mathrm{GP}_t) we use can be written as

GP(μ(t),k(t,t′))\begin{equation} \mathrm{GP}(\mu(t), k(t, t')) \end{equation}

where μ(t)\mu(t) and k(t,t′)k(t,t') are the mean and covariance functions, respectively. In our case as set out above, we have

μ(t)≡0k(t,t′)=k(|t−t′|)=k(Δt)\begin{align} \mu(t) &\equiv 0 \\ k(t,t') &= k(|t - t'|) = k(\Delta t) \end{align}

with the following choices available for the kernel kk

Matérn 3/2 covariance kernel (the default)

k(Δt)=α2(1+3Δtρ)exp⁡(−3Δtρ)\begin{equation} k(\Delta t) = \alpha^2 \left( 1 + \frac{\sqrt{3} \Delta t}{\rho} \right) \exp \left( - \frac{\sqrt{3} \Delta t}{\rho}\right) \end{equation}

with ρ>0\rho>0 and α>0\alpha > 0 the length scale and magnitude, respectively, of the kernel. Note that here and later we use a slightly different definition of α\alpha compared to Riutort-Mayol et al.[1], where this is defined as our α2\alpha^2.

Squared exponential kernel

k(Δt)=α2exp⁡(−12(Δt2)l2)\begin{equation} k(\Delta t) = \alpha^2 \exp \left( - \frac{1}{2} \frac{(\Delta t^2)}{l^2} \right) \end{equation}

Ornstein-Uhlenbeck (Matérn 1/2) kernel

k(Δt)=α2exp⁡(−Δt2ρ2)\begin{equation} k(\Delta t) = \alpha^2 \exp{\left( - \frac{\Delta t}{2 \rho^2} \right)} \end{equation}

Matérn 5/2 covariance kernel

k(Δt)=α(1+5Δtρ+53(Δtl)2)exp⁡(−5Δtρ)\begin{equation} k(\Delta t) = \alpha \left( 1 + \frac{\sqrt{5} \Delta t}{\rho} + \frac{5}{3} \left(\frac{\Delta t}{l} \right)^2 \right) \exp \left( - \frac{\sqrt{5} \Delta t}{\rho}\right) \end{equation}

Hilbert space approximation

In order to make our models computationally tractable, we approximate the Gaussian Process using a Hilbert space approximation to the Gaussian Process[1], centred around mean zero.

GP(0,k(Δt))≈∑j=1m(Sk(λj))12ϕj(t)βj\begin{equation} \mathrm{GP}(0, k(\Delta t)) \approx \sum_{j=1}^m \left(S_k(\sqrt{\lambda_j}) \right)^\frac{1}{2} \phi_j(t) \beta_j \end{equation}

with mm the number of basis functions to use in the approximation, and values of λj\lambda_j given by

λj=(jπ2L)2\begin{equation} \lambda_j = \left( \frac{j \pi}{2 L} \right)^2 \end{equation}

where LL is a positive number termed boundary condition, and βj\beta_{j} are regression weights with standard normal prior

βj∼Normal(0,1)\begin{equation} \beta_j \sim \mathrm{Normal}(0, 1) \end{equation}

The function Sk(x)S_k(x) is the spectral density relating to a particular covariance function kk. In the case of the Matérn kernel of order ν\nu this is given by

Sk(x)=α22πΓ(ν+1/2)(2ν)νΓ(ν)ρ2ν(2νρ2+ω2)−(ν+12)\begin{equation} S_k(x) = \alpha^2 \frac{2 \sqrt{\pi} \Gamma(\nu + 1/2) (2\nu)^\nu}{\Gamma(\nu) \rho^{2 \nu}} \left( \frac{2 \nu}{\rho^2} + \omega^2 \right)^{-\left( \nu + \frac{1}{2}\right )} \end{equation}

For ν=3/2\nu = 3 / 2 (the default in EpiNow2) this simplifies to

Sk(ω)=α24(3/ρ)3((3/ρ)2+ω2)2=(2α(3/ρ)32(3/ρ)2+ω2)2\begin{equation} S_k(\omega) = \alpha^2 \frac{4 \left(\sqrt{3} / \rho \right)^3}{\left(\left(\sqrt{3} / \rho\right)^2 + \omega^2\right)^2} = \left(\frac{2 \alpha \left(\sqrt{3} / \rho \right)^{\frac{3}{2}}}{\left(\sqrt{3} / \rho\right)^2 + \omega^2} \right)^2 \end{equation}

For ν=1/2\nu = 1 / 2 it is

Sk(ω)=α22ρ(1/ρ2+ω2)\begin{equation} S_k(\omega) = \alpha^2 \frac{2}{\rho \left(1 / \rho^2 + \omega^2\right)} \end{equation}

and for ν=5/2\nu = 5 / 2 it is

Sk(ω)=α216(5/ρ)53((5/ρ)2+ω2)3\begin{equation} S_k(\omega) = \alpha^2 \frac{16 \left(\sqrt{5} / \rho \right)^5}{3 \left(\left(\sqrt{5} / \rho\right)^2 + \omega^2\right)^3} \end{equation}

In the case of a squared exponential the spectral kernel density is given by

Sk(ω)=α22πρexp⁡(−12ρ2ω2)\begin{equation} S_k(\omega) = \alpha^2 \sqrt{2\pi} \rho \exp \left( -\frac{1}{2} \rho^2 \omega^2 \right) \end{equation}

The functions ϕj(x)\phi_{j}(x) are the eigenfunctions of the Laplace operator,

ϕj(t)=1Lsin⁡(λj(t*+L))\begin{equation} \phi_j(t) = \frac{1}{\sqrt{L}} \sin\left(\sqrt{\lambda_j} (t^* + L)\right) \end{equation}

with the time points t=1,…,tGPt = 1, \ldots, t_\mathrm{GP} rescaled linearly to be between -1 and 1,

t*=t−12(tGP+1)12(tGP−1)\begin{equation} t^* = \frac{t - \frac{1}{2}(t_\mathrm{GP} + 1)}{\frac{1}{2}(t_\mathrm{GP} - 1)} \end{equation}

where tGPt_\mathrm{GP} is the number of time points to which the Gaussian process is applied. The lengthscale is rescaled in the same way, dividing by the half-range S=(tGP−1)/2S = (t_\mathrm{GP} - 1) / 2.

Relevant default priors are

α∼HalfNormal(0,0.01)ρ∼LogNormal(μρ,σρ)\begin{align} \alpha &\sim \mathrm{HalfNormal}(0, 0.01) \\ \rho &\sim \mathrm{LogNormal} (\mu_\rho, \sigma_\rho)\\ \end{align}

with ρ\rho additionally constrained with an upper bound of 6060 and μρ\mu_{\rho} and σρ\sigma_\rho calculated using a mean of 21 and standard deviation of 7.

Choosing the boundary and number of basis functions

The accuracy of the approximation depends on the boundary LL and the number of basis functions mm relative to the lengthscale. We write L=cSL = cS with cc the boundary factor and SS the half-range of the time points, and ℓ=ρ/S\ell = \rho / S for the lengthscale relative to the half-range. For the Matérn 3/2 kernel,[1] (Section 4.3.1) show that the approximation is accurate when

c≥max⁡(1.2,4.5ℓ)m≥3.42cℓ\begin{align} c &\geq \max(1.2, 4.5 \ell) \\ m &\geq 3.42 \frac{c}{\ell} \end{align}

with constants (3.2,1.75)(3.2, 1.75) for the squared exponential kernel and (4.1,2.65)(4.1, 2.65) for the Matérn 5/2 kernel. Longer lengthscales therefore need a wider boundary, and for a given lengthscale no number of basis functions compensates for a boundary that is too narrow. Shorter lengthscales need more basis functions.

By default, EpiNow2 sets cc from the 95% quantile of the lengthscale prior and then mm from the 5% quantile, so that lengthscales across the bulk of the prior are approximated accurately. This 5%/95% split is a convention, not something[1] derive; it is not revisited by a diagnostic the way mm and cc themselves are (see below), so it is worth keeping in mind as a source of inaccuracy distinct from the mm/cc relationships.[1] do not cover the Ornstein-Uhlenbeck kernel (Matérn 1/2). Reproducing their simulation procedure for it gives m≥8.6cℓm \geq 8.6 \frac{c}{\ell}: Matérn 1/2 is the roughest of the four kernels covered here, so it needs more basis functions than Matérn 3/2 for the same accuracy, not fewer. Its boundary constant was not independently re-derived and keeps the Matérn 3/2 value (c≥4.5ℓc \geq 4.5 \ell), which is not confirmed conservative for this kernel. The periodic kernel does not use a boundary, and[1] (Appendix B) show that it is approximated accurately when m≥3.72/ℓm \geq 3.72 / \ell. By default, EpiNow2 sets mm for the periodic kernel from the 5% quantile of the lengthscale prior.

After fitting, EpiNow2 applies the lengthscale diagnostic of[1] (Section 4.5). A warning is given if the posterior median lengthscale is shorter than the 3.42cS/m3.42 c S / m days that mm basis functions can represent, or longer than the cS/4.5c S / 4.5 days that the boundary supports. For the periodic kernel only the first check applies, with 3.72S/m3.72 S / m days. Either can be addressed by adjusting the lengthscale prior, from which the settings are chosen, or by setting basis_prop and boundary_scale in gp_opts() directly.

Checking the approximation

[1] (Section 4.5.1) give a step-by-step procedure for checking the approximation. Adapted to EpiNow2 with the Matérn 3/2 kernel, it is as follows.

  • Fit the model with the default settings, which are chosen from the lengthscale prior. This replaces the single initial guess of the lengthscale in the paper.
  • Check that the model has converged before reading the lengthscale diagnostic: the diagnostic reads the posterior median lengthscale, which is not meaningful from a chain that has not converged. If it has not, address that first, for example by reconsidering the prior or model specification, and refit; only once the model converges does it make sense to look at the lengthscale diagnostic at all.
  • If the posterior median lengthscale ρ̂\hat{\rho} is too short, refit with m≥3.42cS/ρ̂m \geq 3.42 c S / \hat{\rho}, by setting basis_prop to m/tGPm / t_\mathrm{GP}.
  • If ρ̂\hat{\rho} is too long, refit with boundary_scale set to at least 4.5ρ̂/S4.5 \hat{\rho} / S, and mm chosen as above.
  • Repeat until the diagnostic gives no warning.
  • Passing the diagnostic does not guarantee an accurate approximation. Refit with more basis functions (the paper adds 5) and check that ρ̂\hat{\rho} and predictive performance, for example the ELPD, are stable.
  • The approximation error is largest for the posterior standard deviation and near the boundaries. In EpiNow2 this affects the most recent estimates, so check these in particular.

For other kernels, use their constants from above.

Modelling the reproduction number

The default estimate_infections() model uses the GP to model the first-order difference of log⁡Rt\log R_t, with the trajectory itself built by cumulative summation of GP values GPi\mathrm{GP}_i. Writing TobsT_\mathrm{obs} for the length of the observation window and R1R_1 for the initial reproduction number, this construction is

log⁡Rt=log⁡R1+∑i=1t−1GPi\begin{equation} \log R_t = \log R_1 + \sum_{i = 1}^{t - 1} \mathrm{GP}_i \end{equation}

Without further constraint this parameterisation makes inference difficult: a constant added to log⁡R1\log R_1 and subtracted from the cumulative sum of GP values leaves log⁡Rt\log R_t unchanged for all tt, so R1R_1 and the long-run drift of the GP are not jointly identified by the data. The likelihood is flat along this direction (a ridge in the joint posterior), giving the sampler poor geometry and, on some random seeds, stuck chains and large R̂\hat R values.

This parallels a common issue in standard regression problems, where the intercept and the slope can trade off against each other unless the predictors are centred. There, a common fix is to centre the predictors so the intercept becomes the response value at the mean of the predictors rather than at zero, which decorrelates the intercept from the slope and gives the sampler a cleaner geometry; this is how, for example, brms parameterises intercepts internally. Here the cumulative GP increments play the role of the slope (they determine the drift of log⁡Rt\log R_t) and log⁡R1\log R_1 plays the role of the intercept.

To apply the same fix we look for an alternative parameter that is well-identified by the data. Averaging the construction above over the observation window gives

log⁡R¯:=1Tobs∑t=1Tobslog⁡Rt=log⁡R1+GP¯\begin{equation} \overline{\log R} := \frac{1}{T_\mathrm{obs}} \sum_{t = 1}^{T_\mathrm{obs}} \log R_t = \log R_1 + \overline{\mathrm{GP}} \end{equation}

where

GP¯=1Tobs∑t=1Tobs∑i=1t−1GPi=1Tobs∑i=1Tobs−1(Tobs−i)GPi\begin{equation} \overline{\mathrm{GP}} = \frac{1}{T_\mathrm{obs}} \sum_{t = 1}^{T_\mathrm{obs}} \sum_{i = 1}^{t - 1} \mathrm{GP}_i = \frac{1}{T_\mathrm{obs}} \sum_{i = 1}^{T_\mathrm{obs} - 1} (T_\mathrm{obs} - i) \, \mathrm{GP}_i \end{equation}

is the empirical mean over the observation window of the cumulative sum of GP values. Unlike log⁡R1\log R_1, the mean of log⁡Rt\log R_t over the observation window is informed by the entire data series and so does not suffer from the ridge. We therefore sample log⁡R¯\overline{\log R} as the internal parameter, recover log⁡R1=log⁡R¯−GP¯\log R_1 = \overline{\log R} - \overline{\mathrm{GP}}, and rewrite the construction as

log⁡Rt=log⁡R¯+∑i=1t−1GPi−GP¯.\begin{equation} \log R_t = \overline{\log R} + \sum_{i = 1}^{t - 1} \mathrm{GP}_i - \overline{\mathrm{GP}}. \end{equation}

Centring only over the observation window (not the forecast horizon) ensures that the fitted historical RtR_t does not shift when the forecast horizon is changed.

The user’s prior on the initial reproduction number is then applied to the derived quantity

R1=exp⁡(log⁡R¯−GP¯)\begin{equation} R_1 = \exp(\overline{\log R} - \overline{\mathrm{GP}}) \end{equation}

via a change of variables. Stan explores log⁡R¯\overline{\log R} on the unconstrained scale, so the prior placed there must carry the Jacobian of the map to R1R_1,

|∂R1∂log⁡R¯|=R1,\begin{equation} \left| \frac{\partial R_1}{\partial \overline{\log R}} \right| = R_1, \end{equation}

contributing log⁡R1\log R_1 to the target log density.

The internal parameter is declared as a positive real R¯=exp⁡(log⁡R¯)\overline{R} = \exp(\overline{\log R}), so Stan supplies log⁡R¯\log \overline{R} itself as the Jacobian of its own log transform. The correction added by hand is therefore the difference

log⁡R1−log⁡R¯=−GP¯.\begin{equation} \log R_1 - \log \overline{R} = - \overline{\mathrm{GP}}. \end{equation}

A breakpoint random walk, when used, is centred over the observation window in the same way and enters log⁡Rt\log R_t as a further additive term. Its contribution to the offset between log⁡R1\log R_1 and log⁡R¯\overline{\log R} adds to −GP¯-\overline{\mathrm{GP}}, and since it too is a deterministic function of the other parameters that does not involve log⁡R¯\overline{\log R}, the change of variables and its Jacobian are unchanged.

References

1.
Riutort-Mayol, G., Bürkner, P.-C., Andersen, M. R., Solin, A., & Vehtari, A. (2023). Practical hilbert space approximate bayesian gaussian processes for probabilistic programming. Statistics and Computing, 33(1), 17. https://doi.org/10.1007/s11222-022-10167-2