
Gaussian Process implementation details
Source:vignettes/gaussian_process_implementation_details.Rmd
gaussian_process_implementation_details.RmdOverview
We make use of Gaussian Processes in several places in
EpiNow2. For example, the default model for
estimate_infections() uses a Gaussian Process to model the
1st order difference on the log scale of the reproduction number. This
vignette describes the implementation details of the approximate
Gaussian Process used in EpiNow2.
Definition
The one-dimensional Gaussian Processes () we use can be written as
where and are the mean and covariance functions, respectively. In our case as set out above, we have
with the following choices available for the kernel
Hilbert space approximation
In order to make our models computationally tractable, we approximate the Gaussian Process using a Hilbert space approximation to the Gaussian Process[1], centred around mean zero.
with the number of basis functions to use in the approximation, and values of given by
where is a positive number termed boundary condition, and are regression weights with standard normal prior
The function is the spectral density relating to a particular covariance function . In the case of the Matérn kernel of order this is given by
For
(the default in EpiNow2) this simplifies to
For it is
and for it is
In the case of a squared exponential the spectral kernel density is given by
The functions are the eigenfunctions of the Laplace operator,
with the time points rescaled linearly to be between -1 and 1,
where is the number of time points to which the Gaussian process is applied. The lengthscale is rescaled in the same way, dividing by the half-range .
Relevant default priors are
with additionally constrained with an upper bound of and and calculated using a mean of 21 and standard deviation of 7.
Choosing the boundary and number of basis functions
The accuracy of the approximation depends on the boundary and the number of basis functions relative to the lengthscale. We write with the boundary factor and the half-range of the time points, and for the lengthscale relative to the half-range. For the Matérn 3/2 kernel,[1] (Section 4.3.1) show that the approximation is accurate when
with constants for the squared exponential kernel and for the Matérn 5/2 kernel. Longer lengthscales therefore need a wider boundary, and for a given lengthscale no number of basis functions compensates for a boundary that is too narrow. Shorter lengthscales need more basis functions.
By default, EpiNow2 sets
from the 95% quantile of the lengthscale prior and then
from the 5% quantile, so that lengthscales across the bulk of the prior
are approximated accurately. This 5%/95% split is a convention, not
something[1] derive; it is not
revisited by a diagnostic the way
and
themselves are (see below), so it is worth keeping in mind as a source
of inaccuracy distinct from the
/
relationships.[1] do not cover
the Ornstein-Uhlenbeck kernel (Matérn 1/2). Reproducing their simulation
procedure for it gives
:
Matérn 1/2 is the roughest of the four kernels covered here, so it needs
more basis functions than Matérn 3/2 for the same accuracy, not fewer.
Its boundary constant was not independently re-derived and keeps the
Matérn 3/2 value
(),
which is not confirmed conservative for this kernel. The periodic kernel
does not use a boundary, and[1]
(Appendix B) show that it is approximated accurately when
.
By default, EpiNow2 sets
for the periodic kernel from the 5% quantile of the lengthscale
prior.
After fitting, EpiNow2 applies the lengthscale
diagnostic of[1] (Section 4.5).
A warning is given if the posterior median lengthscale is shorter than
the
days that
basis functions can represent, or longer than the
days that the boundary supports. For the periodic kernel only the first
check applies, with
days. Either can be addressed by adjusting the lengthscale prior, from
which the settings are chosen, or by setting basis_prop and
boundary_scale in gp_opts() directly.
Checking the approximation
[1] (Section 4.5.1) give a
step-by-step procedure for checking the approximation. Adapted to
EpiNow2 with the Matérn 3/2 kernel, it is as follows.
- Fit the model with the default settings, which are chosen from the lengthscale prior. This replaces the single initial guess of the lengthscale in the paper.
- Check that the model has converged before reading the lengthscale diagnostic: the diagnostic reads the posterior median lengthscale, which is not meaningful from a chain that has not converged. If it has not, address that first, for example by reconsidering the prior or model specification, and refit; only once the model converges does it make sense to look at the lengthscale diagnostic at all.
- If the posterior median lengthscale
is too short, refit with
,
by setting
basis_propto . - If
is too long, refit with
boundary_scaleset to at least , and chosen as above. - Repeat until the diagnostic gives no warning.
- Passing the diagnostic does not guarantee an accurate approximation. Refit with more basis functions (the paper adds 5) and check that and predictive performance, for example the ELPD, are stable.
- The approximation error is largest for the posterior standard
deviation and near the boundaries. In
EpiNow2this affects the most recent estimates, so check these in particular.
For other kernels, use their constants from above.
Modelling the reproduction number
The default estimate_infections() model uses the GP
to model the first-order difference of
,
with the trajectory itself built by cumulative summation of GP values
.
Writing
for the length of the observation window and
for the initial reproduction number, this construction is
Without further constraint this parameterisation makes inference difficult: a constant added to and subtracted from the cumulative sum of GP values leaves unchanged for all , so and the long-run drift of the GP are not jointly identified by the data. The likelihood is flat along this direction (a ridge in the joint posterior), giving the sampler poor geometry and, on some random seeds, stuck chains and large values.
This parallels a common issue in standard regression problems, where
the intercept and the slope can trade off against each other unless the
predictors are centred. There, a common fix is to centre
the predictors so the intercept becomes the response value at the
mean of the predictors rather than at zero, which decorrelates the
intercept from the slope and gives the sampler a cleaner geometry; this
is how, for example, brms
parameterises intercepts internally. Here the cumulative GP increments
play the role of the slope (they determine the drift of
)
and
plays the role of the intercept.
To apply the same fix we look for an alternative parameter that is well-identified by the data. Averaging the construction above over the observation window gives
where
is the empirical mean over the observation window of the cumulative sum of GP values. Unlike , the mean of over the observation window is informed by the entire data series and so does not suffer from the ridge. We therefore sample as the internal parameter, recover , and rewrite the construction as
Centring only over the observation window (not the forecast horizon) ensures that the fitted historical does not shift when the forecast horizon is changed.
The user’s prior on the initial reproduction number is then applied to the derived quantity
via a change of variables. Stan explores on the unconstrained scale, so the prior placed there must carry the Jacobian of the map to ,
contributing to the target log density.
The internal parameter is declared as a positive real , so Stan supplies itself as the Jacobian of its own log transform. The correction added by hand is therefore the difference
A breakpoint random walk, when used, is centred over the observation window in the same way and enters as a further additive term. Its contribution to the offset between and adds to , and since it too is a deterministic function of the other parameters that does not involve , the change of variables and its Jacobian are unchanged.