Compare forecasting models

Usage

compare_models(
  obs_rts = NULL,
  obs_cases = NULL,
  models = NULL,
  horizon = 7,
  samples = 1000,
  bound_rt = TRUE,
  timeout = 30,
  serial_interval = NULL,
  min_points = 3,
  rdist = NULL,
  return_raw = FALSE
)

Arguments

obs_rts: Dataframe of Rt observations to forecast with and score against. Should contain a date and rt column. If multiple samples are included this should be denoted using a numeric sample variable.
obs_cases: Dataframe of case observations to use for case prediction and scoring. Should contain a date and cases column. If multiple samples are included this should be denoted using a numeric sample variable.
models: A list of models. A configuration is given in the examples. Each model needs to be wrapped in a function that takes a ... argument and returns a dataframe of samples with each column representing a time horizon. Example: function(...) {EpiSoon::bsts_model(model = function(ss, y){bsts::AddAr(ss, y = y, lags = 3)}, ...)}.
horizon: Numeric, the time horizon over which to predict.
samples: Numeric, number of samples to take.
bound_rt: Logical, defaults to TRUE. Should Rt values be bounded to be greater than or equal to 0.
timeout: Numeric, timeout of model fitting in seconds. Defaults to 30 seconds.
serial_interval: A numeric vector describing the probability distribution the serial interval. See EpiNow::covid_serial_interval for an example of the format.
min_points: Numeric, defaults to 3. The minimum number of time points at which to begin iteratively evaluating the forecast.
rdist: A function to be used to sample the number of cases. Must take two arguments with the first specfying the number of samples and the second the mean. Defaults to rpois if not supplied
return_raw: Logical, should raw cases and rt forecasts be returned. Defaults to FALSE.

Value

A list of dataframes as produced by evaluate model but with an additional model column.

Examples

if (FALSE) {
## List of forecasting bsts models wrapped in functions.
models <- list(
  "AR 3" =
    function(...) {
      EpiSoon::bsts_model(
        model =
          function(ss, y) {
            bsts::AddAr(ss, y = y, lags = 3)
          }, ...
      )
    },
  "Semi-local linear trend" =
    function(...) {
      EpiSoon::bsts_model(
        model =
          function(ss, y) {
            bsts::AddSemilocalLinearTrend(ss, y = y)
          }, ...
      )
    },
  "ARIMA" =
    function(...) {
      fable_model(model = fable::ARIMA(y ~ time), ...)
    }
)



## Compare models
evaluations <- compare_models(EpiSoon::example_obs_rts,
  EpiSoon::example_obs_cases, models,
  horizon = 7, samples = 10,
  serial_interval = example_serial_interval
)

## Example evaluation plot for comparing forecasts
## with actuals for a range of models and time horizons.
plot_forecast_evaluation(evaluations$forecast_rts, EpiSoon::example_obs_rts, c(1, 3, 7)) +
  ggplot2::facet_grid(model ~ horizon) +
  cowplot::panel_border()

## Hack to plot observed cases vs predicted
plot_forecast_evaluation(
  evaluations$forecast_cases,
  EpiSoon::example_obs_cases, c(1, 3, 7)
) +
  ggplot2::facet_wrap(model ~ horizon, scales = "free") +
  cowplot::panel_border()
}