Score a Model Fit

Score a Model Fit

Usage

score_forecast(fit_samples, observations, scores = "all")

Arguments

fit_samples

A dataframe as produced by EpiSoon::forecast.

observations

A dataframe of observations against which to score. Should contain a date and rt column.

scores

Character vector defaulting to "all". Select which scores to return, default is all scores but any subset can be returned.

Value

A dataframe containing the following scores per forecast timepoint: dss, crps, logs, bias, and sharpness as well as the forecast date and time horizon.

Examples

if (FALSE) {
## Fit a model (using a subset of observations)
samples <- forecast_rt(EpiSoon::example_obs_rts[1:10, ],
  model = function(...) {
    EpiSoon::bsts_model(
      model =
        function(ss, y) {
          bsts::AddSemilocalLinearTrend(ss, y = y)
        }, ...
    )
  },
  horizon = 7, samples = 10
)

## Score the model fit (with observations during the time horizon of the forecast)
score_forecast(samples, EpiSoon::example_obs_rts)

## Return just CRPS, bias and sharpness
score_forecast(samples, EpiSoon::example_obs_rts, scores = c("crps", "sharpness", "bias"))

## Return just the CRPS
score_forecast(samples, EpiSoon::example_obs_rts, scores = "crps")
}