Adds a column with the coverage of central prediction intervals
to unsummarised scores as produced by score()
Usage
add_coverage(scores, by, ranges = c(50, 90))Arguments
- scores
A data.table of scores as produced by
score().- by
character vector with column names to add the coverage for.
- ranges
numeric vector of the ranges of the central prediction intervals for which coverage values shall be added.
Value
a data.table with unsummarised scores with columns added for the
coverage of the central prediction intervals. While the overall data.table
is still unsummarised, note that for the coverage columns some level of
summary is present according to the value specified in by.
Details
The coverage values that are added are computed according to the values
specified in by. If, for example, by = "model", then there will be one
coverage value for every model and add_coverage() will compute the coverage
for every model across the values present in all other columns which define
the unit of a single forecast.
Examples
library(magrittr) # pipe operator
# \dontshow{
data.table::setDTthreads(2) # restricts number of cores used on CRAN
# }
score(example_quantile) %>%
add_coverage(by = c("model", "target_type")) %>%
summarise_scores(by = c("model", "target_type")) %>%
summarise_scores(fun = signif, digits = 2)
#> The following messages were produced when checking inputs:
#> 1. 144 values for `prediction` are NA in the data provided and the corresponding rows were removed. This may indicate a problem if unexpected.
#> model target_type interval_score dispersion underprediction
#> 1: EuroCOVIDhub-baseline Cases 28000 4100 10000.0
#> 2: EuroCOVIDhub-baseline Deaths 160 91 2.1
#> 3: EuroCOVIDhub-ensemble Cases 18000 3700 4200.0
#> 4: EuroCOVIDhub-ensemble Deaths 41 30 4.1
#> 5: UMass-MechBayes Deaths 53 27 17.0
#> 6: epiforecasts-EpiNow2 Cases 21000 5700 3300.0
#> 7: epiforecasts-EpiNow2 Deaths 67 32 16.0
#> overprediction coverage_deviation bias ae_median coverage_50 coverage_90
#> 1: 14000.0 -0.110 0.0980 38000 0.33 0.82
#> 2: 66.0 0.120 0.3400 230 0.66 1.00
#> 3: 10000.0 -0.098 -0.0560 24000 0.39 0.80
#> 4: 7.1 0.200 0.0730 53 0.88 1.00
#> 5: 9.0 -0.023 -0.0220 78 0.46 0.88
#> 6: 12000.0 -0.067 -0.0790 28000 0.47 0.79
#> 7: 19.0 -0.043 -0.0051 100 0.42 0.91