Skip to content

Forecast evaluation ​

How the forecasts on the forecasts page have scored against the data that arrived afterwards. How the outbreak-size and reproduction-number estimates moved from release to release is on the in-sample checks page. Scoring is the continuous ranked probability score against a persistence baseline, defined in the forecast scoring Methods section. The same scoring by province is on the province forecast evaluation page and by health zone on the health-zone forecast evaluation page.

Load packages, data and fitted chains
julia
# Shared setup: packages, observations and the fit registry. See
# `docs/pages/_setup.jl`.
using BVDOutbreakSize
include(joinpath(pkgdir(BVDOutbreakSize), "docs", "pages", "_setup.jl"))
validation_forecast_from (generic function with 1 method)
julia
# The fits this page reads, loaded from the cache here.
chn_joint = load_fit("joint")
frozen_lastweek = load_fit("frozen_validation")
frozen_lastweek_streams = frozen_validation_stream_fits();

Summary ​

The overall bullets come first, then a table of each stream's scores, from the sections below. Relative skill is the model's CRPS over the persistence baseline's, so a value below one beats the baseline. Coverage is the fraction of forecasts whose observed value falls inside the 90% predictive interval, nominally 0.9.

  • Joint model across releases: beat the baseline on 1 of 5 streams.

  • Frozen joint model: beat the baseline on 2 of 5 streams.

StreamEvaluationRelative skill90% coverageBiasForecasts
confirmed casesAcross releases5.070.870.42172
confirmed deathsAcross releases6.280.880.1170
isolation bedsAcross releases1.680.970.13162
onset reportsAcross releases0.980.84-0.131
recoveredAcross releases1.440.930.2774
confirmed casesFrozen fit2.380.790.3397
confirmed deathsFrozen fit0.940.880.05341
isolation bedsFrozen fit0.890.990.15344
onset reportsFrozen fit1.060.88-0.1932
recoveredFrozen fit8.520.610.6167

Forecast validation ​

How last week's forecast held up against the data since observed, from the frozen refit in forecast-versus-frozen evaluation. Only the streams the situation reports still update are validated, since a stopped stream repeats its last total and leaves nothing to score. The projected bed occupancy is scored against the beds held a week later.

Fit one week back and validate the one-week-ahead forecast
julia
# frozen_lastweek and frozen_lastweek_streams are computed in the setup
# block above, and `validation_forecast_from` is defined there.
validation_forecast = validation_forecast_from("frozen_validation");

# Each frozen individual (single-stream) fit's own one-week-ahead new-count
# forecast at the same cut-off as `frozen_lastweek`, from
# [`forecast_stream`](@ref) (the same per-stream forecaster
# `stream_forecasts.csv` uses), so the validation plots below can show the
# individual fit alongside the joint rather than the joint alone. Recovered
# has no individual fit and is absent here, as it is throughout this report.
# Only the still-reported streams are fitted at the validation cut-off, so
# a stream the situation reports have stopped updating is absent from
# `frozen_lastweek_streams` and carries no individual series here.
function _validation_individual_new(sid, stream::Symbol)
    haskey(frozen_lastweek_streams, sid) || return nothing
    return Float64.(
        forecast_stream(
            fit_forecast("frozen_validation_$sid"), stream; horizon = 7
        )
    )
end
validation_individual = NamedTuple(
    k => v
        for (k, v) in pairs(
            (;
                cases_new = _validation_individual_new(
                    "cases", :reported_cases
                ),
                deaths_new = _validation_individual_new(
                    "deaths", :suspected_deaths
                ),
                confirmed_new = _validation_individual_new(
                    "confirmed", :confirmed_cases
                ),
                confirmed_deaths_new = _validation_individual_new(
                    "confirmed_deaths", :confirmed_deaths
                ),
            )
        )
        if !isnothing(v)
)
# The frozen individual (treatment-only) fit's own bed-occupancy forecast.
# `nothing` when the beds have stopped being reported, so the treatment fit
# is absent; the bed panel then draws the joint alone.
validation_individual_isolation = haskey(frozen_lastweek_streams, "treatment") ?
    Float64.(
        forecast_stream(
            fit_forecast("frozen_validation_treatment"), :isolation_beds;
            horizon = 7
        )
    ) : nothing

# The observed beds at the current cut-off (the forecast target), so the
# frozen-fit bed forecast is scored against what the beds actually held.
# Held back once the beds stop being reported, since the last count would
# then be carried forward rather than observed at the target date.
_obs_beds = stream_reporting(obs, :isolation_beds) ?
    obs.isolation_history.counts[end] : missing
# Same observed/baseline keying as the plot below, so the table covers every
# fitted count stream (cumulative and new-count rows) plus the bed level.
# A harmonisation-break day between the frozen cut-off and the current one
# puts records into the confirmed cumulative that were never notified in that
# week, so the new-count truth carries a step the forecast was never
# predicting. Take it out, the same correction `score_releases.jl` applies.
# Grid days are relative to a seeding date fixed by the genetic tmrca, so the
# frozen fit's own `n` and the current `obs.n` index the same grid.
validation_breaks = (
    confirmed_cum = confirmed_break_correction(
        obs, frozen_lastweek.o.n, obs.n
    ),
    confirmed_deaths_cum = confirmed_break_correction(
        obs, frozen_lastweek.o.n, obs.n; deaths = true
    ),
)

# Observed cumulative at the target date per stream, keyed by the forecast's
# cumulative column; `baseline` is each stream's origin cumulative (the
# frozen cut-off), so the new count is scored against observed minus origin,
# less any harmonisation the window carries (see `validation_breaks`). Both
# the table and the plot below take the still-reported streams
# (`reporting_cum_cols`, from the setup block): a stream the situation
# reports have stopped updating has an origin and a target reading the same
# repeated total, so its cumulative truth is stale and its new-count truth is
# a guaranteed zero.
validation_observed = (
    cases_cum = obs.reported_cases,
    deaths_cum = obs.total_deaths,
    confirmed_cum = obs.confirmed_cases,
    confirmed_deaths_cum = obs.confirmed_deaths,
    recovered_cum = obs.recovered_cases,
)
validation_baseline = (
    cases_cum = frozen_lastweek.o.reported_cases,
    deaths_cum = frozen_lastweek.o.total_deaths,
    confirmed_cum = frozen_lastweek.o.confirmed_cases,
    confirmed_deaths_cum = frozen_lastweek.o.confirmed_deaths,
    recovered_cum = frozen_lastweek.o.recovered_cases,
)

validation_table = forecast_vs_truth(
    validation_forecast;
    observed = keep_streams(validation_observed, reporting_cum_cols),
    baseline = keep_streams(validation_baseline, reporting_cum_cols),
    breaks = validation_breaks,
    isolation = _obs_beds
);
Forecast-versus-observed validation table
StreamQuantityObservedLower 90%Lower 60%Lower 30%Upper 30%Upper 60%Upper 90%Within 90% PI
DRC confirmed casescumulative by T+711504113791145311498115851164211759yes
DRC confirmed casesnew this week462337411456543600717yes
DRC confirmed deathscumulative by T+75630556155885603563056475677yes
DRC confirmed deathsnew this week243174201216243260290yes
DRC recovered among confirmedcumulative by T+73142305632183280336934273568yes
DRC recovered among confirmednew this week15266228290379437578yes
DRC isolation bedsoccupancy at T+7105553572282398910921319yes
Last week's frozen forecast against the data since observed, for the streams still reported. As in the scores below, the confirmed new-count rows take out any retrospective harmonisation step in the week, since it reattaches records notified earlier. The cumulative rows are scored against the published total, harmonisation included.
Forecast-versus-observed plot
julia
validation_fig = plot_forecast_vs_truth(
    validation_forecast;
    observed = keep_streams(validation_observed, reporting_cum_cols),
    baseline = keep_streams(validation_baseline, reporting_cum_cols),
    breaks = validation_breaks,
    individual = keep_streams(validation_individual, reporting_cum_cols)
);

The one-week-ahead forecast from the frozen fit for each still-reported count stream, cumulative and new, with the 90% predictive interval shaded and the count observed by the cut-off as a dashed black rule. A stream with its own single-stream fit carries that fit's forecast from the same cut-off as a dotted step outline.
Bed forecast-versus-observed plot
julia
validation_beds_fig = plot_forecast_beds_vs_truth(
    validation_forecast;
    isolation = _obs_beds, individual = validation_individual_isolation
);

Last week's projected bed occupancy against the beds occupied now (dashed), with the treatment-only fit's projection as a dotted step outline.
Forecast-versus-now latent plot
julia
# Current fit's draws of the new latent counts over the past week, the last
# seven days of each cumulative-trajectory deterministic.
function _now_new(chn, key)
    mat = chn[key]
    trajs = [collect(v) for v in vec(collect(mat))]
    return Float64[t[end] - t[max(1, length(t) - 7)] for t in trajs]
end
now_latent = (;
    infections_new = _now_new(chn_joint, :cumulative_infections),
    onsets_new = _now_new(chn_joint, :cumulative_onsets),
    deaths_latent_new = _now_new(chn_joint, :cumulative_expected_deaths),
)

validation_latent_fig = plot_forecast_vs_truth_latent(
    validation_forecast; now = now_latent
);

The frozen fit's forecast of the past week's new infections, onsets and deaths against the current fit's estimate for the same window. These quantities are not observed, so the two posterior distributions are compared directly.

Streams no longer reported ​

Forecast for the streams no longer reported
julia
# The last-reported date per stopped stream, and the frozen fit's own
# projection for them. `plot_forecast` draws a panel per new-count column
# the frame carries, so passing the stopped streams' columns alone gives the
# projection without the fabricated truth rule the validation figure would
# otherwise draw against a repeated total.
validation_stopped_streams = let s = stream_report_status(obs),
        ids = [stream_id(c) for c in stopped_cum_cols]

    keep = [r.stream in ids for r in eachrow(s)]
    DataFrame(
        "Stream" => s[keep, :label],
        "Last reported" => s[keep, :last_date]
    )
end
_stopped_new_cols = [
    c
        for c in new_cols(stopped_cum_cols)
        if c in propertynames(validation_forecast)
]
validation_stopped_fig = plot_forecast(
    validation_forecast[!, _stopped_new_cols]
);
StreamLast reported
Suspected cases2026-05-26
Suspected deaths2026-05-26
The streams the situation reports no longer update, with the date each was last reported.

The frozen fit's projection for these streams over the same week, with no observed rule, since their counts have not moved since they stopped.

Forecast scoring across releases ​

Every release's saved one- to four-week-ahead forecast is scored against the data observed since, against a persistence baseline and, where one exists, the stream's own individual fit. The tables in this section are the joint model's, one row per stream. Recovered has no individual fit of its own, so it is compared with the baseline only. Reported cases and suspected deaths stopped being updated partway through the outbreak, and the exports detection series is anchored to an earlier cut-off. Exports therefore has no scored forecast, and reported cases and suspected deaths each rest on a single matched forecast.

Each row shows the number of matched forecasts it rests on.

Four things are excluded from the scores, each for the reason given:

  • One whole reconstruction (results-v1.6.0), whose chain forecasts a near-zero median at every horizon and stream with an occasional five- or six-digit upper tail, the signature of a chain that failed to sample.

  • In the frozen section only, the confirmed-death rows of the fourteen frozen reconstructions cut between 16 July and 15 August 2026. Their forecaster floored that stream at zero, giving a one-week median of zero against an observed 250 to 370.

  • An onset window containing a vintage whose reread total falls, since its increment is not what the situation reports added.

  • A stream that carries no persistence baseline, which is what makes a window scoreable at all.

Nothing is dropped from the archive itself, and the score and overlay files in the data directory record everything that was scored.

The symptom-onset stream is scored on the new reported count each vintage adds (see symptom-onset nowcast and forecast), and on some vintages the reread total falls. Read the onset row through its skill against the baseline, and as resting on a handful of windows.

Load and summarise the cross-release forecast scores
julia
forecast_scores_df = _release_data(
    "forecast_scores.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, crps = Float64,
        log_crps = Float64, dispersion = Float64, overprediction = Float64,
        underprediction = Float64, coverage_50 = Float64,
        coverage_90 = Float64,
        bias = Float64, n_samples = Int,
        log_rel_to_baseline = Float64,
    )
)
forecast_overlay_df = _release_data(
    "forecast_overlay.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, observed = Float64,
        median = Float64, lo30 = Float64, hi30 = Float64, lo60 = Float64,
        hi60 = Float64, lo90 = Float64, hi90 = Float64,
    )
)
# The digitised onset triangle's own per-vintage total, as calendar dates,
# and the windows it makes unscoreable. A vintage that rereads the figure
# lower gives a window an increment the reports did not add, so the window
# is dropped from the scores and the overlay alike (see
# `drop_rescanned_onset_windows`). The current triangle is used to judge
# every release, since the scored truth is read off this one series.
_onset_vintage_dates = grid_date.(obs.onset_report_history.days)
_onset_vintage_totals = obs.onset_report_history.counts
_drop_rescanned(tbl) = drop_rescanned_onset_windows(
    tbl; vintage_dates = _onset_vintage_dates,
    vintage_totals = _onset_vintage_totals
)
forecast_scores_df = _drop_rescanned(forecast_scores_df)
forecast_overlay_df = _drop_rescanned(forecast_overlay_df)

# One row per (stream, fit) pooled over every horizon and release. The
# by-horizon and by-release detail tables carry the same columns at a finer
# grain (see src/scoring.jl). Every fit is kept here, since the
# relative-skill figure below compares the roles against each other. The
# tables rendered in this section select the joint role, and the individual
# fits are tabulated in their own section.
forecast_score_overview_table = forecast_score_overview(forecast_scores_df)
forecast_score_by_horizon_table = forecast_score_by_horizon(forecast_scores_df)
forecast_score_by_release_table = forecast_score_by_release(forecast_scores_df)

joint_score_overview_table = select_fit_role(
    forecast_score_overview_table, "joint"
)
joint_score_by_horizon_table = select_fit_role(
    forecast_score_by_horizon_table, "joint"
)
# The trailing `;` on this last assignment matters: without it, this whole
# setup chunk's last statement (the DataFrame it assigns) is Literate's
# implicitly displayed "result" for the chunk, on top of the deliberate
# display further down -- and a bare DataFrame is html-showable, so it
# goes out as a second, undisplayed-in-source `@raw html` block that (for
# a table this size) can itself hit the PCRE limit described above.
joint_score_by_release_table = select_fit_role(
    forecast_score_by_release_table, "joint"
);
streamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed casesjoint172648.665.070.2092.030.650.61520.42127.410.820.550.870.42
confirmed deathsjoint170558.856.280.3282.320.820.86470.2537.6750.930.390.880.1
isolation bedsjoint16296.531.680.111.550.880.7457.7428.1110.680.670.970.13
onset reportsjoint31137.120.980.2230.90.690.6765.1215.7356.260.550.84-0.1
recoveredjoint7446.311.440.1241.0932.3511.852.110.680.930.27
Scores pooled over every horizon and release, one row per stream: the mean CRPS and its decomposition, coverage, bias, and relative skill against the persistence baseline on the natural and log scales. Each row also carries relative skill against the stream's own individual fit where one exists.
julia
forecast_relative_skill_fig = plot_forecast_relative_skill(
    forecast_score_by_horizon_table
);

Relative skill against the baseline by horizon, one panel per stream and one series per fit role, on a log scale with the reference line at one.
julia
forecast_crps_by_horizon_fig = plot_forecast_crps_by_horizon(
    forecast_score_by_horizon_table
);

The mean CRPS by horizon split into its width, overprediction and underprediction, one stacked bar per horizon and fit role.
Scores by horizon
streamhorizonfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases7joint4997.912.590.22.530.510.5253.1844.180.540.530.880.45
confirmed cases14joint45219.782.680.2072.410.580.59120.0699.140.580.560.840.4
confirmed cases21joint41403.912.540.2071.810.640.65237.83165.230.850.560.850.41
confirmed cases28joint372170.838.060.2231.580.740.731939.26230.11.470.570.920.4
confirmed deaths7joint4954.571.980.3343.060.880.8920.9811.222.390.450.840.1
confirmed deaths14joint44125.792.080.3142.50.780.8549.629.7446.450.360.860.14
confirmed deaths21joint40231.672.120.3252.140.910.9115.3549.3966.930.350.880.1
confirmed deaths28joint372095.3911.530.341.780.780.821949.1569.5176.730.410.950.08
isolation beds7joint4570.881.730.0821.630.710.5836.3826.118.40.580.960.18
isolation beds14joint4195.331.840.1071.680.880.7354.7629.211.380.590.980.13
isolation beds21joint39104.511.650.1171.50.930.7866.528.049.960.770.970.11
isolation beds28joint37120.621.560.1391.450.980.8577.7829.3913.450.760.970.07
onset reports7joint1591.630.70.2620.740.80.741.7815.8134.030.60.80.01
onset reports14joint10170.991.430.2081.450.680.6969.2112.689.180.50.8-0.24
onset reports21joint6194.410.980.150.950.580.58116.6720.7756.980.51-0.15
recovered7joint2414.890.960.1140.988.835.041.010.750.880.19
recovered14joint2035.591.510.1281.3820.812.382.420.60.90.36
recovered21joint1783.382.330.1341.2964.7416.572.060.6510.3
recovered28joint1372.311.010.1220.7851.1717.433.710.6910.26
julia
forecast_skill_by_cutoff_fig = plot_forecast_skill_by_cutoff(
    forecast_score_by_release_table;
    title = "Relative skill against the baseline, by release"
);

Relative skill against the baseline release by release, so a run of releases that lost to the baseline shows as a run.
Scores by release
made_datestreamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
2026-06-07confirmed casesjoint2363.9113.210.91310.98341.0622.860110.17
2026-06-10confirmed casesjoint3534.876.560.7914.88435.6799.20110.27
2026-07-01confirmed casesjoint415862.78750.5832.8515377.31485.470010.66
2026-07-06confirmed casesjoint4421.281.320.2690.96311.85109.4300.510.49
2026-07-08confirmed casesjoint4390.981.210.2530.94239.5151.4700.2510.52
2026-07-23confirmed casesjoint4335.570.880.1690.681.020.82299.2735.680.62110.13
2026-07-25confirmed casesjoint4222.50.950.1340.940.640.63218.044.460110.07
2026-07-26confirmed casesjoint4235.691.020.1411.010.790.67229.655.680.36110.06
2026-07-27confirmed casesjoint4247.971.230.1421.050.930.83240.087.880110.11
2026-07-31confirmed casesjoint4259.641.970.1491.810.640.68235.7323.760.15110.13
2026-08-01confirmed casesjoint4231.261.920.1391.910.690.66226.844.310.11110.05
2026-08-02confirmed casesjoint4278.452.110.1451.750.940.85263.7714.680110.14
2026-08-03confirmed casesjoint4247.581.20.140.910.850.8237.889.70110.08
2026-08-04confirmed casesjoint4221.562.290.1332.260.630.69220.70.860110.03
2026-08-07confirmed casesjoint4213.542.460.142.110.690.73209.7603.7811-0.09
2026-08-11confirmed casesjoint4197.494.050.1353.60.590.71191.5305.9711-0.13
2026-08-15confirmed casesjoint4387.777.020.1934.171.240.91294.0293.750110.37
2026-08-17confirmed casesjoint4458.716.430.2223.511.530.97290.38168.3300.510.46
2026-08-22confirmed casesjoint4336.094.930.1783.5210.87228.94107.1500.7510.45
2026-08-24confirmed casesjoint4246.113.710.1443.130.780.77118.34127.770010.62
2026-08-25confirmed casesjoint4198.5730.122.690.660.65108.4890.10010.56
2026-08-30confirmed casesjoint4155.681.860.0891.480.40.4499.6855.9900.7510.38
2026-08-31confirmed casesjoint4143.981.480.0851.320.480.48106.0937.890110.29
2026-09-01confirmed casesjoint4113.161.270.071.120.320.3697.2115.950110.17
2026-09-06confirmed casesjoint4236.661.950.1411.880.670.65116.22120.4400.2510.6
2026-09-10confirmed casesjoint4313.881.610.2051.50.70.78106.15207.730010.79
2026-09-12confirmed casesjoint4711.142.770.4332.140.540.58165.71545.420000.96
2026-09-13confirmed casesjoint4720.073.430.4262.620.60.61207.2512.870000.93
2026-09-15confirmed casesjoint4666.042.970.4312.180.640.66197.5468.54000.250.92
2026-09-16confirmed casesjoint4648.532.930.4192.220.680.68195.04453.49000.250.93
2026-09-19confirmed casesjoint4942.364.840.5853.611.451.21215.69726.670000.97
2026-09-21confirmed casesjoint4110.91.140.1051.380.230.2966.6644.2400.510.53
2026-09-22confirmed casesjoint4135.091.760.122.10.380.4271.2963.800.2510.61
2026-09-25confirmed casesjoint458.871.020.0490.820.160.1856.572.30110.15
2026-09-26confirmed casesjoint496.552.050.0972.280.220.2877.5618.9900.7510.32
2026-09-27confirmed casesjoint484.841.660.0731.10.340.460.38024.4611-0.36
2026-10-02confirmed casesjoint491.681.680.0671.250.520.4874.1417.550110.33
2026-10-06confirmed casesjoint4212.44.430.1423.371.030.9396.62115.780010.65
2026-10-10confirmed casesjoint4205.592.620.1442.111.050.9990.66114.930010.67
2026-10-20confirmed casesjoint3304.519.790.257.720.830.8269.93234.58000.330.92
2026-10-23confirmed casesjoint399.372.460.0791.450.240.2553.8345.5500.3310.43
2026-10-25confirmed casesjoint3127.993.070.1152.610.250.2771.7856.220010.54
2026-10-30confirmed casesjoint2249.144.810.2913.780.740.7645.09204.050000.96
2026-10-31confirmed casesjoint2103.912.140.1321.980.320.3546.0857.830010.67
2026-11-01confirmed casesjoint253.321.640.071.590.160.1942.211.120110.34
2026-11-04confirmed casesjoint189.44.760.174.170.430.4725.1364.270010.84
2026-11-06confirmed casesjoint195.453.890.1873.510.340.3826.0869.370010.83
2026-11-07confirmed casesjoint143.662.380.0872.10.160.1923.5820.070010.54
2026-11-09confirmed casesjoint135.721.140.071.110.110.1425.3910.340110.41
2026-06-07confirmed deathsjoint141.491.660.9032.2236.9404.5511-0.2
2026-06-10confirmed deathsjoint264.932.270.7353.162.70.991.2411-0.03
2026-07-01confirmed deathsjoint41719681.20.5491.2317103.1192.890110.28
2026-07-06confirmed deathsjoint4197.20.90.8712.3110.35086.850.251-0.49
2026-07-08confirmed deathsjoint4177.590.790.7131.8484.29093.30.251-0.61
2026-07-23confirmed deathsjoint4271.491.710.4842.50.391.08172.61098.8811-0.45
2026-07-25confirmed deathsjoint4241.832.040.5043.461.022.15106.460135.3701-0.57
2026-07-26confirmed deathsjoint4268.462.430.5544.451.052.17101.590166.8701-0.64
2026-07-27confirmed deathsjoint4249.472.180.5343.780.82.12109.30140.1701-0.59
2026-07-31confirmed deathsjoint4274.552.320.5640.881.94108.790165.760.251-0.58
2026-08-01confirmed deathsjoint4313.482.460.6834.281.122.794.480219.0101-0.68
2026-08-02confirmed deathsjoint4258.172.780.5074.41.011.83128.190129.980.251-0.53
2026-08-03confirmed deathsjoint4268.891.920.582.851.22.37123.560145.330.251-0.55
2026-08-04confirmed deathsjoint4298.064.150.6567.751.22.3293.540204.5201-0.67
2026-08-07confirmed deathsjoint4366.295.260.888.711.582.7176.060290.2300.75-0.79
2026-08-11confirmed deathsjoint4301.1714.310.79825.571.232.3796.550204.6101-0.7
2026-08-15confirmed deathsjoint4205.935.290.325.50.840.94180.35025.5811-0.26
2026-08-17confirmed deathsjoint4202.493.640.3043.620.811.02187.02015.4711-0.21
2026-08-22confirmed deathsjoint4178.981.950.3052.450.740.63156.16022.8211-0.26
2026-08-24confirmed deathsjoint492.941.330.1051.220.420.1945.4247.5100.2510.57
2026-08-25confirmed deathsjoint4741.030.0840.990.30.1145.0828.9200.7510.41
2026-08-30confirmed deathsjoint475.880.880.0920.940.290.138.0537.8300.510.48
2026-08-31confirmed deathsjoint486.270.780.1110.850.350.1441.9844.290010.6
2026-09-01confirmed deathsjoint474.640.760.0950.830.30.143.0231.6100.2510.53
2026-09-06confirmed deathsjoint4126.851.350.1561.390.450.1343.5883.2800.2510.68
2026-09-10confirmed deathsjoint4173.521.250.2461.190.840.3343.19130.34000.50.89
2026-09-12confirmed deathsjoint4273.481.980.3581.632.712.3865.52207.960000.95
2026-09-13confirmed deathsjoint4264.392.210.3421.762.812.3680.9183.49000.250.91
2026-09-15confirmed deathsjoint4237.22.070.3361.733.83.1169.24167.96000.250.91
2026-09-16confirmed deathsjoint4222.672.230.321.863.883.1370.49152.18000.50.91
2026-09-19confirmed deathsjoint4292.383.230.4012.665.394.2271.62220.760000.95
2026-09-21confirmed deathsjoint467.051.180.1361.511.521.722.1944.85000.750.8
2026-09-22confirmed deathsjoint454.741.360.1081.730.80.8924.9329.8100.2510.66
2026-09-25confirmed deathsjoint425.480.860.0430.580.190.1723.750.920.82110.09
2026-09-26confirmed deathsjoint440.811.580.1052.050.50.7521.73190.070.50.750.47
2026-09-27confirmed deathsjoint434.131.440.0551.020.350.3220.610.5612.970.751-0.16
2026-10-02confirmed deathsjoint422.790.80.0390.720.240.2219.550.352.8911-0.07
2026-10-06confirmed deathsjoint428.210.60.0430.550.290.2427.760.020.4311-0.03
2026-10-10confirmed deathsjoint448.940.860.070.740.650.5625.7823.1600.510.5
2026-10-20confirmed deathsjoint326.111.20.0450.920.840.7717.318.250.560.6710.19
2026-10-23confirmed deathsjoint318.10.880.0420.780.370.4715.812.270.020.6710.19
2026-10-25confirmed deathsjoint328.071.030.0540.810.390.4416.8811.1900.6710.44
2026-10-30confirmed deathsjoint258.140.210.120.220.060.1118.8139.3400.510.65
2026-10-31confirmed deathsjoint231.570.850.090.890.70.7512.3319.240010.7
2026-11-01confirmed deathsjoint220.470.970.0561.060.760.812.138.3400.510.51
2026-11-04confirmed deathsjoint115.091.80.0581.651.061.0310.214.880110.43
2026-11-06confirmed deathsjoint117.420.70.0690.711.31.299.318.110010.58
2026-11-07confirmed deathsjoint18.650.570.0350.520.690.668.640011-0.01
2026-11-09confirmed deathsjoint19.470.710.040.780.770.768.5200.9511-0.18
2026-07-01isolation bedsjoint490.191.460.1381.513.48076.7100-0.99
2026-07-06isolation bedsjoint442.870.680.0660.721.07021.801-0.64
2026-07-08isolation bedsjoint454.751.490.0731.511.6743.080010.8
2026-07-23isolation bedsjoint480.062.760.1012.641.441.1429.3950.670010.58
2026-07-25isolation bedsjoint480.842.290.1052.241.561.2832.6548.1900.510.53
2026-07-26isolation bedsjoint473.831.80.0971.741.20.9636.9636.8700.510.46
2026-07-27isolation bedsjoint471.582.40.0932.31.20.9539.5232.0600.510.44
2026-07-31isolation bedsjoint471.051.730.0881.690.860.6751.6919.3600.7510.34
2026-08-01isolation bedsjoint4721.030.0940.980.810.6164.517.480.02110.18
2026-08-02isolation bedsjoint483.691.680.1021.510.990.7252.6731.0200.7510.39
2026-08-03isolation bedsjoint478.831.860.0991.730.990.7460.4218.410110.3
2026-08-04isolation bedsjoint4100.741.50.1371.441.160.9863.4737.2800.7510.38
2026-08-07isolation bedsjoint483.811.310.1221.450.80.7169.03014.7811-0.24
2026-08-11isolation bedsjoint474.710.430.1090.440.730.6472.4802.2311-0.08
2026-08-15isolation bedsjoint480.960.860.0920.770.630.4571.729.240110.17
2026-08-17isolation bedsjoint495.081.310.1061.140.920.6667.3527.740110.33
2026-08-22isolation bedsjoint493.151.680.0981.450.640.4578.9814.180110.23
2026-08-24isolation bedsjoint490.981.840.0961.590.720.5366.324.6700.7510.34
2026-08-25isolation bedsjoint475.541.180.0821.040.570.4371.773.740.02110.08
2026-08-30isolation bedsjoint480.951.910.0891.710.630.5262.6118.3300.7510.3
2026-08-31isolation bedsjoint468.551.470.0791.430.570.4962.975.180.4110.14
2026-09-01isolation bedsjoint481.21.950.1032.140.670.6769.01012.1911-0.2
2026-09-06isolation bedsjoint475.081.540.0891.50.60.5768.210.456.4311-0.12
2026-09-10isolation bedsjoint475.831.40.0851.320.590.5470.160.215.4611-0.1
2026-09-12isolation bedsjoint4177.13.070.1742.691.320.9661.26115.8400.2510.6
2026-09-13isolation bedsjoint4186.883.480.1853.031.541.1870.89115.990010.6
2026-09-15isolation bedsjoint4141.542.550.1372.311.070.8672.6368.9100.510.48
2026-09-16isolation bedsjoint4149.483.220.1432.791.150.8981.0668.4300.510.48
2026-09-19isolation bedsjoint4270.825.310.2574.372.151.7248.64222.18000.750.77
2026-09-21isolation bedsjoint473.081.220.0861.250.540.5657.770.115.2211-0.2
2026-09-22isolation bedsjoint486.571.280.0961.320.470.456.8029.770.751-0.33
2026-09-25isolation bedsjoint4102.91.560.1251.640.740.8150.22052.680.51-0.56
2026-09-26isolation bedsjoint4112.731.190.141.230.880.9259.41053.320.751-0.51
2026-09-27isolation bedsjoint4171.952.280.2232.511.161.1750.630121.3201-0.75
2026-10-02isolation bedsjoint477.531.010.0850.980.540.566.89010.6411-0.2
2026-10-06isolation bedsjoint4122.961.770.1111.620.850.6686.1336.8300.7510.4
2026-10-10isolation bedsjoint481.961.770.0771.570.590.5176.215.750110.15
2026-10-20isolation bedsjoint3154.533.010.1382.731.170.9865.6388.9100.3310.64
2026-10-23isolation bedsjoint367.431.750.0651.720.530.5957.88.910.730.6710.19
2026-10-25isolation bedsjoint265.51.990.0662.010.660.7363.811.570.12110.05
2026-10-30isolation bedsjoint253.111.580.0521.550.850.7950.662.450110.13
2026-10-31isolation bedsjoint148.941.570.0511.590.990.9644.804.1511-0.17
2026-11-01isolation bedsjoint160.82.230.0682.391.141.1537.31023.4911-0.45
2026-11-04isolation bedsjoint146.581.260.0481.321.141.1443.43.170110.19
2026-11-06isolation bedsjoint150.491.620.0521.710.890.942.2608.2211-0.26
2026-08-02onset reportsjoint380.540.570.10.570.760.5977.9102.6311-0.12
2026-08-03onset reportsjoint387.721.440.1191.060.890.7776.4911.230110.22
2026-08-04onset reportsjoint384.340.660.1310.610.610.574.122.327.9111-0.01
2026-08-07onset reportsjoint3277.730.810.3870.790.670.59690208.7300.67-0.83
2026-08-11onset reportsjoint3194.711.220.2871.270.80.6783.450111.260.331-0.6
2026-08-15onset reportsjoint3171.731.910.1571.330.460.56107.0564.6800.3310.43
2026-08-17onset reportsjoint2103.331.660.1341.20.420.5378.8324.50110.28
2026-08-22onset reportsjoint267.680.580.1330.770.560.6367.4400.2411-0.04
2026-08-24onset reportsjoint164.90.930.3061.630.760.7857.507.4111-0.21
2026-08-25onset reportsjoint163.70.250.3150.50.670.6261.192.510110.13
2026-09-21onset reportsjoint145.40.670.1520.490.630.6511.7133.690010.83
2026-09-22onset reportsjoint143.050.740.1440.420.590.6113.1629.890010.82
2026-10-02onset reportsjoint2134.61.730.2251.291.011.0620.230114.3700.5-0.78
2026-10-06onset reportsjoint2288.341.780.4791.860.820.8130.350257.9900-0.96
2026-10-10onset reportsjoint1155.540.540.5020.681.261.217.52138.010000.98
2026-07-01recoveredjoint3312.696.710.4461.95289.5523.140110.27
2026-07-06recoveredjoint485.760.80.4070.8668.39017.3711-0.33
2026-07-08recoveredjoint463.080.750.292144.51018.5711-0.38
2026-09-15recoveredjoint457.313.920.0942.3740.2817.0300.510.35
2026-09-16recoveredjoint453.242.660.0881.5840.8712.370110.26
2026-09-19recoveredjoint484.684.340.142.6542.242.160.320.2510.43
2026-09-21recoveredjoint413.860.680.0360.7513.390.470110.1
2026-09-22recoveredjoint413.150.780.0350.8312.660.460.04110.04
2026-09-25recoveredjoint414.960.530.0420.6812.970.981.0111-0.03
2026-09-26recoveredjoint422.440.50.0670.5212.459.9900.510.55
2026-09-27recoveredjoint414.670.330.0450.3912.821.860110.2
2026-10-02recoveredjoint424.230.520.080.612.0712.1700.2510.56
2026-10-06recoveredjoint416.860.870.0450.7813.122.491.2611-0.05
2026-10-10recoveredjoint426.211.240.0580.8813.812.010.410.7510.27
2026-10-20recoveredjoint331.591.80.0871.4710.3921.200.3310.62
2026-10-23recoveredjoint319.651.270.0580.849.510.1500.3310.45
2026-10-25recoveredjoint328.651.940.0881.539.4219.2300.3310.64
2026-10-30recoveredjoint252.394.210.223.278.5943.80000.94
2026-10-31recoveredjoint242.261.990.2081.798.8633.40000.93
2026-11-01recoveredjoint229.241.890.1541.658.7420.51000.50.83
2026-11-04recoveredjoint16.70.310.0420.266.680.020110.04
2026-11-06recoveredjoint116.352.130.1091.876.3310.020010.68
2026-11-07recoveredjoint19.110.630.0590.516.692.420110.35
2026-11-09recoveredjoint17.050.910.0460.917.0500110
Forecasts-versus-now overlay
julia
forecast_overlay_fig = plot_forecast_overlay(
    scored_overlay(forecast_overlay_df)
);

Forecasts made at each release against the value observed since (black), one panel per stream and horizon by the date each forecast was made, with the median and 90% interval coloured by fit role. Each panel's axis is cropped to a small multiple of what the stream reached, and an interval or median past it is clamped and marked with an open triangle.

Frozen-fit forecast evaluation ​

The current model, frozen at earlier data cut-offs (see forecast-versus-frozen evaluation), is scored as the cross-release forecasts above, against the same baseline. The tables in this section are the frozen joint model's, one row per stream. Each still-reported stream's own frozen fit is scored at the one-week-back cut-off, and confirmed cases and confirmed deaths also at each May cut-off. These single-stream fits appear in the skill figures. The May cut-offs predate the first reported bed occupancy and recoveries, so those windows are left unscored. The baseline snapshot was taken weeks after the frozen cut-off, so it can hold later revisions to earlier days (see forecast scoring against a persistence baseline).

Load and summarise the frozen-fit forecast scores
julia
frozen_scores_df = _release_data(
    "forecast_scores_frozen.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, crps = Float64,
        log_crps = Float64, dispersion = Float64, overprediction = Float64,
        underprediction = Float64, coverage_50 = Float64,
        coverage_90 = Float64,
        bias = Float64, n_samples = Int,
        log_rel_to_baseline = Float64,
    )
)
frozen_overlay_df = _release_data(
    "forecast_overlay_frozen.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, observed = Float64,
        median = Float64, lo30 = Float64, hi30 = Float64, lo60 = Float64,
        hi60 = Float64, lo90 = Float64, hi90 = Float64,
    )
)
# Rows a superseded forecaster produced are dropped before anything is
# summarised or drawn, from the scores and the overlay alike, so the
# tables and the figures rest on one set of rows. See
# `drop_superseded_forecasts` for the one exclusion in force and why.
frozen_scores_df = _drop_rescanned(
    drop_superseded_forecasts(frozen_scores_df)
)
frozen_overlay_df = _drop_rescanned(
    drop_superseded_forecasts(frozen_overlay_df)
)

# The frozen joint carries `FROZEN_FIT`, so it is named as the joint role
# here and compared against each stream's own frozen fit. A release
# published before the archive had a `fit` column carries joint rows only.
frozen_score_overview_table = forecast_score_overview(
    frozen_scores_df; joint_fit = FROZEN_FIT
)
frozen_score_by_horizon_table = forecast_score_by_horizon(
    frozen_scores_df; joint_fit = FROZEN_FIT
)
frozen_score_by_release_table = forecast_score_by_release(
    frozen_scores_df; joint_fit = FROZEN_FIT
)

# The tables show the frozen joint alone, as the cross-release tables show
# the joint alone, so a row reads as one model at one cut-off rather than a
# stream interleaving two fits. The archive's single-stream frozen fits stay
# in the scored data and in the figures, which compare the roles against
# each other. Selecting the joint role leaves `fit` single-valued, so it is
# dropped and the model named in the prose instead.
_frozen_joint_only(tbl) = drop_degenerate_fit_column(
    select_fit_role(tbl, "joint")
)
frozen_score_overview_display = _frozen_joint_only(
    frozen_score_overview_table
)
frozen_score_by_horizon_display = _frozen_joint_only(
    frozen_score_by_horizon_table
)
frozen_score_by_release_display = _frozen_joint_only(
    frozen_score_by_release_table
)

# One row per release for the cut-offs more than one release forecast.
# See the comment above `joint_score_by_release_table`'s assignment for why
# this setup chunk's last statement needs a trailing `;`.
frozen_score_by_vintage_table = forecast_score_by_vintage(
    frozen_scores_df; joint_fit = FROZEN_FIT
)
frozen_score_by_vintage_display = _frozen_joint_only(
    frozen_score_by_vintage_table
);
streamncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases397272.82.380.3411.130.540.98126.54129.4716.790.320.790.3
confirmed deaths34154.470.940.2950.521.371.2427.0723.583.820.580.880.05
isolation beds344101.780.890.1450.610.730.6358.8529.4613.470.550.990.15
onset reports32147.41.060.2430.8766.818.5462.050.560.88-0.19
recovered167365.738.520.6094.95125.44238.51.790.340.610.6
julia
frozen_relative_skill_fig = plot_forecast_relative_skill(
    frozen_score_by_horizon_table
);

Relative skill against the baseline by horizon for the frozen cut-offs.
julia
frozen_crps_by_horizon_fig = plot_forecast_crps_by_horizon(
    frozen_score_by_horizon_table;
    title = "CRPS decomposition by horizon, frozen cut-offs"
);

The mean CRPS by horizon for the frozen cut-offs, split as in the cross-release section above.
Scores by horizon
streamhorizonncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases718487.431.050.3850.760.641.1230.2420.9736.230.350.72-0.09
confirmed cases1490234.133.390.263.020.621.06113.79120.3400.320.890.61
confirmed cases2186527.733.620.3722.490.540.9232.04295.6900.190.790.69
confirmed cases2837696.172.230.2461.550.480.83391.28304.8900.490.920.55
confirmed deaths717017.30.70.4110.521.21.176.982.877.450.540.86-0.33
confirmed deaths147646.861.090.1570.711.291.2926.84200.030.790.920.35
confirmed deaths2172102.680.810.2050.381.321.2458.8843.720.090.560.920.43
confirmed deaths2823203.381.420.1781.271.491.376.75125.371.260.30.740.6
isolation beds790531.160.0850.830.580.523018.754.240.690.940.24
isolation beds148886.091.640.141.20.740.6747.5235.752.810.5910.34
isolation beds2184107.830.660.1530.450.780.6766.1824.2217.430.5410.01
isolation beds2882165.960.810.2080.510.830.6895.1639.8330.970.3710
onset reports717104.110.740.2790.746.6712.1645.280.590.88-0.19
onset reports1411155.661.490.191.4576.1415.9363.60.640.82-0.15
onset reports214308.651.370.2381.33126.7152.85129.090.251-0.27
recovered746129.626.720.5744.3339.8388.711.080.30.870.58
recovered1444279.111.990.6037.3880.8196.831.470.320.570.62
recovered2140448.878.720.6265.05142.62304.242.020.350.50.6
recovered2837672.427.780.6414.02266.4403.192.830.380.460.59
julia
frozen_skill_by_cutoff_fig = plot_forecast_skill_by_cutoff(
    frozen_score_by_release_table;
    xlabel = "Frozen cut-off",
    title = "Relative skill against the baseline, by frozen cut-off"
);

Relative skill against the baseline cut-off by cut-off, pooled over the horizons each cut-off forecast.
Scores by frozen cut-off
made_datestreamncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
2026-05-23confirmed cases4631.710.380.2120.351.081.116.86014.850.461-0.45
2026-05-27confirmed cases46141.250.790.9210.740.461.5811.450129.7900.07-0.96
2026-06-08confirmed cases138307.686.160.3533.65133.22174.4700.250.880.66
2026-07-16confirmed cases4294.680.460.1680.29283.4610.980.24110.07
2026-07-18confirmed cases4361.260.780.1850.51333.8527.410110.15
2026-07-19confirmed cases4439.621.070.2090.7351.8387.790110.3
2026-07-20confirmed cases4385.620.860.1850.56332.653.020110.22
2026-07-24confirmed cases4422.921.290.1920.91343.779.220110.32
2026-07-25confirmed cases4460.722.020.2091.5356.82103.910110.34
2026-07-26confirmed cases4315.561.390.1641.2284.5930.970110.24
2026-07-27confirmed cases4346.391.690.1761.3306.9739.420110.27
2026-07-28confirmed cases4396.781.360.1560.85361.5235.260110.22
2026-07-31confirmed cases4498.384.060.2132.72369.58128.800.7510.38
2026-08-04confirmed cases4281.573.010.1562.79253.9927.590110.18
2026-08-08confirmed cases4253.513.20.152.33246.676.120.72110.03
2026-08-10confirmed cases4244.513.730.152.792411.71.8111-0.01
2026-08-15confirmed cases4623.411.380.2815.95384.1239.300.2510.52
2026-08-17confirmed cases4426.555.70.2343.62166.54260.010010.75
2026-08-18confirmed cases4468.253.960.2762.82155.86312.4000.750.85
2026-08-23confirmed cases4562.748.630.3146.31140.78421.96000.750.88
2026-08-24confirmed cases4437.047.760.2466.22160.51276.53000.750.82
2026-08-25confirmed cases4288.324.440.1764.11113.18175.140010.74
2026-08-30confirmed cases4179.542.250.1061.72110.0569.4900.7510.44
2026-09-03confirmed cases4279.852.480.1612.2999.17180.680010.71
2026-09-05confirmed cases4133.121.020.0841.05104.0628.720.34110.19
2026-09-06confirmed cases4560.624.660.2793.72181.89378.730010.74
2026-09-08confirmed cases4439.232.950.2412.460.380.39185.89253.34000.750.76
2026-09-09confirmed cases4610.513.390.3312.620.520.52205.84404.67000.50.85
2026-09-12confirmed cases4810.133.220.4842.440.610.64191.69618.440000.96
2026-09-14confirmed cases4591.662.340.4171.910.520.6111.41480.250000.97
2026-09-15confirmed cases4641.052.920.452.320.620.68126.77514.280000.97
2026-09-18confirmed cases4433.842.280.3532.030.490.5892.05341.79000.250.95
2026-09-19confirmed cases4410.132.180.3252.040.630.67117.34292.79000.50.88
2026-09-20confirmed cases4175.251.170.1631.280.310.3886.6788.5800.250.750.67
2026-09-25confirmed cases4181.573.20.1372.280.460.4883.7497.830010.67
2026-09-29confirmed cases4155.472.130.0991.151.090.94104.2951.180110.41
2026-10-03confirmed cases4141.030.930.0890.5310.9109.5131.520110.32
2026-10-13confirmed cases4218.035.740.1434.380.940.8796.87121.160010.68
2026-10-16confirmed cases4214.855.180.1544.070.650.6678.48136.370010.78
2026-10-18confirmed cases4227.416.590.1464.570.630.61104.28123.130010.62
2026-10-23confirmed cases3152.854.080.1192.440.410.4162.390.5500.3310.66
2026-10-24confirmed cases3115.433.080.0982.650.260.2867.4148.0200.3310.49
2026-10-25confirmed cases3138.113.220.1252.710.260.369.8468.260010.58
2026-10-28confirmed cases2113.12.870.1392.560.320.3541.471.690010.73
2026-10-30confirmed cases2150.043.070.1822.610.430.4641.35108.690010.84
2026-10-31confirmed cases2104.922.170.1342.060.330.3648.8556.070010.63
2026-11-02confirmed cases260.250.930.0710.590.220.2449.25110110.28
2026-11-07confirmed cases1134.488.110.2336.320.490.5153.0581.440010.71
2026-11-09confirmed cases131.981.020.0651.030.10.1327.314.660110.23
2026-05-23confirmed deaths467.260.290.3030.211.041.053.5703.690.591-0.46
2026-05-27confirmed deaths4624.70.760.9760.811.371.432.39022.3100.61-0.9
2026-06-08confirmed deaths13846.180.680.1950.4436.389.450.350.8810.17
2026-08-17confirmed deaths491.871.650.0971.1659.332.5700.510.44
2026-08-18confirmed deaths4115.91.320.1341.0455.9159.990010.63
2026-08-23confirmed deaths4145.191.690.1661.5151.7193.480010.72
2026-08-24confirmed deaths4110.911.60.121.3955.255.7100.2510.53
2026-08-25confirmed deaths487.421.240.11.244.6542.7700.2510.53
2026-08-30confirmed deaths469.430.850.0870.9339.7529.6800.510.5
2026-09-03confirmed deaths494.771.140.1121.141.453.3700.2510.55
2026-09-05confirmed deaths453.980.590.0740.6640.7712.260.95110.17
2026-09-06confirmed deaths4122.51.370.1421.3445.3276.870.310.510.46
2026-09-08confirmed deaths4165.381.60.1931.432.11.8267.2198.1600.2510.73
2026-09-09confirmed deaths4216.822.150.2521.813.12.5278.5138.310010.79
2026-09-12confirmed deaths4265.951.90.3421.552.482.1368.8197.150000.94
2026-09-14confirmed deaths4216.781.810.3171.653.162.828.06188.720000.99
2026-09-15confirmed deaths4206.511.780.3231.693.793.2727.27179.240000.99
2026-09-18confirmed deaths4157.261.460.2891.513.452.9923.78133.480000.98
2026-09-19confirmed deaths4162.391.730.281.834.13.826.75135.640000.97
2026-09-20confirmed deaths4118.961.670.2191.883.353.1921.7197.250000.96
2026-09-25confirmed deaths436.771.170.0640.810.330.321.5415.2300.2510.51
2026-09-29confirmed deaths428.061.040.0490.850.230.2221.891.454.72110.01
2026-10-03confirmed deaths427.780.350.0440.260.240.2123.350.014.4211-0.2
2026-10-13confirmed deaths432.580.690.0440.590.560.4326.885.70110.21
2026-10-16confirmed deaths434.380.710.0480.540.60.4723.6810.440.26110.23
2026-10-18confirmed deaths433.650.650.0510.50.60.4724.287.212.160.7510.05
2026-10-23confirmed deaths319.210.870.0440.740.420.5216.083.1300.6710.23
2026-10-24confirmed deaths317.940.960.0370.960.360.4515.691.690.57110.06
2026-10-25confirmed deaths324.940.980.0490.780.280.3316.718.220110.38
2026-10-28confirmed deaths214.590.730.0390.70.410.4511.792.80110.32
2026-10-30confirmed deaths214.070.570.0370.450.550.6311.731.191.1511-0.02
2026-10-31confirmed deaths226.370.730.0770.780.560.6212.613.770010.6
2026-11-02confirmed deaths214.210.440.0390.320.640.6913.6800.5311-0.1
2026-11-07confirmed deaths115.560.950.0590.811.251.1112.972.590110.3
2026-11-09confirmed deaths19.840.810.0420.890.80.87.3302.5111-0.34
2026-06-08isolation beds184105.060.630.1680.4460.4225.2919.350.4910.09
2026-07-16isolation beds473.222.370.0962.2320.0153.220010.6
2026-07-18isolation beds477.782.550.1032.4418.7159.070010.61
2026-07-19isolation beds4100.013.690.1323.5622.3877.630010.64
2026-07-20isolation beds482.232.860.1062.6719.8962.3300.2510.61
2026-07-24isolation beds476.612.580.0962.4310.1466.470010.73
2026-07-25isolation beds4106.432.660.1362.5613.8492.59000.750.73
2026-07-26isolation beds494.882.390.1222.2622.6172.2700.250.750.63
2026-07-27isolation beds485.562.70.1132.6238.9246.6500.510.48
2026-07-28isolation beds4153.082.930.2012.6141.15111.9400.50.750.67
2026-07-31isolation beds4119.062.730.1392.541.3677.700.50.750.57
2026-08-04isolation beds4110.511.620.1531.5663.2647.2400.750.750.42
2026-08-08isolation beds459.170.940.0780.9651.547.150.480.7510.16
2026-08-10isolation beds455.991.040.0791.0855.830.020.15110
2026-08-15isolation beds4109.361.270.1151.0674.0735.30110.34
2026-08-17isolation beds4103.62.160.1111.8672.6230.9800.7510.43
2026-08-18isolation beds493.51.220.11.0271.3122.1900.7510.34
2026-08-23isolation beds4145.693.530.1493.0360.6785.020010.66
2026-08-24isolation beds4108.372.090.1131.7668.4739.900.510.46
2026-08-25isolation beds471.11.080.0871.0760.24010.8611-0.27
2026-08-30isolation beds471.871.70.0941.8362.2509.6211-0.19
2026-09-03isolation beds476.350.880.0960.8653.15023.20.751-0.38
2026-09-05isolation beds486.591.830.112267.81018.780.751-0.31
2026-09-06isolation beds4243.915.360.5229.48146.37097.550.751-0.44
2026-09-08isolation beds4106.031.470.1081.310.70.5391.1214.140.76110.12
2026-09-09isolation beds4136.552.290.1321.860.950.6681.8654.6900.7510.37
2026-09-12isolation beds4198.783.440.1963.021.51.1649.76149.020010.7
2026-09-14isolation beds4164.322.730.1592.381.251.0175.6888.640010.7
2026-09-15isolation beds4115.222.180.111.930.850.6875.234000.510.43
2026-09-18isolation beds474.761.650.0781.530.550.4762.6712.0900.7510.27
2026-09-19isolation beds484.481.510.0891.370.620.5671.2613.20.020.7510.24
2026-09-20isolation beds478.161.520.0971.570.550.5759.1701911-0.32
2026-09-25isolation beds471.581.080.0821.070.510.558.27013.3111-0.22
2026-09-29isolation beds4101.051.030.111.020.630.5669.27031.780.751-0.33
2026-10-03isolation beds482.311.150.0871.060.570.4979.502.8111-0.09
2026-10-13isolation beds494.941.330.0981.280.640.5773.50.8820.570.751-0.25
2026-10-16isolation beds468.011.030.0711.010.430.3755.93012.080.751-0.29
2026-10-18isolation beds359.691.370.0621.340.830.7958.4801.2111-0.07
2026-10-23isolation beds371.231.930.0691.880.510.5759.2610.761.20.6710.16
2026-10-24isolation beds259.131.870.0591.870.680.7356.612.280.24110.08
2026-10-25isolation beds256.481.640.0581.690.50.5655.150.860.48110.02
2026-10-28isolation beds257.271.330.0581.40.90.8354.302.9711-0.15
2026-10-30isolation beds2531.510.0521.490.860.8149.840.992.17110
2026-10-31isolation beds157.121.930.0591.971.141.1251.8405.2811-0.21
2026-11-02isolation beds154.931.990.0592.111.241.2351.1303.811-0.14
2026-07-26onset reports267.832.340.1192.1265.222.610110.13
2026-07-27onset reports268.121.070.1210.8656.661.4610.0111-0.15
2026-07-28onset reports2790.490.1290.3761.396.9410.6711-0.07
2026-07-31onset reports2117.290.690.2390.675.7241.5800.510.4
2026-08-04onset reports3940.750.1370.6567.91.4524.6511-0.14
2026-08-08onset reports3340.730.770.4770.7271.380269.3500.33-0.88
2026-08-10onset reports3198.521.90.3091.9566.10132.4101-0.69
2026-08-15onset reports3264.452.690.2161.64135.96128.4800.3310.64
2026-08-17onset reports283.461.430.1781.6471.82011.6411-0.25
2026-08-18onset reports2114.861.20.241.4178.01036.850.51-0.33
2026-08-23onset reports164.411.060.2221.3758.2306.1811-0.2
2026-08-24onset reports170.41.180.3222.2462.2508.1511-0.21
2026-08-25onset reports1124.180.470.360.5735.2888.90010.81
2026-09-09onset reports167.920.440.1240.1858.469.460110.23
2026-09-20onset reports156.910.580.1660.2714.2042.701-0.8
2026-10-03onset reports2277.42.340.532.0622.580254.8200-0.98
2026-10-13onset reports123.970.240.0570.322.4601.5111-0.18
2026-07-16recovered4139.341.050.4691.02135.503.8411-0.1
2026-07-18recovered4149.612.220.4552.17146.942.670110.1
2026-07-19recovered4181.472.190.4822.04179.352.120110.07
2026-07-20recovered4171.812.070.4171.88169.632.180110.06
2026-07-24recovered4179.372.260.4091.97175.364.010110.13
2026-07-25recovered4177.71.970.3791.55173.524.180110.12
2026-07-26recovered4139.991.510.3811.61138.990111-0.05
2026-07-27recovered4219.872.160.4831.83218.3801.4911-0.06
2026-07-28recovered4544.073.810.3690.86543.250.50.33110.02
2026-07-31recovered4249.162.430.4021.43242.916.250110.12
2026-08-04recovered4157.982.230.4572.51146.41011.5711-0.19
2026-08-08recovered4125.591.540.4172.02109.72015.8711-0.23
2026-08-10recovered4157.12.030.5372.6116.31040.7911-0.32
2026-08-15recovered4384.4512.470.4425.84335.6248.830110.29
2026-08-17recovered4548.5822.340.7799.67117.93430.640000.94
2026-08-18recovered4596.2726.90.83411.8199.02497.250000.95
2026-08-23recovered4553.2927.620.73914.49114.01439.28000.250.92
2026-08-24recovered4518.5831.070.72616.57112.37406.220000.94
2026-08-25recovered4500.2432.230.71616.3390.53409.710000.96
2026-08-30recovered4436.8823.410.63513.5297.38339.5000.250.92
2026-09-03recovered4505.8634.830.71419.194.33411.53000.250.92
2026-09-05recovered4369.423.640.58214.0488.66280.73000.50.89
2026-09-06recovered4295.0119.240.4310.6295.6199.41000.50.79
2026-09-08recovered4563.2838.310.7520.21133.12430.16000.50.9
2026-09-09recovered4621.7933.550.83916.49139.18482.610000.97
2026-09-12recovered4633.3643.460.79420.69139.51493.85000.250.94
2026-09-14recovered4481.6734.590.67516.8175.86405.810000.94
2026-09-15recovered4487.535.110.68317.6687.32400.18000.50.91
2026-09-18recovered4458.8421.330.68113.673.88384.96000.250.92
2026-09-19recovered4518.1428.610.72814.6384.46433.69000.250.93
2026-09-20recovered4406.2323.030.60914.3581.49324.740010.85
2026-09-25recovered4351.8912.740.5628.9675.95275.940010.84
2026-09-29recovered4393.2818.360.63811.6875.57317.71000.750.88
2026-10-03recovered4388.4319.390.64713.0179.69308.74000.250.91
2026-10-13recovered4546.4816.980.8391069.19477.29000.250.92
2026-10-16recovered4428.999.610.7445.3866.71362.28000.50.88
2026-10-18recovered4434.2713.750.7398.675.84358.43000.50.89
2026-10-23recovered3311.1619.090.6468.2760.3250.86000.670.88
2026-10-24recovered3310.0214.790.668.6656.17253.860010.87
2026-10-25recovered3296.7518.940.64510.5254.68242.070010.86
2026-10-28recovered2249.0918.430.7639.9739.22209.87000.50.91
2026-10-30recovered2243.0117.070.7289.5743.57199.44000.50.88
2026-10-31recovered2254.1611.250.7916.4144.86209.3000.50.88
2026-11-02recovered2207.49.70.5835.5640167.40010.88
2026-11-07recovered1238.0112.920.8515.7773.79164.230010.82
2026-11-09recovered1115.2614.790.50810.0428.4786.790010.78

Frozen skill by release ​

Frozen skill per release
julia
frozen_skill_by_vintage_fig = plot_forecast_skill_by_vintage(
    frozen_score_by_vintage_table
);

Skill at each cut-off that more than one release forecast, one point per release. Releases run in the order they were cut, evenly spaced.
Frozen scores by release
streamreleaserelease_datencrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed casesresults-13592026-07-165242.12.950.4541.06121.4495.4625.20.610.07
confirmed casesresults-13912026-07-185242.12.850.4541.07121.4495.4625.20.610.07
confirmed casesresults-13942026-07-195242.12.860.4541.05121.4495.4625.20.610.07
confirmed casesresults-14412026-07-205245.993.010.4661.07113.62106.9225.450.20.80.16
confirmed casesresults-14462026-07-245245.992.920.4661.07113.62106.9225.450.20.80.16
confirmed casesresults-v1.12.02026-07-255245.992.950.4661.08113.62106.9225.450.20.80.16
confirmed casesresults-14622026-07-265247.612.890.4581.06122.2799.4625.890.60.80.13
confirmed casesresults-14692026-07-275247.613.030.4581.07122.2799.4625.890.60.80.13
confirmed casesresults-14792026-07-285247.613.070.4581.07122.2799.4625.890.60.80.13
confirmed casesresults-14892026-07-315247.613.030.4581.07122.2799.4625.890.60.80.13
confirmed casesresults-v1.13.22026-08-045247.613.030.4581.06122.2799.4625.890.60.80.13
confirmed casesresults-15052026-08-085247.612.930.4581.06122.2799.4625.890.60.80.13
confirmed casesresults-15272026-08-105247.612.950.4581.06122.2799.4625.890.60.80.13
confirmed casesresults-15732026-08-155256.513.130.4711.11131.5899.0425.890.40.80.13
confirmed casesresults-15912026-08-175304.063.680.5091.1785.78190.6927.590.20.60.3
confirmed casesresults-v1.15.02026-08-185304.063.630.5091.1885.78190.6927.590.20.60.3
confirmed casesresults-v1.16.02026-08-235318.573.870.5131.1895.65195.7327.190.20.40.3
confirmed casesresults-16252026-08-245318.423.830.5131.1995.56195.7327.130.20.40.3
confirmed casesresults-v1.17.02026-08-255278.343.480.4711.1282.69170.1325.510.20.40.28
confirmed casesresults-v1.18.02026-08-305278.343.490.4711.1282.69170.1325.510.20.40.28
confirmed casesresults-17392026-09-035288.73.570.4751.1492.73169.1126.860.20.60.22
confirmed casesresults-18282026-09-055285.093.460.5051.17102.17151.5831.3500.60.15
confirmed casesresults-18582026-09-065290.993.570.5091.1988.02171.6231.3500.60.14
confirmed casesresults-19702026-09-085295.583.490.5181.1896.92166.931.7700.60.15
confirmed casesresults-v2.1.02026-09-095295.583.550.5181.1996.92166.931.7700.60.15
confirmed casesresults-21772026-09-125311.383.730.5011.14106.74175.6928.9500.60.19
confirmed casesresults-23602026-09-145174.792.050.4220.9666.8474.4433.5100.80.07
confirmed casesresults-25062026-09-155140.971.720.3620.8558.1853.1829.610.20.80.02
confirmed casesresults-25362026-09-185135.091.640.3660.8348.9255.0831.090.20.80.03
confirmed casesresults-27202026-09-195161.241.860.3990.9156.2273.1231.8900.80.08
confirmed casesresults-v3.0.02026-09-205162.5920.3980.9356.9674.2331.400.80.1
confirmed casesresults-29682026-09-255151.781.910.3960.9455.6563.7232.410.20.80.03
confirmed casesresults-30002026-09-295142.691.790.3790.8956.1455.4231.1200.80.06
confirmed casesresults-30862026-10-035141.831.740.3660.8556.6654.5130.660.20.80.03
confirmed casesresults-31692026-10-135151.551.880.3780.8957.5363.6330.390.20.80.04
confirmed casesresults-31802026-10-165184.862.30.4251.0151.28102.2431.3400.80.09
confirmed casesresults-33062026-10-185151.981.80.4170.950.71.6453.3963.3235.270.20.80.03
confirmed casesresults-33242026-10-235163.571.990.3890.910.541.2760.173.330.1700.80.09
confirmed casesresults-34402026-10-245159.081.930.3730.870.541.2258.372.2628.5200.80.07
confirmed casesresults-v3.1.02026-10-255147.321.780.3740.870.581.3357.1359.9430.250.20.80.04
confirmed casesresults-34862026-10-285148.681.80.3810.890.611.3956.5159.9432.230.20.80.01
confirmed casesresults-34942026-10-305148.681.840.3810.910.511.2756.5159.9432.230.20.80.01
confirmed casesresults-35052026-10-315147.961.780.3780.880.591.3656.8159.9431.210.20.80.02
confirmed casesresults-35152026-11-025148.031.810.3770.880.61.3856.8359.9431.270.20.80.03
confirmed casesresults-35312026-11-075148.681.860.3810.890.51.2656.5159.9432.230.20.80.01
confirmed casesresults-35342026-11-095148.681.820.3810.870.611.456.5159.9432.230.20.80.01
confirmed deathsresults-13592026-07-16536.610.70.5530.6928.040.078.490.60.8-0.39
confirmed deathsresults-13912026-07-18536.610.710.5530.728.040.078.490.60.8-0.39
confirmed deathsresults-13942026-07-19536.610.720.5530.7328.040.078.490.60.8-0.39
confirmed deathsresults-14412026-07-20537.950.730.5790.7428.760.448.760.60.8-0.35
confirmed deathsresults-14462026-07-24537.950.720.5790.7228.760.448.760.60.8-0.35
confirmed deathsresults-v1.12.02026-07-25537.950.740.5790.7428.760.448.760.60.8-0.35
confirmed deathsresults-14622026-07-26537.320.710.6350.8127.4809.840.60.8-0.43
confirmed deathsresults-14692026-07-27537.320.720.6350.8127.4809.840.60.8-0.43
confirmed deathsresults-14792026-07-28537.320.710.6350.827.4809.840.60.8-0.43
confirmed deathsresults-14892026-07-31537.320.70.6350.7927.4809.840.60.8-0.43
confirmed deathsresults-v1.13.22026-08-04537.320.730.6350.8127.4809.840.60.8-0.43
confirmed deathsresults-15052026-08-08537.320.720.6350.827.4809.840.60.8-0.43
confirmed deathsresults-15272026-08-10537.320.710.6350.7927.4809.840.60.8-0.43
confirmed deathsresults-15732026-08-15537.790.730.6310.827.840.149.820.60.8-0.4
confirmed deathsresults-15912026-08-17536.940.720.2880.3720.912.143.90.610
confirmed deathsresults-v1.15.02026-08-18536.940.710.2880.3620.912.143.90.610
confirmed deathsresults-v1.16.02026-08-23538.990.760.270.3424.1711.593.220.610.04
confirmed deathsresults-16252026-08-24538.960.740.270.3424.1511.593.220.610.05
confirmed deathsresults-v1.17.02026-08-25535.050.680.2560.3321.9610.142.940.61-0.03
confirmed deathsresults-v1.18.02026-08-30535.050.680.2560.3321.9610.142.940.61-0.03
confirmed deathsresults-17392026-09-03534.710.660.2740.3522.918.353.450.61-0.02
confirmed deathsresults-18282026-09-05528.720.550.2950.3720.393.265.070.60.8-0.21
confirmed deathsresults-18582026-09-06535.330.670.3140.421.978.265.10.60.8-0.17
confirmed deathsresults-19702026-09-08531.570.610.2960.3822.64.244.730.60.8-0.19
confirmed deathsresults-v2.1.02026-09-09531.570.630.2960.3922.64.244.730.60.8-0.19
confirmed deathsresults-21772026-09-12539.440.740.2930.3725.679.913.860.61-0.05
confirmed deathsresults-23602026-09-14541.870.820.2940.3723.5414.653.680.410.01
confirmed deathsresults-25062026-09-15528.50.530.2520.3120.714.243.550.81-0.1
confirmed deathsresults-25362026-09-18531.720.60.2750.3420.547.353.830.81-0.08
confirmed deathsresults-27202026-09-19533.370.640.2730.3520.938.923.520.61-0.01
confirmed deathsresults-v3.0.02026-09-20531.40.60.2610.3320.437.553.410.81-0.08
confirmed deathsresults-29682026-09-25530.290.60.2660.3418.558.23.540.61-0.06
confirmed deathsresults-30002026-09-29526.130.50.2470.3218.554.143.430.81-0.1
confirmed deathsresults-30862026-10-03529.860.580.2630.3421.015.323.540.81-0.1
confirmed deathsresults-31692026-10-13534.880.670.2770.3523.437.933.520.61-0.03
confirmed deathsresults-31802026-10-16524.930.470.2440.314.786.593.560.61-0.06
confirmed deathsresults-33062026-10-18532.390.620.2970.381.421.4919.88.554.040.41-0.08
confirmed deathsresults-33242026-10-23534.050.640.260.331.151.1121.739.273.050.61-0.06
confirmed deathsresults-34402026-10-24530.010.570.2350.291.21.1119.937.092.980.81-0.07
confirmed deathsresults-v3.1.02026-10-25530.460.590.2610.331.31.3219.317.63.550.81-0.05
confirmed deathsresults-34862026-10-28530.460.590.260.331.251.2219.117.63.750.81-0.08
confirmed deathsresults-34942026-10-30530.460.580.260.331.241.2319.117.63.750.81-0.08
confirmed deathsresults-35052026-10-31530.490.580.2630.341.231.2219.147.63.750.81-0.08
confirmed deathsresults-35152026-11-02530.430.580.2580.331.251.2119.287.63.550.81-0.06
confirmed deathsresults-35312026-11-07530.460.580.260.331.251.2219.117.63.750.81-0.08
confirmed deathsresults-35342026-11-09530.460.590.260.331.311.3219.117.63.750.81-0.08
isolation bedsresults-13592026-07-16477.620.460.1580.4141.551.5434.540.51-0.22
isolation bedsresults-13912026-07-18477.620.460.1580.4141.551.5434.540.51-0.22
isolation bedsresults-13942026-07-19477.620.470.1580.4241.551.5434.540.51-0.22
isolation bedsresults-14412026-07-20477.320.460.1580.4138.491.0237.810.51-0.22
isolation bedsresults-14462026-07-24477.320.450.1580.438.491.0237.810.51-0.22
isolation bedsresults-v1.12.02026-07-25477.320.460.1580.4138.491.0237.810.51-0.22
isolation bedsresults-14622026-07-26471.120.420.1440.3738.043.5329.560.51-0.2
isolation bedsresults-14692026-07-27471.120.420.1440.3738.043.5329.560.51-0.2
isolation bedsresults-14792026-07-28471.120.420.1440.3738.043.5329.560.51-0.2
isolation bedsresults-14892026-07-31471.120.430.1440.3838.043.5329.560.51-0.2
isolation bedsresults-v1.13.22026-08-04471.120.410.1440.3638.043.5329.560.51-0.2
isolation bedsresults-15052026-08-08471.120.420.1440.3738.043.5329.560.51-0.2
isolation bedsresults-15272026-08-10471.120.440.1440.3938.043.5329.560.51-0.2
isolation bedsresults-15732026-08-15476.50.450.1540.435.782.1538.570.51-0.22
isolation bedsresults-15912026-08-17473.460.440.1470.3936.12.1335.230.51-0.2
isolation bedsresults-v1.15.02026-08-18473.460.450.1470.3936.12.1335.230.51-0.2
isolation bedsresults-v1.16.02026-08-23473.460.430.1470.3735.252.0536.160.51-0.2
isolation bedsresults-16252026-08-24473.460.450.1470.435.252.0536.160.51-0.2
isolation bedsresults-v1.17.02026-08-25474.840.440.1510.3939.271.4434.130.51-0.23
isolation bedsresults-v1.18.02026-08-30474.840.460.1510.4139.271.4434.130.51-0.23
isolation bedsresults-17392026-09-03475.350.450.1550.440.750.9833.630.51-0.24
isolation bedsresults-18282026-09-05473.290.440.1490.3935.511.8135.980.51-0.24
isolation bedsresults-18582026-09-06467.650.410.1360.3636.961.3129.390.751-0.2
isolation bedsresults-19702026-09-08476.590.450.1560.437.540.8838.170.51-0.28
isolation bedsresults-v2.1.02026-09-09476.590.460.1560.4137.540.8838.170.51-0.28
isolation bedsresults-21772026-09-12476.80.450.1560.3934.071.341.430.51-0.24
isolation bedsresults-23602026-09-144205.61.180.2410.59126.5279.0800.2510.58
isolation bedsresults-25062026-09-154131.190.80.1790.4785.3645.8300.7510.46
isolation bedsresults-25362026-09-184101.70.610.1470.3872.2929.4100.7510.4
isolation bedsresults-27202026-09-194122.90.730.1670.4377.5545.3400.510.45
isolation bedsresults-v3.0.02026-09-204150.810.930.1990.5490.816000.510.54
isolation bedsresults-29682026-09-254138.520.810.1860.4783.2455.2800.510.49
isolation bedsresults-30002026-09-294121.410.730.1680.4477.6743.7400.510.43
isolation bedsresults-30862026-10-034141.150.820.1870.4788.9952.1600.510.51
isolation bedsresults-31692026-10-134167.91.020.2140.5797.2370.6700.510.56
isolation bedsresults-31802026-10-164170.8810.240.6149.32121.560010.69
isolation bedsresults-33062026-10-184138.370.820.1850.4886.7451.6300.510.46
isolation bedsresults-33242026-10-234176.731.050.2240.58102.5574.1800.2510.6
isolation bedsresults-34402026-10-244147.580.880.1920.599.2748.3100.510.48
isolation bedsresults-v3.1.02026-10-254141.280.840.1870.4893.6947.5900.510.44
isolation bedsresults-34862026-10-284141.280.860.1870.593.6947.5900.510.44
isolation bedsresults-34942026-10-304141.280.830.1870.4893.6947.5900.510.44
isolation bedsresults-35052026-10-314141.280.830.1870.4793.6947.5900.510.44
isolation bedsresults-35152026-11-024141.280.840.1870.4893.6947.5900.510.44
isolation bedsresults-35312026-11-074141.280.840.1870.4893.6947.5900.510.44
isolation bedsresults-35342026-11-094141.280.840.1870.4893.6947.5900.510.44
Frozen-fit forecasts-versus-now overlay
julia
# The frozen joint and the persistence baseline only. A single-stream
# frozen fit sits at the same made dates as the joint, so drawing it here
# would overplot the joint point beside it. The single-stream frozen fits
# are compared against the joint in the skill figures and in the
# validation plot at the one-week-back cut-off.
frozen_overlay_fig = plot_forecast_overlay(
    scored_overlay(
        vcat(
            select_fit_role(frozen_overlay_df, "joint"),
            select_fit_role(frozen_overlay_df, "baseline")
        )
    )
);

The frozen forecasts made at each cut-off against the value observed since (black), one panel per stream and horizon by the cut-off each was made from. Each panel carries the frozen forecast and the persistence baseline, coloured as in the cross-release overlay above.
Freeze the renewal data to a cut-off and re-fit
julia
# Frozen re-fits and released_df are prepared in the setup block above.

Individual fits against the baseline ​

This section scores each stream's own individual fit across releases, as forecast scoring across releases does the joint, against the same persistence baseline.

Individual-fit rows of the cross-release scores
julia
# The relative skill against a stream's individual fit is only ever
# computed on the joint model's row, so on these rows it is missing by
# construction and the column is dropped rather than shown empty.
individual_score_overview_table = drop_individual_fit_columns(
    select_fit_role(forecast_score_overview_table, "individual")
)
individual_score_by_horizon_table = drop_individual_fit_columns(
    select_fit_role(forecast_score_by_horizon_table, "individual")
)
# See the comment above `joint_score_by_release_table`'s assignment for why
# this setup chunk's last statement needs a trailing `;`.
individual_score_by_release_table = drop_individual_fit_columns(
    select_fit_role(forecast_score_by_release_table, "individual")
);
streamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed casesconfirmed155421.363.570.2873.18197.28223.330.750.530.690.48
confirmed deathsconfirmed_deaths155192.622.410.3342.83131.5226.5834.520.670.99-0.1
isolation bedstreatment150113.371.960.1512.16100.791.9110.670.961-0.13
onset reportsonsets31201.311.430.3291.3389.7530.7780.790.520.81-0.11
julia
individual_relative_skill_fig = plot_forecast_relative_skill(
    individual_score_by_horizon_table;
    empty_message = "Empty: no release old enough for its targets to " *
        "have been observed carries an individual-stream " *
        "forecast. Not a missing forecast."
);

Relative skill against the baseline by horizon for each stream's own individual fit, one panel per stream.
Scores by horizon
streamhorizonfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases7confirmed44171.014.410.2973.960.96108.661.390.480.570.5
confirmed cases14confirmed40337.644.110.2863.52138.28198.211.150.50.620.49
confirmed cases21confirmed37522.273.670.2813.03244.81277.240.220.540.730.48
confirmed cases28confirmed34734.013.10.2812.42391.37342.590.040.620.880.46
confirmed deaths7confirmed_deaths4460.812.10.292.8134.7612.1213.930.660.98-0.07
confirmed deaths14confirmed_deaths401562.730.3343.1589.4539.5127.040.620.98-0.08
confirmed deaths21confirmed_deaths37219.372.290.342.82158.3618.4442.570.681-0.13
confirmed deaths28confirmed_deaths34377.172.410.3852.6277.0238.9461.210.741-0.15
isolation beds7treatment42100.172.550.142.9385.873.4510.850.931-0.12
isolation beds14treatment38111.632.140.1492.34101.171.868.5911-0.1
isolation beds21treatment36117.011.820.1531.95105.80.7210.50.971-0.13
isolation beds28treatment34127.761.610.1671.72113.471.3212.970.941-0.17
onset reports7onsets15114.650.880.3751.0757.2719.7537.640.670.730.06
onset reports14onsets10251.962.10.3022.1198.1135.66118.20.40.8-0.26
onset reports21onsets6333.531.680.261.64157.0350.16126.330.331-0.28
Scores by release
made_datestreamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
2026-07-23confirmed casesconfirmed4327.810.860.2060.83319.434.843.5511-0.03
2026-07-25confirmed casesconfirmed4346.241.480.2131.49343.882.30.06110.02
2026-07-26confirmed casesconfirmed4299.071.30.2111.51291.390.167.5111-0.12
2026-07-27confirmed casesconfirmed4267.621.330.1711.27262.035.380.21110.04
2026-07-31confirmed casesconfirmed4402.873.050.2192.65383.8219.050110.09
2026-08-01confirmed casesconfirmed4336.312.790.2092.88322.9813.330110.08
2026-08-02confirmed casesconfirmed4296.982.250.1712.05286.156.724.1111-0.03
2026-08-03confirmed casesconfirmed4291.81.420.1751.14281.4210.080.3110.05
2026-08-04confirmed casesconfirmed4350.843.620.1913.25323.4127.40.04110.11
2026-08-07confirmed casesconfirmed4308.43.550.1922.9294.779.514.1211-0.01
2026-08-11confirmed casesconfirmed4335.756.880.1915.09307.327.620.84110.1
2026-08-15confirmed casesconfirmed4312.225.660.2114.57308.061.822.3411-0.03
2026-08-17confirmed casesconfirmed4299.384.20.2283.61293.3306.0611-0.09
2026-08-22confirmed casesconfirmed4335.334.920.2064.07293.0242.310110.15
2026-08-24confirmed casesconfirmed4315.974.760.1854.04253.0562.920110.26
2026-08-25confirmed casesconfirmed4302.154.570.1844.11236.9365.230110.27
2026-08-30confirmed casesconfirmed4384.814.610.2033.36243.44141.370110.37
2026-08-31confirmed casesconfirmed4298.733.070.1772.74219.1279.610110.26
2026-09-01confirmed casesconfirmed4350.743.930.1933.08251.3499.40110.29
2026-09-06confirmed casesconfirmed4352.022.90.2152.88256.3295.710110.33
2026-09-10confirmed casesconfirmed4448.082.30.2641.92245.86202.2200.2510.48
2026-09-12confirmed casesconfirmed41307.035.090.7473.69117.751189.280001
2026-09-13confirmed casesconfirmed41202.655.730.6944.27121.721080.920001
2026-09-15confirmed casesconfirmed41039.064.630.6573.31126.86912.210001
2026-09-16confirmed casesconfirmed49484.280.6173.27127.78820.220001
2026-09-19confirmed casesconfirmed4650.673.340.4822.97126.04524.63000.250.95
2026-09-21confirmed casesconfirmed4480.054.940.3664.78100.45379.61000.250.94
2026-09-22confirmed casesconfirmed4356.784.650.2875.01112.6244.18000.50.9
2026-09-25confirmed casesconfirmed4368.496.370.2694.4997.52270.97000.50.89
2026-09-26confirmed casesconfirmed4440.889.360.358.2688.29352.59000.250.93
2026-09-27confirmed casesconfirmed4248.184.860.1812.7498.19149.990010.77
2026-10-02confirmed casesconfirmed4177.53.250.1382.5877.28100.2200.2510.7
2026-10-06confirmed casesconfirmed4206.464.310.1523.6180.84125.620010.7
2026-10-10confirmed casesconfirmed4196.332.50.1452.1386.02110.30010.69
2026-10-20confirmed casesconfirmed3367.8611.820.3059.3867.93299.940000.98
2026-10-23confirmed casesconfirmed3418.7610.380.3175.8269.11349.650000.97
2026-10-25confirmed casesconfirmed3520.812.490.4239.6580.07440.740000.99
2026-10-30confirmed casesconfirmed2334.936.460.3824.9548.12286.80001
2026-10-31confirmed casesconfirmed2320.786.610.3745.6146.05274.730000.99
2026-11-01confirmed casesconfirmed2325.19100.3768.5144.14281.050000.99
2026-11-04confirmed casesconfirmed1208.811.130.3658.9631.89176.910000.99
2026-11-06confirmed casesconfirmed128311.530.4899.1633.22249.780001
2026-11-07confirmed casesconfirmed1273.8814.920.45811.1134.22239.660001
2026-11-09confirmed casesconfirmed1319.6810.180.5158.1337.6282.080001
2026-07-23confirmed deathsconfirmed_deaths4701.544.420.4482.32383.29318.2500.2510.56
2026-07-25confirmed deathsconfirmed_deaths4237.782.010.2341.61214.2823.50110.18
2026-07-26confirmed deathsconfirmed_deaths4256.462.320.2552.05231.6524.810110.13
2026-07-27confirmed deathsconfirmed_deaths4310.272.720.2521.79280.8429.430110.21
2026-07-31confirmed deathsconfirmed_deaths4312.362.640.2892.06292.2120.080.07110.08
2026-08-01confirmed deathsconfirmed_deaths4280.922.210.2531.59261.4619.460110.1
2026-08-02confirmed deathsconfirmed_deaths4256.052.760.2762.4238.2316.880.94110.03
2026-08-03confirmed deathsconfirmed_deaths4224.51.610.2451.2212.210.831.46110.02
2026-08-04confirmed deathsconfirmed_deaths4248.773.460.2833.34242.124.292.3511-0.02
2026-08-07confirmed deathsconfirmed_deaths4231.53.330.3253.21223.681.026.7911-0.11
2026-08-11confirmed deathsconfirmed_deaths4244.611.620.33710.79233.591.959.0611-0.13
2026-08-15confirmed deathsconfirmed_deaths4244.836.290.345.85239.910.044.8911-0.1
2026-08-17confirmed deathsconfirmed_deaths4251.324.520.2973.55246.093.931.311-0.02
2026-08-22confirmed deathsconfirmed_deaths42422.640.4823.88219.73022.2711-0.18
2026-08-24confirmed deathsconfirmed_deaths4220.023.150.5436.28144.48075.5411-0.4
2026-08-25confirmed deathsconfirmed_deaths4247.193.430.7969.35151.31095.880.751-0.45
2026-08-30confirmed deathsconfirmed_deaths4260.373.030.9079.28129.820130.550.51-0.51
2026-08-31confirmed deathsconfirmed_deaths4247.042.240.7826.01144.820102.2211-0.4
2026-09-01confirmed deathsconfirmed_deaths4246.662.510.9278.09116.860129.80.51-0.48
2026-09-06confirmed deathsconfirmed_deaths4281.8831.17110.3979.510202.3701-0.64
2026-09-10confirmed deathsconfirmed_deaths4207.511.50.7463.6124.24083.270.751-0.45
2026-09-12confirmed deathsconfirmed_deaths4100.940.730.150.6960.0640.8700.2510.54
2026-09-13confirmed deathsconfirmed_deaths494.040.780.1450.7555.4938.5500.2510.57
2026-09-15confirmed deathsconfirmed_deaths462.490.550.1080.5652.899.60110.31
2026-09-16confirmed deathsconfirmed_deaths457.360.570.1020.5952.384.9800.7510.26
2026-09-19confirmed deathsconfirmed_deaths454.290.60.0950.6348.661.34.32110.01
2026-09-21confirmed deathsconfirmed_deaths444.170.780.080.8931.160.212.811-0.2
2026-09-22confirmed deathsconfirmed_deaths468.071.690.1211.9427.69040.380.51-0.51
2026-09-25confirmed deathsconfirmed_deaths4132.714.480.2563.4526.890105.8201-0.84
2026-09-26confirmed deathsconfirmed_deaths481.453.160.142.7427.910.1753.360.251-0.48
2026-09-27confirmed deathsconfirmed_deaths496.474.080.1753.2226.67069.801-0.7
2026-10-02confirmed deathsconfirmed_deaths493.873.310.1743.2427.18066.6901-0.72
2026-10-06confirmed deathsconfirmed_deaths498.952.10.1772.2930.26068.6901-0.77
2026-10-10confirmed deathsconfirmed_deaths475.261.320.1261.3430.12045.140.251-0.57
2026-10-20confirmed deathsconfirmed_deaths330.981.420.0591.228.790.621.5711-0.09
2026-10-23confirmed deathsconfirmed_deaths348.592.370.0891.6432.7915.800.6710.44
2026-10-25confirmed deathsconfirmed_deaths371.792.620.1211.8235.636.1900.3310.6
2026-10-30confirmed deathsconfirmed_deaths2975.863.591.132.07167.28808.580001
2026-10-31confirmed deathsconfirmed_deaths245.081.220.1191.1820.3324.750010.69
2026-11-01confirmed deathsconfirmed_deaths226.781.260.071.3220.436.360110.33
2026-11-04confirmed deathsconfirmed_deaths114.191.70.0561.5913.480.710110.14
2026-11-06confirmed deathsconfirmed_deaths113.370.540.0540.5511.741.640110.23
2026-11-07confirmed deathsconfirmed_deaths112.450.820.0530.7810.5401.9111-0.25
2026-11-09confirmed deathsconfirmed_deaths112.310.920.0531.0311.1801.1311-0.21
2026-07-23isolation bedstreatment455.731.920.0892.3147.7907.9411-0.21
2026-07-25isolation bedstreatment451.681.460.0821.7547.140.014.5211-0.14
2026-07-26isolation bedstreatment461.671.50.11.8154.820.236.6311-0.12
2026-07-27isolation bedstreatment459.4720.0982.4254.6704.811-0.18
2026-07-31isolation bedstreatment482.762.010.1322.5268.17014.60.751-0.28
2026-08-01isolation bedstreatment488.621.260.1531.6173.94014.670.751-0.25
2026-08-02isolation bedstreatment484.371.690.1412.0969.7014.670.751-0.28
2026-08-03isolation bedstreatment479.931.890.1342.3471.9108.0111-0.21
2026-08-04isolation bedstreatment487.061.30.1411.4882.731.223.1111-0.03
2026-08-07isolation bedstreatment4104.981.640.1712.0485.76019.2111-0.29
2026-08-11isolation bedstreatment4102.610.60.1710.6990.93011.6811-0.21
2026-08-15isolation bedstreatment4127.511.350.2041.798.57028.9411-0.33
2026-08-17isolation bedstreatment4103.761.430.1611.7393.32010.4411-0.19
2026-08-22isolation bedstreatment4144.982.610.2183.23120.25024.7311-0.22
2026-08-24isolation bedstreatment4127.242.580.1812.99115.131.0511.0611-0.11
2026-08-25isolation bedstreatment4131.922.060.1912.39111.86020.0611-0.2
2026-08-30isolation bedstreatment4128.623.030.1713.28117.425.735.4711-0.02
2026-08-31isolation bedstreatment4120.612.590.162.89111.751.797.0711-0.07
2026-09-01isolation bedstreatment4120.72.910.1543.19116.430.683.611-0.05
2026-09-06isolation bedstreatment4124.132.550.1562.63120.731.811.5811-0.01
2026-09-10isolation bedstreatment4129.272.380.1582.45119.720.249.3111-0.06
2026-09-12isolation bedstreatment4134.162.330.1812.8121.99012.1611-0.13
2026-09-13isolation bedstreatment4121.642.270.1572.57120.420.021.211-0.04
2026-09-15isolation bedstreatment4132.572.390.162.69125.50.316.7511-0.06
2026-09-16isolation bedstreatment4130.442.810.163.12123.530.26.711-0.1
2026-09-19isolation bedstreatment4126.152.470.1492.54124.251.850.05110.03
2026-09-21isolation bedstreatment4136.462.280.1552.23134.531.030.91110.01
2026-09-22isolation bedstreatment4182.972.710.2433.35143.66039.3211-0.26
2026-09-25isolation bedstreatment4139.312.110.1552.04135.071.572.6811-0.03
2026-09-26isolation bedstreatment4128.651.360.1521.34119.230.049.3811-0.12
2026-09-27isolation bedstreatment4148.531.970.1912.14127.91020.6211-0.2
2026-10-02isolation bedstreatment4143.431.870.1691.94128.42015.0111-0.17
2026-10-06isolation bedstreatment4144.792.080.1692.47121.5023.2911-0.24
2026-10-10isolation bedstreatment4138.172.990.1533.11133.0705.111-0.11
2026-10-20isolation bedstreatment3132.262.580.1412.8129.1403.1211-0.07
2026-10-23isolation bedstreatment3128.173.330.1112.9382.4145.7600.6710.45
2026-10-25isolation bedstreatment299.53.020.0912.7762.1937.3100.510.48
2026-10-30isolation bedstreatment262.581.860.0651.9647.06015.5211-0.31
2026-10-31isolation bedstreatment149.631.60.0531.6535.11014.5211-0.37
2026-11-01isolation bedstreatment153.141.950.0592.0830.72022.4211-0.48
2026-11-04isolation bedstreatment140.981.110.0421.1637.573.410110.22
2026-11-06isolation bedstreatment156.531.820.0581.9134.08022.4601-0.5
2026-08-02onset reportsonsets3122.90.860.1881.0796.84026.0611-0.28
2026-08-03onset reportsonsets3105.391.730.1661.47100.321.153.9211-0.02
2026-08-04onset reportsonsets3141.841.120.2661.2388.340.0453.460.671-0.26
2026-08-07onset reportsonsets3377.961.10.6331.2982.320295.6401-0.8
2026-08-11onset reportsonsets3243.781.520.4291.985.910157.870.331-0.64
2026-08-15onset reportsonsets3369.664.110.2782.36205.04164.6100.3310.57
2026-08-17onset reportsonsets2246.493.960.2552.28153.6192.8800.510.47
2026-08-22onset reportsonsets2121.151.030.2111.21104.1217.040110.22
2026-08-24onset reportsonsets185.161.210.3922.0982.0903.0711-0.12
2026-08-25onset reportsonsets195.660.370.5110.8188.577.090110.18
2026-09-21onset reportsonsets172.061.070.2360.7610.9761.090000.98
2026-09-22onset reportsonsets172.991.260.2370.6912.8360.160000.98
2026-10-02onset reportsonsets2133.31.710.2141.2217.720115.580.50.5-0.74
2026-10-06onset reportsonsets2352.342.180.592.322.60329.7400-1
2026-10-10onset reportsonsets1123.550.430.4190.5615.39108.170000.96

Saving forecast outputs ​

The one-week-back validation forecast is written in the release forecast's archive format, so it is recorded as a release asset alongside the forecast it is scored against.

Write forecast outputs
julia
output_dir = get(
    ENV, "BVD_OUTPUT_DIR",
    joinpath(pkgdir(BVDOutbreakSize), "output")
)
mkpath(output_dir)
CSV.write(
    joinpath(output_dir, "forecast_validation.csv"),
    forecast_archive(
        [(7, validation_forecast)];
        made_date = frozen_lastweek.o.cutoff, thin = 5
    )
)
"/home/runner/work/BVDOutbreakSize/BVDOutbreakSize/output/forecast_validation.csv"
Write the summary bullets
julia
# The bullets under the summary heading at the top of the page. They read
# tables built further down, so they are written here and read back when
# the site is assembled.
evaluation_forecast_national_summary = let
    scored(tbl) = filter(r -> !ismissing(r.rel_to_baseline), tbl)
    function beat(label, tbl)
        rows = scored(tbl)
        size(rows, 1) == 0 &&
            return string("- **", label, ":** no scored forecasts yet.")
        return string(
            "- **", label, ":** beat the baseline on ",
            count(<(1), rows.rel_to_baseline), " of ", size(rows, 1),
            " streams."
        )
    end
    frozen_joint = select_fit_role(frozen_score_overview_table, "joint")
    overall = [
        beat("Joint model across releases", joint_score_overview_table),
        beat("Frozen joint model", frozen_joint),
    ]
    # One row per stream and evaluation, the cross-release scores first.
    rows(lead, tbl) = let r = scored(tbl)
        DataFrame(
            "Stream" => r.stream,
            "Evaluation" => fill(lead, size(r, 1)),
            "Relative skill" => finite_round.(r.rel_to_baseline),
            "90% coverage" => finite_round.(r.coverage_90),
            "Bias" => finite_round.(r.bias),
            "Forecasts" => r.n,
        )
    end
    table = vcat(
        rows("Across releases", joint_score_overview_table),
        rows("Frozen fit", frozen_joint)
    )
    join(
        vcat(
            [join(overall, "\n")],
            size(table, 1) == 0 ? String[] :
                [markdown_table(table; missing_text = "n/a")]
        ), "\n\n"
    )
end
dashboard_dir = joinpath(
    pkgdir(BVDOutbreakSize), "docs", "src", "summary_assets"
)
mkpath(dashboard_dir)
write(
    joinpath(dashboard_dir, "evaluation_forecast_national.md"),
    evaluation_forecast_national_summary
);