BVDOutbreakSize

Appearance

Forecast evaluation ​

How the forecasts on the forecasts page have scored against the data that arrived afterwards. It also tracks how the outbreak-size and reproduction-number estimates moved from release to release. Scoring is the continuous ranked probability score against a persistence baseline, defined in the forecast scoring Methods section. The same scoring by province is on the province forecast evaluation page and by health zone on the health-zone forecast evaluation page.

Load packages, data and fitted chains
julia
# Shared setup: packages, observations and the fit registry. See
# `docs/pages/_setup.jl`.
using BVDOutbreakSize
include(joinpath(pkgdir(BVDOutbreakSize), "docs", "pages", "_setup.jl"))
validation_forecast_from (generic function with 1 method)
julia
# The fits this page reads, loaded from the cache here.
chn_joint = load_fit("joint")
frozen_lastweek = load_fit("frozen_validation")
frozen_lastweek_streams = frozen_validation_stream_fits()
chn_exports = load_fit("exports")
chn_deaths = load_fit("deaths")
chn_cases = load_fit("cases")
chn_confirmed = load_fit("confirmed")
chn_confirmed_deaths = load_fit("confirmed_deaths")
chn_treatment = load_fit("treatment")
chn_onsets = load_fit("onsets")
frozen_by_cutoff = frozen_fits_by_cutoff()
frozen_C(c) = vec(Array(frozen_by_cutoff[c].chn[:C_T]))
# Every frozen fit is a full joint fit, so it carries the same walk base
# chn_joint does.
frozen_R0(c) = r0_walk_draws(frozen_by_cutoff[c].chn);

Summary ​

The overall bullets come first, then each stream's own scores, from the sections further down this page. Relative skill is the model's CRPS over the persistence baseline's, so a value below one beats the baseline. Coverage is the fraction of forecasts whose observed value falls inside the 90% predictive interval, nominally 0.9.

  • Joint model across releases: beat the baseline on 1 of 5 streams.

  • Frozen joint model: beat the baseline on 2 of 5 streams.

Across releases

  • confirmed cases: relative skill 6.42, 90% coverage 0.93, bias 0.32 over 103 forecasts.

  • confirmed deaths: relative skill 8.92, 90% coverage 0.91, bias -0.14 over 101 forecasts.

  • isolation beds: relative skill 1.46, 90% coverage 0.95, bias 0.2 over 98 forecasts.

  • onset reports: relative skill 0.94, 90% coverage 0.96, bias -0.1 over 24 forecasts.

  • recovered: relative skill 1.67, 90% coverage 1.0, bias -0.14 over 14 forecasts.

Frozen fits

  • confirmed cases: relative skill 2.43, 90% coverage 0.78, bias 0.27 over 255 forecasts.

  • confirmed deaths: relative skill 0.88, 90% coverage 0.87, bias -0.04 over 199 forecasts.

  • isolation beds: relative skill 0.79, 90% coverage 0.98, bias 0.1 over 224 forecasts.

  • onset reports: relative skill 1.01, 90% coverage 0.93, bias -0.13 over 29 forecasts.

  • recovered: relative skill 5.34, 90% coverage 0.68, bias 0.4 over 100 forecasts.

Forecast validation ​

How last week's forecast held up against the data since observed, using the frozen re-fit and one-week projection defined in forecast-versus-frozen evaluation. Only the streams the situation reports are still updating are validated here. A stream that has stopped being reported carries a cumulative total that repeats its last reported value, so there is no observation for the past week to score against. The frozen fit also conditions on the isolation beds, so the projected bed occupancy is scored against the beds held a week later. The bed validation is weak at a one-week-back freeze. The reported occupancy rate starts only on 9 June, so the capacity has no implied-capacity anchor and rides its random walk back to the freeze date. Like the scores further down, the confirmed new-count rows here take out any retrospective harmonisation step the week contained. Such a step reattaches records notified earlier, so it is not something the forecast was predicting. The cumulative rows are scored against the published total, harmonisation included.

Fit one week back and validate the one-week-ahead forecast
julia
# frozen_lastweek and frozen_lastweek_streams are computed in the setup
# block above, and `validation_forecast_from` is defined there.
validation_forecast = validation_forecast_from("frozen_validation");

# Each frozen individual (single-stream) fit's own one-week-ahead new-count
# forecast at the same cut-off as `frozen_lastweek`, from
# [`forecast_stream`](@ref) (the same per-stream forecaster
# `stream_forecasts.csv` uses), so the validation plots below can show the
# individual fit alongside the joint rather than the joint alone. Recovered
# has no individual fit and is absent here, as it is throughout this report.
# Only the still-reported streams are fitted at the validation cut-off, so
# a stream the situation reports have stopped updating is absent from
# `frozen_lastweek_streams` and carries no individual series here.
function _validation_individual_new(sid, stream::Symbol)
    haskey(frozen_lastweek_streams, sid) || return nothing
    return Float64.(
        forecast_stream(
            fit_forecast("frozen_validation_$sid"), stream; horizon = 7
        )
    )
end
validation_individual = NamedTuple(
    k => v
        for (k, v) in pairs(
            (;
                cases_new = _validation_individual_new(
                    "cases", :reported_cases
                ),
                deaths_new = _validation_individual_new(
                    "deaths", :suspected_deaths
                ),
                confirmed_new = _validation_individual_new(
                    "confirmed", :confirmed_cases
                ),
                confirmed_deaths_new = _validation_individual_new(
                    "confirmed_deaths", :confirmed_deaths
                ),
            )
        )
        if !isnothing(v)
)
# The frozen individual (treatment-only) fit's own bed-occupancy forecast.
# `nothing` when the beds have stopped being reported, so the treatment fit
# is absent; the bed panel then draws the joint alone.
validation_individual_isolation = haskey(frozen_lastweek_streams, "treatment") ?
    Float64.(
        forecast_stream(
            fit_forecast("frozen_validation_treatment"), :isolation_beds;
            horizon = 7
        )
    ) : nothing

# The observed beds at the current cut-off (the forecast target), so the
# frozen-fit bed forecast is scored against what the beds actually held.
# Held back once the beds stop being reported, since the last count would
# then be carried forward rather than observed at the target date.
_obs_beds = stream_reporting(obs, :isolation_beds) ?
    obs.isolation_history.counts[end] : missing
# Same observed/baseline keying as the plot below, so the table covers every
# fitted count stream (cumulative and new-count rows) plus the bed level.
# A harmonisation-break day between the frozen cut-off and the current one
# puts records into the confirmed cumulative that were never notified in that
# week, so the new-count truth carries a step the forecast was never
# predicting. Take it out, the same correction `score_releases.jl` applies.
# Grid days are relative to a seeding date fixed by the genetic tmrca, so the
# frozen fit's own `n` and the current `obs.n` index the same grid.
validation_breaks = (
    confirmed_cum = confirmed_break_correction(
        obs, frozen_lastweek.o.n, obs.n
    ),
    confirmed_deaths_cum = confirmed_break_correction(
        obs, frozen_lastweek.o.n, obs.n; deaths = true
    ),
)

# Observed cumulative at the target date per stream, keyed by the forecast's
# cumulative column; `baseline` is each stream's origin cumulative (the
# frozen cut-off), so the new count is scored against observed minus origin,
# less any harmonisation the window carries (see `validation_breaks`). Both
# the table and the plot below take the still-reported streams
# (`reporting_cum_cols`, from the setup block): a stream the situation
# reports have stopped updating has an origin and a target reading the same
# repeated total, so its cumulative truth is stale and its new-count truth is
# a guaranteed zero.
validation_observed = (
    cases_cum = obs.reported_cases,
    deaths_cum = obs.total_deaths,
    confirmed_cum = obs.confirmed_cases,
    confirmed_deaths_cum = obs.confirmed_deaths,
    recovered_cum = obs.recovered_cases,
)
validation_baseline = (
    cases_cum = frozen_lastweek.o.reported_cases,
    deaths_cum = frozen_lastweek.o.total_deaths,
    confirmed_cum = frozen_lastweek.o.confirmed_cases,
    confirmed_deaths_cum = frozen_lastweek.o.confirmed_deaths,
    recovered_cum = frozen_lastweek.o.recovered_cases,
)

validation_table = forecast_vs_truth(
    validation_forecast;
    observed = keep_streams(validation_observed, reporting_cum_cols),
    baseline = keep_streams(validation_baseline, reporting_cum_cols),
    breaks = validation_breaks,
    isolation = _obs_beds
);
Forecast-versus-observed validation table
StreamQuantityObservedLower 90%Lower 60%Lower 30%Upper 30%Upper 60%Upper 90%Within 90% PI
DRC confirmed casescumulative by T+78116812081958246833884048528no
DRC confirmed casesnew this week383387462513605671795no
DRC confirmed deathscumulative by T+73924394539773995403140514089no
DRC confirmed deathsnew this week192213245263299319357no
DRC recovered among confirmedcumulative by T+72088191121422236235324182610yes
DRC recovered among confirmednew this week1869240334451516708yes
DRC isolation bedsoccupancy at T+78035606597098118771034yes

The observation panels histogram the one-week-ahead forecast made from the frozen fit: a cumulative and a new-count panel for each still-reported count stream the forecast carries. The 90% predictive interval is shaded, and the count observed by the current cut-off is a dashed black rule. Where a stream has its own individual (single-stream) fit, that fit's forecast from the same frozen cut-off is overlaid as a dotted step outline on the joint's own histogram bins.

Forecast-versus-observed plot
julia
validation_fig = plot_forecast_vs_truth(
    validation_forecast;
    observed = keep_streams(validation_observed, reporting_cum_cols),
    baseline = keep_streams(validation_baseline, reporting_cum_cols),
    breaks = validation_breaks,
    individual = keep_streams(validation_individual, reporting_cum_cols)
);

The bed panel scores last week's projected occupancy against the beds occupied now (the dashed rule), with the individual (treatment-only) fit's own projection overlaid as a dotted step outline on the joint's own histogram bins.

Bed forecast-versus-observed plot
julia
validation_beds_fig = plot_forecast_beds_vs_truth(
    validation_forecast;
    isolation = _obs_beds, individual = validation_individual_isolation
);

The latent quantities are not observed, so they are scored distribution against distribution: what the frozen fit forecast for the past week's new infections, onsets and deaths against what the current fit now estimates for the same window.

Forecast-versus-now latent plot
julia
# Current fit's draws of the new latent counts over the past week, the last
# seven days of each cumulative-trajectory deterministic.
function _now_new(chn, key)
    mat = chn[key]
    trajs = [collect(v) for v in vec(collect(mat))]
    return Float64[t[end] - t[max(1, length(t) - 7)] for t in trajs]
end
now_latent = (;
    infections_new = _now_new(chn_joint, :cumulative_infections),
    onsets_new = _now_new(chn_joint, :cumulative_onsets),
    deaths_latent_new = _now_new(chn_joint, :cumulative_expected_deaths),
)

validation_latent_fig = plot_forecast_vs_truth_latent(
    validation_forecast; now = now_latent
);

Streams no longer reported ​

The situation reports have stopped updating some of the streams the model fits, listed with the date each was last reported below. The panels show what the frozen fit projected for those streams over the same week, without an observed rule, since the count they would be scored against has not moved since the stream stopped.

Forecast for the streams no longer reported
julia
# The last-reported date per stopped stream, and the frozen fit's own
# projection for them. `plot_forecast` draws a panel per new-count column
# the frame carries, so passing the stopped streams' columns alone gives the
# projection without the fabricated truth rule the validation figure would
# otherwise draw against a repeated total.
validation_stopped_streams = let s = stream_report_status(obs),
        ids = [stream_id(c) for c in stopped_cum_cols]

    keep = [r.stream in ids for r in eachrow(s)]
    DataFrame(
        "Stream" => s[keep, :label],
        "Last reported" => s[keep, :last_date]
    )
end
_stopped_new_cols = [
    c
        for c in new_cols(stopped_cum_cols)
        if c in propertynames(validation_forecast)
]
validation_stopped_fig = plot_forecast(
    validation_forecast[!, _stopped_new_cols]
);
StreamLast reported
Suspected cases2026-05-26
Suspected deaths2026-05-26

Forecast scoring across releases ​

Every release's saved one- to four-week-ahead forecast is scored against the data observed since, against a persistence baseline and, where one exists, the stream's own individual fit as well as the joint. The tables in this section are the joint model's, one row per stream. See forecast scoring against a persistence baseline for how the scores, the relative skill and the baseline are built. Recovered has no individual fit of its own, so its comparison is the baseline against the joint only. Reported cases and suspected deaths stopped being updated by the situation reports partway through the outbreak, and exports' confirmed-detection series is anchored to an earlier cut-off. Exports therefore contributes no scored forecast, and reported cases and suspected deaths each rest on exactly one matched forecast, a single window rather than a settled sample.

Only a minority of the daily releases examined contribute a row to the table below, each a reconstruction of an earlier model version rather than the current fit. Only the newest few releases carry the current model's own individual-stream forecasts, and the backfilled reconstructions carry none at all. Every row also rests on one to a handful of matched forecasts, shown as its own count rather than rounded away.

Four things are excluded from the scores here and in the frozen section below, each for a stated reason rather than for scoring badly, and the second of them applies to the frozen section alone.

  • One whole reconstruction (results-v1.6.0): its chain forecasts a near-zero median at every horizon and stream, with the upper predictive tail occasionally reaching five- and six-digit values, which is the signature of a chain that failed to sample rather than a forecast.

  • Frozen section only: the confirmed-death rows of the fourteen frozen reconstructions cut between 16 July and 15 August 2026, whose forecaster could not project that stream from its own trajectory and floored it at zero, so each carries a one-week median of exactly zero against an observed 250 to 370. Reconstructions cut after that window project the stream normally.

  • An onset window containing a vintage whose reread total falls, since its increment is not what the situation reports added.

  • A stream that carries no persistence baseline, which is what makes a window scoreable at all.

Nothing is dropped from the archive itself; data/forecast_scores*.csv and data/forecast_overlay*.csv record everything that was scored.

The symptom-onset stream is scored on the new reported count each vintage adds rather than on its level, because every vintage rereads the whole figure. Its printed total therefore moves with the scan error as well as with late reporting. On fourteen vintages the reread total falls, which a cumulative onset curve cannot do, and on many others it repeats unchanged. The scored truth cannot absorb that, since it is the increment between the vintages at the two ends of a window. A window containing a falling vintage is therefore left unscored, the rule the province scores already apply to a window holding a harmonisation-break day. It bites hardest at the longer horizons, a four-week window being more likely to contain a reread than a one-week one: the frozen onset row keeps three of its twenty-nine windows, all at one week, and the cross-release row six of thirty-eight. Read the onset row's skill against the baseline rather than its coverage, and read it as resting on a handful of windows.

Load and summarise the cross-release forecast scores
julia
forecast_scores_df = _release_data(
    "forecast_scores.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, crps = Float64,
        log_crps = Float64, dispersion = Float64, overprediction = Float64,
        underprediction = Float64, coverage_50 = Float64,
        coverage_90 = Float64,
        bias = Float64, n_samples = Int,
        log_rel_to_baseline = Float64,
    )
)
forecast_overlay_df = _release_data(
    "forecast_overlay.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, observed = Float64,
        median = Float64, lo30 = Float64, hi30 = Float64, lo60 = Float64,
        hi60 = Float64, lo90 = Float64, hi90 = Float64,
    )
)
# The digitised onset triangle's own per-vintage total, as calendar dates,
# and the windows it makes unscoreable. A vintage that rereads the figure
# lower gives a window an increment the reports did not add, so the window
# is dropped from the scores and the overlay alike (see
# `drop_rescanned_onset_windows`). The current triangle is used to judge
# every release, since the scored truth is read off this one series.
_onset_vintage_dates = grid_date.(obs.onset_report_history.days)
_onset_vintage_totals = obs.onset_report_history.counts
_drop_rescanned(tbl) = drop_rescanned_onset_windows(
    tbl; vintage_dates = _onset_vintage_dates,
    vintage_totals = _onset_vintage_totals
)
forecast_scores_df = _drop_rescanned(forecast_scores_df)
forecast_overlay_df = _drop_rescanned(forecast_overlay_df)

# One row per (stream, fit) pooled over every horizon and release. The
# by-horizon and by-release detail tables carry the same columns at a finer
# grain (see src/scoring.jl). Every fit is kept here, since the
# relative-skill figure below compares the roles against each other. The
# tables rendered in this section select the joint role, and the individual
# fits are tabulated in their own section.
forecast_score_overview_table = forecast_score_overview(forecast_scores_df)
forecast_score_by_horizon_table = forecast_score_by_horizon(forecast_scores_df)
forecast_score_by_release_table = forecast_score_by_release(forecast_scores_df)

joint_score_overview_table = select_fit_role(
    forecast_score_overview_table, "joint"
)
joint_score_by_horizon_table = select_fit_role(
    forecast_score_by_horizon_table, "joint"
)
# The trailing `;` on this last assignment matters: without it, this whole
# setup chunk's last statement (the DataFrame it assigns) is Literate's
# implicitly displayed "result" for the chunk, on top of the deliberate
# display further down -- and a bare DataFrame is html-showable, so it
# goes out as a second, undisplayed-in-source `@raw html` block that (for
# a table this size) can itself hit the PCRE limit described above.
joint_score_by_release_table = select_fit_role(
    forecast_score_by_release_table, "joint"
);

The headline pools every horizon and release into one row per stream for the joint model: the mean CRPS and its decomposition, coverage, bias, and the relative skill against the persistence baseline, on both the natural and the log scale. Each row also carries relative skill against the stream's own individual fit where one exists.

streamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed casesjoint103877.866.420.221.970.750.7793.5883.850.430.690.930.32
confirmed deathsjoint101869.088.920.4582.940.80.95764.919.1785.010.360.91-0.14
isolation bedsjoint9882.461.460.1011.350.850.6852.6324.325.510.660.950.2
onset reportsjoint24131.950.940.1960.90.650.6178.1412.0341.780.710.96-0.1
recoveredjoint14111.511.670.3061.196.16510.3611-0.14

The same relative skill against the baseline, by horizon: one panel per stream, one series per fit role, on a log-scaled skill axis with the reference line at one.

julia
forecast_relative_skill_fig = plot_forecast_relative_skill(
    forecast_score_by_horizon_table
);

What that error is made of, by horizon: the mean CRPS split into its width, its overprediction and its underprediction, one stacked bar per horizon and fit role.

julia
forecast_crps_by_horizon_fig = plot_forecast_crps_by_horizon(
    forecast_score_by_horizon_table
);

Scores by horizon
streamhorizonfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases7joint31115.742.740.242.770.660.6669.0746.050.620.650.840.37
confirmed cases14joint28228.722.830.222.610.670.67153.3274.920.480.710.930.3
confirmed cases21joint24405.612.430.1931.560.740.71312.5792.780.260.7510.29
confirmed cases28joint203534.6510.890.2221.270.850.813390.16144.230.260.6510.3
confirmed deaths7joint3174.772.930.484.391.060.9627.6211.8435.310.390.81-0.08
confirmed deaths14joint27166.862.720.4363.360.970.9168.0523.1175.690.30.89-0.09
confirmed deaths21joint23304.892.380.4432.50.810.89168.6920.27115.920.261-0.18
confirmed deaths28joint203697.0516.630.4721.970.681.073534.0823.91139.070.51-0.23
isolation beds7joint2973.931.780.0891.630.730.5635.5732.256.110.480.930.29
isolation beds14joint2683.761.680.0991.510.880.6950.3828.744.640.620.960.21
isolation beds21joint2383.221.280.1021.20.890.7361.3315.716.170.830.960.12
isolation beds28joint2092.271.190.121.170.940.7970.2816.975.030.80.950.13
onset reports7joint1085.760.640.2480.750.750.6255.053.9226.780.90.9-0.09
onset reports14joint8142.651.390.1661.40.620.6278.115.6248.930.621-0.07
onset reports21joint6194.670.970.150.940.580.58116.6720.7757.230.51-0.15
recovered7joint618.111.50.1851.514.790.732.5911-0.09
recovered14joint389.71.950.3931.4668.015.8115.8711-0.23
recovered21joint3297.163.260.4221.16269.916.0511.211-0.09
recovered28joint2145.990.650.3670.57121.86024.1311-0.26

The same relative skill against the baseline, release by release, so a run of releases that lost to the baseline reads as a run rather than as an average.

julia
forecast_skill_by_cutoff_fig = plot_forecast_skill_by_cutoff(
    forecast_score_by_release_table;
    title = "Relative skill against the baseline, by release"
);

Scores by release
made_datestreamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
2026-06-07confirmed casesjoint2363.9112.530.91310.43341.0622.860110.17
2026-06-10confirmed casesjoint3534.876.730.7915.08435.6799.20110.27
2026-07-01confirmed casesjoint415862.7874.230.5832.8215377.31485.470010.66
2026-07-06confirmed casesjoint4421.281.320.2690.96311.85109.4300.510.49
2026-07-08confirmed casesjoint4390.981.220.2530.96239.5151.4700.2510.52
2026-07-23confirmed casesjoint4335.570.880.1690.671.020.82299.2735.680.62110.13
2026-07-25confirmed casesjoint4222.50.980.1340.980.640.63218.044.460110.07
2026-07-26confirmed casesjoint4235.691.120.1411.120.790.67229.655.680.36110.06
2026-07-27confirmed casesjoint4247.971.280.1421.120.930.83240.087.880110.11
2026-07-31confirmed casesjoint4259.642.130.1491.930.640.68235.7323.760.15110.13
2026-08-01confirmed casesjoint4231.262.020.1392.030.690.66226.844.310.11110.05
2026-08-02confirmed casesjoint4278.452.020.1451.680.940.85263.7714.680110.14
2026-08-03confirmed casesjoint4247.581.280.140.940.850.8237.889.70110.08
2026-08-04confirmed casesjoint4221.562.430.1332.40.630.69220.70.860110.03
2026-08-07confirmed casesjoint4213.542.70.142.20.690.73209.7603.7811-0.09
2026-08-11confirmed casesjoint4197.494.430.1353.920.590.71191.5305.9711-0.13
2026-08-15confirmed casesjoint4387.776.540.1933.81.240.91294.0293.750110.37
2026-08-17confirmed casesjoint4458.716.140.2223.541.530.97290.38168.3300.510.46
2026-08-22confirmed casesjoint4336.095.030.1783.5510.87228.94107.1500.7510.45
2026-08-24confirmed casesjoint4246.114.040.1443.420.780.77118.34127.770010.62
2026-08-25confirmed casesjoint4198.573.210.122.790.660.65108.4890.10010.56
2026-08-30confirmed casesjoint4155.681.820.0891.440.40.4499.6855.9900.7510.38
2026-08-31confirmed casesjoint394.411.80.0721.490.460.4575.3519.060110.22
2026-09-01confirmed casesjoint377.231.260.0611.030.320.3568.48.840110.13
2026-09-06confirmed casesjoint3175.272.570.1292.340.70.6581.5793.700.3310.58
2026-09-10confirmed casesjoint2160.441.590.1731.490.740.7851.24109.20010.78
2026-09-12confirmed casesjoint2332.382.210.361.910.460.5264.18268.210000.95
2026-09-13confirmed casesjoint2359.112.630.3852.320.540.5871.93287.180000.95
2026-09-15confirmed casesjoint1227.822.30.4162.020.560.6243.47184.360000.97
2026-09-16confirmed casesjoint1217.342.780.4012.420.580.6442.14175.20000.99
2026-09-19confirmed casesjoint1300.576.040.5474.861.011.0146.48254.090000.99
2026-06-07confirmed deathsjoint141.491.730.9032.3936.9404.5511-0.2
2026-06-10confirmed deathsjoint264.932.280.7353.0662.70.991.2411-0.03
2026-07-01confirmed deathsjoint41719679.780.5491.1917103.1192.890110.28
2026-07-06confirmed deathsjoint4197.20.910.8712.37110.35086.850.251-0.49
2026-07-08confirmed deathsjoint4177.590.80.7131.8984.29093.30.251-0.61
2026-07-23confirmed deathsjoint4271.491.750.4842.60.391.08172.61098.8811-0.45
2026-07-25confirmed deathsjoint4241.832.020.5043.451.022.15106.460135.3701-0.57
2026-07-26confirmed deathsjoint4268.462.40.5544.391.052.17101.590166.8701-0.64
2026-07-27confirmed deathsjoint4249.472.320.5344.070.82.12109.30140.1701-0.59
2026-07-31confirmed deathsjoint4274.552.370.564.160.881.94108.790165.760.251-0.58
2026-08-01confirmed deathsjoint4313.482.680.6834.581.122.794.480219.0101-0.68
2026-08-02confirmed deathsjoint4258.172.750.5074.371.011.83128.190129.980.251-0.53
2026-08-03confirmed deathsjoint4268.891.940.582.891.22.37123.560145.330.251-0.55
2026-08-04confirmed deathsjoint4298.064.270.65681.22.3293.540204.5201-0.67
2026-08-07confirmed deathsjoint4366.295.110.888.341.582.7176.060290.2300.75-0.79
2026-08-11confirmed deathsjoint4301.1713.150.79824.011.232.3796.550204.6101-0.7
2026-08-15confirmed deathsjoint4205.935.140.325.320.840.94180.35025.5811-0.26
2026-08-17confirmed deathsjoint4202.494.110.3044.030.811.02187.02015.4711-0.21
2026-08-22confirmed deathsjoint4178.9820.3052.460.740.63156.16022.8211-0.26
2026-08-24confirmed deathsjoint492.941.350.1051.240.420.1945.4247.5100.2510.57
2026-08-25confirmed deathsjoint4741.080.0841.050.30.1145.0828.9200.7510.41
2026-08-30confirmed deathsjoint475.880.850.0920.930.290.138.0537.8300.510.48
2026-08-31confirmed deathsjoint360.230.920.0990.990.30.1329.930.330010.57
2026-09-01confirmed deathsjoint353.541.060.0851.110.260.0929.1324.4100.3310.54
2026-09-06confirmed deathsjoint394.072.170.1412.030.40.1229.8764.200.3310.65
2026-09-10confirmed deathsjoint296.571.290.2221.190.780.320.6575.92000.50.9
2026-09-12confirmed deathsjoint2122.21.620.2891.412.252.1522.2999.920000.94
2026-09-13confirmed deathsjoint2127.431.760.31.62.232.0527.4999.950000.92
2026-09-15confirmed deathsjoint176.481.880.3091.772.892.7513.5862.90001
2026-09-16confirmed deathsjoint179.121.730.3271.622.772.6114.964.220000.98
2026-09-19confirmed deathsjoint182.475.280.3294.645.334.6812.8469.620000.95
2026-07-01isolation bedsjoint490.191.450.1381.4813.48076.7100-0.99
2026-07-06isolation bedsjoint442.870.660.0660.6721.07021.801-0.64
2026-07-08isolation bedsjoint454.751.510.0731.5111.6743.080010.8
2026-07-23isolation bedsjoint480.062.750.1012.661.441.1429.3950.670010.58
2026-07-25isolation bedsjoint480.842.10.1052.061.561.2832.6548.1900.510.53
2026-07-26isolation bedsjoint473.831.990.0971.931.20.9636.9636.8700.510.46
2026-07-27isolation bedsjoint471.582.470.0932.371.20.9539.5232.0600.510.44
2026-07-31isolation bedsjoint471.051.670.0881.620.860.6751.6919.3600.7510.34
2026-08-01isolation bedsjoint4721.130.0941.10.810.6164.517.480.02110.18
2026-08-02isolation bedsjoint483.691.650.1021.50.990.7252.6731.0200.7510.39
2026-08-03isolation bedsjoint478.831.740.0991.610.990.7460.4218.410110.3
2026-08-04isolation bedsjoint4100.741.480.1371.421.160.9863.4737.2800.7510.38
2026-08-07isolation bedsjoint483.811.390.1221.550.80.7169.03014.7811-0.24
2026-08-11isolation bedsjoint474.710.440.1090.440.730.6472.4802.2311-0.08
2026-08-15isolation bedsjoint480.960.940.0920.850.630.4571.729.240110.17
2026-08-17isolation bedsjoint495.081.390.1061.210.920.6667.3527.740110.33
2026-08-22isolation bedsjoint493.151.50.0981.30.640.4578.9814.180110.23
2026-08-24isolation bedsjoint490.981.810.0961.540.720.5366.324.6700.7510.34
2026-08-25isolation bedsjoint475.541.10.0820.960.570.4371.773.740.02110.08
2026-08-30isolation bedsjoint480.951.850.0891.650.630.5262.6118.3300.7510.3
2026-08-31isolation bedsjoint360.951.330.0691.260.520.4354.186.230.54110.15
2026-09-01isolation bedsjoint370.871.850.0871.970.610.5757.32013.5511-0.22
2026-09-06isolation bedsjoint362.141.320.0741.290.530.4955.520.66.0211-0.11
2026-09-10isolation bedsjoint253.641.330.0611.290.40.3544.40.278.9611-0.18
2026-09-12isolation bedsjoint2144.422.340.1512.131.090.8462.0182.4100.510.58
2026-09-13isolation bedsjoint2173.543.250.1792.921.561.2465.75107.790010.69
2026-09-15isolation bedsjoint192.682.730.0942.550.830.6651.5241.160010.57
2026-09-16isolation bedsjoint194.953.970.0953.570.840.6656.5138.440010.53
2026-09-19isolation bedsjoint1266.1340.2823.542.632.1153.06213.080000.91
2026-08-02onset reportsjoint380.540.590.10.610.760.5977.9102.6311-0.12
2026-08-03onset reportsjoint387.721.370.1190.990.890.7776.4911.230110.22
2026-08-04onset reportsjoint384.340.680.1310.630.610.574.122.327.9111-0.01
2026-08-07onset reportsjoint3278.250.850.3870.840.670.59690209.2500.67-0.83
2026-08-11onset reportsjoint3194.711.180.2871.210.80.6783.450111.260.331-0.6
2026-08-15onset reportsjoint3172.591.940.1591.330.470.57107.0565.5400.3310.44
2026-08-17onset reportsjoint2103.331.690.1341.30.420.5378.8324.50110.28
2026-08-22onset reportsjoint268.520.620.1350.770.580.6667.4401.0811-0.08
2026-08-24onset reportsjoint164.90.960.3061.90.760.7857.507.4111-0.21
2026-08-25onset reportsjoint163.70.270.3150.540.670.6261.192.510110.13
2026-07-01recoveredjoint3312.696.630.4461.94289.5523.140110.27
2026-07-06recoveredjoint485.760.80.4070.8868.39017.3711-0.33
2026-07-08recoveredjoint463.080.760.292144.51018.5711-0.38
2026-09-15recoveredjoint18.921.050.0490.968.810.110110.09
2026-09-16recoveredjoint18.660.80.0490.78.250.410110.15
2026-09-19recoveredjoint110.180.660.0560.638.8901.2911-0.23

Forecasts made at each release against the value observed since, one panel per stream and horizon, the observed value in black. The median and 90% interval are coloured by fit role: the persistence baseline, the stream's individual fit and the joint. The x-axis is the date each forecast was made, so an incident stream's observed window pairs unambiguously with the forecast that made it. Each panel's axis is cropped to a small multiple of what that stream actually reached, so one very wide interval cannot squash every other series flat. An interval or median too wide for the panel is clamped at the top and marked with an open triangle rather than silently cut off.

Forecasts-versus-now overlay
julia
forecast_overlay_fig = plot_forecast_overlay(
    scored_overlay(forecast_overlay_df)
);

Frozen-fit forecast evaluation ​

The current model, frozen at earlier data cut-offs (see Forecast-versus-frozen evaluation), is scored the same way as the cross-release forecasts above, against the same persistence baseline. The tables in this section are the frozen joint model's, one row per stream. Each stream's own frozen fit is also scored where one exists, at the one-week-back cut-off and for still-reported streams only, and is carried by the skill figures rather than by the tables. Every other cut-off carries the joint alone. The May cut-offs predate the first reported bed occupancy and the first reported recoveries, so those windows are left unscored rather than scored against a series that had not started. The baseline carries a weaker data-vintage guarantee than the cross-release one, since its snapshot was taken weeks after the frozen cut-off and can hold later revisions to earlier days (see forecast scoring against a persistence baseline).

Load and summarise the frozen-fit forecast scores
julia
frozen_scores_df = _release_data(
    "forecast_scores_frozen.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, crps = Float64,
        log_crps = Float64, dispersion = Float64, overprediction = Float64,
        underprediction = Float64, coverage_50 = Float64,
        coverage_90 = Float64,
        bias = Float64, n_samples = Int,
        log_rel_to_baseline = Float64,
    )
)
frozen_overlay_df = _release_data(
    "forecast_overlay_frozen.csv",
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, observed = Float64,
        median = Float64, lo30 = Float64, hi30 = Float64, lo60 = Float64,
        hi60 = Float64, lo90 = Float64, hi90 = Float64,
    )
)
# Rows a superseded forecaster produced are dropped before anything is
# summarised or drawn, from the scores and the overlay alike, so the
# tables and the figures rest on one set of rows. See
# `drop_superseded_forecasts` for the one exclusion in force and why.
frozen_scores_df = _drop_rescanned(
    drop_superseded_forecasts(frozen_scores_df)
)
frozen_overlay_df = _drop_rescanned(
    drop_superseded_forecasts(frozen_overlay_df)
)

# The frozen joint carries `FROZEN_FIT`, so it is named as the joint role
# here and compared against each stream's own frozen fit. A release
# published before the archive had a `fit` column carries joint rows only.
frozen_score_overview_table = forecast_score_overview(
    frozen_scores_df; joint_fit = FROZEN_FIT
)
frozen_score_by_horizon_table = forecast_score_by_horizon(
    frozen_scores_df; joint_fit = FROZEN_FIT
)
frozen_score_by_release_table = forecast_score_by_release(
    frozen_scores_df; joint_fit = FROZEN_FIT
)

# The tables show the frozen joint alone, as the cross-release tables show
# the joint alone, so a row reads as one model at one cut-off rather than a
# stream interleaving two fits. The archive's single-stream frozen fits stay
# in the scored data and in the figures, which compare the roles against
# each other. Selecting the joint role leaves `fit` single-valued, so it is
# dropped and the model named in the prose instead.
_frozen_joint_only(tbl) = drop_degenerate_fit_column(
    select_fit_role(tbl, "joint")
)
frozen_score_overview_display = _frozen_joint_only(
    frozen_score_overview_table
)
frozen_score_by_horizon_display = _frozen_joint_only(
    frozen_score_by_horizon_table
)
frozen_score_by_release_display = _frozen_joint_only(
    frozen_score_by_release_table
)

# One row per release for the cut-offs more than one release forecast.
# See the comment above `joint_score_by_release_table`'s assignment for why
# this setup chunk's last statement needs a trailing `;`.
frozen_score_by_vintage_table = forecast_score_by_vintage(
    frozen_scores_df; joint_fit = FROZEN_FIT
)
frozen_score_by_vintage_display = _frozen_joint_only(
    frozen_score_by_vintage_table
);
streamncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases255291.312.430.3641.150.470.54149.09125.3216.90.40.780.27
confirmed deaths19947.520.880.3640.572.332.2827.8414.744.940.540.87-0.04
isolation beds22490.60.790.1420.590.720.5847.3525.2118.050.50.980.1
onset reports29142.321.010.2290.8271.3820.7350.210.550.93-0.13
recovered100298.925.340.5443.43150.48145.4530.560.680.4

The same relative skill against the baseline, by horizon, for the frozen cut-offs.

julia
frozen_relative_skill_fig = plot_forecast_relative_skill(
    frozen_score_by_horizon_table
);

What that error is made of, by horizon, as in the cross-release section above.

julia
frozen_crps_by_horizon_fig = plot_forecast_crps_by_horizon(
    frozen_score_by_horizon_table;
    title = "CRPS decomposition by horizon, frozen cut-offs"
);

Scores by horizon
streamhorizonncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases712493.741.110.3960.780.480.5534.3624.6334.740.390.7-0.06
confirmed cases1457261.693.830.2843.380.460.52139.09122.600.460.890.57
confirmed cases2154608.33.920.422.66284.95323.3500.260.780.66
confirmed cases2820744.781.890.2341.16522.06222.7200.710.41
confirmed deaths711020.070.810.5020.612.442.367.134.048.910.470.78-0.35
confirmed deaths144347.711.110.1740.72.222.131.5516.110.050.790.980.3
confirmed deaths214099.370.730.220.3570.3629.0100.5510.36
confirmed deaths286203.511.560.161.3997.57105.940010.62
isolation beds76257.551.320.0930.930.520.4429.4325.322.80.630.920.3
isolation beds145780.581.560.1311.111.240.9743.9333.633.020.6710.33
isolation beds215498.570.590.1570.4553.7919.1925.590.371-0.11
isolation beds2851133.560.620.1980.4566.1322.0245.40.311-0.18
onset reports71596.140.680.2660.6550.3514.431.40.530.93-0.13
onset reports1410144.891.360.1711.2880.817.3846.70.70.9-0.07
onset reports214309.051.380.2381.33126.7152.85129.490.251-0.27
recovered731118.315.890.5343.9242.6374.071.610.450.810.46
recovered1426243.968.620.5485.3396.3145.182.480.540.580.43
recovered2123362.645.030.5433.04173.67185.473.50.610.610.37
recovered2820577.054.480.5572.28361.39210.435.230.70.70.29

The same relative skill against the baseline, cut-off by cut-off, pooled over the horizons each cut-off forecast.

julia
frozen_skill_by_cutoff_fig = plot_forecast_skill_by_cutoff(
    frozen_score_by_release_table;
    xlabel = "Frozen cut-off",
    title = "Relative skill against the baseline, by frozen cut-off"
);

Scores by frozen cut-off
made_datestreamncrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
2026-05-23confirmed cases3128.370.340.1840.319.4708.90.681-0.33
2026-05-27confirmed cases31141.720.790.9430.7612.040129.6800.1-0.94
2026-06-08confirmed cases93362.547.340.3984.17155.7206.8500.250.830.68
2026-07-16confirmed cases4294.680.470.1680.29283.4610.980.24110.07
2026-07-18confirmed cases4361.260.780.1850.52333.8527.410110.15
2026-07-19confirmed cases4439.621.090.2090.72351.8387.790110.3
2026-07-20confirmed cases4385.620.870.1850.56332.653.020110.22
2026-07-24confirmed cases4422.921.340.1920.95343.779.220110.32
2026-07-25confirmed cases4460.722.030.2091.54356.82103.910110.34
2026-07-26confirmed cases4315.561.480.1641.27284.5930.970110.24
2026-07-27confirmed cases4346.391.760.1761.33306.9739.420110.27
2026-07-28confirmed cases4396.781.340.1560.84361.5235.260110.22
2026-07-31confirmed cases4498.384.080.2132.74369.58128.800.7510.38
2026-08-04confirmed cases4281.572.970.1562.7253.9927.590110.18
2026-08-08confirmed cases4253.513.220.152.27246.676.120.72110.03
2026-08-10confirmed cases4244.513.740.152.782411.71.8111-0.01
2026-08-15confirmed cases4623.411.740.2816.2384.1239.300.2510.52
2026-08-17confirmed cases4426.555.490.2343.51166.54260.010010.75
2026-08-18confirmed cases4468.253.910.2762.8155.86312.4000.750.85
2026-08-23confirmed cases4562.747.990.3146.34140.78421.96000.750.88
2026-08-24confirmed cases4437.046.820.2465.48160.51276.53000.750.82
2026-08-25confirmed cases4288.324.340.1764.06113.18175.140010.74
2026-08-30confirmed cases4179.542.210.1061.8110.0569.4900.7510.44
2026-09-03confirmed cases3199.663.570.1413.0873.77125.890010.68
2026-09-05confirmed cases386.861.330.071.2971.9914.410.46110.12
2026-09-06confirmed cases3346.214.680.2273.88114.16232.050010.71
2026-09-08confirmed cases2177.683.460.1722.960.30.3373.15104.530010.68
2026-09-09confirmed cases2253.513.20.2462.750.420.4379.73173.78000.50.81
2026-09-12confirmed cases2381.772.590.412.20.530.5965.93315.840000.96
2026-09-14confirmed cases1223.91.760.4071.60.540.6231.54192.360000.98
2026-09-15confirmed cases1255.022.340.4632.070.630.6933.78221.240001
2026-09-18confirmed cases1227.712.720.4612.390.580.6627.64200.070000.99
2026-09-19confirmed cases1183.324.120.3653.580.620.6833.5149.820000.97
2026-09-20confirmed cases1130.582.510.282.310.450.5126.24104.340000.94
2026-05-23confirmed deaths318.050.330.350.253.4704.590.421-0.51
2026-05-27confirmed deaths3127.530.851.1640.972.15025.3900.42-0.92
2026-06-08confirmed deaths9347.970.70.2060.4639.368.090.520.8810.12
2026-08-17confirmed deaths491.871.810.0971.2759.332.5700.510.44
2026-08-18confirmed deaths4115.91.320.1341.0255.9159.990010.63
2026-08-23confirmed deaths4145.191.720.1661.5151.7193.480010.72
2026-08-24confirmed deaths4110.911.550.121.3755.255.7100.2510.53
2026-08-25confirmed deaths487.421.250.11.2244.6542.7700.2510.53
2026-08-30confirmed deaths469.430.840.0870.9339.7529.6800.510.5
2026-09-03confirmed deaths366.071.50.0961.329.0637.0100.3310.48
2026-09-05confirmed deaths337.510.920.0660.8628.767.491.26110.1
2026-09-06confirmed deaths368.491.540.1071.4730.7737.30.420.6710.35
2026-09-08confirmed deaths263.061.870.1291.711.761.5927.0136.0500.510.62
2026-09-09confirmed deaths285.221.760.1791.672.392.0728.3256.890010.76
2026-09-12confirmed deaths2114.031.540.2651.331.961.8123.4490.590000.94
2026-09-14confirmed deaths160.691.440.2411.392.152.111.7148.980000.95
2026-09-15confirmed deaths175.861.660.3091.582.622.511.2964.570000.99
2026-09-18confirmed deaths190.942.090.4061.922.692.511.279.750001
2026-09-19confirmed deaths170.775.180.294.635.274.6710.7160.060000.97
2026-09-20confirmed deaths169.463.920.2993.563.623.3511.0758.390001
2026-06-08isolation beds12485.20.510.1560.446.410.0828.720.521-0.11
2026-07-16isolation beds473.222.350.0962.2420.0153.220010.6
2026-07-18isolation beds477.782.450.1032.3318.7159.070010.61
2026-07-19isolation beds4100.013.480.1323.3122.3877.630010.64
2026-07-20isolation beds482.232.790.1062.6419.8962.3300.2510.61
2026-07-24isolation beds476.612.60.0962.4410.1466.470010.73
2026-07-25isolation beds4106.432.830.1362.7313.8492.59000.750.73
2026-07-26isolation beds494.882.430.1222.2922.6172.2700.250.750.63
2026-07-27isolation beds485.562.770.1132.6838.9246.6500.510.48
2026-07-28isolation beds4153.082.90.2012.641.15111.9400.50.750.67
2026-07-31isolation beds4119.063.040.1392.7841.3677.700.50.750.57
2026-08-04isolation beds4110.511.450.1531.3963.2647.2400.750.750.42
2026-08-08isolation beds459.170.930.0780.9551.547.150.480.7510.16
2026-08-10isolation beds455.991.130.0791.1855.830.020.15110
2026-08-15isolation beds4109.361.220.1151.0174.0735.30110.34
2026-08-17isolation beds4103.62.030.1111.7372.6230.9800.7510.43
2026-08-18isolation beds493.51.230.11.0371.3122.1900.7510.34
2026-08-23isolation beds4145.693.680.1493.1560.6785.020010.66
2026-08-24isolation beds4108.372.490.1132.1168.4739.900.510.46
2026-08-25isolation beds471.11.040.0871.0260.24010.8611-0.27
2026-08-30isolation beds471.871.640.0941.7662.2509.6211-0.19
2026-09-03isolation beds368.390.850.0850.8343.71024.680.671-0.42
2026-09-05isolation beds375.381.730.0941.8454.64020.740.671-0.34
2026-09-06isolation beds3234.885.050.5199.35150.55084.340.671-0.44
2026-09-08isolation beds272.91.390.0761.240.50.3864.616.771.52110.05
2026-09-09isolation beds290.521.790.0891.480.660.4769.7720.750110.3
2026-09-12isolation beds2172.613.080.182.781.381.152.96119.650010.72
2026-09-14isolation beds193.691.590.1011.540.840.736.9456.750010.75
2026-09-15isolation beds166.120.0691.910.550.4637.7228.380010.52
2026-09-18isolation beds155.421.080.0621.060.460.4328.7526.670010.57
2026-09-19isolation beds177.621.170.091.140.730.6838.4939.130010.57
2026-09-20isolation beds137.351.50.0481.590.320.3332.2305.1211-0.21
2026-07-26onset reports267.832.310.1192.0865.222.610110.13
2026-07-27onset reports268.121.060.1210.8656.661.4610.0111-0.15
2026-07-28onset reports2790.50.1290.3861.396.9410.6711-0.07
2026-07-31onset reports2117.290.690.2390.675.7241.5800.510.4
2026-08-04onset reports3940.750.1370.6467.91.4524.6511-0.14
2026-08-08onset reports3336.610.730.4730.6371.380265.2300.33-0.88
2026-08-10onset reports3198.521.850.3091.866.10132.4101-0.69
2026-08-15onset reports3266.423.050.2211.9135.96130.460010.66
2026-08-17onset reports283.461.30.1781.6171.82011.6411-0.25
2026-08-18onset reports2114.861.390.241.778.01036.850.51-0.33
2026-08-23onset reports164.410.950.2221.3658.2306.1811-0.2
2026-08-24onset reports170.40.940.3221.8462.2508.1511-0.21
2026-08-25onset reports1124.180.550.360.6535.2888.90010.81
2026-09-09onset reports169.820.50.1280.2358.4611.360110.24
2026-09-20onset reports150.630.520.150.2614.2036.4301-0.76
2026-07-16recovered4139.341.070.4691.06135.503.8411-0.1
2026-07-18recovered4149.612.250.4552.19146.942.670110.1
2026-07-19recovered4181.472.290.4822.16179.352.120110.07
2026-07-20recovered4171.812.040.4171.84169.632.180110.06
2026-07-24recovered4179.372.380.4092.01175.364.010110.13
2026-07-25recovered4177.72.080.3791.7173.524.180110.12
2026-07-26recovered4139.991.560.3811.66138.990111-0.05
2026-07-27recovered4219.872.180.4831.85218.3801.4911-0.06
2026-07-28recovered4544.073.960.3690.9543.250.50.33110.02
2026-07-31recovered4249.162.450.4021.51242.916.250110.12
2026-08-04recovered4157.982.250.4572.57146.41011.5711-0.19
2026-08-08recovered4125.591.620.4172.14109.72015.8711-0.23
2026-08-10recovered4157.11.850.5372.38116.31040.7911-0.32
2026-08-15recovered4384.4511.890.4425.52335.6248.830110.29
2026-08-17recovered4548.5822.70.7799.99117.93430.640000.94
2026-08-18recovered4596.2724.450.83411.4199.02497.250000.95
2026-08-23recovered4553.2928.690.73914.14114.01439.28000.250.92
2026-08-24recovered4518.5828.30.72614.89112.37406.220000.94
2026-08-25recovered4500.2432.490.71616.4890.53409.710000.96
2026-08-30recovered4436.8824.220.63513.1697.38339.5000.250.92
2026-09-03recovered3399.7131.330.70617.6272.05327.66000.330.9
2026-09-05recovered3306.8221.730.59412.769.91236.9000.330.9
2026-09-06recovered3205.0214.240.3888.1775.87129.15000.670.73
2026-09-08recovered2307.3528.320.71815.8261.07246.29000.50.88
2026-09-09recovered2325.7817.680.81410.9460.31265.480000.95
2026-09-12recovered2291.7327.950.67515.1764.47227.26000.50.9
2026-09-14recovered1152.3113.740.5588.4931.66120.650000.9
2026-09-15recovered1156.4317.20.5810.5238.33118.10010.86
2026-09-18recovered1166.4519.950.63212.7828.76137.690010.89
2026-09-19recovered1164.6810.930.576.5833.81130.860010.86
2026-09-20recovered1142.615.690.5129.8837.44105.160010.78

Frozen skill by release ​

Skill at each cut-off more than one release forecast, one point per release rather than pooled across releases. Releases run in the order they were cut, evenly spaced rather than to calendar scale.

Frozen skill per release
julia
frozen_skill_by_vintage_fig = plot_forecast_skill_by_vintage(
    frozen_score_by_vintage_table
);

Frozen scores by release
streamreleaserelease_datencrpsrel_to_baselinelog_crpslog_rel_to_baselinerel_to_individuallog_rel_to_individualdispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed casesresults-13592026-07-165242.130.4541.08121.4495.4625.20.610.07
confirmed casesresults-13912026-07-185242.12.940.4541.04121.4495.4625.20.610.07
confirmed casesresults-13942026-07-195242.130.4541.07121.4495.4625.20.610.07
confirmed casesresults-14412026-07-205245.993.050.4661.08113.62106.9225.450.20.80.16
confirmed casesresults-14462026-07-245245.992.940.4661.07113.62106.9225.450.20.80.16
confirmed casesresults-v1.12.02026-07-255245.992.970.4661.08113.62106.9225.450.20.80.16
confirmed casesresults-14622026-07-265247.613.030.4581.06122.2799.4625.890.60.80.13
confirmed casesresults-14692026-07-275247.612.960.4581.07122.2799.4625.890.60.80.13
confirmed casesresults-14792026-07-285247.612.970.4581.04122.2799.4625.890.60.80.13
confirmed casesresults-14892026-07-315247.613.040.4581.06122.2799.4625.890.60.80.13
confirmed casesresults-v1.13.22026-08-045247.613.060.4581.08122.2799.4625.890.60.80.13
confirmed casesresults-15052026-08-085247.612.990.4581.06122.2799.4625.890.60.80.13
confirmed casesresults-15272026-08-105247.613.170.4581.08122.2799.4625.890.60.80.13
confirmed casesresults-15732026-08-155256.513.10.4711.08131.5899.0425.890.40.80.13
confirmed casesresults-15912026-08-175304.063.670.5091.1985.78190.6927.590.20.60.3
confirmed casesresults-v1.15.02026-08-185304.063.740.5091.1885.78190.6927.590.20.60.3
confirmed casesresults-v1.16.02026-08-235318.573.640.5131.1895.65195.7327.190.20.40.3
confirmed casesresults-16252026-08-245318.423.90.5131.2295.56195.7327.130.20.40.3
confirmed casesresults-v1.17.02026-08-255278.343.410.4711.182.69170.1325.510.20.40.28
confirmed casesresults-v1.18.02026-08-305278.343.430.4711.1182.69170.1325.510.20.40.28
confirmed casesresults-17392026-09-035288.73.50.4751.192.73169.1126.860.20.60.22
confirmed casesresults-18282026-09-055285.093.460.5051.19102.17151.5831.3500.60.15
confirmed casesresults-18582026-09-065290.993.570.5091.1888.02171.6231.3500.60.14
confirmed casesresults-19702026-09-085295.583.450.5181.1796.92166.931.7700.60.15
confirmed casesresults-v2.1.02026-09-095295.583.70.5181.2796.92166.931.7700.60.15
confirmed casesresults-21772026-09-125311.3840.5011.2106.74175.6928.9500.60.19
confirmed casesresults-23602026-09-145174.792.140.4220.9966.8474.4433.5100.80.07
confirmed casesresults-25062026-09-155140.971.730.3620.8658.1853.1829.610.20.80.02
confirmed casesresults-25362026-09-185135.091.60.3660.8448.9255.0831.090.20.80.03
confirmed casesresults-27202026-09-195161.241.980.3990.9656.2273.1231.8900.80.08
confirmed casesresults-v3.0.02026-09-205162.591.960.3980.9256.9674.2331.400.80.1
confirmed deathsresults-13592026-07-16536.610.690.5530.6928.040.078.490.60.8-0.39
confirmed deathsresults-13912026-07-18536.610.70.5530.7128.040.078.490.60.8-0.39
confirmed deathsresults-13942026-07-19536.610.70.5530.6928.040.078.490.60.8-0.39
confirmed deathsresults-14412026-07-20537.950.740.5790.7428.760.448.760.60.8-0.35
confirmed deathsresults-14462026-07-24537.950.740.5790.7428.760.448.760.60.8-0.35
confirmed deathsresults-v1.12.02026-07-25537.950.710.5790.7328.760.448.760.60.8-0.35
confirmed deathsresults-14622026-07-26537.320.70.6350.7927.4809.840.60.8-0.43
confirmed deathsresults-14692026-07-27537.320.730.6350.8227.4809.840.60.8-0.43
confirmed deathsresults-14792026-07-28537.320.710.6350.827.4809.840.60.8-0.43
confirmed deathsresults-14892026-07-31537.320.70.6350.7927.4809.840.60.8-0.43
confirmed deathsresults-v1.13.22026-08-04537.320.690.6350.7827.4809.840.60.8-0.43
confirmed deathsresults-15052026-08-08537.320.70.6350.827.4809.840.60.8-0.43
confirmed deathsresults-15272026-08-10537.320.70.6350.8127.4809.840.60.8-0.43
confirmed deathsresults-15732026-08-15537.790.710.6310.7827.840.149.820.60.8-0.4
confirmed deathsresults-15912026-08-17536.940.710.2880.3720.912.143.90.610
confirmed deathsresults-v1.15.02026-08-18536.940.720.2880.3720.912.143.90.610
confirmed deathsresults-v1.16.02026-08-23538.990.750.270.3424.1711.593.220.610.04
confirmed deathsresults-16252026-08-24538.960.780.270.3524.1511.593.220.610.05
confirmed deathsresults-v1.17.02026-08-25535.050.670.2560.3221.9610.142.940.61-0.03
confirmed deathsresults-v1.18.02026-08-30535.050.670.2560.3221.9610.142.940.61-0.03
confirmed deathsresults-17392026-09-03534.710.660.2740.3422.918.353.450.61-0.02
confirmed deathsresults-18282026-09-05528.720.550.2950.3820.393.265.070.60.8-0.21
confirmed deathsresults-18582026-09-06535.330.680.3140.421.978.265.10.60.8-0.17
confirmed deathsresults-19702026-09-08531.570.590.2960.3722.64.244.730.60.8-0.19
confirmed deathsresults-v2.1.02026-09-09531.570.60.2960.3722.64.244.730.60.8-0.19
confirmed deathsresults-21772026-09-12539.440.770.2930.3825.679.913.860.61-0.05
confirmed deathsresults-23602026-09-14541.870.80.2940.3823.5414.653.680.410.01
confirmed deathsresults-25062026-09-15528.50.550.2520.3220.714.243.550.81-0.1
confirmed deathsresults-25362026-09-18531.720.620.2750.3620.547.353.830.81-0.08
confirmed deathsresults-27202026-09-19533.370.640.2730.3520.938.923.520.61-0.01
confirmed deathsresults-v3.0.02026-09-20531.40.60.2610.3320.437.553.410.81-0.08
isolation bedsresults-13592026-07-16477.620.470.1580.4141.551.5434.540.51-0.22
isolation bedsresults-13912026-07-18477.620.460.1580.4141.551.5434.540.51-0.22
isolation bedsresults-13942026-07-19477.620.450.1580.441.551.5434.540.51-0.22
isolation bedsresults-14412026-07-20477.320.460.1580.4138.491.0237.810.51-0.22
isolation bedsresults-14462026-07-24477.320.450.1580.3938.491.0237.810.51-0.22
isolation bedsresults-v1.12.02026-07-25477.320.460.1580.438.491.0237.810.51-0.22
isolation bedsresults-14622026-07-26471.120.420.1440.3738.043.5329.560.51-0.2
isolation bedsresults-14692026-07-27471.120.420.1440.3738.043.5329.560.51-0.2
isolation bedsresults-14792026-07-28471.120.410.1440.3638.043.5329.560.51-0.2
isolation bedsresults-14892026-07-31471.120.410.1440.3638.043.5329.560.51-0.2
isolation bedsresults-v1.13.22026-08-04471.120.430.1440.3838.043.5329.560.51-0.2
isolation bedsresults-15052026-08-08471.120.440.1440.3938.043.5329.560.51-0.2
isolation bedsresults-15272026-08-10471.120.420.1440.3738.043.5329.560.51-0.2
isolation bedsresults-15732026-08-15476.50.470.1540.4135.782.1538.570.51-0.22
isolation bedsresults-15912026-08-17473.460.430.1470.3736.12.1335.230.51-0.2
isolation bedsresults-v1.15.02026-08-18473.460.450.1470.3936.12.1335.230.51-0.2
isolation bedsresults-v1.16.02026-08-23473.460.440.1470.3935.252.0536.160.51-0.2
isolation bedsresults-16252026-08-24473.460.440.1470.3835.252.0536.160.51-0.2
isolation bedsresults-v1.17.02026-08-25474.840.450.1510.3939.271.4434.130.51-0.23
isolation bedsresults-v1.18.02026-08-30474.840.440.1510.3939.271.4434.130.51-0.23
isolation bedsresults-17392026-09-03475.350.440.1550.3940.750.9833.630.51-0.24
isolation bedsresults-18282026-09-05473.290.440.1490.3935.511.8135.980.51-0.24
isolation bedsresults-18582026-09-06467.650.410.1360.3636.961.3129.390.751-0.2
isolation bedsresults-19702026-09-08476.590.460.1560.4137.540.8838.170.51-0.28
isolation bedsresults-v2.1.02026-09-09476.590.460.1560.4137.540.8838.170.51-0.28
isolation bedsresults-21772026-09-12476.80.460.1560.4134.071.341.430.51-0.24
isolation bedsresults-23602026-09-144205.61.190.2410.61126.5279.0800.2510.58
isolation bedsresults-25062026-09-154131.190.770.1790.4685.3645.8300.7510.46
isolation bedsresults-25362026-09-184101.70.610.1470.3972.2929.4100.7510.4
isolation bedsresults-27202026-09-194122.90.760.1670.4577.5545.3400.510.45
isolation bedsresults-v3.0.02026-09-204150.810.90.1990.5290.816000.510.54

The frozen forecasts made at each cut-off against the value observed since, one panel per stream and horizon, the observed value in black. Each panel carries the frozen forecast and the persistence baseline, coloured as in the cross-release overlay above, and the x-axis is the cut-off each forecast was made from.

Frozen-fit forecasts-versus-now overlay
julia
# The frozen joint and the persistence baseline only. A single-stream
# frozen fit exists at the one-week-back cut-off alone, so its series lands
# on one made date of a panel spanning every cut-off, overplotting the
# joint point it sits beside rather than reading as a second series. The
# single-stream frozen fits are compared against the joint in the skill
# figures and in the validation plot at that cut-off.
frozen_overlay_fig = plot_forecast_overlay(
    scored_overlay(
        vcat(
            select_fit_role(frozen_overlay_df, "joint"),
            select_fit_role(frozen_overlay_df, "baseline")
        )
    )
);

The frozen re-fits below freeze the renewal data to an earlier cut-off and re-fit, so that a change driven by newer data can be distinguished from one driven by a change of method. Each uses the full headline settings: 1000 draws across two chains.

Freeze the renewal data to a cut-off and re-fit
julia
# Frozen re-fits and released_df are prepared in the setup block above.

Individual fits against the baseline ​

This section carries the same cross-release forecast scoring as Forecast scoring across releases above, for each stream's own individual fit rather than the joint, against the same persistence baseline.

Individual-fit rows of the cross-release scores
julia
# The relative skill against a stream's individual fit is only ever
# computed on the joint model's row, so on these rows it is missing by
# construction and the column is dropped rather than shown empty.
individual_score_overview_table = drop_individual_fit_columns(
    select_fit_role(forecast_score_overview_table, "individual")
)
individual_score_by_horizon_table = drop_individual_fit_columns(
    select_fit_role(forecast_score_by_horizon_table, "individual")
)
# See the comment above `joint_score_by_release_table`'s assignment for why
# this setup chunk's last statement needs a trailing `;`.
individual_score_by_release_table = drop_individual_fit_columns(
    select_fit_role(forecast_score_by_release_table, "individual")
);
streamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed casesconfirmed86330.592.740.2332.55261.3467.891.350.910.920.18
confirmed deathsconfirmed_deaths86250.43.020.4333.7195.9323.7130.760.81-0.05
isolation bedstreatment86100.351.770.1512.0488.540.411.410.971-0.17
onset reportsonsets24208.321.490.3211.47110.9430.0767.310.621-0.12

The same relative skill against the baseline, by horizon, one panel per stream (dataset), for each stream's own individual fit.

julia
individual_relative_skill_fig = plot_forecast_relative_skill(
    individual_score_by_horizon_table;
    empty_message = "Empty: no release old enough for its targets to " *
        "have been observed carries an individual-stream " *
        "forecast. Not a missing forecast."
);

Scores by horizon
streamhorizonfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
confirmed cases7confirmed26151.943.40.2613.1380.7268.882.350.810.810.19
confirmed cases14confirmed23282.423.450.2343.07188.791.731.990.870.910.17
confirmed cases21confirmed20377.792.730.2012.34340.5836.80.41110.15
confirmed cases28confirmed17613.432.280.2271.75542.6570.690.09110.19
confirmed deaths7confirmed_deaths2672.42.650.3973.9949.023.3620.020.691-0.1
confirmed deaths14confirmed_deaths23171.683.060.4464.52125.9310.9334.820.741-0.09
confirmed deaths21confirmed_deaths20316.793.010.4773.87248.0523.0145.720.91-0.08
confirmed deaths28confirmed_deaths17551.053.090.4222.6454.0372.9524.070.9410.1
isolation beds7treatment26101.642.60.1563.0788.680.1912.7711-0.16
isolation beds14treatment2399.211.990.1482.2993.220.065.9311-0.11
isolation beds21treatment2098.641.470.1451.6686.560.3211.770.951-0.18
isolation beds28treatment17101.941.260.1531.4584.341.2916.320.881-0.27
onset reports7onsets10114.750.860.4011.21796.9328.820.91-0.05
onset reports14onsets8231.172.250.2682.26116.2843.9270.960.51-0.08
onset reports21onsets6333.791.660.261.62157.0350.16126.590.331-0.28
Scores by release
made_datestreamfitncrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
2026-07-23confirmed casesconfirmed4327.810.860.2060.82319.434.843.5511-0.03
2026-07-25confirmed casesconfirmed4346.241.520.2131.55343.882.30.06110.02
2026-07-26confirmed casesconfirmed4299.071.420.2111.67291.390.167.5111-0.12
2026-07-27confirmed casesconfirmed4267.621.380.1711.35262.035.380.21110.04
2026-07-31confirmed casesconfirmed4402.873.310.2192.82383.8219.050110.09
2026-08-01confirmed casesconfirmed4336.312.930.2093.06322.9813.330110.08
2026-08-02confirmed casesconfirmed4296.982.160.1711.97286.156.724.1111-0.03
2026-08-03confirmed casesconfirmed4291.81.50.1751.18281.4210.080.3110.05
2026-08-04confirmed casesconfirmed4350.843.850.1913.45323.4127.40.04110.11
2026-08-07confirmed casesconfirmed4308.43.90.1923.01294.779.514.1211-0.01
2026-08-11confirmed casesconfirmed4335.757.530.1915.54307.327.620.84110.1
2026-08-15confirmed casesconfirmed4312.225.270.2114.16308.061.822.3411-0.03
2026-08-17confirmed casesconfirmed4299.384.010.2283.64293.3306.0611-0.09
2026-08-22confirmed casesconfirmed4335.335.020.2064.11293.0242.310110.15
2026-08-24confirmed casesconfirmed4315.975.190.1854.41253.0562.920110.26
2026-08-25confirmed casesconfirmed4302.154.880.1844.27236.9365.230110.27
2026-08-30confirmed casesconfirmed4384.814.510.2033.29243.44141.370110.37
2026-08-31confirmed casesconfirmed3203.033.880.1583.28165.7737.260110.22
2026-09-01confirmed casesconfirmed3242.923.960.1732.91189.5153.410110.24
2026-09-06confirmed casesconfirmed3251.073.680.1993.61192.6458.440110.29
2026-09-10confirmed casesconfirmed2216.462.140.2221.92135.5980.8700.510.43
2026-09-12confirmed casesconfirmed2715.774.760.6953.6864.25651.520001
2026-09-13confirmed casesconfirmed2668.584.890.664463.47605.110001
2026-09-15confirmed casesconfirmed1410.164.150.6693.2442.163680001
2026-09-16confirmed casesconfirmed1375.694.80.6293.7943.88331.810001
2026-09-19confirmed casesconfirmed1297.615.980.5424.8141.56256.050001
2026-07-23confirmed deathsconfirmed_deaths4701.544.530.4482.4383.29318.2500.2510.56
2026-07-25confirmed deathsconfirmed_deaths4237.781.990.2341.6214.2823.50110.18
2026-07-26confirmed deathsconfirmed_deaths4256.462.30.2552.02231.6524.810110.13
2026-07-27confirmed deathsconfirmed_deaths4310.272.880.2521.92280.8429.430110.21
2026-07-31confirmed deathsconfirmed_deaths4312.362.70.2892.15292.2120.080.07110.08
2026-08-01confirmed deathsconfirmed_deaths4280.922.40.2531.7261.4619.460110.1
2026-08-02confirmed deathsconfirmed_deaths4256.052.730.2762.38238.2316.880.94110.03
2026-08-03confirmed deathsconfirmed_deaths4224.51.620.2451.22212.210.831.46110.02
2026-08-04confirmed deathsconfirmed_deaths4248.773.560.2833.44242.124.292.3511-0.02
2026-08-07confirmed deathsconfirmed_deaths4231.53.230.3253.08223.681.026.7911-0.11
2026-08-11confirmed deathsconfirmed_deaths4244.610.680.33710.13233.591.959.0611-0.13
2026-08-15confirmed deathsconfirmed_deaths4244.836.120.345.64239.910.044.8911-0.1
2026-08-17confirmed deathsconfirmed_deaths4251.325.10.2973.95246.093.931.311-0.02
2026-08-22confirmed deathsconfirmed_deaths42422.710.4823.89219.73022.2711-0.18
2026-08-24confirmed deathsconfirmed_deaths4220.023.20.5436.42144.48075.5411-0.4
2026-08-25confirmed deathsconfirmed_deaths4247.193.620.7969.93151.31095.880.751-0.45
2026-08-30confirmed deathsconfirmed_deaths4260.372.930.9079.23129.820130.550.51-0.51
2026-08-31confirmed deathsconfirmed_deaths3199.793.070.7827.79109.7909011-0.42
2026-09-01confirmed deathsconfirmed_deaths3203.764.020.93212.0989.670114.090.331-0.52
2026-09-06confirmed deathsconfirmed_deaths3233.715.41.17516.8461.640172.0701-0.66
2026-09-10confirmed deathsconfirmed_deaths2123.891.660.7474.0367.86056.040.51-0.46
2026-09-12confirmed deathsconfirmed_deaths254.20.720.1350.6627.6926.520010.56
2026-09-13confirmed deathsconfirmed_deaths257.110.790.1460.7824.8332.280010.66
2026-09-15confirmed deathsconfirmed_deaths126.420.650.1120.6416.2210.20110.49
2026-09-16confirmed deathsconfirmed_deaths128.530.620.1250.6215.9112.620010.56
2026-09-19confirmed deathsconfirmed_deaths115.480.990.070.9914.590.890110.13
2026-07-23isolation bedstreatment455.731.910.0892.3247.7907.9411-0.21
2026-07-25isolation bedstreatment451.681.340.0821.647.140.014.5211-0.14
2026-07-26isolation bedstreatment461.671.660.1254.820.236.6311-0.12
2026-07-27isolation bedstreatment459.472.050.0982.554.6704.811-0.18
2026-07-31isolation bedstreatment482.761.940.1322.4168.17014.60.751-0.28
2026-08-01isolation bedstreatment488.621.40.1531.7973.94014.670.751-0.25
2026-08-02isolation bedstreatment484.371.660.1412.0769.7014.670.751-0.28
2026-08-03isolation bedstreatment479.931.770.1342.1871.9108.0111-0.21
2026-08-04isolation bedstreatment487.061.280.1411.4582.731.223.1111-0.03
2026-08-07isolation bedstreatment4104.981.740.1712.1785.76019.2111-0.29
2026-08-11isolation bedstreatment4102.610.610.1710.790.93011.6811-0.21
2026-08-15isolation bedstreatment4127.511.480.2041.8898.57028.9411-0.33
2026-08-17isolation bedstreatment4103.761.520.1611.8493.32010.4411-0.19
2026-08-22isolation bedstreatment4144.982.330.2182.89120.25024.7311-0.22
2026-08-24isolation bedstreatment4127.242.530.1812.9115.131.0511.0611-0.11
2026-08-25isolation bedstreatment4131.921.910.1912.22111.86020.0611-0.2
2026-08-30isolation bedstreatment4128.622.940.1713.17117.425.735.4711-0.02
2026-08-31isolation bedstreatment3117.932.570.1612.94108.450.059.4311-0.14
2026-09-01isolation bedstreatment3117.143.050.1533.47112.3304.811-0.09
2026-09-06isolation bedstreatment3117.442.50.1512.641150.332.1111-0.05
2026-09-10isolation bedstreatment2133.863.310.1743.66115.73018.1211-0.14
2026-09-12isolation bedstreatment21322.140.1812.55113.46018.5411-0.16
2026-09-13isolation bedstreatment2111.492.090.1442.36111.010.040.4411-0.02
2026-09-15isolation bedstreatment1112.093.30.1433.87109.3602.7311-0.08
2026-09-16isolation bedstreatment1113.014.720.1435.499.88013.1311-0.2
2026-09-19isolation bedstreatment1101.371.520.1341.68101.120.250110.03
2026-08-02onset reportsonsets3122.90.90.1881.1396.84026.0611-0.28
2026-08-03onset reportsonsets3105.391.640.1661.38100.321.153.9211-0.02
2026-08-04onset reportsonsets3141.841.150.2661.2888.340.0453.460.671-0.26
2026-08-07onset reportsonsets3378.481.160.6331.3882.320296.1601-0.8
2026-08-11onset reportsonsets3243.781.480.4291.8185.910157.870.331-0.64
2026-08-15onset reportsonsets3370.984.170.2812.35205.04165.9300.3310.57
2026-08-17onset reportsonsets2246.494.040.2552.47153.6192.8800.510.47
2026-08-22onset reportsonsets2117.841.060.2051.17104.1213.730110.2
2026-08-24onset reportsonsets185.161.260.3922.4482.0903.0711-0.12
2026-08-25onset reportsonsets195.660.410.5110.8788.577.090110.18

Estimate evolution across releases ​

How the outbreak-size estimate has moved as situation reports accrued, three series on one calendar axis. The estimate published at each release is in blue, drawn as a median with nested 30/60/90% interval bars because each release is its own fit rather than one continuous model. The current model frozen at earlier cut-offs is in red. The current model on current data is the green band, drawn day by day so the latest estimate reads against the earlier points. Dotted vertical rules mark the release dates. The published series switches from a closed-form integral model to a renewal model on 7 June, so a step there can reflect the change of method rather than of data.

Released estimates and the current-model frozen re-fits
julia
# Released median and 30/60/90% intervals per release, from
# `data/released_estimates.csv`. Each tuple is
# `(date, median, lo30, hi30, lo60, hi60, lo90, hi90)`.
release_evolution = [
    (
        string(r.date), r.median, r.lo30, r.hi30, r.lo60, r.hi60,
        r.lo90, r.hi90,
    ) for r in eachrow(released_df)
]

# The current model frozen at earlier cut-offs, each its own discrete
# estimate: the McCabe-matched cut-offs (20, 23, 27 May), the 8 June Chamla
# confirmed-case anchor and the one-week-back validation fit
# (`frozen_lastweek`, at `validation_cutoff`). Each tuple carries the
# median and 30/60/90% credible bounds from the frozen draws; `round_fn`
# rounds to a whole count for outbreak size, and is passed through
# unrounded for a continuous quantity such as R0.
function _ci369(xs; round_fn = x -> round(Int, x))
    q(p) = round_fn(quantile(xs, p))
    return (q(0.5), q(0.35), q(0.65), q(0.2), q(0.8), q(0.05), q(0.95))
end
frozen_by_cutoff[validation_cutoff] = frozen_lastweek
# The cut-offs every frozen fit above was made at, shared by the
# outbreak-size and R0 by-release overlays below.
_frozen_matched_cutoffs = sort(
    union(
        frozen_cutoffs,
        [validation_cutoff, default_chamla_cutoff()]
    )
)
frozen_matched = [(c, _ci369(frozen_C(c))...) for c in _frozen_matched_cutoffs]

# The current-data, current-model estimate as the cumulative-infection
# trajectory over the day grid (one calendar date per grid day, day 1 is
# the seeding date), summarised by per-day 30/60/90% credible bounds. This
# is the same latent quantity the cumulative-trajectory figure shows, so
# the current estimate rises over time on the release-date axis instead of
# sitting flat. Drawn against calendar dates, it lines up with the
# release and frozen points.
infection_trajectory = let
    mat = chn_joint[:cumulative_infections]
    trajs = [collect(v) for v in vec(collect(mat))]
    # Only over the comparison window — from the earliest release date to the
    # cut-off — not back to the seeding date.
    start_day = obs.n - value(obs.cutoff - Date(release_evolution[1][1]))
    days = max(start_day, 1):obs.n
    dates = [obs.seeding + Day(d - 1) for d in days]
    q(d, p) = quantile(Float64[t[d] for t in trajs], p)
    (
        dates,
        [q(d, 0.35) for d in days], [q(d, 0.65) for d in days],
        [q(d, 0.2) for d in days], [q(d, 0.8) for d in days],
        [q(d, 0.05) for d in days], [q(d, 0.95) for d in days],
    )
end

evolution_fig = plot_estimate_evolution(
    release_evolution;
    renewal = frozen_matched,
    renewal_label = "Current model frozen at earlier cut-offs",
    trajectory = infection_trajectory,
    title = "Outbreak-size estimate as data accrued"
);

Reproduction number by release ​

The reproduction number estimated at each release, the same kind of release-by-release picture as the outbreak-size evolution above. Each release's cut-off reproduction number is drawn as a discrete estimate, a median with nested 30/60/90% interval bars. The current fit's daily over its established window is drawn as the continuous band, and   is marked. The reproduction-number axis is fixed at three across this figure and the by-dataset one below, with an interval running past it clamped and marked with an open triangle.

Reproduction number per release with the current-fit band
julia
rt_release_df = CSV.read(
    joinpath(pkgdir(BVDOutbreakSize), "data", "rt_by_release.csv"), DataFrame
)
rt_release = [
    (
        string(r.date), r.median, r.lo30, r.hi30, r.lo60, r.hi60,
        r.lo90, r.hi90,
    ) for r in eachrow(rt_release_df)
]

# The current fit's daily Rt over its established window, summarised per day
# into a 30/60/90% band, reusing the same walk reconstruction the Rt figure
# uses so the band lines up with the per-release points on the calendar axis.
# The band is drawn only from the first release date onward, so it spans the
# same window as the per-release estimates rather than extending back to the
# renewal start. The first release day is the earliest date in
# `rt_by_release.csv` as a grid day; the walk is still reconstructed from the
# renewal start `_rt_start_plot` (the model knot grid) and the window is
# clamped into the reconstructed range so the quantiles never hit masked days.
rt_release_trajectory = let
    rt_walk_start = clamp(_BREAKPOINT - RT_WALK_LEAD, _rt_start_plot, obs.n)
    mat = reconstruct_rt(
        chn_joint; n = obs.n, breakpoint = _BREAKPOINT,
        rt_start = _rt_start_plot, rt_walk_start = rt_walk_start,
        ramp = RT_INTERVENTION_RAMP
    )
    first_release_day = clamp(
        value(minimum(rt_release_df.date) - obs.seeding) + 1,
        _rt_start_plot, obs.n
    )
    days = first_release_day:obs.n
    dates = [obs.seeding + Day(d - 1) for d in days]
    q(d, p) = quantile(collect(skipmissing(@view mat[:, d])), p)
    (
        dates,
        [q(d, 0.35) for d in days], [q(d, 0.65) for d in days],
        [q(d, 0.2) for d in days], [q(d, 0.8) for d in days],
        [q(d, 0.05) for d in days], [q(d, 0.95) for d in days],
    )
end

# One fixed reproduction-number axis across both figures below. The
# estimates sit around one and the widest stream's 90% upper pulls a free
# axis past four, which flattens every panel onto the lower quarter of its
# range. An interval past the crop is clamped and marked, so nothing is
# silently cut. The basic reproduction number keeps its own axis: it sits
# around two with tails near four, and the same crop would clip the
# estimates themselves.
const _RT_AXIS_MAX = 3.0

rt_evolution_fig = plot_estimate_evolution(
    rt_release;
    trajectory = rt_release_trajectory,
    ylabel = "Reproduction number",
    title = "Reproduction number as data accrued",
    released_label = "Released estimate (per project release)",
    trajectory_label = "Current model, current data",
    refline = 1.0,
    ymax = _RT_AXIS_MAX
);

Reproduction number by release and dataset ​

The same release-by-release reproduction number split into one panel per dataset, so each dataset's history reads against the others and against the joint. Panels share a calendar axis and the fixed reproduction-number range, and   is marked. Each release's cut-off value is a median with nested 30/60/90% interval bars. A dataset the report also fits on its own carries that fit's current-model band behind its points, built as in the overview above. Confirmed deaths carries no band, so its panel shows release points alone. Only the most recent releases published these per-dataset estimates, so every panel spans a much shorter window than the overview above rather than a different history.

Reproduction number per release by fit
julia
# Schema of the per-release, per-fit estimate tables written by
# scripts/score_releases.jl from each release's stream_estimates.csv.
_by_stream_schema = (;
    release = String, date = Date, fit = String,
    median = Float64, lo30 = Float64, hi30 = Float64, lo60 = Float64,
    hi60 = Float64, lo90 = Float64, hi90 = Float64,
)

# Fits in a fixed order, the joint first, so the panels do not reshuffle
# between builds. Labels match the per-stream table on the in-sample page.
# Recovered is absent because it has no individual fit.
_fit_order = [
    "joint", "cases", "deaths", "confirmed", "confirmed_deaths",
    "treatment", "onsets", "exports",
]
_fit_labels = Dict(
    "joint" => "joint", "cases" => "cases (DRC)",
    "deaths" => "deaths (DRC)", "confirmed" => "confirmed (DRC)",
    "confirmed_deaths" => "confirmed deaths (DRC)",
    "treatment" => "isolation (DRC)", "onsets" => "onsets (DRC)",
    "exports" => "exports"
)

# Group a per-fit estimate table into the label => tuples pairs the faceted
# plot takes, keyed on the date so the mixed release tag shapes
# (`results-v1.9.0` and `results-1243`) never reach the axis.
function _fit_groups(df)
    return [
        get(_fit_labels, f, f) =>
            [
            (
                string(r.date), r.median, r.lo30, r.hi30, r.lo60, r.hi60,
                r.lo90, r.hi90,
            ) for r in eachrow(df) if r.fit == f
        ]
            for f in _fit_order
    ]
end

# Per-fit reproduction-number trajectory, reconstructing the walk exactly as
# `plot_rt_streams` does per stream.
function _stream_rt_trajectory(chn, dates; rt_start, rt_walk_start)
    mat = reconstruct_rt(
        chn; n = obs.n, breakpoint = _BREAKPOINT,
        rt_start = rt_start, rt_walk_start = rt_walk_start,
        ramp = RT_INTERVENTION_RAMP
    )
    first_date = isempty(dates) ? obs.seeding : minimum(dates)
    first_day = clamp(value(first_date - obs.seeding) + 1, rt_start, obs.n)
    days = first_day:obs.n
    ds = [obs.seeding + Day(d - 1) for d in days]
    q(d, p) = quantile(collect(skipmissing(@view mat[:, d])), p)
    return (
        ds,
        [q(d, 0.35) for d in days], [q(d, 0.65) for d in days],
        [q(d, 0.2) for d in days], [q(d, 0.8) for d in days],
        [q(d, 0.05) for d in days], [q(d, 0.95) for d in days],
    )
end

# The single-stream chains and their renewal-walk starts, keyed on the fit
# id the per-release tables use. Both the joint walk start and the day-1
# per-stream starts are the ones the per-stream implied-Rt figure on the
# in-sample page uses, so the bands here match it. Confirmed deaths has no trajectory
# here: its panel still draws its release points alone.
_rt_walk_start_joint = clamp(
    _BREAKPOINT - RT_WALK_LEAD, _rt_start_plot, obs.n
)
_stream_chains = (
    "joint" => (;
        chn = chn_joint, rt_start = _rt_start_plot,
        rt_walk_start = _rt_walk_start_joint,
    ),
    "cases" => (; chn = chn_cases, rt_start = 1, rt_walk_start = 1),
    "deaths" => (; chn = chn_deaths, rt_start = 1, rt_walk_start = 1),
    "confirmed" => (; chn = chn_confirmed, rt_start = 1, rt_walk_start = 1),
    "confirmed_deaths" => (;
        chn = chn_confirmed_deaths, rt_start = 1,
        rt_walk_start = 1,
    ),
    "treatment" => (; chn = chn_treatment, rt_start = 1, rt_walk_start = 1),
    "onsets" => (; chn = chn_onsets, rt_start = 1, rt_walk_start = 1),
    "exports" => (; chn = chn_exports, rt_start = 1, rt_walk_start = 1),
)

# Build a fit label => trajectory dictionary from a per-release table,
# restricted to the fits `_stream_chains` names. A fit with no row in `df`
# gets no trajectory, so its panel still draws its release points alone.
function _rt_trajectories(df)
    trajs = Dict{String, Any}()
    for (fid, cfg) in _stream_chains
        fdates = df.date[df.fit .== fid]
        isempty(fdates) && continue
        trajs[get(_fit_labels, fid, fid)] = _stream_rt_trajectory(
            cfg.chn, fdates; rt_start = cfg.rt_start,
            rt_walk_start = cfg.rt_walk_start
        )
    end
    return trajs
end

rt_stream_df = _release_data(
    "rt_by_release_by_stream.csv",
    _by_stream_schema
)
rt_stream_fig = plot_evolution_by_group(
    _fit_groups(rt_stream_df);
    trajectories = _rt_trajectories(rt_stream_df),
    ylabel = "Reproduction number",
    title = "Reproduction number as data accrued, by dataset",
    released_label = "Released estimate (per release)",
    refline = 1.0,
    ymax = _RT_AXIS_MAX,
    empty_note = "No per-dataset reproduction numbers saved yet."
);

Basic reproduction number by release ​

The basic reproduction number estimated at each release, the initial-transmission counterpart of the reproduction number above, before the time-varying decline. Released estimates are blue and the current model frozen at earlier cut-offs is red, each a median with nested 30/60/90% interval bars. The current fit sits behind both as a flat band, and   is marked. Releases only began publishing this quantity recently, so the short blue history reflects that rather than any failed release.

Basic reproduction number per release with frozen re-fits and the current-fit band
julia
# Per-release R0 points from r0_by_release.csv, read through the typed
# fallback so a missing or header-only file (until a release carries
# `rt_state.log_R0` in its posterior draws) does not break the build. The
# schema mirrors rt_by_release.csv.
_r0_schema = (;
    release = String, date = Date, median = Float64,
    lo30 = Float64, hi30 = Float64, lo60 = Float64, hi60 = Float64,
    lo90 = Float64, hi90 = Float64,
)
r0_release_df = _release_data("r0_by_release.csv", _r0_schema)
r0_release = [
    (
        string(r.date), r.median, r.lo30, r.hi30, r.lo60, r.hi60,
        r.lo90, r.hi90,
    ) for r in eachrow(r0_release_df)
]

# The current model frozen at earlier cut-offs, one discrete estimate per
# cut-off, reusing the same frozen fits `frozen_matched` above already
# computed. No extra fits are run. Each tuple carries the median and
# 30/60/90% credible bounds of that frozen fit's own R0 draws, unrounded
# since R0 is continuous.
frozen_r0_matched = [
    (c, _ci369(frozen_R0(c); round_fn = identity)...)
        for c in _frozen_matched_cutoffs
]

# The current fit's R0 posterior is a single distribution rather than a
# daily series, so it summarises into a flat 30/60/90% reference band. The
# window runs from the earliest mark on the axis, the first frozen cut-off
# or release point, to the current cut-off, so the band reads behind both
# series rather than only their recent end.
r0_reference = let
    draws = r0_walk_draws(chn_joint)
    q(p) = quantile(draws, p)
    first_date = min(
        minimum(Date.(_frozen_matched_cutoffs)),
        isempty(r0_release_df.date) ? obs.cutoff :
            minimum(r0_release_df.date)
    )
    dates = [first_date, obs.cutoff]
    (
        dates, fill(q(0.35), 2), fill(q(0.65), 2), fill(q(0.2), 2),
        fill(q(0.8), 2), fill(q(0.05), 2), fill(q(0.95), 2),
    )
end

r0_evolution_fig = plot_estimate_evolution(
    r0_release;
    renewal = frozen_r0_matched,
    renewal_label = "Current model frozen at earlier cut-offs",
    trajectory = r0_reference,
    ylabel = "Basic reproduction number",
    title = "Basic reproduction number as data accrued",
    released_label = "Released estimate (per project release)",
    trajectory_label = "Current model, current data",
    refline = 1.0
);

Basic reproduction number by release and dataset ​

The basic reproduction number estimated at each release, one panel per fit, the by-dataset counterpart of the figure above. Panels share a calendar axis and a y range, and   is marked. Each release is a median with nested 30/60/90% interval bars. Every fit the report runs on its own also carries a current-model reference band.

Basic reproduction number per release by fit
julia
# Per-fit R0 flat reference band, the by-dataset counterpart of
# `r0_reference` above, a single distribution rather than a daily walk, so
# each fit's band is flat across its own release window. `r0_walk_draws`
# probes for the walk base, so a single-stream model built without its own
# renewal walk drops its band instead of erroring.
function _r0_stream_trajectory(chn, dates)
    draws = r0_walk_draws(chn)
    isnothing(draws) && return nothing
    q(p) = quantile(draws, p)
    first_date = isempty(dates) ? obs.seeding : minimum(dates)
    ds = [first_date, obs.cutoff]
    return (
        ds, fill(q(0.35), 2), fill(q(0.65), 2), fill(q(0.2), 2),
        fill(q(0.8), 2), fill(q(0.05), 2), fill(q(0.95), 2),
    )
end

# Build a fit label => trajectory dictionary from a per-release R0 table,
# restricted to the fits `_stream_chains` names, the same restriction the
# reproduction-number-by-dataset trajectories use. A fit with no row in
# `df`, or whose chain carries no walk base, gets no trajectory, so its
# panel still draws its release points alone.
function _r0_trajectories(df)
    trajs = Dict{String, Any}()
    for (fid, cfg) in _stream_chains
        fdates = df.date[df.fit .== fid]
        isempty(fdates) && continue
        traj = _r0_stream_trajectory(cfg.chn, fdates)
        isnothing(traj) || (trajs[get(_fit_labels, fid, fid)] = traj)
    end
    return trajs
end

r0_stream_df = _release_data(
    "r0_by_release_by_stream.csv",
    _by_stream_schema
)
r0_stream_fig = plot_evolution_by_group(
    _fit_groups(r0_stream_df);
    trajectories = _r0_trajectories(r0_stream_df),
    ylabel = "Basic reproduction number",
    title = "Basic reproduction number as data accrued, by dataset",
    released_label = "Released estimate (per release)",
    refline = 1.0,
    empty_note = "No per-dataset basic reproduction numbers saved yet."
);

Saving forecast outputs ​

The one-week-back validation forecast, in the same archive format as the release forecast, so the frozen "last week versus now" forecast is recorded as a release asset alongside the forecast it is scored against.

Write forecast outputs
julia
output_dir = get(
    ENV, "BVD_OUTPUT_DIR",
    joinpath(pkgdir(BVDOutbreakSize), "output")
)
mkpath(output_dir)
CSV.write(
    joinpath(output_dir, "forecast_validation.csv"),
    forecast_archive(
        [(7, validation_forecast)];
        made_date = frozen_lastweek.o.cutoff, thin = 5
    )
)
"/home/runner/work/BVDOutbreakSize/BVDOutbreakSize/output/forecast_validation.csv"
Write the summary bullets
julia
# The bullets under the summary heading at the top of the page. They read
# tables built further down, so they are written here and read back when
# the site is assembled.
evaluation_forecast_national_summary = let
    fmt(x) = ismissing(x) || !isfinite(x) ? "n/a" :
        string(round(x; digits = 2))
    scored(tbl) = filter(r -> !ismissing(r.rel_to_baseline), tbl)
    function beat(label, tbl)
        rows = scored(tbl)
        size(rows, 1) == 0 &&
            return string("- **", label, ":** no scored forecasts yet.")
        return string(
            "- **", label, ":** beat the baseline on ",
            count(<(1), rows.rel_to_baseline), " of ", size(rows, 1),
            " streams."
        )
    end
    function block(lead, tbl)
        rows = scored(tbl)
        size(rows, 1) == 0 && return nothing
        return join(
            vcat(
                [string("**", lead, "**"), ""],
                [
                    string(
                        "- ", r.stream, ": relative skill ",
                        fmt(r.rel_to_baseline), ", 90% coverage ",
                        fmt(r.coverage_90), ", bias ", fmt(r.bias), " over ",
                        r.n, " forecasts."
                    )
                        for r in eachrow(rows)
                ]
            ), "\n"
        )
    end
    frozen_joint = select_fit_role(frozen_score_overview_table, "joint")
    overall = [
        beat("Joint model across releases", joint_score_overview_table),
        beat("Frozen joint model", frozen_joint),
    ]
    blocks = filter(
        !isnothing,
        [
            block("Across releases", joint_score_overview_table),
            block("Frozen fits", frozen_joint),
        ]
    )
    join(vcat([join(overall, "\n")], blocks), "\n\n")
end
dashboard_dir = joinpath(
    pkgdir(BVDOutbreakSize), "docs", "src", "summary_assets"
)
mkpath(dashboard_dir)
write(
    joinpath(dashboard_dir, "evaluation_forecast_national.md"),
    evaluation_forecast_national_summary
);