Skip to content

Health-zone forecast evaluation ​

How the health-zone split on the health-zone forecasts page has scored against what each zone went on to report. The patch totals are scored as in the national forecast evaluation and the zone split against two persistence rules of its own. The same scoring by province is on the province forecast evaluation page. The in-sample zone checks are on the health-zone in-sample checks page.

Load packages, data and fitted chains
julia
# Shared setup: packages, observations and the fit registry. See
# `docs/pages/_setup.jl`.
using BVDOutbreakSize
include(joinpath(pkgdir(BVDOutbreakSize), "docs", "pages", "_setup.jl"))
validation_forecast_from (generic function with 1 method)
julia
# The zone fit melded from the frozen joint, loaded from the cache here.
frozen_local = load_fit("local_frozen_validation");

Summary ​

The overall bullets come first, then a block per province, from the scores below. The log score of the split is higher when the model allocates the week's cases better than the persistence rules.

  • Split: the zone model allocates the week better than both persistence rules in 3 of 5 provinces scored, over 492 observed confirmed cases.

Ituri

  • Log score of the split: -65.0, against -225.73 for share persistence and -87.26 for naive persistence.

  • Total over 265 observed cases: bias 0.4, within the 90% interval: true.

Nord-Kivu

  • Log score of the split: -38.36, against -80.82 for share persistence and -38.46 for naive persistence.

  • Total over 183 observed cases: bias 0.59, within the 90% interval: true.

Haut-Uele

  • Log score of the split: -9.69, against -10.45 for share persistence and -7.67 for naive persistence.

  • Total over 28 observed cases: bias 0.1, within the 90% interval: true.

Tshopo

  • Log score of the split: -17.85, against -15.15 for share persistence and -11.69 for naive persistence.

  • Total over 13 observed cases: bias -0.22, within the 90% interval: true.

Nord-Ubangi

  • Log score of the split: 0.0, against 0.0 for share persistence and 0.0 for naive persistence.

  • Total over 3 observed cases: bias -0.99, within the 90% interval: false.

Across releases

  • 36 of 63 zones scored beat the persistence baseline, from the forecast made on 2026-09-27 onwards.

Forecast by health zone ​

The one-week-ahead zone forecast comes from the health-zone model melded from the frozen fit, run a week past the frozen cut-off. The observed count is the change in each zone's cumulative confirmed cases between the frozen cut-off and the vintage a week later, clamped at zero. The week is scored only when the zone tables carry a vintage on its last day. The split of each patch's weekly total is scored by its multinomial log score at the forecast shares. It is set against share persistence, the cumulative zone shares at the cut-off, and naive persistence, the zone split of the last seven days, each with a pseudo-count of 0.5 per zone. The patch total is scored against a naive persistence total with negative-binomial noise, as in the forecast scoring Methods section.

Zone forecast against observed
julia
# `frozen_zone_stage_inputs` is defined in the shared setup, so the zone
# estimates page reads the same frozen inputs.
frozen_zone_inputs = frozen_zone_stage_inputs(; forecast = true);
# The week is scored when the current zone tables carry a vintage a week
# past the frozen cut-off. Otherwise the section shows a note in place of
# its outputs.
zone_validation_horizon = frozen_zone_inputs.model_data.forecast.horizon
zone_truth = zone_forecast_truth(
    obs, frozen_zone_inputs;
    made_date = frozen_local.o.cutoff, horizon = zone_validation_horizon
)
_zone_validation_missing = Markdown.parse(
    "_The zone tables carry no vintage a week after the frozen cut-off, so " *
        "the zone forecast is not scored in this build._"
)
if any(!ismissing, zone_truth)
    zone_validation_forecast = zone_forecast(
        frozen_local.chn, frozen_zone_inputs
    )
    zone_validation_table = zone_forecast_vs_truth(
        zone_validation_forecast, frozen_zone_inputs; truth = zone_truth
    )
    zone_validation_scores = zone_forecast_scores(
        zone_validation_forecast, frozen_zone_inputs; truth = zone_truth
    )
    zone_validation_fig = plot_zone_forecast(
        zone_validation_table;
        patch_labels = frozen_zone_inputs.patch_labels,
        xlabel = "New confirmed cases over $(zone_validation_horizon) days",
        title = "Zone forecast from $(frozen_local.o.cutoff) against observed"
    )
else
    # The frames stay frames, since the release assets below write them.
    zone_validation_table = DataFrame()
    zone_validation_scores = DataFrame()
    zone_validation_fig = _zone_validation_missing
end;
# The scores rounded for display. Share persistence forecasts the split
# alone, so its total columns are blank. No patch is scored when every
# observed total is zero or incomplete, which leaves an empty frame
# without the columns.
zone_validation_scores_table = isempty(zone_validation_scores) ?
    _zone_validation_missing :
    let s = zone_validation_scores
        fmt(x, d) = ismissing(x) || isnan(x) ? "" :
        string(round(x; digits = d))
        DataFrame(
            "Province" => s.patch, "Rule" => s.method,
            "Observed total" => s.observed,
            "Log score of the split" => [fmt(x, 2) for x in s.log_score],
            "CRPS of the total" => [fmt(x, 1) for x in s.crps],
            "Log CRPS" => [fmt(x, 2) for x in s.log_crps],
            "Dispersion" => [fmt(x, 1) for x in s.dispersion],
            "Bias" => [fmt(x, 2) for x in s.bias],
            "Total within 90%" => [
                ismissing(x) ? "" : string(x)
                for x in s.coverage_90
            ]
        )
end;
zone_validation_table_display = isempty(zone_validation_table) ?
    _zone_validation_missing :
    zone_validation_table;

The fifteen zones with the largest forecast medians against what each went on to report, with every zone in the fold below the scores.
ProvinceRuleObserved totalLog score of the splitCRPS of the totalLog CRPSDispersionBiasTotal within 90%
Iturizone model265-65.028.70.120.30.4true
Iturishare persistence265-225.73
Iturinaive persistence265-87.2627.50.1126.2-0.14true
Nord-Kivuzone model183-38.3637.50.1718.80.59true
Nord-Kivushare persistence183-80.82
Nord-Kivunaive persistence183-38.4619.70.1218.4-0.16true
Haut-Uelezone model28-9.694.40.144.30.1true
Haut-Ueleshare persistence28-10.45
Haut-Uelenaive persistence28-7.673.20.123.2-0.07true
Tshopozone model13-17.852.10.171.8-0.22true
Tshoposhare persistence13-15.15
Tshoponaive persistence13-11.691.70.141.5-0.25true
Nord-Ubangizone model30.02.91.320.0-0.99false
Nord-Ubangishare persistence30.0
Nord-Ubanginaive persistence30.02.20.870.2-0.98false
Scores of each province's split and total for the model and the persistence rules. Share persistence forecasts the split alone, so its total columns are blank.
Zone forecast against observed, every zone
zonepatchcentral_estimatelower_90lower_50upper_50upper_90observedwithin_90
AdjaIturi000000true
AriwaraIturi000010true
AruIturi000036false
AungbaIturi000020true
BambuIturi40110250true
BogaIturi000040true
BuniaIturi7128509714755true
DamasIturi000051true
DrodroIturi000050true
FatakiIturi000180true
GetyIturi000070true
KambalaIturi000010true
KiloIturi000060true
KomandaIturi80317356true
LitaIturi1115214210true
LogoIturi000040true
LolwaIturi0002120true
MahagiIturi000010true
MambasaIturi1818337316true
MandimaIturi6322438814093true
MangalaIturi602143310true
MongbwaluIturi40111274true
Nia-NiaIturi141726535true
NiziIturi26615417213true
NyankundeIturi0002131true
RimbaIturi000030true
RwamparaIturi24614386539true
TchomiaIturi000196true
Patch totalIturi308186252366481265true
AlimbongoNord-Kivu000040true
BeniNord-Kivu101497613319371true
BienaNord-Kivu1003121true
ButemboNord-Kivu1628265019true
GomaNord-Kivu000000true
KalungutaNord-Kivu142724428true
KatwaNord-Kivu26716396416true
KaynaNord-Kivu000020true
KyondoNord-Kivu2006184true
LuberoNord-Kivu1005147true
MabalakoNord-Kivu18310305317true
ManguredjipaNord-Kivu000281true
MaserekaNord-Kivu2007192true
MusieneneNord-Kivu50211267true
MutwangaNord-Kivu2006168true
OichaNord-Kivu142725467true
VuhoviNord-Kivu502112515true
Patch totalNord-Kivu244140193304402183true
Boma MangbetuHaut-Uele4026121true
DunguHaut-Uele000010true
GombariHaut-Uele000010true
IsiroHaut-Uele200484true
PawaHaut-Uele102616305true
RunguHaut-Uele000021true
WambaHaut-Uele1237193417true
Patch totalHaut-Uele30819437528true
Miti-MurhesaSud-Kivu000010true
Patch totalSud-Kivu000010true
BafwasendeTshopo100248false
KabondoTshopo100243true
LubungaTshopo000120true
Makiso-KisanganiTshopo5139161true
MangoboTshopo201480true
TshopoTshopo000021true
Wanie-RukulaTshopo000020true
Patch totalTshopo1127173013true
ButaBas-Uele000120true
GangaBas-Uele100250true
ViadanaBas-Uele100140true
Patch totalBas-Uele201480true
BuluSud-Ubangi000120true
Patch totalSud-Ubangi000120true
WapindaNord-Ubangi000003false
Patch totalNord-Ubangi000003false

Forecast by health zone across releases ​

The archived zone forecast of each release is scored against what each zone went on to report. The scores are those of the province forecast evaluation, with each zone scored as its own stream against a persistence baseline built from its own history. A week is left out when the zone tables carry no vintage on its last day, when it holds a harmonisation-break day, or when its province moved unallocated cases into named zones during it. The figures show the ten zones with the most observed cases over the scored weeks, and the folds hold every zone.

Load and summarise the zone forecast scores
julia
# Written by `scripts/score_releases.jl` from each release's
# `zone_forecast.csv`. Missing files read as empty tables.
zone_release_scores_df = _release_data(
    joinpath("zone", "forecast_scores.csv"),
    (;
        release = String, made_date = Date, stream = String, horizon = Int,
        target_date = Date, fit = String, crps = Float64,
        log_crps = Float64, dispersion = Float64, overprediction = Float64,
        underprediction = Float64, coverage_50 = Float64,
        coverage_90 = Float64,
        bias = Float64, n_samples = Int,
        log_rel_to_baseline = Float64,
    )
)
zone_release_overlay_df = scored_overlay(
    _release_data(
        joinpath("zone", "forecast_overlay.csv"),
        (;
            release = String, made_date = Date, stream = String,
            horizon = Int, target_date = Date, fit = String,
            observed = Float64, median = Float64, lo30 = Float64,
            hi30 = Float64, lo60 = Float64, hi60 = Float64, lo90 = Float64,
            hi90 = Float64,
        )
    )
)
# Every scored zone, ordered by its observed cases over the scored weeks.
zone_release_keys = let o = zone_release_overlay_df[
        zone_release_overlay_df.fit .== JOINT_FIT, :,
    ]
    tot = Dict{String, Float64}()
    for r in eachrow(o)
        k = zone_score_key(r.stream)
        k === nothing || (tot[k] = get(tot, k, 0.0) + r.observed)
    end
    sort!(collect(keys(tot)); by = k -> (-tot[k], k))
end
zone_release_labels = let lab = Dict(
        zip(frozen_zone_inputs.zone_keys, frozen_zone_inputs.zone_labels)
    )
    [get(lab, k, k) for k in zone_release_keys]
end
zone_release_top = first(zone_release_keys, 10)
_zone_release(tbl, keys = zone_release_keys) = zone_score_rows(
    tbl, keys, zone_release_labels[indexin(keys, zone_release_keys)]
)
# Zones in the order of `zone_release_keys`, under a `zone` column.
_zone_release_display(tbl) = let t = drop_degenerate_fit_column(
        drop_individual_fit_columns(tbl)
    )
    t = t[sortperm(indexin(t.stream, zone_release_labels)), :]
    DataFrame([(c == "stream" ? "zone" : c) => t[!, c] for c in names(t)])
end
zone_release_by_horizon = forecast_score_by_horizon(
    _zone_release(zone_release_scores_df, zone_release_top)
)
zone_release_by_release = forecast_score_by_release(
    _zone_release(zone_release_scores_df, zone_release_top)
)
zone_release_overview_display = _zone_release_display(
    forecast_score_overview(_zone_release(zone_release_scores_df))
)
zone_release_by_release_display = _zone_release_display(
    forecast_score_by_release(_zone_release(zone_release_scores_df))
);

_zone_release_empty = "No zone forecast has been scored yet. Scores " *
    "appear from the first release that archives the zone forecast onwards.";
# A plain statement above the tables: when nothing is scored yet, and
# otherwise the first cut-off scored.
zone_release_note = Markdown.parse(
    size(zone_release_scores_df, 1) == 0 ? _zone_release_empty :
        string(
            "Scores run from the forecast made on ",
            minimum(zone_release_scores_df.made_date), "."
        )
);

Scores run from the forecast made on 2026-09-27.

julia
zone_release_crps_fig = plot_forecast_crps_by_horizon(
    zone_release_by_horizon;
    title = "CRPS decomposition, by health zone",
    empty_message = _zone_release_empty
);

The mean CRPS split into its width, overprediction and underprediction, one panel per zone.
julia
zone_release_by_cutoff_fig = plot_forecast_skill_by_cutoff(
    zone_release_by_release;
    title = "Relative skill against the baseline by release, by health zone",
    empty_message = _zone_release_empty
);

Relative skill against the baseline release by release, on a log scale with the reference line at one.
julia
zone_release_overlay_fig = plot_forecast_overlay(
    _zone_release(zone_release_overlay_df, zone_release_top);
    empty_message = _zone_release_empty
);

Each release's zone forecasts against what each zone went on to report, by the cut-off each was made from. Each forecast shows its median and 90% predictive interval beside the persistence baseline and the observed count.
Zone scores across releases, every zone
zonencrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
Mandima410.481.70.1361.559.310.530.64110.04
Bunia412.150.980.210.856.490.984.680.751-0.22
Beni49.752.280.131.897.781.9700.7510.27
Katwa411.350.730.4080.733.911.85.630.250.75-0.13
Rwampara49.441.220.3271.015.072.751.610.510.17
Nizi45.460.940.3420.923.030.132.30.51-0.34
Butembo46.750.840.3920.773.060.732.960.751-0.14
Mambasa47.621.230.9042.21.7105.910.50.75-0.65
Mabalako44.741.170.3181.132.790.980.960.510.01
Nia-Nia45.770.870.3370.863.711.970.090.7510.35
Wamba43.251.010.280.821.650.091.510.51-0.22
Musienene44.970.670.5180.671.930.122.920.51-0.32
Mangala43.480.770.3740.842.10.021.360.751-0.23
Lita42.51.180.2921.042.0800.4211-0.24
Oicha42.910.960.4151.31.770.041.10.51-0.34
Pawa42.631.260.2380.982.030.600.7510.31
Kalunguta43.553.810.2732.462.620.9400.7510.34
Komanda43.860.850.4660.912.250.830.790.510.05
Bambu42.610.750.4730.851.40.051.160.751-0.27
Vuhovi42.571.190.4981.051.550.640.380.510.04
Mongbwalu43.5110.4550.862.411.10110.47
Kyondo41.681.20.451.240.9400.740.751-0.35
Mutwanga42.310.650.6260.920.670.011.630.51-0.43
Lubero41.581.550.4151.630.7600.820.51-0.34
Masereka41.641.50.371.571.210.070.360.751-0.12
Rimba42.70.871.0051.130.0502.6500.75-0.82
Bafwasende42.70.920.6750.880.2302.460.50.75-0.26
Makiso-Kisangani41.51.10.3330.940.970.490.050.7510.19
Boma Mangbetu40.940.850.2810.820.680.150.11110.08
Nyankunde41.311.420.5781.760.3700.940.51-0.53
Mangobo41.420.740.5620.80.350.130.940.250.75-0.36
Gety41.51.870.8051.950.0601.4401-0.81
Isiro41.281.020.4840.950.540.230.510.510.03
Biena41.170.570.4760.560.550.060.560.51-0.04
Kabondo40.720.730.3390.720.2800.440.51-0.33
Ganga40.730.590.3820.570.1600.570.51-0.23
Rungu40.720.560.430.610.0500.670.250.75-0.54
Kilo40.830.630.4160.710.0700.760.51-0.29
Lolwa40.651.190.3651.60.3100.340.751-0.42
Aru40.61.620.3521.720.0300.570.51-0.34
Fataki40.610.610.3280.730.1500.460.751-0.2
Manguredjipa40.560.760.2640.660.4300.13110.05
Kayna40.50.370.2660.360.1200.380.751-0.02
Adja40.230.390.160.390.0100.220.750.75-0.18
Logo40.230.470.1520.460.0500.180.751-0.07
Alimbongo30.30.110.180.170.1500.1511-0.01
Tshopo40.210.330.1440.320.0800.130.7510.04
Buta40.0824.740.05222.160.0800110.26
Viadana40.220.280.1410.330.2200110.42
Dungu40.012.830.0072.690.0100110.1
Gombari40.010.450.0090.410.0100110.11
Ariwara40.030.380.0140.30.0300110.11
Aungba40.030.660.0140.460.0300110.12
Boga40.020.730.0090.460.0200110.09
Damas40.10.350.0480.290.100110.2
Drodro40.110.940.0530.730.1100110.23
Kambala40.021.270.010.880.0200110.1
Mahagi40.031.60.0161.010.0300110.12
Tchomia40.310.760.1320.610.3100110.33
Goma40.039.450.0146.740.0300110.12
Miti-Murhesa40.150.0930.1500110.34
Bulu40.040.190.0270.180.0400110.19
Lubunga40.1430.010.08926.910.1400110.34
Wanie-Rukula40.0722.410.04318.980.0700110.24
Zone scores by release, every zone
made_datezonencrpsrel_to_baselinelog_crpslog_rel_to_baselinedispersionoverpredictionunderpredictioncoverage_50coverage_90bias
2026-09-27Mandima19.993.130.1462.768.51.490110.27
2026-10-02Mandima110.680.880.1560.918.1302.5411-0.32
2026-10-06Mandima110.081.780.131.52100.080110.07
2026-10-10Mandima111.173.040.1132.6410.620.550110.14
2026-09-27Bunia119.531.140.3530.925.15014.3801-0.78
2026-10-02Bunia110.920.820.1890.816.983.940110.44
2026-10-06Bunia17.760.830.1160.677.6500.1111-0.07
2026-10-10Bunia110.391.050.1810.936.1804.2211-0.45
2026-09-27Beni16.93.430.1083.036.190.710110.22
2026-10-02Beni19.331.730.1291.367.671.660110.26
2026-10-06Beni18.071.10.1050.968.010.060110.05
2026-10-10Beni114.696.220.1785.069.235.460010.53
2026-09-27Katwa118.761.020.70.892.62016.1400-0.93
2026-10-02Katwa110.071.120.3311.133.6806.3901-0.63
2026-10-06Katwa16.030.40.1920.444.641.390110.34
2026-10-10Katwa110.530.550.4080.584.725.810010.72
2026-09-27Rwampara19.81.430.3951.233.3506.4501-0.64
2026-10-02Rwampara19.071.140.2590.916.112.960110.44
2026-10-06Rwampara112.941.180.481.085.147.80010.76
2026-10-10Rwampara15.951.150.1740.725.690.260110.12
2026-09-27Nizi13.630.740.3530.912.2801.3511-0.4
2026-10-02Nizi16.51.340.4381.342.7903.7101-0.56
2026-10-06Nizi17.060.930.3680.732.9204.1401-0.61
2026-10-10Nizi14.640.80.2080.784.130.510110.22
2026-09-27Butembo111.150.780.7740.71.5209.6301-0.85
2026-10-02Butembo14.891.180.2791.282.6902.211-0.51
2026-10-06Butembo15.090.910.2460.843.781.310110.4
2026-10-10Butembo15.850.730.2710.674.251.60110.43
2026-09-27Mambasa16.0411.811.45824.460.2105.8300-0.92
2026-10-02Mambasa13.071.050.5111.321.501.5811-0.48
2026-10-06Mambasa14.450.870.3161.023.4401.0111-0.33
2026-10-10Mambasa116.911.041.331.491.7015.2101-0.89
2026-09-27Mabalako15.330.830.5380.971.6103.7201-0.69
2026-10-02Mabalako12.362.90.173.512.2300.1311-0.15
2026-10-06Mabalako14.282.210.2091.913.480.80110.3
2026-10-10Mabalako16.9710.3540.853.833.140010.59
2026-09-27Nia-Nia16.651.820.2511.494.721.930110.45
2026-10-02Nia-Nia18.240.560.4340.5844.240010.65
2026-10-06Nia-Nia13.331.350.2551.612.9700.3611-0.19
2026-10-10Nia-Nia14.870.870.4090.823.151.720110.48
2026-09-27Wamba11.460.940.170.691.460011-0.01
2026-10-02Wamba11.951.170.210.821.580.380110.33
2026-10-06Wamba14.560.770.3830.631.4903.0701-0.63
2026-10-10Wamba15.021.360.3551.412.0502.9601-0.56
2026-09-27Musienene18.910.740.9720.631.0607.8501-0.9
2026-10-02Musienene15.50.920.5610.971.6703.8301-0.65
2026-10-06Musienene13.190.490.2890.522.710.480110.29
2026-10-10Musienene12.260.450.250.622.260011-0.03
2026-09-27Mangala16.551.230.541.12.204.3601-0.61
2026-10-02Mangala12.160.410.2780.652.1100.0411-0.07
2026-10-06Mangala12.250.480.2690.612.170.080110.14
2026-10-10Mangala12.981.020.4080.961.9301.0511-0.38
2026-09-27Lita12.561.070.2890.92.0300.5311-0.32
2026-10-02Lita12.091.050.2510.732.0500.0411-0.08
2026-10-06Lita12.421.160.2861.372.1300.2911-0.23
2026-10-10Lita12.931.460.3421.392.0900.8411-0.35
2026-09-27Oicha11.970.340.5410.820.7601.2101-0.58
2026-10-02Oicha13.712.030.6422.971.0902.6101-0.65
2026-10-06Oicha12.760.70.2570.752.1800.5811-0.28
2026-10-10Oicha13.235.190.223.953.050.180110.15
2026-09-27Pawa12.551.910.1861.682.180.370110.25
2026-10-02Pawa12.532.40.1751.912.350.190110.15
2026-10-06Pawa11.851.410.1570.931.770.080110.15
2026-10-10Pawa13.580.780.4340.731.811.770010.69
2026-09-27Kalunguta12.392.270.2331.391.980.410110.26
2026-10-02Kalunguta13.53.930.2712.852.520.980110.38
2026-10-06Kalunguta13.13.490.2192.482.860.240110.19
2026-10-10Kalunguta15.235.840.373.963.122.120010.53
2026-09-27Komanda14.660.550.340.553.221.440110.42
2026-10-02Komanda14.50.870.5650.772.631.880010.64
2026-10-06Komanda12.341.320.3161.231.8600.4811-0.27
2026-10-10Komanda13.951.390.6421.471.2902.6601-0.6
2026-09-27Bambu11.830.90.3621.141.3700.4611-0.34
2026-10-02Bambu14.831.190.811.09103.8301-0.73
2026-10-06Bambu11.990.850.3591.171.6300.3711-0.27
2026-10-10Bambu11.790.320.3590.421.580.210110.28
2026-09-27Vuhovi12.143.390.5774.650.6801.4601-0.63
2026-10-02Vuhovi11.630.650.2390.541.5900.0411-0.09
2026-10-06Vuhovi11.982.710.2681.781.920.060110.1
2026-10-10Vuhovi14.540.950.9070.772.022.520010.78
2026-09-27Mongbwalu14.21.330.2940.743.220.980110.36
2026-10-02Mongbwalu13.210.890.3760.772.330.880110.45
2026-10-06Mongbwalu13.420.880.4780.862.121.310110.49
2026-10-10Mongbwalu13.230.950.6731.011.991.240110.58
2026-09-27Kyondo11.040.930.3620.860.7400.311-0.34
2026-10-02Kyondo11.191.110.4141.020.7500.4411-0.36
2026-10-06Kyondo11.321.760.2981.871.320011-0.05
2026-10-10Kyondo13.171.20.7261.540.9402.2301-0.66
2026-09-27Mutwanga12.283.040.814.460.3201.9601-0.77
2026-10-02Mutwanga14.971.461.0862.150.5704.3901-0.84
2026-10-06Mutwanga11.060.270.3150.470.8900.1811-0.25
2026-10-10Mutwanga10.920.150.2950.210.890.020110.16
2026-09-27Lubero10.451.220.2371.420.4300.0111-0.21
2026-10-02Lubero10.631.810.2551.510.630011-0.03
2026-10-06Lubero12.360.880.5630.950.9401.4201-0.56
2026-10-10Lubero12.874.340.6046.381.0301.8401-0.58
2026-09-27Masereka11.842.610.6033.340.4901.3501-0.68
2026-10-02Masereka11.291.10.2610.961.2100.0811-0.15
2026-10-06Masereka11.831.440.3111.251.610.220110.23
2026-10-10Masereka11.591.30.3041.241.520.060110.12
2026-09-27Rimba10.731.460.4951.460.0600.6701-0.72
2026-10-02Rimba15.370.991.6821.050.0205.3400-0.95
2026-10-06Rimba13.991.691.3612.550.0703.9101-0.9
2026-10-10Rimba10.710.170.4830.450.0600.6601-0.72
2026-09-27Bafwasende10.242.410.1522.370.2400110.44
2026-10-02Bafwasende10.212.920.1312.890.2100110.41
2026-10-06Bafwasende11.90.870.7590.820.2201.6801-0.91
2026-10-10Bafwasende18.430.91.6570.820.2608.1700-0.99
2026-09-27Makiso-Kisangani10.861.30.1630.980.8600110.01
2026-10-02Makiso-Kisangani10.921.450.1841.360.890.020110.18
2026-10-06Makiso-Kisangani10.960.810.1930.740.7700.1911-0.28
2026-10-10Makiso-Kisangani13.271.090.7910.941.351.920010.85
2026-09-27Boma Mangbetu11.011.270.2751.320.6400.3711-0.43
2026-10-02Boma Mangbetu10.71.050.1980.860.6500.0611-0.18
2026-10-06Boma Mangbetu11.080.590.3370.610.750.330110.49
2026-10-10Boma Mangbetu10.970.850.3120.830.690.290110.46
2026-09-27Nyankunde11.241.940.642.250.1501.0901-0.72
2026-10-02Nyankunde11.472.250.6013.60.4601.0111-0.53
2026-10-06Nyankunde11.811.480.7681.610.2301.5801-0.72
2026-10-10Nyankunde10.710.610.3040.790.6500.0611-0.17
2026-09-27Mangobo11.470.930.5671.030.3201.1601-0.76
2026-10-02Mangobo12.470.980.8081.090.2902.1900-0.92
2026-10-06Mangobo10.694.70.3186.480.2900.411-0.53
2026-10-10Mangobo11.060.310.5550.380.530.530010.76
2026-09-27Gety11.531.050.8181.080.0501.4801-0.84
2026-10-02Gety11.531.020.8271.040.0501.4801-0.81
2026-10-06Gety11.419.720.74913.530.0901.3201-0.75
2026-10-10Gety11.5413.790.82521.540.0501.501-0.84
2026-09-27Isiro11.041.280.3231.070.750.290110.48
2026-10-02Isiro11.251.030.6211.10.620.640010.78
2026-10-06Isiro10.540.590.2240.510.4600.0811-0.27
2026-10-10Isiro12.31.110.7681.050.3501.9401-0.86
2026-09-27Biena11.010.990.5050.980.2800.7401-0.61
2026-10-02Biena12.051.060.6861.140.5301.5201-0.68
2026-10-06Biena10.920.360.4110.370.710.220110.61
2026-10-10Biena10.710.260.3010.260.670.040110.52
2026-09-27Kabondo10.266.340.1565.340.2600110.43
2026-10-02Kabondo10.40.650.2560.60.2500.1511-0.33
2026-10-06Kabondo10.890.540.430.50.3100.5801-0.7
2026-10-10Kabondo11.350.820.5130.90.3101.0401-0.73
2026-09-27Ganga10.134.410.0874.230.1300110.34
2026-10-02Ganga11.260.780.6410.750.1201.1401-0.84
2026-10-06Ganga11.270.760.6420.710.1201.1501-0.88
2026-10-10Ganga10.250.150.1580.170.2500110.45
2026-09-27Rungu11.570.840.830.820.0401.5400-0.94
2026-10-02Rungu10.040.030.0280.030.0400110.19
2026-10-06Rungu10.630.760.4321.30.0500.5701-0.72
2026-10-10Rungu10.620.890.4290.890.0500.5701-0.72
2026-09-27Kilo12.41.951.0852.420.0402.3601-0.88
2026-10-02Kilo10.710.620.4871.210.0500.6601-0.72
2026-10-06Kilo10.120.060.0610.060.1200110.24
2026-10-10Kilo10.070.10.0320.080.0700110.18
2026-09-27Lolwa10.660.870.3291.180.4400.2111-0.31
2026-10-02Lolwa10.621.310.3521.60.300.3211-0.41
2026-10-06Lolwa10.631.390.3411.680.3500.2811-0.38
2026-10-10Lolwa10.681.380.4392.070.1300.5501-0.61
2026-09-27Aru10.030.770.0150.570.0300110.13
2026-10-02Aru10.030.890.0150.540.0300110.12
2026-10-06Aru10.761.130.5171.130.0400.7201-0.77
2026-10-10Aru11.62.130.862.780.0401.5601-0.86
2026-09-27Fataki10.650.950.3841.350.2400.4111-0.49
2026-10-02Fataki11.481.610.7881.790.0601.4101-0.81
2026-10-06Fataki10.20.160.0920.170.200110.3
2026-10-10Fataki10.10.090.0470.090.100110.21
2026-09-27Manguredjipa10.260.210.1390.190.2600110.38
2026-10-02Manguredjipa10.413.150.22.580.4100110.44
2026-10-06Manguredjipa10.910.690.4340.640.4200.4911-0.48
2026-10-10Manguredjipa10.653.160.2812.680.6300.0211-0.15
2026-09-27Kayna11.570.850.8440.840.0301.5401-0.87
2026-10-02Kayna10.10.060.0470.050.100110.21
2026-10-06Kayna10.160.090.0830.090.1600110.29
2026-10-10Kayna10.1810.980.0898.550.1800110.3
2026-09-27Adja100.070.0020.09000110.04
2026-10-02Adja10.911.090.6261.10.0100.900-0.92
2026-10-06Adja10.020.030.0090.020.0200110.09
2026-10-10Adja10.010.010.0030.010.0100110.06
2026-09-27Logo10.050.480.0260.340.0500110.16
2026-10-02Logo10.771.350.5271.360.0300.7401-0.79
2026-10-06Logo10.070.110.0370.090.0700110.2
2026-10-10Logo10.040.060.0190.050.0400110.15
2026-10-02Alimbongo10.590.210.3840.450.1300.4611-0.59
2026-10-06Alimbongo10.160.030.0790.050.1600110.28
2026-10-10Alimbongo10.150.180.0760.140.1500110.28
2026-09-27Tshopo10.058.280.037.670.0500110.2
2026-10-02Tshopo10.590.670.4070.660.0700.5201-0.67
2026-10-06Tshopo10.10.130.070.130.100110.31
2026-10-10Tshopo10.10.120.0680.110.100110.3
2026-09-27Buta10.0651.330.04144.740.0600110.23
2026-10-02Buta10.0614.740.04213.270.0600110.24
2026-10-06Buta10.0929.860.05727.10.0900110.28
2026-10-10Buta10.1123.720.06821.290.1100110.3
2026-09-27Viadana10.270.110.1680.140.2700110.46
2026-10-02Viadana10.20.310.1250.30.200110.4
2026-10-06Viadana10.172.760.1072.590.1700110.37
2026-10-10Viadana10.274.90.1624.440.2700110.44
2026-09-27Dungu10.011.70.0061.650.0100110.09
2026-10-02Dungu10.013.320.0093.210.0100110.11
2026-10-06Dungu10.013.410.0063.280.0100110.09
2026-10-10Dungu10.013.410.0083.130.0100110.1
2026-09-27Gombari10.020.40.0110.370.0200110.12
2026-10-02Gombari10.010.320.0060.290.0100110.09
2026-10-06Gombari10.020.740.0130.680.0200110.13
2026-10-10Gombari10.010.390.0080.350.0100110.1
2026-09-27Ariwara10.050.580.0230.410.0500110.15
2026-10-02Ariwara10.030.370.0130.310.0300110.12
2026-10-06Ariwara10.020.330.0110.260.0200110.1
2026-10-10Ariwara10.020.210.0080.180.0200110.09
2026-09-27Aungba10.030.830.0140.540.0300110.12
2026-10-02Aungba10.020.50.010.370.0200110.1
2026-10-06Aungba10.041.040.020.760.0400110.14
2026-10-10Aungba10.030.410.0120.290.0300110.11
2026-09-27Boga10.020.80.0090.430.0200110.08
2026-10-02Boga10.031.790.0131.140.0300110.11
2026-10-06Boga10.020.390.0080.250.0200110.08
2026-10-10Boga10.010.560.0060.410.0100110.08
2026-09-27Damas10.240.70.1140.570.2400110.33
2026-10-02Damas10.050.190.0260.160.0500110.17
2026-10-06Damas10.090.30.0350.220.0900110.18
2026-10-10Damas10.040.140.0170.110.0400110.13
2026-09-27Drodro10.110.750.0510.530.1100110.22
2026-10-02Drodro10.090.80.0450.660.0900110.22
2026-10-06Drodro10.141.420.0691.130.1400110.27
2026-10-10Drodro10.090.910.0470.730.0900110.23
2026-09-27Kambala10.011.150.0070.840.0100110.09
2026-10-02Kambala10.021.210.0070.750.0200110.09
2026-10-06Kambala10.032.070.0161.420.0300110.12
2026-10-10Kambala10.020.80.0090.570.0200110.09
2026-09-27Mahagi10.041.530.0180.950.0400110.13
2026-10-02Mahagi10.021.050.010.650.0200110.1
2026-10-06Mahagi10.041.430.01810.0400110.14
2026-10-10Mahagi10.043.070.0161.820.0400110.12
2026-09-27Tchomia10.651.10.2770.880.6500110.49
2026-10-02Tchomia10.30.920.1240.70.300110.32
2026-10-06Tchomia10.150.390.070.350.1500110.26
2026-10-10Tchomia10.130.40.0580.330.1300110.24
2026-09-27Goma10.025.640.0114.490.0200110.11
2026-10-02Goma10.024.250.013.510.0200110.11
2026-10-06Goma10.0528.060.02319.880.0500110.15
2026-10-10Goma10.0310.40.0136.350.0300110.11
2026-09-27Miti-Murhesa10.070.0440.0700110.24
2026-10-02Miti-Murhesa10.150.1010.1500110.37
2026-10-06Miti-Murhesa10.150.0950.1500110.35
2026-10-10Miti-Murhesa10.210.1320.2100110.4
2026-09-27Bulu10.040.050.0280.050.0400110.19
2026-10-02Bulu10.042.610.0252.560.0400110.19
2026-10-06Bulu10.031.10.0171.080.0300110.15
2026-10-10Bulu10.063.180.0373.020.0600110.22
2026-09-27Lubunga10.1115.590.07314.460.1100110.32
2026-10-02Lubunga10.1129.120.07426.880.1100110.32
2026-10-06Lubunga10.1230.10.0826.840.1200110.33
2026-10-10Lubunga10.260.940.1352.140.200110.41
2026-09-27Wanie-Rukula10.0612.750.03810.610.0600110.22
2026-10-02Wanie-Rukula10.0724.340.04421.510.0700110.25
2026-10-06Wanie-Rukula10.0737.930.04632.030.0700110.25
2026-10-10Wanie-Rukula10.0726.060.04621.760.0700110.24

Saving zone forecast outputs ​

The frozen zone fit's one-week-ahead forecast against the observed zone increments, and its scores against the two persistence rules, are written as release assets.

Write zone forecast outputs
julia
output_dir = get(
    ENV, "BVD_OUTPUT_DIR",
    joinpath(pkgdir(BVDOutbreakSize), "output")
)
mkpath(output_dir)
CSV.write(
    joinpath(output_dir, "zone_forecast_validation.csv"),
    zone_validation_table
)
CSV.write(
    joinpath(output_dir, "zone_forecast_scores.csv"),
    zone_validation_scores
)
"/home/runner/work/BVDOutbreakSize/BVDOutbreakSize/output/zone_forecast_scores.csv"
Write the summary bullets
julia
# The bullets under the summary heading at the top of the page. They read
# the scores built further down, so they are written here and read back
# when the site is assembled.
evaluation_forecast_zone_summary = let s = zone_validation_scores
    fmt(x) = ismissing(x) || (x isa Real && isnan(x)) ? "n/a" :
        string(round(x; digits = 2))
    if isempty(s)
        string(
            "- **Zone forecasts:** the zone tables end at or before the ",
            "frozen cut-off, so no zone forecast is scored in this build."
        )
    else
        patches = unique(s.patch)
        rule(p, m) = let r = s[(s.patch .== p) .& (s.method .== m), :]
            size(r, 1) == 0 ? nothing : first(eachrow(r))
        end
        beats(p) = let m = rule(p, "zone model")
            m === nothing ? false :
                all(
                    r -> r === nothing || m.log_score >= r.log_score,
                    [rule(p, "share persistence"), rule(p, "naive persistence")]
                )
        end
        observed_total = sum(s[s.method .== "zone model", :observed])
        overall = [
            string(
                "- **Split:** the zone model allocates the week better ",
                "than both persistence rules in ", count(beats, patches),
                " of ", length(patches), " provinces scored, over ",
                observed_total, " observed confirmed cases."
            ),
        ]
        function detail(p)
            m = rule(p, "zone model")
            m === nothing && return string("**", p, "**", "\n\n- Not scored.")
            return join(
                [
                    string("**", p, "**"), "",
                    string(
                        "- Log score of the split: ", fmt(m.log_score),
                        ", against ",
                        fmt(
                            rule(p, "share persistence") === nothing ?
                                missing : rule(p, "share persistence").log_score
                        ),
                        " for share persistence and ",
                        fmt(
                            rule(p, "naive persistence") === nothing ?
                                missing : rule(p, "naive persistence").log_score
                        ),
                        " for naive persistence."
                    ),
                    string(
                        "- Total over ", m.observed,
                        " observed cases: bias ", fmt(m.bias),
                        ", within the 90% interval: ",
                        ismissing(m.coverage_90) ? "n/a" :
                            string(m.coverage_90), "."
                    ),
                ], "\n"
            )
        end
        join(vcat([join(overall, "\n")], [detail(p) for p in patches]), "\n\n")
    end
end
dashboard_dir = joinpath(
    pkgdir(BVDOutbreakSize), "docs", "src", "summary_assets"
)
mkpath(dashboard_dir)
write(
    joinpath(dashboard_dir, "evaluation_forecast_zone.md"),
    evaluation_forecast_zone_summary
);
Add the across-release scores to the summary
julia
# A block on the zone forecasts scored across releases, after the
# per-province blocks above.
zone_release_summary = let ov = forecast_score_overview(
        _zone_release(zone_release_scores_df)
    )
    rows = filter(r -> !ismissing(r.rel_to_baseline), ov)
    size(rows, 1) == 0 ?
        "**Across releases**\n\n- $(_zone_release_empty)" :
        string(
            "**Across releases**\n\n- ", count(<(1), rows.rel_to_baseline),
            " of ", size(rows, 1), " zones scored beat the persistence ",
            "baseline, from the forecast made on ",
            minimum(zone_release_scores_df.made_date), " onwards."
        )
end
open(joinpath(dashboard_dir, "evaluation_forecast_zone.md"), "a") do io
    write(io, "\n\n", zone_release_summary)
end;