Modelling of emerging threats and epidemics

Robert Koch Colloquium, 19 May 2025

Sebastian Funk

https://epiforecasts.io

Models are a tool to combine data (what we know) with assumptions and theory (what we think) to learn about what we don’t know.

Example: January 2020

Can COVID-19 be controlled by contact tracing?

Example: January 2020

Can COVID-19 be controlled by contact tracing?

Hellewell et al., Lancet Glob Health, 2020

Probability of control vs. tracing effort

Probability of control depends on intensity of transmission and contact tracing effort.

Hellewell et al., Lancet Glob Health, 2020

We illustrate the potential impact that flawed model inferences can have on public health policy with the model described […] by Joel Hellewell and colleagues, which is part of the scientific evidence informing the UK Government’s response to COVID-19.

Gudrasani & Ziauddeen, Lancet Glob Health, 2020

“All models are wrong, but some are useful”

George Box

“All models are wrong, but some are useful”

  • wrong: how wrong?
  • some: which ones?

How wrong are models?

What happens next?

Time series of case incidence.

Epidemiological context

Acute Respiratory Infection (ARI) consultations in Germany.

Source: ERVISS

What really happened

Acute Respiratory Infection (ARI) consultations in Germany.

Source: ERVISS

What really happened

Acute Respiratory Infection (ARI) consultations in Germany.

Source: ERVISS

How wrong are models? — Forecast evaluation

Assess quality of models by how closely prediction matches reality

Funk et al., PLOS Comp Biol, 2019

Types of predictive modelling

Forecasts vs. Scenarios

Image from: https://covid19scenariomodelinghub.org/

Forecast hubs support systematic collection of forecasts

Reich et al., Am J Public Health, 2022

Ensembles beat individual models

Sherratt et al., eLife, 2023

Case forecasts degrade quickly

Sherratt et al., eLife, 2023

No model type dominates

Sherratt et al., medRxiv, 2025

Unpredictable changes in human behaviour made forecasting harder

Gimma et al., PLOS Medicine, 2022

Comparing human forecasts to models

Bosse et al., PLOS Comp Biol, 2022
Bosse et al., Wellcome Open Res, 2024

Humans are better than models at predicting cases, but not deaths

Bosse et al., PLOS Comp Biol, 2022

Broader evaluations: not just forecasts

Any model of the future is a prediction and can be evaluated as such.

Howerton et al., Nat Comm, 2023

Which models are useful?

Usefulness doesn’t have to coincide with accuracy

Saltelli, arXiv, 2018

Few frameworks exist to assess utility

Rao et al., medRxiv, 2025

Collaboration with Robert Koch Institute / WHO Pandemic Hub.

Outlook: how can we improve modelling in future epidemics

Speed matters

An 80 % right paper before a policy decision is made it is worth ten 95 % right papers afterwards

Whitty, BMC Medicine, 2015

Preparing for the next epidemic: predictable tasks

Figure courtesy of Adam Kucharski

Predictable task: mpox nowcasting

UKHSA, 2022
Overton et al., PLOS Comp Biol, 2023

Unpredictable task: the impact of network structure on mpox transmission

Endo et al. Science, 2022

Initiatives to create and disseminate tools for predictable tasks.

What should we do next?

  • Models aren’t crystal balls — but used carefully, they can give us a glimpse of the near future and guide decision-making.
  • We must evaluate both their accuracy and their real-world utility.
  • We should invest in modelling infrastructure that’s flexible, reusable, and ready.

Thank you

Slides at

https://epiforecasts.io/slides/rki_20250519.html