How do we evaluate modelling work, and what does that imply about modelling responsibly?
TipGet involved
TipWhat?
We brought together around 30 participants to explore how evaluation links to professional norms, values, and responsibilities. We shared expertise across epidemiological and public health contexts, and put discussion into action by co-designing modeller training materials, and setting individual next steps to improve practice.
We aimed to:
- compare perspectives on evaluation in epidemiological and public health contexts
- collaboratively map out what this means for professional norms, values, and responsibilities
- experiment with what this looks like in practice by co-designing training material for modellers
TipWhat happened?
| Time | Content | Lead |
|---|---|---|
| 09:45 | Arrival | |
| 10:00 | Welcome activity | Participants |
| 10:15 | Introduction & aims | Kath Sherratt, Sebastian Funk, Erica Thompson |
| Evaluation & scientific responsibility | ||
| 10:30 | Modelling & scientific responsibility | Sebastian Funk |
| 10:40 | Evaluation across epidemiological contexts | Panel: Matt Keeling (Uni. Warwick), Kaitlyn Johnson (LSHTM), Jonathon Mellor (UKHSA) |
| 11:10 | Implications for scientific responsibility | Participants: Nested group discussion |
| 11:30 | Coffee break | |
| 11:45 | Training scientifically responsible modellers | Participants: Small group co-design |
| 12:30 | Lunch | |
| Evaluation & social responsibility | ||
| 13:30 | Modelling & social responsibility | Erica Thompson |
| 13:40 | Evaluation across public health contexts | Panel: Graham Medley (LSHTM), Juan Vesga (UKHSA), Katy Gaythorpe (Imperial/Vaccine Impact Modelling Consortium) |
| 14:10 | Implications for social responsibility | Participants: Nested group discussion |
| 14:30 | Fresh air break | |
| 15:00 | Review so far | Kath Sherratt |
| 15:05 | Training socially responsible modellers | Participants: Small group co-design |
| 15:50 | Gallery of co-designed training materials | |
| Implementation | ||
| 16:00 | Crowd sourcing: big priorities and smallest next steps | Participants: Rapid rating exercise |
| 16:20 | Summary and actions | Kath Sherratt, Sebastian Funk, Erica Thompson |
| 16:30 | Close |
Workshop design motivated by principles of structured groupwork.
TipWhat next?
Quick wins
We crowd-sourced ideas for next best action for an individual to improve evaluation and responsibility in their work. These are the highest-ranked actions to take now1:
| Smallest next steps |
|---|
| Make my PhD code publicly available, even though it’s horrid |
| Before doing any modelling, think about and write down evaluation criteria for afterwards |
| Embed PPIE/community in modelling |
| Write an analysis plan beforehand with collaborators and stakeholders |
Long term
This is just one effort in a much wider community working to improve evaluation and understand resonsibility in infectious disease modelling. Let us know you’re interested and we’ll keep in touch with workshop outcomes.
Organised by Kath Sherratt (LSHTM), Seb Funk (LSHTM), and Erica Thompson (UCL), and supported by LSHTM CMMID.
Footnotes
Table shows highest ranked actions among participants’ next steps. Each participant wrote one small action they would personally take. All actions were anonymously swapped among participants and scored from one to five (best). We repeated the swap-and-score process until each action had at least three independent scores. Table shows actions with an average of 5/5.↩︎