Function reference • scoringutils

Package documentation

scoringutils scoringutils-package: scoringutils: Utilities for Scoring and Assessing Predictions

Functions to check and analyse inputs

avail_forecasts(): Display Number of Forecasts Available

check_forecasts(): Check forecasts

find_duplicates(): Find duplicate forecasts

log_shift(): Log transformation with an additive shift

print(<scoringutils_check>): Print output from check_forecasts()

transform_forecasts(): Transform forecasts and observed values

Functions for convenient forecast evaluation

score(): Evaluate forecasts

add_coverage(): Add coverage of central prediction intervals

correlation(): Correlation Between Metrics

pairwise_comparison(): Do Pairwise Comparisons of Scores

pit(): Probability Integral Transformation (data.frame Format)

summarise_scores() summarize_scores(): Summarise scores as produced by score()

Lower-level scoring functions

abs_error(): Absolute Error

ae_median_quantile(): Absolute Error of the Median (Quantile-based Version)

ae_median_sample(): Absolute Error of the Median (Sample-based Version)

bias_quantile(): Determines Bias of Quantile Forecasts

bias_range(): Determines Bias of Quantile Forecasts based on the range of the prediction intervals

bias_sample(): Determines bias of forecasts

brier_score(): Brier Score

crps_sample(): Ranked Probability Score

dss_sample(): Dawid-Sebastiani Score

interval_score(): Interval Score

logs_binary(): Log Score for Binary outcomes

logs_sample(): Logarithmic score

mad_sample(): Determine dispersion of a probabilistic forecast

pit_sample(): Probability Integral Transformation (sample-based version)

quantile_score(): Quantile Score

se_mean_sample(): Squared Error of the Mean (Sample-based Version)

squared_error(): Squared Error

Data wrangling helpers

merge_pred_and_obs(): Merge Forecast Data And Observations

sample_to_quantile(): Change Data from a Sample Based Format to a Quantile Format

set_forecast_unit(): Set unit of a single forecast manually

Functions for plotting and data visualisation

plot_avail_forecasts(): Visualise Where Forecasts Are Available

plot_correlation(): Plot Correlation Between Metrics

plot_heatmap(): Create a Heatmap of a Scoring Metric

plot_interval_coverage(): Plot Interval Coverage

plot_pairwise_comparison(): Plot Heatmap of Pairwise Comparisons

plot_pit(): PIT Histogram

plot_predictions(): Plot Predictions vs True Values

plot_quantile_coverage(): Plot Quantile Coverage

plot_ranges(): Plot Metrics by Range of the Prediction Interval

plot_score_table(): Plot Coloured Score Table

plot_wis(): Plot Contributions to the Weighted Interval Score

make_NA() make_na(): Make Rows NA in Data for Plotting

theme_scoringutils(): Scoringutils ggplot2 theme

Internal functions

check_equal_length(): Check Length

check_metrics(): Check whether the desired metrics are available in scoringutils

check_not_null(): Check Variable is not NULL

check_predictions(): Check Prediction Input For Lower-level Scoring Functions

check_quantiles(): Check that quantiles are valid

check_summary_params(): Check input parameters for summarise_scores()

check_true_values(): Check Observed Value Input For Lower-level Scoring Functions

collapse_messages(): Collapse several messages to one

compare_two_models(): Compare Two Models Based on Subset of Common Forecasts

delete_columns(): Delete Columns From a Data.table

geom_mean_helper(): Calculate Geometric Mean

get_forecast_unit(): Get unit of a single forecast

get_prediction_type(): Get prediction type of a forecast

get_protected_columns(): Get protected columns from a data frame

get_target_type(): Get type of the target true values of a forecast

infer_rel_skill_metric(): Infer metric for pairwise comparisons

is_scoringutils_check(): Check whether object has been checked with check_forecasts()

pairwise_comparison_one_group(): Do Pairwise Comparison for one Set of Forecasts

permutation_test(): Simple permutation test

prediction_is_quantile(): Check if predictions are quantile forecasts

quantile_to_range_long(): Change Data from a Plain Quantile Format to a Long Range Format

range_long_to_quantile(): Change Data from a Range Format to a Quantile Format

sample_to_range_long(): Change Data from a Sample Based Format to a Long Interval Range Format

score_binary(): Evaluate forecasts in a Binary Format

score_quantile(): Evaluate forecasts in a Quantile-Based Format

score_sample(): Evaluate forecasts in a Sample-Based Format (Integer or Continuous)

scoringutils scoringutils-package: scoringutils: Utilities for Scoring and Assessing Predictions

Example data and information

example_binary: Binary Forecast Example Data

example_continuous: Continuous Forecast Example Data

example_integer: Integer Forecast Example Data

example_point: Point Forecast Example Data

example_quantile: Quantile Example Data

example_quantile_forecasts_only: Quantile Example Data - Forecasts only

example_truth_only: Truth data only

available_metrics(): Available metrics in scoringutils

metrics: Summary information for selected metrics