# Compare Two Models Based on Subset of Common Forecasts

Source:`R/pairwise-comparisons.R`

`compare_two_models.Rd`

This function compares two models based on the subset of forecasts for which
both models have made a prediction. It gets called
from `pairwise_comparison_one_group()`

, which handles the
comparison of multiple models on a single set of forecasts (there are no
subsets of forecasts to be distinguished). `pairwise_comparison_one_group()`

in turn gets called from from `pairwise_comparison()`

which can handle
pairwise comparisons for a set of forecasts with multiple subsets, e.g.
pairwise comparisons for one set of forecasts, but done separately for two
different forecast targets.

## Usage

```
compare_two_models(
scores,
name_model1,
name_model2,
metric,
one_sided = FALSE,
test_type = c("non_parametric", "permutation"),
n_permutations = 999
)
```

## Arguments

- scores
A data.table of scores as produced by

`score()`

.- name_model1
character, name of the first model

- name_model2
character, name of the model to compare against

- metric
A character vector of length one with the metric to do the comparison on. The default is "auto", meaning that either "interval_score", "crps", or "brier_score" will be selected where available. See

`available_metrics()`

for available metrics.- one_sided
Boolean, default is

`FALSE`

, whether two conduct a one-sided instead of a two-sided test to determine significance in a pairwise comparison.- test_type
character, either "non_parametric" (the default) or "permutation". This determines which kind of test shall be conducted to determine p-values.

- n_permutations
numeric, the number of permutations for a permutation test. Default is 999.