Determines bias from quantile forecasts. For an increasing number of quantiles this measure converges against the sample based bias version for integer and continuous forecasts.
Arguments
- predictions
vector of length corresponding to the number of quantiles that holds predictions
- quantiles
vector of corresponding size with the quantiles for which predictions were made. If this does not contain the median (0.5) then the median is imputed as being the mean of the two innermost quantiles.
- true_value
a single true value
Details
For quantile forecasts, bias is measured as
$$ B_t = (1 - 2 \cdot \max \{i | q_{t,i} \in Q_t \land q_{t,i} \leq x_t\}) \mathbf{1}( x_t \leq q_{t, 0.5}) \\ + (1 - 2 \cdot \min \{i | q_{t,i} \in Q_t \land q_{t,i} \geq x_t\}) 1( x_t \geq q_{t, 0.5}),$$
where \(Q_t\) is the set of quantiles that form the predictive distribution at time \(t\). They represent our belief about what the true value $x_t$ will be. For consistency, we define \(Q_t\) such that it always includes the element \(q_{t, 0} = - \infty\) and \(q_{t,1} = \infty\). \(1()\) is the indicator function that is \(1\) if the condition is satisfied and $0$ otherwise. In clearer terms, \(B_t\) is defined as the maximum percentile rank for which the corresponding quantile is still below the true value, if the true value is smaller than the median of the predictive distribution. If the true value is above the median of the predictive distribution, then $B_t$ is the minimum percentile rank for which the corresponding quantile is still larger than the true value. If the true value is exactly the median, both terms cancel out and \(B_t\) is zero. For a large enough number of quantiles, the percentile rank will equal the proportion of predictive samples below the observed true value, and this metric coincides with the one for continuous forecasts.
Bias can assume values between -1 and 1 and is 0 ideally (i.e. unbiased).
Author
Nikos Bosse nikosbosse@gmail.com
Examples
predictions <- c(
705.500, 1127.000, 4006.250, 4341.500, 4709.000, 4821.996,
5340.500, 5451.000, 5703.500, 6087.014, 6329.500, 6341.000,
6352.500, 6594.986, 6978.500, 7231.000, 7341.500, 7860.004,
7973.000, 8340.500, 8675.750, 11555.000, 11976.500
)
quantiles <- c(0.01, 0.025, seq(0.05, 0.95, 0.05), 0.975, 0.99)
true_value <- 8062
bias_quantile(predictions, quantiles, true_value = true_value)
#> [1] -0.8