Metrics¶

Evaluation metrics for distributional regression models, including both traditional point prediction metrics and distribution-aware measures.

Distributional Metrics¶

CRPS (Continuous Ranked Probability Score)¶

`drn.metrics.crps(obs, grid, cdf_on_grid)` ¶

Compute CRPS using the provided grid and CDF values with PyTorch tensors.

:param obs: observed value(s) :param grid: a grid over y values :param cdf_on_grid: tensor of corresponding CDF values or a 2D tensor where each column is a CDF :return: CRPS value(s) as a PyTorch tensor

Quantile Score¶

`drn.metrics.quantile_score(y_true, y_pred, p)` ¶

Compute the quantile score for predictions at a specific quantile.

:param y_true: Actual target values as a Pandas Series or PyTorch tensor. :param y_pred: Predicted target values as a numpy array or PyTorch tensor. :param p: The cumulative probability as a float :return: The quantile score as a PyTorch tensor.

Quantile Losses¶

`drn.metrics.quantile_losses(p, model, model_name, X, y, max_iter=1000, tolerance=5e-05, l=None, u=None, print_score=True)` ¶

Calculate and optionally print the quantile loss for the given data and model.

:param p: The cumulative probability ntile as a float :param model: The trained model. :param model_name: The name of the trained model. :param X: Input features as a Pandas DataFrame or numpy array. :param y: True target values as a Pandas Series or numpy array. :param max_iter: The maximum number of iterations for the quantile search algorithm. :param tolerance: The tolerance for convergence of the the quantile search algorithm. :param l: The lower bound for the quantile search :param u: The upper bound for the quantile search :param print_score: A boolean indicating whether to print the score. :return: The quantile loss as a PyTorch tensor.

Point Prediction Metrics¶

RMSE (Root Mean Squared Error)¶

`drn.metrics.rmse(y, y_hat)` ¶

Compute the Root Mean Square Error (RMSE) between the true values and predictions.

:param y: True target values. Can be a Pandas Series or a PyTorch tensor. :param y_hat: Predicted target values. Should be a PyTorch tensor. :return: The RMSE as a PyTorch tensor.

Utility Functions¶

Binary Search for Quantiles¶

`drn.utils.binary_search_icdf(distribution, p, l=None, u=None, max_iter=1000, tolerance=1e-07)` ¶

Generic binary search implementation for inverse CDF (quantiles).

This function can be used by any distribution that has a cdf method but doesn't have its own icdf implementation.

Args: distribution: Distribution object with a cdf method p: cumulative probability value at which to evaluate icdf l: lower bound for the quantile search u: upper bound for the quantile search max_iter: maximum number of iterations tolerance: stopping criteria for convergence

Returns: A tensor of shape (1, batch_shape) containing the inverse CDF values.