ReliabilityModelPlot.reliability_diagram

static ReliabilityModelPlot.reliability_diagram(df_eval, figsize=(5, 5), color='tab:blue', title=None, ax=None)

Calibration curve — mean predicted score vs. empirical positive rate, per bin.

Points on the diagonal are perfectly calibrated; points below it mean the score overstates the true positive rate (over-confident), above it under-confident.

Parameters:

• df_eval (pd.DataFrame) – Output of ReliabilityModel.eval() (per-bin mean_score / empirical_pos).

• figsize (tuple, default=(5, 5)) – Figure size (used only when ax is None).

• color (str, default="tab:blue") – Line/marker color of the model curve.

• title (str, optional) – Axes title.

• ax (matplotlib.axes.Axes, optional) – Axes to draw on; a new figure is created if None.

Returns:

• fig (matplotlib.figure.Figure) – The created (or parent) figure.

• ax (matplotlib.axes.Axes) – The axes drawn on.

Examples

ReliabilityModelPlot().reliability_diagram() draws the calibration curve from ReliabilityModel.eval() — mean predicted score vs. empirical positive rate, against the diagonal of perfect calibration.

import aaanalysis as aa
import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_classification
aa.options["verbose"] = False
aa.plot_settings()
X, labels = make_classification(n_samples=160, n_features=10, n_informative=6, random_state=42)
X_train, labels_train = X[:120], labels[:120]
# include one clearly out-of-distribution sample so the trust status is visible
X_new = np.vstack([X[120:], (X_train[labels_train == 1][0] + 20.0)[None, :]])
rm = aa.ReliabilityModel(random_state=42).fit(X=X_train, labels=labels_train)
df_rel = rm.predict(X=X_new)
df_eval = rm.eval(X=X[120:], labels=labels[120:])

aa.ReliabilityModelPlot().reliability_diagram(df_eval=df_eval, figsize=(5, 5),
                                              color="tab:blue", title="Calibration", ax=None)
plt.tight_layout()
plt.show()