ModelEvaluatorPlot.learning_curve

static ModelEvaluatorPlot.learning_curve(df_curve, *, metric=None, figsize=(6, 4), colors=None, show_ci=True)

Plot a learning curve: one metric versus training size per model, with a CI band.

One line per model connects the mean cross-validated score at each training size; the shaded band shows the bootstrap confidence interval of the mean (falling back to the fold std where the CI is NaN). A curve still rising at the largest size suggests the task is sampling-limited; a flat curve suggests it has saturated.

Added in version 1.2.0.

Parameters:

• df_curve (pd.DataFrame, shape (n_models * n_train_sizes * n_metrics, 8)) – Learning-curve table from ModelEvaluator.learning_curve() with columns model, train_size, metric, score, score_std, ci_low, ci_high, and n_scores. Model and metric values must be non-empty strings; training sizes must be integer-valued and at least 2, score counts positive integer-valued, and scores and standard deviations finite (with non-negative standard deviations). Each (model, training size, metric) tuple must occur once and each model/metric curve must have at least two training sizes. Confidence bounds must be finite and ordered, or both NaN when the curve was computed with ci=None.

• metric (str or None, default=None) – Metric to plot; must be one of the metrics in df_curve. Defaults to "mcc" when present, otherwise the first metric of df_curve.

• figsize (tuple of int or float, default=(6, 4)) – Positive figure dimensions (width, height) in inches; changes the rendered figure size.

• colors (str, list of str, or None, default=None) – One color per model (in first-appearance order); a single color name counts as one color, not as a sequence of characters. If None, uses the package color list.

• show_ci (bool, default=True) – If True, draw the confidence band (ci_low / ci_high, or score_std where the CI is NaN) around each curve. If False, draws only the score lines.

Returns:

• fig (Figure) – Figure object for the learning-curve plot.

• ax (Axes) – Axes object of the learning-curve line plot.

Raises:

ValueError – If df_curve is not a DataFrame with the learning-curve columns or holds no rows, its values do not form a valid learning-curve table, metric is not one of its metrics, figsize is not a positive numeric pair, show_ci is not boolean, or colors is not a valid color string/list with at least one color per model.

See also

• ModelEvaluator.learning_curve(): the respective computation method.

Examples

ModelEvaluatorPlot.learning_curve draws one metric versus training size from ModelEvaluator.learning_curve, one line per model with a shaded bootstrap confidence band. First, the dataset, feature matrix, and learning curve:

import aaanalysis as aa
aa.options["verbose"] = False  # Disable verbosity

# DOM_GSEC example dataset + a small feature set (see [Breimann25]_)
df_seq = aa.load_dataset(name="DOM_GSEC")
labels = df_seq["label"].to_list()
df_feat = aa.load_features(name="DOM_GSEC").head(20)

# Build the CPP feature matrix X
sf = aa.SequenceFeature()
df_parts = sf.get_df_parts(df_seq=df_seq)
X = sf.feature_matrix(features=df_feat["feature"], df_parts=df_parts)

me = aa.ModelEvaluator(models=["rf", "log_reg"], random_state=42, verbose=False)
df_curve = me.learning_curve(X, labels, train_sizes=[0.1, 0.25, 0.5, 0.75, 1.0], n_rounds=3,
                             metrics=["balanced_accuracy", "mcc"])

Plot df_curve for a chosen metric (default mcc) with a figsize and per-model colors:

import matplotlib.pyplot as plt
aa.plot_settings()
fig, ax = aa.ModelEvaluatorPlot.learning_curve(df_curve=df_curve, metric="balanced_accuracy",
                                               figsize=(6, 4), colors=["tab:blue", "tab:orange"])
plt.tight_layout()
plt.show()

Set show_ci=False to draw only the mean curves:

fig, ax = aa.ModelEvaluatorPlot.learning_curve(df_curve=df_curve, metric="mcc", show_ci=False)
plt.tight_layout()
plt.show()