MODELCATALOG

The trivial baseline. Use it in every experiment.

sklearn.dummy

dummy

The trivial baseline. Use it in every experiment.

1 Get data Something to practise on
2 Split & tune An honest test set
3 Prepare Into numbers a model accepts
4 Compose One object, no leakage
5 Model The algorithm itself
6 Unsupervised No labels available
7 Measure Is it any good, and why
8 Utilities Glue and extension

What it is for

Predicts the most frequent class, or the mean, with no learning whatsoever. Its whole purpose is to sit in your results table so you know whether the real model is earning anything.

On imbalanced data the dummy often scores 95% accuracy, which is exactly the point — seeing that number next to your model's stops you reporting it as a success.

Contents

DummyClassifier common strategy='most_frequent', 'stratified' or 'prior'
DummyRegressor strategy='mean' or 'median'

Worked examples 1

Runnable, with the arguments that change the outcome called out. Output shapes are shown as comments where they help.

The baseline that belongs in every results table

On imbalanced data the dummy often scores 95% accuracy. Seeing that next to your model's number is what stops you reporting it as a success.

from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, average_precision_score

# 0.5% positive rate - fraud-shaped
dummy = DummyClassifier(strategy="most_frequent").fit(X_train, y_train)
p_dummy = dummy.predict_proba(X_test)[:, 1]

print("DUMMY   accuracy", round(accuracy_score(y_test, dummy.predict(X_test)), 4))
print("DUMMY   PR AUC  ", round(average_precision_score(y_test, p_dummy), 4))
# DUMMY   accuracy 0.9948      <- looks superb
# DUMMY   PR AUC   0.0052      <- and is completely useless

p_model = model.predict_proba(X_test)[:, 1]
print("MODEL   accuracy", round(accuracy_score(y_test, model.predict(X_test)), 4))
print("MODEL   PR AUC  ", round(average_precision_score(y_test, p_model), 4))
# MODEL   accuracy 0.9951      <- barely different from the dummy
# MODEL   PR AUC   0.3140      <- 60x the dummy: THIS is the real gain
#
# Two metrics, opposite conclusions. The dummy row is what makes it
# obvious which one to report.