Build & train

scikit-learn training pattern

Canonical classical-ML example for understanding the entire lifecycle.

Inputs

clean train/validation text + labels

Outputs

fitted sklearn Pipeline + validation metrics

Core explanation

I use the same SageMaker training lifecycle regardless of framework; the main variation is the framework image and training script.

Mental model

What this means before worrying about syntax.

Train preprocessing and estimator objects, persist the fitted objects, and package them as the model artifact.

Classical ML often has fitted preprocessing state plus estimator state.
The fitted vectorizer must travel with the classifier.
A strong linear baseline is valuable even if a transformer may be evaluated later.

Lifecycle position

complaint CSV -> TF-IDF -> LogisticRegression -> model artifact

Implementation anatomy

The sequence you would actually build or configure.

1
Split data before fitting text features.
2
Fit TF-IDF on train only.
3
Transform validation with the fitted vectorizer.
4
Fit LogisticRegression.
5
Evaluate class-sensitive metrics.
6
Persist vectorizer + model or one sklearn Pipeline.

Objects / terms worth remembering

TfidfVectorizerLogisticRegressionjoblibModelTrainer

Concrete example

A real model-development or production flow.

Text: my account shows a debit for a transaction I did not do

↓

TF-IDF uses learned vocabulary/IDF

↓

LogisticRegression returns category probabilities

↓

Business rule auto-routes only if max probability >= 0.75

↓

Low-confidence case goes to review

Implementation lab

Concrete code, data manipulation and configuration for this lifecycle step.

Train a leakage-safe sklearn Pipeline

python

from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression

pipeline = Pipeline([
    ("tfidf", TfidfVectorizer(
        lowercase=True,
        ngram_range=(1, 2),
        min_df=args.min_df,
        max_features=100_000,
    )),
    ("clf", LogisticRegression(
        C=args.C,
        max_iter=1500,
        class_weight="balanced",
    )),
])

# fit() learns BOTH TF-IDF state and classifier weights from train only.
pipeline.fit(train["text"], train["label"])

Evaluate validation probabilities and threshold coverage

python

For routing systems, coverage + error rate above the decision threshold is often more meaningful than raw accuracy.

import numpy as np
from sklearn.metrics import classification_report, f1_score

proba = pipeline.predict_proba(validation["text"])
classes = pipeline.classes_

best_idx = proba.argmax(axis=1)
pred = classes[best_idx]
confidence = proba.max(axis=1)

print("macro_f1 =", f1_score(
    validation["label"], pred, average="macro"
))
print(classification_report(validation["label"], pred))

threshold = args.confidence_threshold
auto_mask = confidence >= threshold

print("auto-route coverage =", auto_mask.mean())
print(
    "accuracy among auto-routed =",
    (pred[auto_mask] == validation.loc[auto_mask, "label"]).mean()
)

Persist one pipeline instead of separate incompatible pieces

python

import joblib
import os

joblib.dump(
    pipeline,
    os.path.join(os.environ["SM_MODEL_DIR"], "pipeline.joblib")
)

Design decisions that matter

Pipeline vs separate artifacts.
Class weights.
Probability threshold and calibration.
When a transformer materially improves the business outcome.

Common failure modes

Near-perfect metrics should trigger leakage/duplicate checks.
Refitting the vectorizer changes feature space and breaks model compatibility.
Probability threshold is a business operating decision, not a training default.

Production readiness check

Items to close before treating this step as production-ready.

✓ preprocessor packaged✓ label mapping✓ threshold✓ class imbalance✓ calibration✓ leakage/duplicates