MODELCATALOG

Maximum-margin classifiers. Excellent on text.

sklearn.svm

svm

Maximum-margin classifiers. Excellent on text.

1 Get data Something to practise on
2 Split & tune An honest test set
3 Prepare Into numbers a model accepts
4 Compose One object, no leakage
5 Model The algorithm itself
6 Unsupervised No labels available
7 Measure Is it any good, and why
8 Utilities Glue and extension

What it is for

Finds the boundary with the widest margin between classes. LinearSVC in particular is a very strong text classifier and frequently edges out logistic regression on TF-IDF features.

Contents

LinearSVC common Text classification. Fast, scales well, handles sparse input
SVC Kernel SVM. Powerful on small data, scales badly past roughly 10k rows
LinearSVR, SVR Regression equivalents
OneClassSVM Novelty detection

Worked examples 1

Runnable, with the arguments that change the outcome called out. Output shapes are shown as comments where they help.

LinearSVC on text, with probabilities added

Frequently edges out logistic regression on TF-IDF. Its one gap is that it has no predict_proba — calibration fixes that.

from sklearn.svm import LinearSVC
from sklearn.calibration import CalibratedClassifierCV
from sklearn.pipeline import Pipeline

# Fast, handles sparse input directly, scales to large vocabularies
svc = LinearSVC(C=1.0, class_weight="balanced", max_iter=5000, dual="auto")

# LinearSVC gives decision_function, not predict_proba. If you need to
# threshold on a probability, wrap it.
clf = CalibratedClassifierCV(svc, method="sigmoid", cv=5)

pipe = Pipeline([("tfidf", TfidfVectorizer(ngram_range=(1, 2), min_df=3)),
                 ("clf", clf)])
pipe.fit(X_train, y_train)

print(pipe.predict_proba(X_test[:3]).round(3))

Worth knowing

  1. LinearSVC has no predict_proba. Wrap it in CalibratedClassifierCV if you need probabilities.