Tabular classification

What it is

For all the attention that language models attract, the models making real decisions in a bank are overwhelmingly tabular. Credit approval, fraud scoring, churn, propensity, collections prioritisation — structured data, supervised learning, well-understood algorithms.

The important and slightly unfashionable fact is that gradient-boosted trees still win here. On tabular data with a few hundred features, XGBoost or LightGBM generally beats a neural network, trains in minutes, and comes with mature explainability tooling. The interesting engineering is in the features, the validation design and the monitoring — not the algorithm.

Model families 4

Sub-categories within this task. The distinction that matters is not which is best, but which fits your constraints.

Gradient-boosted trees

Megabytes. CPU trains fine.

Use when: The default and usually the winner. Handles mixed types, missing values and non-linear interactions without much preparation.
Avoid when: You need a model a regulator can read line by line, or you need smooth extrapolation beyond the training range.
Examples: XGBoostLightGBMCatBoost

Logistic regression scorecards

Kilobytes.

Use when: Credit decisioning, where explainability is a legal requirement rather than a preference. Weight-of-evidence binning plus logistic regression remains the industry standard for a reason — every point of the score traces to an attribute.
Avoid when: Complex interactions matter more than interpretability.
Examples: statsmodels / scikit-learn with WOE binning

Neural tabular models

1M–50M.

Use when: Very high cardinality categoricals, or you want to combine tabular features with text or image inputs in one model.
Avoid when: Standard tabular problems — they rarely beat boosting and cost far more to tune.
Examples: TabTransformerFT-TransformerSAINT

AutoML

Ensembles of the above.

Use when: Establishing a strong baseline quickly, or when you want to know what is achievable before investing in features.
Avoid when: The resulting ensemble has to pass model validation — explaining a stacked ensemble is materially harder.
Examples: AutoGluonSageMaker Autopilot

In a bank 4

The third line in each is the part that makes it harder than the textbook version.

Credit risk and application scoring

Probability of default from application and bureau data, feeding an approve/decline/refer decision.

The wrinkleDenials require a statement of specific reasons under Regulation B, so you must be able to say which attributes drove an individual decision. This is why scorecards persist and why boosting, where used, is paired with SHAP and tight monotonic constraints.

Transaction fraud scoring

Score card and payment transactions in real time.

The wrinkleExtreme imbalance — often well under 0.1% positive — and a latency budget in the tens of milliseconds. Evaluate on precision-recall, never ROC AUC, and measure at the operating point you will actually run at.

Customer churn and attrition

Identify accounts likely to close so retention can intervene.

The wrinkleThe label depends on a definition you choose — is dormancy churn? Most of the difficulty is in defining the target, not modelling it.

AML alert prioritisation

Rank alerts from rules engines so investigators work the likeliest first.

The wrinkleYour labels come from past investigator decisions, which carry their own biases. The model learns to predict investigator behaviour, not criminality — worth stating plainly to stakeholders.

Code 3

Named functions and the arguments that actually change the outcome. Comments flag the traps.

XGBoost on imbalanced data, validated by time

The two most common mistakes in banking tabular work are a random split and optimising ROC AUC on imbalanced data. Both are corrected here.

import xgboost as xgb
import numpy as np
from sklearn.metrics import average_precision_score, precision_recall_curve

# 1. TIME-BASED split. A random split leaks the future and flatters
#    every metric you will report.
train = df[df.booking_date <  "2025-01-01"]
valid = df[(df.booking_date >= "2025-01-01") & (df.booking_date < "2025-04-01")]
test  = df[df.booking_date >= "2025-04-01"]

# 2. scale_pos_weight handles imbalance without resampling
neg, pos = (train.y == 0).sum(), (train.y == 1).sum()

model = xgb.XGBClassifier(
    n_estimators=2000,
    learning_rate=0.03,
    max_depth=5,
    subsample=0.8,
    colsample_bytree=0.8,
    scale_pos_weight=neg / pos,
    eval_metric="aucpr",          # PR, not ROC - ROC lies on imbalanced data
    early_stopping_rounds=100,
    # Monotonic constraints encode domain knowledge and make the model
    # defensible: more missed payments must never reduce risk.
    # monotone_constraints={"months_delinquent": 1, "income": -1},
)

model.fit(train[FEATURES], train.y,
          eval_set=[(valid[FEATURES], valid.y)], verbose=100)

p = model.predict_proba(test[FEATURES])[:, 1]
print("PR AUC:", average_precision_score(test.y, p))

# 3. Choose the operating point deliberately - what recall can the
#    investigation team actually absorb?
prec, rec, thr = precision_recall_curve(test.y, p)
for target in (0.5, 0.7, 0.9):
    i = np.argmin(np.abs(rec - target))
    print(f"recall {rec[i]:.2f} -> precision {prec[i]:.2f} @ {thr[i]:.4f}")

Per-decision explanations with SHAP

This is what makes boosting usable where individual decisions must be justified.

import shap

explainer = shap.TreeExplainer(model)
sv = explainer.shap_values(test[FEATURES])

# Population level - what drives the model overall
shap.summary_plot(sv, test[FEATURES])

# Individual level - the reasons for ONE decision, which is what an
# adverse action notice or a customer complaint actually requires
def reasons(i, k=4):
    contrib = sorted(zip(FEATURES, sv[i]), key=lambda x: -abs(x[1]))[:k]
    return [{"feature": f,
             "value": test[FEATURES].iloc[i][f],
             "effect": "increases risk" if v > 0 else "reduces risk",
             "magnitude": round(float(abs(v)), 4)}
            for f, v in contrib]

for r in reasons(0):
    print(r)

The same model as a SageMaker training job

Built-in XGBoost: you supply data and hyperparameters, AWS supplies the algorithm container.

from sagemaker.estimator import Estimator
from sagemaker import image_uris

container = image_uris.retrieve("xgboost", region, version="1.7-1")

xgb_est = Estimator(
    image_uri=container,
    role=role,
    instance_count=1,
    instance_type="ml.m5.xlarge",     # CPU is fine; GPU is rarely worth it
    output_path=f"s3://{bucket}/models",
)

xgb_est.set_hyperparameters(
    objective="binary:logistic",
    eval_metric="aucpr",
    num_round=2000,
    early_stopping_rounds=100,
    max_depth=5, eta=0.03,
    subsample=0.8, colsample_bytree=0.8,
    scale_pos_weight=neg / pos,
)

# Built-in XGBoost expects CSV with the LABEL IN THE FIRST COLUMN
# and no header row. This trips up nearly everyone once.
xgb_est.fit({"train": train_s3, "validation": valid_s3})

What catches people out 5

Target leakage is the number one killer

A feature computed after the outcome — a collections flag on a default model, a chargeback field on a fraud model — produces spectacular validation results and a worthless production model. If your AUC is above 0.95 on a hard problem, look for leakage before celebrating.
Split by time, and by entity

Random splits leak the future. They also leak across records: the same customer appearing in train and test inflates results. Split by time and group by customer.
ROC AUC is misleading when positives are rare

At 0.1% positive rate, ROC AUC stays high while precision is hopeless, because the enormous true-negative count dominates. Use precision-recall AUC and report precision at the recall you will operate at.
Fair lending testing is not optional

Models used in credit decisions get tested for disparate impact across protected classes. Excluding the protected attribute does not solve it, because proxies remain. Plan for the testing and for having to defend feature choices.
Monitor the inputs, not just the outputs

Performance decay shows up in feature distributions well before it shows in outcomes — especially where outcomes take months to mature. This is what SageMaker Model Monitor's baseline constraints are for.