Build & train
Channels & InputData
Keeps logically different training inputs separate and named.
named S3/filesystem/dataset sources
container-visible named input paths
Channels are named inputs to the training container. I might define train and validation channels, and my training script decides how to use each.
Mental model
What this means before worrying about syntax.
A channel is a named data feed to a training job. train, validation and test are conventions, not built-in scikit-learn modules.
- A channel is a named logical input to the training job.
- train, validation and test are conventions, not framework modules.
- Your training code decides how each channel is consumed.
S3 URI -> InputData(channel_name=...) -> training job -> /opt/ml/input/data/<channel>
Implementation anatomy
The sequence you would actually build or configure.
- 1
Create one input per logically distinct dataset.
- 2
Point each to a versioned S3 source.
- 3
Choose delivery/input mode.
- 4
Read the corresponding container location.
- 5
Keep test data outside training when you want an independent evaluation step.
Concrete example
A real model-development or production flow.
train -> s3://bank-ml/complaints/v3/train/
validation -> s3://bank-ml/complaints/v3/validation/
trainer fits on train
trainer reports validation metrics
separate evaluation step uses untouched test set
Implementation lab
Concrete code, data manipulation and configuration for this lifecycle step.
Two named inputs
from sagemaker.train.configs import InputData
inputs = [
InputData(
channel_name="train",
data_source="s3://bank-ml/complaints/v17/train/",
content_type="text/csv",
),
InputData(
channel_name="validation",
data_source="s3://bank-ml/complaints/v17/validation/",
content_type="text/csv",
),
]
trainer.train(input_data_config=inputs)Read channels robustly in train.py
SageMaker exposes channel-specific environment variables such as SM_CHANNEL_TRAIN.
from pathlib import Path
import os
import pandas as pd
train_dir = Path(os.environ["SM_CHANNEL_TRAIN"])
val_dir = Path(os.environ["SM_CHANNEL_VALIDATION"])
train = pd.concat(
[pd.read_csv(p) for p in train_dir.glob("*.csv")],
ignore_index=True,
)
validation = pd.concat(
[pd.read_csv(p) for p in val_dir.glob("*.csv")],
ignore_index=True,
)
print("train:", train.shape)
print("validation:", validation.shape)Keep final test outside the training job
# A useful production pattern:
#
# Training job:
# train channel -> fit model
# validation channel -> tune/report during development
#
# Separate evaluation job:
# test dataset -> final acceptance gate
#
# Why?
# The trainer cannot accidentally keep tuning against the final test set.Design decisions that matter
- Need a test channel or a separate evaluation job?
- Use custom names for weights/reference/labels where clearer.
Common failure modes
- A validation channel does not automatically call model.score().
- Do not fit preprocessing on validation/test channels.
- Channel name and input delivery mode are different concepts.
Production readiness check
Items to close before treating this step as production-ready.