MODELCATALOG

Weights you download and serve yourself, sized against the hardware they need.

Open-weight models

Every variant below is placed on the same hardware tiers as the model catalog. The tier comes from total parameter count: at 16-bit precision each billion parameters needs roughly 2GB of GPU memory before any working space, so size decides the instance.

Checked as of 2026-10-23

Hardware tiers

TierParametersWhat it means SageMaker instance$ / hour
CPU 0–150M Runs on a laptop. No GPU needed.
ml.m5.large CPU, 2 vCPU $0.12
CPU or small GPU 150M–1B Fine on CPU for single requests. GPU if you need throughput.
ml.m5.xlarge CPU, 4 vCPU $0.23
ml.g4dn.xlarge 1x T4, 16GB $0.74
16GB GPU 1B–7B Needs a GPU. Comfortable on a T4 or L4 in half precision.
ml.g4dn.xlarge 1x T4, 16GB $0.74
ml.g5.xlarge 1x A10G, 24GB $1.41
40GB GPU 7B–25B Single large GPU. Quantise to fit smaller cards.
ml.g5.12xlarge 4x A10G, 96GB $7.09
ml.p4d.24xlarge 8x A100, 320GB $25.25
80GB GPU 25B–80B One H100 or A100 80GB, at reduced precision.
ml.p4de.24xlarge 8x A100, 640GB $31.56
ml.p5.48xlarge 8x H100, 640GB $63.30
Multi-GPU 80B–+ Will not fit on one card. Needs sharding across several.
ml.p5.48xlarge 8x H100, 640GB $63.30+

Approximate on-demand SageMaker real-time inference rates, us-east-1, from the AWS price list, checked 2026-10-23. Quantising to 8 or 4 bits roughly halves or quarters memory and can drop a model one tier. The multi-GPU rate is a floor: the largest models need more than one 8-GPU node at full precision.

Llama & Muse Meta Open, restricted checked 2026-10-23

Meta's open models. Llama 4 is the previous generation; the new line is called Muse and lives in a different HF org.

ModelParamsContext Hardware tierCheapest instanceGatedStatus
Muse Spark Open meta-models/Muse-Spark-Open-816B-A42B Released 2026-10-15. Muse Model License. Context is the card's '1M at inference'. 816B42B active 1M Multi-GPU ml.p5.48xlarge $63.30/hr+ Yes current
Muse Glimmer 30B meta-models/Muse-Glimmer-30B Dense, Apache-2.0. Parameter count includes the vision encoder. 29.6B 131.072K 80GB GPU ml.p4de.24xlarge $31.56/hr No current
Llama 4 Maverick meta-llama/Llama-4-Maverick-17B-128E-Instruct April 2025. Llama 4 Community License. On Bedrock and JumpStart. 400B17B active 1M Multi-GPU ml.p5.48xlarge $63.30/hr+ Yes legacy
Llama 4 Scout meta-llama/Llama-4-Scout-17B-16E-Instruct April 2025. Llama 4 Community License. On Bedrock and JumpStart. 109B17B active 10M Multi-GPU ml.p5.48xlarge $63.30/hr+ Yes legacy

Mistral Mistral AI Open source checked 2026-10-23

A full size ladder from 3B to 675B, nearly all Apache-2.0 and none of it gated.

ModelParamsContext Hardware tierCheapest instanceGatedStatus
Mistral Medium 3.5 mistralai/Mistral-Medium-3.5-128B Dense. Modified MIT licence with a US$20M monthly revenue cap. 128B 262.144K Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
Mistral Large 3 mistralai/Mistral-Large-3-675B-Instruct-2512 Apache-2.0. On Bedrock. The card also says 39B active. 675B41B active 262.144K Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
Mistral Small 4 mistralai/Mistral-Small-4-119B-2603 Apache-2.0. Context per the card; config.json says 1,048,576. 119B6.5B active 262.144K Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
Ministral 3 14B mistralai/Ministral-3-14B-Instruct-2512 Dense, Apache-2.0. On Bedrock. 13.9B 262.144K 40GB GPU ml.g5.12xlarge $7.09/hr No current
Ministral 3 3B mistralai/Ministral-3-3B-Instruct-2512 Dense, Apache-2.0. On Bedrock and JumpStart. Size from HF safetensors metadata. 3.9B 262.144K 16GB GPU ml.g4dn.xlarge $0.74/hr No current

Qwen Alibaba Open, restricted checked 2026-10-23

The widest size range of any open family, from 27B up to a 2.4-trillion-parameter model.

ModelParamsContext Hardware tierCheapest instanceGatedStatus
Qwen3.8 2.4T-A95B Qwen/Qwen3.8-2.4T-A95B Qwen3.8-Max License. Extensible to 1,010,000 tokens. 2400B95B active 262.144K Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
Qwen3.8 Flash-Next Qwen/Qwen3.8-Flash-Next Qwen Community License 1.0. 180B includes 51B of n-gram embeddings. 180B6B active 262.144K Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
Qwen3.8 27B Qwen/Qwen3.8-27B Dense, Apache-2.0. Safetensors total is 27.8B. 27B 262.144K 80GB GPU ml.p4de.24xlarge $31.56/hr No current
Qwen3.6 35B-A3B Qwen/Qwen3.6-35B-A3B April 2026, Apache-2.0. Still heavily downloaded. 35B3B active 262.144K 80GB GPU ml.p4de.24xlarge $31.56/hr No legacy

Gemma Google Open source checked 2026-10-23

Google's open models: small, efficient, and Apache-2.0 from Gemma 4 on.

ModelParamsContext Hardware tierCheapest instanceGatedStatus
Gemma 4 31B google/gemma-4-31B-it Dense. On Bedrock and JumpStart. Size from HF safetensors metadata. 31.3B 262.144K 80GB GPU ml.p4de.24xlarge $31.56/hr No current
Gemma 4 26B-A4B google/gemma-4-26B-A4B-it Mixture of experts. On Bedrock and JumpStart. 25.8B3.8B active 262.144K 80GB GPU ml.p4de.24xlarge $31.56/hr No current
Gemma 4 12B (Unified) google/gemma-4-12B-it Added 2026-05-23. Video is read as frames. 11.9B 262.144K 40GB GPU ml.g5.12xlarge $7.09/hr No current
Gemma 4 E4B google/gemma-4-E4B-it 4.5B 'effective' parameters, built for on-device. On JumpStart. 8B 131.072K 40GB GPU ml.g5.12xlarge $7.09/hr No current
Gemma 4 E2B google/gemma-4-E2B-it 2.3B 'effective' parameters, built for on-device. On Bedrock. 5.1B 131.072K 16GB GPU ml.g4dn.xlarge $0.74/hr No current
Gemma 3 27B google/gemma-3-27b-it Gemma Terms of Use. 27.4B 131.072K 80GB GPU ml.p4de.24xlarge $31.56/hr Yes legacy

DeepSeek DeepSeek Open source checked 2026-10-23

Very large, very sparse mixture-of-experts models under the MIT licence.

ModelParamsContext Hardware tierCheapest instanceGatedStatus
DeepSeek-V4.1-Flash deepseek-ai/DeepSeek-V4.1-Flash Card: 552B backbone plus 196B 'Engram' memory; total here is the safetensors count. 8B active in prefill, 16B in decode. 763B16B active 1.04858M Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
DeepSeek-V4-Pro deepseek-ai/DeepSeek-V4-Pro-0813 Official 0813 release. Safetensors total is 1.65T. 1600B49B active 1.04858M Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
DeepSeek-V4-Flash deepseek-ai/DeepSeek-V4-Flash-0731 Official 0731 release. Safetensors total is 304B. 284B13B active 1.04858M Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
DeepSeek-V4-Flash-Vision-Exp deepseek-ai/DeepSeek-V4-Flash-Vision-Exp Experimental. Active parameters not stated; context from config.json. 305B 1.04858M Multi-GPU ml.p5.48xlarge $63.30/hr+ No current
DeepSeek-V3.2 deepseek-ai/DeepSeek-V3.2 December 2025. On Bedrock. Size from safetensors, context from config.json. 685B 163.84K Multi-GPU ml.p5.48xlarge $63.30/hr+ No legacy