OpenAI · Released August 2025
gpt-oss-20b
What it takes to run gpt-oss-20b locally: memory by quantization, the GPUs that handle it, and how fast it runs.
Updated
- Parameters
- 20.9B (3.6B active)
- Architecture
- Mixture of experts
- Context
- 128K tokens
- License
- Apache 2.0 license
- Memory at MXFP4
- ~15GB
What do you need to run gpt-oss-20b?
gpt-oss-20b needs about 15GB of VRAM at MXFP4 with 8K context (14GB for the weights plus KV cache and overhead). The smallest GPU that can hold it is a 16GB card like the RTX 4060 Ti 16GB, with little memory to spare. For a good balance of price and speed we recommend the RTX 3090 (~120 tokens/sec). Because it is a mixture-of-experts model with 3.6B active parameters, it generates text much faster than a dense model of the same size, and it tolerates partial CPU offload better.
What to buy
GPUs for gpt-oss-20b
Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.
Memory
gpt-oss-20b VRAM requirements by quantization
gpt-oss-20b ships natively in MXFP4, so there is no need to pick a quantization. Totals include the KV cache for the context length shown plus runtime overhead.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| MXFP4 | NativeDefault | 14 GB | 15 GB | 16 GB |
KV cache note: 24 layers alternate between a 128-token sliding window and full attention, so only the 12 full-attention layers grow with context: 2 x 12 x 8 KV heads x 64 dims x 2 bytes = 24,576 bytes (24 KiB) per token, about 3.2GB at the full 128K. The sliding layers add a negligible fixed buffer.
Performance
How fast gpt-oss-20b runs
Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.
RTX PRO 5000 72GB
MXFP4 · 72GB
215tok/s
RTX 5090
MXFP4 · 32GB
~190tok/s
RTX PRO 6000
MXFP4 · 96GB
~190tok/s
RTX 4090
MXFP4 · 24GB
~135tok/s
RTX 5080
MXFP4 · 16GB
~130tok/s
RTX 6000 Ada
MXFP4 · 48GB
~130tok/s
RTX 5070 Ti
MXFP4 · 16GB
~125tok/s
RTX 3090
MXFP4 · 24GB
~120tok/s
M4 Max 128GB
MXFP4 · 128GB
120tok/s
RTX 4080 Super
MXFP4 · 16GB
~110tok/s
Measured
gpt-oss-20b benchmarks
- RTX PRO 5000 72GBMXFP4 · llama.cpp215 tok/s Hardware Corner: RTX Pro 5000 Blackwell LLM Performance
- M4 Max 128GBMXFP4 · llama.cpp120 tok/s llama.cpp discussion #16578: Performance of llama.cpp on NVIDIA DGX Spark
- RX 9060 XT 16GBMXFP4 · llama.cpp105 tok/s llama.cpp discussion #10879: Performance of llama.cpp with Vulkan
- RX 9060 XT 16GBMXFP4 · llama.cpp93 tok/s llama.cpp discussion #15021: Performance of llama.cpp on AMD ROCm (HIP)
- DGX SparkMXFP4 · llama.cpp86 tok/s llama.cpp discussion #16578: Performance of llama.cpp on NVIDIA DGX Spark
- DGX SparkMXFP4 · llama.cpp83 tok/s llama.cpp repository: benches/dgx-spark/dgx-spark.md
- RTX 4060 Ti 16GBMXFP4 · llama.cpp63 tok/s Hardware Corner: RTX 4060 Ti 16GB LLM Performance
Overview
About gpt-oss-20b
gpt-oss-20b is OpenAI's small open-weight reasoning model, a 21B mixture-of-experts with 3.6B active parameters whose expert weights ship natively in 4-bit MXFP4. OpenAI has not released a successor, and it remains one of the best reasoning and tool-calling models that fits a 16GB GPU. The native weights take about 14GB, so 16GB cards and Macs with 24GB or more run it fully in memory, and 12GB cards run it with a little CPU offload. Reasoning effort is adjustable (low, medium, high) but cannot be turned off.
Strengths
- +Strong math, reasoning, and tool calling for a model that fits 16GB
- +Ships in native MXFP4, so there is no quantization quality loss to manage
- +Very fast: 3.6B active parameters and a tiny KV cache (24 KiB per token)
- +Apache 2.0 license
Weaknesses
- −Thin world knowledge and more hallucination than dense models of similar size
- −Reasoning cannot be disabled, which adds latency to simple requests
- −Text only, and it relies on the harmony chat format, which some tools mishandle
Setup
How to run gpt-oss-20b
$ ollama run gpt-oss:20b$ llama-server -hf ggml-org/gpt-oss-20b-GGUF -c 8192 -ngl 99$ mlx_lm.chat --model mlx-community/gpt-oss-20b-MXFP4-Q8Weights: openai/gpt-oss-20b · GGUF: ggml-org/gpt-oss-20b-GGUF
Alternatives
Models like gpt-oss-20b
Alibaba Qwen
Qwen3.6 35B-A3B
35B (3B active) · 256K ctx
Gemma 4 26B A4B
26B (3.8B active) · 256K ctx
Alibaba Qwen
Qwen3.5 9B
9B · 256K ctx
OpenAI
gpt-oss-120b
117B (5.1B active) · 128K ctx
Frequently asked questions
How much VRAM do you need to run gpt-oss-20b?
About 15GB at MXFP4 with 8K tokens of context: 13.8GB of weights, 0.2GB of KV cache, and 1.0GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.
What is the cheapest GPU that can run gpt-oss-20b?
The smallest card in our database that can hold it is the RTX 4060 Ti 16GB (16GB), at MXFP4 with little memory to spare, generating ~51 tokens/sec.
Is gpt-oss-20b free for commercial use?
gpt-oss-20b is released under the Apache 2.0 license. That is a permissive license, so commercial use is allowed.
How long a context does gpt-oss-20b support?
Up to 131,072 tokens (128K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.
Sources: gpt-oss-20b model card (Hugging Face), OpenAI: Introducing gpt-oss. Capability score basis: Placed between the Qwen3 32B and gpt-oss-120b anchors from its Artificial Analysis Intelligence Index v3 standing at high reasoning effort (October 2025).