OpenAI · Released August 2025
gpt-oss-120b
What it takes to run gpt-oss-120b locally: memory by quantization, the GPUs that handle it, and how fast it runs.
Updated
- Parameters
- 117B (5.1B active)
- Architecture
- Mixture of experts
- Context
- 128K tokens
- License
- Apache 2.0 license
- Memory at MXFP4
- ~68GB
What do you need to run gpt-oss-120b?
gpt-oss-120b needs about 68GB of VRAM at MXFP4 with 8K context (65GB for the weights plus KV cache and overhead). The smallest GPU that can hold it is a 72GB card like the RTX PRO 5000 72GB, with little memory to spare. For a good balance of price and speed we recommend the RTX PRO 6000 (~170 tokens/sec). Because it is a mixture-of-experts model with 5.1B active parameters, it generates text much faster than a dense model of the same size, and it tolerates partial CPU offload better.
What to buy
GPUs for gpt-oss-120b
Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.
Memory
gpt-oss-120b VRAM requirements by quantization
gpt-oss-120b ships natively in MXFP4, so there is no need to pick a quantization. Totals include the KV cache for the context length shown plus runtime overhead.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| MXFP4 | NativeDefault | 65 GB | 68 GB | 69 GB |
KV cache note: 36 layers alternate between a 128-token sliding window and full attention, so only the 18 full-attention layers grow with context: 2 x 18 x 8 KV heads x 64 dims x 2 bytes = 36,864 bytes (36 KiB) per token, about 4.8GB at the full 128K. The sliding layers add a negligible fixed buffer.
Performance
How fast gpt-oss-120b runs
Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.
RTX PRO 6000
MXFP4 · 96GB
~170tok/s
RTX PRO 5000 72GB
MXFP4 · 72GB
~145tok/s
M5 Ultra 512GB
MXFP4 · 512GB
~93tok/s
M5 Ultra 256GB
MXFP4 · 256GB
~93tok/s
M3 Ultra 512GB
MXFP4 · 512GB
~70tok/s
M5 Max 128GB
MXFP4 · 128GB
~64tok/s
DGX Spark
MXFP4 · 128GB
59tok/s
M4 Max 128GB
MXFP4 · 128GB
~58tok/s
Ryzen AI Max+ 395
MXFP4 · 128GB
47tok/s
Ryzen AI Max+ PRO 495
MXFP4 · 192GB
~37tok/s
Measured
gpt-oss-120b benchmarks
- DGX SparkMXFP4 · llama.cpp59 tok/s llama.cpp repository: benches/dgx-spark/dgx-spark.md
- Ryzen AI Max+ 395MXFP4 · llama.cpp47 tok/s llama.cpp discussion #16578: Performance of llama.cpp on NVIDIA DGX Spark
Overview
About gpt-oss-120b
gpt-oss-120b is OpenAI's larger open-weight reasoning model: 117B total parameters and 5.1B active, with expert weights shipped natively in MXFP4. It became the default model for 96GB to 128GB unified-memory machines because it is both strong and unusually fast for its size, and OpenAI has not replaced it. The native weights take about 65GB, so it runs fully in memory on an 80GB or 96GB GPU, a Mac with 96GB or more, a Strix Halo 128GB system, or a DGX Spark; a 24GB GPU plus 64GB of system RAM also works with the experts offloaded. Newer models such as Qwen3.5 122B-A10B edge it on benchmarks, but few match its speed.
Strengths
- +Excellent reasoning, math, and tool calling with only 5.1B active parameters
- +Native MXFP4 weights: no quantization decisions and no quality loss
- +Small KV cache (36 KiB per token) leaves room for long contexts
- +Apache 2.0 license
Weaknesses
- −Text only
- −Weaker world knowledge and creative writing than its benchmark scores suggest
- −Reasoning is always on, adding latency to simple requests
- −Needs about 68GB with context, beyond any single consumer GPU without offload
Setup
How to run gpt-oss-120b
$ ollama run gpt-oss:120b$ llama-server -hf ggml-org/gpt-oss-120b-GGUF -c 8192 -ngl 99Weights: openai/gpt-oss-120b · GGUF: ggml-org/gpt-oss-120b-GGUF
Alternatives
Models like gpt-oss-120b
Alibaba Qwen
Qwen3.5 122B-A10B
122B (10B active) · 256K ctx
Z.ai
GLM-4.5-Air
106B (12B active) · 128K ctx
Meta
Llama 4 Scout
109B (17B active) · 10.5M ctx
OpenAI
gpt-oss-20b
20.9B (3.6B active) · 128K ctx
Frequently asked questions
How much VRAM do you need to run gpt-oss-120b?
About 68GB at MXFP4 with 8K tokens of context: 65.4GB of weights, 0.3GB of KV cache, and 2.6GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.
What is the cheapest GPU that can run gpt-oss-120b?
The smallest card in our database that can hold it is the RTX PRO 5000 72GB (72GB), at MXFP4 with little memory to spare, generating ~145 tokens/sec.
Is gpt-oss-120b free for commercial use?
gpt-oss-120b is released under the Apache 2.0 license. That is a permissive license, so commercial use is allowed.
How long a context does gpt-oss-120b support?
Up to 131,072 tokens (128K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.
Sources: gpt-oss-120b model card (Hugging Face), OpenAI: Introducing gpt-oss. Capability score basis: Llama Builds scale anchor (80 to 85 band), consistent with its Artificial Analysis Intelligence Index v3 standing at high reasoning effort (October 2025).