Tool
Can your GPU run it?
Pick a model and a GPU. We'll show how much memory it needs, whether it fits, and roughly how fast it will run.
Qwen3.8 27B · Q4_K_M · 8K context on RTX 3090
Fits with room to spare. Fast. Faster than you can read.
- Weights
- 16.5 GB
- KV cache
- 0.5 GB
- Overhead
- 1.1 GB
Estimates for llama.cpp-style runtimes and a single user. Real numbers vary with runtime, drivers, and settings. How we estimate.
The math
How VRAM use is calculated
A running model needs memory for three things:
- Weights. Parameters × bits per weight ÷ 8. An 8B model at Q4_K_M (about 4.85 bits) is roughly 4.9GB.
- KV cache. Every token in the context stores keys and values for every layer: 2 × layers × KV heads × head size × 2 bytes. Long contexts can need more memory than the weights.
- Overhead. Compute buffers and the CUDA, ROCm, or Metal context, usually 0.5–2GB.
Speed comes from memory bandwidth. To generate each token the GPU reads every active weight once, so tokens per second is roughly bandwidth divided by model size, minus some efficiency loss. That is why a 1,792 GB/s RTX 5090 is so much faster than a 448 GB/s RTX 5060 Ti on the same model.
Quick reference
VRAM needed by popular models
| Model | Size | Q4_K_M | Q8_0 |
|---|---|---|---|
| Llama 3.1 8B Instruct | 8B | 6.7 GB | 10 GB |
| Qwen3 14B | 14.8B | 11 GB | 18 GB |
| gpt-oss-20b | 20.9B (3.6B active) | 15 GBMXFP4 | — |
| Qwen3 30B-A3B (2507) | 30.5B (3.3B active) | 21 GB | 35 GB |
| Gemma 3 27B | 27.4B | 19 GB | 31 GB |
| Qwen3 32B | 32.8B | 23 GB | 39 GB |
| Llama 3.3 70B Instruct | 70.6B | 47 GB | 81 GB |
| gpt-oss-120b | 117B (5.1B active) | 68 GBMXFP4 | — |
Frequently asked questions
How much VRAM do I need for Llama 3.1 8B Instruct?
About 6.7GB at Q4_K_M with 8K tokens of context, or 10GB at Q8_0. See which GPUs run it.
How much VRAM do I need for Qwen3 14B?
About 11GB at Q4_K_M with 8K tokens of context, or 18GB at Q8_0. See which GPUs run it.
How much VRAM do I need for gpt-oss-20b?
About 15GB at MXFP4 with 8K tokens of context. See which GPUs run it.
How much VRAM do I need for Qwen3 30B-A3B (2507)?
About 21GB at Q4_K_M with 8K tokens of context, or 35GB at Q8_0. See which GPUs run it.
How much VRAM do I need for Gemma 3 27B?
About 19GB at Q4_K_M with 8K tokens of context, or 31GB at Q8_0. See which GPUs run it.
How do you calculate VRAM for an LLM?
Add three things: the model weights (parameters × bits per weight ÷ 8), the KV cache (which grows with context length), and 0.5–2GB of runtime overhead. A 32B model at 4.85 bits per weight (Q4_K_M) is about 19.4GB of weights before context.
What happens if a model does not fit in VRAM?
Runtimes like llama.cpp, Ollama, and LM Studio can keep part of the model in system RAM. It still runs, but every token has to read those weights over the much slower system memory bus, so speed often drops by 5–10× for dense models. Mixture-of-experts models handle offloading far better.