Recommendations · October 2026
The best local LLMs for the RTX 5080
Every model worth running on 16GB of GDDR7, which quantization to pick, and how fast each one generates text.
Our picks at a glance
- Best overall (also coding, reasoning, vision, largest that fits): Qwen3.8 27B (Q3_K_M, ~55 tokens/sec)
- Runner-up: Qwen3.5 9B (Q8_0, ~66 tokens/sec)
- Fastest good model: Gemma 4 26B A4B (Q3_K_M, ~155 tokens/sec)
We may earn a commission when you buy through links on this page. It never changes what we recommend. How we make money
01 · Best overall · also coding, reasoning, vision, largest that fits
Qwen3.8 27B
Qwen3.8 27B (27B) fits, with little room to spare in the RTX 5080's memory at Q3_K_M, using about 15GB with 8K context and generating ~55 tokens/sec. It is the most capable model in our catalog that runs well on this card. It is also our top pick for coding, reasoning and working with images. It is also the largest model that runs fully on this GPU.
Qwen3.8 27B, released in August 2026, is one of the strongest open models that fits a single 24GB GPU, with image input, a 262K native context, and switchable thinking. It uses the same hybrid layout as the Qwen3.5 and 3.6 27B models, where three of every four layers use linear attention, so long contexts cost far less memory than on older dense models. At Q4_K_M it needs about 18GB with 8K context, leaving room for 64K or more of context on a 24GB card, and 32GB cards can run Q6_K. It supersedes Qwen3.6 27B and Qwen3 32B, but its long reasoning traces make speed on slower hardware a real consideration.
- Quantization
- Q3_K_M
- Memory used
- 14.8 GB
- Speed (est.)
- ~55 tok/s
- Max context
- ~14K
$ ollama run qwen3.8:27bModel details →02 · Runner-up
Qwen3.5 9B
Qwen3.5 9B (9B) fits comfortably in the RTX 5080's memory at Q8_0, using about 12GB with 8K context and generating ~66 tokens/sec. It is the next-best model that runs well here, and the one to try if the top pick doesn't suit you.
Qwen3.5 9B is the largest of the small Qwen3.5 models released in March 2026, with built-in image understanding, a 262K context, and switchable thinking. It uses Qwen's hybrid attention, where most layers are linear-attention layers with a fixed-size state, so long contexts cost far less memory than on older 8B models. It is one of the strongest models that fits an 8GB GPU: at Q4_K_M it needs about 7GB with 8K context, and 12GB cards can run Q8_0. It replaces Qwen3 8B and, for most uses, Qwen3 14B.
- Quantization
- Q8_0
- Memory used
- 12.0 GB
- Speed (est.)
- ~66 tok/s
- Max context
- ~103K
$ ollama run qwen3.5:9bModel details →03 · Fastest good model
Gemma 4 26B A4B
Gemma 4 26B A4B (26B (3.8B active)) fits, with little room to spare in the RTX 5080's memory at Q3_K_M, using about 14GB with 8K context and generating ~155 tokens/sec. Pick it when responsiveness matters more than squeezing out maximum quality.
Gemma 4 26B A4B is Google's mixture-of-experts Gemma, with 26B total parameters and 3.8B active (128 experts, 8 used per token), image input, a 256K context, and a thinking mode. Because so few parameters are active, it generates text at small-model speed while matching much larger dense models on reasoning benchmarks. At Q4_K_M it needs about 17GB with 8K context, so it fits a 24GB GPU with plenty of room for context; on 16GB cards it runs at Q3_K_M or with a few experts in system RAM. It replaces Gemma 3 27B for most local uses.
- Quantization
- Q3_K_M
- Memory used
- 14.1 GB
- Speed (est.)
- ~155 tok/s
- Max context
- ~59K
$ ollama run gemma4:26bModel details →The full list
Every model that runs on the RTX 5080
| # | Model | Best quant | Memory | Speed | Fit |
|---|---|---|---|---|---|
| 1 | Qwen3.8 27B27B | Q3_K_M | 14.8 GB | ~55 tok/s | Fits, tight |
| 2 | Qwen3.5 9B9B | Q8_0 | 12.0 GB | ~66 tok/s | Runs well |
| 3 | gpt-oss-20b20.9B (3.6B active) | MXFP4 | 15.0 GB | ~130 tok/s | Fits, tight |
| 4 | Gemma 4 26B A4B26B (3.8B active) | Q3_K_M | 14.1 GB | ~155 tok/s | Fits, tight |
| 5 | Gemma 4 E4B8B | Q8_0 | 10.5 GB | ~75 tok/s | Runs well |
| 6 | Phi-414.7B | Q5_K_M | 13.2 GB | ~67 tok/s | Runs well |
Ranked by our capability score, adjusted for quantization and speed. Memory includes 8K context. Speeds are estimates for a single user; seemethodology.
Limits
Popular models that don't run well on 16GB
- Qwen3.6 35B-A3BPartial offload
Needs about 24GB at Q4_K_M with 8K context. It will run with 37% of the weights in system RAM at about 48 tokens/sec with CPU offload, which is usable because only a fraction of a mixture-of-experts model is active per token. See what hardware it needs.
- Gemma 4 31BPartial offload
Needs about 22GB at Q4_K_M with 8K context. It will run with 36% of the weights in system RAM at about 5.8 tokens/sec with CPU offload, which is too slow for comfortable chat. See what hardware it needs.
- gpt-oss-120bWon't fit
Needs about 68GB at MXFP4 with 8K context. It does not fit, even with CPU offload. See what hardware it needs.
- DeepSeek V4 FlashWon't fit
Needs about 168GB at FP4 with 8K context. It does not fit, even with CPU offload. See what hardware it needs.
Setup
Best settings and software for the RTX 5080
NVIDIA cards have the broadest software support: everything runs on CUDA, usually on day one. With Qwen3.8 27B at Q3_K_M, you can fit roughly 14K tokens of context with a 16-bit KV cache; an 8-bit KV cache about doubles that.
- Ollama or LM Studio: the easiest way to download and chat with models.
- llama.cpp: full control over quantization, context, and offloading.
- vLLM or SGLang: best throughput when serving several users or agents at once.
- ExLlamaV3 / TabbyAPI: fast single-user inference with EXL3 quants that fill VRAM precisely.
New to quantization? Q4_K_M is the safe default; read our quantization explainer to see when Q5, Q6, or Q8 is worth the extra memory.
Upgrade path
Need more memory?
Memory capacity decides which models you can run. These are the most popular options with more memory than the RTX 5080.
Frequently asked questions
What is the best LLM for the RTX 5080?
Qwen3.8 27B is the best all-round model for the RTX 5080. At Q3_K_M it uses about 15GB with 8K context and generates ~55 tokens/sec.
What quantization should I use on the RTX 5080?
Use the highest-precision quant that leaves room for your context. Q4_K_M is the best default: it is about a quarter the size of the original 16-bit weights with only a small quality loss. If a model fits at Q5_K_M, Q6_K, or Q8_0 with room to spare, use that instead. Avoid going below Q3 except for very large models.
How much context can I use with Qwen3.8 27B on the RTX 5080?
At Q3_K_M with a 16-bit KV cache, roughly 14K tokens fit in 16GB. Quantizing the KV cache to 8-bit (for example with llama.cpp's --cache-type-k q8_0 --cache-type-v q8_0) roughly doubles that.
Can the RTX 5080 run Qwen3.6 35B-A3B?
Only with partial CPU offload. At Q4_K_M it needs about 24GB, so 37% of the weights would sit in system RAM and speed drops to about 48 tokens/sec with CPU offload.
Can the RTX 5080 run Gemma 4 31B?
Only with partial CPU offload. At Q4_K_M it needs about 22GB, so 36% of the weights would sit in system RAM and speed drops to about 5.8 tokens/sec with CPU offload.