NVIDIA · Blackwell · 2025
NVIDIA GeForce RTX 5090 for local AI
The fastest consumer card for local LLMs, but shortage pricing has more than doubled its cost.
Updated · Speeds marked ~ are estimates; seemethodology
- VRAM
- 32GB
- Bandwidth
- 1,792 GB/s
- Power
- 575W
- Runs well
- 8 models
RTX 5090
Typically about $4,300
Can the RTX 5090 run LLMs?
The RTX 5090 has 32GB of GDDR7 and 1,792 GB/s of memory bandwidth, which decides how big a model fits and how fast it generates text. It comfortably runs models up to about 35B parameters: the largest in our catalog is Qwen3.6 35B-A3B at Q5_K_M (28GB with 8K context, ~195 tokens/sec). Our top all-round pick for it is Qwen3.8 27B (Q6_K, ~59 tokens/sec). Llama 3.3 70B Instruct only runs with partial CPU offload, at about 2.7 tokens/sec with CPU offload.
Recommendations
Best models for the RTX 5090
Best overall · also coding, reasoning, vision
Qwen3.8 27BQ6_K · 25GB · ~59 tokens/sec
Runs wellRunner-up · also fastest, largest that fits
Qwen3.6 35B-A3BQ5_K_M · 28GB · ~195 tokens/sec
Runs well
Compatibility
What runs on the RTX 5090
Every current model in our catalog, at the best quantization that fits in 32GB with 8K tokens of context. "Partial offload" means part of the model runs from system RAM (we assume 64GB), which is much slower.
| Model | Parameters | Best quant | Memory | Speed | Verdict |
|---|---|---|---|---|---|
| Gemma 4 E4B | 8B | Q8_0 | 10 GB | ~130 tok/s | Runs well |
| Qwen3.5 9B | 9B | Q8_0 | 12 GB | ~115 tok/s | Runs well |
| Phi-4 | 14.7B | Q8_0 | 18 GB | ~84 tok/s | Runs well |
| gpt-oss-20b | 20.9B (3.6B active) | MXFP4 | 15 GB | ~190 tok/s | Runs well |
| Gemma 4 26B A4B | 26B (3.8B active) | Q6_K | 23 GB | ~165 tok/s | Runs well |
| Qwen3.8 27B | 27B | Q6_K | 25 GB | ~59 tok/s | Runs well |
| Gemma 4 31B | 31B | Q5_K_M | 26 GB | ~59 tok/s | Runs well |
| Qwen3.6 35B-A3B | 35B (3B active) | Q5_K_M | 28 GB | ~195 tok/s | Runs well |
| GLM-4.5-Air | 106B (12B active) | Q4_K_M | 68 GB | ~10 tok/s | Partial offload |
| Llama 4 Scout | 109B (17B active) | Q4_K_M | 70 GB | ~7.2 tok/s | Partial offload |
| gpt-oss-120b | 117B (5.1B active) | MXFP4 | 68 GB | ~25 tok/s | Partial offload |
| Qwen3.5 122B-A10B | 122B (10B active) | Q4_K_M | 77 GB | ~11 tok/s | Partial offload |
| DeepSeek V4 Flash | 284B (13B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3-Flash | 320B (18B active) | — | — | — | Won't fitWhat it needs |
| GLM-5.3 | 744B (40B active) | — | — | — | Won't fitWhat it needs |
Performance
RTX 5090 vs. alternatives on Qwen3.8 27B
Generation speed in tokens per second for a single user. Token generation is limited mostly by memory bandwidth, which is why bandwidth predicts these results so well.
RTX 5090
Q6_K · 32GB
~59tok/s
RTX 5080
Q3_K_M · 16GB
~55tok/s
RTX 4090
Q4_K_M · 24GB
~47tok/s
RTX PRO 6000
Q8_0 · 96GB
~46tok/s
RTX 3090
Q4_K_M · 24GB
~39tok/s
Measured
RTX 5090 benchmark results
| Model | Quant | Runtime | Generation | Prompt | Source |
|---|---|---|---|---|---|
| Llama 2 7B Chat | Q4_0 | llama.cpp (CUDA) | 290 tok/s | 14075 tok/s | llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026 |
| Llama 2 7B Chat | Q4_0 | llama.cpp (Vulkan) | 265 tok/s | 10380 tok/s | llama.cpp discussion #10879: Performance of llama.cpp with Vulkan · October 2026 |
Our take
Should you buy the RTX 5090 for AI?
The RTX 5090 pairs 32GB of GDDR7 with 1,792 GB/s of bandwidth, about 78 percent more than the RTX 4090, so it generates tokens faster than any other consumer card. The extra 8GB over 24GB cards lets 32B-class dense models run at 5 to 6 bit with room for long context, and large MoE models run well with partial CPU offload. It draws up to 575W through a single 12V-2x6 connector, so plan for a quality 1,000W or larger power supply and good case airflow. In October 2026 the memory shortage pushed new cards to $4,300 or more, more than twice the $1,999 launch MSRP.
What we like
- Highest memory bandwidth of any consumer GPU at 1,792 GB/s
- 32GB fits 32B-class models at high quality with long context
- Full CUDA support across llama.cpp, vLLM, ExLlamaV2 and SGLang
- FP4 and FP8 tensor support for newer quantization formats
Watch out for
- Street prices above $4,300 new in October 2026
- 575W board power needs a large PSU and strong cooling
- Still too small for 70B dense models without offload
Best for: Fastest single-GPU inference for models up to about 32B · Developers who need top CUDA speed in a consumer card · Multi-GPU rigs where per-card speed matters
Specifications
RTX 5090 specs that matter for AI
- Memory
- 32GB GDDR7
- Memory bandwidth
- 1,792 GB/s
- Memory bus
- 512-bit
- Architecture
- Blackwell
- Power
- 575W
- Interface
- PCIe 5.0 x16
- Released
- 2025
- Launch price
- $1,999
- Street price (new)
- ~$4,300 (2026-10)
- Street price (used)
- ~$3,800 (2026-10)
- Software
- CUDA, Vulkan
Sources: NVIDIA GeForce RTX 5090 product page
Cross-shop
Alternatives to the RTX 5090
Frequently asked questions
How much VRAM does the RTX 5090 have?
The NVIDIA GeForce RTX 5090 has 32GB of GDDR7 with 1,792 GB/s of memory bandwidth on a 512-bit bus.
What is the largest LLM the RTX 5090 can run?
Fully in GPU memory, the largest model in our catalog that runs comfortably is Qwen3.6 35B-A3B (35B (3B active)) at Q5_K_M, using about 28GB with 8K context at ~195 tokens/sec.
Can the RTX 5090 run Llama 3.3 70B?
Only with partial CPU offload. At Q4_K_M the model needs about 47GB with 8K context, so part of it spills into system RAM and speed drops to about 2.7 tokens/sec with CPU offload. For usable 70B speeds you want 48GB of VRAM or more (for example two 24GB cards).
What is the best LLM to run on the RTX 5090?
Our top pick is Qwen3.8 27B at Q6_K, which fits in 32GB with room for 8K of context and runs at ~59 tokens/sec. See our full list of the best models for the RTX 5090 for coding, reasoning, and vision picks.
Is the RTX 5090 good for local AI in 2026?
The fastest consumer card for local LLMs, but shortage pricing has more than doubled its cost.