# NVIDIA GeForce RTX 3090 for local LLMs

> The default value pick for 24GB of CUDA memory and the backbone of most multi-GPU builds.

The RTX 3090 has 24GB of GDDR6X and 936 GB/s of memory bandwidth, which decides how big a model fits and how fast it generates text. It comfortably runs models up to about 35B parameters: the largest in our catalog is Qwen3.6 35B-A3B at Q3_K_M (19GB with 8K context, ~160 tokens/sec). Our top all-round pick for it is Qwen3.8 27B (Q4_K_M, ~39 tokens/sec).

## Key specs

- Memory: 24GB GDDR6X
- Memory bandwidth: 936 GB/s
- Architecture: Ampere (2020)
- Power: 350W
- Typical price: ~$1,050 used (checked 2026-10)
- Software: CUDA, Vulkan

## Best models for this GPU

- **Best overall (also coding, reasoning, vision):** Qwen3.8 27B at Q4_K_M, ~18.1GB with 8K context, ~39 tokens/sec
- **Runner-up:** Gemma 4 31B at Q4_K_M, ~22.5GB with 8K context, ~33 tokens/sec
- **Fastest good model (also largest that fits):** Qwen3.6 35B-A3B at Q3_K_M, ~19.2GB with 8K context, ~160 tokens/sec

## Compatibility (best quantization that fits, 8K context)

| Model | Params | Quant | Memory | Speed | Verdict |
| --- | --- | --- | --- | --- | --- |
| Gemma 4 E4B | 8B | Q8_0 | 10.5GB | ~66 tok/s | Runs well |
| Qwen3.5 9B | 9B | Q8_0 | 12.0GB | ~59 tok/s | Runs well |
| Phi-4 | 14.7B | Q8_0 | 18.3GB | ~41 tok/s | Runs well |
| gpt-oss-20b | 20.9B (3.6B active) | MXFP4 | 15.0GB | ~120 tok/s | Runs well |
| Gemma 4 26B A4B | 26B (3.8B active) | Q5_K_M | 20.1GB | ~110 tok/s | Runs well |
| Qwen3.8 27B | 27B | Q4_K_M | 18.1GB | ~39 tok/s | Runs well |
| Gemma 4 31B | 31B | Q4_K_M | 22.5GB | ~33 tok/s | Fits, tight |
| Qwen3.6 35B-A3B | 35B (3B active) | Q3_K_M | 19.2GB | ~160 tok/s | Runs well |
| GLM-4.5-Air | 106B (12B active) | Q4_K_M | 68.3GB | ~8.3 tok/s | Partial offload |
| Llama 4 Scout | 109B (17B active) | Q4_K_M | 69.5GB | ~5.9 tok/s | Partial offload |
| gpt-oss-120b | 117B (5.1B active) | MXFP4 | 68.3GB | ~21 tok/s | Partial offload |
| Qwen3.5 122B-A10B | 122B (10B active) | - | - | - | Won't fit |
| DeepSeek V4 Flash | 284B (13B active) | - | - | - | Won't fit |
| GLM-5.3-Flash | 320B (18B active) | - | - | - | Won't fit |
| GLM-5.3 | 744B (40B active) | - | - | - | Won't fit |

## Pros
- 24GB at 936 GB/s for less than half the price of a used RTX 4090
- NVLink support for dual-card setups
- Huge community knowledge base for multi-GPU builds
- Full CUDA support, including FlashAttention 2

## Cons
- Hot rear-mounted memory on many cards
- 350W with sharp transient power spikes
- Used cards may have mining or heavy-use history

Full page: https://llamabuilds.ai/gpus/rtx-3090 · Best models: https://llamabuilds.ai/gpus/rtx-3090/best-models

Sources: [NVIDIA GeForce RTX 3090 and 3090 Ti product page](https://www.nvidia.com/en-us/geforce/graphics-cards/30-series/rtx-3090-3090ti/), [TechPowerUp GPU Database, GeForce RTX 3090](https://www.techpowerup.com/gpu-specs/geforce-rtx-3090.c3622), [BestValueGPU RTX 3090 price history](https://bestvaluegpu.com/history/new-and-used-rtx-3090-price-history-and-specs/)

---
Source: https://llamabuilds.ai/gpus/rtx-3090 · Updated October 14, 2026 · Llama Builds. Speeds marked ~ are estimates from our bandwidth model (https://llamabuilds.ai/methodology); measured results are labeled as such.