NVIDIA · Ampere · 2020

NVIDIA GeForce RTX 3090 for local AI

The default value pick for 24GB of CUDA memory and the backbone of most multi-GPU builds.

Updated October 14, 2026 · Speeds marked ~ are estimates; seemethodology

VRAM: 24GB
Bandwidth: 936 GB/s
Power: 350W
Runs well: 8 models

RTX 3090

Typically about $1,050 used

Check price at Amazon Find it used on eBay

Can the RTX 3090 run LLMs?

The RTX 3090 has 24GB of GDDR6X and 936 GB/s of memory bandwidth, which decides how big a model fits and how fast it generates text. It comfortably runs models up to about 35B parameters: the largest in our catalog is Qwen3.6 35B-A3B at Q3_K_M (19GB with 8K context, ~160 tokens/sec). Our top all-round pick for it is Qwen3.8 27B (Q4_K_M, ~39 tokens/sec). Llama 3.3 70B Instruct only runs with partial CPU offload, at about 1.8 tokens/sec with CPU offload.

Recommendations

Best models for the RTX 3090

Full list

Best overall · also coding, reasoning, vision
Qwen3.8 27B
Q4_K_M · 18GB · ~39 tokens/sec
Runs well
Runner-up
Gemma 4 31B
Q4_K_M · 22GB · ~33 tokens/sec
Fits, tight
Fastest good model · also largest that fits
Qwen3.6 35B-A3B
Q3_K_M · 19GB · ~160 tokens/sec
Runs well

Read: the best LLMs to run on the RTX 3090

Compatibility

What runs on the RTX 3090

Every current model in our catalog, at the best quantization that fits in 24GB with 8K tokens of context. "Partial offload" means part of the model runs from system RAM (we assume 64GB), which is much slower.

Which open-weight models run on the NVIDIA GeForce RTX 3090, with the best quantization and estimated speed
Model	Parameters	Best quant	Memory	Speed	Verdict
Gemma 4 E4B	8B	Q8_0	10 GB	~66 tok/s	Runs well
Qwen3.5 9B	9B	Q8_0	12 GB	~59 tok/s	Runs well
Phi-4	14.7B	Q8_0	18 GB	~41 tok/s	Runs well
gpt-oss-20b	20.9B (3.6B active)	MXFP4	15 GB	~120 tok/s	Runs well
Gemma 4 26B A4B	26B (3.8B active)	Q5_K_M	20 GB	~110 tok/s	Runs well
Qwen3.8 27B	27B	Q4_K_M	18 GB	~39 tok/s	Runs well
Gemma 4 31B	31B	Q4_K_M	22 GB	~33 tok/s	Fits, tight
Qwen3.6 35B-A3B	35B (3B active)	Q3_K_M	19 GB	~160 tok/s	Runs well
GLM-4.5-Air	106B (12B active)	Q4_K_M	68 GB	~8.3 tok/s	Partial offload
Llama 4 Scout	109B (17B active)	Q4_K_M	70 GB	~5.9 tok/s	Partial offload
gpt-oss-120b	117B (5.1B active)	MXFP4	68 GB	~21 tok/s	Partial offload
Qwen3.5 122B-A10B	122B (10B active)	—	—	—	Won't fitWhat it needs
DeepSeek V4 Flash	284B (13B active)	—	—	—	Won't fitWhat it needs
GLM-5.3-Flash	320B (18B active)	—	—	—	Won't fitWhat it needs
GLM-5.3	744B (40B active)	—	—	—	Won't fitWhat it needs

Performance

RTX 3090 vs. alternatives on Qwen3.8 27B

Generation speed in tokens per second for a single user. Token generation is limited mostly by memory bandwidth, which is why bandwidth predicts these results so well.

Estimated (method)

Measured

RTX 3090 benchmark results

All benchmarks

Model	Quant	Runtime	Generation	Prompt	Source
Llama 2 7B Chat	Q4_0	llama.cpp (CUDA)	170 tok/s	5155 tok/s	llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026
Llama 2 7B Chat	Q4_0	llama.cpp (Vulkan)	165 tok/s	4665 tok/s	llama.cpp discussion #10879: Performance of llama.cpp with Vulkan · October 2026
Llama 2 7B Chat	Q4_0	llama.cpp (CUDA)	160 tok/s	5175 tok/s	llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026
4× · Qwen3.6 35B-A3B	Q5_K_M	llama.cpp (CUDA)	150 tok/s	—	llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026
Llama 3.1 8B Instruct	Q4_K_M	llama.cpp (CUDA)	115 tok/s	4030 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024
Llama 3.1 8B Instruct	F16	llama.cpp (CUDA)	47 tok/s	4445 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024
4× · Qwen3.8 27B	Q5_K_M	llama.cpp (CUDA)	39 tok/s	—	llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026
4× · Llama 3.3 70B Instruct	Q4_K_M	llama.cpp (CUDA)	17 tok/s	300 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024
2× · Llama 3.3 70B Instruct	Q4_K_M	llama.cpp (CUDA)	17 tok/s	355 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024
6× · Llama 3.3 70B Instruct	F16	llama.cpp (CUDA)	5.8 tok/s	580 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024

Our take

Should you buy the RTX 3090 for AI?

The RTX 3090 has 24GB of GDDR6X on a 384-bit bus for 936 GB/s, close to the RTX 4090's 1,008 GB/s for less than half the used price. That makes it the most popular card for local LLMs, running 32B models at 4 to 5 bit on one card and 70B models across two. It is one of the last GeForce cards with NVLink, which some dual-card builds use for faster tensor-parallel inference. Expect a 350W card whose memory runs hot and often benefits from new thermal pads; used prices were about $1,050 to $1,450 in October 2026.

What we like

24GB at 936 GB/s for less than half the price of a used RTX 4090
NVLink support for dual-card setups
Huge community knowledge base for multi-GPU builds
Full CUDA support, including FlashAttention 2

Watch out for

Hot rear-mounted memory on many cards
350W with sharp transient power spikes
Used cards may have mining or heavy-use history

Best for: Value 24GB single-card builds · Dual and quad 3090 rigs for 70B and larger models · LoRA fine-tuning on a budget

Specifications

RTX 3090 specs that matter for AI

Memory: 24GB GDDR6X
Memory bandwidth: 936 GB/s
Memory bus: 384-bit
Architecture: Ampere
Power: 350W
Interface: PCIe 4.0 x16
Released: 2020
Launch price: $1,499
Street price (used): ~$1,050 (2026-10)
Software: CUDA, Vulkan

Sources: NVIDIA GeForce RTX 3090 and 3090 Ti product page, TechPowerUp GPU Database, GeForce RTX 3090, BestValueGPU RTX 3090 price history

Builds

PC builds with the RTX 3090

All builds

Black HYTE Y40 case with purple-lit fans, an MSI triple-fan RTX 3090, and a Thermalright AIO cooler on an MSI motherboard.

Sleeper 3090

Updated single-GPU NVIDIA RTX 3090 workstation

RTX 3090

$6,486

High-end · 96GB VRAM

FJÄLLBO

AI workstation arranged on an IKEA table

4× RTX 3090

$3,090

Mid-range · 48GB VRAM

Hardigg 3090

Highly portable dual-3090 LLM rig

2× RTX 3090

Tall multi-tier black aluminum open frame stacked with EVGA RTX 3090 cards and bundles of power cables, lit with RGB lighting.

$23,182

Server · 288GB VRAM

The Machine

Enterprise ML server with 12x RTX 3090s

12× RTX 3090

Cross-shop

Alternatives to the RTX 3090

All GPUs

NVIDIA2022

24GB

GDDR6X · 1,008 GB/s

RTX 4090

~$2,500 used

AMD2022

24GB

GDDR6 · 960 GB/s

RX 7900 XTX

~$1,050 used

NVIDIA2016

24GB

GDDR5 · 346 GB/s

Tesla P40

~$230 used

NVIDIA2020

48GB

GDDR6 ECC · 768 GB/s

RTX A6000

~$4,400 used

Frequently asked questions

How much VRAM does the RTX 3090 have?

The NVIDIA GeForce RTX 3090 has 24GB of GDDR6X with 936 GB/s of memory bandwidth on a 384-bit bus.

What is the largest LLM the RTX 3090 can run?

Fully in GPU memory, the largest model in our catalog that runs comfortably is Qwen3.6 35B-A3B (35B (3B active)) at Q3_K_M, using about 19GB with 8K context at ~160 tokens/sec.

Can the RTX 3090 run Llama 3.3 70B?

Only with partial CPU offload. At Q4_K_M the model needs about 47GB with 8K context, so part of it spills into system RAM and speed drops to about 1.8 tokens/sec with CPU offload. For usable 70B speeds you want 48GB of VRAM or more (for example two 24GB cards).

What is the best LLM to run on the RTX 3090?

Our top pick is Qwen3.8 27B at Q4_K_M, which fits in 24GB with room for 8K of context and runs at ~39 tokens/sec. See our full list of the best models for the RTX 3090 for coding, reasoning, and vision picks.

Is the RTX 3090 good for local AI in 2026?

The default value pick for 24GB of CUDA memory and the backbone of most multi-GPU builds.