NVIDIA · Ada Lovelace · 2023

NVIDIA RTX 4000 Ada Generation for local AI

A single-slot 130W card that fits 20GB into small or dense builds, with modest bandwidth.

Updated October 14, 2026 · Speeds marked ~ are estimates; seemethodology

VRAM: 20GB
Bandwidth: 360 GB/s
Power: 130W
Runs well: 8 models

RTX 4000 Ada

Typically $1,250 at launch

Check price at Amazon Find it used on eBay

Can the RTX 4000 Ada run LLMs?

The RTX 4000 Ada has 20GB of GDDR6 ECC and 360 GB/s of memory bandwidth, which decides how big a model fits and how fast it generates text. It comfortably runs models up to about 26B parameters: the largest in our catalog is Gemma 4 26B A4B at Q4_K_M (17GB with 8K context, ~63 tokens/sec). Our top all-round pick for it is Qwen3.8 27B (Q4_K_M, ~17 tokens/sec). Llama 3.3 70B Instruct only runs with partial CPU offload, at about 1.5 tokens/sec with CPU offload.

Recommendations

Best models for the RTX 4000 Ada

Full list

Best overall · also coding, reasoning, vision
Qwen3.8 27B
Q4_K_M · 18GB · ~17 tokens/sec
Fits, tight
Runner-up
Gemma 4 26B A4B
Q4_K_M · 17GB · ~63 tokens/sec
Runs well
Fastest good model · also largest that fits
Qwen3.6 35B-A3B
Q3_K_M · 19GB · ~88 tokens/sec
Fits, tight

Read: the best LLMs to run on the RTX 4000 Ada

Compatibility

What runs on the RTX 4000 Ada

Every current model in our catalog, at the best quantization that fits in 20GB with 8K tokens of context. "Partial offload" means part of the model runs from system RAM (we assume 64GB), which is much slower.

Which open-weight models run on the NVIDIA RTX 4000 Ada Generation, with the best quantization and estimated speed
Model	Parameters	Best quant	Memory	Speed	Verdict
Gemma 4 E4B	8B	Q8_0	10 GB	~30 tok/s	Runs well
Qwen3.5 9B	9B	Q8_0	12 GB	~26 tok/s	Runs well
Phi-4	14.7B	Q6_K	15 GB	~23 tok/s	Runs well
gpt-oss-20b	20.9B (3.6B active)	MXFP4	15 GB	~62 tok/s	Runs well
Gemma 4 26B A4B	26B (3.8B active)	Q4_K_M	17 GB	~63 tok/s	Runs well
Qwen3.8 27B	27B	Q4_K_M	18 GB	~17 tok/s	Fits, tight
Gemma 4 31B	31B	Q3_K_M	19 GB	~18 tok/s	Fits, tight
Qwen3.6 35B-A3B	35B (3B active)	Q3_K_M	19 GB	~88 tok/s	Fits, tight
GLM-4.5-Air	106B (12B active)	—	—	—	Won't fitWhat it needs
Llama 4 Scout	109B (17B active)	—	—	—	Won't fitWhat it needs
gpt-oss-120b	117B (5.1B active)	—	—	—	Won't fitWhat it needs
Qwen3.5 122B-A10B	122B (10B active)	—	—	—	Won't fitWhat it needs
DeepSeek V4 Flash	284B (13B active)	—	—	—	Won't fitWhat it needs
GLM-5.3-Flash	320B (18B active)	—	—	—	Won't fitWhat it needs
GLM-5.3	744B (40B active)	—	—	—	Won't fitWhat it needs

Performance

RTX 4000 Ada vs. alternatives on Qwen3.8 27B

Generation speed in tokens per second for a single user. Token generation is limited mostly by memory bandwidth, which is why bandwidth predicts these results so well.

Estimated (method)

Measured

RTX 4000 Ada benchmark results

All benchmarks

Model	Quant	Runtime	Generation	Prompt	Source
Llama 2 7B Chat	Q4_0	llama.cpp (CUDA)	62 tok/s	2780 tok/s	llama.cpp discussion #15013: Performance of llama.cpp on NVIDIA CUDA · October 2026
Llama 3.1 8B Instruct	Q4_K_M	llama.cpp (CUDA)	59 tok/s	2450 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024
Llama 3.1 8B Instruct	F16	llama.cpp (CUDA)	21 tok/s	3120 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024
4× · Llama 3.3 70B Instruct	Q4_K_M	llama.cpp (CUDA)	7.4 tok/s	280 tok/s	XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub) · May 2024

Our take

Should you buy the RTX 4000 Ada for AI?

The RTX 4000 Ada Generation packs 20GB of ECC GDDR6 on a 160-bit bus into a single-slot card that draws just 130W. Its 360 GB/s bandwidth is modest, but the slim design lets builders fit several cards in a standard tower or a compact case. That makes it a good fit for multi-GPU builds where slot space and power are the main limits. NVIDIA's launch price was $1,250, and no reliable October 2026 street price was available.

What we like

Single-slot, 130W design
20GB with ECC
Fits several cards in one tower

Watch out for

Only 360 GB/s bandwidth
High price per GB

Best for: Dense multi-GPU builds · Small form factor AI workstations

Specifications

RTX 4000 Ada specs that matter for AI

Memory: 20GB GDDR6 ECC
Memory bandwidth: 360 GB/s
Memory bus: 160-bit
Architecture: Ada Lovelace
Power: 130W
Interface: PCIe 4.0 x16
Released: 2023
Launch price: $1,250
Software: CUDA, Vulkan

Sources: NVIDIA RTX 4000 Ada Generation product page

Builds

PC builds with the RTX 4000 Ada

All builds

Frame from PewDiePie's video showing his black aluminum-extrusion open rack with three large top fans, dense power cabling, and Seasonic Prime power supplies.

$20,429

Server · 140GB VRAM

PewDiePie "Odysseus"

PewDiePie's first NVIDIA multi-GPU inference rig

7× RTX 4000 Ada

Cross-shop

Alternatives to the RTX 4000 Ada

All GPUs

NVIDIA2023

16GB

GDDR6 · 288 GB/s

RTX 4060 Ti 16GB

~$380 used

Intel2025

24GB

GDDR6 · 456 GB/s

Arc Pro B60

~$650

NVIDIA2020

48GB

GDDR6 ECC · 768 GB/s

RTX A6000

~$4,400 used

Frequently asked questions

How much VRAM does the RTX 4000 Ada have?

The NVIDIA RTX 4000 Ada Generation has 20GB of GDDR6 ECC with 360 GB/s of memory bandwidth on a 160-bit bus.

What is the largest LLM the RTX 4000 Ada can run?

Fully in GPU memory, the largest model in our catalog that runs comfortably is Gemma 4 26B A4B (26B (3.8B active)) at Q4_K_M, using about 17GB with 8K context at ~63 tokens/sec.

Can the RTX 4000 Ada run Llama 3.3 70B?

Only with partial CPU offload. At Q4_K_M the model needs about 47GB with 8K context, so part of it spills into system RAM and speed drops to about 1.5 tokens/sec with CPU offload. For usable 70B speeds you want 48GB of VRAM or more (for example two 24GB cards).

What is the best LLM to run on the RTX 4000 Ada?

Our top pick is Qwen3.8 27B at Q4_K_M, which fits in 20GB with room for 8K of context and runs at ~17 tokens/sec. See our full list of the best models for the RTX 4000 Ada for coding, reasoning, and vision picks.

Is the RTX 4000 Ada good for local AI in 2026?

A single-slot 130W card that fits 20GB into small or dense builds, with modest bandwidth.