# The best GPUs for running AI models locally (2026)

> For most people, a used **RTX 3090** is still the GPU to buy: 24GB at 936 GB/s for about $1,050, enough to run Qwen3.8 27B at ~34 tok/s (estimated). On a tighter budget, the **RTX 5060 Ti 16GB** (about $620 new) runs gpt-oss-20b and Qwen3.5 9B entirely on the card. The **RTX 5090** is the fastest consumer card, but the 2026 memory shortage has pushed it to about $4,300.

If you want to run large language models on your own computer, the graphics card matters more than anything else in the box. Two numbers decide almost everything. The amount of memory sets the largest model you can load, and the speed of that memory sets how quickly the model writes. Buy the most VRAM you can afford first, then the most bandwidth, and only then look at the rest of the spec sheet.

That makes 24GB the sweet spot in 2026. The strongest open models that fit on one card, such as [Qwen3.8 27B](/models/qwen3-8-27b) and [Gemma 4 31B](/models/gemma-4-31b), need about 18GB to 23GB at 4-bit, and the cheapest fast way to get 24GB is still a used [RTX 3090](/gpus/rtx-3090). Below that, 16GB cards handle models up to about 20B parameters. Above it, you are paying for speed or for 32GB to 96GB of memory, and prices climb steeply.

> **About these prices:** Prices are October 2026 street prices, and they move week to week. The 2026 memory shortage pushed most cards well above their launch prices, from about 45 percent over MSRP for the [RTX 5060 Ti 16GB](/gpus/rtx-5060-ti-16gb) to more than double for the [RTX 5090](/gpus/rtx-5090). We quote what you can actually pay today, new or used.

## How we picked

We rank cards on memory first, because a model that doesn't fit in VRAM either won't run or spills into system RAM that is many times slower. Bandwidth comes second: generating each token means reading the model's active weights from memory, so a card with twice the bandwidth writes roughly twice as fast. Then price, measured as cost per gigabyte of reasonably fast memory. Software support breaks ties. NVIDIA's CUDA runs everything, AMD's ROCm and Vulkan run most popular tools, and Intel's stack works but needs more setup.

We have not lab-tested these cards. Speeds in this guide come from our estimate model, which predicts single-user generation speed from memory bandwidth and model size and is calibrated against published llama.cpp results. The [methodology page](/methodology) explains how it works and where it can be wrong. Estimates carry a tilde (~) and assume a llama.cpp-style runtime, 8K of context and one user. They cover generation speed only, not how fast a card reads a long prompt. Measured results live on our [benchmarks page](/benchmarks).

We judged every card on the open models people actually run in October 2026. For dense models that means Qwen3.8 27B and Gemma 4 31B. For fast mixture-of-experts (MoE) models, which only read a small slice of their weights per token, we used [Qwen3.6 35B-A3B](/models/qwen3-6-35b-a3b) and [gpt-oss-20b](/models/gpt-oss-20b). For the big ones, [gpt-oss-120b](/models/gpt-oss-120b) and [Qwen3.5 122B-A10B](/models/qwen3-5-122b-a10b). The chart shows why bandwidth matters so much: the same model at the same quantization runs about twice as fast on an RTX 5090 as on an RTX 3090, and more than three times as fast on a 3090 as on a 128GB mini PC.

Estimated generation speed for Qwen3.8 27B at Q4_K_M:

- RTX 5090: ~80 tok/s
- RTX 4090: ~47 tok/s
- RTX 3090: ~39 tok/s
- RX 7900 XTX: ~36 tok/s
- R9700: ~24 tok/s
- M5 Max 128GB: ~23 tok/s
- Arc Pro B70: ~20 tok/s
- Arc Pro B60: ~15 tok/s
- DGX Spark: ~12 tok/s
- Ryzen AI Max+ 395: ~11 tok/s

## Best overall: a used RTX 3090

**Best overall: NVIDIA GeForce RTX 3090** (24GB, 936 GB/s, ~$1,050 used). The default value pick for 24GB of CUDA memory and the backbone of most multi-GPU builds. Details: https://llamabuilds.ai/gpus/rtx-3090

The [RTX 3090](/gpus/rtx-3090) came out in 2020, and it is still the card we would buy first. It has 24GB of GDDR6X at 936 GB/s, within 8 percent of the [RTX 4090](/gpus/rtx-4090)'s bandwidth, and used cards sold for about $1,050 to $1,450 in October 2026. At about $44 per gigabyte, it is the cheapest way to get fast memory in a card that drops into a normal desktop and runs current CUDA releases. Buy it if you want the best models that fit on one card and you are comfortable buying used.

It runs Qwen3.8 27B at Q5_K_M (about 21GB) at ~34 tok/s, estimated, or at Q4_K_M with 64K of context in about 22GB. Gemma 4 31B fits at Q4_K_M (about 22.5GB) at ~33 tok/s, though only with a modest context of around 16K tokens. MoE models are much quicker: [Gemma 4 26B A4B](/models/gemma-4-26b-a4b) at Q5_K_M runs at ~109 tok/s and gpt-oss-20b at ~119 tok/s. With 64GB of system RAM holding the rest of the model, gpt-oss-120b runs at ~21 tok/s. The [best models for the RTX 3090](/gpus/rtx-3090/best-models) page lists everything else.

The downsides come with age. Many cards spent years in mining rigs, the memory chips on the back of the board run hot and often need new thermal pads, and the 350W rating hides sharp power spikes, so don't skimp on the power supply. Newer cards also accelerate FP8 and FP4 formats in hardware, which the 3090 can't. That matters for some vLLM deployments and very little for llama.cpp's GGUF files.

If you prefer AMD, the [RX 7900 XTX](/gpus/rx-7900-xtx) sells for about the same $1,050 used with 24GB at 960 GB/s, but prompt processing is slower and some tools are CUDA-only ([RTX 3090 vs RX 7900 XTX](/compare/rtx-3090-vs-rx-7900-xtx)). The RTX 4090 is nearly 20 percent faster at generation (~40 tok/s on Qwen3.8 27B at Q5_K_M, estimated) and much faster at reading long prompts, but used cards cost about $2,500 ([RTX 3090 vs RTX 4090](/compare/rtx-3090-vs-rtx-4090)). If you won't buy used, the [RTX 5070 Ti](/gpus/rtx-5070-ti) costs about $1,200 new with 896 GB/s, but only 16GB ([RTX 3090 vs RTX 5070 Ti](/compare/rtx-3090-vs-rtx-5070-ti)). Our [RTX 3090 guide](/guides/rtx-3090-for-local-ai) covers buying and setting one up, and the [Sleeper 3090](/builds/sleeper-rtx-3090) shows a complete single-card build.

## Best budget: RTX 5060 Ti 16GB

**Best budget: NVIDIA GeForce RTX 5060 Ti 16GB** (16GB, 448 GB/s, ~$620). The cheapest new way to get 16GB of CUDA memory, and a strong first LLM card. Details: https://llamabuilds.ai/gpus/rtx-5060-ti-16gb

The [RTX 5060 Ti 16GB](/gpus/rtx-5060-ti-16gb) is the cheapest new card we'd buy for local AI. About $620 gets you 16GB of GDDR7 at 448 GB/s, full CUDA support and a 180W board power that works with modest power supplies. It's the card for adding local AI to a gaming PC or building a first rig on a budget.

Sixteen gigabytes covers models up to about 20B parameters. gpt-oss-20b fits entirely on the card (about 15GB) and runs at ~74 tok/s, estimated, and [Qwen3.5 9B](/models/qwen3-5-9b) runs at practically lossless Q8_0 at ~32 tok/s. MoE models hold up well when they spill over: Qwen3.6 35B-A3B at Q4_K_M runs at ~42 tok/s with part of it in system RAM. Dense 27B to 31B models are the limit. Qwen3.8 27B squeezes in only at Q3_K_M (about 15GB), with little room left for context.

The weak spot is the 128-bit memory bus, which leaves it roughly 40 percent slower than a 3090 on models both cards can hold. It also uses a PCIe 5.0 x8 link, which matters little for one card but can slow a two-card setup on an older PCIe 3.0 motherboard. Check that the listing says 16GB, because an 8GB card shares the name.

On a tighter budget, a used [RTX 3060 12GB](/gpus/rtx-3060-12gb) costs about $270 and runs Qwen3.5 9B at Q6_K at ~30 tok/s ([RTX 3060 12GB vs RTX 5060 Ti 16GB](/compare/rtx-3060-12gb-vs-rtx-5060-ti-16gb)). For about $650 new, the [Intel Arc Pro B60](/gpus/arc-pro-b60) gives you 24GB, enough for Qwen3.8 27B at Q5_K_M, but its 456 GB/s bus and less mature software hold it to ~13 tok/s on that model ([RTX 3090 vs Arc Pro B60](/compare/rtx-3090-vs-arc-pro-b60)). The [RX 9060 XT 16GB](/gpus/rx-9060-xt-16gb) costs about $470 but has nearly 30 percent less bandwidth ([RTX 5060 Ti vs RX 9060 XT](/compare/rtx-5060-ti-16gb-vs-rx-9060-xt-16gb)), and a used [RTX 4060 Ti 16GB](/gpus/rtx-4060-ti-16gb) at about $380 is slower still ([RTX 4060 Ti vs RTX 5060 Ti](/compare/rtx-4060-ti-16gb-vs-rtx-5060-ti-16gb)). If you can stretch to about $1,200, the RTX 5070 Ti doubles the bandwidth and runs gpt-oss-20b at ~125 tok/s. Two 5060 Tis make a 32GB pair for about $1,240, and the [GPUter build](/builds/gputer-rtx-5060-ti) shows a complete single-card system.

## Fastest consumer GPU: RTX 5090

**Fastest consumer GPU: NVIDIA GeForce RTX 5090** (32GB, 1,792 GB/s, ~$4,300). The fastest consumer card for local LLMs, but shortage pricing has more than doubled its cost. Details: https://llamabuilds.ai/gpus/rtx-5090

The [RTX 5090](/gpus/rtx-5090) is the fastest card you can put in a normal desktop. Its 1,792 GB/s of bandwidth is almost double a 3090's, and its 32GB holds the best single-card models at higher quality with room for long context. Buy it if speed is worth more to you than money, for example for coding agents that generate thousands of tokens per task.

Estimated, it runs Qwen3.8 27B at Q6_K (about 24GB) at ~60 tok/s, or at Q4_K_M at ~80 tok/s, about twice the 3090's ~39 tok/s. Gemma 4 31B runs at Q5_K_M at ~59 tok/s, and Qwen3.6 35B-A3B fits at Q5_K_M with room to spare and runs at ~196 tok/s.

The catch is price. Launch MSRP was $1,999; in October 2026 new cards listed for about $4,300 to $5,000 because of the GDDR7 shortage. It also draws up to 575W through a single 12V-2x6 connector, so plan on a quality 1,000W or larger power supply and good case airflow. And 32GB still isn't enough for gpt-oss-120b. With the rest of the model in system RAM, it runs at ~25 tok/s, barely ahead of a 3090 doing the same (~21 tok/s), because the slow system memory sets the pace.

Two used 3090s cost about $2,100 for 48GB: more memory, but not more speed. The RTX 4090 at about $2,500 used sits in between ([RTX 4090 vs RTX 5090](/compare/rtx-4090-vs-rtx-5090), [RTX 3090 vs RTX 5090](/compare/rtx-3090-vs-rtx-5090)). The [RTX 5080](/gpus/rtx-5080) is not a cheaper route to the same thing, since it has only 16GB ([RTX 5080 vs RTX 5090](/compare/rtx-5080-vs-rtx-5090)).

## Best for multi-GPU builds: Radeon AI PRO R9700

**Best for multi-GPU builds: AMD Radeon AI PRO R9700** (32GB, 640 GB/s, ~$1,800). 32GB for far less than NVIDIA workstation cards and built for stacking, but no faster per card than an RX 9070 XT. Details: https://llamabuilds.ai/gpus/radeon-ai-pro-r9700

The [Radeon AI PRO R9700](/gpus/radeon-ai-pro-r9700) is the cheapest well-supported way to put a lot of new VRAM in one tower. Each card has 32GB at 640 GB/s, a blower cooler that exhausts out the back and a 300W rating, so four fit in a workstation case for 128GB. At about $1,800 each (up from a $1,299 MSRP), that's about $56 per gigabyte, against about $134 for an RTX 5090.

One card runs Qwen3.6 35B-A3B at Q5_K_M at ~78 tok/s and Qwen3.8 27B at Q6_K at ~18 tok/s, estimated. Four cards hold gpt-oss-120b entirely in VRAM at ~37 tok/s and Qwen3.5 122B-A10B at Q6_K at ~20 tok/s.

Be clear about what you are buying: capacity, not speed. On dense models one R9700 is slower than a used 3090 (~24 vs ~39 tok/s on Qwen3.8 27B at Q4_K_M, estimated), and tools that only support CUDA won't run. ROCm officially supports the card, and both llama.cpp and vLLM work on it, so the common local stack is covered. Plan on Linux, where ROCm support is most complete.

The [Arc Pro B70](/gpus/arc-pro-b70) is the cheapest new 32GB card at about $1,300, with similar bandwidth (608 GB/s) but software that still needs more setup ([R9700 vs Arc Pro B70](/compare/radeon-ai-pro-r9700-vs-arc-pro-b70)). Four used 3090s give you 96GB for about $4,200 and run gpt-oss-120b faster (~56 tok/s), but they draw about 1,400W on their own and usually need risers or an open frame, like the [FJÄLLBO build](/builds/local-ai-workstation-4x-nvidia-3090-ikea). If you want CUDA at 32GB, see [RTX 5090 vs R9700](/compare/rtx-5090-vs-radeon-ai-pro-r9700).

## Most memory on one card: RTX PRO 6000 Blackwell

**Most memory on one card: NVIDIA RTX PRO 6000 Blackwell** (96GB, 1,792 GB/s, ~$16,000). The most memory on a single PCIe card at RTX 5090-class bandwidth, at a very steep price. Details: https://llamabuilds.ai/gpus/rtx-pro-6000-blackwell

The [RTX PRO 6000 Blackwell](/gpus/rtx-pro-6000-blackwell) puts 96GB on one card at the same 1,792 GB/s as the RTX 5090. That is enough to run 100B-class models entirely in VRAM, without the splitting, risers and power planning that multi-GPU rigs need. It is a card for professionals and well-funded enthusiasts. At about $16,000 in October 2026, close to double its 2025 street price, one card costs more than a complete four-3090 workstation.

The speed difference is large. Estimated, gpt-oss-120b runs at ~172 tok/s, Qwen3.5 122B-A10B at Q4_K_M at ~105 tok/s and [GLM-4.5-Air](/models/glm-4-5-air) at Q5_K_M at ~80 tok/s. A 128GB mini PC manages ~35 tok/s on gpt-oss-120b. Two cards (192GB) can run [DeepSeek V4 Flash](/models/deepseek-v4-flash) at its native FP4, at an estimated ~72 tok/s.

NVIDIA sells three versions with the same memory: a 600W Workstation Edition, a 300W Max-Q blower card that stacks four to a tower (as in the [a16z workstation](/builds/a16zs-personal-ai-workstation-with-four-nvidia-rtx-6000-pro-blackwell-max-q-gpus)), and a passively cooled Server Edition. If 72GB is enough, the [RTX PRO 5000 Blackwell 72GB](/gpus/rtx-pro-5000-blackwell-72gb) holds gpt-oss-120b for about $9,100, the most recent street price we found ([RTX 5090 vs RTX PRO 6000](/compare/rtx-5090-vs-rtx-pro-6000-blackwell)).

## Best for 100B-class models: Ryzen AI Max+ 395

**Best for 100B-class models: AMD Ryzen AI Max+ 395 (128GB)** (128GB, 256 GB/s, ~$3,300). The cheapest way to run 100B-class MoE models in one quiet box, though prices have climbed past $3,300. Details: https://llamabuilds.ai/gpus/ryzen-ai-max-plus-395

Sometimes the best GPU for local AI isn't a graphics card. The [Ryzen AI Max+ 395](/gpus/ryzen-ai-max-plus-395), sold in mini PCs and known as Strix Halo, shares up to 128GB of memory between its CPU and its integrated GPU. Windows lets the GPU use up to 96GB, and Linux about 120GB after a kernel setting change. That's as much GPU memory as the $16,000 RTX PRO 6000, in a quiet box whose chip runs at 45 to 120W.

Bandwidth is only 256 GB/s, which suits MoE models that read a small slice of their weights per token. gpt-oss-120b runs at ~35 tok/s, estimated, and Qwen3.5 122B-A10B at Q4_K_M at ~18 tok/s. A 24GB card with 64GB of system RAM manages ~21 tok/s on gpt-oss-120b, so the mini PC wins on the big models before you even count noise and power.

Dense models are another story. Gemma 4 31B at Q4_K_M runs at ~10 tok/s here against ~33 tok/s on a 3090, and prompt processing is slower than on NVIDIA hardware, so long documents take a while to read in. 128GB systems sold for about $3,300 to $4,350 in October 2026, up from about $2,000 in 2025. Our guide to the [best Macs and mini PCs for local LLMs](/best/macs-and-mini-pcs-for-local-llms) compares it with the [Mac Studio](/gpus/mac-studio-m5-max-128gb) and NVIDIA's [DGX Spark](/gpus/dgx-spark) ([DGX Spark vs Ryzen AI Max+ 395](/compare/dgx-spark-vs-ryzen-ai-max-plus-395)).

## Our picks compared

The table puts our picks next to the main alternatives. The 120B column shows where the lines fall for a big model: 16GB cards can't hold gpt-oss-120b even with 64GB of system RAM, 24GB and 32GB cards run it with part of the model offloaded (amber), and only the 96GB card and the mini PC run it entirely in fast memory.

| GPU | Memory | Bandwidth | Price |
| --- | --- | --- | --- |
| RTX 3090 | 24GB | 936 GB/s | ~$1,050 used |
| RTX 5060 Ti 16GB | 16GB | 448 GB/s | ~$620 |
| RTX 5090 | 32GB | 1,792 GB/s | ~$4,300 |
| R9700 | 32GB | 640 GB/s | ~$1,800 |
| RTX PRO 6000 | 96GB | 1,792 GB/s | ~$16,000 |
| Ryzen AI Max+ 395 | 128GB | 256 GB/s | ~$3,300 |
| RTX 4090 | 24GB | 1,008 GB/s | ~$2,500 used |
| RX 7900 XTX | 24GB | 960 GB/s | ~$1,050 used |
| RTX 5070 Ti | 16GB | 896 GB/s | ~$1,200 |
| Arc Pro B60 | 24GB | 456 GB/s | ~$650 |
| Arc Pro B70 | 32GB | 608 GB/s | ~$1,300 |

## How much VRAM do you need?

A model needs memory for its weights, for the context cache that grows with every token in the conversation, and for some runtime overhead. At 4-bit (Q4_K_M) with 8K of context, this is roughly where each memory size lands:

| VRAM | What fits comfortably | Example |
| --- | --- | --- |
| 8GB | Small models up to about 9B | Qwen3.5 9B at Q4_K_M, about 7GB |
| 12GB to 16GB | Models up to about 20B | gpt-oss-20b, about 15GB |
| 24GB | Dense 27B to 31B, MoE up to 35B | Qwen3.8 27B at Q4_K_M, about 18GB |
| 32GB | The same models at Q5 or Q6, or with long context | Qwen3.8 27B at Q6_K, about 24GB |
| 70GB and up | 100B-class MoE models | gpt-oss-120b, about 68GB |

| Model | Size | Q4_K_M | Q6_K | Q8_0 |
| --- | --- | --- | --- | --- |
| Qwen3.5 9B | 9B | 7.2GB | 9.5GB | 12.0GB |
| gpt-oss-20b | 20.9B (3.6B active) | 15.0GB (MXFP4) | - | - |
| Gemma 4 26B A4B | 26B (3.8B active) | 17.3GB | 23.2GB | 29.5GB |
| Qwen3.8 27B | 27B | 18.1GB | 24.8GB | 31.6GB |
| Gemma 4 31B | 31B | 22.5GB | 29.7GB | 35.9GB |
| Qwen3.6 35B-A3B | 35B (3B active) | 23.7GB | 32.0GB | 41.0GB |
| gpt-oss-120b | 117B (5.1B active) | 68.3GB (MXFP4) | - | - |
| Qwen3.5 122B-A10B | 122B (10B active) | 77.0GB | 104.5GB | 134.3GB |

_Total memory with 8K context, estimated._

Context adds up. Qwen3.8 27B at Q4_K_M needs about 18GB at 8K tokens, about 22GB at 64K and about 26GB at 128K. Gemma 4 31B grows faster, from about 22.5GB at 8K to about 27GB at 64K. Quantization is the other lever: Q4_K_M keeps most of a model's quality at about 30 percent of its 16-bit size, and Q8_0 is close to lossless at about half. Our guides to [how much VRAM you need](/guides/how-much-vram-do-you-need) and [quantization](/guides/llm-quantization-explained) go deeper, and the [VRAM calculator](/tools/vram-calculator) checks any model, quantization and context length against your card.

## New or used?

Used is where the value is in 2026. The shortage hit new cards hardest, so a used RTX 3090 at about $1,050 gets you more memory than a new $1,200 RTX 5070 Ti. When you buy used, pick a seller or marketplace that accepts returns, and run a long generation job as soon as the card arrives while you watch temperatures. Hot memory or throttling usually points to tired thermal pads, which are a cheap fix, but you want to find out inside the return window.

Old datacenter cards are the other used route, and they ask more of you. The [Tesla P40](/gpus/tesla-p40) is the cheapest 24GB card at about $230 to $330, but it has no tensor cores, very slow FP16, passive cooling that needs a fan shroud, and no support in CUDA 13 ([RTX 3090 vs Tesla P40](/compare/rtx-3090-vs-tesla-p40)). The [Tesla V100 32GB](/gpus/tesla-v100-32gb) has 900 GB/s for about $700, but most units are SXM2 modules that need an adapter board and custom cooling, as in this [V100 workstation](/builds/nvidia-v100-modded-workstation-sxm2), and CUDA 13 dropped it too. The [Instinct MI50 32GB](/gpus/instinct-mi50-32gb) was a bargain at about $150 before the shortage; at $600 or more, and without support in current ROCm releases, it no longer is ([RTX 3090 vs MI50](/compare/rtx-3090-vs-instinct-mi50-32gb)).

Used workstation cards hold their value. A 48GB [RTX A6000](/gpus/rtx-a6000) sold for about $4,400 and an [RTX 6000 Ada](/gpus/rtx-6000-ada) for $4,750 or more, which is RTX 5090 money for more memory but much less bandwidth.

## What about two or more GPUs?

Adding cards adds memory, not speed. llama.cpp and the apps built on it split a model's layers across GPUs, and each token passes through the cards one after another, so two cards run a model at about the speed of one. What you gain is room for bigger models and better quantizations. Estimated, two 3090s (48GB) run Qwen3.8 27B at Q8_0 at ~22 tok/s and cut gpt-oss-120b's offloading enough to reach ~32 tok/s. Four (96GB) hold gpt-oss-120b entirely, at ~56 tok/s.

Tensor parallelism in vLLM and similar runtimes can make several cards faster than one, but it works best with matched cards and a fast link between them. The 3090 is one of the last GeForce cards with NVLink, which is why dual-3090 rigs such as the [Hardigg 3090](/builds/portable-25l-nvlinked-dual-3090-llm-rig) still use it.

Plan the platform before you buy the cards. Most consumer motherboards have one full-speed x16 slot and at best split it into two x8 slots, so two GPUs is the practical limit. Three or more call for a workstation or server platform with more PCIe lanes (Threadripper, Xeon or EPYC), as in [The Machine](/builds/the-machine-12x-rtx-3090) with twelve 3090s or [PewDiePie's seven-card RTX 4000 Ada rig](/builds/pewdiepie-ai-gpu-rig-nvidia-rtx-4000). Size the power supply for the cards' combined rating plus 20 to 30 percent headroom. If you are building a Linux server, our guide to [NVIDIA GPUs on Linux](/guides/nvidia-gpus-on-linux) covers drivers and setup.

## What to avoid

**8GB cards.** The [RTX 5060](/gpus/rtx-5060) costs about $450, and models like Qwen3.5 9B at Q4_K_M (about 7GB) are its ceiling. Spend about $170 more on the 16GB RTX 5060 Ti.

**12GB cards at new-card prices.** The [RTX 5070](/gpus/rtx-5070) has fast memory (672 GB/s), but at about $800 for 12GB it costs more than the 16GB 5060 Ti and holds smaller models.

**The RTX 5080 for AI alone.** At about $1,600 it has the same 16GB as the RTX 5070 Ti and only about 7 percent more bandwidth.

**A used RTX 4090 at full price.** At $2,500 to $3,000 it costs more than twice as much as a 3090 for the same 24GB. It makes sense only if you need its faster prompt processing.

**Old hardware at shortage prices.** Pascal cards like the P40 have lost CUDA 13 support, and the MI50 has lost current ROCm support. They still run llama.cpp, but they are tinkerer's cards, not safe buys.

**Low-memory lookalikes.** The RTX 5060 Ti, RTX 3060 and RX 9060 XT all come in 8GB versions with the same name. The 8GB RTX 3060 also has a narrower bus. Check the memory size before you pay.

## Frequently asked questions

### Is 16GB of VRAM enough for local LLMs?

Yes, for models up to about 20B parameters. A 16GB card holds gpt-oss-20b (about 15GB) or Qwen3.5 9B at Q8_0 (about 12GB) entirely in VRAM, and an RTX 5060 Ti 16GB runs gpt-oss-20b at about 74 tok/s (estimated). Dense 27B to 31B models such as Qwen3.8 27B need about 18GB to 23GB at 4-bit, so they call for a 24GB card or a 3-bit quantization with little room for context.

### Is the RTX 3090 still worth buying for AI in 2026?

Yes. A used RTX 3090 has 24GB of VRAM at 936 GB/s and sold for about $1,050 to $1,450 in October 2026, the cheapest way to get fast 24GB memory with current CUDA support. It runs Qwen3.8 27B at Q5_K_M at about 34 tok/s (estimated). Expect a 350W card whose memory may need new thermal pads, and buy from a seller that accepts returns.

### Is the RTX 5090 worth it for running LLMs?

Only if speed matters more to you than cost. The RTX 5090's 32GB and 1,792 GB/s make it about twice as fast as an RTX 3090, running Qwen3.8 27B at Q4_K_M at about 80 tok/s against about 39 tok/s (both estimates). It cost about $4,300 new in October 2026, more than double its $1,999 MSRP, while two used 3090s give you 48GB for about $2,100.

### How much VRAM do I need to run gpt-oss-120b?

About 68GB with 8K of context at its native MXFP4 format. That fits on a single 96GB RTX PRO 6000 Blackwell (about 172 tok/s, estimated), a 72GB RTX PRO 5000, or a 128GB unified-memory system such as a Ryzen AI Max+ 395 mini PC (about 35 tok/s). A 24GB GPU with 64GB of system RAM can also run it by offloading part of the model, at about 21 tok/s.

### Are AMD and Intel GPUs good for local LLMs?

Yes, with caveats. AMD's RX 7900 XTX and Radeon AI PRO R9700 run llama.cpp well through ROCm or Vulkan, and Intel's Arc Pro B60 and B70 offer 24GB and 32GB at low prices. AMD cards are slower at prompt processing than comparable NVIDIA cards, Intel's software needs more setup, and some tools only support CUDA. If you mainly use llama.cpp and apps built on it, they are reasonable choices.

### Do two GPUs make an LLM run faster?

Not with the usual llama.cpp setup. Splitting a model across two GPUs adds their memory together, but it runs at about the speed of one card because each token passes through the cards in turn. Two RTX 3090s (48GB) let you run bigger models or higher-quality quantizations, and tensor-parallel runtimes such as vLLM can add speed with matched cards.

---
Source: https://llamabuilds.ai/best/gpus-for-local-llms · Updated October 14, 2026 · Llama Builds. Speeds marked ~ are estimates from our bandwidth model (https://llamabuilds.ai/methodology); measured results are labeled as such.