The best local LLMs for the RTX 3090

Qwen3.8 27B (27B) fits comfortably in the RTX 3090's memory at Q4_K_M, using about 18GB with 8K context and generating ~39 tokens/sec. It is the most capable model in our catalog that runs well on this card. It is also our top pick for coding, reasoning and working with images.

Qwen3.8 27B, released in August 2026, is one of the strongest open models that fits a single 24GB GPU, with image input, a 262K native context, and switchable thinking. It uses the same hybrid layout as the Qwen3.5 and 3.6 27B models, where three of every four layers use linear attention, so long contexts cost far less memory than on older dense models. At Q4_K_M it needs about 18GB with 8K context, leaving room for 64K or more of context on a 24GB card, and 32GB cards can run Q6_K. It supersedes Qwen3.6 27B and Qwen3 32B, but its long reasoning traces make speed on slower hardware a real consideration.

Gemma 4 31B (31B) fits, with little room to spare in the RTX 3090's memory at Q4_K_M, using about 22GB with 8K context and generating ~33 tokens/sec. It is the next-best model that runs well here, and the one to try if the top pick doesn't suit you.

Gemma 4 31B is Google's flagship open model, a dense 31B with image input, a 256K context, a thinking mode, and an Apache 2.0 license. It is the strongest Gemma yet and a well-rounded choice for writing, multilingual chat, vision, and coding, though Qwen3.8 27B scores higher on reasoning and agentic benchmarks. At Q4_K_M it needs about 21.5GB with 8K context plus a fixed buffer of about 1GB for its sliding-window layers, which fits a 24GB GPU with a modest context; a 32GB RTX 5090 or a Mac with 36GB or more is more comfortable. As a dense model it is slower than Gemma 4 26B A4B on the same hardware.

Qwen3.6 35B-A3B (35B (3B active)) fits comfortably in the RTX 3090's memory at Q3_K_M, using about 19GB with 8K context and generating ~160 tokens/sec. Pick it when responsiveness matters more than squeezing out maximum quality. It is also the largest model that runs fully on this GPU.

Qwen3.6 35B-A3B is the April 2026 refresh of Qwen's fast mixture-of-experts model, focused on agentic coding and on preserving its reasoning across multi-turn tool use. With only 3B active parameters it generates text very quickly, and its hybrid linear attention keeps the KV cache tiny: the full 262K context needs about 5GB. At Q4_K_M it needs about 24GB with 8K context, which is tight on a 24GB card and comfortable on 32GB GPUs and Macs with 36GB or more; 12GB to 16GB GPUs run it well with experts offloaded to system RAM. It replaces Qwen3 30B-A3B and Qwen3 Coder 30B-A3B for most local users.