Google · Released April 2026
Gemma 4 31B
What it takes to run Gemma 4 31B locally: memory by quantization, the GPUs that handle it, and how fast it runs.
Updated
- Parameters
- 31B
- Architecture
- Dense
- Context
- 256K tokens
- License
- Apache 2.0 license
- Memory at Q4_K_M
- ~22GB
What do you need to run Gemma 4 31B?
Gemma 4 31B needs about 22GB of VRAM at Q4_K_M with 8K context (20GB for the weights plus KV cache and overhead). The smallest GPU that can hold it is a 20GB card like the RTX 4000 Ada at Q3_K_M, with little memory to spare. For a good balance of price and speed we recommend the RTX 5090 (~59 tokens/sec).
What to buy
GPUs for Gemma 4 31B
Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.
Memory
Gemma 4 31B VRAM requirements by quantization
Lower-bit quantizations shrink the model at some cost to quality. Q4_K_M is the usual sweet spot. Totals include the KV cache for the context length shown plus runtime overhead.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| Q2_K | Rough | 12 GB | 15 GB | 17 GB |
| Q3_K_M | Usable | 16 GB | 19 GB | 21 GB |
| Q4_K_M | RecommendedDefault | 20 GB | 22 GB | 24 GB |
| Q5_K_M | Very good | 23 GB | 26 GB | 28 GB |
| Q6_K | Near-lossless | 27 GB | 30 GB | 32 GB |
| Q8_0 | Practically lossless | 33 GB | 36 GB | 38 GB |
| BF16 | Full precision | 61 GB | 66 GB | 68 GB |
KV cache note: 60 layers: 50 sliding-window layers (1,024-token window, 16 KV heads x 256 dims) and 10 global layers (4 KV heads x 512 dims). Only the global layers grow with context: 2 x 10 x 4 x 512 x 2 bytes = 81,920 bytes per token, about 21GB at the full 256K. The sliding layers add a fixed buffer of about 0.8GB to 1.3GB depending on batch size.
Performance
How fast Gemma 4 31B runs
Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.
RTX 5090
Q5_K_M · 32GB
~59tok/s
RTX PRO 6000
Q8_0 · 96GB
~42tok/s
RTX 4090
Q4_K_M · 24GB
~40tok/s
RTX 3090
Q4_K_M · 24GB
~33tok/s
RTX PRO 5000 72GB
Q8_0 · 72GB
~32tok/s
RX 7900 XTX
Q4_K_M · 24GB
~30tok/s
V100 32GB
Q5_K_M · 32GB
~25tok/s
RTX 6000 Ada
Q8_0 · 48GB
~23tok/s
M5 Ultra 512GB
Q8_0 · 512GB
~20tok/s
M5 Ultra 256GB
Q8_0 · 256GB
~20tok/s
Measured
Gemma 4 31B benchmarks
- 2× RX 7900 XTXQ8_0 · llama.cpp32 tok/s llama.cpp discussion #15021: Performance of llama.cpp on AMD ROCm (HIP)
- 2× RX 7900 XTXQ8_0 · llama.cpp24 tok/s llama.cpp discussion #10879: Performance of llama.cpp with Vulkan
- 2× RX 7900 XTXQ8_0 · llama.cpp23 tok/s llama.cpp discussion #15021: Performance of llama.cpp on AMD ROCm (HIP)
Overview
About Gemma 4 31B
Gemma 4 31B is Google's flagship open model, a dense 31B with image input, a 256K context, a thinking mode, and an Apache 2.0 license. It is the strongest Gemma yet and a well-rounded choice for writing, multilingual chat, vision, and coding, though Qwen3.8 27B scores higher on reasoning and agentic benchmarks. At Q4_K_M it needs about 21.5GB with 8K context plus a fixed buffer of about 1GB for its sliding-window layers, which fits a 24GB GPU with a modest context; a 32GB RTX 5090 or a Mac with 36GB or more is more comfortable. As a dense model it is slower than Gemma 4 26B A4B on the same hardware.
Strengths
- +Excellent writing, multilingual chat, and image understanding
- +Thinking mode and 256K context
- +Compact long-context KV cache for a dense 31B model
- +Apache 2.0 license
Weaknesses
- −Tight on 24GB GPUs at Q4_K_M once context grows
- −Slower than mixture-of-experts models of similar quality
- −Trails Qwen3.8 27B on reasoning and agentic coding benchmarks
Setup
How to run Gemma 4 31B
$ ollama run gemma4:31b$ llama-server -hf unsloth/gemma-4-31B-it-GGUF:Q4_K_M -c 8192 -ngl 99Weights: google/gemma-4-31B-it · GGUF: unsloth/gemma-4-31B-it-GGUF
Alternatives
Models like Gemma 4 31B
Alibaba Qwen
Qwen3.8 27B
27B · 256K ctx
Gemma 4 26B A4B
26B (3.8B active) · 256K ctx
Alibaba Qwen
Qwen3.6 35B-A3B
35B (3B active) · 256K ctx
Gemma 3 27B
27.4B · 128K ctx
Frequently asked questions
How much VRAM do you need to run Gemma 4 31B?
About 22GB at Q4_K_M with 8K tokens of context: 19.6GB of weights, 1.7GB of KV cache, and 1.2GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.
What is the cheapest GPU that can run Gemma 4 31B?
The smallest card in our database that can hold it is the RTX 4000 Ada (20GB), at Q3_K_M with little memory to spare, generating ~18 tokens/sec.
Is Gemma 4 31B free for commercial use?
Gemma 4 31B is released under the Apache 2.0 license. That is a permissive license, so commercial use is allowed.
How long a context does Gemma 4 31B support?
Up to 262,144 tokens (256K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.
Sources: Gemma 4 31B IT model card (Hugging Face), Artificial Analysis: Gemma 4 31B vs Qwen3.5 9B, Ollama gemma4 library. Capability score basis: Artificial Analysis Intelligence Index v4: 39 in reasoning mode (April 2026), mapped onto the Llama Builds scale above gpt-oss-120b.