Skip to content
Llama Builds

Alibaba Qwen · Released April 2025

Qwen3 32B

What it takes to run Qwen3 32B locally: memory by quantization, the GPUs that handle it, and how fast it runs.

Updated

Parameters
32.8B
Architecture
Dense
Context
32K tokens
License
Apache 2.0 license
Memory at Q4_K_M
~23GB

What do you need to run Qwen3 32B?

Qwen3 32B needs about 23GB of VRAM at Q4_K_M with 8K context (20GB for the weights plus KV cache and overhead). The smallest GPU that can hold it is a 20GB card like the RTX 4000 Ada at Q3_K_M, with little memory to spare. For a good balance of price and speed we recommend the RTX 3090 (~41 tokens/sec).

See the full hardware guide for Qwen3 32B

What to buy

GPUs for Qwen3 32B

Cheapest that runs it

RTX 4000 Ada

20GB · $1,250 MSRP

Quant
Q3_K_M
Speed (est.)
~18 tok/s
Fits, tight
Check price at Amazon

Best value

RTX 3090

24GB · ~$1,050 used

Quant
Q3_K_M
Speed (est.)
~41 tok/s
Runs well
Check price at Amazon

Fastest

RTX 5090

32GB · ~$4,300

Quant
Q5_K_M
Speed (est.)
~58 tok/s
Runs well
Check price at Amazon

Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.

Memory

Qwen3 32B VRAM requirements by quantization

Lower-bit quantizations shrink the model at some cost to quality. Q4_K_M is the usual sweet spot. Totals include the KV cache for the context length shown plus runtime overhead.

Memory needed for Qwen3 32B at each quantization level
QuantizationQualityWeightsTotal @ 8K ctxTotal @ 32K ctx
Q2_KRough12 GB15 GB22 GB
Q3_K_MUsable16 GB19 GB26 GB
Q4_K_MRecommendedDefault20 GB23 GB30 GB
Q5_K_MVery good23 GB27 GB33 GB
Q6_KNear-lossless27 GB31 GB37 GB
Q8_0Practically lossless35 GB39 GB45 GB
BF16Full precision66 GB70 GB77 GB

KV cache note: 2 x 64 layers x 8 KV heads x 128 dims x 2 bytes = 262,144 bytes (256 KiB) per token: about 2.1GB at 8K and 8.6GB at the native 32K context, which is why long contexts do not fit next to the weights on a 24GB card.

Performance

How fast Qwen3 32B runs

Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.

  1. RTX 5090

    Q5_K_M · 32GB

    ~58tok/s

  2. RTX 4090

    Q3_K_M · 24GB

    ~48tok/s

  3. RTX 3090

    Q3_K_M · 24GB

    ~41tok/s

  4. RTX PRO 6000

    Q8_0 · 96GB

    ~40tok/s

  5. RX 7900 XTX

    Q3_K_M · 24GB

    ~37tok/s

  6. RTX PRO 5000 72GB

    Q8_0 · 72GB

    ~30tok/s

  7. V100 32GB

    Q5_K_M · 32GB

    ~24tok/s

  8. RTX 6000 Ada

    Q8_0 · 48GB

    ~22tok/s

  9. MI50 32GB

    Q5_K_M · 32GB

    ~19tok/s

  10. M5 Ultra 512GB

    Q8_0 · 512GB

    ~19tok/s

Estimated (method)

Measured

Qwen3 32B benchmarks

Overview

About Qwen3 32B

Qwen3 32B was the strongest dense model most people could run on a single 24GB GPU in 2025, and it is our reference point for 32B-class speed. It has been superseded by the Qwen3.5, 3.6, and 3.8 27B models, which are smaller, add vision and a 262K context, and score far higher on reasoning and agentic coding. At Q4_K_M it needs about 23GB with 8K context, so a 24GB card fits it with little room for longer context; 32GB cards and Macs with 36GB or more are more comfortable. Because it is dense, every token reads all 32B parameters, so it is much slower than a 30B-A3B mixture-of-experts model on the same GPU.

Strengths

  • +Toggleable thinking with strong math and coding for a 2025 model
  • +Dense weights quantize predictably; Q4_K_M loses little quality
  • +Apache 2.0 license and broad runtime support

Weaknesses

  • −Superseded by Qwen3.8 27B, which is smaller and much stronger
  • −Large KV cache (256 KiB per token) limits context on 24GB cards
  • −Native context is only 32K tokens
  • −Text only
ChatReasoningCodingMultilingualMathAgents & toolsToggleable reasoning

Setup

How to run Qwen3 32B

Ollama
$ ollama run qwen3:32b
llama.cpp
$ llama-server -hf Qwen/Qwen3-32B-GGUF:Q4_K_M -c 8192 -ngl 99
MLX (Mac)
$ mlx_lm.chat --model mlx-community/Qwen3-32B-4bit

Weights: Qwen/Qwen3-32B · GGUF: Qwen/Qwen3-32B-GGUF

Alternatives

Models like Qwen3 32B

Alibaba Qwen

Qwen3.8 27B

27B · 256K ctx

~18GBat Q4_K_M
CodingAgents & toolsReasoning

Google

Gemma 4 31B

31B · 256K ctx

~22GBat Q4_K_M
ChatVisionReasoning

Alibaba Qwen

Qwen3.6 35B-A3B

35B (3B active) · 256K ctx

~24GBat Q4_K_M
CodingAgents & toolsReasoning

Alibaba Qwen

Qwen3 30B-A3B (2507)

30.5B (3.3B active) · 256K ctx

~21GBat Q4_K_M
ChatReasoningCoding

Frequently asked questions

How much VRAM do you need to run Qwen3 32B?

About 23GB at Q4_K_M with 8K tokens of context: 19.8GB of weights, 2.1GB of KV cache, and 1.2GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.

What is the cheapest GPU that can run Qwen3 32B?

The smallest card in our database that can hold it is the RTX 4000 Ada (20GB), at Q3_K_M with little memory to spare, generating ~18 tokens/sec.

Is Qwen3 32B free for commercial use?

Qwen3 32B is released under the Apache 2.0 license. That is a permissive license, so commercial use is allowed.

How long a context does Qwen3 32B support?

Up to 32,768 tokens (32K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.

Sources: Qwen3 32B model card (Hugging Face), Qwen3 launch post. Capability score basis: Llama Builds scale anchor (70 to 75 band), consistent with its Artificial Analysis Intelligence Index v3 standing in reasoning mode (October 2025).