# Mistral Small 3.2 24B Instruct: hardware requirements
> Mistral Small 3.2 24B Instruct (Mistral AI) is a 24B dense model with a 128K-token context window, released under the Apache 2.0 license. It needs about 17GB of memory at Q4_K_M with 8K context.
Mistral Small 3.2 is the June 2025 update of Mistral's dense 24B model, with image input, reliable function calling, and a 128K context under Apache 2.0. Mistral replaced the line in March 2026 with Mistral Small 4, a 119B mixture-of-experts that needs roughly 75GB at 4-bit, so 24GB GPU owners are now better served by Gemma 4 31B or Qwen3.8 27B. At Q4_K_M it needs about 17GB with 8K context, which fits a 24GB card with room for longer context, or a 16GB card at Q3_K_M. It has no reasoning mode.

## Recommended hardware
- **Minimum GPU:** RTX 4060 Ti 16GB (16GB), Q3_K_M, ~20 tokens/sec
- **Best value:** RTX 4060 Ti 16GB (16GB), Q3_K_M, ~20 tokens/sec
- **Fastest single GPU:** RTX 5080 (16GB), Q3_K_M, ~62 tokens/sec
- **Mac / unified memory:** Ryzen AI Max+ 395 (128GB), Q8_0, ~7.6 tokens/sec

## Memory by quantization
| Quant | Weights | Total @ 8K ctx | Total @ 32K ctx |
| --- | --- | --- | --- |
| Q2_K | 8.9GB | 11.1GB | 15.1GB |
| Q3_K_M | 11.5GB | 13.8GB | 17.8GB |
| Q4_K_M | 14.3GB | 16.7GB | 20.7GB |
| Q5_K_M | 16.8GB | 19.3GB | 23.3GB |
| Q6_K | 19.5GB | 22.0GB | 26.1GB |
| Q8_0 | 25.1GB | 27.7GB | 31.8GB |
| BF16 | 47.2GB | 50.5GB | 54.5GB |

## GPUs that run it fully in memory (fastest first)
| GPU | Memory | Quant | Speed | Price |
| --- | --- | --- | --- | --- |
| RTX 5080 | 16GB | Q3_K_M | ~62 tok/s | ~$1,600 |
| RTX 5070 Ti | 16GB | Q3_K_M | ~59 tok/s | ~$1,200 |
| RTX 5090 | 32GB | Q8_0 | ~54 tok/s | ~$4,300 |
| RTX PRO 6000 | 96GB | Q8_0 | ~54 tok/s | ~$16,000 |
| RTX 4080 Super | 16GB | Q3_K_M | ~49 tok/s | ~$1,150 used |
| RTX 4090 | 24GB | Q5_K_M | ~46 tok/s | ~$2,500 used |
| RTX 4070 Ti Super | 16GB | Q3_K_M | ~45 tok/s | ~$900 used |
| RTX PRO 5000 72GB | 72GB | Q8_0 | ~41 tok/s | ~$9,100 |
| RTX 3090 | 24GB | Q5_K_M | ~38 tok/s | ~$1,050 used |
| RX 7900 XTX | 24GB | Q5_K_M | ~35 tok/s | ~$1,050 used |
| RX 9070 XT | 16GB | Q3_K_M | ~34 tok/s | ~$800 |
| RTX 5060 Ti 16GB | 16GB | Q3_K_M | ~30 tok/s | ~$620 |
| RTX 6000 Ada | 48GB | Q8_0 | ~30 tok/s | ~$4,750 used |
| M5 Ultra 512GB | 512GB | Q8_0 | ~26 tok/s | n/a |
| M5 Ultra 256GB | 256GB | Q8_0 | ~26 tok/s | ~$9,499 |
| V100 32GB | 32GB | Q8_0 | ~23 tok/s | ~$700 used |
| RTX A6000 | 48GB | Q8_0 | ~22 tok/s | ~$4,400 used |
| RTX 4000 Ada | 20GB | Q4_K_M | ~20 tok/s | $1,250 MSRP |
| RTX 4060 Ti 16GB | 16GB | Q3_K_M | ~20 tok/s | ~$380 used |
| MI50 32GB | 32GB | Q8_0 | ~18 tok/s | ~$600 used |
| RX 9060 XT 16GB | 16GB | Q3_K_M | ~18 tok/s | ~$470 |
| M3 Ultra 512GB | 512GB | Q8_0 | ~18 tok/s | $9,499 MSRP |
| R9700 | 32GB | Q8_0 | ~16 tok/s | ~$1,800 |
| M5 Max 128GB | 128GB | Q8_0 | ~16 tok/s | ~$5,100 |
| Arc Pro B60 | 24GB | Q5_K_M | ~14 tok/s | ~$650 |

## How to run it
- Ollama: `ollama run mistral-small3.2:24b`
- llama.cpp: `llama-server -hf unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF:Q4_K_M -c 8192 -ngl 99`
- Weights: https://huggingface.co/mistralai/Mistral-Small-3.2-24B-Instruct-2506

## Strengths
- Fast, reliable function calling and structured output
- Image input and 128K context
- Apache 2.0 license
- Fits 24GB GPUs with room for long context

## Weaknesses
- No reasoning mode; behind 2026 models on math and hard coding tasks
- Superseded by Mistral Small 4, which is far larger
- Large KV cache for a 24B model (160 KiB per token)
Full page: https://llamabuilds.ai/models/mistral-small-3-2-24b · Hardware guide: https://llamabuilds.ai/models/mistral-small-3-2-24b/requirements

---
Source: https://llamabuilds.ai/models/mistral-small-3-2-24b · Updated October 14, 2026 · Llama Builds. Speeds marked ~ are estimates from our bandwidth model (https://llamabuilds.ai/methodology); measured results are labeled as such.