Meta · Released December 2024
Llama 3.3 70B Instruct
What it takes to run Llama 3.3 70B Instruct locally: memory by quantization, the GPUs that handle it, and how fast it runs.
Updated
- Parameters
- 70.6B
- Architecture
- Dense
- Context
- 128K tokens
- License
- Llama 3.3 Community License
- Memory at Q4_K_M
- ~47GB
What do you need to run Llama 3.3 70B Instruct?
Llama 3.3 70B Instruct needs about 47GB of VRAM at Q4_K_M with 8K context (43GB for the weights plus KV cache and overhead). The smallest GPU that runs it comfortably at Q3_K_M is a 48GB card like the RTX A6000. For a good balance of price and speed we recommend the RTX 6000 Ada (~22 tokens/sec).
What to buy
GPUs for Llama 3.3 70B Instruct
Speeds are estimates at the quantization shown with 8K context. We may earn a commission when you buy through these links. Learn more.
Memory
Llama 3.3 70B Instruct VRAM requirements by quantization
Lower-bit quantizations shrink the model at some cost to quality. Q4_K_M is the usual sweet spot. Totals include the KV cache for the context length shown plus runtime overhead.
| Quantization | Quality | Weights | Total @ 8K ctx | Total @ 32K ctx |
|---|---|---|---|---|
| Q2_K | Rough | 26 GB | 30 GB | 38 GB |
| Q3_K_M | Usable | 34 GB | 39 GB | 47 GB |
| Q4_K_M | RecommendedDefault | 43 GB | 47 GB | 55 GB |
| Q5_K_M | Very good | 50 GB | 55 GB | 63 GB |
| Q6_K | Near-lossless | 58 GB | 63 GB | 71 GB |
| Q8_0 | Practically lossless | 75 GB | 81 GB | 89 GB |
| BF16 | Full precision | 141 GB | 149 GB | 157 GB |
KV cache note: 2 x 80 layers x 8 KV heads x 128 dims x 2 bytes = 327,680 bytes (320 KiB) per token: about 2.7GB at 8K context and 43GB at the full 128K.
Performance
How fast Llama 3.3 70B Instruct runs
Single-user generation speed on popular GPUs that fit the model entirely in memory, at the best quantization each can hold.
RTX 6000 Ada
Q3_K_M · 48GB
~22tok/s
RTX PRO 6000
Q8_0 · 96GB
~19tok/s
RTX PRO 5000 72GB
Q6_K · 72GB
~18tok/s
RTX A6000
Q3_K_M · 48GB
~16tok/s
M5 Ultra 512GB
Q8_0 · 512GB
~8.9tok/s
M5 Ultra 256GB
Q8_0 · 256GB
~8.9tok/s
M3 Ultra 512GB
Q8_0 · 512GB
~6.0tok/s
M5 Max 128GB
Q8_0 · 128GB
~5.3tok/s
M4 Max 128GB
Q8_0 · 128GB
~4.7tok/s
DGX Spark
Q8_0 · 128GB
~2.7tok/s
Measured
Llama 3.3 70B Instruct benchmarks
- RTX PRO 5000 72GBQ4_K_XL · llama.cpp24 tok/s Hardware Corner: RTX Pro 5000 Blackwell LLM Performance
- 4× RTX 4090Q4_K_M · llama.cpp20 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- 2× RTX 4090Q4_K_M · llama.cpp19 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- RTX 6000 AdaQ4_K_M · llama.cpp19 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- 4× RTX 6000 AdaQ4_K_M · llama.cpp18 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- 4× RTX 3090Q4_K_M · llama.cpp17 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- 2× RTX 3090Q4_K_M · llama.cpp17 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- RTX A6000Q4_K_M · llama.cpp15 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- 4× RTX A6000Q4_K_M · llama.cpp14 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
- 4× RTX 4000 AdaQ4_K_M · llama.cpp7.4 tok/s XiongjieDai/GPU-Benchmarks-on-LLM-Inference (GitHub)
Overview
About Llama 3.3 70B Instruct
Llama 3.3 70B was Meta's last dense flagship and, for most of 2025, the model that justified buying a second 24GB GPU. It is still a capable writer and generalist, but 2026 releases such as Qwen3.8 27B and Qwen3.5 122B-A10B beat it on reasoning and coding while needing less memory or running much faster. At Q4_K_M it needs about 47GB with 8K context, so plan on 48GB of VRAM (two 24GB cards with a short context, or one 48GB workstation card) or a Mac with 64GB or more of unified memory. Generation is slow on Macs because every token reads all 70B parameters.
Strengths
- +Strong general writing and instruction following for a non-reasoning model
- +Dense weights quantize predictably; Q4_K_M holds up well
- +Excellent runtime support and a large ecosystem of fine-tunes
Weaknesses
- −Needs about 47GB at Q4_K_M, more than any single consumer GPU
- −Slow on bandwidth-limited hardware because all 70B parameters are read for every token
- −Outclassed on reasoning and coding by 2026 models a third of its size
- −Text only, with no reasoning mode
Setup
How to run Llama 3.3 70B Instruct
$ ollama run llama3.3:70b$ llama-server -hf bartowski/Llama-3.3-70B-Instruct-GGUF:Q4_K_M -c 8192 -ngl 99$ mlx_lm.chat --model mlx-community/Llama-3.3-70B-Instruct-4bitWeights: meta-llama/Llama-3.3-70B-Instruct · GGUF: bartowski/Llama-3.3-70B-Instruct-GGUF
Alternatives
Models like Llama 3.3 70B Instruct
Meta
Llama 4 Scout
109B (17B active) · 10.5M ctx
Alibaba Qwen
Qwen3.8 27B
27B · 256K ctx
Alibaba Qwen
Qwen3.5 122B-A10B
122B (10B active) · 256K ctx
OpenAI
gpt-oss-120b
117B (5.1B active) · 128K ctx
Frequently asked questions
How much VRAM do you need to run Llama 3.3 70B Instruct?
About 47GB at Q4_K_M with 8K tokens of context: 42.5GB of weights, 2.7GB of KV cache, and 1.9GB of runtime overhead. Higher-precision quants and longer contexts need more; see the table above.
What is the cheapest GPU that can run Llama 3.3 70B Instruct?
The smallest card in our database that runs it comfortably is the RTX A6000 (48GB), at Q3_K_M and ~16 tokens/sec.
Is Llama 3.3 70B Instruct free for commercial use?
Llama 3.3 70B Instruct is released under the Llama 3.3 Community License. Commercial use is allowed with some conditions; read the license before shipping a product.
How long a context does Llama 3.3 70B Instruct support?
Up to 131,072 tokens (128K). Long contexts need a lot of extra memory for the KV cache, so most local setups run 8K to 32K.
Sources: Llama 3.3 70B Instruct model card (Hugging Face), bartowski Llama 3.3 70B GGUF files. Capability score basis: Placed between the Llama 3.1 8B and Qwen3 32B anchors from its Artificial Analysis Intelligence Index v3 standing (October 2025).