Mistral Nemo 12B
by Mistral AI · mistral family
12B
parameters
text-generation code-generation reasoning multilingual tool-use summarization
Mistral Nemo 12B was built jointly by Mistral AI and NVIDIA. It features a 128K context window and uses a Tekken tokenizer that's more efficient across languages than prior Mistral models. With 3.4M+ Ollama pulls, it's one of the most popular models at its size. At Q4 it fits on 12 GB GPUs comfortably, making it a strong contender alongside Gemma 3 12B. Excellent at function calling, multilingual tasks, and general instruction following.
Quick Start with Ollama
ollama run 12b-instruct-q4_K_M | Creator | Mistral AI |
| Parameters | 12B |
| Architecture | transformer-decoder |
| Context | 128K tokens |
| Released | Jul 18, 2024 |
| License | Apache 2.0 |
| Ollama | mistral-nemo |
Quantization Options
| Format | File Size | VRAM Required | Quality | Ollama Tag |
|---|---|---|---|---|
| Q4_K_M rec | 7.1 GB | 9.5 GB |
★
★
★
★
★
| 12b-instruct-q4_K_M |
| Q8_0 | 12.9 GB | 16 GB |
★
★
★
★
★
| 12b-instruct-q8_0 |
| F16 | 24.5 GB | 28 GB |
★
★
★
★
★
| 12b-instruct-fp16 |
Compatible Hardware
for Q4_K_M (9.5 GB VRAM)
Compatible Hardware
| Hardware | VRAM | Type | Fit | Est. Speed |
|---|---|---|---|---|
| Mac Studio M4 Ultra 512GB | 512 GB | mac | Runs | ~86 tok/s |
| Mac Pro M2 Ultra 192GB | 192 GB | mac | Runs | ~84 tok/s |
| Mac Studio M4 Ultra 192GB | 192 GB | mac | Runs | ~86 tok/s |
| Mac Studio M4 Max 128GB | 128 GB | mac | Runs | ~57 tok/s |
| MacBook Pro M4 Max 128GB | 128 GB | mac | Runs | ~57 tok/s |
| MacBook Pro M3 Max 96GB | 96 GB | mac | Runs | ~42 tok/s |
| Mac mini M4 Pro 64GB | 64 GB | mac | Runs | ~29 tok/s |
| Mac Studio M4 Max 64GB | 64 GB | mac | Runs | ~57 tok/s |
| MacBook Pro M4 Max 64GB | 64 GB | mac | Runs | ~57 tok/s |
| Mac mini M4 Pro 48GB | 48 GB | mac | Runs | ~29 tok/s |
| MacBook Pro M3 Max 48GB | 48 GB | mac | Runs | ~42 tok/s |
| MacBook Pro M4 Max 48GB | 48 GB | mac | Runs | ~57 tok/s |
| MacBook Pro M4 Pro 48GB | 48 GB | mac | Runs | ~29 tok/s |
| Mac Studio M4 Max 36GB | 36 GB | mac | Runs | ~57 tok/s |
| MacBook Pro M3 Pro 36GB | 36 GB | mac | Runs | ~16 tok/s |
| NVIDIA GeForce RTX 5090 | 32 GB | gpu | Runs | ~189 tok/s |
| iMac M4 32GB | 32 GB | mac | Runs | ~13 tok/s |
| Mac mini M4 32GB | 32 GB | mac | Runs | ~13 tok/s |
| MacBook Air M4 32GB | 32 GB | mac | Runs | ~13 tok/s |
| AMD Radeon RX 7900 XTX | 24 GB | gpu | Runs | ~101 tok/s |
| NVIDIA GeForce RTX 3090 | 24 GB | gpu | Runs | ~99 tok/s |
| NVIDIA GeForce RTX 4090 | 24 GB | gpu | Runs | ~106 tok/s |
| iMac M3 24GB | 24 GB | mac | Runs | ~11 tok/s |
| Mac mini M2 24GB | 24 GB | mac | Runs | ~11 tok/s |
| Mac mini M4 Pro 24GB | 24 GB | mac | Runs | ~29 tok/s |
| MacBook Air M2 24GB | 24 GB | mac | Runs | ~11 tok/s |
| MacBook Air M4 24GB | 24 GB | mac | Runs | ~13 tok/s |
| MacBook Pro M4 Pro 24GB | 24 GB | mac | Runs | ~29 tok/s |
| AMD Radeon RX 7900 XT | 20 GB | gpu | Runs | ~84 tok/s |
| MacBook Pro M3 Pro 18GB | 18 GB | mac | Runs | ~16 tok/s |
| AMD Radeon RX 6800 XT | 16 GB | gpu | Runs | ~54 tok/s |
| AMD Radeon RX 7800 XT | 16 GB | gpu | Runs | ~66 tok/s |
| Intel Arc A770 | 16 GB | gpu | Runs | ~59 tok/s |
| NVIDIA GeForce RTX 4060 Ti 16GB | 16 GB | gpu | Runs | ~30 tok/s |
| NVIDIA GeForce RTX 4070 Ti Super | 16 GB | gpu | Runs | ~71 tok/s |
| NVIDIA GeForce RTX 4080 Super | 16 GB | gpu | Runs | ~77 tok/s |
| NVIDIA GeForce RTX 4080 | 16 GB | gpu | Runs | ~75 tok/s |
| NVIDIA GeForce RTX 5070 Ti | 16 GB | gpu | Runs | ~94 tok/s |
| NVIDIA GeForce RTX 5080 | 16 GB | gpu | Runs | ~101 tok/s |
| iMac M1 16GB | 16 GB | mac | Runs | ~7 tok/s |
| iMac M4 16GB | 16 GB | mac | Runs | ~13 tok/s |
| Mac mini M1 16GB | 16 GB | mac | Runs | ~7 tok/s |
| Mac mini M4 16GB | 16 GB | mac | Runs | ~13 tok/s |
| MacBook Air M2 16GB | 16 GB | mac | Runs | ~11 tok/s |
| MacBook Air M3 16GB | 16 GB | mac | Runs | ~11 tok/s |
| MacBook Air M4 16GB | 16 GB | mac | Runs | ~13 tok/s |
| MacBook Pro M1 16GB | 16 GB | mac | Runs | ~7 tok/s |
| MacBook Pro M2 Pro 16GB | 16 GB | mac | Runs | ~21 tok/s |
| AMD Radeon RX 7700 XT | 12 GB | gpu | Runs | ~45 tok/s |
| NVIDIA GeForce RTX 3060 12GB | 12 GB | gpu | Runs | ~38 tok/s |
| NVIDIA GeForce RTX 3080 12GB | 12 GB | gpu | Runs | ~96 tok/s |
| NVIDIA GeForce RTX 4070 Super | 12 GB | gpu | Runs | ~53 tok/s |
| NVIDIA GeForce RTX 4070 Ti | 12 GB | gpu | Runs | ~53 tok/s |
| NVIDIA GeForce RTX 4070 | 12 GB | gpu | Runs | ~53 tok/s |
| NVIDIA GeForce RTX 5070 | 12 GB | gpu | Runs | ~71 tok/s |
| NVIDIA GeForce RTX 2080 Ti | 11 GB | gpu | Runs (tight) | ~65 tok/s |
| NVIDIA GeForce RTX 3080 10GB | 10 GB | gpu | Runs (tight) | ~80 tok/s |
| AMD Radeon RX 7600 | 8 GB | gpu | CPU Offload | ~30 tok/s |
| Intel Arc A750 | 8 GB | gpu | CPU Offload | ~54 tok/s |
| NVIDIA GeForce RTX 3060 Ti | 8 GB | gpu | CPU Offload | ~47 tok/s |
| NVIDIA GeForce RTX 3070 | 8 GB | gpu | CPU Offload | ~47 tok/s |
| NVIDIA GeForce RTX 4060 Ti 8GB | 8 GB | gpu | CPU Offload | ~30 tok/s |
| NVIDIA GeForce RTX 4060 | 8 GB | gpu | CPU Offload | ~29 tok/s |
| MacBook Air M1 8GB | 8 GB | mac | CPU Offload | ~7 tok/s |
| MacBook Air M2 8GB | 8 GB | mac | CPU Offload | ~11 tok/s |
Benchmark Scores
68.0
mmlu