Skip to content

Dolphin Mixtral 8x7B

by Cognitive Computations · dolphin family

47B

parameters

text-generation code-generation reasoning multilingual creative-writing

Dolphin Mixtral 8x7B is an uncensored, instruction-tuned mixture-of-experts model built on Mistral's Mixtral architecture. With 47 billion total parameters and approximately 12 billion active per inference, it delivers strong performance across text generation, coding, and multilingual tasks without alignment restrictions. This model combines the powerful MoE architecture of Mixtral with Dolphin's uncensored training approach, making it a popular choice for researchers and developers who need high-quality outputs without content filtering. It requires significant VRAM but rewards users with excellent reasoning and creative writing capabilities.

Quick Start with Ollama

ollama run 8x7b-q4_K_M
Resources Ollama
Creator Cognitive Computations
Parameters 47B
Architecture mixture-of-experts
Context 32K tokens
Released Jan 20, 2024
License Apache 2.0
Ollama dolphin-mixtral:8x7b

Quantization Options

Format File Size VRAM Required Quality Ollama Tag
Q4_K_M rec 24 GB 26 GB 8x7b-q4_K_M
Q8_0 47 GB 49 GB 8x7b-q8_0
F16 90 GB 94 GB 8x7b-fp16

Compatible Hardware

Q4_K_M requires 26 GB VRAM

Compatible Hardware

HardwareVRAMTypeFitEst. Speed
Mac Studio M4 Ultra 512GB512 GBmacRuns~32 tok/s
Mac Pro M2 Ultra 192GB192 GBmacRuns~31 tok/s
Mac Studio M4 Ultra 192GB192 GBmacRuns~32 tok/s
Mac Studio M4 Max 128GB128 GBmacRuns~21 tok/s
MacBook Pro M4 Max 128GB128 GBmacRuns~21 tok/s
MacBook Pro M5 Max 128GB128 GBmacRuns~21 tok/s
NVIDIA RTX PRO 6000 Blackwell96 GBgpuRuns~74 tok/s
MacBook Pro M3 Max 96GB96 GBmacRuns~15 tok/s
Mac mini M4 Pro 64GB64 GBmacRuns~11 tok/s
Mac Studio M4 Max 64GB64 GBmacRuns~21 tok/s
MacBook Pro M4 Max 64GB64 GBmacRuns~21 tok/s
MacBook Pro M5 Max 64GB64 GBmacRuns~21 tok/s
NVIDIA RTX 6000 Ada Generation48 GBgpuRuns~37 tok/s
NVIDIA RTX A600048 GBgpuRuns~30 tok/s
NVIDIA RTX PRO 5000 Blackwell48 GBgpuRuns~37 tok/s
Mac mini M4 Pro 48GB48 GBmacRuns~11 tok/s
MacBook Pro M3 Max 48GB48 GBmacRuns~15 tok/s
MacBook Pro M4 Max 48GB48 GBmacRuns~21 tok/s
MacBook Pro M4 Pro 48GB48 GBmacRuns~11 tok/s
MacBook Pro M5 Max 48GB48 GBmacRuns~16 tok/s
MacBook Pro M5 Pro 48GB48 GBmacRuns~11 tok/s
Mac Studio M4 Max 36GB36 GBmacRuns~21 tok/s
MacBook Pro M3 Pro 36GB36 GBmacRuns~6 tok/s
MacBook Pro M5 Max 36GB36 GBmacRuns~16 tok/s
NVIDIA RTX 5000 Ada Generation32 GBgpuRuns~28 tok/s
NVIDIA GeForce RTX 509032 GBgpuRuns~69 tok/s
iMac M4 32GB32 GBmacRuns~5 tok/s
Mac mini M4 32GB32 GBmacRuns~5 tok/s
MacBook Air M5 32GB32 GBmacRuns~5 tok/s
MacBook Air M4 32GB32 GBmacRuns~5 tok/s
MacBook Pro M5 32GB32 GBmacRuns~5 tok/s
AMD Radeon RX 7900 XTX24 GBgpuCPU Offload~11 tok/s
NVIDIA GeForce RTX 3090 Ti24 GBgpuCPU Offload~12 tok/s
NVIDIA GeForce RTX 309024 GBgpuCPU Offload~11 tok/s
NVIDIA GeForce RTX 409024 GBgpuCPU Offload~12 tok/s
NVIDIA RTX A500024 GBgpuCPU Offload~9 tok/s
iMac M3 24GB24 GBmacCPU Offload~1 tok/s
Mac mini M2 24GB24 GBmacCPU Offload~1 tok/s
Mac mini M4 Pro 24GB24 GBmacCPU Offload~3 tok/s
MacBook Air M2 24GB24 GBmacCPU Offload~1 tok/s
MacBook Air M4 24GB24 GBmacCPU Offload~2 tok/s
MacBook Air M5 24GB24 GBmacCPU Offload~2 tok/s
MacBook Pro M4 Pro 24GB24 GBmacCPU Offload~3 tok/s
MacBook Pro M5 24GB24 GBmacCPU Offload~2 tok/s
MacBook Pro M5 Pro 24GB24 GBmacCPU Offload~3 tok/s
AMD Radeon RX 7900 XT20 GBgpuCPU Offload~9 tok/s
NVIDIA RTX 4000 Ada Generation20 GBgpuCPU Offload~4 tok/s
MacBook Pro M3 Pro 18GB18 GBmacCPU Offload~2 tok/s
59 hardware device(s) cannot run this model at Q4_K_M.

Benchmark Scores

70.0
mmlu