Mixtral 8x7B

Mixtral 8x7B is Mistral AI's mixture-of-experts model, utilizing eight expert networks of 7B parameters each with a routing mechanism that activates two experts per token. This architecture gives it 47B total parameters but only uses about 13B during inference, providing excellent efficiency. The model delivers performance competitive with much larger dense models while maintaining faster inference speeds. It excels at reasoning, multilingual tasks, and code generation, and is particularly well-suited for users who need high-quality output with reasonable hardware requirements.

Compatible Hardware for Q4_K_M

Showing compatibility for the recommended quantization (Q4_K_M, 29.7 GB VRAM).