AMD Radeon RX 7700 XT

AMD · 12GB GDDR6 · Can run 29 models

Buy Amazon
Manufacturer AMD
VRAM 12 GB
Memory Type GDDR6
Architecture RDNA 3
Stream Procs 3,456
Bandwidth 432 GB/s
TDP 245W
MSRP $449
Released Sep 6, 2023

AI Notes

The RX 7700 XT offers 12GB VRAM at a competitive price, handling 7B models well and running 13B with Q4 quantization. The 432 GB/s bandwidth provides moderate inference speeds. ROCm support requires additional setup compared to NVIDIA, but the extra VRAM over 8GB cards opens up more model options.

Compatible Models

Model Parameters Best Quant VRAM Used Fit Est. Speed
Gemma 3 1B 1B Q8_0 2 GB Runs ~216 tok/s
Llama 3.2 1B 1B Q8_0 3 GB Runs ~144 tok/s
DeepSeek R1 1.5B 1.5B Q8_0 3 GB Runs ~144 tok/s
Gemma 2 2B 2B Q8_0 4 GB Runs ~108 tok/s
Llama 3.2 3B 3B Q8_0 5 GB Runs ~86 tok/s
Phi-3 Mini 3.8B 3.8B Q8_0 5.8 GB Runs ~74 tok/s
Phi-4 Mini 3.8B 3.8B Q4_K_M 4.5 GB Runs ~96 tok/s
Gemma 3 4B 4B Q4_K_M 5 GB Runs ~86 tok/s
Qwen 3 4B 4B Q4_K_M 4.5 GB Runs ~96 tok/s
DeepSeek R1 7B 7B Q8_0 9 GB Runs ~48 tok/s
Mistral 7B 7B Q8_0 9 GB Runs ~48 tok/s
Qwen 2.5 7B 7B Q8_0 9 GB Runs ~48 tok/s
Qwen 2.5 Coder 7B 7B Q8_0 9 GB Runs ~48 tok/s
DeepSeek R1 8B 8B Q4_K_M 7.5 GB Runs ~58 tok/s
Llama 3.1 8B 8B Q8_0 10 GB Runs ~43 tok/s
Qwen 3 8B 8B Q4_K_M 7.5 GB Runs ~58 tok/s
Mistral Nemo 12B 12B Q4_K_M 9.5 GB Runs ~45 tok/s
DeepSeek R1 14B 14B Q4_K_M 9.9 GB Runs ~44 tok/s
Phi-4 14B 14B Q4_K_M 9.9 GB Runs ~44 tok/s
Qwen 2.5 14B 14B Q4_K_M 9.9 GB Runs ~44 tok/s
Gemma 2 9B 9B Q8_0 11 GB Runs (tight) ~39 tok/s
Gemma 3 12B 12B Q4_K_M 10.5 GB Runs (tight) ~41 tok/s
Qwen 2.5 Coder 14B 14B Q4_K_M 12 GB CPU Offload ~36 tok/s
Qwen 3 14B 14B Q4_K_M 12 GB CPU Offload ~36 tok/s
StarCoder2 15B 15B Q8_0 17 GB CPU Offload ~25 tok/s
Codestral 22B 22B Q4_K_M 14.7 GB CPU Offload ~29 tok/s
Devstral 24B 24B Q4_K_M 17 GB CPU Offload ~25 tok/s
Mistral Small 3.1 24B 24B Q4_K_M 18 GB CPU Offload ~24 tok/s
Gemma 2 27B 27B Q4_K_M 17.7 GB CPU Offload ~24 tok/s
14 model(s) are too large for this hardware.