NVIDIA GeForce RTX 4070 Super

NVIDIA · 12GBGDDR6X · Can run 63 models

Manufacturer	NVIDIA
VRAM	12 GB
Memory Type	GDDR6X
Architecture	Ada Lovelace
CUDA Cores	7,168
Tensor Cores	224
Bandwidth	504 GB/s
TDP	220W
MSRP	$599
Released	Jan 17, 2024

AI Notes

The RTX 4070 Super is a strong mid-range option with 12GB VRAM and 504 GB/s bandwidth. It runs 7B models at high speed and can handle 13B models with Q4 quantization. Offers a good balance of performance and power efficiency for local AI inference.

Compatible Models

Model	Parameters	Best Quant	VRAM Used	Fit	Est. Speed
Qwen 3 0.6B	600M	Q4_K_M	2.5 GB	Runs	~202 tok/s
Qwen 3.5 0.8B	800M	Q4_K_M	1.5 GB	Runs	~336 tok/s
Gemma 3 1B	1B	Q8_0	2 GB	Runs	~252 tok/s
Llama 3.2 1B	1B	Q8_0	3 GB	Runs	~168 tok/s
DeepSeek R1 1.5B	1.5B	Q8_0	3 GB	Runs	~168 tok/s
SmolLM2 1.7B	1.7B	Q8_0	2.7 GB	Runs	~187 tok/s
Gemma 2 2B	2B	Q8_0	4 GB	Runs	~126 tok/s
Gemma 3n E2B	2B	Q4_K_M	3.3 GB	Runs	~153 tok/s
Gemma 4 E2B	2B	Q4_K_M	4 GB	Runs	~126 tok/s
Qwen 3.5 2B	2B	Q4_K_M	3 GB	Runs	~168 tok/s
Llama 3.2 3B	3B	Q8_0	5 GB	Runs	~101 tok/s
StarCoder2 3B	3B	Q4_K_M	3.5 GB	Runs	~144 tok/s
Phi-3 Mini 3.8B	3.8B	Q8_0	5.8 GB	Runs	~87 tok/s
Phi-4 Mini 3.8B	3.8B	Q4_K_M	4.5 GB	Runs	~112 tok/s
Gemma 3 4B	4B	Q4_K_M	5 GB	Runs	~101 tok/s
Gemma 3n E4B	4B	Q4_K_M	4.5 GB	Runs	~112 tok/s
Gemma 4 E4B	4B	Q4_K_M	6 GB	Runs	~84 tok/s
Qwen 3 4B	4B	Q4_K_M	4.5 GB	Runs	~112 tok/s
Qwen 3.5 4B	4B	Q4_K_M	4.5 GB	Runs	~112 tok/s
Yi 1.5 6B	6B	Q4_K_M	5 GB	Runs	~101 tok/s
Codestral Mamba 7B	7B	Q4_K_M	6.9 GB	Runs	~73 tok/s
DeepSeek R1 7B	7B	Q8_0	9 GB	Runs	~56 tok/s
Falcon 3 7B	7B	Q4_K_M	6.8 GB	Runs	~74 tok/s
InternLM 2.5 7B	7B	Q4_K_M	5.5 GB	Runs	~92 tok/s
Mistral 7B	7B	Q8_0	9 GB	Runs	~56 tok/s
OpenChat 3.5 7B	7B	Q4_K_M	6.9 GB	Runs	~73 tok/s
Qwen 2.5 7B	7B	Q8_0	9 GB	Runs	~56 tok/s
Qwen 2.5 Coder 7B	7B	Q8_0	9 GB	Runs	~56 tok/s
Qwen 2.5 VL 7B	7B	Q4_K_M	7 GB	Runs	~72 tok/s
StarCoder2 7B	7B	Q4_K_M	5.5 GB	Runs	~92 tok/s
WizardLM 2 7B	7B	Q4_K_M	6.9 GB	Runs	~73 tok/s
Aya Expanse 8B	8B	Q4_K_M	6.5 GB	Runs	~78 tok/s
Cogito 8B	8B	Q4_K_M	7.5 GB	Runs	~67 tok/s
DeepSeek R1 8B	8B	Q4_K_M	7.5 GB	Runs	~67 tok/s
Dolphin 3 8B	8B	Q4_K_M	6 GB	Runs	~84 tok/s
Granite 3.3 8B	8B	Q8_0	10 GB	Runs	~50 tok/s
Llama 3.1 8B	8B	Q8_0	10 GB	Runs	~50 tok/s
Nemotron 3 Nano 8B	8B	Q4_K_M	7.5 GB	Runs	~67 tok/s
Nous Hermes 2 8B	8B	Q4_K_M	6 GB	Runs	~84 tok/s
Qwen 3 8B	8B	Q4_K_M	7.5 GB	Runs	~67 tok/s
Qwen 3.5 9B	9B	Q4_K_M	7.5 GB	Runs	~67 tok/s
Yi 1.5 9B	9B	Q4_K_M	6.5 GB	Runs	~78 tok/s
Yi Coder 9B	9B	Q4_K_M	8 GB	Runs	~63 tok/s
Falcon 3 10B	10B	Q4_K_M	8.5 GB	Runs	~59 tok/s
Llama 3.2 Vision 11B	11B	Q4_K_M	8.5 GB	Runs	~59 tok/s
Mistral Nemo 12B	12B	Q4_K_M	9.5 GB	Runs	~53 tok/s
DeepSeek R1 14B	14B	Q4_K_M	9.9 GB	Runs	~51 tok/s
Phi-4 14B	14B	Q4_K_M	9.9 GB	Runs	~51 tok/s
Qwen 2.5 14B	14B	Q4_K_M	9.9 GB	Runs	~51 tok/s
Gemma 2 9B	9B	Q8_0	11 GB	Runs (tight)	~46 tok/s
Gemma 3 12B	12B	Q4_K_M	10.5 GB	Runs (tight)	~48 tok/s
Phi-4 Reasoning 14B	14B	Q4_K_M	11 GB	Runs (tight)	~46 tok/s
Qwen 2.5 Coder 14B	14B	Q4_K_M	12 GB	CPU Offload	~13 tok/s
Qwen 3 14B	14B	Q4_K_M	12 GB	CPU Offload	~13 tok/s
StarCoder2 15B	15B	Q8_0	17 GB	CPU Offload	~9 tok/s
InternLM 2.5 20B	20B	Q4_K_M	12 GB	CPU Offload	~13 tok/s
gpt-oss 20B	21B	MXFP4	15 GB	CPU Offload	~10 tok/s
Codestral 22B	22B	Q4_K_M	14.7 GB	CPU Offload	~10 tok/s
Devstral 24B	24B	Q4_K_M	17 GB	CPU Offload	~9 tok/s
Magistral Small 24B	24B	Q4_K_M	17 GB	CPU Offload	~9 tok/s
Mistral Small 3.1 24B	24B	Q4_K_M	18 GB	CPU Offload	~8 tok/s
Gemma 2 27B	27B	Q4_K_M	17.7 GB	CPU Offload	~8 tok/s
Qwen 3.5 35B A3B	35B	Q4_K_M	12 GB	CPU Offload	~13 tok/s

51 model(s) are too large for this hardware.