Qwen 2.5 Coder 32B Instruct on NVIDIA GeForce RTX 3090

NVIDIA GeForce RTX 3090 handles Qwen 2.5 Coder 32B Instruct well at 18 tok/s at Q4_K_M. A solid choice for this model.

Model Size

32.5B

Device VRAM

24 GB

Bandwidth

936 GB/s

Quantizations Tested

Performance by Quantization

Each row shows Qwen 2.5 Coder 32B Instruct performance at a different quality level on NVIDIA GeForce RTX 3090.

Quantization	Speed	TTFT	Fits in VRAM	Rating	Confidence
Q4_K_M	18 tok/s	450ms	✓ Yes	Good	estimated

Notes

Q4_K_M

Fits on the 3090 at Q4 with ~5.6GB headroom. Slightly slower than 4090 due to lower compute, but the 936 GB/s bandwidth is strong.

About Qwen 2.5 Coder 32B Instruct

Qwen 2.5 Coder 32B Instruct (32.5B) is a coding, ai coding, ai building model. The coding model that defines the builder workflow. Matches GPT-4 on HumanEval. This is what Cursor and Continue.dev users run locally when they want to eliminate API dependency. Apache 2.0 license. The cornerstone of the 'Full AI Builder' profile.

View all Qwen 2.5 Coder 32B Instruct hardware options →