QwenQwen3-32B Instruct on NVIDIA GeForce RTX 4090
Yes — RTX 4090 handles Qwen3-32B Instruct well at Q4_K_M — 30 tok/s. Solid daily-driver performance on 24 GB VRAM.
Model Size
32B
Device VRAM
24 GB
Bandwidth
1008 GB/s
Quantizations
2
Performance by Quantization
Each row is Qwen3-32B Instruct at a different quantization on NVIDIA GeForce RTX 4090: less precision usually means lower VRAM use and higher tokens per second, with a quality tradeoff. Pick a row that fits in 24 GB and matches how sensitive your workload is to quantization.
| Quantization | Speed | TTFT | Fits in VRAM | Rating | Confidence |
|---|---|---|---|---|---|
| Q5_K_M | 25 tok/s | 454ms | ✗ Offload | Good | estimated |
| Q4_K_M | 30 tok/s | – | ✓ Yes | Good | estimated |
Notes
Q5_K_M
Q5_K_M file is 23.2 GB, which with runtime overhead pushes past the 24 GB VRAM limit. Use Q4_K_M (19.8 GB, ~21.5 GB with overhead) for reliable full-VRAM operation on this card.
Q4_K_M
Q4_K_M fits comfortably in 24GB VRAM. Conservative estimate below external 32B Q4 results (33.8-38.9 tok/s) and community reports (~30 tok/s).
About Qwen3-32B Instruct
Qwen3-32B Instruct (32B) is a chat, coding, ai coding, reasoning, multi-purpose model. Qwen 3 dense 32B instruct for high-quality local inference where VRAM allows; 32K default and 128K max context. Apache 2.0.
View all Qwen3-32B Instruct hardware options →About NVIDIA GeForce RTX 4090
NVIDIA GeForce RTX 4090 has 24 GB at 1008 GB/s. Street price: $1,799.
See all models NVIDIA GeForce RTX 4090 can run →Builds with NVIDIA GeForce RTX 4090
Estimate method: Calibrated from measured 32B Q4 benchmarks on RTX 4090 and corroborated by community Q5_K_M reports. Reference hardware source: hardware-corner.net (2026-04-11)
Performance varies by driver version, inference engine, quantization method, context length, and system configuration. Figures shown are estimates based on community benchmarks and may not reflect your exact setup. Product names are trademarks of their respective owners. OwnRig is independent and not affiliated with any hardware or AI model provider.