InternLM · Apache 2.0
Shanghai AI Lab's efficient 7B model with strong coding and reasoning.
InternLM 2.5 7B Chat (7.74B) requires 6.7 GB VRAM at recommended quality (Q6_K). At efficient quality (Q4_K_M), it fits in 4.5 GB VRAM, making it compatible with the NVIDIA GeForce RTX 3060 12GB. On NVIDIA GeForce RTX 4090, expect approximately 88 tok/s at Q8_0. For the best experience, Starter AI Desktop ($582) is recommended.
— OwnRig methodology, data updated 2026-03-15
| Quality | Quantization | VRAM | File Size |
|---|---|---|---|
| full | Q8_0 | 8.6 GB | 7.6 GB |
| recommended | Q6_K | 6.7 GB | 5.9 GB |
| recommended | Q5_K_M | 5.6 GB | 4.9 GB |
| efficient | Q4_K_M | 4.5 GB | 3.9 GB |
KV cache VRAM at Q6_K quality. Longer context = more memory.
| Context | KV Cache | Total VRAM |
|---|---|---|
| 2K | 102 MB | 6.8 GB |
| 4K | 307 MB | 7 GB |
| 8K | 512 MB | 7.2 GB |
| 16K | 1 GB | 7.7 GB |
| 32K | 2 GB | 8.7 GB |
Performance data for InternLM 2.5 7B Chat across different hardware.
| Device | Quantization | Speed | Rating | Fits in VRAM |
|---|---|---|---|---|
| NVIDIA GeForce RTX 3060 12GB | Q5_K_M | 35 tok/s | Good | ✓ |
| NVIDIA GeForce RTX 4090 | Q8_0 | 88 tok/s | Excellent | ✓ |
| NVIDIA GeForce RTX 4060 8GB | Q4_K_M | 30 tok/s | Good | ✓ |
| NVIDIA GeForce RTX 4070 Ti 12GB | Q5_K_M | 46 tok/s | Excellent | ✓ |
| NVIDIA GeForce RTX 3080 10GB | Q5_K_M | 50 tok/s | Excellent | ✓ |
| Apple M3 Pro (18GB Unified) | Q4_K_M | 15 tok/s | Acceptable | ✓ |
InternLM 2.5 7B Chat is commonly used with Cursor, Continue, Aider, Open WebUI, LM Studio. For an AI coding workflow, pair it with an embedding model like nomic-embed-text for local RAG.
Data confidence: estimated. Last updated: 2026-03-15. Source