How much VRAM does LLaVA 1.6 13B need?

LLaVA 1.6 13B requires 9.1 GB VRAM at recommended quality (Q5_K_M). At lower quality settings, it can fit in as little as 6.2 GB.

What is the best GPU for LLaVA 1.6 13B?

The NVIDIA Grace Blackwell Ultra GB300 delivers the best performance for LLaVA 1.6 13B, achieving 270 tok/s at Q5_K_M with an excellent rating.

Can I run LLaVA 1.6 13B on an RTX 4060 Ti?

Yes. On the NVIDIA GeForce RTX 4060 Ti 16GB, LLaVA 1.6 13B runs at 22 tok/s (Q4_K_M, good).

What quantization should I use for LLaVA 1.6 13B?

For the best quality, use Q5_K_M (9.1 GB VRAM). If your GPU has limited VRAM, Q3_K_M (6.2 GB) is the most efficient option with acceptable quality.

ChatReasoningMulti-purpose13B

Chat

LLaVA 1.6 13B

LLaVA · Llama 2 Community License

Multimodal model that processes images and text together. Built on Vicuna 13B with a vision encoder. Can analyze screenshots, diagrams, and photos. Useful for builders who need to process visual content in their workflows.

Parameters: 13B
Architecture: Dense
Context: 4,096 tokens
Released: 2024-01-30
Engines: llama.cpp, ollama

Parameters

13B

VRAM

9.1 GB

Context

Formats

GPUs

LLaVA 1.6 13B (13B) requires 9.1 GB VRAM at recommended quality (Q5_K_M). At efficient quality (Q4_K_M), it fits in 7.7 GB VRAM, making it compatible with the NVIDIA RTX 4060 Laptop (40-60W). On NVIDIA Grace Blackwell Ultra GB300, expect approximately 270 tok/s at Q5_K_M. For the best experience, Starter AI Desktop ($582) is recommended.

Source: OwnRig methodology

VRAM (Recommended)

9.1 GB

Quantization

Q5_K_M

File Size

7.8 GB

Max Context

4K tokens

Primary Use

Chat

Memory

VRAM Requirements

Quality	Quantization	VRAM	File Size
recommended	Q5_K_M	9.1 GB	7.8 GB
efficient	Q4_K_M	7.7 GB	6.5 GB
compressed	Q3_K_M	6.2 GB	5.1 GB

Scaling

Context Length Impact

KV cache VRAM at Q5_K_M quality. Longer context = more memory.

Context	KV Cache	Total VRAM
2K	205 MB	9.3 GB
4K	410 MB	9.5 GB

Compatible GPUs

21 devices


NVIDIA Grace Blackwell Ultra GB300	Q5_K_M	270 tok/s	Excellent
NVIDIA RTX PRO 6000 Blackwell	Q5_K_M	124 tok/s	Excellent
NVIDIA RTX PRO 6000 Blackwell Max-Q	Q5_K_M	114 tok/s	Excellent
NVIDIA GeForce RTX 4060 Ti 16GB	Q4_K_M	22 tok/s	Good
NVIDIA GeForce RTX 4070 Ti 12GB	Q4_K_M	28 tok/s	Good
NVIDIA GeForce RTX 4090	Q5_K_M	30 tok/s	Good
AMD Radeon RX 7900 XTX	Q5_K_M	26 tok/s	Good
AMD Radeon Pro W7900	Q5_K_M	32 tok/s	Good
AMD Radeon RX 9070	Q4_K_M	38 tok/s	Good
NVIDIA GeForce RTX 5060 Ti 16GB	Q4_K_M	25 tok/s	Good
Apple M3 Pro (18GB Unified)	Q4_K_M	8 tok/s	Acceptable
NVIDIA GeForce RTX 3080 10GB	Q3_K_M	18 tok/s	Acceptable
NVIDIA RTX 4060 Laptop (40-60W)	Q3_K_M	13 tok/s	Acceptable
NVIDIA RTX 4070 Laptop (80-115W)	Q3_K_M	15 tok/s	Acceptable
NVIDIA RTX 4080 Laptop (120-150W)	Q4_K_M	20 tok/s	Acceptable
NVIDIA RTX 4090 Laptop (150-175W)	Q4_K_M	19 tok/s	Acceptable
AMD Radeon RX 9060 XT 16GB	Q4_K_M	19 tok/s	Acceptable
AMD Radeon RX 9060 XT 8GB	Q4_K_M	19 tok/s	Acceptable
NVIDIA GeForce RTX 4060 8GB	Q3_K_M	22 tok/s	Marginal
AMD Radeon RX 7600	Q3_K_M	17 tok/s	Marginal
NVIDIA GeForce RTX 5060 8GB	Q3_K_M	25 tok/s	Marginal

Showing 21 of 21 entries

Hardware

Recommended Builds

Complete PC builds that can run LLaVA 1.6 13B.

Extreme

Extreme AI Workstation

Dual GPUs that run the biggest AI models at a smart price

2x NVIDIA GeForce RTX 3090 (Used)·48 GBVRAM

Runs 8 models

$3,972

High-end

High-End AI Workstation

Chat, generate images, and code with AI, all at once

RTX 4090·24 GBVRAM

Runs 8 models

$3,433

FAQ

Frequently Asked Questions

How much VRAM does LLaVA 1.6 13B need?: LLaVA 1.6 13B requires 9.1 GB VRAM at recommended quality (Q5_K_M). At lower quality settings, it can fit in as little as 6.2 GB.
What is the best GPU for LLaVA 1.6 13B?: The NVIDIA Grace Blackwell Ultra GB300 delivers the best performance for LLaVA 1.6 13B, achieving 270 tok/s at Q5_K_M with an excellent rating.
Can I run LLaVA 1.6 13B on an RTX 4060 Ti?: Yes. On the NVIDIA GeForce RTX 4060 Ti 16GB, LLaVA 1.6 13B runs at 22 tok/s (Q4_K_M, good).
What quantization should I use for LLaVA 1.6 13B?: For the best quality, use Q5_K_M (9.1 GB VRAM). If your GPU has limited VRAM, Q3_K_M (6.2 GB) is the most efficient option with acceptable quality.

All models

Data confidence: estimated. Source

VRAM requirements are calculated from model parameters and may vary by inference engine, context length, and batch size. Performance estimates are based on community benchmarks and should be verified for your specific configuration.LLaVA is a trademark of its respective owner. OwnRig is not affiliated with or endorsed by the model creator.