all-MiniLM-L6-v2 on NVIDIA GeForce RTX 3060 12GB

NVIDIA GeForce RTX 3060 12GB runs all-MiniLM-L6-v2 excellently at FP16. This is a strong pairing.

Model Size

23M

Device VRAM

12 GB

Bandwidth

360 GB/s

Quantizations Tested

Performance by Quantization

Each row shows all-MiniLM-L6-v2 performance at a different quality level on NVIDIA GeForce RTX 3060 12GB.

Quantization	Speed	TTFT	Fits in VRAM	Rating	Confidence
FP16	—	5ms	✓ Yes	Excellent	estimated

Notes

FP16

23M params — negligible VRAM. Fastest embedding option for builders.

At 0.25GB, runs alongside any model with zero meaningful impact.

About all-MiniLM-L6-v2

all-MiniLM-L6-v2 (23M) is a embeddings, ai building model. Ultra-lightweight embedding model at 23M params. Fastest option for local embeddings. Lower quality than nomic-embed but practically free in VRAM. Ideal when running alongside large coding models and every MB of VRAM matters. The tiny model for builders who need RAG without the VRAM cost.

View all all-MiniLM-L6-v2 hardware options →