How much VRAM does all-MiniLM-L6-v2 need?

all-MiniLM-L6-v2 requires 256 MB VRAM at recommended quality (FP16).

What is the best GPU for all-MiniLM-L6-v2?

The NVIDIA GeForce RTX 4070 Ti 12GB delivers the best performance for all-MiniLM-L6-v2, achieving 12000 tok/s at FP16 with an excellent rating.

EmbeddingsAI building23M

Embeddings

all-MiniLM-L6-v2

MiniLM · Apache 2.0

Ultra-lightweight embedding model at 23M params. Fastest option for local embeddings. Lower quality than nomic-embed but practically free in VRAM. Ideal when running alongside large coding models and every MB of VRAM matters. The tiny model for builders who need RAG without the VRAM cost.

Parameters: 23M
Architecture: Dense
Context: 512 tokens
Released: 2021-08-12
Engines: ollama, sentence-transformers
Builder Tools: AnythingLLM, Open WebUI

Parameters

23M

VRAM

256 MB

Context

Formats

GPUs

all-MiniLM-L6-v2 (23M) requires 256 MB VRAM at recommended quality (FP16). On NVIDIA GeForce RTX 4070 Ti 12GB, expect approximately 12000 tok/s at FP16. For the best experience, Starter AI Desktop ($582) is recommended.

Source: OwnRig methodology

VRAM (Recommended)

256 MB

Quantization

FP16

File Size

0.09 GB

Max Context

1K tokens

Primary Use

Embeddings

Compatible GPUs

19 devices


NVIDIA Grace Blackwell Ultra GB300	FP16	3000 tok/s	Excellent
NVIDIA GeForce RTX 3060 12GB	FP16	–	Excellent
NVIDIA GeForce RTX 3080 10GB	FP16	5000 tok/s	Excellent
NVIDIA GeForce RTX 4060 8GB	FP16	8500 tok/s	Excellent
NVIDIA RTX 4060 Laptop (40-60W)	FP16	5100 tok/s	Excellent
NVIDIA RTX 4070 Laptop (80-115W)	FP16	5950 tok/s	Excellent
NVIDIA GeForce RTX 4070 Ti 12GB	FP16	12000 tok/s	Excellent
NVIDIA RTX 4080 Laptop (120-150W)	FP16	8400 tok/s	Excellent
NVIDIA GeForce RTX 4090	FP16	–	Excellent
AMD Radeon RX 7600	FP16	500 tok/s	Excellent
AMD Radeon RX 7900 XTX	FP16	–	Excellent
AMD Radeon Pro W7900	FP16	–	Excellent
NVIDIA RTX PRO 6000 Blackwell	FP16	3000 tok/s	Excellent
NVIDIA RTX PRO 6000 Blackwell Max-Q	FP16	2760 tok/s	Excellent
AMD Radeon RX 9070	FP16	–	Excellent
AMD Radeon RX 9060 XT 16GB	FP16	–	Excellent
AMD Radeon RX 9060 XT 8GB	FP16	–	Excellent
NVIDIA GeForce RTX 5060 8GB	FP16	9775 tok/s	Excellent
Apple M3 Pro (18GB Unified)	FP16	1200 tok/s	Good

Showing 19 of 19 entries

Builder Context

all-MiniLM-L6-v2 is commonly used with AnythingLLM, Open WebUI.

Hardware

Recommended Builds

Complete PC builds that can run all-MiniLM-L6-v2.

Mid-range

AI Builder Workstation

Run every AI tool you need. Nothing leaves your machine

RTX 4090·24 GBVRAM

Runs 10 models

$2,773

FAQ

Frequently Asked Questions

How much VRAM does all-MiniLM-L6-v2 need?: all-MiniLM-L6-v2 requires 256 MB VRAM at recommended quality (FP16).
What is the best GPU for all-MiniLM-L6-v2?: The NVIDIA GeForce RTX 4070 Ti 12GB delivers the best performance for all-MiniLM-L6-v2, achieving 12000 tok/s at FP16 with an excellent rating.

All models

Data confidence: verified. Source

VRAM requirements are calculated from model parameters and may vary by inference engine, context length, and batch size. Performance estimates are based on community benchmarks and should be verified for your specific configuration.MiniLM is a trademark of its respective owner. OwnRig is not affiliated with or endorsed by the model creator.