Vision · 1 of 6

Florence-2 base

Hugging Face — microsoft/Florence-2-base

0.46 GB of weights on Hugging Face

A tiny vision model with named tasks. You give it a task tag — caption, detect, read, segment — and it does that one job.

Overview

Microsoft’s 0.23B-parameter vision foundation model. It is not a chatbot that sees images. It runs one task at a time, chosen with a prompt tag: <OD> for detection, <OCR> for reading, <CAPTION> for describing, <SEGMENT> for regions. MIT licensed, so the weights are yours.

Params: 0.23B
License: MIT
Input: image + task prompt
Output: text, boxes, masks
Hardware: runs on CPU; a GPU makes it quick
Download: 0.46 GB

What it is good at

Captioning a photo
Detecting objects as boxes or points
Reading text from an image
Segmenting regions
Labeling a dataset before you train

Not good at: open-ended conversation about an image, long documents, or fine world knowledge. It is a captioner and a labeler, not an analyst.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~463 MB
int8	1 byte per param	~232 MB
int4	0.5 bytes per param	~116 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Florence-2-base", trust_remote_code=True, dtype="auto"
)
processor = AutoProcessor.from_pretrained(
    "microsoft/Florence-2-base", trust_remote_code=True
)

image = Image.open("photo.jpg")
inputs = processor(image, text="<CAPTION>", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=64)
print(processor.decode(out[0], skip_special_tokens=True))

Vision · 2 of 6

SmolVLM2 500M

Hugging Face — HuggingFaceTB/SmolVLM2-500M-Video-Instruct

2.03 GB of weights on Hugging Face

Half a billion parameters that watch a clip and answer questions about it.

Overview

Hugging Face’s 0.51B open vision-language model, trained for video as well as stills. Give it frames and a question and it answers in text. Apache 2.0 licensed, so commercial use needs no permission.

Params: 0.51B
License: Apache 2.0
Input: image or frames + text
Output: text
Hardware: runs on CPU; a GPU helps with video
Download: 2.03 GB

What it is good at

Question answering over a short clip
Captioning video frames
Image question answering
Multimodal experiments on a laptop
A small baseline you can actually retrain

Not good at: high-resolution documents, charts with fine detail, or long conversational context.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.0 GB
int8	1 byte per param	~507 MB
int4	0.5 bytes per param	~254 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoProcessor, AutoModelForImageTextToText
from PIL import Image

repo = "HuggingFaceTB/SmolVLM2-500M-Video-Instruct"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype="auto")

messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": "What is happening here?"},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, images=[Image.open("photo.jpg")],
    return_tensors="pt",
)
print(model.generate(**inputs, max_new_tokens=64))

Vision · 3 of 6

Florence-2 large

Hugging Face — microsoft/Florence-2-large

1.55 GB of weights on Hugging Face

The larger half of Florence-2. Same task tags, more detail in the answer.

Overview

Microsoft’s 0.78B-parameter vision model. It keeps the prompt-tag interface of Florence-2 base and spends the extra parameters on detail: fuller captions, tighter boxes. MIT licensed.

Params: 0.78B
License: MIT
Input: image + task prompt
Output: text, boxes, masks
Hardware: CPU works; a GPU is faster
Download: 1.55 GB

What it is good at

Higher-detail captions
Detection on crowded images
Reading dense text
Image region segmentation
Dataset labeling where base is too coarse

Not good at: conversation, reasoning over several images, or video. It is the same job as base, done better.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.6 GB
int8	1 byte per param	~777 MB
int4	0.5 bytes per param	~388 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Florence-2-large", trust_remote_code=True, dtype="auto"
)
processor = AutoProcessor.from_pretrained(
    "microsoft/Florence-2-large", trust_remote_code=True
)

image = Image.open("photo.jpg")
inputs = processor(image, text="<DETAILED_CAPTION>", return_tensors="pt")
out = model.generate(**inputs, max_new_tokens=96)
print(processor.decode(out[0], skip_special_tokens=True))

Vision · 4 of 6

Moondream 2

Hugging Face — vikhyatk/moondream2

3.86 GB of weights on Hugging Face

A small vision-language model. It looks at an image and tells you what is in it — on a laptop CPU.

Overview

A 1.93B-parameter vision-language model. It looks at an image and answers questions about it: what is in it, where things are, what is written on it. Small enough to run on a laptop CPU. Apache 2.0 licensed, so the weights are yours.

Params: 1.93B
License: Apache 2.0
Input: image + text
Output: text
Hardware: runs on CPU; GPU makes it fast
Download: 3.86 GB

What it is good at

Image captioning
Visual questions on a single image
Object detection with points or boxes
Labeling images for a dataset
Reading text from images

Not good at: long documents, charts at scale, or fine world knowledge. It is a captioner and a labeler, not an analyst.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~3.9 GB
int8	1 byte per param	~1.9 GB
int4	0.5 bytes per param	~964 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers einops

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "vikhyatk/moondream2", trust_remote_code=True, dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained("vikhyatk/moondream2", trust_remote_code=True)

image = Image.open("photo.jpg")
print(model.caption(image, length="short")["caption"])
print(model.query(image, "How many people?")["answer"])

Vision · 5 of 6

Qwen3-VL 2B

Hugging Face — Qwen/Qwen3-VL-2B-Instruct

4.26 GB of weights on Hugging Face

Reads documents, charts, and screens as well as photos, and thinks before it answers.

Overview

Qwen’s 2.13B-parameter vision-language model, tuned for documents and screens rather than only natural images. It can reason step by step before answering. Apache 2.0 licensed.

Params: 2.13B
License: Apache 2.0
Input: image + text
Output: text
Hardware: GPU recommended; CPU is slow
Download: 4.26 GB

What it is good at

Document and chart questions
Screenshot and interface understanding
General image question answering
Counting and spatial questions
Multilingual text inside images

Not good at: very long video, or images larger than it can hold in one pass.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~4.3 GB
int8	1 byte per param	~2.1 GB
int4	0.5 bytes per param	~1.1 GB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoProcessor, AutoModelForImageTextToText

repo = "Qwen/Qwen3-VL-2B-Instruct"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype="auto")

messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": "Read the table in this screenshot."},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, images=[Image.open("shot.png")],
    return_tensors="pt",
)
print(model.generate(**inputs, max_new_tokens=128))

Vision · 6 of 6

Granite Vision 3.2

Hugging Face — ibm-granite/granite-vision-3.2-2b

5.95 GB of weights on Hugging Face

IBM’s document reader. It was built for forms, tables, and scanned pages.

Overview

IBM’s 2.98B-parameter vision-language model, trained for documents: forms, invoices, tables, scanned pages. Apache 2.0 licensed, with the weights published for commercial use.

Params: 2.98B
License: Apache 2.0
Input: document image + text
Output: text
Hardware: GPU recommended
Download: 5.95 GB

What it is good at

Document question answering
Extracting tables from scans
OCR on messy layouts
Form field extraction
Enterprise document pipelines

Not good at: open-ended chat, artistic images, or video. Point it at paperwork.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~6.0 GB
int8	1 byte per param	~3.0 GB
int4	0.5 bytes per param	~1.5 GB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoProcessor, AutoModelForImageTextToText

repo = "ibm-granite/granite-vision-3.2-2b"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForImageTextToText.from_pretrained(repo, dtype="auto")

messages = [{
    "role": "user",
    "content": [
        {"type": "image"},
        {"type": "text", "text": "What is the invoice total?"},
    ],
}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, images=[Image.open("scan.png")],
    return_tensors="pt",
)
print(model.generate(**inputs, max_new_tokens=96))

STT · 1 of 6

Moonshine Tiny

Hugging Face — moonshine-ai/moonshine-tiny

0.11 GB of weights on Hugging Face

27 million parameters turning speech into text far faster than real time.

Overview

Useful Sensors’ smallest speech recognition model. It transcribes English audio quickly enough to run live on modest hardware. MIT licensed, so the weights are yours.

Params: 27.1M
License: MIT
Input: audio
Output: text
Speed: RTFx 753 — about 750× real time
Download: 0.11 GB

What it is good at

Live transcription
Voice commands on a device
Low-power and embedded hardware
English audio
Large batch jobs on a CPU

Not good at: heavy accents, noisy rooms, or languages other than English. When accuracy matters, move up the shelf.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~54 MB
int8	1 byte per param	~27 MB
int4	0.5 bytes per param	~14 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition", model="moonshine-ai/moonshine-tiny"
)
print(transcriber("audio.wav")["text"])

STT · 2 of 6

Whisper tiny.en

Hugging Face — openai/whisper-tiny.en

0.15 GB of weights on Hugging Face

OpenAI’s smallest transcription model, English-only and easy to run anywhere.

Overview

Whisper at 38M parameters, trained for English only. Dropping the other languages buys accuracy at this size. Apache 2.0 licensed.

Params: 37.8M
License: Apache 2.0
Input: audio
Output: text
Speed: RTFx 348
Download: 0.15 GB

What it is good at

English transcription
Podcast and meeting capture
Offline captioning
Batch jobs on a CPU
A first model to benchmark against

Not good at: other languages — use the multilingual Whisper sizes for those — or noisy audio.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~76 MB
int8	1 byte per param	~38 MB
int4	0.5 bytes per param	~19 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition", model="openai/whisper-tiny.en"
)
print(transcriber("audio.wav")["text"])

STT · 3 of 6

Moonshine Base

Hugging Face — moonshine-ai/moonshine-base

0.25 GB of weights on Hugging Face

Moonshine’s larger size: noticeably more accurate, still ahead of real time.

Overview

The 62M version of Moonshine. It keeps the speed advantage over Whisper at this size while cutting the error rate. MIT licensed.

Params: 61.5M
License: MIT
Input: audio
Output: text
Speed: RTFx 566
Download: 0.25 GB

What it is good at

English transcription at speed
Voice assistants
Low-latency captioning
CPU-only servers
Realtime pipelines

Not good at: languages other than English, or audio with heavy background noise.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~123 MB
int8	1 byte per param	~62 MB
int4	0.5 bytes per param	~31 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition", model="moonshine-ai/moonshine-base"
)
print(transcriber("audio.wav")["text"])

STT · 4 of 6

Whisper base.en

Hugging Face — openai/whisper-base.en

0.29 GB of weights on Hugging Face

A comfortable middle point in the Whisper family: English-only, cheap to run.

Overview

Whisper at 73M parameters, English only. It sits where many people stop: clearly better than tiny, still fast on a laptop CPU. Apache 2.0 licensed.

Params: 72.6M
License: Apache 2.0
Input: audio
Output: text
Speed: RTFx 321
Download: 0.29 GB

What it is good at

English transcription
Meeting and interview notes
Caption generation
CPU batch jobs
A practical default for English-only work

Not good at: multilingual use, or matching larger sizes on difficult audio.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~145 MB
int8	1 byte per param	~73 MB
int4	0.5 bytes per param	~36 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition", model="openai/whisper-base.en"
)
print(transcriber("audio.wav")["text"])

STT · 5 of 6

Distil-Whisper small.en

Hugging Face — distil-whisper/distil-small.en

1.00 GB of weights on Hugging Face

Whisper small, distilled down to about a third of the layers.

Overview

Distil-Whisper compresses Whisper small to 166M parameters by keeping two encoder layers for every three in the original. Same task, same interface, faster. MIT licensed.

Params: 0.17B
License: MIT
Input: audio
Output: text
Speed: RTFx 332
Download: 1.00 GB

What it is good at

English transcription at a better error rate
Long-form audio in a single pass
Cost-sensitive batch work
Dropping latency without a bigger model
A strong midpoint on this shelf

Not good at: other languages — this one is English only.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~332 MB
int8	1 byte per param	~166 MB
int4	0.5 bytes per param	~83 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition", model="distil-whisper/distil-small.en"
)
print(transcriber("audio.wav")["text"])

STT · 6 of 6

Whisper large-v3-turbo

Hugging Face — openai/whisper-large-v3-turbo

1.62 GB of weights on Hugging Face

The Whisper you reach for when accuracy matters more than latency.

Overview

OpenAI’s large-v3 distilled to four decoder layers instead of 32. It keeps most of the accuracy at a fraction of the speed cost, and it is multilingual. MIT licensed.

Params: 0.81B
License: MIT
Input: audio, many languages
Output: text
Speed: RTFx 200
Download: 1.62 GB

What it is good at

Multilingual transcription
General-purpose English accuracy
Meeting and interview capture
Server-side batch work
The top of an accuracy ladder

Not good at: realtime on very small hardware. For live audio, move down the shelf.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.6 GB
int8	1 byte per param	~809 MB
int4	0.5 bytes per param	~404 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

transcriber = pipeline(
    "automatic-speech-recognition", model="openai/whisper-large-v3-turbo"
)
print(transcriber("audio.wav")["text"])

TTS · 1 of 6

Kokoro 82M

Hugging Face — hexgrad/Kokoro-82M

0.33 GB of weights on Hugging Face

82 million parameters that speak clearly, in a browser or on a laptop.

Overview

Kokoro keeps the core model tiny and puts the voice in a separate style model, which is how it sounds this good at this size. Apache 2.0 licensed, so you can ship it.

Params: 82M
License: Apache 2.0
Input: text
Output: audio, 24 kHz
Voices: multiple, via a style model
Download: 0.33 GB

What it is good at

Reading text aloud
Voiceovers and narration
On-device speech
Batch audio generation
Prototyping a voice product cheaply

Not good at: cloning an arbitrary voice from a short sample, or expressive acting.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~164 MB
int8	1 byte per param	~82 MB
int4	0.5 bytes per param	~41 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install kokoro soundfile

from kokoro import KPipeline

pipeline = KPipeline(lang_code="a")  # American English
for _, _, audio in pipeline("Talachu runs small models on ordinary hardware."):
    pass
import soundfile as sf
sf.write("out.wav", audio, 24000)

TTS · 2 of 6

CosyVoice2 0.5B

Hugging Face — FunAudioLLM/CosyVoice2-0.5B

0.99 GB of weights on Hugging Face

Alibaba’s conversational voice model, with published numbers in two languages.

Overview

CosyVoice2 is a 0.5B text-to-speech model built for natural, conversational delivery and voice cloning. Apache 2.0 licensed, with benchmark figures published on the card.

Params: 0.5B
License: Apache 2.0
Input: text (+ reference audio to clone)
Output: audio
Languages: English and Chinese
Download: 0.99 GB

What it is good at

Conversational delivery
Voice cloning from a reference clip
English and Chinese speech
Dialogue systems
Natural pacing and pauses

Not good at: languages outside the pair it was trained on, or very long single passages.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.0 GB
int8	1 byte per param	~500 MB
int4	0.5 bytes per param	~250 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install funasr

from funasr import AutoModel

model = AutoModel.from_pretrained("FunAudioLLM/CosyVoice2-0.5B")
result = model.generate(input="Hello from talachu.", cache={})
print(result)

TTS · 3 of 6

Parler TTS mini

Hugging Face — parler-tts/parler-tts-mini-v1

3.51 GB of weights on Hugging Face

Describe the voice you want in plain words and it reads the text that way.

Overview

Parler-TTS is driven by a text description of the voice: pace, pitch, tone. You write the prompt, it speaks. Apache 2.0 licensed.

Params: 0.88B
License: Apache 2.0
Input: text + a voice description
Output: audio
Voices: described in words, not preset
Download: 3.51 GB

What it is good at

Controlling a voice without training
Describing style in plain text
Variation across takes
Research on expressive speech
English narration

Not good at: cloning a specific speaker from a short sample.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.8 GB
int8	1 byte per param	~878 MB
int4	0.5 bytes per param	~439 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install parler-tts

from parler_tts import ParlerTTSForConditionalGeneration

model = ParlerTTSForConditionalGeneration.from_pretrained(
    "parler-tts/parler-tts-mini-v1", dtype="auto"
)
prompt = "Small models, on ordinary hardware."
description = "A female speaker with a slightly low pitch, recorded in a quiet room."
# follow the model card for tokenizer + feature steps, then:
# audio = model.generate(input_ids, attention_mask, description_input_ids)

TTS · 4 of 6

Qwen3-TTS 0.6B

Hugging Face — Qwen/Qwen3-TTS-12Hz-0.6B-Base

2.51 GB of weights on Hugging Face

Qwen’s speech voice running on a 12 Hz stream — small, quick, multilingual.

Overview

Qwen3-TTS generates speech on a 12 Hz latent stream, which keeps generation fast while the model stays under a billion parameters. Apache 2.0 licensed.

Params: 0.91B
License: Apache 2.0
Input: text
Output: audio, 12 Hz stream
Languages: multilingual
Download: 2.51 GB

What it is good at

Fast speech synthesis
Multilingual output
Batch voice generation
Research on streaming speech
A small voice you can fine-tune

Not good at: strong emotion control, or matching dedicated cloning systems.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.8 GB
int8	1 byte per param	~915 MB
int4	0.5 bytes per param	~457 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoModelForTextToSpeech, AutoProcessor

repo = "Qwen/Qwen3-TTS-12Hz-0.6B-Base"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForTextToSpeech.from_pretrained(repo, dtype="auto")
# follow the model card for the tokenizer steps, then:
# audio = model.generate(**inputs)
# soundfile.write("out.wav", audio[0], samplerate)

TTS · 5 of 6

Dia2 1B

Hugging Face — nari-labs/Dia2-1B

4.31 GB of weights on Hugging Face

Talks back: it reads dialogue, with speaker tags and non-verbal sounds.

Overview

Dia2 is a 1.08B text-to-speech model that handles conversational turns. Inline tags produce laughter, sighs, and backchannel sounds. Apache 2.0 licensed.

Params: 1.08B
License: Apache 2.0
Input: text with optional tags
Output: audio
Voices: multi-speaker with inline tags
Download: 4.31 GB

What it is good at

Multi-speaker dialogue
Non-verbal sounds and laughter
Audiobook-style narration
Character voice work
Experiments with conversational rhythm

Not good at: cloning a named speaker from a short sample, or very long documents in one pass.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~2.2 GB
int8	1 byte per param	~1.1 GB
int4	0.5 bytes per param	~538 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoModelForTextToSpeech, AutoProcessor

repo = "nari-labs/Dia2-1B"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForTextToSpeech.from_pretrained(repo, dtype="auto")
prompt = "[S1] Did you run it? [S2] Yes, on my laptop. [S1] <laughter> Told you."
# follow the model card for the tokenizer steps, then generate and write the audio

TTS · 6 of 6

VibeVoice 1.5B

Hugging Face — microsoft/VibeVoice-1.5B

5.41 GB of weights on Hugging Face

Microsoft’s long-form voice: minutes of speech from a single prompt.

Overview

VibeVoice generates long-form, multi-speaker speech that runs to several minutes from one prompt. MIT licensed, so there is nothing to negotiate before you ship it.

Params: 2.7B
License: MIT
Input: text, multi-speaker
Output: audio, long-form
Voices: multiple speakers in one prompt
Download: 5.41 GB

What it is good at

Long passages in one pass
Multi-speaker narration
Audiobook and podcast drafts
Consistent voice over minutes
Research on long-form synthesis

Not good at: running on a small laptop — plan for a GPU. For short clips, move down the shelf.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~5.4 GB
int8	1 byte per param	~2.7 GB
int4	0.5 bytes per param	~1.4 GB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import AutoProcessor, AutoModelForTextToSpeech

repo = "microsoft/VibeVoice-1.5B"
processor = AutoProcessor.from_pretrained(repo)
model = AutoModelForTextToSpeech.from_pretrained(repo, dtype="auto")
prompt = "<|voice_1|> We built it small. <|voice_2|> And it runs on one machine."
# follow the model card for the tokenizer steps, then generate the audio track

Language · 1 of 6

Qwen3.5 0.8B

Hugging Face — Qwen/Qwen3.5-0.8B

1.75 GB of weights on Hugging Face

The floor of the ladder: a usable instruction model at under a billion parameters.

Overview

Qwen’s 0.87B-parameter model — the smallest entry here that still follows instructions reliably. Apache 2.0 licensed.

Params: 0.87B
License: Apache 2.0
Context: 262K tokens
Modality: text in, text out
Hardware: runs on CPU
Download: 1.75 GB

What it is good at

Short, structured outputs
Classification and routing
Summarising a paragraph
Local drafts and autocomplete
A cheap first model to evaluate

Not good at: hard reasoning, long documents, or code that needs several steps. Move up the shelf for those.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.7 GB
int8	1 byte per param	~873 MB
int4	0.5 bytes per param	~437 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

chat = pipeline("text-generation", model="Qwen/Qwen3.5-0.8B")
print(chat("In one sentence: why run models locally?", max_new_tokens=48)[0][
    "generated_text"
])

Language · 2 of 6

SmolLM2 1.7B

Hugging Face — HuggingFaceTB/SmolLM2-1.7B-Instruct

3.42 GB of weights on Hugging Face

A small instruct model that is honest about its limits — and cheap to run.

Overview

Hugging Face’s 1.71B-parameter instruct model, released with its training recipe in full. Apache 2.0 licensed.

Params: 1.71B
License: Apache 2.0
Context: 8,192 tokens
Modality: text in, text out
Hardware: comfortable on CPU
Download: 3.42 GB

What it is good at

Short instructions and chat
Classification and tagging
Summarising text
Teaching and experimentation
Fitting in 8K of context

Not good at: mathematics beyond simple arithmetic, or holding a long document in mind.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~3.4 GB
int8	1 byte per param	~1.7 GB
int4	0.5 bytes per param	~856 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

chat = pipeline("text-generation", model="HuggingFaceTB/SmolLM2-1.7B-Instruct")
messages = [{"role": "user", "content": "List three uses for a small model."}]
print(chat(messages, max_new_tokens=96)[0]["generated_text"])

Language · 3 of 6

Granite 4.1 3B

Hugging Face — ibm-granite/granite-4.1-3b

6.81 GB of weights on Hugging Face

IBM’s enterprise workhorse at 3B, with a long context and a clean license.

Overview

IBM’s 3.40B-parameter instruction model, aimed at business use with a 131K context window. Apache 2.0 licensed, with no extra terms attached to the weights.

Params: 3.40B
License: Apache 2.0
Context: 131K tokens
Modality: text in, text out
Hardware: GPU helps; CPU works slowly
Download: 6.81 GB

What it is good at

Long-document questions
Business and technical writing
Structured extraction
Tool calling patterns
Summarising across many pages

Not good at: competitive coding benchmarks, or very large reasoning chains.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~6.8 GB
int8	1 byte per param	~3.4 GB
int4	0.5 bytes per param	~1.7 GB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

chat = pipeline("text-generation", model="ibm-granite/granite-4.1-3b")
messages = [{"role": "user", "content": "Summarise this contract in five bullets."}]
print(chat(messages, max_new_tokens=160)[0]["generated_text"])

Language · 4 of 6

Phi-4 mini

Hugging Face — microsoft/Phi-4-mini-instruct

7.67 GB of weights on Hugging Face

Microsoft’s reasoning-leaning small model, MIT licensed and 131K deep.

Overview

A 3.84B-parameter instruct model trained for reasoning per parameter. MIT licensed, so the weights carry no conditions.

Params: 3.84B
License: MIT
Context: 131K tokens
Modality: text in, text out
Hardware: GPU recommended
Download: 7.67 GB

What it is good at

Multi-step reasoning
Math and word problems
Code assistance
Long-context extraction
A strong model at a small size

Not good at: very long free-form generation, or matching 8B models on broad knowledge.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~7.7 GB
int8	1 byte per param	~3.8 GB
int4	0.5 bytes per param	~1.9 GB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

chat = pipeline("text-generation", model="microsoft/Phi-4-mini-instruct")
messages = [{"role": "user", "content": "Explain overfitting in one paragraph."}]
print(chat(messages, max_new_tokens=160)[0]["generated_text"])

Language · 5 of 6

OLMo-3 7B

Hugging Face — allenai/Olmo-3-7B-Instruct

14.60 GB of weights on Hugging Face

Ai2’s fully open 7B: weights, data, and training code, all published.

Overview

The 7.30B instruct model from Ai2, whose project publishes the training data and the recipe alongside the weights. Apache 2.0 licensed, shipped in bf16 at 14.6 GB.

Params: 7.30B
License: Apache 2.0
Context: 65K tokens
Modality: text in, text out
Hardware: GPU required
Download: 14.60 GB

What it is good at

Broad general questions
Long-document understanding
Technical writing
Reproducing or auditing a model
Fine-tuning with a known recipe

Not good at: running on a CPU, or matching 8B frontier-small models on hard reasoning.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~15 GB
int8	1 byte per param	~7.3 GB
int4	0.5 bytes per param	~3.6 GB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

chat = pipeline("text-generation", model="allenai/Olmo-3-7B-Instruct")
messages = [{"role": "user", "content": "What changed between OLMo 2 and 3?"}]
print(chat(messages, max_new_tokens=160)[0]["generated_text"])

Language · 6 of 6

Qwen3 8B

Hugging Face — Qwen/Qwen3-8B

16.38 GB of weights on Hugging Face

The top of the ladder: a general 8B model with a thinking mode you can switch off.

Overview

Qwen’s 8.19B-parameter model, with an optional reasoning mode that it only enters when the question needs it. Apache 2.0 licensed.

Params: 8.19B
License: Apache 2.0
Context: 32K tokens natively, 131K with YaRN
Modality: text in, text out
Hardware: GPU required
Download: 16.38 GB

What it is good at

General question answering
Code generation and review
Summarising long inputs
Mathematical reasoning
A ceiling to compare smaller models against

Not good at: realtime CPU use. Below this size is where you go when the hardware is small.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~16 GB
int8	1 byte per param	~8.2 GB
int4	0.5 bytes per param	~4.1 GB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install transformers torch

from transformers import pipeline

chat = pipeline("text-generation", model="Qwen/Qwen3-8B")
messages = [{"role": "user", "content": "Write a haiku about a quiet server."}]
print(chat(messages, max_new_tokens=64)[0]["generated_text"])

Embeddings · 1 of 7

all-MiniLM-L6-v2

Hugging Face — sentence-transformers/all-MiniLM-L6-v2

0.09 GB of weights on Hugging Face

The default first embedding model: 23M parameters, 91 MB, and everywhere.

Overview

A sentence embedding model that maps text to 384 dimensions. It is the usual starting point for search and clustering because it is small, fast, and good enough. Apache 2.0 licensed.

Params: 22.7M
Dims: 384
Context: 256 tokens
Languages: English
MTEB: 56.26 (English v1)
License: Apache 2.0

What it is good at

Semantic search over English text
Clustering related documents
Deduplicating a corpus
Semantic similarity scores
Running on a laptop CPU

Not good at: other languages, long passages, or retrieval where precision really matters.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~45 MB
int8	1 byte per param	~23 MB
int4	0.5 bytes per param	~11 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")
vectors = model.encode(["cheap compute", "small models", "expensive training"])
print(vectors.shape)  # (3, 384)

Embeddings · 2 of 7

bge-small-en-v1.5

Hugging Face — BAAI/bge-small-en-v1.5

0.13 GB of weights on Hugging Face

Same footprint as MiniLM, meaningfully better retrieval scores.

Overview

The small member of BAAI’s embedding family: 33M parameters, 384 dimensions, built specifically for retrieval. MIT licensed.

Params: 33.4M
Dims: 384
Context: 512 tokens
Languages: English
MTEB: 62.17 (English v1)
License: MIT

What it is good at

Retrieval-augmented generation
Question answering over a corpus
Search where MiniLM falls short
Clustering and deduplication
Cheap indexing at scale

Not good at: multilingual text, or passages longer than 512 tokens.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~67 MB
int8	1 byte per param	~33 MB
int4	0.5 bytes per param	~17 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("BAAI/bge-small-en-v1.5")
# BGE expects the query side to carry a short instruction
q = model.encode(["Represent this sentence for searching relevant passages: "
                  "how do I cut memory use?"])
p = model.encode(["Lower the quantisation, then shorten the context window."])
print(float(q @ p.T))

Embeddings · 3 of 7

Granite Embedding 97M

Hugging Face — ibm-granite/granite-embedding-97m-multilingual-r2

0.19 GB of weights on Hugging Face

Under 100M parameters, 52 languages fully supported, and 32K of context.

Overview

IBM’s multilingual embedding model: 97M parameters, 384 dimensions, a 32K token window, and coverage across 200+ languages with 52 of them handled properly. Apache 2.0 licensed.

Params: 97.4M
Dims: 384
Context: 32,768 tokens
Languages: 200+ supported, 52 enhanced
MTEB: 60.3 (ML Retrieval, 18 sets)
License: Apache 2.0

What it is good at

Multilingual retrieval
Cross-lingual search
Long-document indexing
Mixing code and natural language
Enterprise corpora in many languages

Not good at: beating the 0.6B models on English-only retrieval.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~195 MB
int8	1 byte per param	~97 MB
int4	0.5 bytes per param	~49 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("ibm-granite/granite-embedding-97m-multilingual-r2")
vectors = model.encode(["cheap compute", "侵入式服务器", "petit modèle"])
print(vectors.shape)  # (3, 384)

Embeddings · 4 of 7

nomic-embed-text-v1.5

Hugging Face — nomic-ai/nomic-embed-text-v1.5

0.55 GB of weights on Hugging Face

8,192 tokens of context from 137M parameters, and you can shrink the vectors.

Overview

Nomic’s open embedding model: 137M parameters, 768 dimensions, an 8K context window, and Matryoshka support so you can truncate the vector for storage. Apache 2.0 licensed.

Params: 136.7M
Dims: 768
Context: 8,192 tokens
Languages: English
MTEB: 62.28 (English v1)
License: Apache 2.0

What it is good at

Long-document retrieval
Clustering large corpora
Trading storage against accuracy
Semantic search in English
Production indexes with fixed dimension budgets

Not good at: non-English text, or the very top of the retrieval leaderboard.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~273 MB
int8	1 byte per param	~137 MB
int4	0.5 bytes per param	~68 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5")
vectors = model.encode(["a long passage that still fits in 8K tokens"])
# Matryoshka: keep the first 256 dimensions for a cheaper index
import numpy as np
compact = np.ascontiguousarray(vectors[:, :256])

Embeddings · 5 of 7

mxbai-embed-large-v1

Hugging Face — mixedbread-ai/mxbai-embed-large-v1

0.67 GB of weights on Hugging Face

1,024 dimensions from 335M parameters — the strongest pure-English score here below 0.6B.

Overview

Mixedbread’s large embedding model: 335M parameters producing 1,024-dimensional vectors, with Matryoshka and binary quantisation support. Apache 2.0 licensed.

Params: 335.1M
Dims: 1,024
Context: 512 tokens
Languages: English
MTEB: 64.68 (English v1)
License: Apache 2.0

What it is good at

English semantic search
High-precision retrieval
Reranking pipelines
Binary quantised indexes
Clustering where 384 dims is not enough

Not good at: long passages beyond 512 tokens, or multilingual corpora.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~670 MB
int8	1 byte per param	~335 MB
int4	0.5 bytes per param	~168 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("mixedbread-ai/mxbai-embed-large-v1")
vectors = model.encode(["how to lower inference memory"])
print(vectors.shape)  # (1, 1024)

Embeddings · 6 of 7

Qwen3-Embedding 0.6B

Hugging Face — Qwen/Qwen3-Embedding-0.6B

1.19 GB of weights on Hugging Face

The top of the shelf: 100+ languages, 32K context, and the best score here.

Overview

Qwen’s 0.6B embedding model, instruction-aware and Matryoshka-ready: vectors can be cut to any dimension from 32 up to 1,024. Apache 2.0 licensed.

Params: 0.60B
Dims: 1,024 (MRL, 32–1,024)
Context: 32,768 tokens
Languages: 100+
MTEB: 70.70 (English v2)
License: Apache 2.0

What it is good at

Multilingual retrieval
Instruction-aware search
Long-document indexing
Vector storage under budget
A high ceiling without going to 4B

Not good at: staying tiny — at 0.6B it is the largest model on this shelf.

Memory

Precision	Rule	Size
fp16	2 bytes per param	~1.2 GB
int8	1 byte per param	~596 MB
int4	0.5 bytes per param	~298 MB

Memory rule from the source: GB = params × bytes per param.

Quickstart

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Qwen/Qwen3-Embedding-0.6B")
docs = model.encode(
    ["cheap models you can run yourself"],
    prompt_name="document",
)
query = model.encode(["What can I run on a laptop?"], prompt_name="query")
print(float(query @ docs.T))

Embeddings · 7 of 7

EmbeddingGemma 2

Hugging Face — google/embeddinggemma-2

1.49 GB of weights on Hugging Face

Five kinds of data, one vector space. A voice note, a photo, and a line of code, searchable with one index.

Overview

Google’s 0.74B multimodal embedder, released 06 October 2026. It turns text, code, images, video, and audio into vectors in one shared 768-dimension space. Apache 2.0 licensed, ungated — a build server pulls the weights without asking.

The model is modular. A 270M text core does text and code. A 170M vision encoder and a 300M audio encoder load only when a task asks for them. Quantized, the text-only load runs in about 191 MB of RAM; the full model in about 567 MB.

Params: 0.74B full; 270M text-only load
License: Apache 2.0 (ungated)
Input: text, code, image, video, audio
Output: 768-dim vectors (cut to 512 / 256 / 128)
Hardware: runs on a phone; CPU or GPU
Download: 1.49 GB

What it is good at

Searching text, images, video, and audio with one index
Code search and local codebase retrieval
Retrieval-augmented generation on a phone
Small vector stores — cut vectors to 128 dims

Not good at: it is an embedder, not a chatbot — it does not generate text. Its scores are self-reported and no independent test exists yet, so measure it on your own data before you commit an index.

Memory

Precision	Rule	Size
bf16	2 bytes per param	~1.49 GB
int8	1 byte per param	~740 MB
int4	0.5 bytes per param	~370 MB

The LiteRT-LM int4 bundles are 165 MB (text-only) and 485 MB (full). One warning from the model card: use bfloat16 or float32 — plain float16 breaks the output.

Quickstart

pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("google/embeddinggemma-2")

docs = model.encode(["a voice memo about a broken window"])
query = model.encode(["what broke?"])
print(float(query @ docs.T))

Text and code embed straight away. The image and audio encoders load only when a task asks for them.

Thirty-one open-weight models, by modality

Florence-2 base

Overview

What it is good at

Memory

Quickstart

SmolVLM2 500M

Overview

What it is good at

Memory

Quickstart

Florence-2 large

Overview

What it is good at

Memory

Quickstart

Moondream 2

Overview

What it is good at

Memory

Quickstart

Qwen3-VL 2B

Overview

What it is good at

Memory

Quickstart

Granite Vision 3.2

Overview

What it is good at

Memory

Quickstart

Moonshine Tiny

Overview

What it is good at

Memory

Quickstart

Whisper tiny.en

Overview

What it is good at

Memory

Quickstart

Moonshine Base

Overview

What it is good at

Memory

Quickstart

Whisper base.en

Overview

What it is good at

Memory

Quickstart

Distil-Whisper small.en

Overview

What it is good at

Memory

Quickstart

Whisper large-v3-turbo

Overview

What it is good at

Memory

Quickstart

Kokoro 82M

Overview

What it is good at

Memory

Quickstart

CosyVoice2 0.5B

Overview

What it is good at

Memory

Quickstart

Parler TTS mini

Overview

What it is good at

Memory

Quickstart

Qwen3-TTS 0.6B

Overview

What it is good at

Memory