Small models.
Open weights.
No permission needed.
A hand-picked shelf of small, permissively licensed models — with the memory math to prove they fit where you want to run them.
-
Cheaper. Serving a small model costs a fraction of what a big one does. Smaller model, smaller bill.
-
Lower latency. Less compute per request means shorter waits, and no network round-trip at all when the model runs on your machine.
-
Runs anywhere. Laptop, phone, a five-dollar VPS. Pick the home that fits the job; the weights do not care.
-
Fits where big models cannot. The memory math decides — and small wins. Prove it to yourself, right here.
Why small models? Because model memory and real inference memory are not the same thing.
Model
Your hardware
Weights1.0 GB
KV cache3.5 GB
Activations + runtime500 MB
Total5.0 GB of 24 GB
Fits — with room to spare
Likely ~68 tokens/sec per user (272 across 4) — estimate
KV cache is 70% of the total — cut context, users, or KV precision before you cut the weights
Model memory: params × bytes. Real inference memory:
weights + KV cache + activations + overhead. KV cache is
the part that grows with context and users, not with parameters.
See the full math →