Find the smallest Gemma 4 that does your job, and train it to do it perfectly.
Two stages: find, then distill.
Find the smallest sufficient model
Start at the largest Gemma 4 size and step down through smaller ones. The pipeline runs your task against each and finds the smallest model that already hits ~80% satisfaction.
Distill it to 100% on your task
Take that small model and train it on outputs from the larger teacher until it stably hits 100% on your task. The result is a small, fast model that behaves the way you need.
A smaller model is a different product.
Cheaper to serve
Fewer parameters means fewer GPU cycles per request. Your inference bill scales with model size, not just usage.
Faster to run
Lower latency for the same task. Especially noticeable in interactive product flows where each call is on the critical path.
Runs anywhere
Small enough for laptops, phones, and WebGPU. Run it inside the browser, or offline on-device, when that matters.
Gemma 4 family.
Distil currently supports the Gemma 4 family of open-weight models. Other families will be added as the workflow proves out.
| Model | Parameters | Role |
|---|---|---|
| Gemma 4 31b | 31B (dense) | Teacher |
| Gemma 4 26b A4b | 26B MoE, 4B active | Teacher |
| Gemma 4 E4B | 4B effective | Teacher / Student |
| Gemma 4 E2B | 2B effective | Student |
More from the toolkit.
Code-first SDK for fine-tuning open-weight models.
Learn moreInspect and manage training runs from your browser or coding agent.
Learn moreTrain custom models on managed GPUs and use them instantly.
Learn moreServe open-weight models in your app without running inference infrastructure.
Learn more