Model distillation for open-weight AI models
We train a smaller, faster model to reproduce the behavior of a larger model on the tasks your business actually needs, so you can run it privately and at a lower cost.
What is model distillation?
Distillation is a way of transferring knowledge from a large model, called the teacher, into a smaller model, called the student. Instead of training the student only on raw data, we train it to imitate the teacher's outputs on examples drawn from your real workload.
Think of it like an experienced specialist training an apprentice for one particular job. The apprentice does not need to learn everything the specialist knows, only what is needed to do that job well. The result is a model that is much lighter, and that performs closely to the teacher on the tasks you selected.
Because the student is smaller, it typically needs less memory and compute per request. That can mean lower hosting bills, faster responses, and the ability to run on hardware you already own.

When distillation makes sense
- You have one or a few well-defined tasks, such as classification, extraction, summarization, routing, or drafting in a fixed format.
- You are running a large model at volume and the cost or latency is becoming a problem.
- You need the model to run inside your own environment for privacy, compliance, or reliability reasons.
- You have, or can collect, representative examples of the inputs the model will see in production.
Distillation is usually not the right choice if your needs are broad and open-ended, if your tasks change every month, or if you only process a small number of requests. In those cases a hosted general model or a light fine-tune may be simpler. We will tell you this during the free scoping call.
What you receive
- Model weights for the trained student model, in a standard format, which you own under our agreement.
- Evaluation report comparing the student and teacher on your evaluation set, including failure cases and known limitations.
- Deployment guide describing hardware requirements, recommended serving setup, and configuration.
- Training configuration and data processing scripts, so the work can be reproduced or extended.
Our process
- Scoping. We define target tasks, quality thresholds, size and latency budgets, and confirm the teacher model's license allows distillation.
- Evaluation set. Together we assemble a set of real examples and the expected outputs. This becomes the yardstick for every decision.
- Data generation. We collect inputs from your workload and generate teacher outputs, filtering and reviewing them for quality.
- Student training. We train one or more candidate student sizes and compare them against the teacher.
- Review and handover. You review results at each milestone. Once acceptance criteria are met, we hand over weights, the report, and the deployment guide.
Typical timelines
A focused distillation project for a single task often takes four to eight weeks. Projects covering several tasks, or with strict latency targets, can take eight to fourteen weeks. Timelines are estimates that depend on data availability and how quickly review cycles run. See pricing for indicative costs.
Frequently asked questions
How much smaller can a distilled model be?
It depends on how narrow the target tasks are. Students that are several times smaller than the teacher are common for focused tasks, but we only commit to a size after measuring quality on your evaluation set.
Will the student model be as good as the teacher?
On the specific tasks it is trained for, a student can come close to the teacher, and sometimes matches it. On unrelated tasks it will usually be weaker. We report exactly where it stands so you can decide.
Do I need my own labeled data?
Not always. Distillation can use the teacher to generate training targets from your unlabeled inputs. A small set of human-reviewed examples is still valuable for evaluation.
Can the teacher be a model I already use?
The teacher must be a model whose license permits use for training other models. We review license terms with you during scoping and only use teachers that allow it.
Curious whether distillation fits your workload?
Book a free scoping call. We will look at your use case, your data, and your constraints, and tell you honestly whether a custom model makes sense.
Book a free scoping call