Owning a competitive AI model used to mean founding a lab and spending like a government. That barrier dropped because two things now sit together: reasoning-trace distillation, and strong open-weight base models you can download and keep training.
A company that wants a private model, or a country that wants a sovereign one, can treat this as an engineering program with a budget and an end date. Distillation is a standard training method that got cheap enough, for everyone.
The Big Labs are doing it, so can you
When a strong model works a hard problem, it produces more than a final answer. It produces intermediate steps: the false start it discarded, the constraint it noticed late, the check it ran before committing. Those steps are a reasoning trace.
Distillation means capturing those traces, and often the answers too, and using them as training data for a second, smaller or cheaper model, so the second model starts to produce similar reasoning. Labs have done this for years as ordinary practice. Frontier labs are doing it today, including training on reasoning traces produced by other frontier labs.
Two changes made it useful outside a frontier lab.
The training signal got richer. A second model trained only on final answers learns to copy conclusions. A second model trained on traces learns the steps that led there, which is why a capable but cheaper model can get sharp on the kinds of problems the stronger model is good at.
The starting point got stronger. A strong open-weight base model already handles language, code and the general competence that used to take a fortune to build from nothing. Open-weight means you can download the weights and continue training them on your own traces, instead of pretraining a model from scratch.
Together, that turns “build a competitive model” into a bounded project. Collect traces on problems that look like your work, discard the bad ones, train, score the result against tasks your own experts already know how to grade, and decide which queries still go to a frontier model. You still need compute, data and people who know how to run the training, which is a different undertaking from founding a lab.
Open-weight releases were a choice. The publishers wanted other organizations to be able to train on top of those models. Once a capable base model is available to download, a frontier model’s reasoning traces become training data you can actually use.
Why a country cares
Sovereign AI means running a model you control, on infrastructure you control, without depending on a foreign API for the answers.
The old path was to stand up a frontier lab, spend like one, and hope to catch up. Most countries kept buying API access and wrote papers about a foundation model they would train from scratch later.
The current path is narrower. Take a strong open-weight base. Train it on a frontier model’s reasoning output for your problems: your language, your law and your public services. Run the result on infrastructure you control, with data that doesn’t leave the building.
You still need a real program. Downloaded weights do not become a national capability on their own. You need people who can train, people who can catch a fluent wrong answer, and a compute budget you can actually fund. If a ministry can’t hire those people, it should keep using a vendor API.
What dropped off the invoice is pretraining a foundation model from scratch. A data-residency clause in a vendor contract still leaves you calling that vendor’s API, so it does not replace running the model on your own infrastructure. Vendor terms get updated, access gets throttled and a region can be carved out of the product. Those are ordinary vendor decisions, and a model you trained and host is not affected by a terms update.
Language is easy to underestimate if your board works in English. A frontier model that is strong in the language of the labs that trained it can be weak in yours. A model trained on your courts, your regulations and your operational documents already has that context.
Why a company cares
Most companies don’t need a model that can do everything. They need one that is good at their work and does not send that work to a third party: support tickets, contracts, internal policy, the product names and process language that a general model mishandles.
Renting a frontier model covers a lot of that, and for many companies it should stay that way. Training your own model costs people and GPUs. If usage stays low, the vendor bill is usually cheaper.
The case for training your own is two conditions at once.
The work is sensitive enough that sending it to a vendor is the actual risk. And the volume or the specificity is high enough that a general frontier model, billed per request, costs more than it returns. Your best staff already know how to think through a claim, a design review, an escalation. If you capture those reasoning steps, you have training data no vendor sells.
Send the hard, rare, expensive-if-wrong cases to a frontier model and run the routine volume on the model you trained.
A practical test: sit with the people who do the work and sort a month of real requests into two piles. Pile one is work that a trained specialist could teach someone new in a quarter. Pile two is work where a wrong answer is expensive. If pile one is most of the month, training your own model is worth costing out. If pile two is most of the month, stay on the vendor.
Give one person ownership. IT will want to own the runtime. A line of business will want to own the traces, because those traces are how your people actually work. If those groups never sit in the same meeting, you will train on generic problems and then point the model at the one workflow that needed specialist knowledge.
Limits
The model you train will be good at the lesson you ran, and it will lose, sometimes obviously, to a frontier model trained on a much wider set of work.
A model trained on another model’s traces will copy that model’s habits, including the bad ones. If you don’t score it on your actual work, fluent output will pass review even when the answers are wrong.
Downloading weights does not finish the program, because the training run still needs engineers, legal still has to approve the traces you train on, and compute still costs money. What you no longer have to do is pretrain a foundation model from scratch in order to have a model you can run yourself.
Ask your team this
- Do we have real problems, worked examples and expert reasoning we are allowed to use as training data, or do we only want to say we have a model?
- What does the median task look like, and what does the hard, rare task look like? If the median is our work and the rare cases are few, plan for a trained in-house model plus a vendor model for the hard cases. If almost every request is a hard case, stay on the vendor.
- Does the data have to stay in the building? If it does, you are deciding whether to train and host, and the work left is doing that competently.
- Who owns evaluation? Name a person. Ranking on a vendor’s public leaderboard does not tell you how the model does on your workload.
- What would make us stop? Agree in advance on a budget limit, a quality bar and a calendar date.
Treat it as a build-versus-buy decision, which is a choice you did not really have a year or two ago.