IT EN
Start a trial

Operational notes Observatory

Distilling a model you cannot export: what it means if you expose an API

8 min read

A copper still in a distillery, its condenser spout pointing towards an empty bowl, in black and white
What comes out of the spout is a distillate: it concentrates some properties of the original and loses others. The same holds for models.

Today, 31 July 2026, Reuters published an exclusive investigation: researchers linked to the People’s Liberation Army and to other Chinese security bodies have reportedly used outputs from OpenAI and Anthropic models to train domestic systems for surveillance, cyberwarfare, drones and tactical decision-making. The agency says it reviewed more than 80 Chinese academic papers and patents: research compiled by the Washington-based Jamestown Foundation and shared exclusively with Reuters, plus around two dozen cases the agency identified itself.

One qualification, because it changes the weight of everything that follows: this is a single wire-agency investigation, and the list of documents reviewed has not been published, so no reader can currently repeat the check independently. What follows is what Reuters states, attributed as such, set alongside the documents that can be read in full: the two suppliers’ own contracts.

What the investigation says it found

The recurring technique is distillation: you query a large model, collect its outputs and use them as training material for a small model that then runs locally on modest compute. It is not unlawful in itself, and it is in routine use across the industry — the dispute is about the unauthorised extraction of capabilities from proprietary systems, not about the method.

The cases reported by the agency:

  • a paper published last year by researchers from the PLA’s Unit 96941 — for Reuters, a Beijing-based military intelligence and cyberwarfare unit — on using GPT-3.5 to summarise military source code; a domestic model was trained on those summaries so it could operate entirely inside Chinese military networks. The stated rationale: external models are unsuitable for classified information;
  • at the North University of China, described as closely linked to the country’s weapons industry, Claude 3 Haiku used to generate synthetic training data for a text-classification model intended for social media monitoring and content moderation;
  • a 2024 paper from the PLA’s National University of Defense Technology on shrinking an image-processing model for deployment on unmanned aerial vehicles, to analyse live video and support navigation and targeting in real time, including when communications are disrupted;
  • a study published earlier this year by the Academy of Military Sciences on running a target-recognition model on tactical hardware in simulated maritime operations involving drones, ships and uncrewed submersibles;
  • in January, a paper from the Army Engineering University on data-free distillation, the reconstruction of a model’s capabilities without access to its parameters, with proposed defences to conceal what public outputs leak.

Reuters also reports the context: central and local government have long promoted “model lightweighting” and edge computing, funding work that lets AI models run on drones, satellites and other devices with limited processing power.

Who answered and who did not

Anthropic said it does not offer commercial access to Claude in China or to Beijing-controlled companies, and that it uses monitoring systems to detect policy violations; it added that a distilled model can lose the original system’s safety safeguards, transferring sensitive capabilities beyond the control of whoever built them. The White House, the Pentagon, China’s Foreign Ministry, the PLA and OpenAI did not respond to the agency’s requests for comment. Beijing’s position on allegations of this kind is the familiar one: American AI “hegemonism”, and comparable practices by US companies. Moonshot, Reuters reports, denied last week that Kimi K3 was built through distillation — we wrote about those allegations on 27 July, when they were statements on a social platform rather than formal acts.

One technical limit scales the story down: a distilled model reproduces selected capabilities, not the general intelligence of the system it was extracted from. Trevor Koverko, co-founder of the data firm Sapien, puts it to the agency this way: “It is best understood as transferring selected capabilities into a cheaper, locally controlled system, not achieving independence from frontier AI.”

The documents you can read: the contracts

The yardstick against which the breach is measured is public, and worth knowing, because you will use it in a supplier questionnaire.

Anthropic puts the ban in its Usage Policy (effective 15 September 2025), under “Do Not Abuse our Platform”: prohibited is the “Utilization of inputs and outputs to train an AI model (e.g., ‘model scraping’ or ‘model distillation’) without prior authorization from Anthropic”. It reappears in contractual form in the Commercial Terms of Service (17 June 2025): the customer may not “access the Services to build a competing product or service, including to train competing AI models”.

OpenAI puts it elsewhere, and that is the detail that makes people ask the wrong question. Distillation does not appear in the Usage policies (effective 29 October 2025): what sits there is the prohibition on “weapons development, procurement, or use, including conventional weapons or CBRNE” and on “national security or intelligence purposes without our review and approval”. The extraction ban is in the Terms of Use in the version applicable outside the United States (effective 1 January 2026), in the “What you cannot do” list: “Automatically or programmatically extract data or Output” and “Use Output to develop models that compete with OpenAI”. On the enterprise side, the Services Agreement (1 January 2026) defines “Reverse Engineer” to include, expressly, “model extraction or stealing attacks”.

So asking a supplier “does your usage policy ban distillation?” produces different answers depending on where the ban sits, and a “no” does not mean there is none: read the whole chain — usage policies, consumer terms, enterprise agreement.

The operational lesson

The point is not geopolitical, which is why it also concerns organisations that will never sell anything to a defence ministry: a contract clause is not a technical control. A written prohibition acts afterwards — account suspension, termination, litigation. It does not stop outputs from leaving, because the outputs are the product you sold; and anyone querying a model at industrial scale from a jurisdiction where the clause cannot be enforced will never feel it. The same pattern showed up on access: US models reachable, lawfully, through the overseas subsidiaries of blacklisted Chinese groups, while export control was busy with hardware.

It cuts both ways. If you buy a model through an API, the supplier is promising a degree of control over outputs that it does not have: clauses, abuse monitoring and suspensions are not physical barriers. That is not an argument against APIs, it is an argument for not drafting one as though it were a guarantee — a limit that is neither written down nor verifiable does not exist, and one written down but not measurable is worth little more. If you expose a model through an API — customer-facing service, product assistant, partner endpoint — your outputs are somebody else’s potential training set. If they carry your own work (pricing rules, assessment criteria, diagnostic procedures), distillation is a trade-secret leakage channel that no clause intercepts on its own.

What to do now:

  1. Measure, don’t just prohibit. Per-account request limits, detection of systematic querying patterns, alerts on unusual volumes from a handful of credentials: that is the only layer acting while the event happens.
  2. Reduce the information per response. Returning the result without the chain of reasoning that produced it lowers the value of distillation: teaching the answer is easy, teaching the reasoning is not.
  3. Write it into the contract, knowing what for. A ban on using outputs to train models grounds termination and damages. It stops nobody.
  4. Separate data that may leave from data that may not. For classified, health or sensitive industrial information the answer is not a better clause: it is a model that runs where the data already is. That is the choice we set out in hardware and open-weight models, where the subject is ownership of the stack, not just its licence.

Those research groups distil for the same reason a hospital or a bank should care about the story: they want a system that works inside a perimeter. The difference lies in how you get there — by extracting it from somebody else’s service, or by building it on models you can install and infrastructure you can inspect. That is the standard we apply when delivering a dedicated AI system, always in two forms and never just one: on-premise in the customer’s own environment, or CSIDIA’s dedicated cloud — an environment reserved for a single customer, accessed via a dedicated VPN, with a data centre in Italy staffed directly by us. In both cases the model’s outputs stay inside a boundary you can describe in a tender document and verify at acceptance testing. What counts is the method, not whichever supplier is fashionable this quarter.

Do you expose a model or an assistant to your customers without knowing what leaves in its answers? Let’s talk for thirty minutes.

Sources