Operational notes Observatory

Who decides how fast the model you bought gets better?

7 min read

Macro of the toothed wheels of a mechanical movement, meshed together, in black and white
The pace is set by the wheel upstream. None of the wheels downstream gets to choose it.

The AI adoption plan you approved for 2027 rests on a promise nobody ever put in writing for you: that the model you are building on will keep improving by itself, at the pace of the last two years. It is not a contract clause. It is an assumption — and for eight days now the four suppliers holding it up have been saying the opposite in public.

Eight days

On 12 September 2026 Dario Amodei, chief executive of Anthropic, published on his site the essay We Must Pace the Frontier. The thesis fits in one line: “We must slow the pace at which we improve the capabilities of AI models.” The plan has three steps. The first, he writes, Anthropic takes alone: outside evaluators with permissions similar to those of employees running the same risk checks. The second does not — frontier AI companies in democratic countries must “coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress”. The third calls on governments.

The same text addresses the legal obstacle: for antitrust reasons, the author writes, the US government would have to “issue a narrow waiver for certain kinds of safety conversations”. And it says what the coordination is for: slowing down together would buy the time to do the safety work “without sacrificing commercial advantage”.

In the hours that followed, Fortune reported on 19 September, the heads of OpenAI, SpaceXAI — the company behind Grok — and Google DeepMind responded in public expressing agreement.

On Friday 18 September four subscribers filed a class action in the federal court for the Northern District of California against Anthropic, PBC, OpenAI OpCo, LLC, SpaceXAI LLC and Google LLC: case no. 3:26-cv-10693, Section 1 of the Sherman Act. The claim fits in one sentence: “An agreement among competitors to reduce the quality of their products and the rate at which those products improve is an agreement to restrict output.” The boundary the claimants draw is narrow: they challenge only “an agreement among competitors about how fast their competing products will improve”; each company stays free to slow down on its own.

Asked for comment on 19 September, the four companies had not replied; none has filed a defence.

Filed, stated, alleged

Three levels, not to be confused. The essay is published by its author: a primary source, verifiable word for word. The endorsements of 12 September are public statements, reported by major outlets. The agreement is an allegation in a party filing: not a fact established by a judge, and it may never be one.

For a buyer, though, the distinction weighs less than it seems. Even if the case ended in nothing, the text behind it remains: the essay is published and says the pace must come down, and according to Fortune’s account the other three suppliers voiced their agreement in the hours that followed. That is not a law of nature: it is a business decision, and you are not the one taking it.

Who decides whether the change counts

Regulation (EU) 2024/1689, in Article 3, point 23, defines a “substantial modification” as “a change to an AI system after its placing on the market or putting into service which is not foreseen or planned in the initial conformity assessment carried out by the provider and as a result of which the compliance of the AI system with the requirements set out in Chapter III, Section 2 is affected […]”.

Read the condition at the centre again: not foreseen or planned in the initial conformity assessment carried out by the provider. A supplier that plans the next generations of the model inside its own assessment keeps those changes outside the definition. It sets the pace, and it also decides whether the change counts.

The obligations, meanwhile, stay where they are. Article 26(2) provides that “Deployers shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support.” Article 25(1)(b) adds that anyone making a substantial modification to a high-risk system already on the market is considered a provider. And in Italy the frame has just tightened: legislative decree 160 of 9 September 2026 — Official Gazette no. 214 of 15 September 2026, in force from the 30th — aligns national law with the European regulation on policing uses of AI and on civil and criminal liability, with a new criminal offence for omitted human oversight.

The asymmetry is all there: the supplier governs the speed of the product and the legal characterisation of its own updates, while you answer for oversight of a system that changes when it decides.

Four lines to write before you renew

The version. Do not buy the model: buy an identified version, with a maintenance window and written notice of withdrawal. Without that, the subject of the contract changes while the contract stays the same.

The test. A set of your own cases, on your own documents, re-run at every change of version. Public comparisons measure the supplier’s product; this one measures your work.

The date. Tie your conformity documentation to the version you validated, with the test result attached: if the version disappears, you know what to redo.

The way out. A second path you can run yourselves, tried at least once on real data. If the alternative is a supplier that signed up to the same pace, it is not an alternative: it is the same risk with a different invoice.

Why a closed system

The pace becomes yours: weights installed inside your perimeter do not change on the night a supplier ships an update, and the version you validated stays that version for as long as you need. Concentration stops being a risk: if the four coordinate, the market no longer offers the second option you had in the plan. Evaluation comes back in house: the first step of the essay — evaluators with internal access to the model — is an ordinary function in a closed system, because the evaluators are yours. Human oversight becomes demonstrable, as the regulation requires: you know the version, parameters and logs. That is why we work with open-weight models on hardware we staff: what stays installed counts for more than what gets announced.

The two axes, applied to this case

Complying. The control we put into service is a version register: which model produced which outcome, with which configuration and on which date, and which test was re-run at the change — the dated trail to show an inspector, in the terms of Articles 25 and 26 of Regulation (EU) 2024/1689. Not an opinion in a PDF.

Deciding. The same set-up holds documents, business systems, archives, plants and sensors together in a single operating model, on which AI agents execute decisions with a human operator in command — for large companies, the public sector, healthcare and defence. On-premise on self-contained machines that need no deep integration into your network, or a dedicated cloud with a data centre in Italy staffed by us, always under shared management: you need no one in house to administer models.

Could you say which version of the model produced the decision you took last month? Half an hour with one of our engineers is enough to see where you stand.

What we do not know

We do not know whether an agreement exists: the filing asserts it, no judge has established it, the defendants have not replied. We do not know how much they will really slow down, or on which product lines: none of the public texts states a measure. We do not know whether the coordination will touch services sold in Europe, which the cited documents do not name, nor what enterprise contracts say about keeping versions available.

Sources