IT EN
Start a trial

Operational notes Observatory

Granite 4.2 from IBM: fully dense, the declared 512K context stops at 128K in the engine

7 min read

Close-up, angled view of a black storage unit with identical bays, each with a small lock and a vertical indicator mark, blurred white background, black-and-white photograph
Every bay is identical to the next: none sits idle, none weighs less. The row keeps going past the focus point, where the card says there is still room.

On 25 August 2026, at 17:17 UTC, IBM published the weights of Granite 4.2 on Hugging Face: three models — 3, 8 and 30 billion parameters — built for agentic work inside an enterprise, Apache 2.0 licensed, no hidden experts in the architecture. It is the generation that brings native reasoning to the Granite family, and the first to declare an extended context of up to 512,000 tokens. We downloaded the model cards, the configuration files and the GitHub repository that accompanies them to see how much of that claim actually reaches the engine that serves the model. We verify it here, as of 29 August.

The facts, in order

  • Publication: namespace reserved on 7 August 2026; weights and card uploaded on 25 August at 17:17:32 UTC (30B) and 17:17:24 UTC (8B) — the time that matches the release date the card itself declares. Organisation ibm-granite: no verification badge (isVerified: false), but an enterprise plan on Hugging Face, 232 models already published and 5,254 followers — a long publishing history, not a new account.
  • Licence: Apache 2.0, declared in the metadata (license: apache-2.0) and confirmed by the LICENSE file in the project’s GitHub repository — 201 lines, the full licence text. No second restriction file found, either in the card or in the repository: no revenue threshold, no sector exclusion, no separate-negotiation clause for commercial redistribution.
  • Adoption, today: 1,791 downloads and 86 likes for the 30B, 2,604 downloads and 50 likes for the 8B — the numbers of a four-day-old release.
  • Architecture: GraniteForCausalLM, decoder-only, dense — no expert-routing field in the configuration of any of the three sizes.
  • Context: max_position_embeddings: 131072 in the configuration file of all three sizes — 128K, not more. The card, on the same page, declares otherwise.

Fully dense: what it actually weighs

Most of the recent models we have checked in this column were mixture-of-experts: a minority of parameters active per token, the rest still kept in memory to serve different requests in the same batch. Granite 4.2 does not have that problem, and does not have its advantage either: GraniteForCausalLM is a dense transformer, every parameter works on every token. The 30B’s model.safetensors.index.json declares 58,553,540,608 bytes of bf16 weights; divided by two bytes per parameter — what each number takes up in that format — the sum, ours, gives 29.28 billion parameters, consistent with the 30B label on the name. For the 8B, on the 16,761,102,336 bytes the index declares, the same arithmetic gives 8.38 billion.

For local use IBM also publishes quantised versions: FP8, NVFP4, MXFP4 and a full set of GGUF formats, from Q8_0 down to Q2_K. Verified via HTTP header: the 30B in Q4_K_M weighs 17,721,455,328 bytes — 16.5 GiB, fitting a single 24 GB card with room for a moderate context; in Q8_0, near-lossless, it reaches 31,111,705,312 bytes, 29.0 GiB. The 8B in Q4_K_M weighs 5,347,917,952 bytes, 5.0 GiB: it runs on a laptop with a mid-range dedicated card. One warning in the card concerns anyone thinking of combining quantisation with a long context: the NVFP4 and MXFP4 versions are calibrated with GPTQ on two thousand samples, and “Max context length is 2K during calibration.” — worth keeping in mind before trusting these formats beyond a few thousand tokens.

The context the card promises, and the one the config delivers

Here the card contradicts itself. The summary table at the top of every page declares: “Natively Supports 128K (Long-context extension to 512K)”. The technical blog explains where the number comes from: pre-training runs in five phases, and the fifth, IBM states, is “extending the context window to 512K tokens” — a fact about the base’s training, not necessarily about the published checkpoint.

The configuration file of all three sizes — the one an inference engine actually reads — declares max_position_embeddings: 131072 and rope_scaling: null: 128K, with no extension configured. Every serving command on the same page confirms it: vllm serve uses --max-model-len 131072, SGLang uses --context-length 131072. Neither reaches 512K. One clue, on the 30B size alone, points to a copy-paste that fell behind: its config declares rope_theta: 50000000, five times the value on the 3B and 8B (10,000,000, verified in the same files) — a parameter that is typically raised precisely to support longer contexts — while the card’s prose, identical across all three pages, still reads “θ = 10,000,000” for the 30B too.

What it would actually cost to serve 512K tokens is not a minor detail: a dense model’s attention cache grows with every token held in memory. For the 30B — 64 layers and 8 key-value heads read from the config, 128 dimensions per head derived by dividing hidden size by head count, bf16 — the sum, ours, gives 256 KiB of cache per token of context. At the 131,072 tokens the config actually delivers, that is already 32 GiB for a single request; at the 524,288 tokens declared in training, a single session would need 128 GiB, before the weights are even loaded. That is likely part of why the published checkpoint stops at a quarter of that number.

How it behaves as an agent

Only the 8B and 30B receive tool-use training: the blog is explicit — “The 3B model runs none of these.” — referring to three agentic reinforcement-learning environments: an agent that reads code and runs tests inside an isolated sandbox, one that plans and executes shell commands, one that queries the web across multiple hops. These are trained capabilities, not merely described ones: the model learns to propose execution, not just to talk about it.

How long one of those trajectories may run is written in two places, with two different numbers. The card says rollouts “can span up to 200 tool-calling turns”. The technical blog instead publishes the stage-by-stage table for the 30B chain: the Terminal stage lists 64 rollout turns, and the highest figure anywhere in the table is 128, at the SWE 2 stage. That same blog notes the column “counts the environment interactions GRPO itself sees per rollout” — not necessarily the same quantity. Neither figure have we verified in execution. How far the model can go on its own, either way, remains a decision for whoever deploys it.

What we do not know

We have not run the model against our own tools, nor measured the benchmarks: the evaluation tables are computed by IBM, with the NeMo Evaluator harness IBM chose. The project’s GitHub repository holds a disclosures folder, meant to publish structured metadata aligned with the Model Documentation Form of the EU’s GP AI Code of Practice, and it invokes IBM’s first-place finish on Stanford’s Foundation Model Transparency Index — a placement independent sources confirm. But the folder, today, holds only a notice: “The IBM Granite 4.2 disclosure files described below are currently being prepared and will be published in this repository soon.” They do not exist yet. We have not independently verified the Stanford score, nor the 15 trillion pre-training tokens the blog declares.

How we resolve it

For a model like this, with a clean licence and trained agentic behaviour, the question that matters is not whether to install it, but with what permissions: which shell it can actually touch, which repositories it can actually modify, which web requests it can actually make, before an operator authorises it case by case. It is the same principle behind holding models, data, contracts and decisions together in a single operating model, where AI agents execute with an operator in command — never alone on the perimeter that matters. On-premise on autonomous machines that require no deep integration into the client’s network, or a dedicated cloud with a data centre in Italy: always with shared management, for large enterprises, defence and public administration.

See the service · Talk to an engineer

Do you need to decide how much autonomy to give an agent before it touches a production repository or shell? Half an hour with one of our specialists, at no cost: we bring the permissions map the card does not write.

Sources