GLM-5.3: sixteen benchmarks, zero on safety

On 28 August 2026 the zai-org organisation — Z.ai, formerly Zhipu AI — published the weights of GLM-5.3 on Hugging Face. In the repository’s initial commit, verified on the public history, the card described itself this way: “GLM-5.3: Frontier Coding with Emergent Cyber Capabilities” — a line later dropped from subsequent revisions, but present in the version published that day. The jump the vendor claims is measurable: more than double the predecessor’s score on the exploitation benchmarks the card highlights. For anyone assessing an open-weight model for a security or defence use case, the question is not whether that jump is real — the published table can be read and independently checked — but what, in that same card, measures how safe it actually is to put into service. Checked line by line, entry by entry: nothing.

The facts, in order

16 June 2026 — GLM-5.2 ships under an MIT licence. On 17 July, CAISI, the NIST body that evaluates frontier models, publishes an independent assessment that we verified line by line: the model refuses most attack requests posed as a direct question, but refuses none of them once the same task reaches an agent holding the tools.
14 August 2026 — Z.ai makes GLM-5.3 available only through its own hosted services, the GLM Coding Plan and ZCode, with no downloadable weights. The official documentation states: “GLM-5.3 is now available to all GLM Coding Plan users” — with no date given for API access or the weights.
A trade outlet attributes to Z.ai a statement we could not trace on any official company channel: “API access and open weights will be released in stages following rigorous safety evaluations.” We treat this as a third-hand statement, not as a document we verified at the source.
26 August 2026 — Z.ai publishes the GLM-5.3-Flash weights under an MIT licence, as we verified here.
28 August 2026, 15:00 UTC — publication of the GLM-5.3 weights, not Flash. Architecture, from the configuration file: 78 layers, 256 routed experts plus one shared, eight active per token, a context window of 1,048,576 tokens — the same figure already measured on GLM-5.2, consistent with the card’s own statement that “GLM-5.3 uses the same base model as GLM-5.2”. On disk: 141 safetensors files, 755.6 GB — 703.7 GiB — a total that neither the card nor the configuration file reduces to a single headline parameter count; trade press rounds it to roughly 753 billion, a figure we have not independently confirmed.
Licence: not MIT like its Flash sibling, but a licence of its own (the “GLM-5.3 License”) carrying the same freedoms to use and redistribute commercially; one condition applies, for whoever exceeds 10 billion US dollars in annual revenue from a “Model as a Service” business — such a licensee “must pass Z.AI’s security review before using the Software or its derivative works for any commercial purpose.” Below that threshold, no review is required.

Sixteen benchmarks, none on safety

The card reports a table of sixteen benchmarks, the one the vendor uses to claim the jump over GLM-5.2. The sixteen rows fall into three families only: coding and software-engineering tasks (Terminal-Bench 2.1 and 3.0, DeepSWE, NL2Repo, ProgramBench, FrontierSWE, SWE-Marathon, PostTrainBench), offensive and defensive cybersecurity (CyberGym, ExploitGym, ExploitBench), and general tool-use and agentic tasks (Toolathlon, AutomationBench, Agents’ Last Exam, HLE w/ Tools, GDPval-AA). Checked row by row: none measures resistance to misuse, the consistency of refusals, or factual reliability. The card’s language field, in its YAML text, declares only en and zh: no other language appears, either as a stated training language or as an evaluation language.

On cyber capability specifically, the card is as explicit as the table: “As we scaled post-training, cyber capability developed faster than we expected”, followed by the claim of being “state of the art on CyberGym for vulnerability discovery”. On the numbers: CyberGym rises to 84.5 from GLM-5.2’s 77.2; ExploitBench, a coverage score across 41 tasks, rises to 54.4 from 24.4; on ExploitGym, 869 tasks under two time budgets, GLM-5.3 solves 105 and 130, against the predecessor’s 29 and 39. These are the vendor’s own table figures: we have not run the model ourselves, and offer no independent confirmation of them.

The repository also hosts a .eval_results folder, which feeds the “eval-results” tag shown in the model’s public metadata. It holds four files: one for DeepSWE, one for HLE, two for Terminal-Bench. Checked against the commit history: they were added between 28 and 31 August under commits titled “community evaluation results”, co-authored by two Hugging Face staff members alongside the zai-org account, and they report only figures already present in the card’s table. The structure meant to make a result machine-verifiable covers, in practice, a quarter of the sixteen published tests — and none of it touches safety.

The review that was declared, not published

Nowhere in the repository published on 28 August is there any trace of the “rigorous safety evaluations” the third-hand statement attributes to Z.ai: no file, no section of the card, no clause of the licence names them. The family’s technical report — arXiv 2602.15763, “GLM-5: from Vibe Coding to Agentic Engineering”, forty pages, the same source already cited for GLM-5.3-Flash — is no exception: downloaded and searched in full, it contains not one occurrence of the words “safety”, “red team”, “misuse”, “risk” or “guardrail”. The few near-neighbours that do appear are technical: “alignment” refers to the post-training stages, “bias” to the optimisation offset, and in one case to a company name. It describes instead the pretraining pipeline — 27 trillion tokens, classifiers built against synthetic content, separate filters for web, code and scientific material — without naming the actual sources or saying how personal or copyrighted data was handled.

For GLM-5.2, the closest precedent, the safety evaluation never came from the vendor: it came from CAISI, which measured zero refusals out of ten once the same attack request moved from a direct question to an agentic task, noting that on the safety chapters the vendor had published no results of its own. Three days after GLM-5.3’s weights went live — a model the vendor itself describes as more capable than its predecessor specifically on exploitation tasks — CAISI’s public list of assessments, checked directly on the NIST site, stops at GLM-5.2 and Kimi K3: no entry yet for GLM-5.3. That does not mean one will not eventually arrive: it means that, as of this check, none has, from either side — while the model is already downloadable and runnable in-house, and the licence requires no review at all from anyone who stays below that revenue threshold.

What to do, in practice

Do not treat “rigorous safety evaluations” as proof: it is a third-hand statement, not a document. Ask the vendor for the report, not the announcement.
If your use case involves an agent with tools, not just a chat, test exploitation capability and hijack resistance yourself: the card does not, and GLM-5.2 shows the behaviour changes between the two modes.
Do not mistake the four .eval_results files for a safety audit: they are coding figures added by outsiders after publication, not a new evaluation.
If your context requires testing in languages other than English and Chinese, note that the card declares none: verification in Italian or any other European language is entirely on you.
Before adopting a model with rising cyber capability, record the weight version, hash and acquisition date: if an independent evaluation does arrive, you will know exactly which checkpoint it covers.

The jump GLM-5.3 claims on exploitation benchmarks is real on the vendor’s own table; the check on how safe it is to put into production exists, for now, only as a reported line. These are two different registers — what the model can do, and how it behaves when someone tries to make it do the wrong thing — and only the second tells you whether it is worth adopting before someone else verifies it in your place.

Assessing an open-weight model with declared cyber capability for an agentic use case? Let’s talk in a thirty-minute session: we verify the card, the licence and the real behaviour before the decision gets made by a tender document without these answers.

The facts, in order

Sixteen benchmarks, none on safety

The review that was declared, not published

What to do, in practice

Sources