---
title: "GLM-5.3: sixteen benchmarks, zero on safety"
description: "On 28 August 2026 Z.ai published the weights of GLM-5.3: the card lists sixteen coding and cyber benchmarks, none for safety or for languages beyond English and Chinese."
url: https://csidia.com/en/notes/glm-5-3-sedici-benchmark-zero-sicurezza/
lang: en
date: 2026-09-01
label: "Observatory"
publisher: CSIDIA S.r.l. (https://csidia.com/)
italiano: https://csidia.com/note-operative/glm-5-3-sedici-benchmark-zero-sicurezza/
---

# GLM-5.3: sixteen benchmarks, zero on safety

On 28 August 2026 the **zai-org** organisation — Z.ai, formerly Zhipu AI — published the weights of **GLM-5.3** on Hugging Face. In the repository’s initial commit, verified on the public history, the card described itself this way: “GLM-5.3: Frontier Coding with Emergent Cyber Capabilities” — a line later dropped from subsequent revisions, but present in the version published that day. The jump the vendor claims is measurable: more than double the predecessor’s score on the exploitation benchmarks the card highlights. For anyone assessing an open-weight model for a security or defence use case, the question is not whether that jump is real — the published table can be read and independently checked — but what, in that same card, measures how safe it actually is to put into service. Checked line by line, entry by entry: nothing.

## The facts, in order

- **16 June 2026** — GLM-5.2 ships under an MIT licence. On 17 July, CAISI, the NIST body that evaluates frontier models, publishes an independent assessment that [we verified line by line](https://csidia.com/en/notes/glm-5-2-caisi-valutazione-sicurezza): the model refuses most attack requests posed as a direct question, but refuses none of them once the same task reaches an agent holding the tools.
- **14 August 2026** — Z.ai makes GLM-5.3 available only through its own hosted services, the GLM Coding Plan and ZCode, with no downloadable weights. The official documentation states: “GLM-5.3 is now available to all GLM Coding Plan users” — with no date given for API access or the weights.
- A trade outlet attributes to Z.ai a statement we could not trace on any official company channel: “API access and open weights will be released in stages following rigorous safety evaluations.” We treat this as a third-hand statement, not as a document we verified at the source.
- **26 August 2026** — Z.ai publishes the GLM-5.3-Flash weights under an MIT licence, [as we verified here](https://csidia.com/en/notes/glm-5-3-flash-zai-costo-reale-perimetro).
- **28 August 2026, 15:00 UTC** — publication of the GLM-5.3 weights, not Flash. Architecture, from the configuration file: 78 layers, 256 routed experts plus one shared, eight active per token, a context window of 1,048,576 tokens — the same figure already measured on GLM-5.2, consistent with the card’s own statement that “GLM-5.3 uses the same base model as GLM-5.2”. On disk: 141 safetensors files, 755.6 GB — 703.7 GiB — a total that neither the card nor the configuration file reduces to a single headline parameter count; trade press rounds it to roughly 753 billion, a figure we have not independently confirmed.
- **Licence**: not MIT like its Flash sibling, but a licence of its own (the “GLM-5.3 License”) carrying the same freedoms to use and redistribute commercially; one condition applies, for whoever exceeds 10 billion US dollars in annual revenue from a “Model as a Service” business — such a licensee “must pass Z.AI’s security review before using the Software or its derivative works for any commercial purpose.” Below that threshold, no review is required.

## Sixteen benchmarks, none on safety

The card reports a table of sixteen benchmarks, the one the vendor uses to claim the jump over GLM-5.2. The sixteen rows fall into three families only: coding and software-engineering tasks (Terminal-Bench 2.1 and 3.0, DeepSWE, NL2Repo, ProgramBench, FrontierSWE, SWE-Marathon, PostTrainBench), offensive and defensive cybersecurity (CyberGym, ExploitGym, ExploitBench), and general tool-use and agentic tasks (Toolathlon, AutomationBench, Agents’ Last Exam, HLE w/ Tools, GDPval-AA). Checked row by row: none measures resistance to misuse, the consistency of refusals, or factual reliability. The card’s `language` field, in its YAML text, declares only `en` and `zh`: no other language appears, either as a stated training language or as an evaluation language.

On cyber capability specifically, the card is as explicit as the table: “As we scaled post-training, cyber capability developed faster than we expected”, followed by the claim of being “state of the art on CyberGym for vulnerability discovery”. On the numbers: CyberGym rises to 84.5 from GLM-5.2’s 77.2; ExploitBench, a coverage score across 41 tasks, rises to 54.4 from 24.4; on ExploitGym, 869 tasks under two time budgets, GLM-5.3 solves 105 and 130, against the predecessor’s 29 and 39. These are the vendor’s own table figures: we have not run the model ourselves, and offer no independent confirmation of them.

The repository also hosts a `.eval_results` folder, which feeds the “eval-results” tag shown in the model’s public metadata. It holds four files: one for DeepSWE, one for HLE, two for Terminal-Bench. Checked against the commit history: they were added between 28 and 31 August under commits titled “community evaluation results”, co-authored by two Hugging Face staff members alongside the zai-org account, and they report only figures already present in the card’s table. The structure meant to make a result machine-verifiable covers, in practice, a quarter of the sixteen published tests — and none of it touches safety.

## The review that was declared, not published

Nowhere in the repository published on 28 August is there any trace of the “rigorous safety evaluations” the third-hand statement attributes to Z.ai: no file, no section of the card, no clause of the licence names them. The family’s technical report — arXiv 2602.15763, “GLM-5: from Vibe Coding to Agentic Engineering”, forty pages, the same source already cited for GLM-5.3-Flash — is no exception: downloaded and searched in full, it contains not one occurrence of the words “safety”, “red team”, “misuse”, “risk” or “guardrail”. The few near-neighbours that do appear are technical: “alignment” refers to the post-training stages, “bias” to the optimisation offset, and in one case to a company name. It describes instead the pretraining pipeline — 27 trillion tokens, classifiers built against synthetic content, separate filters for web, code and scientific material — without naming the actual sources or saying how personal or copyrighted data was handled.

For GLM-5.2, the closest precedent, the safety evaluation never came from the vendor: it came from CAISI, which measured zero refusals out of ten once the same attack request moved from a direct question to an agentic task, noting that on the safety chapters the vendor had published no results of its own. Three days after GLM-5.3’s weights went live — a model the vendor itself describes as more capable than its predecessor specifically on exploitation tasks — CAISI’s public list of assessments, checked directly on the NIST site, stops at GLM-5.2 and Kimi K3: no entry yet for GLM-5.3. That does not mean one will not eventually arrive: it means that, as of this check, none has, from either side — while the model is already downloadable and runnable in-house, and the licence requires no review at all from anyone who stays below that revenue threshold.

## What to do, in practice

- Do not treat “rigorous safety evaluations” as proof: it is a third-hand statement, not a document. Ask the vendor for the report, not the announcement.
- If your use case involves an agent with tools, not just a chat, test exploitation capability and hijack resistance yourself: the card does not, and GLM-5.2 shows the behaviour changes between the two modes.
- Do not mistake the four `.eval_results` files for a safety audit: they are coding figures added by outsiders after publication, not a new evaluation.
- If your context requires testing in languages other than English and Chinese, note that the card declares none: verification in Italian or any other European language is entirely on you.
- Before adopting a model with rising cyber capability, record the weight version, hash and acquisition date: if an independent evaluation does arrive, you will know exactly which checkpoint it covers.

The jump GLM-5.3 claims on exploitation benchmarks is real on the vendor’s own table; the check on how safe it is to put into production exists, for now, only as a reported line. These are two different registers — what the model can do, and how it behaves when someone tries to make it do the wrong thing — and only the second tells you whether it is worth adopting before someone else verifies it in your place.

*Assessing an open-weight model with declared cyber capability for an agentic use case? [Let’s talk in a thirty-minute session](https://csidia.com/en/contact): we verify the card, the licence and the real behaviour before the decision gets made by a tender document without these answers.*

## Sources

- [Hugging Face — zai-org/GLM-5.3, model card](https://huggingface.co/zai-org/GLM-5.3)
- [Hugging Face — zai-org/GLM-5.3, commit history](https://huggingface.co/zai-org/GLM-5.3/commits/main)
- [Hugging Face — zai-org/GLM-5.3, licence text](https://huggingface.co/zai-org/GLM-5.3/blob/main/LICENSE)
- [arXiv 2602.15763 — GLM-5: from Vibe Coding to Agentic Engineering](https://arxiv.org/abs/2602.15763)
- [NIST/CAISI — Assessment of Z.ai’s GLM-5.2 (17 July 2026)](https://www.nist.gov/news-events/news/2026/07/caisi-assessment-zais-glm-52)
- [NIST/CAISI — page of published assessments](https://www.nist.gov/caisi)
- [Z.ai Developer Docs — GLM-5.3, model overview](https://docs.z.ai/guides/llm/glm-5.3)
- [explainx.ai — GLM-5.3 Launch: Benchmarks, Pricing & Access (statement attributed to Z.ai on safety evaluations)](https://www.explainx.ai/blog/glm-5-3-launch-cyber-defense-benchmarks-august-2026)

Note by CSIDIA S.r.l. (Vignola, MO, Italy): https://csidia.com/en/notes/glm-5-3-sedici-benchmark-zero-sicurezza/ · Who we are and what we do: https://csidia.com/en/at-a-glance/ · engagement@csidia.com