GLM-5.3-Flash Benchmarks: Official and Independent Results

Quick answer: GLM-5.3-Flash scores 57 on the Artificial Analysis Intelligence Index, three points behind the much larger GLM-5.3 at 60. Z.ai's own evaluation puts it at 84.3 on Terminal-Bench 2.1 and 63.4 on DeepSWE v1.1. Unusually for a launch, the headline claims were independently verified within days, because the model had already been publicly tested for a week under the codename Ox Alpha.

Why these numbers arrived pre-tested

Most launch benchmarks are vendor claims that wait weeks for independent scrutiny. This release inverted that.

On August 20, 2026 an unattributed model appeared on third-party AI platforms as Ox Alpha, free, with a million-token context and tool calling enabled. Developers used it heavily for a week without knowing who built it. As TechNode reported, Zhipu confirmed on August 26 that Ox Alpha was GLM-5.3-Flash and released the weights.

The practical consequence for anyone reading benchmark tables: a large developer population had already formed a view on this model's quality with no branding attached. Independent measurement landed fast because the model was already everywhere.

Official benchmarks from Z.ai

From the GLM-5.3-Flash model card. Methodology notes follow the table and are worth reading before you quote any of these.

The generational movement is the striking part. DeepSWE goes from 46.2 to 63.4. AutomationBench nearly doubles, 26.2 to 48.8. GDPval-AA v2 rises from 1504 to 1773.

Terminal-Bench 2.1 at 84.3 is the number that travelled. As DataCamp noted in its analysis, that sits near Claude Opus 4.8 at 85.0 and behind GPT-5.6 Terra at 87.4. On AutomationBench, the 48.8 is a clear lead over Opus 4.8 at 41.0 and DeepSeek-V4-Vision-Exp at 38.8, the largest margin over Opus on any benchmark in the set.

Z.ai's summary claim on the model card is that GLM-5.3-Flash outperforms GLM-5.2 across benchmarks and real-world workloads at one-tenth the price, while approaching Claude Opus 4.8 on coding and agentic benchmarks.

Methodology notes you should actually read

The model card is specific about how each number was produced. Honest reporting means passing that along:

• Terminal-Bench 2.1 was evaluated inside the Claude Code harness, version 2.1.207, at temperature 1.0 with a 6-hour timeout and 65,536 max new tokens. Harness choice materially affects agentic scores, and this is not a neutral one.

• Humanity's Last Exam with tools used GPT-5.6-luna (medium) as the judge model, with a 163,840-token maximum generation length and a 300,000-token context under a context-management strategy. LLM-as-judge adds its own variance.

• DeepSWE ran under the mini-swe-agent harness at temperature 0.95, top_p 1.0, 400K context, 6-hour timeout.

• NL2Repo used both rule-based and LLM-based judgement to block unauthorised pip and curl operations, specifically to prevent reward hacking.

• AutomationBench was v1.0.6, incorporating the null-type handling fix from PR #13. Version matters on this benchmark and the number is not comparable across versions.

• BabyVision used images resized so the shorter side is at least 1.5K pixels, matched to other baselines.

• GDPval-AA v2 was evaluated by Artificial Analysis, not Z.ai. That row is independent.

• Z.ai Code Bench is an in-house evaluation. The Claude Opus 4.8 coding parity claim rests on it and cannot be reproduced externally.

None of this makes the numbers wrong. It means vendor benchmarks stay vendor benchmarks until someone else runs them.

Independent benchmarks: Artificial Analysis

Someone else did run them, within days.

Artificial Analysis scored GLM-5.3-Flash at 57 on the Intelligence Index, a composite across reasoning, knowledge, mathematics and coding. Among open-weight models of similar size, the median is 29.

The comparison that matters most is internal. GLM-5.3 scores 60 on the same index.

Three index points for roughly a seventh of the cost per task. As Artificial Analysis wrote at release, that places GLM-5.3-Flash in line with GPT-5.6 Terra and Muse Spark 1.2, sitting comfortably on the Pareto frontier for intelligence versus cost per task.

The verbosity measurement

Artificial Analysis also recorded GLM-5.3-Flash generating 150M output tokens across the Intelligence Index, against a 110M median for comparable open-weight models. Somewhat above average rather than extreme, but it compounds: the model cannot have thinking disabled, and reasoning traces bill as output tokens. Benchmark scores at max effort are real, but they are not free.

Independent benchmarks: structured extraction

The model card carries LlamaIndex ExtractBench results, run as one-shot structured output from files. This is the most useful table on the page for anyone building a document pipeline, and it is the one launch coverage keeps flattening into a single number.

A 45-point gap between short and medium is not noise. GLM-5.3-Flash is close to excellent at extracting structured data from short documents and roughly coin-flip on medium ones.

The mean of 80.75 tells you almost nothing useful. The split tells you exactly what to build: if your workload is invoices, receipts, forms, purchase orders or single-page records, this model is a strong and very cheap fit. If it is multi-page contracts or long reports, chunk them into shorter units before extraction, and benchmark on your own documents before committing.

GLM-5.3-Flash vs GLM-5.3

The gap is three points on a composite index. The price gap is much larger. For most production traffic that trade favours Flash heavily, with the flagship reserved for the hard tail. Flash also adds vision, which the flagship does not have.

What the benchmarks do not tell you

The architecture is doing something unusual. As the vLLM Recipes project documents, the 45-layer model combines KDA linear-attention layers with NoPE sparse MLA layers and routes each token through 8 of 288 experts. Linear attention carries a constant-size state instead of a growing KV cache, which is what makes the long-context economics work. Z.ai credits the hybrid design with sharply reducing long-context serving costs. That claim has not been independently audited.

Context window numbers disagree. The checkpoint declares 1,048,576 tokens and the vLLM recipe confirms it. Z.ai's own footnotes describe evaluating DeepSWE under 400K, and Artificial Analysis lists the served context as 400K. The weights support a million tokens; what a given deployment serves depends on configuration and KV cache budget.

Adoption is its own signal. The model card shows over 441,000 downloads in the first month, 79 community quantizations, 11 fine-tunes and 36 active discussion threads. For a model that spent its first week anonymous, that is a fast ecosystem.

Frequently asked questions

What is GLM-5.3-Flash's Intelligence Index score? 57 on the Artificial Analysis Intelligence Index at max reasoning effort, against a median of 29 for open-weight models of similar size.

What does GLM-5.3-Flash score on Terminal-Bench? 84.3 on Terminal-Bench 2.1, evaluated inside the Claude Code harness at temperature 1.0. That is near Claude Opus 4.8 at 85.0 and behind GPT-5.6 Terra at 87.4.

What does GLM-5.3-Flash score on DeepSWE? 63.4 Pass@1 on DeepSWE v1.1, up from 46.2 for GLM-5.2, run under the mini-swe-agent harness with 400K context.

Does GLM-5.3-Flash beat Claude Opus 4.8? On AutomationBench, yes: 48.8 against 41.0. On Terminal-Bench 2.1 it trails slightly, 84.3 against 85.0. Z.ai's coding parity claim rests on its in-house Z.ai Code Bench, which is not externally reproducible. Individual benchmark wins are not model equivalence.

How does GLM-5.3-Flash compare to GLM-5.3? 57 versus 60 on the Intelligence Index, at $0.09 versus $0.68 cost per task. Flash also supports vision, which GLM-5.3 does not.

Are GLM-5.3-Flash's benchmarks independently verified? Partly, and faster than usual. The Artificial Analysis Intelligence Index score and the GDPval-AA v2 result are independent, as is ExtractBench. Terminal-Bench, DeepSWE and NL2Repo were run by Z.ai in specific harnesses, and Z.ai Code Bench is in-house.

Is GLM-5.3-Flash good at document extraction? Excellent on short documents at 96.30 on the ExtractBench short split, and much weaker on medium documents at 51.56. Chunk long documents before extraction.

What was the Ox Alpha model? Ox Alpha was the anonymous codename GLM-5.3-Flash was tested under from August 20, 2026, before Zhipu confirmed its identity on August 26.

How many output tokens does GLM-5.3-Flash generate? Artificial Analysis measured 150M across its Intelligence Index, against a 110M median for comparable open-weight models. Thinking cannot be disabled on this model, only reduced via reasoning_effort.

Benchmark it yourself

zai-org/GLM-5.3-Flash is live on Qubrid AI at $0.0863 per 1M input tokens. Point any OpenAI-compatible SDK at https://platform.qubrid.com/v1 and evaluate it on your own workload, which is the only benchmark that decides anything.

Get your API key at platform.qubrid.com