Grok 4.5's hallucination rate doubled to 54% while its accuracy went up

Artificial Analysis published its independent evaluation of Grok 4.5 on July 8, the day after xAI shipped the model. The headline number that xAI and its supporters have been quoting is the Intelligence Index score: 54, fourth place, behind Claude Fable 5, GPT-5.5 and Claude Opus 4.8, and a 16 point jump over Grok 4.3. That is a real improvement and it is worth saying plainly.

Further down the same evaluation is a number nobody put on a slide. On AA-Omniscience, the benchmark that tests what a model knows and how it behaves when it does not know, Grok 4.5's hallucination rate went from 25% on Grok 4.3 to 54%. Its accuracy went up too, from 35% to 52%. The composite Omniscience score rose from 18 to 26.

Both things are true at once, and the second one is the constraint.

What that number actually measures

AA-Omniscience does not just ask whether a model gets the answer right. It tracks what the model does with questions it cannot answer: admit the gap, or produce a confident wrong answer anyway. The hallucination rate is the share of the time it picks the second option.

So Grok 4.5 knows more than Grok 4.3 and is also substantially more willing to make something up. Artificial Analysis calls this a common pattern, and it is. Scale up the parameter count and the model's sense of what it does not know tends not to scale with it. Grok 4.5 is 1.5 trillion parameters, three times its predecessor, per Musk's own disclosure.

The practical read is narrow and specific. If your workload is an agent calling tools, running code, and getting real feedback from the environment, a wrong guess gets caught by the next step. If your workload is generating text that a human or a downstream system will trust without checking, the hallucination rate is a hard limit, not a footnote.

What it is genuinely good at

The engineering here is not hype, and the cost numbers are the most interesting part of the release.

Token efficiency is the mechanism behind the price, not just the sticker. Grok 4.5 used about 14,000 output tokens per Intelligence Index task, more than 60% fewer than Opus 4.8. Cheap per token and fewer tokens per task compound. That is how a coding agent task lands at $2.49 instead of $11.80.

API pricing is $2 per million input tokens and $6 per million output, with cache hits discounted 75% to $0.50. Two things that do not appear in the marketing: the context window dropped to 500,000 tokens from Grok 4.3's 1 million, and inputs over 200,000 tokens are billed at a higher rate.

The gap between the claim and the measurement

On July 9, Musk posted that Grok 4.5 had "landed at number 1 on multiple benchmarks." During the private beta, he was reported as saying it may match or exceed Anthropic's Opus.

Independent measurement puts it fourth on the composite index and first on exactly one thing: agentic tool use, where it beat every model Artificial Analysis has charted. That is a real win, and it is the reason to run this model. It is not the same claim.

If you are evaluating Grok 4.5, the number to test is the one that decides whether you can use it at all. Run it on your own task distribution, count how often it answers a question it should have declined, and size your validation loop for that. The benchmark tells you where to look. It does not tell you what your checks have to catch.