A Near-Frontier AI Result From Kimi K3.1 Pro, With One Catch

🔍 Read the full analysis: A Near-Frontier AI Result From Kimi K3.1 Pro, With One Catch on ThorstenMeyerAI.com

Age 18–24?Offer from Amazon

Prime made for students and young adults

• Fast, free delivery for dorm and study essentials
• Prime Video and Amazon Music included
• Member-only deals

Try Prime for Young Adults →Free trial for eligible 18–24 year olds

As an affiliate, we earn on qualifying purchases.Hide

Moonshot AI launched Kimi K3.1 Pro on October 23, and an evaluation by Artificial Analysis ranked it fourth on its Intelligence Index, close to leading proprietary models. The model is available through Moonshot’s apps and API, but the company has only promised to publish its weights by November 4, and has not published a licence for them.

Artificial Analysis gave K3.1 Pro an Intelligence Index score of 55.1, behind Claude Opus 5.5 at 57.6, Claude Sonnet 5.5 at 56.0 and Muse Wonder Full at 55.8. Its evaluation lists a cost of $2.11 per index task, compared with $5.98 for Opus 5.5 at its highest reasoning setting. These figures describe the benchmark setup, not a guaranteed cost for every real-world task.

Artificial Analysis lists the model as a mixture-of-experts system with 2.8 trillion total parameters and 104 billion active parameters, a one-million-token context window and text and image input. Moonshot made it available through the Kimi apps, Kimi Work, Kimi Code and its API. The company describes it as a more heavily post-trained version of Kimi K3.

Artificial Analysis recorded a 67% score on Terminal-Bench 4.0, its highest result in that evaluation at the time, up from 13% for Kimi K3. On GDPval-AA, a simulated knowledge-work evaluation, K3.1 Pro scored 1,846 Elo, behind Opus 5.5 and Muse Wonder Full. Its score on Humanity’s Last Exam was 52%, compared with 61% for Opus 5.5.

A Smaller Gap on Agentic Tasks

The result matters because open-weight models have generally scored below leading proprietary systems on broad evaluations. Artificial Analysis put the best open-weight model it had measured before K3.1 Pro at 46.3, while Opus 5.5 scored 57.6. K3.1 Pro’s 55.1 score narrows that measured gap, particularly on the agentic coding benchmark.

That could interest developers and businesses weighing model capability against cost or control over where software runs. But the distinction between a model being offered through an API and its weights being available for others to download is material. K3.1 Pro is not yet a usable self-hosted option; even when weights are released, a model of this size would require substantial computing resources. The missing licence also leaves unanswered what users may do with the weights.

The benchmark price comparison is promising but limited. Artificial Analysis’s per-task cost is calculated for its own evaluation, and costs in production can vary with prompts, output length, reasoning settings and usage patterns. The score and price do not, on their own, establish that K3.1 Pro is a cheaper or better choice for every organization.

Launch Claims Versus Evaluations

Moonshot presented K3.1 Pro as its most capable “3T-class open-weight model” and said it could match strong proprietary models in coding, agentic work and knowledge tasks. The company’s launch materials included demonstrations involving game engines, 3D modelling, CAD and finance. Those are vendor claims and demonstrations, rather than findings independently confirmed in the supplied material.

Moonshot reported a 92.7% result on BenchCAD with tools, compared with 96.2% for Opus 5.5 and 95.9% for GPT-6 Astra. It also cited an internal financial benchmark score of 64.4 on OfficeQA Pro v2. On an internal hallucination test, the company said the claim-level hallucination rate fell from 23.4% for Kimi K3 to 8.3% for K3.1 Pro, a reported reduction of about 65%.

Artificial Analysis measured a smaller change on its own AA-Omniscience evaluation: K3.1 Pro’s hallucination rate was 44%, down from 53% for Kimi K3. Moonshot cautioned that its internal hallucination test was deliberately designed to provoke errors and that its rates do not represent overall error rates in daily use. Results from these separate tests should not be treated as interchangeable.

“The company called K3.1 Pro its most capable “3T-class open-weight model.””

— Moonshot AI, in its launch description

Weights, Licence and Real-World Results

The most consequential open questions concern the promised release. Moonshot said the weights would be published by November 4, but the supplied information does not confirm that release or identify a licence. Until both are available, users cannot assess the terms for redistribution, commercial use or modification, or verify how readily the model can be deployed outside Moonshot’s services.

Benchmark scores also do not establish how the model will perform across everyday work. Artificial Analysis’s results offer an external comparison, while Moonshot’s demonstrations and internal tests remain company-reported. The available material does not provide enough detail to independently reproduce all of the company’s claims about complete game projects, finance performance or reduced hallucinations. Actual operating costs and hardware requirements for self-hosting also remain uncertain.

The November 4 Release Test

The next stated milestone is Moonshot’s promised November 4 publication of the weights. Readers and developers will be able to assess whether the files appear on schedule and, if they do, inspect the accompanying licence and technical documentation. The supplied material gives no confirmed release plan beyond that deadline.

Further independent evaluations could clarify whether the launch-day results hold across other tasks and testing methods. For now, the clearest conclusion is narrower: K3.1 Pro has scored close to leading proprietary models in Artificial Analysis’s evaluation, but its status as an open-weight model depends on a release and licence that are still pending.

Key Questions

What is Kimi K3.1 Pro?

Kimi K3.1 Pro is a Moonshot AI model launched on October 23, 2026. Artificial Analysis lists it as a mixture-of-experts model with 2.8 trillion total parameters and 104 billion active parameters.

How did it score against other models?

Artificial Analysis gave it an Intelligence Index score of 55.1, ranking it fourth in the listed comparison, behind Claude Opus 5.5, Claude Sonnet 5.5 and Muse Wonder Full.

Can users download and run its weights now?

Not according to the supplied information. Moonshot said the weights would be published by November 4, and no K3.1 Pro licence had been published as of October 24.

Is the claim of 65% fewer hallucinations independently verified?

No. That figure comes from Moonshot’s internal test. Artificial Analysis measured a decrease from 53% to 44% on a separate evaluation, and Moonshot said its internal test does not represent everyday error rates.

Source: ThorstenMeyerAI.com