This article originally appeared on Weijin Research on Huxiu on October 29, 2026. Original Chinese title: 「中国开源模型被“打回原形”了吗」. It has been translated and adapted for an English-speaking audience.

The gap between Chinese open-source models and the US AI frontier is being reassessed. After Artificial Analysis upgraded its Intelligence Index, which measures the overall capabilities of AI models, to v5, the rankings of Chinese models fell sharply across the board. Overnight, it seemed, the gap had widened by several positions.

Bar chart ranking AI models by the Artificial Analysis Intelligence Index, separated into open-weights and proprietary models.
Now, the top of the rankings consists almost entirely of closed-source US frontier models, far ahead of Chinese open-source models. Although Kimi K3.1 Pro made the top ten, its weights have not been released; Qwen3.8 Max ranks 13th and has likewise not released its weights. Among Chinese models currently in service that have actually released their weights, DeepSeek V4.1 Flash 1017 and GLM 5.3 rank 14th and 15th, respectively, and both score below Claude Haiku 5.5, Anthropic's entry-level model.
This is a stark contrast to the previous rankings. Just a few days ago, Kimi K3.1 Pro was behind only Claude Opus 5.5, Claude Sonnet 5.5, and Muse Wonder Full.
Many people have therefore dredged up old charges, including industrial-scale distillation, overfitting to benchmarks, and attention-grabbing marketing tactics, arguing that Chinese open-source models have finally "shown their true colors" this time. Dean Ball, OpenAI's head of strategic futures, said the new rankings had finally, for the first time, broadly matched his subjective judgment.
What changed was the yardstick. Last night, Artificial Analysis significantly overhauled how it measures model capabilities. For the first time, 60% of the weight in the new Intelligence Index comes from private test sets. It added FrontierCode 2.0, which evaluates real-world coding tasks, and Terminal-Bench-Science, which assesses the ability to execute complex, long-horizon scientific research workflows in a terminal environment; it also upgraded three tests: HLE (high-difficulty knowledge and reasoning), AA-Omniscience (knowledge reliability and hallucination), and CritPt (research-level physics reasoning).
Moreover, the old index weighted general capability (General) and agentic capability (Agents) at 30% each, and coding tasks (Coding) and scientific reasoning (Scientific Reasoning) at 20% each; this time, each was set at 25%, with agentic capability further narrowed to knowledge work (Knowledge Work) and scientific reasoning broadened to science (Science).
Clearly, the new index not only continues to emphasize AI coding capability but also substantially raises the importance of the ability to carry out scientific research tasks. At the same time, moving from public question sets to private test sets helps reduce the interference caused by models optimizing for public benchmarks or even memorizing test questions.
As the year draws to a close, such an adjustment makes sense. Unlike a single evaluation benchmark, the "yardstick" for measuring the overall capabilities of AI models embodies judgments about which capabilities will matter in the future.
The industry has spent a full year chasing AI coding and agentic capabilities, during which Anthropic overtook OpenAI. But AI coding cannot satisfy all of the market's expectations that AI will raise productivity and create economic value; market penetration of AI coding appears to be nearing saturation in the short term.
AI is changing the world not only by restructuring software development but also by accelerating biomedical research, compressing decades of progress in physics, chemistry, and other sciences into a few years.
In its prospectus, Anthropic estimates that the global biopharmaceutical industry is worth $2 trillion, a considerable sum. Both OpenAI and Anthropic are pushing AI into real scientific research processes, trying to build wet-lab feedback loops with end-to-end data flows and autonomous robots. They have even beaten mathematicians and scientists to publishing independent discoveries by their models.
In this sense, Chinese open-source models have indeed revealed their true form. Since the start of this year, leading Chinese model developers have devoted a great deal of effort to coding and agents. This summer, software practitioners around the world finally discovered that GLM-5.2 and Kimi-3 had also become "usable" and could partially replace expensive frontier AI. But Chinese AI companies, intentionally or not, neglected pretraining and post-training of general-purpose models for scientific research. After all, DeepSeek had specifically developed models for mathematical reasoning and theorem proving.
As a result, on the latest Terminal-Bench-Science benchmark, a cliff-like gap has emerged between the performance of Chinese open-source models and frontier AI.
Leading closed-source models can pass about 60% of the tasks, while most Chinese open-source models remain at around 10%. This directly affects their overall performance on the Intelligence Index. Meanwhile, Gemini 4 Carbon, which Google released today, reclaimed the top spot in frontier model evaluations, largely because of its strong performance on benchmarks such as mathematics and science.

Bar chart ranking models on Terminal-Bench-Science 0.1 scores, with the best model at 64.8%; data compiled by Artificial Analysis.
This is a bitter choice. For a long time, Chinese AI companies have been constrained in both access to computing power and capital investment, and they were relatively late to start developing high-quality data.
In an investor memo that leaked unexpectedly in the middle of the year, DeepSeek founder Liang Wenfeng, discussing the gap with overseas frontier models, listed high-quality data annotation as one of the key problems; half of the team's researchers were involved in annotating data, and it would take a year to solve the problem properly. This was still only the overall bottleneck in high-quality data, not one specific to scientific research data.
With limited investment, resources naturally flowed more heavily toward areas where the technological path, user demand, and commercialization opportunities had already been validated. From the vantage point of the start of the year, that was unquestionably AI coding.
But from the vantage point of the end of the year, the focus of competition is extending further into scientific research workflows. If AI can connect research hypotheses, experiment execution, and feedback on results, its role will no longer be limited to helping humans complete tasks; it may also accelerate the next round of technological progress. The closed loop of scientific research and the recursive evolution of AI participating in its own R&D may therefore become an important narrative throughout 2027.
Of course, as Nathan Lambert, who has long studied open-source models, cautions, the decline of open-source models has been greatly exaggerated, and a single yardstick can hardly measure how different models perform in different application scenarios. As AI continues to penetrate every industry, "yardsticks" with different weightings will be needed to measure the capability structures of different frontier AI models. The cost-effectiveness advantage of Chinese open-source models in AI coding has not disappeared despite the change in ranking methodology and the drop in the overall rankings.
He also raised concerns that private test sets reduce external auditability and will affect the industry's trust in the independence of evaluation organizations.
But the shortfall in frontier capabilities likewise cannot be ignored just because AI coding still has commercial value. As AI capabilities gradually draw from and give back to society as a whole, Chinese AI model developers and the computing power and data ecosystems behind them will all face a considerable challenge.

