Which Chinese model is the best depends on which evaluator you ask. Vals ranks Qwen 4 Max first among Chinese models, just ahead of Kimi K3.1 Pro. Artificial Analysis ranks Kimi first, 3.6 points ahead. In our coding agent test both passed every hidden test at their default settings, and Qwen 4 Max cost about $0.65 per task, the mean of its two fully recorded runs, against about $1.10 for Kimi in its one fully recorded run.
Which should you use?
- For the lowest bill on agent work, try Qwen 4 Max at its default,
high. In our test it passed all 59 hidden tests three times, at about $0.65 per task in the two fully recorded runs. - For terminal and shell heavy work, look at Kimi K3.1 Pro. On Vals’ Terminal-Bench 4.0 run it scores 63.6% against 48.48% for Qwen 4 Max.
- For the top setting without waiting an hour, Kimi at its default took 16 to 29 minutes in our test. Qwen at
xhightook about 47 minutes, and one run reached our 60-minute limit. - If you need weights you can host yourself, or a choice of providers, Kimi K3.1 Pro has both. Qwen 4 Max is API only.
What we measured
We ran one coding task through Atomic Agent, our open-source agent, on OpenRouter. It is the same task we used for Gemini 4 Carbon vs Fable 5.2: a warehouse stock library of about 1,400 lines with 13 planted bugs and one missing feature. The agent reads a spec, fixes the bugs, builds the feature, and runs the tests. We then score the result against 59 hidden tests the agent never sees. The untouched code passes 20 of them.
Each setting ran three times on the same build of the agent, which is included in the 0.7.0 release. We pinned Qwen 4 Max to Alibaba and Kimi K3.1 Pro to Moonshot AI on OpenRouter. For Qwen we sent the effort value explicitly. For Kimi at its default we sent no effort value, so it ran at Moonshot’s default, which the docs list as max.
| Model and setting | Hidden tests | Time per run | Steps | Cost per task |
|---|---|---|---|---|
| Qwen 4 Max, high (API default, sent explicitly) | 59, 59, 59 | 24.1, 27.2, 23.7 min (avg 25.0) | 66 to 92 | about $0.65 in the two fully recorded runs |
| Qwen 4 Max, xhigh | 59, 59, 59 | 47.3, 60.5, 47.4 min | 73 to 79 | $0.84 to $1.28 recorded, incomplete |
| Kimi K3.1 Pro, default (documented as max) | 59, 59, 59 | 28.8, 24.5, 15.9 min | 71 to 138 | about $1.10 in its one fully recorded run |
| Kimi K3.1 Pro, low | 52, 56, 59 | 10.9, 14.2, 14.5 min (avg 13.2) | 57 to 89 | $0.98 |
Qwen runs on November 16, Kimi runs on November 9, same agent build, 60-minute limit per run.
The Qwen high cost of about $0.65 is the mean of the two fully recorded runs ($0.62 and $0.68). The third and longest run was not fully recorded and likely cost more. The Kimi default cost is from the one run whose requests were fully recorded.
On xhigh every run recorded only part of its requests, so the real cost is higher than shown. The second xhigh run did not stop on its own: our 60-minute limit stopped it, and its code passed all 59 tests by then.
Kimi low ran in an earlier batch with a 20-minute limit, and all three runs finished under it.
What the numbers show:
- At the defaults, Qwen 4 Max cost about $0.65 against about $1.10 for Kimi in its one fully recorded run, roughly 40% less, in about the same time. Qwen wrote more output tokens: about 49,000 and 59,000 in the two fully recorded Qwen runs, against about 35,000 in the one fully recorded Kimi run. Its output price is less than half of Kimi’s.
- At the top settings, Kimi looked about twice as fast. Qwen at xhigh averaged about 47 minutes on two runs, and the third reached the limit. The run conditions may favour Kimi here, as the limits below explain. Xhigh is also the setting behind most of Qwen’s published scores.
- Xhigh gained nothing on our task, where high already passed everything. It took about twice as long. Qwen’s xhigh runs wrote about 1,200 to 1,800 reasoning tokens per request against about 500 to 600 at high, which suggests a higher setting reached Alibaba, though it does not prove it.
- Kimi at low was the fastest setting, but two of three runs stopped short of 59, and at $0.98 it cost more than Qwen at high.
- In our completed 60-minute batch, Qwen 3.8 Max finished none of three runs (26, 58, 24 of 59 tests); one run in an earlier, aborted batch did finish in 45 minutes. All six Qwen 4 Max runs ended with every test passing.
Cache reads did show up on the Qwen bill. OpenRouter’s records show about 2.7 million input tokens per fully recorded high run billed as cache reads, even though OpenRouter lists implicit caching as off for this model. At $1.50 that input would be over $4; the run cost $0.62. The agent sends no cache markers for Qwen, so this is Alibaba’s own caching.
For scale, on the same task and build, Claude Opus 5.5 at its default took 4.2 minutes and $0.97, and GPT-6.1 Sol took 2.4 minutes and $0.29. Both Chinese models are slow on this task.
Limits of this test. It is one task, three runs per setting, and every default run scored 59 of 59, so it shows cost and time to the same result, not which model is smarter. Kimi’s three runs went one after another on November 9. Qwen’s six ran at once on one key on November 16. That can favour Kimi’s times. Each model also ran on its own lab’s servers, so treat the time comparison as rough. We did not test Kimi at high or Qwen at low.
Effort levels
| Qwen 4 Max | Kimi K3.1 Pro | |
|---|---|---|
| Levels | low, high, xhigh | low, high, max |
| API default | high | max |
| Other values | minimal becomes low, medium becomes high, max becomes xhigh | not documented |
| Sampling | defaults temperature 1.0, top_p 0.95, top_k 20 | accepts only temperature 1 |
Qwen from the Qwen 4 Max documentation, November 16. Kimi from Moonshot’s K3.1 Pro quickstart, via our Kimi K3.1 Pro review.
The names line up badly. Both labs have three levels, but Qwen’s default is its middle one and Kimi’s default is its top one. A comparison at the defaults puts Qwen at high against Kimi at its documented default, max. A comparison by top level puts Qwen at xhigh against Kimi at max. We show both above.
The published scores are mostly at the top settings. Vals and Arena test Qwen 4 Max at xhigh. Arena labels Kimi K3.1 Pro at max, and Moonshot says it used max for its own evaluations. Artificial Analysis does not state the Qwen level.
Pricing
| Per 1M tokens | Qwen 4 Max | Kimi K3.1 Pro |
|---|---|---|
| Input | $1.50 | $2 |
| Output, including thinking | $4.50 | $10 |
| Cache read | $0.075 | $0.20 |
| 5-minute cache write | $1.875 | $2 |
| Faster variant | none | Highspeed, $4 / $20 |
| Providers on OpenRouter | 1 (Alibaba) | 18 |
Qwen 4 Max prices from Model Studio (International) and OpenRouter, checked November 16. Kimi prices from Moonshot’s pricing page and OpenRouter, checked November 16.
Qwen 4 Max is 25% cheaper on input and 55% cheaper on output. Its cache reads cost less than half as much, which matters in an agent, since the loop resends the same long prefix at every step.
Open weights change the Kimi row. On November 16 OpenRouter listed 24 Kimi K3.1 Pro endpoints from 18 providers. Moonshot charges $2 and $10. The endpoint with the lowest input price, Relace on fp4, charges $0.24 and $8.40, and Alibaba itself hosts Kimi at $2.30 and $11.50. Endpoints differ in price and quantization (Moonshot’s own is listed as mxfp4), and we ran only Moonshot’s endpoint. Qwen 4 Max has one route at one price.
Independent results
These are the figures each evaluator listed on November 16, as compiled in our Qwen 4 Max review:
| Evaluator | Qwen 4 Max | Kimi K3.1 Pro | Leader |
|---|---|---|---|
| Artificial Analysis Intelligence Index | 42.9 | 46.5 | Kimi |
| Artificial Analysis cost per task | $3.27 | $3.68 | Qwen cheaper |
| Artificial Analysis time per task | not listed | 40.1 min | |
| Vals Index | 63.20% (11th of 38) | 62.52% | Qwen |
| Vals cost per test | $9.33 | $6.44 | Kimi cheaper |
| Vals Terminal-Bench 4.0 | 48.48% | 63.6% | Kimi |
| Arena Code WebDev | 1750, 6th | 1686, 13th | Qwen |
| Arena Agent, net improvement | 7.46%, 14th | 3.95%, 21st | Qwen |
| Arena Agent median task cost | $1.47 | $1.15 | Kimi cheaper |
Arena ratings for both are preliminary. Vals and Arena tested Qwen 4 Max at xhigh, and Arena lists Kimi at max. Artificial Analysis updates its figures, so check the live page before you quote them.
Vals and both Arena boards rank Qwen 4 Max higher. Artificial Analysis and the Terminal-Bench run on Vals rank Kimi higher, and the Terminal-Bench gap, 15 points, is the largest of the percentage gaps. On cost per task, Qwen is cheaper on Artificial Analysis, and Kimi is cheaper on Vals and Arena Agent.
The Vals Index gap is the narrowest: 0.68 points. It puts Qwen 4 Max at the top of the Chinese models on that index, and it is too small to call Qwen clearly stronger.
Benchmarks from the labs
Alibaba’s launch table includes a Kimi K3.1 column. These are Alibaba’s numbers, not ours and not Moonshot’s:
| Benchmark | Qwen 4 Max | Kimi K3.1 |
|---|---|---|
| DeepSWE v1.1 | 75.7 | 75.5 |
| Terminal-Bench 4.0 | 48.5 | 67.2 |
| ProgramBench | 31.5 | 25.0 |
| Agents’ Last Exam | 46.7 | 43.4 |
| APEX-Agents | 72.1 | 65.5 |
| JobBench | 68.5 | 61.2 |
| AutomationBench (AA) | 72.5 | 73.3 |
| AA-Briefcase (Elo) | 1686 | 1674 |
| HLAB-Vals | 21.3 | 11.7 |
| HLE | 51.7 | 51.1 |
Vendor-reported, from the Qwen 4 Max blog post, November 16. Alibaba labels the column “Kimi K3.1” and does not state effort levels. It takes AutomationBench and AA-Briefcase from Artificial Analysis, HLAB-Vals from Vals, and APEX-Agents from Mercor.
On Alibaba’s own table, Qwen 4 Max leads on 8 of these 10 rows. Only two of those leads are under a point: DeepSWE by 0.2 and HLE by 0.6. Kimi leads on AutomationBench and on Terminal-Bench 4.0 by almost 19 points.
The three Terminal-Bench 4.0 figures for Kimi do not match each other. Moonshot reports 65.7% for K3.1 Pro, Alibaba reports 67.2, and Vals measured 63.6%. All three are well above Qwen 4 Max, whose figure is the same from Alibaba and Vals, 48.5 and 48.48%.
Specs side by side
| Qwen 4 Max | Kimi K3.1 Pro | |
|---|---|---|
| Released | November 16, 2026 | October 23, 2026 |
| Total / active parameters | 2.4T / 71B | 2.8T / 104B |
| Context window | 1M tokens | 1M tokens |
| Input | text, images, video | text, images, video |
| API model ID | qwen4-max | kimi-k3.1-pro |
| OpenRouter ID | qwen/qwen4-max | moonshotai/kimi-k3.1-pro |
| Open weights | no | yes, Kimi K3.1-Pro License |
From Alibaba’s blog and documentation, and from Moonshot’s tech blog.
Weights and running it yourself
Kimi K3.1 Pro weights are on Hugging Face at moonshotai/Kimi-K3.1-Pro. The repository was created on November 2, 2026, and the Hugging Face API lists a custom license named kimi-k3.1-pro and its stored size at about 1.56 TB. The license follows MIT-style terms, with two extra conditions for very large commercial users, which our Kimi review describes.
Qwen 4 Max weights are closed. Artificial Analysis lists it as proprietary and Vals lists its weights as private. Alibaba says smaller Qwen 4 models are coming to open source and gives no date. Repos named Qwen4-27B from accounts other than huggingface.co/Qwen are not from Alibaba.
Open weights do not mean a laptop. At 1.56 TB, Kimi K3.1 Pro needs a server with several data-center GPUs. What the weights do give you is the option to host it on your own hardware or pick a provider, which matters if your company will not send code to one vendor. If you want most of the work on your own machine, Atomic Agent’s Fusion mode can use a cloud model as the orchestrator while local workers do the edits. We have not run either of these models in that role.
How to try both
To repeat our test on your own work, download Atomic Agent for Mac, Windows or Linux, or install the terminal version.
curl -fsSL https://atomicagent.io/install | shatomic-agentOn Windows, run irm https://atomicagent.io/install.ps1 | iex in PowerShell. The source is on GitHub.
Connect OpenRouter with your own key and choose qwen/qwen4-max or moonshotai/kimi-k3.1-pro from the live model list; see our OpenRouter guide. To set the effort level, add one line to your openrouter entry under llm.providers in ~/.atomic-agent/config.json:
"extraBody": { "reasoning": { "effort": "high" } }The setting applies to every model on that entry. Without it, each model runs at its own default, which is high on Qwen 4 Max and max on Kimi. Kimi on OpenRouter is served by many providers at different prices and quantizations, so pin the provider if you want results like ours. More in our models guide.
For each model on its own, see our Qwen 4 Max review and our Kimi K3.1 Pro review. Qwen 4 Max against Claude is in Qwen 4 Max vs Opus 5.5.
The takeaway
There is no single best Chinese model this week. Vals and Arena put Qwen 4 Max on top, Artificial Analysis puts Kimi K3.1 Pro on top, and the gaps are small except on Terminal-Bench 4.0, where Kimi leads by about 15 points.
In our agent task, at their defaults, both solved everything, and Qwen 4 Max cost about 40% less than Kimi’s one fully recorded run, in about the same time. At the top settings Kimi finished in about half the time in our runs, which may have favoured it. Pick Qwen 4 Max for the bill on an API, and Kimi K3.1 Pro for terminal-heavy work or when you need the weights.


