Benchmark your model.
Validate your WebMCP.

Model-token estimates at public API prices, not subscription/proxy bills. Rates are per million tokens with all rates and costs shown in USD. Hy3 is estimated at paid API rates.

Terra uses Standard short-context rates; every observed input is below 40k. DeepSeek uses off-peak rates: recorded runs are outside weekday UTC 01–04 and 06–10 peak windows. Peak rates per million tokens are $0.30 input, $0.006 cached input and $1.20 output. Hy3 uses TokenHub Guangzhou pay-as-you-go pricing.

Luna and MiMo rates checked on 2026-09-23. Luna uses Standard API-equivalent rates, not the ChatGPT subscription bill: $0.20 input, $0.02 cached input, $0.25 cache writes and $1.20 output per million tokens. The maximum observed request input was 84,709 tokens, below the 272k long-context threshold. MiMo uses official overseas USD real-time rates: $0.14 input, $0.0028 cached input and $0.28 output per million tokens. This is a USD-equivalent comparison, not an FX conversion of the domestic bill; credits, plans and discounts are excluded.

Cache accounting was checked against 994 recorded Luna requests and 1,230 MiMo requests: input, cache and output components match the reports. Cache hits use cached-input rates without being charged again as uncached input; output already includes reasoning tokens. Two incomplete MiMo records retain recorded cost lower bounds only.

Cost = each token category × its rate / 1,000,000. Uncached input excludes cache hits; output already includes reasoning, with no second charge. All recorded cache writes are zero; an unspecified write rate does not imply free future writes. All-49 totals include final failed attempts; incomplete usage yields a recorded lower bound only. Superseded attempts, tax, external tool fees and infrastructure are excluded. Up to nine decimals are shown; downloads retain components.

Hy3 CNY tariffs are converted at 1 USD = 6.7065 CNY (2026-09-16 observation, fixed for this estimate): USD = CNY / 6.7065. This is not a separate Tencent USD tariff. FX source ↗

Each task has WebMCP on/off rows, with five models side by side. Cells show exact token counts, Agent seconds, steps, tool calls and score. Timeouts and failures retain observed usage; missing usage is —. Scroll horizontally on narrow screens.

Controls

The same 8 local sites and 49 tasks. Each arm has 10 minutes and no step limit. Both retain all DeepDeck default tools, including Chrome DevTools MCP; only WebMCP changes. Each arm resets the site and starts a fresh session/profile, always on then off. Calibration tasks and custom templates are excluded.

Models & interfaces

Terra uses ChatGPT login via openai-codex / gpt-5.6-terra. DeepSeek and Hy3 use a local OpenAI-compatible proxy with cn:deepseek-v4.1-flash and cn:hy3. No reasoning effort was explicitly requested (recorded as provider-default). The effective thinking level was not retained, so it cannot be classified as off, medium, or high. Proxy upstream identities were not independently verified. Different interfaces can also affect latency. Terra spans two corpus revisions around an environment readiness fix; hashes are retained in the download. This is not a strict same-build cross-model ranking.

Luna / MiMo

Luna uses openai-codex / gpt-5.6-luna. MiMo uses the official Xiaomi API / mimo-v2.6-flash with default thinking enabled, verified in formal-run responses. Both runs share corpus and task-set hashes and leave daily settings unchanged.

Accounting

The overview uses the common tasks passed by all ten arms with known incomplete/unverified steps excluded; site charts use each model’s own complete-usage both-pass subset without known completion gaps. Both show totals, not medians. Tokens sum input, output and cache reads/writes, not billed cost. Agent time excludes site build/reset, desktop startup, rate-limit waits and superseded attempts. One run per task: caching, load, randomness and order can affect results.

Scoring scope

A pass means satisfying the existing scorer, not manual verification of every answer detail. For example, hev-9 checks order creation automatically; reported order status needs manual review. DeepSeek md-7 retains its suspected format-related false negative. WebMCP-on answers are not automatically ground truth.

GPT-5.6 Terra

Both arms passed 49/49. Five initial environment failures were followed by a readiness fix and paired reruns of lh-6, lh-7, lh-8 and id-3. All eight reruns passed and replace the original pairs. Original records are retained; environment failures and superseded attempts are excluded from efficiency totals.

DeepSeek v4.1 Flash

On passed 48/49; off passed 47/49. Off timed out on blog-find-post and hev-9. On answered 3 for md-7 but missed the scorer’s format regex: a suspected scoring false negative, retained as scored. Model failures were not rerun and tasks were not changed. Runs resumed after pauses; unsaved partial attempts are excluded.

Hy3

On passed 49/49; off passed 48/49. Off timed out on hev-9 after 10 minutes and 76 tool calls; the failure is retained. Off on lh-8 hit an API rate limit, then passed a single-arm retry after the limit reset. This pair spans batches; the original error is retained. Efficiency excludes rate-limit waits, superseded attempts and unsaved interrupted execution.

GPT-5.6 Luna

Full Sep 22 run: both arms scored 49/49. Off did not cancel the appointment (ea-7), open the financial summary (id-8), or complete order lookup (hev-9). Raw passes are retained; these pairs are excluded from efficiency. The summary page is also unverified in id-8 on. Usage is complete; costs use official Standard API-equivalent rates.

MiMo V2.6 Flash

Full Sep 23 run: on 49/49, off 46/49. Off timed out on md-8, id-8 and hev-9; md-8 encountered page HTTP 500, and the latter two have incomplete usage. On id-8 read summary data without verifying the page step. Both ea-7 arms cancelled the appointment; on hev-9 completed lookup in about 590 seconds. Costs use official overseas USD real-time API-equivalent rates; the thinking-disabled calibration is excluded.