THREE EXPERIMENTS. THE POTENTIAL OF WEBMCP.
Explore once.
Build on it.
Can the next task benefit from what an agent has already learned? We tested what happens when website workflows become reusable WebMCP tools: finding books, reading posts, and organizing discussions.
Explore the resultsWORKFLOW ILLUSTRATION · EXPERIMENTS MEASURE REUSE
Time on the book task
Both groups 5/5; 52.0% fewer tokens
02 / X−10.3%Time on simple post reading
Both groups 5/5; 8.4% more tokens
03 / HACKER NEWS3/3Complete WebMCP deliveries
CU baseline 0/3; 73.3% of comments retrieved
Times are medians. Baseline capabilities differ across experiments; success rates are not pooled. HN groups ran at different times with unequal completion.
THE RESULTS
See the difference. Keep the context.
Compare the final valid groups by task and metric. Same model, fresh sessions, and field-by-field verification of the delivered answers.
MEDIAN PER RUN · LOWER IS LESS
Books to Scrape
Five runs per group in the same batch. The baseline allowed DevTools and ad hoc browser scripts. Installed-tool reuse; authoring and preparation excluded.
BOOKS TO SCRAPE
BATCH READING / 5 RUNS PER GROUP
Repeated reads.
Fewer agent steps.
Scan four Fiction pages and 65 books, select the three cheapest, then return price, UPC, stock, and links.
All five WebMCP runs chose the batch-detail tool, with no navigation calls. The baseline already allowed ad hoc browser scripts. Both groups delivered every required field.
Tasks with clear rules and batchable reads can benefit from turning repeated operations into reusable tools.
- Complete success
- 5/5 → 5/5
- Median tool calls
- 5 → 3
- Median total tokens
- 92,559 → 44,440
General browser → WebMCP · 2026-09-10
X / THREE FIXED POSTS
SIMPLE READING / 5 RUNS PER GROUP
Simple reading.
A tool for the task.
Read the author, date, full original text, and link of three fixed posts, excluding quoted posts, replies, and recommendations.
Both groups succeeded in all five runs. WebMCP completed every run without DevTools fallback; the baseline allowed ad hoc browser scripts.
WebMCP used 10.3% less time and 8.4% more total tokens. This simple task showed a time benefit; token savings still depend on tool calls and context.
- Complete success
- 5/5 → 5/5
- Median time
- 62.176s → 55.748s
- Median total tokens
- 134,038 → 145,301
- Median tool calls
- 7 → 8
General browser → WebMCP · 2026-09-12
HACKER NEWS / TEN DISCUSSIONS
STRUCTURED READING / 3 FINAL RUNS PER GROUP
Reading the words
is only part of the task.
Ten on-site discussions, up to ten valid comments each. Beyond the text, the task required identifying which comment each reply belongs to.
The baseline acquired pages only through Computer Use, with file notes and offline scripts allowed. The last two runs retrieved all target text, but left most parent relationships unverified: 0/3 complete tasks.
WebMCP returns comments, parent relationships, and exhaustion status, with on-site navigation. Across three runs, all 282 target comments had correct content, parent relationships, and character formatting.
COMPLETE TASK SUCCESSES
Parents among returned comments: baseline 55/220 verified, 165 unknown; WebMCP 282/282 verified. A further 80 baseline comments were not acquired.
Story IDs were fixed, but comments changed: 94 per WebMCP run, then 100 per baseline run. The baseline also encountered navigation, focus, and stale-element issues; all runs ended before the 30-minute limit. This reflects this Harness integration and execution, not a universal limit of Computer Use.
WHAT BECOMES POSSIBLE
Turn verified workflows
into a reusable capability.
The experiments point to an opportunity: less repeated exploration, and more direct access to the capabilities and information a task needs. The gains depend on how the tools are designed.
Batch repeated work
Combine repeated detail reads so the model can focus on selection and judgment.
Make task information explicit
Return relationships, order, and completion state so the result can be checked.
Carry only useful context
Discover tool definitions and read source on demand instead of carrying them through every step.
METHOD & EVIDENCE
Evidence, with boundaries.
These are observations from three specific read-only tasks. They demonstrate potential, not a promise of equal gains on every website.
Were scripts and notes allowed?
Book and X baselines allowed DevTools and temporary browser scripts. The final HN baseline acquired web content only through Computer Use, with file tools and offline processing allowed. The old no-notes baseline was superseded; direct Python HTTP fetching does not count as Computer Use.
How were success, time, and tokens measured?
Success requires every output field and execution condition, not just page visits or partial correctness. Time runs from task start to finish, excluding preparation and between-run lock-screen waits. Values are medians. Total tokens include cached reads and repeated input, not monetary cost.
Why no pooled success rate or overall speedup?
Baseline capabilities differ, so results are not pooled. Book and X used interleaved same-batch runs. HN is a historical comparison with live comments and incomplete baseline output, so it is not an equal-work speedup. Samples are only 3–5 runs per group; cache and service load were not fully controlled. No statistical significance is claimed.
Which results are outside the main comparison?
X Following had empty pages and mismatched samples, so it does not support an efficiency comparison or prove anti-bot interference. Interrupted HN note-taking trials, the no-notes baseline, and direct HTTP scripts are kept separate. One historical Book build trial is insufficient for a reuse break-even claim. All tasks were read-only; writing actions and arbitrary desktop software were not evaluated.
Explore the data behind the results.
Download the full Chinese report and public summary data for all three experiments and their comparison conditions.
EXPLORE WITH DEEPDECK