Armature published research on how coding agents, programs that help write and change code, find and choose tools. The company measured 16,893 sessions with Claude Code, Codex and Cursor.
In the first published wave, the authors kept 5,292 sessions across 51 codebases in 18 sectors. All three agents picked the same tool in only 42% of the cells.
Claim check:
- Armature published research results on how coding agents find and choose tools. (confirmed by the publication itself: evidence; «Today we are sharing everything: aggregated results and leaderboards per category, but also every observation and even the entire traces with the user prompts, thinking traces and actual code diffs applied by the agent.»)
- Armature measured 16,893 sessions with Claude Code, Codex and Cursor. (confirmed by the publication itself: evidence; «Which tools do Claude Code, Codex and Cursor choose? We measured 16,893 sessions to find out.»)
- In the first published wave, the authors kept 5,292 sessions across 51 codebases in 18 sectors. (confirmed by the publication itself: evidence; «Out of these 16,893 runs, we started by keeping 5,292 sessions on 51 codebases and 18 sectors that we considered valid and ready to be published.»)
- All three agents picked the same tool in only 42% of the cells. (confirmed by the publication itself: evidence; «All three agents pick the same tool in only 42% of the cells»)
Publications:
Primary sources:
score 67.5 · kind research · revision 1 · stories st-8u0fhc