What Anthropic’s 26% tells us about AI research

Anthropic says Claude was leading 26% of its measured AI research and development work as of August. It published the figure on September 17 as part of a proposal for greater visibility into activity inside frontier labs.

“Leading” has a particular meaning in the index: completing most of a task from a high-level prompt while a person supervises. Anthropic reported no measured category operating fully autonomously. More than 90% of the measured work involved at least substantial collaboration with Claude.

This is a more useful disclosure than a general claim that researchers are becoming more productive. It also takes some care to interpret.

A share of work is not a speed multiplier

The index organises research activity into a basket of tasks and weights it using an approximation of person-time. Anthropic then rates how much AI participates in each category. The company uses Claude in constructing and assessing the index, and acknowledges the difficulty of comparing it with other labs.

A rising share tells us that more of the measured work is being performed at a higher level of automation. It does not directly tell us how many additional experiments succeed, how much sooner a model is ready or how much human effort the organisation has saved. Those outcomes depend on what happens around the automated task.

Imagine a research team that becomes much faster at implementing experiments. If choosing promising experiments remains difficult, some of the saved time may produce a larger number of uninformative runs. If review becomes the bottleneck, the team may accumulate results faster than it can assess them. These are possibilities to investigate, not claims about Anthropic’s actual workflow.

Keep the denominator visible

A fixed basket helps make comparisons over time more meaningful: the measurement continues to refer to a defined set of work. It also creates a question about new work. As software takes on familiar tasks, people may move into activities the original basket does not represent well.

For another company building a similar measure, this argues for keeping two records. One follows automation of the original tasks. The other records changes in the work itself: new responsibilities, exceptions and effort spent reviewing outputs. Combining them too early can hide whether the business is doing the same work faster or doing a different mix of work.

There is a practical reason to preserve that distinction. A manager deciding where to invest needs to know which constraint has moved. If drafting becomes faster while checking remains slow, another improvement to drafting may have little effect on the completed output.

Anthropic’s disclosure gives outside readers a defined measure to question and potentially track. Our inference is that its most useful contribution may be the measurement discipline: name the task, define the degree of delegation and keep the human role visible.

The next informative comparison would put that index beside outcomes from the research process, with a method stable enough to compare across periods. That would help show how greater automation translates into progress on the work the lab is trying to complete.