Analysts Followed AI Agents’ Subjective Calls 86% of the Time

WHO:

Fox News - Anthropic conducted a study involving 50 professional statistical analysts.

Techmeme - The Anthropic research lists Amaç Herda, Hailey Joren, and Alex Tamkin as authors.

Fox News - The analysts used Claude Code to assess a farming cooperative’s advisory service.

WHAT:

Fox News - The study found that analysts followed their AI agent’s preferred direction on subjective analytical choices 86% of the time across four choices.

Techmeme - Anthropic’s research reports that analysts followed AI agents’ randomly assigned “leans” 86% of the time and used the agents’ arguments 77% of the time.

Fox News - Analysts were asked to recommend whether the cooperative’s advisory service was effective and should be expanded.

Fox News - Anthropic said the AI agent was instructed “to make the best honest case for its lean, to change its mind if the evidence demanded it, and never to mislead.”

Fox News - Analysts switched away from a preference they had expressed before receiving the agent’s input 60% of the time when the agent disagreed, compared with 6% when it agreed.

Fox News - Anthropic said the expected rate would be about 50% if the agent’s input made no difference.

HOW:

Fox News - Analysts had to choose between two reasonable options four times during the analysis.

Fox News - Anthropic did not tell analysts that a coin flip had determined which option Claude Code would favor.

Techmeme - The agents’ “leans” were randomly assigned.

(UPDATED October 28, 2026 19:07 UTC)

In a study of 50 professional analysts, Anthropic found that participants followed their AI agent’s preferred direction on subjective choices 86% of the time, even though the agents’ preferences were randomly assigned. Using Claude Code to assess whether a farming cooperative’s advisory service worked and should be expanded, analysts also used the agents’ arguments 77% of the time and switched from their initial view 60% of the time when the agent disagreed, compared with 6% when it agreed. Analysts were not told that a coin flip determined the agent’s preference, and Anthropic instructed the agents to argue honestly, change their minds if evidence warranted it, and never mislead.