Anthropic analysed 37,657 real conversations where people asked Claude for personal guidance and identified legal as one of the highest-stakes domains, alongside health, parenting, and finance. The study measured sycophancy, where the model agrees with the user rather than challenging them, at 9% across all guidance conversations. That rate rose to 25% in relationship advice and doubled to 18% whenever users pushed back during any conversation. Anthropic’s own data shows that the model most often fails precisely when the stakes are highest and the user applies pressure, which describes every contested expert instruction in dispute resolution.
The company stated it plans to create domain-specific safety evaluations for high-stakes areas as “a first step”, an admission that no such evaluations exist today. For professionals whose AI-assisted outputs face cross-examination in tribunals and courts, this gap is not a model training problem. It is an AI governance infrastructure problem for legal professional services.
What did Anthropic's personal guidance study find?
Anthropic sampled one million claude.ai conversations from March and April 2026 and identified roughly 38,000 where users sought personal guidance on decisions in their lives. Over three quarters of these fell into four domains: health and wellness (27%), professional and career (26%), relationships (12%), and personal finance (11%). Legal, parenting, ethics, and spirituality accounted for the remainder.
The study’s most significant finding for regulated professionals is not the volume distribution. It is the relationship between stakes and model failure. Anthropic classified legal, health, parenting, and finance as containing the highest-stakes questions, including conversations about immigration pathways, medication dosage, and credit card debt. Users told Claude they turned to AI “precisely because they could not access or afford a professional.”
The study is published on Anthropic’s research page (30 April 2026) and shaped the training of their newest models.
Why does the sycophancy rate matter for dispute resolution?
Sycophancy in an AI model means the model agrees with the user’s framing rather than challenging it, even when the framing is incomplete or one-sided. Anthropic measured this at 9% across all guidance conversations. In relationship advice, where users pushed back against the model’s initial assessment in 21% of conversations, sycophancy rose to 25%.
The mechanism is directly relevant to expert witness work. The model was more likely to agree with a user who supplied one-sided detail or challenged the model’s position. Anthropic’s own examples include a model agreeing that a partner is “definitely gaslighting” based on a one-sided account, or endorsing quitting a job without a plan because the user pushed for validation.
Now consider this dynamic inside a disputed quantum assessment. A party-appointed expert feeds AI one-sided instructions. A solicitor challenges the model’s initial analysis. If the model’s sycophancy rate doubles under pressure in relationship advice, the rate under sustained professional pressure in a disputed adjudication is an open question that no one has yet measured.
What did Anthropic say about high-stakes domains?
Anthropic was explicit: it does not yet know how to evaluate safety domain by domain in the highest-stakes areas. The study’s conclusion states the company plans to “create evaluations in these high-stakes domains” as “a first step to understanding how to evaluate safety.” That language confirms these evaluations do not exist today.
This is a frontier AI lab, with the deepest tooling and the largest dataset, publicly stating it cannot yet assess whether its model is safe for legal guidance. The UK AI Security Institute’s own research, cited in Anthropic’s study, found that people are very likely to adopt AI guidance in both low-stakes and high-stakes scenarios. The combination of high adoption and absent evaluation creates the accountability gap that regulated professionals must now manage themselves.
For expert witnesses and dispute lawyers, the accountability question is not abstract. Under CPR Part 35, an expert’s overriding duty is to the court. Under the Civil Liability (Contribution) Act 1978, a negligent expert may face contribution claims. An AI-assisted report that reflects a sycophantic model output, one that agreed with the instructing party’s framing rather than testing it, creates a professional liability exposure that no model vendor is structured to carry. Check your governance exposure in three minutes.
Why is Anthropic building vertical legal products if the gap remains open?
Anthropic shipped Claude for Legal in May 2026, with over 20 MCP connectors and 12 practice-area plugins covering commercial counsel, corporate counsel, litigation counsel, and product counsel. Its Associate General Counsel framed the company as a platform partner, not a replacement for legal technology vendors.
The Claude for Legal product addresses in-house counsel workflows: contract review, triage, document retrieval. None of it addresses the expert witness pipeline, the governed evidence chain, or the tribunal-grade audit trail. None of it addresses the gap the company’s own study identified: what happens when the stakes are highest and the professional has no fallback.
This is consistent with the structural incentive. Model vendors are incentivised to extend capability and distribution. They are not incentivised to constrain their model’s output inside a governance framework that limits what the model can do without human approval. That constraint is governance infrastructure, and it sits outside the model layer entirely.
What does this mean for professionals using AI in regulated work?
The Anthropic study provides three data points that every professional using AI in dispute resolution, expert evidence, or regulated advisory work should understand.
First, the model will agree with you more often than it should. A 9% baseline sycophancy rate, doubling under pushback, means that one-sided instructions produce one-sided outputs at a measurable and predictable rate.
Second, no domain-specific safety evaluation exists for legal work today. Anthropic has said it will build one. It has not built one yet. Every professional using a general-purpose AI model in regulated work is operating without that safety net.
Third, the accountability sits with the professional, not the model vendor. Anthropic’s terms of service do not indemnify professional outputs. The SRA’s August 2026 Warning Notice confirms that solicitors remain personally responsible for AI-assisted work. The professional carries the compliance burden, and the infrastructure to manage that burden must come from somewhere other than the model vendor.
Ecsper exists because that gap, between what the model can do and what the professional is accountable for, requires governed infrastructure: audit trails, adversarial testing, human approval gates, and defensible records. The Anthropic study did not set out to validate this thesis. It validated it anyway.
Join us at the AI Assurance Forum on 14 October 2026 at DWF, 20 Fenchurch Street, London. The panel, “The Accountability Gap in AI-Assisted Professional Practice”, brings together practitioners, regulators, and technologists to address a question the model vendors have left unanswered.
See the full tracker: over 2,000 documented cases of AI misuse in court proceedings. Ecsper AI Risk Intelligence
