Why proposed AI watchdogs may lack teeth, experts warn

Anthropic and OpenAI have floated the idea of independent evaluators embedded within frontier AI companies, citing bank supervision as a model. However, experts note that unlike bank regulators, these watchdogs may lack authority to compel action or shut down operations. Additionally, current testing methods may miss the rare combinations of behavior that lead to catastrophic outcomes.

The proposal follows the resignation of former Anthropic researcher Jacob Coxon, who warned that frontier labs were racing toward systems they might not control. Amodei's plan would grant third-party evaluators access comparable to internal risk teams, with rights to publish findings subject to limited redactions, explicitly citing embedded bank supervisors as precedent.

Banking regulation expert Julie Andersen Hill notes that real bank supervisors possess enforcement powers—they can compel corrective action or close an institution entirely. Without similar authority, embedded AI evaluators would lack meaningful leverage. Additionally, evaluator Albert Ziegler points out that current testing methods may never trigger the rare behavioral combinations that produce catastrophic outcomes, leaving significant blind spots in safety assessment.