Researcher Survey Reveals Widespread Noncompliance With AI Bans in Peer Review

A Microsoft Research experiment at the 2026 International Conference on Machine Learning found that papers reviewed under strict AI bans and permissive AI policies had nearly identical acceptance rates and quality assessments. An anonymous survey of reviewers revealed that 22.5 percent of those instructed not to use AI tools admitted to using large language models anyway. The finding suggests that AI restrictions in scientific review processes have minimal practical impact when enforcement mechanisms are absent.

The Microsoft Research team examined nearly 25,000 submissions to a major machine learning conference, randomly assigning papers to reviewers operating under different AI usage rules. The results showed virtually no difference in outcomes: acceptance rates hovered around 27 percent regardless of policy, and review quality metrics remained nearly identical. However, automated text analysis suggested that roughly half of reviews submitted under the strict ban still contained AI-generated content, indicating non-compliance may have been even more widespread than reviewers self-reported.

The study identified workload pressures and ambiguous guidelines as key factors driving violations. Reviewers admitted using language models for multiple tasks including brainstorming, summarizing papers, and drafting feedback—activities ostensibly forbidden under the conservative policy. These findings underscore the practical challenges of policing tool usage in distributed, volunteer-dependent peer review systems.