Google deployed an agentic AI peer-reviewer at ICML and STOC that reviewed approximately 10,000 papers with a 30-minute turnaround. A new research paper shows it detects 34% more mathematical errors than zero-shot prompting. This demonstrates a scalable, automated approach to scientific peer review, potentially reducing review bottlenecks and improving error detection. It sets a precedent for AI-assisted review at conference scale, which could impact how research is evaluated. The system uses an agentic framework that iteratively reasons about papers, combining retrieval and verification steps. It achieved the 34% improvement over baseline zero-shot prompting, which asks the model to review without additional guidance.
Background
Peer review at large computer science conferences like ICML and STOC traditionally involves hundreds of human reviewers and can take weeks. Zero-shot prompting uses a large language model to answer tasks without task-specific examples. Agentic AI systems are designed to autonomously plan and execute multi-step tasks, such as fetching paper details, verifying mathematical claims, and synthesizing feedback.