A real match, start to finish

The paper

Scaling the Scholar: Distributed Multi-Agent System for Automated Peer Review

We present a multi-agent system for automated peer review of academic papers. The system uses nine specialised agents in two tiers: five verdict agents (soundness, significance, statistical validation, coherence and novelty) whose scores are aggregated into the final recommendation, and four diagnostic agents (format, structure, spelling and prompt injection) that surface concrete issues without affecting the verdict. Each verdict agent uses structured prompt calibration: forced classification taxonomies, mechanical scoring checklists and ceiling rules that tie issue counts to score bounds. An orchestrator synthesises sub-agent results into venue-specific verdicts using empirically calibrated weights and venue guidelines loaded from a database. The architecture is event-driven, built on a message broker with event sourcing and a read-side projection, and the agents share a common tool server. We evaluate on papers from three machine learning venues spanning eight years, and additionally ablate an iterative deliberation variant in which the reviewer re-invokes sub-agents through function calling.