Worst-case research · 25
Why can superhuman Go AI lose to seemingly weak play?
An ICML adversarial policy beat KataGo at superhuman settings in more than 97% of games—not by playing stronger Go, but by inducing systematic mistakes. Average strength and worst-case robustness are separate properties.
Superhuman average play does not cover every distribution
Ordinary self-play optimizes common game distributions; adversarial training actively seeks rare, repeatable failures. The measurements answer average strength and worst-case robustness separately.
Ordinary and adversarial trajectories visit different regions of the state space. The cyclic attack exploits a global state mistake
The public attack builds a large cyclic group whose life, death, or final value is misjudged while the adversary scores elsewhere. Our teaching board explains structure without supplying a move-by-move exploit script.
Read the central cycle and outside territory together rather than only local captures. The 97% result is not a universal human win rate
It belongs to a trained adversarial policy against specified superhuman victims. It does not imply any person beats any current KataGo, although the paper reports that human experts could understand and execute the core strategy.
Victim version, visits, and adversary identity sit beside the experimental win rate. Patching a known exploit invites adaptive attacks
AAAI 2025 tests hand-built adversarial positions, iterated adversarial training, and architecture changes. Some stop old attacks, but none withstand adversaries retrained against the defense.
A fixed-attack gate and an adaptive-attack gate are different tests. Rules can be part of the failure mechanism
The public gift attack uses positional superko to block an expected recapture. Robustness testing therefore treats rules as input; the same failure line may disappear under another ko rule.
One future recapture is open under one ko rule and closed under another. Report normal and worst-case performance together
Ordinary self-play Elo can hide rare failures; an attack win rate alone can exaggerate everyday danger. A release card needs normal strength, a fixed attack suite, and adaptive red-team results.
A three-column scorecard resists collapsing capability into one number. Treat blind spots as a trust boundary
Analysis tools should surface uncertainty around cycles, very long life-and-death, and rule-sensitive states, with model switching, deeper search, and human review. Superhuman AI still needs a way to abstain.
The final board marks shapes that deserve a second opinion and the audit path.
Peer-reviewed research
When reading an AI blind-spot claim
- Separate average and worst case
- Identify victim version and visits
- Put rules in the test
- Separate fixed and adaptive attacks
- Do not extrapolate to every model