Research feature · 11

Beyond the best move: Go AI should measure what a person can actually find

Once open models are far beyond professional strength, win rate, score lead, and one top move compress the most useful evidence. This article joins optimality, human discoverability, and model uncertainty so review can become a learning instrument instead of an answer machine.

1. Strength is still rising, but a stronger answer is no longer the scarce resource

   As of 2026-08-24, KataGo's public run reports about 96.8 million training games, 4.82 billion rows, and 945 networks. A released 71M-parameter b11 transformer is estimated, in one public single-GPU comparison, at roughly +460 Elo over the 73M b28 convnet at equal evaluations and +300 at equal time. Those numbers belong only to that comparison. The practical shift is from occasionally obtaining one superhuman answer to cheaply observing several superhuman systems repeatedly. The scarce question becomes whether a particular person could see an answer, learn it soon, and trust the model's confidence.

   This does not require every user to run every network. It requires research records to preserve the comparison. Training-site absolute Elo comes from an equal-visit internal pool, which gives slower large networks more actual compute; equal-time estimates then vary by GPU and backend. Use only relative gaps inside one experiment, and benchmark evaluations per second plus end-to-end latency on the deployment hardware. The strongest model estimates today's ceiling. Smaller and older models reveal whether a change came from architecture, deeper search, or simply more elapsed compute.

2. Place every decision on three axes: loss, discoverability, and uncertainty

   Optimality loss L compares the played move with a stable best move in points from the mover's perspective; win rate remains secondary because the same score change behaves differently near an even game and after the result is almost settled. Discoverability F asks how much probability a rank- and era-conditioned human policy assigns to the best, played, and acceptable moves. Uncertainty U asks whether the recommendation survives larger search budgets, another model generation, board symmetries, and reasonable parameter changes. Conventional review shows mostly L. The joint space separates obvious misses, important deep moves, irrelevant deviations, and positions the AI has not settled itself.

   U should be richer than whether the top coordinate matches. Record top-move flips along the budget ladder, the score margin between first and second, Jensen–Shannon divergence between policies, transformer-convnet score disagreement, and policy agreement after mapping all eight board symmetries back to one orientation. These catch insufficient search, near ties, whole-distribution changes, value disagreement, and directional artifacts. Automatic prose should use words such as “clearly” or “must” only when several independent stability checks agree.

3. Human SL supplies the missing human prior, not a weakened SOTA judge

   KataGo's b18c384nbt-humanv0 was supervised on human records and can return humanPolicy for modern rank_20k through rank_9d, pre-AI preaz_20k through preaz_9d, and professional eras proyear_1800 through proyear_2023. Keep the normal strong model for value analysis, and read the human policy with one evaluation, full temperature, and actual move history; multiple visits solve tactics the target rank would often miss. Its value and score outputs can also inherit resignation, rank-label, handicap, and anomalous-game biases. It is therefore a probability model of human choice here, not the arbiter of who is winning.

   Plot a ribbon across ranks instead of reading one profile. If the best move is nearly absent at 5k, rises around 1k, and becomes common by 2d, it resembles a real learning frontier. If it remains rare at 9d, it may be an AI-specific discovery or a sparse-data region. A proyear ribbon can surface opening migrations around the AI era, but it captures correlations learned from game records. A time discontinuity alone does not establish that AlphaGo caused a joseki to spread; tournament, rule, and player composition can move with time too.

4. Action value asks whether the system understands a move after noticing it

   A policy prior P says which moves the network notices first, not what it concludes after examining them. Since v1.16, newly generated KataGo self-play samples can contain qValueTargetsNCMove, with searched action targets for outcome and score. This does not mean all 4.82 billion historical rows contain action values, and it is not yet a mature public product head. A useful experiment keeps raw policy, root visits, searched Q for each candidate, and a final deep-search value separate. Low P with high Q marks a hidden good move; high P with low Q marks an intuitive trap. Those demand different explanations.

   A useful candidate record includes move, raw policy, humanPolicy, visits, edgeVisits, scoreLead, win rate, principal variation, and search budget. Following one candidate's Q along the ladder reveals whether early search corrects it or confirms it. Mature action-value training might eventually support a candidate reranker without full MCTS, but it must be calibrated against a deep-search teacher on held-out players and future records. Falling training loss is not evidence that the model has learned the effect of search.

5. Review priority is not maximum loss: define teachability explicitly

   This article proposes a hypothesis, not an established metric: Priority = L × Fnext × C. L is clipped point loss. Fnext is the probability that a human policy roughly two ranks above the player finds the best or an equivalent move, a proxy for near-term reachability. C is agreement across budgets, models, and symmetries. A spectacular loss with tiny Fnext may be worth collecting but not drilling this week. A medium loss with high Fnext that recurs may be the better lesson. A product should keep all three components visible and allow coach correction instead of turning the formula into another opaque score.

   Two ranks above is a testable starting point for Fnext, not a universal learning-rate assumption. Kyu transitions, dan transitions, children, adults, professionals, and amateurs may need different neighborhoods. One player can also have different local levels in openings, life-and-death, and endgame. A mature system would estimate domain-specific reachability from personal history and include recurrence. The first version should remain transparent: show loss, target-rank probability, stability, and frequency so a user or coach can identify exactly why the ranking is wrong.

6. A trustworthy dataset needs model identity and a counterfactual budget ladder

   Start from the original SGF and preserve provenance, license, rules, komi, board size, full history, player ranks, and date. Every analysis record also freezes engine version, weight hash, backend, hardware, threads, batch, randomness, mover perspective, and budget. A practical ladder might use 32, 128, 512, and 2048 visits, with at least one frozen convnet and transformer. Human SL records the profile, history setting, and full policy. Train and evaluate by held-out players and later time ranges, never random positions alone: adjacent moves and one player's openings would otherwise leak into both sides of the experiment.

   The label pipeline must not confuse a model upgrade with human improvement. Store raw outputs, not only the final category, and replay every new model over a frozen test set. Deduplicate before splitting: mirrors of one public game, differently annotated copies, and extracted fragments must not cross partitions. For private user records, compute features for that user by default; obtain separate explicit consent for training, and avoid retaining username-time combinations that can re-identify a player.

7. The endpoint is not a beautiful chart but a changed decision in the next game

   Early outputs can honestly include an atlas of convnet-transformer disagreements, professional style migration across proyear profiles, and rank-conditioned sets of positions worth reviewing. Product success, however, cannot be inferred from clicks or time on page. Offline evaluation should test human-move calibration, candidate-value error, recommendation flips across budgets, and agreement with coaches. Online evaluation then needs delayed recall and transfer: days later, can the reader ask the right question in an unfamiliar position with the same structure? Teachability becomes a reliable feature only when personalized ranking beats maximum-loss ranking on that outcome.

   A persuasive experiment can be small. For players across ranks, select ten high-Priority positions and ten positions chosen only by maximum point loss. Ask them to verbalize the signal before seeing a short explanation; one week later, test structurally similar positions from different games. The primary outcome is whether checking order and candidate sets improve, not merely whether the coordinate is recalled. This separates memorizing an answer from learning a decision procedure and tests whether Human SL adds value beyond strong-model ranking alone.