M9: Bot batch checks what holds between difficulties

`make bot-matches` failed on its default seeds because its difficulty checks
(easy places worst, hard takes the most eliminations) are noisier than the
differences they look for over 20 matches.

Measured on three 20-seed batches (seeds 1-20, 21-40, 101-120): easy bots won
0, 4 and 2 matches against an even share of 6.7, and took the fewest
eliminations every time. Mean place doesn't separate the levels, and hard is
not reliably ahead of normal by any whole-match measure.

The batch now checks that easy bots win less than an even share and take the
fewest eliminations, and skips the difficulty checks under 20 matches, where
they mean nothing. Hard against easy stays covered by the duel test.

Closes #76

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-12 08:26:02 -05:00
co-authored by Claude Opus 5.5
parent d1736035b0
commit 881f449d10
2 changed files with 21 additions and 10 deletions
+2 -2
View File
@@ -30,8 +30,8 @@ test-selfcheck: import
smoke: import
@tools/smoke.sh
## Plays 20 bot-only matches and checks each has a winner, few bots drown and
## difficulty shows. Takes several minutes, so it isn't part of `check`.
## Plays 20 bot-only matches and checks each has a winner, few bots drown and easy
## bots do worst. Takes about ten minutes, so it isn't part of `check`.
## BOT_MATCH_SEEDS and BOT_MATCH_FIRST change how many seeds and where they start.
bot-matches: import
@tools/bot_matches.sh