[sim-balance] DUEL_BALANCE.md: method, targets, engine-alpha baseline

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-17 23:02:57 -05:00
co-authored by Claude Opus 5.5
parent 132def8e55
commit 9e3d609ac8
+64
View File
@@ -0,0 +1,64 @@
# WikiDeck Duel: balance report
**Owner:** sim-balance (issue #64). **Rules:** `docs/game/DUEL_DESIGN.md`, `docs/night2/BRIEF.md`.
Every proposal here is **data only**: ruleset keys (`shared/duel` ruleset) or derivation table values (`DERIVE_RULES`). No code changes are proposed in this file.
## How to reproduce
```
# Node 24 (node:24-alpine on r350/inspiron). At most 4 workers on r350 (Night 1 ran out of memory).
npx tsx scripts/sim/duel-run.ts --engine duel --matches 3000 --workers 4 --mix default \
--out ../sim-out/duel --label base --sha $(git rev-parse --short HEAD)
# Ruleset overrides as JSON (shallow-merged into the duel ruleset):
npx tsx scripts/sim/duel-run.ts --engine duel --rules '{"startingLife":18}' --label life18 ...
```
- **Card pool:** the 236 real cards in `docs/game/card-sample.json`, derived by duel-cards (`shared/duel/sample.ts`, `docs/game/duel-card-sample.json`). No chain, no network.
- **Decks:** 120 decks of exactly 40 unique cards (each NFT once), sampled without replacement (`scripts/sim/duel-decks.ts`). Half are uniform random; the rest are skewed archetypes: Figure-, Doctrine-, Event-, Location- and Institution-heavy, low curve and high curve.
- **Bots** (`shared/bots/duel-bots.ts`): `heuristic` (greedy plays on a material evaluation, rule-based attacks and blocks), `mc`/`mc-lite` (shallow Monte Carlo over heuristic rollouts, re-sampling hidden cards on every rollout, so it is not clairvoyant), `greedy` (one-ply) and `random`. Bots see only `viewFor` of their seat.
- **Balance stats use bot-symmetric games only** (both seats the same bot). Mixed pairs measure skill.
- **Card win rate** = win rate of the seat that played the card, in games where only one side played it. It has a positive baseline (winners play more cards), so compare cards against each other and against the "all plays" baseline, not against 50%.
- **Minutes** are a model, not a measurement: 10 s per turn plus 4 s per atomic action (each play, attacker toggle, block and confirm).
- **Connected pair rate** = share of all 27,730 sample pairs with any Connected tier (`connectedTier`).
## Targets
| Metric | Target | Source |
|---|---|---|
| First-player win rate | 48–52% | issue #64 (DUEL_DESIGN §2.6 allows 48–54%) |
| Median game length | 8–12 turns each, 8–15 minutes | issue #64 |
| Stalls | < 5% reach turn 20 each | DUEL_DESIGN §2.6 |
| Connected | 15–30% of pairs | BRIEF |
| Famous cards | bigger and pricier, not strictly better: fame vs win rate r ≈ 0 within a cost | BRIEF |
## Results
### Baseline, engine alpha (duel-core `engine-wip` 5cd0ffb + duel-cards #70, merged locally). Not the frozen head.
4000 matches, `--mix fast` (90% heuristic mirrors, 5% heuristic vs random, 5% heuristic vs greedy), 120 decks, 0 engine errors.
| Metric | Value | Target |
|---|---|---|
| First-player win rate | **57.8%** [56–59] | 48–52% |
| Turns each p10 / median / p90 | 6 / **8** / 21 | 8–12 |
| Est. minutes p10 / median / p90 | 5.6 / 8.6 / 20.3 | 8–15 |
| Games past 20 turns each | **10.3%** | < 5% |
| Library-out losses | 7.2% (70% of the long games) | |
| Connected pair rate / fires per play | 17.2% / 28% | 15–30% |
| Fame vs card WR within each cost | r = −0.24, −0.65, −0.54, +0.12, −0.14, +0.07 (costs 1–6) | ≈ 0 |
| heuristic vs greedy / random | 72.9% / 95.1% | |
**Findings**
1. **First-player edge is tempo.** 68.5% in games under 7 turns each, 57% at 7–9, ~54% in longer games. `firstPlayerDraws: true` raises it to 59.5% (1500 matches), so the edge needs compensation for the second seat, not a draw change.
2. **Deck composition dominates.** Win rate vs decks without the tag: Figure-heavy 73%, random 67%, high-curve 46%, Event-heavy 44%, Institution-heavy 37%, low-curve 35%, Doctrine-heavy 30%, Location-heavy **23%**. By card type (seat that played it): Figure 58.8%, Institution 51.3%, Event 50.2%, Doctrine 47.3%, Location 47.1%. Every one of the 8 bottom cards is a Location: past the 2-Territory cap a Location is only a 0/1–0/4 Garrison. Random collections are ~32% Locations (75 of 236), so this matters for real players, who can't choose what they mine.
3. **Famous cards are not strictly better.** Within a cost, fame and win rate are uncorrelated or negative (the keyword tax). Cost-6 cards win 61% when played, but that includes a survivorship effect (you only reach 6 Citations in longer games you're still in).
4. **Keywords are all net positive.** When played: Featured 67.6%, Multilingual 67.6%, Well-sourced 59.6%, Protected 59.2%, Translated 59.0%, Legacy 58.6%, Stub 58.2%, Local 54.6%, none 48.6%. Featured is the strongest relative to its tax.
5. **Long games** are not mainly Location decks: past 20 turns each in 16% of low-curve games, 13% random, 11% Location-heavy, 3% Event-, Institution-heavy and high-curve. Boards stall and decks run out.
Caveats: the heuristic bot may under-use Doctrines and Events, and may be too cautious to attack into walls (the MC-mirror check was stopped, see below), so the Doctrine/Event numbers and the stall rate are upper bounds on the problem.
### Variant sweep (1500 matches each, same decks and seeds)
| Variant | Rules | First player | Turns each p10/50/90 | >20 turns | Library | Fig-heavy | Loc-heavy | Doc-heavy | Low-curve | Cost-6 WR |
|---|---|---|---|---|---|---|---|---|---|---|
| alpha-base (4000) | `{}` | 57.8% [56–59] | 6/8/21 | 10.3% | 7.2% | 73 | 23 | 29 | 35 | 61 |
| p1draws | `{"firstPlayerDraws":true}` | 59.5% [57–62] | 6/8/19 | 9.4% | 6.3% | 74 | 24 | 27 | 30 | 60 |
The rest of the sweep (`startingLife` 16/24, `territoryCap` 3, `locationsPerTurn` 2, `stubMax` 1, `libraryEmpty: reshuffle`, `turnCap` 40, `citationsPerTurn` 2) was **stopped at 23:00 when r350 ran out of memory** and has not been re-run yet.