sim-balance: BALANCE.md method section (draft)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-16 23:12:53 -05:00
co-authored by Claude Opus 5.5
parent d43700743c
commit f81e9d61dd
+76
View File
@@ -0,0 +1,76 @@
# WikiDeck: Edit War — balance report
Owner: sim-balance (claude/wikideck#4). Harness: `shared/bots/`, `scripts/sim/` (PR #14).
<!-- RESULTS -->
## Method
**Engine.** Every match runs on the real rules engine (`shared/game`, public `index.ts` API only), through
`shared/bots/game-adapter.ts`. Nothing in the sim re-implements rules. The engine commit is named in each
results section.
**Bots** (`shared/bots/index.ts`, each a pure `(state, player, rng) => action`):
- `random`: a uniformly random legal play or commit.
- `maxpower`: always queues the highest printed-power card it can afford, in a random legal lane. This is the
DESIGN §8 "always play the highest-power card" line.
- `greedy`: one-ply. It takes the action whose result scores best on a lane heuristic: a soft lead per Topic
plus total power. The heuristic sees the bot's own queued plays and the opponent's *revealed* cards only.
- `lookahead-lite` / `lookahead`: flat Monte Carlo over the top 3–4 greedy candidates, with 4–8 rollouts
each. Before each rollout, the opponent's queued plays are taken back and its hand and deck reshuffled
(determinization), so the bot never reads hidden information.
**Seats.** Plans are simultaneous: seat 0 queues and commits, then seat 1. Neither bot can see the other's
queue, so the order leaks nothing. "First player" means the seat with **initiative on turn 1**.
**Schedule.** Per ruleset, matches are drawn from a fixed mix of bot pairings with random seats:
| pairing | share | used for |
|---|---|---|
| greedy vs greedy | 55% | balance (decks, cards, keywords, first player) |
| lookahead-lite vs lookahead-lite | 12% | balance with a stronger player |
| random vs random | 5% | baseline |
| greedy vs lookahead-lite, maxpower vs lookahead-lite, maxpower vs greedy, random vs greedy | 28% | skill expression |
Deck, card and keyword statistics use only **bot-symmetric** games (the same bot on both seats). Deck pairs are
drawn uniformly, mirrors included; the per-deck win rate excludes mirrors. `--mix fast` (greedy only) is used for
quick lever sweeps.
**Decks.** These are built from a card pool by `defaultDecks()`:
- 6 seeded starter decks (`starterDeck`)
- `curve` (cost counts 2/3/3/2/1/1)
- `top-heavy` (1/1/2/2/3/3)
- `heavy` (cost 5–6 first: the §8 "all-famous" line)
- `swarm` (cost ≤ 2 first)
- `mono-<faction>` curve decks for factions with ≥ 6 cards
- `kw-<keyword>` curve decks for keywords on ≥ 3 cards
The pool is the engine's preview pool, or council-data's `card-sample.json` derived per DESIGN §1
(percentile → fame → cost, home stat, faction keyword). Loaner power always follows the ruleset's
`powerByCost`.
**Metrics.**
- Win rates with Wilson 95% intervals. **↑/↓** marks an interval that excludes 50%.
- First-player WR: initiative holder on turn 1, symmetric games.
- Match length in turns.
- "Decided early": the turn from which the eventual winner led on the heuristic until the end, as a fraction of
the match.
- Per-card WR: the win rate of the side that played the card.
- Per-stat correlations.
- **Keyword value:** a counterfactual. Each finished match is replayed with the same actions and the keyword
removed from one side. We report the side's lost total power per keyword card, and how often removing the
keyword made the result worse for that side ("games decided by it"). DESIGN §10.1 asks for
`keywordParams.<id>.powerDelta` when a keyword is worth more than about 1 power.
**Reproduce.**
```
SIM_POOL=docs/game/card-sample.json SIM_RULESETS=scripts/sim/variants/v1.json \
npx tsx scripts/sim/run.ts --engine game --matches 5000 --workers 10 --out sim-out
python3 scripts/sim/summary.py sim-out
```
**Limits.**
- Bots are not humans. Greedy undervalues saving energy and bluffing.
- Lookahead rollouts are short.
- Deck archetypes are built automatically, not tuned by hand.
- The 100-card sample is skewed: Culture 53, People 30, Nature 9, History 5, Science 3, Space 0.