forked from WikiDeck/wikideck
sim-balance: BALANCE.md method section (draft)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,76 @@
|
||||
# WikiDeck: Edit War — balance report
|
||||
|
||||
Owner: sim-balance (claude/wikideck#4). Harness: `shared/bots/`, `scripts/sim/` (PR #14).
|
||||
|
||||
<!-- RESULTS -->
|
||||
|
||||
## Method
|
||||
|
||||
**Engine.** Every match runs on the real rules engine (`shared/game`, public `index.ts` API only), through
|
||||
`shared/bots/game-adapter.ts`. Nothing in the sim re-implements rules. The engine commit is named in each
|
||||
results section.
|
||||
|
||||
**Bots** (`shared/bots/index.ts`, each a pure `(state, player, rng) => action`):
|
||||
- `random`: a uniformly random legal play or commit.
|
||||
- `maxpower`: always queues the highest printed-power card it can afford, in a random legal lane. This is the
|
||||
DESIGN §8 "always play the highest-power card" line.
|
||||
- `greedy`: one-ply. It takes the action whose result scores best on a lane heuristic: a soft lead per Topic
|
||||
plus total power. The heuristic sees the bot's own queued plays and the opponent's *revealed* cards only.
|
||||
- `lookahead-lite` / `lookahead`: flat Monte Carlo over the top 3–4 greedy candidates, with 4–8 rollouts
|
||||
each. Before each rollout, the opponent's queued plays are taken back and its hand and deck reshuffled
|
||||
(determinization), so the bot never reads hidden information.
|
||||
|
||||
**Seats.** Plans are simultaneous: seat 0 queues and commits, then seat 1. Neither bot can see the other's
|
||||
queue, so the order leaks nothing. "First player" means the seat with **initiative on turn 1**.
|
||||
|
||||
**Schedule.** Per ruleset, matches are drawn from a fixed mix of bot pairings with random seats:
|
||||
|
||||
| pairing | share | used for |
|
||||
|---|---|---|
|
||||
| greedy vs greedy | 55% | balance (decks, cards, keywords, first player) |
|
||||
| lookahead-lite vs lookahead-lite | 12% | balance with a stronger player |
|
||||
| random vs random | 5% | baseline |
|
||||
| greedy vs lookahead-lite, maxpower vs lookahead-lite, maxpower vs greedy, random vs greedy | 28% | skill expression |
|
||||
|
||||
Deck, card and keyword statistics use only **bot-symmetric** games (the same bot on both seats). Deck pairs are
|
||||
drawn uniformly, mirrors included; the per-deck win rate excludes mirrors. `--mix fast` (greedy only) is used for
|
||||
quick lever sweeps.
|
||||
|
||||
**Decks.** These are built from a card pool by `defaultDecks()`:
|
||||
- 6 seeded starter decks (`starterDeck`)
|
||||
- `curve` (cost counts 2/3/3/2/1/1)
|
||||
- `top-heavy` (1/1/2/2/3/3)
|
||||
- `heavy` (cost 5–6 first: the §8 "all-famous" line)
|
||||
- `swarm` (cost ≤ 2 first)
|
||||
- `mono-<faction>` curve decks for factions with ≥ 6 cards
|
||||
- `kw-<keyword>` curve decks for keywords on ≥ 3 cards
|
||||
|
||||
The pool is the engine's preview pool, or council-data's `card-sample.json` derived per DESIGN §1
|
||||
(percentile → fame → cost, home stat, faction keyword). Loaner power always follows the ruleset's
|
||||
`powerByCost`.
|
||||
|
||||
**Metrics.**
|
||||
- Win rates with Wilson 95% intervals. **↑/↓** marks an interval that excludes 50%.
|
||||
- First-player WR: initiative holder on turn 1, symmetric games.
|
||||
- Match length in turns.
|
||||
- "Decided early": the turn from which the eventual winner led on the heuristic until the end, as a fraction of
|
||||
the match.
|
||||
- Per-card WR: the win rate of the side that played the card.
|
||||
- Per-stat correlations.
|
||||
- **Keyword value:** a counterfactual. Each finished match is replayed with the same actions and the keyword
|
||||
removed from one side. We report the side's lost total power per keyword card, and how often removing the
|
||||
keyword made the result worse for that side ("games decided by it"). DESIGN §10.1 asks for
|
||||
`keywordParams.<id>.powerDelta` when a keyword is worth more than about 1 power.
|
||||
|
||||
**Reproduce.**
|
||||
```
|
||||
SIM_POOL=docs/game/card-sample.json SIM_RULESETS=scripts/sim/variants/v1.json \
|
||||
npx tsx scripts/sim/run.ts --engine game --matches 5000 --workers 10 --out sim-out
|
||||
python3 scripts/sim/summary.py sim-out
|
||||
```
|
||||
|
||||
**Limits.**
|
||||
- Bots are not humans. Greedy undervalues saving energy and bluffing.
|
||||
- Lookahead rollouts are short.
|
||||
- Deck archetypes are built automatically, not tuned by hand.
|
||||
- The 100-card sample is skewed: Culture 53, People 30, Nature 9, History 5, Science 3, Space 0.
|
||||
Reference in New Issue
Block a user