sim-balance: BALANCE.md report v1 with v1.1 recommendations

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-17 00:23:59 -05:00
co-authored by Claude Opus 5.5
parent b1d60b73c5
commit fc64277b38
3 changed files with 301 additions and 4 deletions
+291 -1
View File
@@ -2,7 +2,297 @@
Owner: sim-balance (claude/wikideck#4). Harness: `shared/bots/`, `scripts/sim/` (PR #14).
<!-- RESULTS -->
## Report v1: 2026-11-17 00:45 EST
**Engine:** `night/2026-11-16` @ 482de6c/4a4941c (engine #25 and #27, keywords and Portals; card data #28).
**Bots:** view-only (`viewFor`); see Method.
**Card pool:** `docs/game/card-sample.json` (236 cards), derived per §1 with percentiles from the sample.
**Decks:** the 4 §2.1 starter decks, 4 "realistic owned" decks (12 random cards from the sample), and archetypes.
**Runs:**
- full field: 3,000 matches per ruleset (report mix, 4 workers; r350 is memory-limited tonight)
- focused starters + owned round robin: 2,000 matches per ruleset
- greedy lever sweeps: 2,000–3,000 per variant
### TL;DR for the council
1. **Seat and initiative are fine:** 51.5% [49.6–53.5] for the turn-1 initiative holder (n = 2,421 symmetric games), and seat 0 51.2%. Draws are 0.4%. The tie-break (total power) decides 4.7% of games. *This full-field run started before the bot-side queue-count fix (PR #14); the effect is at most a 0.02 nudge to the heuristic. The focused starters run, after the fix, gives a borderline 53.4% [51.0–55.8] under v1 and **51.0% [48.6–53.4]** under the recommended v1.1. Initiative only matters through reveal order, and v1.1 removes most of the order-sensitive power.*
2. **Keyword values are lopsided, confirming the critic's figures within 0.2:**
| keyword | power per card |
|---|---|
| rising | +1.76 |
| translated | +1.53 |
| nostalgic | +1.45 |
| local | +0.65 |
| **viral** | **+0.19** |
| **encyclopedic** | **+0.19** |
Values are counterfactual replays. viral and encyclopedic are dead: raising viral to amount 2 / cap 3 only reached +0.24, and encyclopedic amount 3 only +0.33. **Their conditions are the problem, not their numbers.**
3. **The starter decks are not 45–55% in v1:** pairings run 23–77%. Field Guide (rising + nostalgic) wins 65% overall and Headliners (local + viral) 40%. **With v1.1 items 1–6, every pairing is 45–55%.**
4. **Owned decks lose to the starters**, because they are about 51% viral and squashed into cost 2–4. That is the wrong way round for "NFTs give variety, not power".
5. **`costThresholds` squash fame.** The owned cost mix is 3/23/37/22/10/5. `[26,39,51,63,78]` gives 13/20/22/20/15/10, which hits the target and confirms the critic and designer-a.
6. **Pure extremes are dead under every lever:** heavy (cost 5–6 only) at 15%, swarm (cost ≤ 2) at 2%. Top-heavy (curve 1/1/2/2/3/3) is **57%**, so the §8 top-heavy retarget (≥ 40%) **passes**. Swarm and heavy aren't fixable without breaking curve decks: every lever that helps one hurts the other (sweep below). Treat them as non-decks.
7. **Matches are always 6 turns.** Only 6.5% of decisive games are decided in the first third, and 19.9% by half-time, so most games stay live to the end.
8. **Skill expression is healthy:**
- greedy beats maxpower ("always play the biggest card") 59.5%. §8 wants the naive line under 55% against a smart bot; it is at 40.5%.
- lookahead-lite beats greedy 63.7%.
- greedy beats random 77.5%.
### Proposed v1.1 rule changes (concrete)
| # | Change | Evidence | Status |
|---|---|---|---|
| 1 | `costThresholds: [26, 39, 51, 63, 78]` (designer-a's quantile rule) | owned cost mix 3/23/37/22/10/5 → 13/20/22/20/15/10 | **adopt** |
| 2 | `rising.powerDelta: -1` | rising +1.76 per card, the top keyword | **adopt** |
| 3 | viral: "+1, plus +1 per earlier reveal this turn, max +3" (critic) | viral +0.19; buffing amount/cap alone gives +0.24 | **adopt.** It needs the base +1. |
| 4 | encyclopedic: "+2 if *you* have no other cards here" (critic) | +0.19; amount 3 alone gives +0.33 | **adopt.** It needs the condition change. |
| 5 | Culture takes its home-stat keyword (critic) | owned decks drop from 51% viral to 18% | see the v1.1 results |
| 6 | `nostalgic.powerDelta -1`, `translated.powerDelta -1` | +1.45 / +1.53 per card, close to rising. Without them Field Guide beats Headliners 60% under 1–5. | **adopt**: with 1–6, every starter pairing is 45–55% |
| 7 | No change for heavy or swarm | no lever fixes both | – |
| 8 | No change for seat or initiative | 48–52% target met | – |
### Starter round robin and owned vs starters (focused run: §2.1 starters plus 4 owned decks, 2,000 matches per ruleset)
| | v1 | v1.1 critic package (1–5) | **v1.1 critic + item 6** |
|---|---|---|---|
| starter pairings, min–max | **23–77%** | 40–60% | **45–55%** ✅ |
| starter deck WR (FG / Head / Pio / Star) | 65 / 40 / 57 / 54 | 50 / 48 / 54 / 54 | 51 / 55 / 52 / 52 |
| owned vs starters (FG / Head / Pio / Star) | 37 / 54 / 40 / 41 | 52 / 45 / 46 / 43 | 50 / 40 / 46 / 47 |
| first player (turn-1 initiative) | 53.4% [51.0–55.8] | 52.5% [50.1–54.9] | **51.0% [48.6–53.4]** ✅ |
| lookahead-lite > greedy | 60% | 70% | 66% |
Each starter pairing has about 50–58 symmetric games (±13 pp at 95%). Each owned-vs-starter cell has about 200 games (±7 pp).
**Net keyword value under v1.1** (counterfactual trigger value per card, plus `powerDelta`), from the 3,000-match critic-package full field:
| keyword | v1.1 net per card |
|---|---|
| rising | 1.74 − 1 = **+0.74** |
| viral | 0.16 + 1 = **+1.16** |
| encyclopedic | ≈ 0.23 + 1 = **+1.23** (approximated) |
| translated | +1.59, or **+0.59** with item 6 |
| nostalgic | +1.58, or **+0.58** with item 6 |
| local | **+0.66** |
So every keyword lands within 0.6–1.2. This matches the critic's numbers (viral +1.15, encyclopedic +0.9, rising +0.9) to within 0.35.
**Recommendation: ship items 1–5 plus item 6** (`nostalgic.powerDelta -1`, `translated.powerDelta -1`). Without item 6, Field Guide still beats Headliners 60% and Headliners sits at 40–41% against every starter. With it, every pairing is inside 45–55. Owned decks then win 40–50% against the starters: the variety-not-power goal, slightly on the weak side. If v1.2 wants owned decks a bit stronger, the next lever is `local.amount 2` (Headliners and owned decks are about 25% local).
Raw tables:
#### `v1` (DEFAULT_RULESET), focused: 2000 matches, 9 draws, 0 engine errors
- First player (turn-1 initiative), symmetric games: **53.4% [51.0–55.8] (n=1641)**
- Decided early: 4.1% of decisive games locked in during the first third, 15.5% by half-time
- Skill: greedy>maxpower 61.6% (n=146), lookahead-lite>greedy 60.0% (n=100), greedy>random 81.7% (n=104)
| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks |
|---|---|---|---|---|---|
| field-guide | – | 77% (n=52) | 57% (n=53) | 69% (n=54) | 64.8% |
| headliners | 23% (n=52) | – | 36% (n=56) | 35% (n=54) | 39.6% |
| pioneers | 43% (n=53) | 64% (n=56) | – | 52% (n=58) | 56.9% |
| stargazers | 31% (n=54) | 65% (n=54) | 48% (n=58) | – | 54.3% |
| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all |
|---|---|---|---|---|---|
| owned-0 | 42% | 48% | 35% | 37% | 40.0% |
| owned-1 | 26% | 62% | 40% | 45% | 47.4% |
| owned-2 | 45% | 57% | 44% | 43% | 52.7% |
| owned-3 | 35% | 47% | 40% | 37% | 42.3% |
| **all owned** | **37%** (n=233) | **54%** (n=197) | **40%** (n=200) | **41%** (n=197) | |
| keyword | power per card | games decided by it | cards played |
|---|---|---|---|
| rising | +1.75 | 12.0% | 1822 |
| translated | +1.55 | 18.6% | 2263 |
| nostalgic | +1.54 | 10.3% | 2197 |
| local | +0.71 | 6.2% | 4300 |
| viral | +0.22 | 3.3% | 6034 |
| encyclopedic | +0.17 | 0.8% | 1042 |
#### `v1.1-critic`: 2000 matches, 4 draws, 0 engine errors
- First player (turn-1 initiative), symmetric games: **52.5% [50.1–54.9] (n=1646)**
- Decided early: 3.5% of decisive games locked in during the first third, 11.4% by half-time
- Skill: greedy>maxpower 59.6% (n=146), lookahead-lite>greedy 70.0% (n=100), greedy>random 80.8% (n=104)
| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks |
|---|---|---|---|---|---|
| field-guide | – | 60% (n=52) | 51% (n=53) | 51% (n=55) | 50.1% |
| headliners | 40% (n=52) | – | 41% (n=56) | 41% (n=54) | 48.3% |
| pioneers | 49% (n=53) | 59% (n=56) | – | 55% (n=58) | 54.2% |
| stargazers | 49% (n=55) | 59% (n=54) | 45% (n=58) | – | 54.1% |
| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all |
|---|---|---|---|---|---|
| owned-0 | 46% | 43% | 46% | 49% | 43.8% |
| owned-1 | 50% | 43% | 40% | 34% | 44.1% |
| owned-2 | 52% | 47% | 56% | 51% | 52.7% |
| owned-3 | 63% | 49% | 43% | 42% | 52.4% |
| **all owned** | **52%** (n=233) | **45%** (n=198) | **46%** (n=198) | **43%** (n=199) | |
| keyword | power per card | games decided by it | cards played |
|---|---|---|---|
| rising | +1.71 | 12.8% | 1505 |
| nostalgic | +1.68 | 13.6% | 1789 |
| translated | +1.64 | 17.6% | 4491 |
| local | +0.70 | 6.3% | 4081 |
| encyclopedic | +0.21 | 1.7% | 2177 |
| viral | +0.13 | 0.9% | 2524 |
#### `v1.1-critic-nerf`: 2000 matches, 14 draws, 0 engine errors
- First player (turn-1 initiative), symmetric games: **51.0% [48.6–53.4] (n=1636)**
- Decided early: 4.0% of decisive games locked in during the first third, 12.7% by half-time
- Skill: greedy>maxpower 58.2% (n=146), lookahead-lite>greedy 66.0% (n=100), greedy>random 79.8% (n=104)
| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks |
|---|---|---|---|---|---|
| field-guide | – | 49% (n=51) | 50% (n=52) | 55% (n=55) | 50.6% |
| headliners | 51% (n=51) | – | 50% (n=54) | 46% (n=54) | 55.3% |
| pioneers | 50% (n=52) | 50% (n=54) | – | 47% (n=58) | 51.8% |
| stargazers | 45% (n=55) | 54% (n=54) | 53% (n=58) | – | 51.9% |
| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all |
|---|---|---|---|---|---|
| owned-0 | 51% | 43% | 52% | 46% | 46.0% |
| owned-1 | 40% | 35% | 47% | 33% | 43.1% |
| owned-2 | 52% | 42% | 37% | 56% | 51.1% |
| owned-3 | 58% | 38% | 47% | 55% | 50.1% |
| **all owned** | **50%** (n=231) | **40%** (n=197) | **46%** (n=199) | **47%** (n=197) | |
| keyword | power per card | games decided by it | cards played |
|---|---|---|---|
| rising | +1.72 | 15.6% | 1534 |
| translated | +1.69 | 16.3% | 4315 |
| nostalgic | +1.66 | 13.3% | 1686 |
| local | +0.71 | 6.4% | 4113 |
| encyclopedic | +0.21 | 1.8% | 2221 |
| viral | +0.13 | 0.7% | 2571 |
#### Critic package, full field (3,000 matches)
#### `v1.1-critic`: 3000 matches, 14 draws, 0 engine errors
- First player (turn-1 initiative), symmetric games: **49.7% [47.7–51.7] (n=2418)**
- Decided early: 8.0% of decisive games locked in during the first third, 17.9% by half-time
- Skill: greedy>maxpower 54.6% (n=238), lookahead-lite>greedy 62.7% (n=161), greedy>random 84.0% (n=169)
| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks |
|---|---|---|---|---|---|
| field-guide | – | 38% (n=8) | 50% (n=8) | 29% (n=7) | 60.6% |
| headliners | 62% (n=8) | – | 36% (n=11) | 56% (n=9) | 46.8% |
| pioneers | 50% (n=8) | 64% (n=11) | – | 50% (n=12) | 59.7% |
| stargazers | 71% (n=7) | 44% (n=9) | 50% (n=12) | – | 64.8% |
| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all |
|---|---|---|---|---|---|
| owned-0 | 33% | 64% | 36% | 50% | 55.3% |
| owned-1 | 50% | 50% | 33% | 43% | 54.9% |
| owned-2 | 55% | 58% | 27% | 42% | 52.7% |
| owned-3 | 11% | 42% | 9% | 50% | 52.8% |
| **all owned** | **38%** (n=37) | **54%** (n=41) | **26%** (n=39) | **46%** (n=37) | |
| archetype | WR vs other decks |
|---|---|
| curve | 58.6% [51.7–65.2] (n=203) |
| top-heavy | 52.2% [45.4–58.9] (n=205) |
| heavy | 23.4% [18.1–29.7] (n=205) |
| swarm | 0.9% [0.2–3.3] (n=220) |
| mono-Culture | 55.4% [48.9–61.6] (n=233) |
| mono-People | 47.5% [41.0–54.1] (n=221) |
| mono-Nature | 40.3% [34.0–46.9] (n=216) |
| keyword | power per card | games decided by it | cards played |
|---|---|---|---|
| rising | +1.74 | 21.1% | 3184 |
| translated | +1.59 | 15.8% | 3450 |
| nostalgic | +1.58 | 19.7% | 2871 |
| local | +0.66 | 7.5% | 4696 |
| encyclopedic | +0.23 | 2.0% | 4548 |
| viral | +0.16 | 1.5% | 4733 |
### v1 full-field detail
#### `v1` (DEFAULT_RULESET): 3000 matches, 13 draws, 0 engine errors
- First player (turn-1 initiative), symmetric games: **51.5% [49.6–53.5] (n=2421)**
- Decided early: 6.5% of decisive games locked in during the first third, 19.9% by half-time
- Skill: greedy>maxpower 59.5% (n=237), lookahead-lite>greedy 63.7% (n=160), greedy>random 77.5% (n=169)
| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks |
|---|---|---|---|---|---|
| field-guide | – | 70% (n=10) | 57% (n=7) | 83% (n=6) | 67.4% |
| headliners | 30% (n=10) | – | 33% (n=12) | 56% (n=9) | 51.7% |
| pioneers | 43% (n=7) | 67% (n=12) | – | 64% (n=14) | 67.0% |
| stargazers | 17% (n=6) | 44% (n=9) | 36% (n=14) | – | 53.3% |
| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all |
|---|---|---|---|---|---|
| owned-0 | 29% | 40% | 15% | 58% | 51.0% |
| owned-1 | 33% | 33% | 67% | 56% | 51.6% |
| owned-2 | 27% | 62% | 62% | 25% | 59.7% |
| owned-3 | 43% | 50% | 56% | 70% | 57.7% |
| **all owned** | **31%** (n=42) | **48%** (n=48) | **44%** (n=36) | **51%** (n=43) | |
| archetype | WR vs other decks |
|---|---|
| curve | 50.0% [43.5–56.5] (n=222) |
| top-heavy | 56.8% [50.2–63.1] (n=222) |
| heavy | 15.3% [11.0–20.9] (n=203) |
| swarm | 2.0% [0.9–4.6] (n=250) |
| mono-Culture | 42.2% [35.7–48.9] (n=211) |
| mono-People | 51.7% [44.9–58.4] (n=207) |
| mono-Nature | 56.0% [49.6–62.3] (n=232) |
| keyword | power per card | games decided by it | cards played |
|---|---|---|---|
| rising | +1.76 | 15.6% | 4027 |
| translated | +1.53 | 17.5% | 1266 |
| nostalgic | +1.45 | 16.4% | 3610 |
| local | +0.65 | 6.1% | 6781 |
| viral | +0.19 | 2.3% | 8030 |
| encyclopedic | +0.19 | 1.4% | 1002 |
*Starter-vs-starter cells here have small n; the focused round robin above has the real numbers. Keyword table: counterfactual trigger value per card (excluding any powerDelta).*
### Lever sweeps (greedy mirrors, real pool)
Before keywords were implemented (engine core @ c938407), I swept these levers. Each row is 3,000 greedy-mirror games. Columns are win rate vs the other decks.
| variant | heavy | swarm | curve | first player |
|---|---|---|---|---|
| v1 | 11.9% | 0.9% | 64.5% | 50.5% |
| `energyCarryMax 2` | 17.8% | 0.9% | 59.3% | 51.0% |
| `energyByTurn [2..7]` | **42.4%** | 0.0% | 41.3% | 49.7% |
| `powerByCost [0,2,3,4,6,8,10]` | 8.1% | 13.4% | 65.6% | 49.3% |
| `powerByCost [0,2,4,5,7,8,10]` | 0.3% | **36.1%** | 62.1% | 49.7% |
| `laneCapacity 4` | 10.3% | 0.6% | 63.6% | 50.7% |
| `onTopicBonus [0,2,2,2,2,3,3]` | 10.2% | 6.8% | 62.2% | 49.7% |
**Reading:** energy 2→7 rescues heavy but kills swarm and curve, and flatter power rescues swarm but kills heavy. Neither helps the normal decks. Keep v1's curve.
### Caveats
- Bots aren't people. Greedy doesn't plan energy carry and queues one card at a time; lookahead-lite is shallow.
- Hidden cards in determinizations are sampled from the loaner pool, not from the opponent's real decklist.
- Some critic-package items need engine hooks that don't exist yet, so this run approximates them:
| item | modelled as | exact? |
|---|---|---|
| viral base +1 | `powerDelta +1` | yes |
| encyclopedic "your side only" | `powerDelta +1`, about the critic's measured +0.9 | no: an approximation until engine-b adds `encyclopedic {sides 1}` |
`powerDelta > 0` is a sim stand-in: the engine's derivation clamps `powerDelta` to [−3, 0].
- Owned decks are 4 random 12-card draws, not curated. They have no curve fill, so some are lopsided.
## Method
+7 -1
View File
@@ -2,5 +2,11 @@
"v1.1-critic": { "keywordParams": {
"rising": { "amount": 1, "cap": 2, "powerDelta": -1 },
"viral": { "amount": 1, "cap": 2, "powerDelta": 1 },
"encyclopedic": { "amount": 2, "powerDelta": 1 } } }
"encyclopedic": { "amount": 2, "powerDelta": 1 } } },
"v1.1-critic-nerf": { "keywordParams": {
"rising": { "amount": 1, "cap": 2, "powerDelta": -1 },
"viral": { "amount": 1, "cap": 2, "powerDelta": 1 },
"encyclopedic": { "amount": 2, "powerDelta": 1 },
"nostalgic": { "amount": 2, "powerDelta": -1 },
"translated": { "amount": 1, "powerDelta": -1 } } }
}
+3 -2
View File
@@ -80,8 +80,9 @@ export interface LookaheadOptions { k?: number; rollouts?: number; horizon?: num
* Flat Monte Carlo lookahead: shortlist the top-k greedy candidates, then for each run
* `rollouts` epsilon-greedy playouts for both seats, up to `horizon` actions, scoring
* terminal results (or the clamped heuristic at the horizon).
* Rollouts run on the true state, so the bot is clairvoyant about hidden info and seeded
* draws: an upper bound on skill, not a fair human-like player.
* With an engine that has views (observe/assume, e.g. shared/game), the bot sees only its seat's view:
* every rollout starts from a fresh determinization (hidden cards resampled), so it is not clairvoyant.
* Without views (toy engine) rollouts use the true state.
*/
export function lookaheadBot<S, A>(engine: EngineAdapter<S, A>, opts: LookaheadOptions = {}): BotFn<unknown, A> {
const k = opts.k ?? 4, rollouts = opts.rollouts ?? 8, horizon = opts.horizon ?? 40, eps = opts.epsilon ?? 0.25;