From fc64277b386eb13548f28d12467dac78e6d7af31 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 17 Nov 2026 00:23:59 -0500 Subject: [PATCH] sim-balance: BALANCE.md report v1 with v1.1 recommendations Co-Authored-By: Claude Opus 5.5 --- docs/game/BALANCE.md | 292 +++++++++++++++++++++++++- scripts/sim/variants/v1.1-critic.json | 8 +- shared/bots/index.ts | 5 +- 3 files changed, 301 insertions(+), 4 deletions(-) diff --git a/docs/game/BALANCE.md b/docs/game/BALANCE.md index 8204c77..0ecfe83 100644 --- a/docs/game/BALANCE.md +++ b/docs/game/BALANCE.md @@ -2,7 +2,297 @@ Owner: sim-balance (claude/wikideck#4). Harness: `shared/bots/`, `scripts/sim/` (PR #14). - +## Report v1: 2026-11-17 00:45 EST + +**Engine:** `night/2026-11-16` @ 482de6c/4a4941c (engine #25 and #27, keywords and Portals; card data #28). +**Bots:** view-only (`viewFor`); see Method. +**Card pool:** `docs/game/card-sample.json` (236 cards), derived per §1 with percentiles from the sample. +**Decks:** the 4 §2.1 starter decks, 4 "realistic owned" decks (12 random cards from the sample), and archetypes. +**Runs:** +- full field: 3,000 matches per ruleset (report mix, 4 workers; r350 is memory-limited tonight) +- focused starters + owned round robin: 2,000 matches per ruleset +- greedy lever sweeps: 2,000–3,000 per variant + +### TL;DR for the council +1. **Seat and initiative are fine:** 51.5% [49.6–53.5] for the turn-1 initiative holder (n = 2,421 symmetric games), and seat 0 51.2%. Draws are 0.4%. The tie-break (total power) decides 4.7% of games. *This full-field run started before the bot-side queue-count fix (PR #14); the effect is at most a 0.02 nudge to the heuristic. The focused starters run, after the fix, gives a borderline 53.4% [51.0–55.8] under v1 and **51.0% [48.6–53.4]** under the recommended v1.1. Initiative only matters through reveal order, and v1.1 removes most of the order-sensitive power.* +2. **Keyword values are lopsided, confirming the critic's figures within 0.2:** + + | keyword | power per card | + |---|---| + | rising | +1.76 | + | translated | +1.53 | + | nostalgic | +1.45 | + | local | +0.65 | + | **viral** | **+0.19** | + | **encyclopedic** | **+0.19** | + + Values are counterfactual replays. viral and encyclopedic are dead: raising viral to amount 2 / cap 3 only reached +0.24, and encyclopedic amount 3 only +0.33. **Their conditions are the problem, not their numbers.** +3. **The starter decks are not 45–55% in v1:** pairings run 23–77%. Field Guide (rising + nostalgic) wins 65% overall and Headliners (local + viral) 40%. **With v1.1 items 1–6, every pairing is 45–55%.** +4. **Owned decks lose to the starters**, because they are about 51% viral and squashed into cost 2–4. That is the wrong way round for "NFTs give variety, not power". +5. **`costThresholds` squash fame.** The owned cost mix is 3/23/37/22/10/5. `[26,39,51,63,78]` gives 13/20/22/20/15/10, which hits the target and confirms the critic and designer-a. +6. **Pure extremes are dead under every lever:** heavy (cost 5–6 only) at 15%, swarm (cost ≤ 2) at 2%. Top-heavy (curve 1/1/2/2/3/3) is **57%**, so the §8 top-heavy retarget (≥ 40%) **passes**. Swarm and heavy aren't fixable without breaking curve decks: every lever that helps one hurts the other (sweep below). Treat them as non-decks. +7. **Matches are always 6 turns.** Only 6.5% of decisive games are decided in the first third, and 19.9% by half-time, so most games stay live to the end. +8. **Skill expression is healthy:** + - greedy beats maxpower ("always play the biggest card") 59.5%. §8 wants the naive line under 55% against a smart bot; it is at 40.5%. + - lookahead-lite beats greedy 63.7%. + - greedy beats random 77.5%. + +### Proposed v1.1 rule changes (concrete) + +| # | Change | Evidence | Status | +|---|---|---|---| +| 1 | `costThresholds: [26, 39, 51, 63, 78]` (designer-a's quantile rule) | owned cost mix 3/23/37/22/10/5 → 13/20/22/20/15/10 | **adopt** | +| 2 | `rising.powerDelta: -1` | rising +1.76 per card, the top keyword | **adopt** | +| 3 | viral: "+1, plus +1 per earlier reveal this turn, max +3" (critic) | viral +0.19; buffing amount/cap alone gives +0.24 | **adopt.** It needs the base +1. | +| 4 | encyclopedic: "+2 if *you* have no other cards here" (critic) | +0.19; amount 3 alone gives +0.33 | **adopt.** It needs the condition change. | +| 5 | Culture takes its home-stat keyword (critic) | owned decks drop from 51% viral to 18% | see the v1.1 results | +| 6 | `nostalgic.powerDelta -1`, `translated.powerDelta -1` | +1.45 / +1.53 per card, close to rising. Without them Field Guide beats Headliners 60% under 1–5. | **adopt**: with 1–6, every starter pairing is 45–55% | +| 7 | No change for heavy or swarm | no lever fixes both | – | +| 8 | No change for seat or initiative | 48–52% target met | – | + +### Starter round robin and owned vs starters (focused run: §2.1 starters plus 4 owned decks, 2,000 matches per ruleset) + +| | v1 | v1.1 critic package (1–5) | **v1.1 critic + item 6** | +|---|---|---|---| +| starter pairings, min–max | **23–77%** | 40–60% | **45–55%** ✅ | +| starter deck WR (FG / Head / Pio / Star) | 65 / 40 / 57 / 54 | 50 / 48 / 54 / 54 | 51 / 55 / 52 / 52 | +| owned vs starters (FG / Head / Pio / Star) | 37 / 54 / 40 / 41 | 52 / 45 / 46 / 43 | 50 / 40 / 46 / 47 | +| first player (turn-1 initiative) | 53.4% [51.0–55.8] | 52.5% [50.1–54.9] | **51.0% [48.6–53.4]** ✅ | +| lookahead-lite > greedy | 60% | 70% | 66% | + +Each starter pairing has about 50–58 symmetric games (±13 pp at 95%). Each owned-vs-starter cell has about 200 games (±7 pp). + +**Net keyword value under v1.1** (counterfactual trigger value per card, plus `powerDelta`), from the 3,000-match critic-package full field: + +| keyword | v1.1 net per card | +|---|---| +| rising | 1.74 − 1 = **+0.74** | +| viral | 0.16 + 1 = **+1.16** | +| encyclopedic | ≈ 0.23 + 1 = **+1.23** (approximated) | +| translated | +1.59, or **+0.59** with item 6 | +| nostalgic | +1.58, or **+0.58** with item 6 | +| local | **+0.66** | + +So every keyword lands within 0.6–1.2. This matches the critic's numbers (viral +1.15, encyclopedic +0.9, rising +0.9) to within 0.35. + +**Recommendation: ship items 1–5 plus item 6** (`nostalgic.powerDelta -1`, `translated.powerDelta -1`). Without item 6, Field Guide still beats Headliners 60% and Headliners sits at 40–41% against every starter. With it, every pairing is inside 45–55. Owned decks then win 40–50% against the starters: the variety-not-power goal, slightly on the weak side. If v1.2 wants owned decks a bit stronger, the next lever is `local.amount 2` (Headliners and owned decks are about 25% local). + +Raw tables: + +#### `v1` (DEFAULT_RULESET), focused: 2000 matches, 9 draws, 0 engine errors + +- First player (turn-1 initiative), symmetric games: **53.4% [51.0–55.8] (n=1641)** +- Decided early: 4.1% of decisive games locked in during the first third, 15.5% by half-time +- Skill: greedy>maxpower 61.6% (n=146), lookahead-lite>greedy 60.0% (n=100), greedy>random 81.7% (n=104) + +| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks | +|---|---|---|---|---|---| +| field-guide | – | 77% (n=52) | 57% (n=53) | 69% (n=54) | 64.8% | +| headliners | 23% (n=52) | – | 36% (n=56) | 35% (n=54) | 39.6% | +| pioneers | 43% (n=53) | 64% (n=56) | – | 52% (n=58) | 56.9% | +| stargazers | 31% (n=54) | 65% (n=54) | 48% (n=58) | – | 54.3% | + +| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all | +|---|---|---|---|---|---| +| owned-0 | 42% | 48% | 35% | 37% | 40.0% | +| owned-1 | 26% | 62% | 40% | 45% | 47.4% | +| owned-2 | 45% | 57% | 44% | 43% | 52.7% | +| owned-3 | 35% | 47% | 40% | 37% | 42.3% | +| **all owned** | **37%** (n=233) | **54%** (n=197) | **40%** (n=200) | **41%** (n=197) | | + + +| keyword | power per card | games decided by it | cards played | +|---|---|---|---| +| rising | +1.75 | 12.0% | 1822 | +| translated | +1.55 | 18.6% | 2263 | +| nostalgic | +1.54 | 10.3% | 2197 | +| local | +0.71 | 6.2% | 4300 | +| viral | +0.22 | 3.3% | 6034 | +| encyclopedic | +0.17 | 0.8% | 1042 | + +#### `v1.1-critic`: 2000 matches, 4 draws, 0 engine errors + +- First player (turn-1 initiative), symmetric games: **52.5% [50.1–54.9] (n=1646)** +- Decided early: 3.5% of decisive games locked in during the first third, 11.4% by half-time +- Skill: greedy>maxpower 59.6% (n=146), lookahead-lite>greedy 70.0% (n=100), greedy>random 80.8% (n=104) + +| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks | +|---|---|---|---|---|---| +| field-guide | – | 60% (n=52) | 51% (n=53) | 51% (n=55) | 50.1% | +| headliners | 40% (n=52) | – | 41% (n=56) | 41% (n=54) | 48.3% | +| pioneers | 49% (n=53) | 59% (n=56) | – | 55% (n=58) | 54.2% | +| stargazers | 49% (n=55) | 59% (n=54) | 45% (n=58) | – | 54.1% | + +| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all | +|---|---|---|---|---|---| +| owned-0 | 46% | 43% | 46% | 49% | 43.8% | +| owned-1 | 50% | 43% | 40% | 34% | 44.1% | +| owned-2 | 52% | 47% | 56% | 51% | 52.7% | +| owned-3 | 63% | 49% | 43% | 42% | 52.4% | +| **all owned** | **52%** (n=233) | **45%** (n=198) | **46%** (n=198) | **43%** (n=199) | | + + +| keyword | power per card | games decided by it | cards played | +|---|---|---|---| +| rising | +1.71 | 12.8% | 1505 | +| nostalgic | +1.68 | 13.6% | 1789 | +| translated | +1.64 | 17.6% | 4491 | +| local | +0.70 | 6.3% | 4081 | +| encyclopedic | +0.21 | 1.7% | 2177 | +| viral | +0.13 | 0.9% | 2524 | + +#### `v1.1-critic-nerf`: 2000 matches, 14 draws, 0 engine errors + +- First player (turn-1 initiative), symmetric games: **51.0% [48.6–53.4] (n=1636)** +- Decided early: 4.0% of decisive games locked in during the first third, 12.7% by half-time +- Skill: greedy>maxpower 58.2% (n=146), lookahead-lite>greedy 66.0% (n=100), greedy>random 79.8% (n=104) + +| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks | +|---|---|---|---|---|---| +| field-guide | – | 49% (n=51) | 50% (n=52) | 55% (n=55) | 50.6% | +| headliners | 51% (n=51) | – | 50% (n=54) | 46% (n=54) | 55.3% | +| pioneers | 50% (n=52) | 50% (n=54) | – | 47% (n=58) | 51.8% | +| stargazers | 45% (n=55) | 54% (n=54) | 53% (n=58) | – | 51.9% | + +| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all | +|---|---|---|---|---|---| +| owned-0 | 51% | 43% | 52% | 46% | 46.0% | +| owned-1 | 40% | 35% | 47% | 33% | 43.1% | +| owned-2 | 52% | 42% | 37% | 56% | 51.1% | +| owned-3 | 58% | 38% | 47% | 55% | 50.1% | +| **all owned** | **50%** (n=231) | **40%** (n=197) | **46%** (n=199) | **47%** (n=197) | | + + +| keyword | power per card | games decided by it | cards played | +|---|---|---|---| +| rising | +1.72 | 15.6% | 1534 | +| translated | +1.69 | 16.3% | 4315 | +| nostalgic | +1.66 | 13.3% | 1686 | +| local | +0.71 | 6.4% | 4113 | +| encyclopedic | +0.21 | 1.8% | 2221 | +| viral | +0.13 | 0.7% | 2571 | + + + +#### Critic package, full field (3,000 matches) + +#### `v1.1-critic`: 3000 matches, 14 draws, 0 engine errors + +- First player (turn-1 initiative), symmetric games: **49.7% [47.7–51.7] (n=2418)** +- Decided early: 8.0% of decisive games locked in during the first third, 17.9% by half-time +- Skill: greedy>maxpower 54.6% (n=238), lookahead-lite>greedy 62.7% (n=161), greedy>random 84.0% (n=169) + +| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks | +|---|---|---|---|---|---| +| field-guide | – | 38% (n=8) | 50% (n=8) | 29% (n=7) | 60.6% | +| headliners | 62% (n=8) | – | 36% (n=11) | 56% (n=9) | 46.8% | +| pioneers | 50% (n=8) | 64% (n=11) | – | 50% (n=12) | 59.7% | +| stargazers | 71% (n=7) | 44% (n=9) | 50% (n=12) | – | 64.8% | + +| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all | +|---|---|---|---|---|---| +| owned-0 | 33% | 64% | 36% | 50% | 55.3% | +| owned-1 | 50% | 50% | 33% | 43% | 54.9% | +| owned-2 | 55% | 58% | 27% | 42% | 52.7% | +| owned-3 | 11% | 42% | 9% | 50% | 52.8% | +| **all owned** | **38%** (n=37) | **54%** (n=41) | **26%** (n=39) | **46%** (n=37) | | + +| archetype | WR vs other decks | +|---|---| +| curve | 58.6% [51.7–65.2] (n=203) | +| top-heavy | 52.2% [45.4–58.9] (n=205) | +| heavy | 23.4% [18.1–29.7] (n=205) | +| swarm | 0.9% [0.2–3.3] (n=220) | +| mono-Culture | 55.4% [48.9–61.6] (n=233) | +| mono-People | 47.5% [41.0–54.1] (n=221) | +| mono-Nature | 40.3% [34.0–46.9] (n=216) | + +| keyword | power per card | games decided by it | cards played | +|---|---|---|---| +| rising | +1.74 | 21.1% | 3184 | +| translated | +1.59 | 15.8% | 3450 | +| nostalgic | +1.58 | 19.7% | 2871 | +| local | +0.66 | 7.5% | 4696 | +| encyclopedic | +0.23 | 2.0% | 4548 | +| viral | +0.16 | 1.5% | 4733 | + + + +### v1 full-field detail + +#### `v1` (DEFAULT_RULESET): 3000 matches, 13 draws, 0 engine errors + +- First player (turn-1 initiative), symmetric games: **51.5% [49.6–53.5] (n=2421)** +- Decided early: 6.5% of decisive games locked in during the first third, 19.9% by half-time +- Skill: greedy>maxpower 59.5% (n=237), lookahead-lite>greedy 63.7% (n=160), greedy>random 77.5% (n=169) + +| starters (row vs col) | field-guide | headliners | pioneers | stargazers | vs all decks | +|---|---|---|---|---|---| +| field-guide | – | 70% (n=10) | 57% (n=7) | 83% (n=6) | 67.4% | +| headliners | 30% (n=10) | – | 33% (n=12) | 56% (n=9) | 51.7% | +| pioneers | 43% (n=7) | 67% (n=12) | – | 64% (n=14) | 67.0% | +| stargazers | 17% (n=6) | 44% (n=9) | 36% (n=14) | – | 53.3% | + +| owned (random 12 from card-sample) vs | field-guide | headliners | pioneers | stargazers | all | +|---|---|---|---|---|---| +| owned-0 | 29% | 40% | 15% | 58% | 51.0% | +| owned-1 | 33% | 33% | 67% | 56% | 51.6% | +| owned-2 | 27% | 62% | 62% | 25% | 59.7% | +| owned-3 | 43% | 50% | 56% | 70% | 57.7% | +| **all owned** | **31%** (n=42) | **48%** (n=48) | **44%** (n=36) | **51%** (n=43) | | + +| archetype | WR vs other decks | +|---|---| +| curve | 50.0% [43.5–56.5] (n=222) | +| top-heavy | 56.8% [50.2–63.1] (n=222) | +| heavy | 15.3% [11.0–20.9] (n=203) | +| swarm | 2.0% [0.9–4.6] (n=250) | +| mono-Culture | 42.2% [35.7–48.9] (n=211) | +| mono-People | 51.7% [44.9–58.4] (n=207) | +| mono-Nature | 56.0% [49.6–62.3] (n=232) | + +| keyword | power per card | games decided by it | cards played | +|---|---|---|---| +| rising | +1.76 | 15.6% | 4027 | +| translated | +1.53 | 17.5% | 1266 | +| nostalgic | +1.45 | 16.4% | 3610 | +| local | +0.65 | 6.1% | 6781 | +| viral | +0.19 | 2.3% | 8030 | +| encyclopedic | +0.19 | 1.4% | 1002 | + + +*Starter-vs-starter cells here have small n; the focused round robin above has the real numbers. Keyword table: counterfactual trigger value per card (excluding any powerDelta).* + + +### Lever sweeps (greedy mirrors, real pool) + +Before keywords were implemented (engine core @ c938407), I swept these levers. Each row is 3,000 greedy-mirror games. Columns are win rate vs the other decks. + +| variant | heavy | swarm | curve | first player | +|---|---|---|---|---| +| v1 | 11.9% | 0.9% | 64.5% | 50.5% | +| `energyCarryMax 2` | 17.8% | 0.9% | 59.3% | 51.0% | +| `energyByTurn [2..7]` | **42.4%** | 0.0% | 41.3% | 49.7% | +| `powerByCost [0,2,3,4,6,8,10]` | 8.1% | 13.4% | 65.6% | 49.3% | +| `powerByCost [0,2,4,5,7,8,10]` | 0.3% | **36.1%** | 62.1% | 49.7% | +| `laneCapacity 4` | 10.3% | 0.6% | 63.6% | 50.7% | +| `onTopicBonus [0,2,2,2,2,3,3]` | 10.2% | 6.8% | 62.2% | 49.7% | + +**Reading:** energy 2→7 rescues heavy but kills swarm and curve, and flatter power rescues swarm but kills heavy. Neither helps the normal decks. Keep v1's curve. + +### Caveats +- Bots aren't people. Greedy doesn't plan energy carry and queues one card at a time; lookahead-lite is shallow. +- Hidden cards in determinizations are sampled from the loaner pool, not from the opponent's real decklist. +- Some critic-package items need engine hooks that don't exist yet, so this run approximates them: + + | item | modelled as | exact? | + |---|---|---| + | viral base +1 | `powerDelta +1` | yes | + | encyclopedic "your side only" | `powerDelta +1`, about the critic's measured +0.9 | no: an approximation until engine-b adds `encyclopedic {sides 1}` | + + `powerDelta > 0` is a sim stand-in: the engine's derivation clamps `powerDelta` to [−3, 0]. +- Owned decks are 4 random 12-card draws, not curated. They have no curve fill, so some are lopsided. + ## Method diff --git a/scripts/sim/variants/v1.1-critic.json b/scripts/sim/variants/v1.1-critic.json index ee33a08..3729771 100644 --- a/scripts/sim/variants/v1.1-critic.json +++ b/scripts/sim/variants/v1.1-critic.json @@ -2,5 +2,11 @@ "v1.1-critic": { "keywordParams": { "rising": { "amount": 1, "cap": 2, "powerDelta": -1 }, "viral": { "amount": 1, "cap": 2, "powerDelta": 1 }, - "encyclopedic": { "amount": 2, "powerDelta": 1 } } } + "encyclopedic": { "amount": 2, "powerDelta": 1 } } }, + "v1.1-critic-nerf": { "keywordParams": { + "rising": { "amount": 1, "cap": 2, "powerDelta": -1 }, + "viral": { "amount": 1, "cap": 2, "powerDelta": 1 }, + "encyclopedic": { "amount": 2, "powerDelta": 1 }, + "nostalgic": { "amount": 2, "powerDelta": -1 }, + "translated": { "amount": 1, "powerDelta": -1 } } } } diff --git a/shared/bots/index.ts b/shared/bots/index.ts index 3e2dc28..65395f8 100644 --- a/shared/bots/index.ts +++ b/shared/bots/index.ts @@ -80,8 +80,9 @@ export interface LookaheadOptions { k?: number; rollouts?: number; horizon?: num * Flat Monte Carlo lookahead: shortlist the top-k greedy candidates, then for each run * `rollouts` epsilon-greedy playouts for both seats, up to `horizon` actions, scoring * terminal results (or the clamped heuristic at the horizon). - * Rollouts run on the true state, so the bot is clairvoyant about hidden info and seeded - * draws: an upper bound on skill, not a fair human-like player. + * With an engine that has views (observe/assume, e.g. shared/game), the bot sees only its seat's view: + * every rollout starts from a fresh determinization (hidden cards resampled), so it is not clairvoyant. + * Without views (toy engine) rollouts use the true state. */ export function lookaheadBot(engine: EngineAdapter, opts: LookaheadOptions = {}): BotFn { const k = opts.k ?? 4, rollouts = opts.rollouts ?? 8, horizon = opts.horizon ?? 40, eps = opts.epsilon ?? 0.25;