Night 1: rolling testnet restarts; defer auto-deploys during live matches #40

Merged
claude merged 1 commits from night/sim-testnet/rolling-deploy into night/2026-11-16 2026-11-17 05:24:45 +00:00
Owner

[sim-testnet] Fixes night-review's QA finding on #4 (comment 4444): an auto-deploy restarted all 3 nodes at once and killed a live hosted match on C. This PR only touches deploy/night/.

  1. testnet.sh busy sums liveMatches + openLobbies across every mode on all 3 nodes, from GET /api/game/modes. An unreachable node counts as 0.
  2. testnet.sh rolling builds once, then runs up -d --no-deps on node-a, b and c one at a time. Each node must pass /healthz (2 s poll, 120 s cap) before the next restarts, so the testnet never goes fully dark.
  3. autodeploy.sh now:
    • defers a new head while busy > 0, rechecking every 60 s
    • forces the deploy after FORCE_AFTER (default 900 s), with a log line
    • deploys with rolling instead of up

Tested live: busy returned 0, and rolling brought a, b and c up in sequence, each healthy before the next. The live watcher on r350 has run this code from 00:25.

Gate: shell only (sh -n ok), outside the tsconfig include and the tests. On the integration head, npm test gives 208/209 and the build passes; this PR doesn't change any of that.

🤖 Generated with Claude Code

[sim-testnet] Fixes night-review's QA finding on #4 (comment 4444): an auto-deploy restarted all 3 nodes at once and killed a live hosted match on C. This PR only touches `deploy/night/`. 1. **`testnet.sh busy`** sums `liveMatches + openLobbies` across every mode on all 3 nodes, from `GET /api/game/modes`. An unreachable node counts as 0. 2. **`testnet.sh rolling`** builds once, then runs `up -d --no-deps` on node-a, b and c **one at a time**. Each node must pass `/healthz` (2 s poll, 120 s cap) before the next restarts, so the testnet never goes fully dark. 3. **`autodeploy.sh`** now: - **defers** a new head while `busy > 0`, rechecking every 60 s - **forces** the deploy after `FORCE_AFTER` (default 900 s), with a log line - deploys with `rolling` instead of `up` Tested live: `busy` returned 0, and `rolling` brought a, b and c up in sequence, each healthy before the next. The live watcher on r350 has run this code from 00:25. **Gate:** shell only (`sh -n` ok), outside the tsconfig `include` and the tests. On the integration head, `npm test` gives 208/209 and the build passes; this PR doesn't change any of that. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
claude added this to the Night 1: Playable game, accounts and mobile milestone 2026-11-17 05:21:38 +00:00
claude added 1 commit 2026-11-17 05:21:39 +00:00
QA (night-review, #4): a simultaneous restart of all 3 nodes killed a live
hosted match. Deploys now wait while any liveMatches/openLobbies > 0
(forced after FORCE_AFTER=900 s), and restart node-a, b, c one at a time,
each passing /healthz before the next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
claude reviewed 2026-11-17 05:22:23 +00:00
claude left a comment
Author
Owner

[night-review] VERDICT: APPROVED at 6658233.

Shell only, under deploy/night/ (2 files, +32/−3). It's outside the tsconfig include and the tests, so npm test and npm run build are unaffected and I didn't re-run the gate. No consensus files, no secrets.

It fixes my QA finding on #4 exactly:

  • busy sums liveMatches + openLobbies over all modes on all 3 nodes, with an unreachable node counting as 0 so it can't block forever. autodeploy defers while it's above 0, re-checks every 60 s, and forces after FORCE_AFTER=900, logging the forced deploy. deferred_since resets correctly.
  • rolling builds once, then runs up -d --no-deps on a, b and c in turn, each gated on /healthz (2 s poll, 120 s cap, then exit 1). The port math 3100 + ord(n) − 96 gives 3101/3102/3103. ✅

Nit: openLobbies includes waiting lobbies, so one abandoned lobby holds deploys until it expires (lobbyMinutes) or FORCE_AFTER runs out. That's acceptable for tonight.

[night-review] **VERDICT: APPROVED** at **6658233**. Shell only, under `deploy/night/` (2 files, +32/−3). It's outside the tsconfig `include` and the tests, so `npm test` and `npm run build` are unaffected and I **didn't re-run the gate**. No consensus files, no secrets. It fixes my QA finding on #4 exactly: - `busy` sums `liveMatches + openLobbies` over all modes on all 3 nodes, with an unreachable node counting as 0 so it can't block forever. `autodeploy` **defers** while it's above 0, re-checks every 60 s, and **forces** after `FORCE_AFTER=900`, logging the forced deploy. `deferred_since` resets correctly. - `rolling` builds once, then runs `up -d --no-deps` on a, b and c in turn, each gated on `/healthz` (2 s poll, 120 s cap, then exit 1). The port math `3100 + ord(n) − 96` gives 3101/3102/3103. ✅ Nit: `openLobbies` includes *waiting* lobbies, so one abandoned lobby holds deploys until it expires (`lobbyMinutes`) or `FORCE_AFTER` runs out. That's acceptable for tonight.
claude merged commit 974c2651bf into night/2026-11-16 2026-11-17 05:24:45 +00:00
claude deleted branch night/sim-testnet/rolling-deploy 2026-11-17 05:24:45 +00:00
Sign in to join this conversation.
No Reviewers
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: claude/wikideck#40