H4: Long-run stability (make stability), and two memory leaks fixed

make stability (tools/stability.sh, StabilityReport) runs one dedicated
server for 50 bot matches back to back, unthrottled, sampling its resident
memory, and fails on a crash, hang, engine or script error, a match with
no winner, or memory that grows after the first five matches.

The first run had no crash, hang or error, but showed two leaks:
- ServerLoop kept every tick's duration in an array (8 bytes a tick,
  about 21 MB a day at 30 Hz) and copied and sorted all of it for every
  progress line. TickTimes keeps exact statistics in fixed memory: a
  one-microsecond histogram up to 65 ms, plus the rare slower ticks.
- Resident memory grew about 28 MB a match (905 MB after 42 matches):
  glibc gives each thread its own malloc arena, and every next island is
  built on a worker thread. The server launchers (packaging/server,
  packaging/dist, so also the .deb, Docker image and systemd unit) and
  tools/godot.sh for --server now set MALLOC_ARENA_MAX=1 unless already
  set. Tick times were unchanged (p50 501 vs 507 us, p99 1635 vs 1625).

After: 50 matches (seed 4242) in 756 s, all with a winner, exit 0, no
errors; static memory +1.6 KiB and resident +145 KiB a match, flat at
204 MB from match 26 on. No match crashed, so there is no crash seed to
turn into a regression test.

Tests: TickTimes against a sorted list, its fixed memory, the launchers'
setting, and StabilityReport. make check, dist-check and
server-package-check pass.

Closes #57

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-14 07:15:23 -05:00
co-authored by Claude Opus 5.5
parent 38812120df
commit f87d5eae0d
19 changed files with 486 additions and 34 deletions
+1
View File
@@ -48,6 +48,7 @@ continuity.
| `make build-captures` | Renders the fort and the wall test card: a row per material, a column per state (half built, intact, worn, battered), and a battered ramp (`captures/build_*.png`), and the barge and sea chests (`captures/vessels.png`), about 2 min. Needs Docker. Look at them. |
| `tools/capture.sh canopy captures/canopy.png` | H11: the shoulder camera on a player against a pine, with canopies in the way hidden (`canopy_before` puts them back). |
| `make style-captures` | Renders a hamlet and a dock with and without the woodcut print pass (`captures/style_*.png`). Needs Docker. |
| `make stability` | H4: one dedicated server plays 50 bot matches back to back, unthrottled, under `nice` (about 13 min), sampling its resident memory. Fails on a crash, hang, engine or script error, a match without a winner, or memory growing more than 256 KiB (static) or 1 MiB (resident) a match after the first five. `STABILITY_MATCHES`, `STABILITY_SEED`, `STABILITY_DIR` scale it. Report in `build/stability/report.txt`. |
| `make soak` | The 24-client network soak (N8): a real server and 24 headless `SoakPlayer` clients in three processes, each behind a `UdpRelay` adding 60 ms, +/-20 ms jitter and 2% loss each way, for two whole matches in real time (about 30 min, under `nice`). Fails on a world-hash mismatch, errors, unreadable packets, a client that didn't play every match, memory growth between matches, or a p99 tick over budget. `SOAK_MATCHES`, `SOAK_CLIENTS`, `SOAK_PROCS`, `SOAK_LATENCY`, `SOAK_JITTER`, `SOAK_LOSS` scale it. Results in `build/soak/report.txt`. |
| `make bench` | H1: the server's p50/p99/max tick with 24 bots on three seeds, and the client's draw calls, triangles, objects and memory at six moments of a match (about 3 min, needs Docker). Fails on anything over the budgets in docs/ARCHITECTURE.md. Report in `build/bench/report.txt`. `BENCH_SEEDS`, `BENCH_TICKS`, `BENCH_CLIENT=0` scale it. |
| `make dist` | R1: exports the game with the official Godot templates (checked against `tools/engine.sha512`, installed into `.godot-home/`) to `build/dist/tidewall-<version>-linux-x86_64.tar.gz` with a `.sha256`. One binary and one pack: `tidewall.x86_64` is the game, `tidewall-server` runs it headless as the server. The version is `application/config/version` in project.godot; the commit goes in as `build_commit.txt` (`BuildInfo`). |