H4: Long-run stability (make stability), and two memory leaks fixed

make stability (tools/stability.sh, StabilityReport) runs one dedicated
server for 50 bot matches back to back, unthrottled, sampling its resident
memory, and fails on a crash, hang, engine or script error, a match with
no winner, or memory that grows after the first five matches.

The first run had no crash, hang or error, but showed two leaks:
- ServerLoop kept every tick's duration in an array (8 bytes a tick,
  about 21 MB a day at 30 Hz) and copied and sorted all of it for every
  progress line. TickTimes keeps exact statistics in fixed memory: a
  one-microsecond histogram up to 65 ms, plus the rare slower ticks.
- Resident memory grew about 28 MB a match (905 MB after 42 matches):
  glibc gives each thread its own malloc arena, and every next island is
  built on a worker thread. The server launchers (packaging/server,
  packaging/dist, so also the .deb, Docker image and systemd unit) and
  tools/godot.sh for --server now set MALLOC_ARENA_MAX=1 unless already
  set. Tick times were unchanged (p50 501 vs 507 us, p99 1635 vs 1625).

After: 50 matches (seed 4242) in 756 s, all with a winner, exit 0, no
errors; static memory +1.6 KiB and resident +145 KiB a match, flat at
204 MB from match 26 on. No match crashed, so there is no crash seed to
turn into a regression test.

Tests: TickTimes against a sorted list, its fixed memory, the launchers'
setting, and StabilityReport. make check, dist-check and
server-package-check pass.

Closes #57

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-14 07:15:23 -05:00
co-authored by Claude Opus 5.5
parent 38812120df
commit f87d5eae0d
19 changed files with 486 additions and 34 deletions
+5 -1
View File
@@ -1,7 +1,7 @@
# Tidewall build entry points. Everything runs headless with the repo-local engine.
GODOT := tools/godot.sh
.PHONY: app-icon deb deb-check bench dist dist-check build-captures poi-captures art figure figure-captures sea-captures models item-captures audio audio-check check import parse test test-selfcheck smoke capture-test terrain-capture binaries verify-engine balance bot-matches match match-captures client-check client-captures hud-captures style-captures menu-captures chart-captures discovery-check fuzz soak soak-selfcheck server-package-check
.PHONY: app-icon deb deb-check bench dist dist-check build-captures poi-captures art figure figure-captures sea-captures models item-captures audio audio-check check import parse test test-selfcheck smoke capture-test terrain-capture binaries verify-engine balance bot-matches match match-captures client-check client-captures hud-captures style-captures menu-captures chart-captures discovery-check fuzz stability soak soak-selfcheck server-package-check
## A8: regenerates every sound in assets/audio/ from tools/audio/ (about 6 s).
audio: import
@@ -172,6 +172,10 @@ terrain-capture: import
## The 24-client network soak: a real server and 24 headless clients behind a relay that
## adds latency, jitter and loss, for SOAK_MATCHES (default 2) whole matches in real
## time, about half an hour. Checks for desyncs, errors, leaks and the tick budget.
## H4: 50 bot matches back to back on one unthrottled server; report in build/stability/report.txt.
stability: import
@tools/stability.sh
soak: import
@tools/soak.sh