make stability (tools/stability.sh, StabilityReport) runs one dedicated
server for 50 bot matches back to back, unthrottled, sampling its resident
memory, and fails on a crash, hang, engine or script error, a match with
no winner, or memory that grows after the first five matches.
The first run had no crash, hang or error, but showed two leaks:
- ServerLoop kept every tick's duration in an array (8 bytes a tick,
about 21 MB a day at 30 Hz) and copied and sorted all of it for every
progress line. TickTimes keeps exact statistics in fixed memory: a
one-microsecond histogram up to 65 ms, plus the rare slower ticks.
- Resident memory grew about 28 MB a match (905 MB after 42 matches):
glibc gives each thread its own malloc arena, and every next island is
built on a worker thread. The server launchers (packaging/server,
packaging/dist, so also the .deb, Docker image and systemd unit) and
tools/godot.sh for --server now set MALLOC_ARENA_MAX=1 unless already
set. Tick times were unchanged (p50 501 vs 507 us, p99 1635 vs 1625).
After: 50 matches (seed 4242) in 756 s, all with a winner, exit 0, no
errors; static memory +1.6 KiB and resident +145 KiB a match, flat at
204 MB from match 26 on. No match crashed, so there is no crash seed to
turn into a regression test.
Tests: TickTimes against a sorted list, its fixed memory, the launchers'
setting, and StabilityReport. make check, dist-check and
server-package-check pass.
Closes#57
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>