N8: 24-client network soak, and the server fixes it found

make soak runs a real server and 24 headless clients in three processes for
two whole matches in real time. Each client reaches the server through its own
UdpRelay, which adds 60 ms, +/-20 ms jitter and 2% loss each way, so ENet runs
unchanged over a bad link. SoakPlayer drives each client from what a client
knows: it drops, glides, heads for high ground, harvests and tries to build.
tools/soak_report.gd checks world hashes against the server's, errors,
unreadable packets, every client playing every match with its connection
intact, memory between matches, and the p99 tick with the network's share.
make soak-selfcheck proves a drifted client world is caught.

Server changes: --matches=N, --world-hash-every=N, tick timing that includes
after_tick, and memory in match and progress lines.

What the soak found and fixed:
- Snapshot building for 24 clients took the p99 tick to 13.8 ms, over the
  12 ms budget, against a 2.8 ms simulation. Encoded records are now shared
  between clients for the tick, unchanged records skip the field diff, and
  clients are spread over the snapshot interval by handle. p99 is 7.3 ms.
- ENet peers had a timeout limit of 0, which gives up on a peer as soon as any
  reliable message has waited the minimum timeout. Now ENet's default, 32.

Not fixed: one or two clients still lose their connection about 15 s into the
second match's drop in each run (see the issue), so the soak still fails and
N8 stays open.

Refs #39

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-12 21:56:01 -05:00
co-authored by Claude Opus 5.5
parent 65b1fe015c
commit 05c7568ec1
25 changed files with 1135 additions and 39 deletions
+2
View File
@@ -37,6 +37,8 @@ continuity.
| `make discovery-check` | Starts a list server and a game server as processes and checks `--browse` finds the game both on the LAN (loopback) and through the list (about 15 s). |
| `make fuzz` | Feeds 1,000,000 malformed messages (`FUZZ_MESSAGES=N`, `FUZZ_SEED=N`) to every network parser and fails on any engine error or memory growth (about 40 s). |
| `make client-captures` | Renders what a network client sees on the barge, gliding and landed (`captures/client_<stage>.png`). Needs Docker. Look at them. |
| `make soak` | The 24-client network soak (N8): a real server and 24 headless `SoakPlayer` clients in three processes, each behind a `UdpRelay` adding 60 ms, +/-20 ms jitter and 2% loss each way, for two whole matches in real time (about 30 min, under `nice`). Fails on a world-hash mismatch, errors, unreadable packets, a client that didn't play every match, memory growth between matches, or a p99 tick over budget. `SOAK_MATCHES`, `SOAK_CLIENTS`, `SOAK_PROCS`, `SOAK_LATENCY`, `SOAK_JITTER`, `SOAK_LOSS` scale it. Results in `build/soak/report.txt`. |
| `make soak-selfcheck` | About a minute: tampers with one client's world and checks the soak's comparison catches it and only it. |
| `make capture-test` | Proves the capture harness works and rejects a blank frame. Needs Docker, so it isn't part of `make check`. |
Run one group of tests with `tools/run_tests.sh --root=res://test/sim` or