Files
tidewall/tools/bench.sh
T
claudeandClaude Opus 5.5 9ce0865c9b H1: make bench, render budgets, and a third of the figure draw calls
make bench (tools/bench.sh, tools/bench_report.gd) measures the server's
p50/p99/max tick with 24 bots on three seeds over a whole match, and the
client's draw calls, primitives, objects, nodes and memory at six moments
of a match rendered in the capture container. It fails on anything over
the budgets now recorded in docs/ARCHITECTURE.md: p99 tick 12 ms, 2,000
draw calls and 2,000,000 primitives per frame (all passes), 1 GiB video
and 512 MiB static memory. Every capture now reports its frame's render
counts in a "render" field.

The first run put the barge, with 24 figures together, at 2,197 draw
calls. Each figure part had one surface per colour (about 43 per figure,
drawn again for shadows). FigureRig now merges a part's surfaces into one
per finish (coat, gilt, plain) and carries the Palette colours as vertex
colours, with shared materials. The barge is down to 1,207 draw calls,
with the same triangles and the same look.

Frame rates on a real GPU are not measured here; that stays with R6.

Closes #54

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 23:30:16 -05:00

64 lines
2.5 KiB
Bash
Executable File

#!/usr/bin/env bash
# H1: repeatable performance benchmark.
#
# Server: runs the headless server with 24 bots, unthrottled, for BENCH_TICKS ticks on
# each of BENCH_SEEDS seeds, and reads its p50/p99/max tick time.
# Client: renders the moments of the match `make match` saved (whole island, barge,
# landed, a build, the final players) and a crowded pine forest in the capture
# container, and reads each frame's draw calls, primitives, objects and memory.
#
# tools/bench_report.gd checks every number against the budgets in
# docs/ARCHITECTURE.md and writes build/bench/report.txt and report.json.
#
# Software rendering counts what is drawn, not how fast: frame rates on a real GPU are
# left to the hands-on playtest (R6).
#
# tools/bench.sh about 3 minutes; needs Docker for the client
# BENCH_SEEDS="2 7" BENCH_TICKS=6000 tools/bench.sh
# BENCH_CLIENT=0 tools/bench.sh the server only
set -uo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")/.."
seeds="${BENCH_SEEDS:-2 7 20261109}"
ticks="${BENCH_TICKS:-12000}"
client="${BENCH_CLIENT:-1}"
[[ "$ticks" =~ ^[0-9]+$ ]] || { echo "bench FAIL: bad BENCH_TICKS '$ticks'"; exit 2; }
out="build/bench"
mkdir -p "$out"
rm -f "$out"/*.jsonl
status=0
for seed in $seeds; do
[[ "$seed" =~ ^[0-9]+$ ]] || { echo "bench FAIL: bad seed '$seed'"; exit 2; }
echo "bench: server, seed $seed, $ticks ticks, 24 bots" >&2
raw="$(timeout --signal=KILL 900 nice -n 15 tools/godot.sh -- --server --seed="$seed" --bots=24 --ticks="$ticks" --unthrottled 2>&1)"
code=$?
line="$(grep -m1 '"event":"stats"' <<<"$raw" || true)"
if [[ "$code" -ne 0 || -z "$line" ]]; then
echo "bench FAIL: the server on seed $seed exited $code"
grep -v '^{' <<<"$raw" | tail -n 10
status=1
continue
fi
echo "$line" >>"$out/server.jsonl"
done
if [[ "$client" == "1" ]]; then
if [[ ! -f build/match/match.tclog ]]; then
echo "bench: no saved match yet; playing one (make match)" >&2
tools/match.sh >/dev/null || { echo "bench FAIL: make match failed"; exit 1; }
fi
for stage in match_lobby match_barge match_landed match_built match_final props_forest; do
echo "bench: client, $stage" >&2
if ! report="$(tools/capture.sh "$stage" "captures/bench_$stage.png" 2>/dev/null)"; then
echo "bench FAIL: the $stage capture failed"
status=1
continue
fi
echo "$report" | grep -m1 '^{' >>"$out/client.jsonl"
done
fi
tools/godot.sh --script res://tools/bench_report.gd -- --dir="res://$out" --client="$client" || status=1
exit "$status"