F7: Offscreen render capture harness

tools/capture.sh <scene-or-scenario> <out.png> renders the real client
in a throwaway tidewall-capture-* Docker container (debian:13, Xvfb,
Mesa software rendering), with no network, no host display, GPU or
audio, and removes the container afterwards. The tidewall-capture image
rebuilds only when its recipe changes.

Forward+ through Lavapipe is the default, matching the game;
CAPTURE_RENDERER=compatibility uses OpenGL through llvmpipe. The script
fails if Godot silently falls back to another renderer, and fails on a
blank frame, judged by ImageStats (luminance variance and colour count).

make capture-test renders the shore test scene with both renderers and
proves an empty scene is rejected. ImageStats has unit tests in
make check. CLAUDE.md now requires a capture for visual issues, and
requires that it be opened and looked at.

Closes #7

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-09 20:13:22 -05:00
co-authored by Claude Opus 5.5
parent d9d4c04312
commit 68635d5fba
16 changed files with 453 additions and 1 deletions
+13
View File
@@ -23,10 +23,23 @@ continuity.
| `make smoke` | Boots the headless server for 1,000 ticks with 24 bots, prints tick timing, and checks exit codes and logs. Part of `make check`. | | `make smoke` | Boots the headless server for 1,000 ticks with 24 bots, prints tick timing, and checks exit codes and logs. Part of `make check`. |
| `tools/godot.sh -- --server --seed=N --bots=N --ticks=N` | Runs a headless dedicated server. See `src/server/server_config.gd` for every option. Logs are JSON lines carrying the seed. Exit codes: 0 clean, 1 script error, 2 bad command line. | | `tools/godot.sh -- --server --seed=N --bots=N --ticks=N` | Runs a headless dedicated server. See `src/server/server_config.gd` for every option. Logs are JSON lines carrying the seed. Exit codes: 0 clean, 1 script error, 2 bad command line. |
| `tools/godot.sh ...` | Runs the pinned engine. Headless by default. | | `tools/godot.sh ...` | Runs the pinned engine. Headless by default. |
| `tools/capture.sh <scene-or-scenario> <out.png>` | Renders the real client offscreen in a throwaway `tidewall-capture-*` Docker container (Xvfb, Mesa software rendering) and saves a PNG. Fails on a blank frame or a renderer fallback. |
| `make capture-test` | Proves the capture harness works and rejects a blank frame. Needs Docker, so it isn't part of `make check`. |
Run one group of tests with `tools/run_tests.sh --root=res://test/sim` or Run one group of tests with `tools/run_tests.sh --root=res://test/sim` or
`tools/run_tests.sh --filter=name`. `tools/run_tests.sh --filter=name`.
## Visual work
- Every issue that changes what the game looks like attaches or describes a capture from
`tools/capture.sh`, made after the change. Scenarios live in `test/capture/<name>.tscn`.
- **Open every capture and look at it** (the Read tool shows images). The harness only
catches blank frames. A broken, ugly or misleading capture means the issue isn't done.
- The default renderer is Forward+ through Mesa's Lavapipe, which matches the game.
`CAPTURE_RENDERER=compatibility` uses OpenGL through llvmpipe. Colours differ between
the two, so compare captures made with the same renderer.
- Software rendering shows what is drawn, not how fast. Never report frame rates from it.
## Rules ## Rules
- The engine lives in `tools/godot/`, is git-ignored, and is pinned by - The engine lives in `tools/godot/`, is git-ignored, and is pinned by
+5 -1
View File
@@ -1,7 +1,7 @@
# Tidewall build entry points. Everything runs headless with the repo-local engine. # Tidewall build entry points. Everything runs headless with the repo-local engine.
GODOT := tools/godot.sh GODOT := tools/godot.sh
.PHONY: check import parse test test-selfcheck smoke binaries verify-engine .PHONY: check import parse test test-selfcheck smoke capture-test binaries verify-engine
## Everything that must pass before a push. ## Everything that must pass before a push.
check: verify-engine binaries parse test test-selfcheck smoke check: verify-engine binaries parse test test-selfcheck smoke
@@ -29,3 +29,7 @@ test-selfcheck: import
## Boots the headless server for 1,000 ticks and checks exit codes, logs and timing. ## Boots the headless server for 1,000 ticks and checks exit codes, logs and timing.
smoke: import smoke: import
@tools/smoke.sh @tools/smoke.sh
## Renders the capture test scenes in the offscreen container. Needs Docker.
capture-test: import
@tools/capture_selfcheck.sh
+3
View File
@@ -0,0 +1,3 @@
[gd_scene format=3]
[node name="CaptureBlank" type="Node"]
+6
View File
@@ -0,0 +1,6 @@
[gd_scene load_steps=2 format=3]
[ext_resource type="Script" path="res://test/capture/smoke_scene.gd" id="1"]
[node name="CaptureSmoke" type="Node3D"]
script = ExtResource("1")
+67
View File
@@ -0,0 +1,67 @@
## The capture harness's test scene: a small shore with sea, sand, a breakwater and a
## post, under a procedural sky. Everything is built in code, so there are no assets.
extends Node3D
func _ready() -> void:
var environment: Environment = Environment.new()
var sky_material: ProceduralSkyMaterial = ProceduralSkyMaterial.new()
sky_material.sky_top_color = Color(0.32, 0.5, 0.68)
sky_material.sky_horizon_color = Color(0.82, 0.78, 0.68)
sky_material.ground_bottom_color = Color(0.18, 0.2, 0.2)
var sky: Sky = Sky.new()
sky.sky_material = sky_material
environment.background_mode = Environment.BG_SKY
environment.sky = sky
environment.ambient_light_source = Environment.AMBIENT_SOURCE_SKY
environment.ambient_light_energy = 0.35
environment.tonemap_mode = Environment.TONE_MAPPER_FILMIC
var world: WorldEnvironment = WorldEnvironment.new()
world.environment = environment
add_child(world)
var sun: DirectionalLight3D = DirectionalLight3D.new()
sun.rotation_degrees = Vector3(-38.0, -50.0, 0.0)
sun.light_energy = 0.75
sun.shadow_enabled = true
add_child(sun)
_box(Vector3(60.0, 1.0, 60.0), Vector3(0.0, -0.5, -10.0), Color(0.58, 0.5, 0.36))
_box(Vector3(200.0, 0.2, 200.0), Vector3(0.0, -0.3, 80.0), Color(0.07, 0.2, 0.26))
for index: int in 6:
var height: float = 1.4 + 0.25 * (index % 2)
var x: float = -7.5 + index * 3.0
_box(Vector3(2.6, height, 2.0), Vector3(x, height * 0.5, 4.0), Color(0.45, 0.42, 0.38))
_cylinder(0.35, 5.0, Vector3(5.0, 2.5, -2.0), Color(0.55, 0.36, 0.22))
_cylinder(0.6, 0.8, Vector3(5.0, 5.4, -2.0), Color(0.72, 0.22, 0.18))
var camera: Camera3D = Camera3D.new()
camera.position = Vector3(-6.0, 6.0, -16.0)
add_child(camera)
camera.look_at(Vector3(1.0, 1.0, 2.0))
camera.make_current()
func _box(size: Vector3, at: Vector3, colour: Color) -> void:
var mesh: BoxMesh = BoxMesh.new()
mesh.size = size
_place(mesh, at, colour)
func _cylinder(radius: float, height: float, at: Vector3, colour: Color) -> void:
var mesh: CylinderMesh = CylinderMesh.new()
mesh.top_radius = radius
mesh.bottom_radius = radius
mesh.height = height
_place(mesh, at, colour)
func _place(mesh: PrimitiveMesh, at: Vector3, colour: Color) -> void:
var material: StandardMaterial3D = StandardMaterial3D.new()
material.albedo_color = colour
material.roughness = 0.85
mesh.material = material
var instance: MeshInstance3D = MeshInstance3D.new()
instance.mesh = mesh
instance.position = at
add_child(instance)
+1
View File
@@ -0,0 +1 @@
uid://f5bdhjyvldby
+48
View File
@@ -0,0 +1,48 @@
extends TestCase
func _solid(colour: Color) -> Image:
var image: Image = Image.create(64, 64, false, Image.FORMAT_RGBA8)
image.fill(colour)
return image
func test_solid_frame_is_blank() -> void:
var stats: Dictionary = ImageStats.measure(_solid(Color(0.3, 0.3, 0.3)))
assert_near(stats["variance"], 0.0, 0.000001)
assert_eq(stats["colours"], 1)
assert_true(ImageStats.is_blank(stats))
func test_two_flat_halves_are_still_blank() -> void:
# A cleared frame with one flat rectangle has variance but almost no colours.
var image: Image = _solid(Color.BLACK)
image.fill_rect(Rect2i(0, 0, 32, 64), Color.WHITE)
var stats: Dictionary = ImageStats.measure(image)
assert_true(stats["variance"] > ImageStats.BLANK_VARIANCE)
assert_true(ImageStats.is_blank(stats))
func test_gradient_with_detail_is_not_blank() -> void:
var image: Image = Image.create(128, 128, false, Image.FORMAT_RGBA8)
for y: int in 128:
for x: int in 128:
image.set_pixel(x, y, Color(x / 128.0, y / 128.0, float((x * y) % 7) / 7.0))
var stats: Dictionary = ImageStats.measure(image)
assert_eq(stats["samples"], (128 / ImageStats.STRIDE) * (128 / ImageStats.STRIDE))
assert_false(ImageStats.is_blank(stats))
func test_mean_of_known_image() -> void:
var stats: Dictionary = ImageStats.measure(_solid(Color.WHITE))
assert_near(stats["mean"], 1.0, 0.0001)
func test_empty_image() -> void:
var stats: Dictionary = ImageStats.measure(Image.create(1, 1, false, Image.FORMAT_RGBA8))
assert_eq(stats["samples"], 1)
func test_capture_scenes_exist() -> void:
assert_true(ResourceLoader.exists("res://test/capture/smoke.tscn"))
assert_true(ResourceLoader.exists("res://test/capture/blank.tscn"))
+1
View File
@@ -0,0 +1 @@
uid://bimwta516hla6
+61
View File
@@ -0,0 +1,61 @@
## Renders a scene for a number of frames and saves the last one as a PNG.
## Runs inside the tidewall-capture container via tools/capture.sh, never on the host
## display. Prints one JSON line with the image statistics and exits:
## 0 saved and not blank, 1 blank or failed, 2 bad arguments.
##
## --scene=res://...tscn --out=res://captures/x.png [--frames=N] [--allow-blank]
extends SceneTree
var _scene: String = ""
var _out: String = ""
var _frames: int = 30
var _allow_blank: bool = false
func _initialize() -> void:
for arg: String in OS.get_cmdline_user_args():
if arg.begins_with("--scene="):
_scene = arg.trim_prefix("--scene=")
elif arg.begins_with("--out="):
_out = arg.trim_prefix("--out=")
elif arg.begins_with("--frames=") and arg.trim_prefix("--frames=").is_valid_int():
_frames = maxi(arg.trim_prefix("--frames=").to_int(), 1)
elif arg == "--allow-blank":
_allow_blank = true
else:
printerr("capture: unknown argument '%s'" % arg)
quit(2)
return
if _scene.is_empty() or _out.is_empty():
printerr("capture: --scene and --out are required")
quit(2)
return
if DisplayServer.get_name() == "headless":
printerr("capture: no display; run this through tools/capture.sh")
quit(2)
return
_capture.call_deferred()
func _capture() -> void:
var packed: PackedScene = load(_scene) as PackedScene
if packed == null:
printerr("capture: can't load scene '%s'" % _scene)
quit(1)
return
root.add_child(packed.instantiate())
for _index: int in _frames:
await process_frame
await RenderingServer.frame_post_draw
var image: Image = root.get_texture().get_image()
var stats: Dictionary = ImageStats.measure(image)
var blank: bool = ImageStats.is_blank(stats)
var error: Error = image.save_png(_out)
stats["scene"] = _scene
stats["size"] = [image.get_width(), image.get_height()]
stats["renderer"] = RenderingServer.get_current_rendering_method()
stats["adapter"] = RenderingServer.get_video_adapter_name()
stats["blank"] = blank
stats["saved"] = error == OK
print("CAPTURE " + JSON.stringify(stats))
quit(0 if error == OK and (not blank or _allow_blank) else 1)
+1
View File
@@ -0,0 +1 @@
uid://00d2aqely5kg
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env bash
# Renders the real client offscreen and saves a PNG, without touching the host display,
# GPU or audio. Godot runs in a throwaway Docker container with a private Xvfb and Mesa
# software rendering.
#
# tools/capture.sh <scene-or-scenario> <out.png> [--frames=N] [--allow-blank]
#
# <scene-or-scenario> is a res:// path to a .tscn, or the name of a scenario in
# test/capture/ (for example `smoke` means res://test/capture/smoke.tscn).
#
# Environment:
# CAPTURE_SIZE=1280x720 frame size
# CAPTURE_RENDERER=forward_plus Vulkan via Lavapipe (default, the shipping renderer), or
# compatibility for OpenGL via llvmpipe
#
# Exits 0 when the PNG was written and isn't blank (pixel variance and colour count),
# 1 when the render failed or came out blank, 2 on bad usage. Open the image and look
# at it anyway: these checks only catch the obvious failures.
set -euo pipefail
repo="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
image="tidewall-capture"
context="$repo/tools/capture"
usage() {
sed -n '2,20p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//' >&2
exit 2
}
[[ $# -ge 2 ]] || usage
target="$1"
out="$2"
shift 2
extra=()
for arg in "$@"; do
case "$arg" in
--frames=* | --allow-blank) extra+=("$arg") ;;
*) echo "capture.sh: unknown option '$arg'" >&2; usage ;;
esac
done
if [[ "$target" == res://* ]]; then
scene="$target"
else
[[ "$target" =~ ^[a-z0-9_]+$ ]] || { echo "capture.sh: bad scenario name '$target'" >&2; exit 2; }
scene="res://test/capture/$target.tscn"
fi
[[ -f "$repo/${scene#res://}" ]] || { echo "capture.sh: no such scene: $scene" >&2; exit 2; }
size="${CAPTURE_SIZE:-1280x720}"
[[ "$size" =~ ^[0-9]+x[0-9]+$ ]] || { echo "capture.sh: bad CAPTURE_SIZE '$size'" >&2; exit 2; }
case "${CAPTURE_RENDERER:-forward_plus}" in
compatibility) render=(--rendering-method gl_compatibility --rendering-driver opengl3) ;;
forward_plus) render=(--rendering-method forward_plus --rendering-driver vulkan) ;;
*) echo "capture.sh: CAPTURE_RENDERER must be compatibility or forward_plus" >&2; exit 2 ;;
esac
# Rebuild the image only when its recipe changes.
recipe_hash="$(cat "$context/Dockerfile" "$context/entrypoint.sh" | sha256sum | cut -c1-16)"
current="$(docker image inspect "$image" --format '{{ index .Config.Labels "tidewall.recipe" }}' 2>/dev/null || true)"
if [[ "$current" != "$recipe_hash" ]]; then
echo "capture.sh: building $image" >&2
docker build --quiet --label "tidewall.recipe=$recipe_hash" -t "$image" "$context" >/dev/null
fi
# Refresh the global class cache headless on the host, so class_name lookups resolve.
"$repo/tools/godot.sh" --import --quit >/dev/null 2>&1 || true
# The container can only write inside the repo, so render to a scratch file there.
mkdir -p "$repo/captures"
# Keep Godot from importing captures as project resources.
touch "$repo/captures/.gdignore"
scratch="captures/.capture-$$-$RANDOM.png"
name="tidewall-capture-$$-$RANDOM"
cleanup() {
docker rm -f "$name" >/dev/null 2>&1 || true
rm -f "$repo/$scratch"
}
trap cleanup EXIT
set +e
output="$(timeout --signal=KILL 300 nice -n 15 docker run --rm --name "$name" \
--network none \
--user "$(id -u):$(id -g)" \
--memory 3g --cpus 4 \
-e "CAPTURE_SCREEN=${size}x24" \
-v "$repo:/repo" \
"$image" \
"${render[@]}" --resolution "$size" \
--script res://tools/capture.gd -- --scene="$scene" --out="res://$scratch" "${extra[@]}" 2>&1)"
code=$?
set -e
report="$(grep -m1 '^CAPTURE ' <<<"$output" | cut -c9- || true)"
if [[ -z "$report" ]]; then
echo "capture.sh: render failed (exit $code)" >&2
tail -n 30 <<<"$output" >&2
exit 1
fi
if [[ -f "$repo/$scratch" ]]; then
mkdir -p "$(dirname "$out")"
mv "$repo/$scratch" "$out"
fi
echo "$report"
# Godot silently falls back to OpenGL when Vulkan fails; that is not the requested capture.
wanted="${render[1]}"
if ! grep -q "\"renderer\":\"$wanted\"" <<<"$report"; then
echo "capture.sh: asked for the $wanted renderer, but Godot fell back to another" >&2
grep -iE 'vulkan|error' <<<"$output" | tail -n 10 >&2 || true
exit 1
fi
if [[ "$code" -ne 0 ]]; then
echo "capture.sh: capture of $scene failed or came out blank (exit $code)" >&2
grep -iE 'error|fail' <<<"$output" | grep -v '^CAPTURE ' | tail -n 15 >&2 || true
exit 1
fi
echo "capture.sh: wrote $out" >&2
+23
View File
@@ -0,0 +1,23 @@
# Offscreen render harness for tools/capture.sh: a private X server (Xvfb) and Mesa's
# software rasterizers, so Godot can render the real client without any host display,
# GPU or audio device. Built as image tidewall-capture.
FROM debian:13
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
xvfb xauth \
libgl1 libgl1-mesa-dri libglx-mesa0 libegl1 libgles2 \
mesa-vulkan-drivers libvulkan1 \
libx11-6 libxcursor1 libxinerama1 libxrandr2 libxi6 libxext6 libxrender1 \
libxkbcommon0 libxkbcommon-x11-0 \
libfontconfig1 libdbus-1-3 libasound2t64 libpulse0 libudev1 \
&& rm -rf /var/lib/apt/lists/*
# Never touch a real GPU, even if a device were passed in by mistake.
ENV LIBGL_ALWAYS_SOFTWARE=1 \
GALLIUM_DRIVER=llvmpipe \
VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.json \
HOME=/tmp/home
COPY entrypoint.sh /usr/local/bin/tidewall-capture
ENTRYPOINT ["/usr/local/bin/tidewall-capture"]
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
# Runs inside the tidewall-capture container. Starts a private Xvfb, then runs the
# repo's pinned Godot against it. Arguments go to Godot after the repo's own flags.
set -euo pipefail
mkdir -p "$HOME"
export DISPLAY=:77
Xvfb "$DISPLAY" -screen 0 "${CAPTURE_SCREEN:-1280x720x24}" -nolisten tcp -noreset \
>/tmp/xvfb.log 2>&1 &
xvfb_pid=$!
trap 'kill "$xvfb_pid" 2>/dev/null || true' EXIT
for _ in $(seq 1 50); do
[[ -e /tmp/.X11-unix/X77 ]] && break
sleep 0.1
done
if [[ ! -e /tmp/.X11-unix/X77 ]]; then
echo "tidewall-capture: Xvfb did not start" >&2
cat /tmp/xvfb.log >&2
exit 3
fi
# This container's X server is private, so a window here is allowed.
export TIDEWALL_GODOT_DISPLAY=1
/repo/tools/godot.sh --display-driver x11 --audio-driver Dummy "$@"
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env bash
# Proves the capture harness works and catches blank renders. Needs Docker; not part of
# `make check`, because it builds an image and renders in software. Run: make capture-test
set -uo pipefail
cd "$(dirname "${BASH_SOURCE[0]}")/.."
status=0
dir="captures/selfcheck"
mkdir -p "$dir"
# expect <exit> <renderer> <scenario>
expect() {
local wanted="$1" renderer="$2" scenario="$3" code
CAPTURE_RENDERER="$renderer" tools/capture.sh "$scenario" "$dir/$scenario-$renderer.png" \
>"$dir/$scenario-$renderer.log" 2>&1
code=$?
if [[ "$code" -ne "$wanted" ]]; then
echo "capture-test FAIL $scenario ($renderer): exit $code, expected $wanted"
tail -n 15 "$dir/$scenario-$renderer.log"
status=1
else
echo "capture-test ok $scenario ($renderer): exit $code, $(head -n 1 "$dir/$scenario-$renderer.log" | grep -oE '"variance":[0-9.e-]+')"
fi
}
expect 0 forward_plus smoke
expect 0 compatibility smoke
expect 1 forward_plus blank
leftover="$(docker ps -a --filter name=tidewall-capture- --format '{{.Names}}')"
if [[ -n "$leftover" ]]; then
echo "capture-test FAIL: containers left behind: $leftover"
status=1
fi
[[ "$status" -eq 0 ]] && echo "capture-test: ok. Now open $dir/smoke-forward_plus.png and look at it."
exit "$status"
+45
View File
@@ -0,0 +1,45 @@
## Measures a captured image, so a blank or broken render fails automatically.
## A capture still has to be looked at by a person or agent; this only catches the
## obvious failures.
class_name ImageStats
extends RefCounted
## Below this luminance variance (on a 0..1 scale) an image counts as blank.
const BLANK_VARIANCE: float = 0.0005
## A real scene has far more distinct colours than a cleared or broken frame.
const BLANK_MAX_COLOURS: int = 16
## Sampling stride, in pixels. Enough for a 1280x720 frame in milliseconds.
const STRIDE: int = 4
## Returns mean and variance of luminance, and the number of distinct sampled colours.
static func measure(image: Image) -> Dictionary:
var source: Image = image
if source.is_compressed():
source = image.duplicate() as Image
source.decompress()
var sum: float = 0.0
var sum_squares: float = 0.0
var samples: int = 0
var colours: Dictionary = {}
for y: int in range(0, source.get_height(), STRIDE):
for x: int in range(0, source.get_width(), STRIDE):
var colour: Color = source.get_pixel(x, y)
var luminance: float = colour.get_luminance()
sum += luminance
sum_squares += luminance * luminance
samples += 1
colours[colour.to_rgba32()] = true
if samples == 0:
return {"samples": 0, "mean": 0.0, "variance": 0.0, "colours": 0}
var mean: float = sum / samples
return {
"samples": samples,
"mean": mean,
"variance": maxf(sum_squares / samples - mean * mean, 0.0),
"colours": colours.size(),
}
static func is_blank(stats: Dictionary) -> bool:
return stats["variance"] < BLANK_VARIANCE or stats["colours"] <= BLANK_MAX_COLOURS
+1
View File
@@ -0,0 +1 @@
uid://dwf5l8spiiso2