F4 Headless test harness #4

Closed
opened 2026-11-09 05:26:03 +00:00 by claude · 1 comment
Owner

Depends on: F1

Write a small in-repo test runner with no third-party addon. It runs under --headless.

Acceptance

  • Discovers test/**/test_*.gd, reports pass and fail per test, and exits non-zero on failure.
  • Treats script errors and engine error output during a test as failures.
  • make test runs it, and make check includes it.
  • A deliberately failing fixture proves that failures are detected.
**Depends on:** F1 Write a small in-repo test runner with no third-party addon. It runs under `--headless`. **Acceptance** - Discovers `test/**/test_*.gd`, reports pass and fail per test, and exits non-zero on failure. - Treats script errors and engine error output during a test as failures. - `make test` runs it, and `make check` includes it. - A deliberately failing fixture proves that failures are detected.
claude added this to the v0.1 Foundations milestone 2026-11-09 05:26:03 +00:00
claude added the type/chorepriority/high labels 2026-11-09 05:26:03 +00:00
claude added this to the Tidewall v1.0 project 2026-11-09 15:00:04 +00:00
claude moved this to To Do in Tidewall v1.0 on 2026-11-09 15:00:19 +00:00
eric referenced this issue from a commit 2026-11-09 22:13:16 +00:00
eric closed this issue 2026-11-09 22:13:16 +00:00
Author
Owner

Done in dee417d. The runner in test/framework/ discovers test/**/test_*.gd, reports pass or fail per test, and exits non-zero on failure. It uses Godot's Logger API, so engine and script errors during a test fail that test, unless the test calls expect_error(). Finding no tests also counts as a failure.

make test runs it and make check includes it. make test-selfcheck runs fixtures in test/fixtures/ that must fail for specific reasons: a failed assertion, a null dereference, an engine error, a parse error, a script that isn't a test, an expected error that never happens, and an empty directory. A passing fixture must exit 0. All 8 behave as expected. The self-check found a real bug in the runner's load-failure path during development, now fixed.

A hang is caught by the 900-second timeout in tools/run_tests.sh, not per test.

Done in dee417d. The runner in `test/framework/` discovers `test/**/test_*.gd`, reports pass or fail per test, and exits non-zero on failure. It uses Godot's `Logger` API, so engine and script errors during a test fail that test, unless the test calls `expect_error()`. Finding no tests also counts as a failure. `make test` runs it and `make check` includes it. `make test-selfcheck` runs fixtures in `test/fixtures/` that must fail for specific reasons: a failed assertion, a null dereference, an engine error, a parse error, a script that isn't a test, an expected error that never happens, and an empty directory. A passing fixture must exit 0. All 8 behave as expected. The self-check found a real bug in the runner's load-failure path during development, now fixed. A hang is caught by the 900-second timeout in `tools/run_tests.sh`, not per test.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: claude/tidewall#4