Market-journal reconciliation benchmark

This commit is contained in:
2026-10-21 10:06:56 -04:00
parent 1f11210b5b
commit 687b628424
4 changed files with 78 additions and 1 deletions
+2
View File
@@ -46,6 +46,8 @@ Observed in the development Docker environment:
These numbers are a reproducible **baseline**, not acceptance SLOs: host CPU/storage and production concurrency were not selected, outputs include synthetic article payloads, warm caches apply, one transaction amortizes insertion fsync, and real validation/startup/HTTP/market workloads are excluded. They do not establish a supported production envelope.
The synthetic market-journal harness [benchmark](../scripts/benchmark-market-reconciliation.ts), available as `npm run benchmark:market -- 10000`, separately measures bounded reconciliation of terminal order/listing records in its own temporary SQLite directory. On the same development Docker environment, 10,000 orders plus 10,000 listings completed in 157 reconciliation passes in 772 ms (about 25,903 records/s), with pass-latency p50/p95/max of 4.57/9.13/20.04 ms and a 7,180,288-byte SQLite file. This dataset intentionally avoids active-order delivery and listing validation branches; it measures local keyset/SQLite overhead only. It is not a target-hardware result or throughput/SLO claim and excludes concurrent HTTP traffic, event-loop interference from other work, PostgreSQL, chain validation and adversarial workloads.
## Remaining gates
M2 still needs target-hardware/data/SLO decisions, populated long-journal/large-event benchmarks, bounded worker/event-loop latency under load, production financial database/schema and measured recovery/retention/storage growth. The new scheduler bounds work per pass, but does not yet provide queue isolation, multi-worker coordination, or a demonstrated production throughput envelope. Ledger PoW replay and DAG state memory still scale with retained history; indexed queries do not solve those costs. Startup validation checks undo-record counts but not every possible future undo mutation against a second reconstructed undo model; comprehensive corruption/fuzz coverage and independent review remain required.
+1 -1
View File
@@ -63,7 +63,7 @@ Acceptance: exact-amount property/boundary tests, versioned signature tests, zer
Can run alongside P0/P1. Addresses SEC-01/SEC-06 rather than adding more fixed caps.
**Implemented M2 increments, 2026-10-20–21:** [canonical indexes and durable market observation](canonical-indexes.md). Canonical ownership/transaction SQL indexes now serve NFT, deck, custody availability and mock-market queries; per-block undo and anchored connect/disconnect events commit with ledger updates. Market checkpoint/reorg holds persist atomically and bounded catch-up blocks new mutations. Order/listing reconciliation now advances in durable, stable-ID keyset batches of at most 128 records per table, resumes after restart, re-fences when the chain tip/revision changes, and runs from an HTTP lifecycle poller rather than relying only on page loads. Final TypeScript/Vite build and **174 offline tests passed**; focused market tests passed 23/23 and the disposable PostgreSQL integration suite passed 31/31. A disposable 10,000-NFT SQL baseline is recorded, **not** a target-hardware SLO or complete M2 acceptance. Queue isolation, target-scale throughput/latency, production financial storage/recovery, retained-DAG capacity and operator dataset/SLO targets remain open.
**Implemented M2 increments, 2026-10-20–21:** [canonical indexes and durable market observation](canonical-indexes.md). Canonical ownership/transaction SQL indexes now serve NFT, deck, custody availability and mock-market queries; per-block undo and anchored connect/disconnect events commit with ledger updates. Market checkpoint/reorg holds persist atomically and bounded catch-up blocks new mutations. Order/listing reconciliation now advances in durable, stable-ID keyset batches of at most 128 records per table, resumes after restart, re-fences when the chain tip/revision changes, and runs from an HTTP lifecycle poller rather than relying only on page loads. Final TypeScript/Vite build and **174 offline tests passed**; focused market tests passed 23/23 and the disposable PostgreSQL integration suite passed 31/31. A new synthetic local benchmark processed 10,000 terminal orders plus 10,000 listings in 772 ms on the development Docker host; this is a reproducible harness result, **not** a target-hardware SLO or complete M2 acceptance. Queue isolation, target-scale/concurrent throughput and latency, production financial storage/recovery, retained-DAG capacity and operator dataset/SLO targets remain open.
**Next financial-storage increment:** [ADR 003 — PostgreSQL](postgres-financial-storage.md) selects and integrates the custody SQL backend with existing HTTP callers, exact constrained accounting, explicit migration/runtime roles, source-freezing SQLite import and dump/restore rehearsal. The adapter remains on the fictitious sandbox domain; it is not a production deposit/withdrawal worker. SQLite node-only backup deliberately refuses PostgreSQL-bound financial state. Production hardware/SLO tests, coordinated backup/PITR/failover, real payment schema/worker controls and independent review remain open, so this does not close M2/M4 or authorize funds.