Bounded reconciliation increment

docs/canonical-indexes.md

@@ -1,6 +1,6 @@
 # M2 increment — canonical indexes and recoverable market observation
 
-Status: implemented local infrastructure, **not full M2 acceptance or production authorization**. Evidence date: 2026-10-20. Related: [production plan](production-plan.md), [custodial design](custodial-market-design.md).
+Status: implemented local infrastructure, **not full M2 acceptance or production authorization**. Evidence date: 2026-10-21. Related: [production plan](production-plan.md), [custodial design](custodial-market-design.md).
 
 ## Delivered
 
@@ -9,13 +9,15 @@ Status: implemented local infrastructure, **not full M2 acceptance or production
 - Events contain sequence, connect/disconnect type, block identity/height/parent, transaction IDs and a hash chained to the previous event. An independently generated local stream ID binds the history; it is **not consensus data or a public proof of honest observation**. Pages contain at most 64 events and no article payloads. Financial/chain history is not silently pruned.
 - On first open, original blocks are replayed and independently proof/state validated before additive local index backfill. Later opens compare indexed block identity, ownership, transaction inclusion and event continuity/anchors to that validated chain. Inconsistent state fails closed; existing consensus version, genesis, signatures and free-transfer/trade rules are unchanged.
 - [Market reconciliation](../server/market/store.ts) uses indexed current delivery/ownership queries and a persisted `(stream ID, sequence, anchor)` checkpoint. Every observed disconnect of a previously completed order causes a durable review hold, even if delivery reappeared before polling. Review is never automatically repaid. Hold updates and checkpoint advancement commit together in the market database.
-- Catch-up consumes one bounded event page per reconciliation pass. While behind, ordinary market mutation/payout/expiry decisions stop; diagnostic/read access remains available and exposes event cursor/head/backlog. Periodic polling and subsequent requests advance catch-up. No claim of instant catch-up or background-thread isolation.
+- Order and listing review now uses stable-ID keyset batches capped at 128 rows each. A strict, durable job record stores stage, continuation ID, observed time, chain tip/revision, and the bounded canonical event page. Each batch commits row effects and its continuation atomically; the canonical event cursor advances only after both tables finish. A chain tip/revision change during a multi-batch scan rewinds the job and captures a fresh event page, preventing a mixed-view scan from being marked caught up. Completion checks the current event head so events appended during the scan remain behind the write fence.
+- While any job/event backlog exists, ordinary market mutations and time-dependent payout/expiry decisions stop; diagnostic/read access remains available. Subsequent requests advance bounded work, and the HTTP lifecycle has an unref'd 250ms poller that drives catch-up independently of user page loads. The poller is not a separately isolated worker: SQLite work is synchronous on the server event loop. There is no claim of instant catch-up, a throughput SLO, or horizontal-worker safety.
+- Reconciliation restart, mid-job event append/reorg fencing, disconnect review, and injected SQLite batch/cursor failures are tested. A failed batch leaves both order effects and durable keyset progress unchanged; restart resumes committed work. Restore mismatch (different stream, different history anchor or a cursor ahead of the restored ledger) is refused rather than resetting the cursor. Restore the matching stopped-node ledger/market backup; any alternative recovery requires reviewed evidence, not deleting a checkpoint.
 - Restore mismatch (different stream, different history anchor or a cursor ahead of the restored ledger) is refused rather than resetting the cursor. Restore the matching stopped-node ledger/market backup; any alternative recovery requires reviewed evidence, not deleting a checkpoint.
 - [HTTP NFT lookup and custody card checks](../server/testnet/http.ts) use the same index. Wallet deck pagination accepts an optional `tip` query; stale tips return 409. Existing callers without `tip` remain compatible. There is no new public event/admin endpoint, no payment observer and no NFT-lock signer.
 
 ## Tests and measurements
 
-The final Docker test-target build passed TypeScript/Vite compilation. All **149 tests passed, zero failures/skips**, including native RandomX multi-process HTTP/peer/restart and backup rehearsal. New failure evidence includes:
+The final Docker test-target build passed TypeScript/Vite compilation. All **174 offline tests passed, zero failures/skips**, including native RandomX multi-process HTTP/peer/restart and backup rehearsal. The focused market suite passed **23 tests**; the disposable PostgreSQL suite passed **31 tests**. New failure evidence includes:
 
 - indexed ownership/sequence versus full replay for transfers, multi-card trades and forks;
 - append/disconnect/reconnect ordering, side-branch/mempool exclusion, bounded defensive-copy pages and durable restart;
@@ -23,7 +25,9 @@ The final Docker test-target build passed TypeScript/Vite compilation. All **149
 - additive migration of a prior-schema copy, refusing corrupt ownership and event records;
 - production market adapter exercised with full snapshots forbidden;
 - multi-page catch-up and persisted restart checkpoint;
-- injected cursor-write failure rolling back the payout hold, followed by restart recovery;
+- 128-row order/listing keyset progress, restart continuation, and a new canonical event arriving during an in-flight job;
+- chain revision change fencing that resets continuation and refreshes the event page before the scan continues;
+- injected batch-write failure rolling back order effects and keyset progress, plus cursor-write failure rolling back checkpoint advancement while preserving committed review holds;
 - numeric cursor reuse with a different historical hash rejected;
 - real HTTP NFT results and stale-tip deck pagination.
 
@@ -44,6 +48,6 @@ These numbers are a reproducible **baseline**, not acceptance SLOs: host CPU/sto
 
 ## Remaining gates
 
-M2 still needs target-hardware/data/SLO decisions, market-order incremental scheduling rather than scanning every existing bounded order/listing, bounded worker/event-loop latency under load, production financial database/schema and measured recovery/retention/storage growth. Ledger PoW replay and DAG state memory still scale with retained history; indexed queries do not solve those costs. Startup validation checks undo-record counts but not every possible future undo mutation against a second reconstructed undo model; comprehensive corruption/fuzz coverage and independent review remain required.
+M2 still needs target-hardware/data/SLO decisions, populated long-journal/large-event benchmarks, bounded worker/event-loop latency under load, production financial database/schema and measured recovery/retention/storage growth. The new scheduler bounds work per pass, but does not yet provide queue isolation, multi-worker coordination, or a demonstrated production throughput envelope. Ledger PoW replay and DAG state memory still scale with retained history; indexed queries do not solve those costs. Startup validation checks undo-record counts but not every possible future undo mutation against a second reconstructed undo model; comprehensive corruption/fuzz coverage and independent review remain required.
 
 Custodial reservations still stop at `awaiting-lock`. This increment does **not** consume ordinary transfers as custodial settlement evidence: doing that would bypass the reviewed sale-lock/cancel protocol. Real Monero deposit/withdrawal integration, production treasury keys, host attestations and a reviewed versioned NFT lock/completion/cancellation specification remain blocked. No real funds, credentials, external processor or genesis change were used.