# M2 increment — canonical indexes and recoverable market observation

Status: implemented local infrastructure, **not full M2 acceptance or production authorization**. Evidence date: 2026-10-20. Related: [production plan](production-plan.md), [custodial design](custodial-market-design.md).

## Delivered

- [Canonical index](../server/testnet/canonical-index.ts) stores current NFT ownership/sequence, canonical mint/block identity and transaction inclusion in SQLite. Owner/mint ordering and transaction indexes replace whole-chain scans for deck, NFT and marketplace queries.
- [Ledger persistence](../server/testnet/ledger.ts) commits accepted blocks, pending transactions, index connect/disconnect undo and canonical events in **one SQLite transaction**. A losing side branch or mempool submission does not create a canonical event. Disconnects run tip-first; connects run ancestor-first. No await separates the database commit from updating the in-memory tip.
- Events contain sequence, connect/disconnect type, block identity/height/parent, transaction IDs and a hash chained to the previous event. An independently generated local stream ID binds the history; it is **not consensus data or a public proof of honest observation**. Pages contain at most 64 events and no article payloads. Financial/chain history is not silently pruned.
- On first open, original blocks are replayed and independently proof/state validated before additive local index backfill. Later opens compare indexed block identity, ownership, transaction inclusion and event continuity/anchors to that validated chain. Inconsistent state fails closed; existing consensus version, genesis, signatures and free-transfer/trade rules are unchanged.
- [Market reconciliation](../server/market/store.ts) uses indexed current delivery/ownership queries and a persisted `(stream ID, sequence, anchor)` checkpoint. Every observed disconnect of a previously completed order causes a durable review hold, even if delivery reappeared before polling. Review is never automatically repaid. Hold updates and checkpoint advancement commit together in the market database.
- Catch-up consumes one bounded event page per reconciliation pass. While behind, ordinary market mutation/payout/expiry decisions stop; diagnostic/read access remains available and exposes event cursor/head/backlog. Periodic polling and subsequent requests advance catch-up. No claim of instant catch-up or background-thread isolation.
- Restore mismatch (different stream, different history anchor or a cursor ahead of the restored ledger) is refused rather than resetting the cursor. Restore the matching stopped-node ledger/market backup; any alternative recovery requires reviewed evidence, not deleting a checkpoint.
- [HTTP NFT lookup and custody card checks](../server/testnet/http.ts) use the same index. Wallet deck pagination accepts an optional `tip` query; stale tips return 409. Existing callers without `tip` remain compatible. There is no new public event/admin endpoint, no payment observer and no NFT-lock signer.

## Tests and measurements

The final Docker test-target build passed TypeScript/Vite compilation. All **149 tests passed, zero failures/skips**, including native RandomX multi-process HTTP/peer/restart and backup rehearsal. New failure evidence includes:

- indexed ownership/sequence versus full replay for transfers, multi-card trades and forks;
- append/disconnect/reconnect ordering, side-branch/mempool exclusion, bounded defensive-copy pages and durable restart;
- injected event-write failure rolling back blocks, indexes and mempool together;
- additive migration of a prior-schema copy, refusing corrupt ownership and event records;
- production market adapter exercised with full snapshots forbidden;
- multi-page catch-up and persisted restart checkpoint;
- injected cursor-write failure rolling back the payout hold, followed by restart recovery;
- numeric cursor reuse with a different historical hash rejected;
- real HTTP NFT results and stale-tip deck pagination.

[Benchmark script](../scripts/benchmark-canonical-index.ts), available as `npm run benchmark:index -- 10000`, creates/removes its own temporary SQL database. Input is bounded to 1–100,000 synthetic blocks; **these are not valid mined blocks** and are never submitted to a ledger/peer. It measures only derived index behavior, not PoW, network throughput or operating capacity.

Observed in the development Docker environment:

| Metric | Result |
| --- | --- |
| Fixture | 10,000 NFTs/blocks, 9,998 transfers, 16 owner identities |
| SQL fixture build (single transaction) | 5,096 ms |
| 1,000 query groups (NFT + 24-item owner page + optional transaction lookup), median / p95 | 0.619 / 1.167 ms |
| Read all events, maximum 64 per page | 157 pages, 54.5 ms total |
| Verify derived indexes and anchored history | 524 ms |
| Database plus WAL before close | 76,856,784 bytes |

These numbers are a reproducible **baseline**, not acceptance SLOs: host CPU/storage and production concurrency were not selected, outputs include synthetic article payloads, warm caches apply, one transaction amortizes insertion fsync, and real validation/startup/HTTP/market workloads are excluded. They do not establish a supported production envelope.

## Remaining gates

M2 still needs target-hardware/data/SLO decisions, market-order incremental scheduling rather than scanning every existing bounded order/listing, bounded worker/event-loop latency under load, production financial database/schema and measured recovery/retention/storage growth. Ledger PoW replay and DAG state memory still scale with retained history; indexed queries do not solve those costs. Startup validation checks undo-record counts but not every possible future undo mutation against a second reconstructed undo model; comprehensive corruption/fuzz coverage and independent review remain required.

Custodial reservations still stop at `awaiting-lock`. This increment does **not** consume ordinary transfers as custodial settlement evidence: doing that would bypass the reviewed sale-lock/cancel protocol. Real Monero deposit/withdrawal integration, production treasury keys, host attestations and a reviewed versioned NFT lock/completion/cancellation specification remain blocked. No real funds, credentials, external processor or genesis change were used.