From 8cfcf6c025d159a49af5d346b2e6d5fe3c6ba136 Mon Sep 17 00:00:00 2001 From: Eric Meehan Date: Tue, 20 Oct 2026 19:31:08 -0400 Subject: [PATCH] Strengthened the disposable reorg rehearsal --- docs/custody-sale-rehearsal.md | 6 +++--- docs/market-host-configuration.md | 2 +- docs/production-plan.md | 2 +- server/market/canonical-inbox.ts | 23 ++++++++++++++++++++++- server/market/postgres-custody.ts | 27 +++++++++++++++++++++++---- tests/postgres/custody.test.ts | 17 ++++++++++++----- 6 files changed, 62 insertions(+), 15 deletions(-) diff --git a/docs/custody-sale-rehearsal.md b/docs/custody-sale-rehearsal.md index 74c016c..ff90500 100755 --- a/docs/custody-sale-rehearsal.md +++ b/docs/custody-sale-rehearsal.md @@ -37,13 +37,13 @@ The same explicitly disposable PostgreSQL instance now has `observeCanonicalEven The adapter is not automatically scheduled: there is no HTTP route or background worker. PostgreSQL tests include synthetic event pages for malformed/retry cases and one integration using blocks accepted by a local `Ledger` configured with a **test-only fake PoW engine**, including an equal-work side branch and a reorg. This tests the local adapter/API boundary, not real RandomX validation, external provenance or production multi-node behavior. Hash-anchor consistency is not proof of honest observation; the stream ID and unkeyed hashes are stored beside the events, so a schema owner able to coherently rewrite the complete history can also recompute the hashes. The included corruption test detects a partial edit, not a fully rewritten valid history. Production must anchor custody checkpoints outside the database's mutable trust domain. The disposable inbox is capped at 4,096 total events, stores the append-only event history and fully rescans it during audits. That is intentionally a small-fixture design, not production storage/cursor architecture or a scale claim. -The synthetic PostgreSQL event-page seam now links an explicitly terminal candidate transaction ID to its posted order. A later **submitted disconnect event** containing that exact ID rolls back inbox advancement and commits a durable `posted-sale-terminal-disconnected` hold; the original journal is not reversed or paid again. This exercises order-to-event linkage and atomic review behavior. The hold only detects disconnects actually delivered to this seam; it cannot detect an unseen intermediate fork. +The synthetic PostgreSQL event-page seam now links an explicitly terminal candidate transaction ID to its posted order. A later **submitted disconnect event** containing that exact ID commits both the canonical event page and a durable per-order `sale-reorg-review` record plus global `posted-sale-terminal-disconnected` mutation hold in one transaction; the original journal is not reversed or paid again. This remains true if the transaction reappears later in the same replacement-branch page: review does not auto-clear on apparent re-inclusion. Generic incident resolution re-audits the retained review record and therefore cannot clear it. This exercises order-to-event linkage and atomic review behavior. The hold only detects disconnects actually delivered to this seam; it cannot detect an unseen intermediate fork. The local-ledger adapter is **not** linked to these terminal candidate IDs: active v6 blocks cannot contain candidate sale transactions, and local free transfers are not accepted as sale completion evidence. The adapter's integration test demonstrates canonical event ingestion/reorg only, not sale rollback handling. The terminal-link test supplies a synthetic canonical page under disposable-only guards. Production still requires the candidate transaction format in a separately reviewed/activated network, an actual local canonical feed containing those IDs, and per-order integration with durable node history. No production sale-observation guarantee is claimed. ## Validation and remaining gates -TypeScript/Vite Docker build passed; **172 offline regressions and 27 PostgreSQL integration tests passed, zero failures/skips**. [Candidate tests](../tests/sale-candidate.test.ts) cover signature compatibility/recipient tampering and candidate transition rules. [PostgreSQL tests](../tests/postgres/custody.test.ts) add: +TypeScript/Vite Docker build passed; **172 offline regressions and 28 PostgreSQL integration tests passed, zero failures/skips**. [Candidate tests](../tests/sale-candidate.test.ts) cover signature compatibility/recipient tampering and candidate transition rules. [PostgreSQL tests](../tests/postgres/custody.test.ts) add: - explicit instance/database consent and exact signed reservation matching; - seed-binding changes, malformed/cyclic/oversized collections, and denial of ordinary cancellation after binding; @@ -53,7 +53,7 @@ TypeScript/Vite Docker build passed; **172 offline regressions and 27 PostgreSQL - one-atomic settlement and signed custom-recipient revenue across current-policy changes; - conflicting outcome → durable cross-instance/restart hold without a second credit; - anchored canonical-event connect/disconnect paging, overlapping/stale consumer retries, reconnect, partial history mutation audit, forged disconnect-ancestor rejection and tamper-triggered hold; actual locally validated blocks using test-only fake PoW, excluded equal-work branch and a canonical reorg through the adapter (no candidate sale link); -- synthetic exact candidate-terminal transaction disconnect → order-linked durable hold, one preserved seller credit and no second payout; +- synthetic exact candidate-terminal transaction disconnect/reappearance in one page → committed event history plus order-linked durable hold, generic incident resolution denied, one preserved seller credit and no second payout; - injected terminal-evidence failure → complete rollback and safe retry; corrupted evidence → audit hold. A first bounded-checkpoint implementation attempted metadata UPDATE and was rejected by the real runtime grants. It was corrected to retain insert-only first/terminal evidence; no privilege was widened. Test evidence is for this implementation boundary, not production approval. diff --git a/docs/market-host-configuration.md b/docs/market-host-configuration.md index 5743457..902feff 100755 --- a/docs/market-host-configuration.md +++ b/docs/market-host-configuration.md @@ -32,4 +32,4 @@ Existing signature bytes remain valid: omission retains the original message; ex The custody SQLite fixture records newly settled simulated host revenue under its snapshotted recipient. PostgreSQL production settlement/host withdrawals are still unimplemented: configuring an address does not create automatic transfers. Historical unallocated host revenue remains historical; it is not silently reassigned. The offline sale-lock candidate now preserves the hosting recipient in a tagged signed-terms extension. A [disposable PostgreSQL accounting rehearsal](custody-sale-rehearsal.md) verifies that the signed snapshot receives simulated revenue even after current host policy changes; this is not active-chain delivery or a live host withdrawal service. -Validation: current TypeScript/Vite build, 172 regression tests and 22 PostgreSQL integration tests passed. Tests cover strict flag/address validation, paused/disabled behavior, continuing existing-order delivery, preserved balances/cancellations, signed-recipient tampering and recipient snapshots across config changes, including the isolated accounting rehearsal. Real XMR remains disabled. +Validation: current TypeScript/Vite build, 172 offline regressions and 28 PostgreSQL integration tests passed. Tests cover strict flag/address validation, paused/disabled behavior, continuing existing-order delivery, preserved balances/cancellations, signed-recipient tampering and recipient snapshots across config changes, including the isolated accounting rehearsal. Real XMR remains disabled. diff --git a/docs/production-plan.md b/docs/production-plan.md index 70959df..7351f2f 100755 --- a/docs/production-plan.md +++ b/docs/production-plan.md @@ -88,7 +88,7 @@ Blocked on P0 ADR; use P1 amount types and P2 durable events. Consensus work is **Accounting rehearsal increment, 2026-10-20:** [signed candidate → PostgreSQL reservation resolution](custody-sale-rehearsal.md) now exercises actual restricted-role SQL terminal postings from bounded synthetic transcripts, behind explicit disposable database/instance guards and without an HTTP route. Exact terms/hosting recipients and reservation commitments are checked; mature completion credits seller/fees once, cancellation/non-admission releases buyer funds, and conflicting post-settlement evidence durably holds mutations rather than repaying. Insert-only first/terminal evidence is audited. Final build, 172 offline tests and 22 PostgreSQL integration tests passed. This supersedes the earlier absence of any balance-posting test integration, **not** the production/activation boundary: caller-supplied seeds/blocks are not canonical evidence, intermediate unseen disconnects cannot be detected, and no signer, canonical inbox, live deposit/withdrawal or current-chain paid sale is enabled. M3 remains open. -**Canonical/per-order reorg rehearsal increment, 2026-10-20:** the gated disposable PostgreSQL observer stores bounded pages from an in-process `Ledger` adapter, checks stream/anchor/hash/tip/ancestor continuity, supports retries, and audits retained history. A local-ledger test using fake PoW covers an excluded equal-work branch and reorg. A separate synthetic-event test links the deterministic candidate terminal transaction ID to its posted order; disconnecting that ID commits a durable review hold without undoing or repeating the payout. Final build and 28 PostgreSQL tests pass; offline regressions remain 172. The worker is not scheduled, active v6 cannot emit candidate IDs, and synthetic linkage is not canonical sale evidence. Stream hashes are not tamper-proof against the schema owner; 4,096-event cap/full scan are non-production. M3 remains blocked on reviewed candidate activation, actual candidate IDs from a production local ledger, observer/checkpoint design and production readiness. +**Canonical/per-order reorg rehearsal increment, 2026-10-20:** the gated disposable PostgreSQL observer stores bounded pages from an in-process `Ledger` adapter, checks stream/anchor/hash/tip/ancestor continuity, supports retries, and audits retained history. A local-ledger test using fake PoW covers an excluded equal-work branch and reorg. A separate synthetic-event test links the deterministic candidate terminal transaction ID to its posted order; disconnecting that ID (even followed by same-page re-inclusion) commits the event history and durable order review hold atomically, without undoing or repeating the payout. Generic incident resolution cannot clear a sale-reorg record. Final build and 28 PostgreSQL tests pass; offline regressions remain 172. The worker is not scheduled, active v6 cannot emit candidate IDs, and synthetic linkage is not canonical sale evidence. Stream hashes are not tamper-proof against the schema owner; 4,096-event cap/full scan are non-production. M3 remains blocked on reviewed candidate activation, actual candidate IDs from a production local ledger, observer/checkpoint design and production readiness. - If a reviewed trustless design is selected: implement its actual cross-chain validity/evidence rules, compatible cryptographic primitives, both-party failure/refund paths and commission semantics. Run maintained reference vectors, negative tests, fuzz/property tests and specialist audit before claiming atomicity. - If a custodial/attested fallback is explicitly approved: define chain-enforced NFT lock, order identity, authorized releaser, cancellation/refund/expiry and stale-signature rules. Separate host control keys from seller wallet and processor payout keys. Reserve/refund procedures must not permit both buyer refund and later seller delivery/release. Describe who can steal, censor or lock assets and how recovery is authorized. diff --git a/server/market/canonical-inbox.ts b/server/market/canonical-inbox.ts index ba79895..89e965a 100755 --- a/server/market/canonical-inbox.ts +++ b/server/market/canonical-inbox.ts @@ -12,7 +12,7 @@ const pageSchema = z.object({ streamId: z.string().uuid(), previousAnchor: hex, head: z.number().int().nonnegative().safe(), caughtUp: z.boolean() }).strict(); export type CanonicalInboxPage = z.infer; export type CanonicalInboxResult = { streamId: string; cursor: number; head: number; height: number; tip: string; - replayed: boolean; appendedEvents: z.infer[] }; + replayed: boolean; appendedEvents: z.infer[]; reviewRequired?: boolean }; export class CanonicalInboxError extends Error { readonly code: string; @@ -127,4 +127,25 @@ export async function ingestCanonicalPage(client: PoolClient, raw: unknown): Pro /** Full append-only inbox verification. Expected to run under the caller's coherent audit snapshot. */ export async function auditCanonicalInbox(client: PoolClient): Promise { await readCanonicalHistory(client); + const reviews = await client.query(`SELECT key,value FROM wk_custody.custody_meta + WHERE key LIKE 'sale-reorg-review:%' ORDER BY key`); + for (const row of reviews.rows) { + let record: unknown; + try { record = JSON.parse(row.value); } catch { throw new CanonicalInboxError('sale-review-record-invalid'); } + const schema = z.object({ version: z.literal(1), streamId: z.string().uuid(), orderId: z.string().uuid(), + terminalTransactionId: hex, stateRoot: hex, eventSequence: z.number().int().positive().safe(), eventHash: hex }).strict(); + const parsed = schema.safeParse(record), keyMatch = /^sale-reorg-review:([a-f0-9-]{36}):(\d+)$/.exec(row.key); + if (!parsed.success || !keyMatch || keyMatch[1] !== parsed.data.orderId || Number(keyMatch[2]) !== parsed.data.eventSequence) { + throw new CanonicalInboxError('sale-review-record-invalid'); + } + const event = await client.query('SELECT value FROM wk_custody.custody_meta WHERE key=$1', + [eventKey(parsed.data.eventSequence)]); + if (!event.rowCount) throw new CanonicalInboxError('sale-review-event-missing'); + const disconnected = storedEvent(event.rows[0].value); + if (disconnected.type !== 'disconnect' || disconnected.eventHash !== parsed.data.eventHash || + !disconnected.transactionIds.includes(parsed.data.terminalTransactionId)) { + throw new CanonicalInboxError('sale-review-event-mismatch'); + } + } + if (reviews.rowCount) throw new CanonicalInboxError('posted-sale-terminal-disconnected'); } diff --git a/server/market/postgres-custody.ts b/server/market/postgres-custody.ts index 687aa8c..7f508c2 100755 --- a/server/market/postgres-custody.ts +++ b/server/market/postgres-custody.ts @@ -348,27 +348,46 @@ export class PostgresCustody { }); } /** Disposable observer rehearsal only; input must come from Ledger.canonicalEvents(), never peers/HTTP. */ - observeCanonicalEvents(page: unknown) { + async observeCanonicalEvents(page: unknown) { check(this.options.disposableSaleRehearsal === true, 'Canonical event inbox rehearsal is unavailable on this custody instance'); - return this.transaction(true, async c => { + const result = await this.transaction(true, async c => { check(!this.diagnostics().mutationHold, 'Custody reconciliation hold'); check(!(await c.query("SELECT 1 FROM wk_custody.custody_meta WHERE key='integrity_incident_v1'")).rowCount, 'Custody integrity incident unresolved'); try { const result = await ingestCanonicalPage(c, page); + const reviews: { orderId: string; terminalTransactionId: string; stateRoot: string; eventSequence: number; eventHash: string }[] = []; for (const event of result.appendedEvents) { if (event.type !== 'disconnect') continue; for (const txid of event.transactionIds) { const terminal = await c.query('SELECT value FROM wk_custody.custody_meta WHERE key=$1', [`sale-rehearsal-tx:${txid}`]); - if (terminal.rowCount) throw new CustodyIntegrityError('posted-sale-terminal-disconnected'); + if (terminal.rowCount) { + const link = JSON.parse(terminal.rows[0].value); + if (link.version !== 1 || typeof link.orderId !== 'string' || typeof link.stateRoot !== 'string') { + throw new CustodyIntegrityError('sale-terminal-index-invalid'); + } + reviews.push({ orderId: link.orderId, terminalTransactionId: txid, stateRoot: link.stateRoot, + eventSequence: event.sequence, eventHash: event.eventHash }); + } } } - return result; + for (const review of reviews) { + await c.query('INSERT INTO wk_custody.custody_meta(key,value) VALUES($1,$2) ON CONFLICT(key) DO NOTHING', + [`sale-reorg-review:${review.orderId}:${review.eventSequence}`, JSON.stringify({ version: 1, + streamId: result.streamId, ...review })]); + } + if (reviews.length) await recordIntegrityIncident(c, 'posted-sale-terminal-disconnected'); + return { ...result, appendedEvents: [], reviewRequired: reviews.length > 0 }; } catch (error) { if (error instanceof CanonicalInboxError) throw new CustodyIntegrityError(`canonical-inbox-${error.code}`); throw error; } }); + if (result.reviewRequired) { + this.integrityHold = true; + this.auditState = { ...this.auditState, status: 'failed', lastAttemptAt: Date.now(), failureCode: 'posted-sale-terminal-disconnected' }; + } + return result; } /** Consume one bounded page directly from this process's locally validated ledger; no HTTP/peer feed. */ async observeLedgerCanonicalEvents(ledger: Ledger) { diff --git a/tests/postgres/custody.test.ts b/tests/postgres/custody.test.ts index f123e6d..afd5ab3 100755 --- a/tests/postgres/custody.test.ts +++ b/tests/postgres/custody.test.ts @@ -90,7 +90,7 @@ function canonicalEvents(streamId: string, linkedTxId = '1'.repeat(64)) { const a = 'a'.repeat(64), b = 'b'.repeat(64), c = 'c'.repeat(64); add('connect', 1, a, GENESIS_HASH, [linkedTxId]); add('disconnect', 1, a, GENESIS_HASH, [linkedTxId]); - add('connect', 1, b, GENESIS_HASH, ['2'.repeat(64)]); + add('connect', 1, b, GENESIS_HASH, [linkedTxId]); add('connect', 2, c, b, ['3'.repeat(64)]); return rows; } @@ -486,7 +486,7 @@ test('sale rehearsal requires explicit disposable instance consent and exact tra const o = await f.execute(buyer, { type: 'reserve', listingId: l.id, priceAtomicXmr: l.priceAtomicXmr, feeBps: 250 }) as CustodyOrder; const t1 = saleTranscript(o, f.market.id); assert.throws(() => f.market.observeSaleRehearsal(o.id, t1.input), /unavailable/); - assert.throws(() => f.market.observeCanonicalEvents({}), /unavailable/); + await assert.rejects(f.market.observeCanonicalEvents({}), /unavailable/); const rehearsal = await PostgresCustody.open(f.runtime, { ...options, disposableSaleRehearsal: true }); try { const wrong = structuredClone(t1.input); wrong.lock = { ...t1.input.lock, terms: { ...t1.input.lock.terms, priceAtomicXmr: '1' } } as typeof wrong.lock; @@ -646,13 +646,20 @@ test('disconnect of the exact posted candidate terminal transaction commits an o assert.equal(credited.availableAtomicXmr, '96500000000'); const first = await observer.observeCanonicalEvents(canonicalPage(streamId, events, 0, 1)); assert.equal(first.cursor, 1); - await assert.rejects(observer.observeCanonicalEvents(canonicalPage(streamId, events, 1, 1)), /posted-sale-terminal-disconnected/); - await assert.rejects(observer.observeCanonicalEvents(canonicalPage(streamId, events, 1, 1)), /hold|incident unresolved/); - assert.equal((await f.admin.query("SELECT count(*)::int AS n FROM wk_custody.custody_meta WHERE key LIKE 'canonical-inbox:v1:event:%'")).rows[0].n, 1); + // The same transaction reappears on the replacement branch in this page; still review, never auto-clear. + const review = await observer.observeCanonicalEvents(canonicalPage(streamId, events, 1, 2)); + assert.equal(review.reviewRequired, true); + assert.equal(review.cursor, 3); // Disconnect and reappearance are durably retained with the review marker. + await assert.rejects(observer.observeCanonicalEvents(canonicalPage(streamId, events, 1, 2)), /hold|incident unresolved/); + assert.equal((await f.admin.query("SELECT count(*)::int AS n FROM wk_custody.custody_meta WHERE key LIKE 'canonical-inbox:v1:event:%'")).rows[0].n, 3); assert.equal((await f.activity(seller)).availableAtomicXmr, credited.availableAtomicXmr); assert.equal((await f.admin.query("SELECT count(*)::int AS n FROM wk_custody.journals WHERE id=$1", [`settle:${o.id}`])).rows[0].n, 1); const incident = await f.admin.query("SELECT value FROM wk_custody.custody_meta WHERE key='integrity_incident_v1'"); assert.match(incident.rows[0].value, /posted-sale-terminal-disconnected/); + await assert.rejects(observer.audit(), /unresolved-integrity-incident/); + await assert.rejects(resolveCustodyIncident(f.admin, randomUUID()), /posted-sale-terminal-disconnected/); + assert.equal((await f.admin.query("SELECT count(*)::int AS n FROM wk_custody.custody_meta WHERE key LIKE 'sale-reorg-review:%'")).rows[0].n, 1); + await assert.rejects(PostgresCustody.open(f.runtime, options), /unresolved-integrity/); await assert.rejects(f.execute(other, { type: 'mock-credit' }), /incident is unresolved/); } finally { await observer.close(); await rehearsal.close(); } });