Custodial financial storage

docs/custodial-market-design.md

@@ -15,7 +15,9 @@ The operator has authorized proceeding with implementation instead of the previo
 - [Custody UI](../src/CustodyPanel.tsx): marketplace “Custody preview” tab with reviewed signatures, private/paged balances and activity, test funding, listing/reservation and withdrawal-hold cancellation. A lost response retains the original signed envelope for replay rather than creating a new debit nonce.
 - HTTP endpoints `/api/custody` and `/api/custody/command` are off by default; explicit `WIKIDECK_CUSTODY_MODE=sandbox` enables testnet only. All other enabled modes and mainnet sandbox are rejected before node resources open. Requests use the node's origin/admission/body limits, per-route mutation throttling and sanitized errors. No observer/admin settlement or balance-credit endpoint exists.
 - [Stopped-node recovery](../server/recovery.ts) includes the optional custody sandbox database and its identity while continuing to accept existing two-database backups. Financial receipts are not pruned. Restore/startup checks do not substitute for production reconciliation.
-- Offline fixture methods exercise fee allocation and prepared-withdrawal identity/retry/cancellation boundaries. They never access Monero and are not callable through HTTP. Public reservations stop at `awaiting-lock`; there is **no automatic NFT delivery or settled sale** in this increment. No real-address withdrawal, deposit subaddress, wallet RPC, signer, author payout, reorg compensation or PostgreSQL adapter is implemented yet.
+- Offline SQLite fixture methods exercise fee allocation and prepared-withdrawal identity/retry/cancellation boundaries. They never access Monero and are not callable through HTTP. Public reservations stop at `awaiting-lock`; there is **no automatic NFT delivery or settled sale** in this increment. No real-address withdrawal, deposit subaddress, wallet RPC, signer, author payout or reorg compensation is implemented yet.
+
+**Financial storage update:** [ADR 003](postgres-financial-storage.md) selects PostgreSQL and records the integrated asynchronous custody backend, restricted runtime/schema roles, deferred accounting constraints, explicit SQLite freeze/import, and full dump/restore rehearsal. SQLite remains an opt-in-compatible local sandbox default, not an automatic fallback. Real custody and production operating acceptance remain blocked.
 
 Validation: TypeScript/Vite build and 144 offline tests passed in the serial native-RandomX test run; the initial parallel run encountered a peer-gossip timeout. Browser smoke verified a disposable wallet's signed test credit, withdrawal hold and cancellation restoring the exact balance, plus a 390px viewport without horizontal overflow. No real funds were used. Tests and remaining acceptance gates are below; this is not launch readiness.
 
@@ -40,7 +42,7 @@ Validation: TypeScript/Vite build and 144 offline tests passed in the serial nat
 | Treasury worker / isolated signer | Serializes spend construction, validates approved withdrawal intents, destination, amount, fee cap and policy; persists prepared transactions before relay. Independent limits and approvals, not blind signing for API requests. |
 | Reconciler and operations | Independently compares ledger, wallet and chain observations; alerts/halts affected operations; administers reviewed compensations, not arbitrary balance edits. |
 
-Recommend PostgreSQL for the eventual production financial journal: exact integer `NUMERIC(...,0)` amounts, row locking/serializable transactions, unique constraints and managed point-in-time backups. This is a **proposed database decision**, not a dependency change. SQLite remains suitable for disposable models; the existing market SQLite and disconnected settlement lab are not a production custody ledger. Keep financial order transitions and journal entries in one database transaction; bridge external chain/wallet effects with a durable outbox, never a cross-database pretend transaction. One active writer/treasury worker initially; failover must fence the old worker before spending resumes.
+PostgreSQL is now selected and integrated for the custody sandbox under [ADR 003](postgres-financial-storage.md): exact `NUMERIC` with explicit integer checks (avoiding silent scale rounding), transactional row/advisory locking, unique constraints and separately reviewed backup/PITR operations. SQLite remains suitable for disposable compatibility models; no automatic promotion/fallback occurs. Financial order transitions and journal entries share one PostgreSQL transaction; external chain/wallet effects require a durable outbox, never a cross-database pretend transaction. One financial writer at a time initially; readers use coherent snapshots. Future treasury failover must fence the old worker before spending resumes.
 
 Minimal entities: custodians/network bindings, authenticated customer identities/recovery enrollment, deposit address mappings, observed outputs and inclusion history, ledger accounts/journals/entries, immutable sale terms, purchase reservations, chain events/cursors, withdrawal intents/attempts/prepared transactions, outbox/inbox, signed receipts and audited cases/approvals. Mutable projections are rebuildable; history and external-effect identity are not discarded.
 

docs/postgres-financial-storage.md

@@ -0,0 +1,84 @@
+# ADR 003 — PostgreSQL financial storage
+
+Status: **selected and integrated for the custody sandbox; real-money release not authorized**. Evidence date: 2026-10-20. Related: [custody design](custodial-market-design.md), [production plan](production-plan.md).
+
+## Decision and scope
+
+Use **PostgreSQL** for financial state. Exact arithmetic, transactional constraints, concurrent coherent reads, cross-process locks, established backup/restore tooling and future WAL/PITR operations make it preferable to extending an embedded financial database indefinitely. PostgreSQL 17.11 was exercised in the disposable fixture; pin and review the intended deployed release/image separately.
+
+This is an integrated backend, not a second disconnected balance demo: [custody storage selection](../server/market/custody-storage.ts) supplies [the existing HTTP custody routes](../server/testnet/http.ts) with [PostgreSQL custody](../server/market/postgres-custody.ts). Signed command domains, user identities, amounts and UI stay unchanged. SQLite remains the default disposable/local sandbox for compatibility; choose PostgreSQL explicitly. Chain DAG/canonical indexes and legacy mock marketplace remain in their original SQLite databases. Moving those independent concerns is not needed to use PostgreSQL for customer liabilities.
+
+**All balances still represent fictitious sandbox XMR.** No deposit observer/subaddress allocation, Monero signer, real-address withdrawal, NFT sale witness or production payout is introduced. The PostgreSQL backend supports public listing/reservation/cancellation/withdrawal-hold commands; old prepared fixture withdrawals are preserved by import but cannot be cancelled or paid through HTTP. Old SQLite fixture settlement methods are not production PostgreSQL payment workers.
+
+## Storage and transaction guarantees
+
+- Dedicated `wk_custody` schema with version/digest-checked **explicit migration**, not DDL at HTTP startup. A runtime connection with administrative privileges or schema-owner membership is rejected.
+- Money uses unconstrained-precision PostgreSQL `NUMERIC` plus checks for integral, nonnegative, bounded atomic values. Using `NUMERIC(30,0)` alone could silently round a fractional input before checking; this implementation rejects fractions instead. Account balances and individual entry legs remain below $10^{30}$ atomic units. The driver returns monetary values as strings; no global floating-point parser override.
+- Journals/entries and replay receipts are immutable to the application. Deferred constraint triggers reject empty/unbalanced journals at commit, including entry inserts. Committed journals reject appended entries. An isolated `SECURITY DEFINER` entry trigger with a fixed schema search path locks accounts and updates balance projections; the runtime cannot directly update balances.
+- Reservations, listing/order state, withdrawal holds, journal entries, blocked outbox and receipts commit in the **same database transaction**. Exactly the same signed nonce returns its original receipt; changed terms fail. No receipt pruning or reinterpretation of mock balances as live deposits.
+- One writer per financial domain initially: transaction-scoped advisory lock shared across pools/processes, plus row locks and constraints. Reads use a repeatable-read snapshot. This improves separation/concurrency of readers and durable server operations, **not a claim of unlimited concurrent writers**. Sharding/partitioning or finer write locking requires measurements and another reviewed change.
+- Pool maximum four connections per service; per-instance active admission sixteen; finite connect/statement/lock/idle transaction timeouts; synchronous commit explicitly on. Scale the total pool budget across instances deliberately. No wallet/network side effect is performed in a database transaction.
+- No automatic retry after an uncertain commit. A client refreshes or resends the original signed envelope, letting durable receipts establish whether it committed. PostgreSQL failure never causes SQLite fallback. Startup audits compare every account projection with exact journal sums and check backing totals; these are internal accounting checks, not proof that real Monero assets exist.
+
+These controls do not make a compromised application harmless: it still has posting privileges, and an administrative database owner can alter schema/history. Restricted roles prevent accidental/direct mutation, not fraudulent balanced postings by a fully compromised service. Independent monitoring, statement review, external backups and authorization/treasury isolation remain required.
+
+## Configuration and operator workflow
+
+The following are configuration values, not credentials:
+
+| Setting | Behavior |
+| --- | --- |
+| `WIKIDECK_CUSTODY_MODE=sandbox` | Enables the existing testnet-only sandbox. Live/mainnet custody stays disabled. |
+| `WIKIDECK_CUSTODY_STORAGE=postgres` | Selects PostgreSQL; default `sqlite` is retained for existing sandbox users. |
+| `WIKIDECK_CUSTODY_PG_URL_FILE` | Absolute path to a private secret file containing a PostgreSQL URL. Never a Vite/browser variable. |
+| `WIKIDECK_CUSTODY_PG_CA_FILE` | Optional private file with trusted PEM CA certificate(s); otherwise use system trust. TLS hostname/certificate verification remains enabled. |
+| `WIKIDECK_CUSTODY_PG_LOCAL_TEST=1` | **Disposable loopback tests only:** disables TLS solely for literal loopback/localhost. Never use for production. |
+
+URL query parameters/fragments are rejected, preventing `sslmode` options from overriding transport policy. Missing/malformed configuration fails with a generic error, not a URL/password dump. Secret files must be regular non-symlink bounded files; mount them read-only with restricted filesystem ACLs. Production network firewall, verified TLS, password/secret rotation and external database logs need an operator review. The automated fixture tests configuration policy, not an actual production certificate renewal/handshake drill.
+
+The operator CLI is [scripts/custody-db.ts](../scripts/custody-db.ts), available through `npm run custody:db -- ACTION`. Credentials are provided only by the secret-file environment setting, never positional arguments or browser UI:
+
+1. Provision a dedicated database, a migration/schema-owner identity, and a separate existing login role without superuser/create-role/create-db/replication/bypass-RLS privileges. Create passwords privately outside this tool; no credentials are generated or displayed by the CLI.
+2. With migration credentials, run action `migrate`. It checks the pinned version/digest or creates a fresh schema transactionally. Unknown versions are refused; no reset/drop/rebuild mode exists.
+3. With migration credentials, run action `grant-runtime ROLE`. It validates the literal role identifier and grants schema usage, reads, account/journal column-specific inserts, entry/receipt inserts and operational record updates. It never grants direct balance updates, history deletion, schema ownership or migrations. Use a **new dedicated role**; the helper grants required privileges but is not a scrubber for inherited privileges from an existing unrelated role.
+4. If the data directory has existing SQLite custody, stop the node, preserve a stopped-node backup, and run `import-sqlite DATA_DIR` against a migrated **empty** target using migration credentials. See cutover behavior below.
+5. Switch the server to runtime credentials and explicit PostgreSQL selection. Startup verifies schema, restricted identity, custodian binding and accounting before serving. A new directory can initialize a fresh sandbox identity; it does not inherit another host's credit.
+6. Action `audit`, using runtime credentials, prints exact aggregate sandbox totals only. It does not print customer histories or database credentials. Running it on a previously uninitialized schema initializes a sandbox custodian identity just like startup, so do not use it as a supposedly read-only probe of an empty migration target.
+
+The CLI is included in the runtime image. No existing deployment, database secret or fund wallet was changed automatically by this implementation.
+
+## Explicit SQLite cutover and rollback limits
+
+[Importer](../server/market/custody-import.ts) is bounded to the existing sandbox source size (128 MiB), not a general live-money migration utility. The CLI takes the same exclusive stopped-node directory lock as node startup. Calling the underlying library assumes the caller has already stopped all source writers.
+
+1. Acquire SQLite immediate transaction and take a consistent snapshot. Verify source identity/integrity and canonical amount encoding. Hash identity and all logical rows in deterministic insertion order.
+2. **Freeze source first:** commit its `postgres_cutover` digest before contacting the target. SQLite startup refuses this marker. This closes the crash window where PostgreSQL might commit while SQLite remains writable.
+3. Lock PostgreSQL financial domain. Require an empty migrated target. Import accounts initially at zero and replay original journal entries through the database posting trigger; compare rebuilt balances to source projections. Preserve custodian identity, all receipts/digests, fictitious-credit claims, listing/order/withdrawal order, prepared attempt/transaction identities and blocked outbox state.
+4. Validate accounting, then commit the imported source digest. Repeating the **same** frozen-source import returns success without duplicating balances. A changed source/different imported target is rejected. Failure before commit rolls target rows back; the source stays frozen. Correct the privately diagnosed issue and resume the same import—do not delete markers to restart SQLite.
+5. PostgreSQL startup records a private fsynced local binding of custodian ID. A directory with that binding refuses SQLite even if no source database exists. An imported source must match the target's stored source digest.
+
+There is no live dual-write, automatic migration or automatic rollback. An old pre-cutover backup cannot know about a later cutover: restoring it and running SQLite would fork accounting operationally. Fence old hosts/credentials and archive old backups as **historical, non-spending**. This cannot be solved by a local marker alone; coordinated restore and authority controls are release gates.
+
+## Backup, restore and failure handling
+
+Existing node backup packages cover local SQLite databases only. They now **refuse** a PostgreSQL-bound data directory or frozen cutover source instead of claiming to capture all customer liabilities. No automatic cross-database backup coordinator is implemented.
+
+Recommended operations before any production release:
+
+- Managed PostgreSQL storage with encrypted backups, WAL archival/PITR, monitored replica/backup lag, checked restore retention and selected RPO/RTO. Define synchronous-replica requirements based on acceptable loss/failover risk; `synchronous_commit=on` alone does not configure a synchronous replica.
+- Fence writers/treasury services during backup/restore cutover and record matching node ledger/market checkpoints, custodian identity, database dump/backup identity and external withdrawal state. Do not restore a financial snapshot in isolation and start spending.
+- Use full-database dumps that include schema/functions/triggers/ACLs, plus separately protected role configuration; a schema-filtered dump may omit necessary trigger dependencies. Restore into an isolated database first, using administrative credentials; connect only the restricted runtime for audit/replay checks after restore.
+- Reconcile Monero external history before resuming real spending once that integration exists. No local backup can reverse an external payment. Database timeouts/connection loss leave requests failed closed; retry the original signed command, not a new withdrawal.
+- Monitor application and database errors without logging statements/parameters containing private customer data in production. Fixture logs deliberately show failed constraint probes on fictitious data; do not adopt their logging/auth configuration.
+
+The integration suite performs a real full `pg_dump`/isolated `pg_restore` rehearsal preserving ID, exact liabilities and receipt replay. It does **not** certify managed backup schedules, PITR, encrypted off-host custody, power-loss/fsync behavior or production failover.
+
+## Evidence and remaining gates
+
+[Disposable test compose](../compose.postgres-test.yaml) runs PostgreSQL on tmpfs in an isolated internal network, no published ports; test runner shares the database network namespace and uses loopback. Trust authentication is **fixture-only**, not a deployment template. [Tests](../tests/postgres/custody.test.ts) are a separate explicit suite and refuse to run without `WIKIDECK_PG_DISPOSABLE_TEST=1`.
+
+Verified: restricted runtime authorization; deferred balance constraints; fractional/negative/oversized rejection; committed journal append rejection; independent-pool simultaneous nonce/listing/withdrawal races; insufficient-balance rollback; injected receipt failure atomicity; exact amounts beyond JavaScript safe integer; reconnect/replay; import of prepared withdrawals and pending orders; failed-import freeze/rollback/resume; real HTTP backend integration; configuration policy; fallback/backup refusal; full dump/restore audit.
+
+Final evidence: TypeScript/Vite test-image build passed; **149 existing regression tests and 10 isolated PostgreSQL integration tests passed**, zero failures/skips. PostgreSQL fixture resources were removed after testing. Existing user data directories and deployed node database settings were not switched. This verifies the current sandbox implementation, not production acceptance.
+
+**Not closed:** live Monero and NFT settlement; production dataset/SLOs and pool contention benchmarks; audit scheduling and very-large-history startup cost; normalized durable real-deposit/withdrawal/compensation states; account recovery/second factor; external security/accounting review; coordinated node/financial/treasury backup; deployment TLS drills, PITR and failover rehearsal; funded exposure/fee/loss policies; legal/operations sign-off. The scalable database choice does not turn the existing sandbox into a production custodian.

docs/production-plan.md

@@ -65,6 +65,8 @@ Can run alongside P0/P1. Addresses SEC-01/SEC-06 rather than adding more fixed c
 
 **Implemented M2 increment, 2026-10-20:** [canonical indexes and durable market observation](canonical-indexes.md). Canonical ownership/transaction SQL indexes now serve NFT, deck, custody availability and mock-market queries; per-block undo and anchored connect/disconnect events commit with ledger updates. Market checkpoint/reorg holds persist atomically and bounded catch-up blocks new mutations. Final TypeScript/Vite build and 149 offline tests passed. A disposable 10,000-NFT SQL baseline is recorded, **not** a target-hardware SLO or complete M2 acceptance. Production financial storage, incremental order scheduling, retained-DAG capacity and operator dataset/latency targets remain open.
 
+**Next financial-storage increment:** [ADR 003 — PostgreSQL](postgres-financial-storage.md) selects and integrates the custody SQL backend with existing HTTP callers, exact constrained accounting, explicit migration/runtime roles, source-freezing SQLite import and dump/restore rehearsal. The adapter remains on the fictitious sandbox domain; it is not a production deposit/withdrawal worker. SQLite node-only backup deliberately refuses PostgreSQL-bound financial state. Production hardware/SLO tests, coordinated backup/PITR/failover, real payment schema/worker controls and independent review remain open, so this does not close M2/M4 or authorize funds.
+
 - Indexed canonical ownership, NFT/transfer lookup and paged wallet/explorer APIs: browser signing must not require loading all history. Tip-pinned queries detect reorgs and preserve complete ownership semantics.
 - Incremental market reconciliation from durable canonical/reorg events. Maintain rollback-safe indexes and a recoverable cursor; catch-up after restart must not miss delivery or payout holds. Bound worker time/concurrency; move expensive work off the request event loop where appropriate.
 - A versioned production market/settlement schema with append-only financial events and balanced accounting entries for buyer liabilities, host revenue, author liabilities, seller liabilities, paid/refunded amounts and actual network fees. Derived views are rebuildable; financial history and idempotency keys are not discarded to reclaim space.