Persistent PostgreSQL sandbox deployment

This commit is contained in:
2026-10-20 16:02:18 -04:00
parent 84a505c4a5
commit 4cac26d84b
14 changed files with 333 additions and 1 deletions
+54
View File
@@ -0,0 +1,54 @@
# Persistent local PostgreSQL custody sandbox
Status: implemented deployment rehearsal, **not a real-money or public production deployment**. Date: 2026-10-20. Related: [financial storage](postgres-financial-storage.md), [production plan](production-plan.md).
## Start and inspect
Use [compose.custody.yaml](../compose.custody.yaml) **by itself**, with a stable Compose project name. Existing single-node/three-node Compose files remain unchanged and do not start PostgreSQL. Do not combine this file with [the disposable test fixture](../compose.postgres-test.yaml).
The VS Code **Start persistent custody sandbox** task runs the fresh/persistent stack using project `wikideck-custody-local`. Equivalent CLI invocation is `docker compose -p wikideck-custody-local -f compose.custody.yaml up -d --build`. Browser: [http://127.0.0.1:3422/](http://127.0.0.1:3422/), then **Marketplace → Custody preview**. Override `WIKIDECK_CUSTODY_PORT` if needed; origin configuration follows that port. The default bind is loopback only.
Services:
| Service | Lifecycle / purpose |
| --- | --- |
| `provision` | One-shot, network disabled. Generates random passwords and local CA/server certificate into scoped named volumes. Reuses complete credentials on later starts; partial/missing sets fail instead of silently rotating them. |
| `database` | Persistent PostgreSQL 17 with no published database port, internal network, TLS-only TCP HBA, SCRAM passwords, private key mounted only here. Health-gated startup. |
| `migrate` | Explicit one-shot local bootstrap: separate owner/runtime roles, versioned schema, restricted grants. Holds admin secrets unavailable to Wikideck. Unknown schema/credential state fails closed. |
| `wikideck` | Existing non-root application runtime with only the runtime credential and public CA mounted read-only, its own persistent node volume, mock payments and PostgreSQL-backed custody sandbox. |
Completed one-shot services are expected to show **Exited (0)**. PostgreSQL and Wikideck should remain running; a stopped migrator with exit zero is not a missing database.
## Persistent volumes and isolation
Named volumes separately retain PostgreSQL data, node data, admin credentials, schema-owner credentials, runtime credentials and database TLS material. Generated secrets are not written into source, command arguments, Compose environment or browser variables. The app receives no admin/owner credential or server private key. Generated files are scoped by volume mounts; the admin password file is readable inside the trusted database/provisioning containers for the PostgreSQL entrypoint, not mounted in the app.
This is a **new independent market**, not a migration of any existing node or wallet balance. Do not attach an existing SQLite custody volume and expect automatic import. Follow the stopped-node, backup and source-freezing procedure in [ADR 003](postgres-financial-storage.md) for that case. Do not point multiple independent market identities at this one custody schema.
Keep the same project name. A different project name creates different volumes and therefore a new database/custodian. Existing preview containers/ports and user data were not changed by the rehearsal.
## TLS and credentials
The database presents a certificate with SAN `DNS:database`; the driver validates the chain and hostname using the mounted CA. No insecure transport override is set. Server-side HBA rejects plaintext TCP even with correct credentials. Unix-socket local trust remains only for the official local PostgreSQL bootstrap/health path; it is not a published network endpoint. PostgreSQL logs are configured not to print SQL error statements/parameters that could contain bootstrap credentials.
Certificates expire after **30 days**. This is deliberate local-rehearsal material, not a certificate lifecycle solution for production. Provisioning refuses certificates with less than one day remaining and does not regenerate passwords or certificates over an existing database. The CA private key is temporary during first provisioning and not retained; renewal therefore requires a deliberately reviewed replacement CA/certificate and coordinated trust update. Do not delete random secret volumes to “fix” renewal. A running database may keep a certificate loaded beyond expiry; connection verification still fails closed for new clients. External monitoring/expiry alerts and a tested rotation procedure are required before longer operation.
Passwords are not automatically rotated on restart. Missing credentials paired with existing PostgreSQL storage will not reset database roles. Preserve matching secret and data volumes securely; losing them requires controlled recovery. Docker administrators can read volumes; this is not protection against host compromise. Pin intended images/digests, configure encrypted backups and host firewall, and review actual deployment infrastructure before public/valuable use.
## Check and stop
The **Verify persistent custody after recreation** task runs [the deployment check](../scripts/custody-deployment-check.ts) in the app container. It verifies TLS, plaintext rejection, runtime role, sandbox API flags and exact journal accounting; prints only the public custodian ID and aggregate fictitious amounts. It does not reveal credentials or customer records.
The optional **Check persistent custody TLS and credit** task adds a single 1-XMR **fictitious** test credit under an ephemeral test identity before auditing. It does not create Monero outputs, touch existing wallets or transfer NFTs. Repeating that task creates another fictitious customer credit; use the read-only verification task for routine checks. Test identities are not recoverable users and should never be used with valuable funds.
The **Stop persistent custody without deleting data** task uses Compose `down` without volume removal. Restarting with the same project/file recreates containers and reuses database, credentials, TLS and node state. **Never add `--volumes`/`-v` to normal shutdown:** it destroys custody history and credentials for this stack. Unclean node shutdown may leave the existing exclusive node lock; verify the old process is gone and follow the stale-lock review procedure—do not clear it automatically.
Existing SQLite-only node backup intentionally refuses PostgreSQL-bound nodes. This stack does not yet supply a coordinated backup job: full PostgreSQL backups, protected role/secret material and matching node state must be covered together. Named-volume persistence is **not a backup**, and Docker Desktop resets, disk loss or volume removal can still lose data. WAL/PITR, off-host encrypted copies, restore/failover exercises and RPO/RTO remain production gates.
## Evidence and limits
Rehearsal: provisioned a fresh isolated stack, observed healthy PostgreSQL and successful migrations, checked an actual **TLSv1.3** session under `custody_runtime`, and submitted a signed fictitious credit. Removed all four containers and both networks using `down`, retained volumes, recreated with `up`, and verified the **same custodian ID, one journal and 1,000,000,000,000 atomic units** in both assets and claims. Re-running setup retained existing credentials, role grants and schema; no second credit was added. Actual plaintext TCP authentication was rejected by the database. The browser served the app at port 3422.
TypeScript/Vite runtime/test builds pass. Final offline regression run: **164 tests passed, zero failures/skips**. Deployment regression tests verify sandbox configuration, no published database port, runtime secret isolation, persistent data, TLS/SCRAM HBA and fail-closed partial provisioning. This is not independent certification of secrets provisioning or production hardening.
Still absent: real Monero observer/signer/deposits/withdrawals, NFT candidate activation/settlement, managed certificates/rotation, automatic coordinated backups, production capacity/availability guarantees and release approval. The existing production plan remains authoritative.
+2
View File
@@ -47,6 +47,8 @@ The operator CLI is [scripts/custody-db.ts](../scripts/custody-db.ts), available
The CLI is included in the runtime image. No existing deployment, database secret or fund wallet was changed automatically by this implementation.
**Persistent local deployment now available:** [deployment guide](persistent-custody-sandbox.md) and [standalone Compose stack](../compose.custody.yaml) provision isolated named volumes, scoped generated credentials, verified PostgreSQL TLS/SCRAM, separate roles and a one-shot migration service. Recreated containers retained a signed fictitious credit and custodian identity in rehearsal. This explicitly started a new sandbox, not a cutover of existing node data or a live-custody deployment; production backup, certificate rotation and activation gates remain open.
## Explicit SQLite cutover and rollback limits
[Importer](../server/market/custody-import.ts) is bounded to the existing sandbox source size (128 MiB), not a general live-money migration utility. The CLI takes the same exclusive stopped-node directory lock as node startup. Calling the underlying library assumes the caller has already stopped all source writers.
+2
View File
@@ -67,6 +67,8 @@ Can run alongside P0/P1. Addresses SEC-01/SEC-06 rather than adding more fixed c
**Next financial-storage increment:** [ADR 003 — PostgreSQL](postgres-financial-storage.md) selects and integrates the custody SQL backend with existing HTTP callers, exact constrained accounting, explicit migration/runtime roles, source-freezing SQLite import and dump/restore rehearsal. The adapter remains on the fictitious sandbox domain; it is not a production deposit/withdrawal worker. SQLite node-only backup deliberately refuses PostgreSQL-bound financial state. Production hardware/SLO tests, coordinated backup/PITR/failover, real payment schema/worker controls and independent review remain open, so this does not close M2/M4 or authorize funds.
**Deployment increment:** [persistent local custody stack](persistent-custody-sandbox.md) now starts PostgreSQL and the actual app together, with private TLS/SCRAM networking, separate owner/runtime roles, explicit migrations and scoped persistent credential/data volumes. A signed fictitious credit and custodian identity survived container/network removal and recreation; the server rejected plaintext authentication. Existing deployments and SQLite custody were not switched. Production certificate rotation, coordinated backup/restore and external operating review remain open; all XMR remains simulated.
- Indexed canonical ownership, NFT/transfer lookup and paged wallet/explorer APIs: browser signing must not require loading all history. Tip-pinned queries detect reorgs and preserve complete ownership semantics.
- Incremental market reconciliation from durable canonical/reorg events. Maintain rollback-safe indexes and a recoverable cursor; catch-up after restart must not miss delivery or payout holds. Bound worker time/concurrency; move expensive work off the request event loop where appropriate.
- A versioned production market/settlement schema with append-only financial events and balanced accounting entries for buyer liabilities, host revenue, author liabilities, seller liabilities, paid/refunded amounts and actual network fees. Derived views are rebuildable; financial history and idempotency keys are not discarded to reclaim space.