Added pilot-bound Monero observation persistence and the one-shot monero:pilot-observe command

This commit is contained in:
2026-10-22 20:04:05 -04:00
parent 36963fed30
commit 176d1927a8
12 changed files with 640 additions and 12 deletions
+1
View File
@@ -48,6 +48,7 @@ COPY scripts/custody-deployment-init.ts ./scripts/custody-deployment-init.ts
COPY scripts/custody-deployment-check.ts ./scripts/custody-deployment-check.ts
COPY scripts/monero-daemon-check.ts ./scripts/monero-daemon-check.ts
COPY scripts/pilot-monero-deposit-address.ts ./scripts/pilot-monero-deposit-address.ts
COPY scripts/pilot-monero-observe-once.ts ./scripts/pilot-monero-observe-once.ts
COPY --from=randomx-build /native/build/wikideck-randomx ./native/wikideck-randomx
RUN mkdir -p /app/data && chown node:node /app/data
USER node
+6
View File
@@ -65,6 +65,12 @@ Keep daemon RPC private to the pilot/operator network, bound to an appropriate p
This probe does not establish that the node is honest or independently canonical, and `synchronized: true` is a point-in-time daemon report—not a finality guarantee or credit authorization. No deposit addresses are issued and no financial API is enabled by a successful result.
### Pilot-bound observation persistence (schema v14)
Pilot Wallet-RPC output scans are stored in separate pilot tables, not the existing mainnet-custody observation tables. The one-shot observer verifies the configured wallet fingerprint, resumes a bounded durable cursor with overlap, binds recognized outputs only to an exact registered reservation, and records unknown-wallet outputs as `unallocated` suspense observations. Replays are idempotent; missing, changed, and reappeared outputs produce append-only events for operator reorg review. This pipeline never posts a journal, creates a credit, or changes a balance.
The pilot observer role is separate from reservation/address roles and cannot read custody balances/journals or write credit records. Use `custody:db grant-pilot-monero-observer-runtime <ROLE>` with `WIKIDECK_PILOT_CUSTODY_OPERATIONS=pilot-observation-only-no-credit` after schema migration and identity initialization. Run one operator-controlled scan with `npm run monero:pilot-observe`; it requires the protected pilot observer PostgreSQL URL/CA, Mainnet Wallet-RPC config, matching mTLS gateway config, and wallet generation/fingerprint settings. `WIKIDECK_PILOT_OBSERVER_MAX_BLOCKS` (default 5000; max 100000) bounds each pass; `WIKIDECK_PILOT_OBSERVER_OVERLAP` (default 100; max 10000) requests a rescan window. Re-run manually to advance the cursor. No background service or HTTP endpoint is enabled.
## Initial checks
The network manifest reports `profile: pilot`, `network: wikideck-randomx-pilot-1`, and `launchStatus: private-pilot-mock-only`. Verify startup refuses a missing pilot-mode opt-in and any non-mock payment setting. Deployment tests check loopback binding, distinct profile selection, database secret isolation, and absence of an in-stack Monero daemon. A successful mock deployment is only evidence for this first slice, not for daemon connectivity or real-XMR readiness.
+3 -2
View File
@@ -11,7 +11,8 @@ Updated: 2026-10-22. Implementation plan for a private, operator-supervised func
- **Durable pilot allocation implemented:** a signed pilot request now reserves the next never-reused index transactionally, serializes concurrent allocators, replays idempotently, and rejects request/beneficiary conflicts. PostgreSQL enforces the reservation's custodian, market-chain, and Monero-network binding with a composite foreign key to the immutable pilot identity. A dedicated role can read schema digests and pilot identity, execute the isolation check, and read/insert reservations only; it cannot read balances, register addresses, or access observation, credit, or withdrawal tables. This code does not call Wallet-RPC, return a Monero address, or connect to HTTP.
- **Wallet-RPC address create/recover implemented:** the operator/library-only function verifies the configured wallet fingerprint, checks the reserved index and stable allocation label, persists a create intent before `create_address`, then reads back and validates the exact Mainnet subaddress before registering it once. Lost responses recover by the stable label without repeating creation; ambiguous missing responses remain in a durable `creating` state and stop for operator reconciliation. No HTTP enrollment, observer, credit, withdrawal, or sale path is enabled.
- **Operator address command implemented:** `monero:pilot-address` loads an owner-only signed request file, requires the pilot profile plus exact operator-action acknowledgement, checks the configured custodian and watch-only wallet generation/fingerprint/account, loads the protected Wallet-RPC and mTLS configs, verifies the wallet fingerprint, and invokes the durable create/recover library. It returns the address to that one-shot operator caller only; it is not wired into server startup, HTTP, or the browser.
- **Validation:** serial offline suite passed 189/189; disposable PostgreSQL suite passed 44/44, including schema v13 migration and the pilot reservation/address lifecycle tests. The focused address test verifies lost-response recovery and that an unresolved create is not sent twice. No operator database or real Monero node was accessed.
- **Pilot observation persistence implemented:** schema v14 stores pilot-only wallet cursors, scan records, assigned/unallocated outputs, and append-only `seen`/`changed`/`reappeared`/`not-seen` events. The separate observer role matches outputs only to a registered reservation with the same wallet generation, fingerprint, account/index, and address; unknown outputs remain suspense. A one-shot bounded scanner resumes with overlap through the new `monero:pilot-observe` command. This is observation-only: no journal, credit, HTTP route, or background service is enabled.
- **Validation:** focused schema and observer PostgreSQL tests passed; full disposable PostgreSQL suite passed 46/46 and full serial offline suite passed 191/191. Tests cover assigned versus suspense outputs, replay, disappearance/reappearance evidence, exact large atomic amounts, zero financial posting, and observer role denials. No operator database or live node was accessed.
- **Step 1 is not complete.** The current pilot stack does not connect to that external daemon and still has no Wallet-RPC gateway, observer, real-fund deposit or withdrawal path. It is useful for isolated UI/mock-flow work only and must not receive funds.
- **Validation history:** earlier pilot-only checks passed 10/10. An earlier full-suite attempt had a transient RandomX peer-gossip failure; the most recent serial offline suite subsequently exited 0.
- **Steps 2–5:** not started. Existing deposit/withdrawal APIs still require the separately gated mainnet profile; the new pilot record is identity metadata only. Sale-lock transactions remain candidate-only and are not accepted by active consensus.
@@ -123,4 +124,4 @@ Keep the functional essentials: signed authorization, correct network binding, p
## Next coding task
**Next: exercise `monero:pilot-address` against a protected Wallet-RPC mTLS gateway in a disposable operator-controlled wallet, then add pilot-bound observation for those registered allocations.** First test wallet type, network, external-daemon reachability, and recovery behavior; do not use a funded wallet. Keep HTTP enrollment and customer credit disabled. Do not accept deposits until observer/reorg handling, reviewed credit, actual withdrawal execution, and recovery are connected.
**Next: add an operator-only pilot credit-review preparation path that consumes these pilot-bound assigned observations, validates confirmation/unlock/double-spend and freshness, and produces a review subject without posting a journal.** Keep unallocated outputs in suspense; do not add automatic credit, customer HTTP enrollment, or withdrawal activation. Continue after this with the separate pending-credit posting/release roles and reorg hold resolution, still gated until cash-out and recovery work is ready.
+1
View File
@@ -17,6 +17,7 @@
"custody:backup": "node --experimental-strip-types scripts/custody-pair-backup.ts",
"monero:daemon-check": "node --experimental-strip-types scripts/monero-daemon-check.ts",
"monero:pilot-address": "node --experimental-strip-types scripts/pilot-monero-deposit-address.ts",
"monero:pilot-observe": "node --experimental-strip-types scripts/pilot-monero-observe-once.ts",
"node:backup": "node --experimental-strip-types scripts/node-backup.ts",
"ops:check": "node --experimental-strip-types scripts/ops-check.ts",
"miner": "tsx server/testnet/miner.ts",
+14 -1
View File
@@ -4,7 +4,8 @@ import { resolve, join } from 'node:path';
import { postgresConfig } from '../server/market/postgres-config.ts';
import { migrateCustody, grantCustodyRuntime, grantMoneroObserverRuntime,
grantMoneroDepositCreditRuntime, grantMoneroDepositAllocatorRuntime, grantMoneroWithdrawalRuntime,
grantPilotDepositReservationRuntime, grantPilotDepositAddressRuntime } from '../server/market/postgres-schema.ts';
grantPilotDepositReservationRuntime, grantPilotDepositAddressRuntime,
grantPilotMoneroObserverRuntime } from '../server/market/postgres-schema.ts';
import { initializeMainnetCustodyIdentity, parseMainnetCustodianId } from '../server/market/mainnet-custody-identity.ts';
import { importSqliteCustody } from '../server/market/custody-import.ts';
import { PostgresCustody } from '../server/market/postgres-custody.ts';
@@ -16,6 +17,7 @@ import { assertCustodyDbActionAllowedOnNetwork, assertMainnetSchemaMigrationAckn
assertMainnetCustodyOperationAcknowledged, assertPilotCustodyActionAllowed, assertPilotSchemaMigrationAcknowledged,
assertPilotCustodyIdentityAcknowledged, assertPilotReservationGrantAcknowledged,
assertPilotAddressGrantAcknowledged,
assertPilotObserverGrantAcknowledged,
parseCustodyDbArguments } from '../server/market/custody-db-args.ts';
import { prepareDepositCreditReview, postReviewedDepositPendingCredit, prepareDepositCreditReleaseReview,
releaseReviewedDepositCredit, reconcileDepositCreditReorgHolds, prepareDepositReorgResolution,
@@ -30,6 +32,7 @@ const custodyActions = ['migrate', 'migrate-mainnet-schema', 'initialize-mainnet
'initialize-pilot-custody-identity', 'grant-runtime', 'grant-monero-observer-runtime',
'grant-pilot-deposit-reservation-runtime',
'grant-pilot-deposit-address-runtime',
'grant-pilot-monero-observer-runtime',
'grant-monero-withdrawal-runtime',
'grant-monero-deposit-allocator-runtime', 'grant-monero-credit-runtime',
'import-sqlite', 'audit', 'resolve-integrity'];
@@ -43,6 +46,7 @@ assertPilotCustodyIdentityAcknowledged(action, process.env.WIKIDECK_PILOT_CUSTOD
process.env.WIKIDECK_PILOT_CUSTODY_OPERATIONS);
assertPilotReservationGrantAcknowledged(action, process.env.WIKIDECK_PILOT_CUSTODY_OPERATIONS);
assertPilotAddressGrantAcknowledged(action, process.env.WIKIDECK_PILOT_CUSTODY_OPERATIONS);
assertPilotObserverGrantAcknowledged(action, process.env.WIKIDECK_PILOT_CUSTODY_OPERATIONS);
if (action === 'initialize-mainnet-custody-identity' && process.env.WIKIDECK_MAINNET_CUSTODY_IDENTITY !== 'initialize-mainnet-custody-v1') {
throw new Error('Mainnet custody identity initialization requires WIKIDECK_MAINNET_CUSTODY_IDENTITY=initialize-mainnet-custody-v1');
}
@@ -97,6 +101,15 @@ try {
await grantPilotDepositAddressRuntime(pool, argument ?? '');
console.log('Pilot Wallet-RPC address allocator role granted for explicit operator action only; no HTTP enrollment or financial posting.');
}
if (action === 'grant-pilot-monero-observer-runtime') {
if (!IS_PILOT || process.env.WIKIDECK_PILOT_CUSTODY_OPERATIONS !== 'pilot-observation-only-no-credit' || extraArgument) {
throw new Error('Pilot observer grant requires pilot profile, observation-only acknowledgement, and one role name');
}
const identity = await pool.query('SELECT 1 FROM wk_custody.pilot_custody_identity WHERE singleton=1');
if (!identity.rowCount) throw new Error('Initialize pilot custody identity before granting observation access');
await grantPilotMoneroObserverRuntime(pool, argument ?? '');
console.log('Pilot Monero observation-only role granted; no address allocation, credit, withdrawal, or public enrollment privileges.');
}
if (action === 'grant-runtime') { await grantCustodyRuntime(pool, argument ?? ''); console.log('Runtime grants applied; no credentials printed.'); }
if (action === 'grant-monero-observer-runtime') {
await grantMoneroObserverRuntime(pool, argument ?? '');
+68
View File
@@ -0,0 +1,68 @@
import { randomUUID } from 'node:crypto';
import { Pool } from 'pg';
import { IS_PILOT } from '../shared/network.ts';
import { createMoneroWalletObserver, loadMoneroObserverConfig, moneroWalletFingerprint } from '../server/market/monero-observer.ts';
import { loadMoneroMtlsTransport, loadMoneroMtlsTransportConfig } from '../server/market/monero-mtls-transport.ts';
import { postgresConfig } from '../server/market/postgres-config.ts';
import { parsePilotCustodianId, verifyPilotCustodyIdentity } from '../server/market/pilot-custody-identity.ts';
import { readPilotMoneroObservationCursor, persistPilotMoneroObservationScan } from '../server/market/pilot-monero-observation-store.ts';
if (process.platform === 'win32') throw new Error('Pilot observer requires POSIX deployment with protected-file permission checks');
if (!IS_PILOT || process.env.WIKIDECK_PILOT_MODE !== 'private-mock-only') {
throw new Error('Pilot observer requires the private mock-only pilot profile');
}
if (process.env.WIKIDECK_PILOT_CUSTODY_OPERATIONS !== 'pilot-observation-only-no-credit') {
throw new Error('Set WIKIDECK_PILOT_CUSTODY_OPERATIONS=pilot-observation-only-no-credit for this explicit observer action');
}
const custodianId = parsePilotCustodianId(process.env.WIKIDECK_PILOT_CUSTODIAN_ID ?? '');
const walletGenerationId = process.env.WIKIDECK_MONERO_WALLET_GENERATION_ID ?? '';
const expectedFingerprint = process.env.WIKIDECK_MONERO_WALLET_FINGERPRINT ?? '';
if (!/^[a-f0-9-]{36}$/i.test(walletGenerationId) || !/^[a-f0-9]{64}$/.test(expectedFingerprint)) {
throw new Error('Invalid protected pilot watch-only wallet generation or fingerprint');
}
function positiveBounded(name: string, fallback: number, maximum: number): number {
const value = process.env[name];
if (value === undefined) return fallback;
if (!/^[1-9]\d{0,5}$/.test(value)) throw new Error(`Invalid ${name}`);
const parsed = Number(value);
if (!Number.isSafeInteger(parsed) || parsed > maximum) throw new Error(`Invalid ${name}`);
return parsed;
}
const maxBlocks = positiveBounded('WIKIDECK_PILOT_OBSERVER_MAX_BLOCKS', 5000, 100_000);
const overlap = positiveBounded('WIKIDECK_PILOT_OBSERVER_OVERLAP', 100, 10_000);
const observerConfig = await loadMoneroObserverConfig({ ...process.env, WIKIDECK_MONERO_OBSERVER: '1' });
const transportConfig = await loadMoneroMtlsTransportConfig();
if (observerConfig.network !== 'mainnet' || new URL(observerConfig.url).href !== new URL(transportConfig.url).href) {
throw new Error('Pilot observer requires matching Mainnet Wallet-RPC and mTLS gateway endpoints');
}
const scanner = createMoneroWalletObserver(observerConfig, await loadMoneroMtlsTransport(transportConfig));
const primaryAddress = await scanner.verifyMainnetWallet();
const walletFingerprint = await moneroWalletFingerprint(primaryAddress);
if (walletFingerprint !== expectedFingerprint) throw new Error('Configured pilot wallet fingerprint does not match protected Wallet-RPC');
const dbUrlFile = process.env.WIKIDECK_PILOT_OBSERVER_PG_URL_FILE;
const dbCaFile = process.env.WIKIDECK_PILOT_OBSERVER_PG_CA_FILE;
if (!dbUrlFile || !dbCaFile) throw new Error('Pilot observer PostgreSQL secret URL and CA files are required');
const pool = new Pool(await postgresConfig({ ...process.env,
WIKIDECK_CUSTODY_PG_URL_FILE: dbUrlFile,
WIKIDECK_CUSTODY_PG_CA_FILE: dbCaFile,
WIKIDECK_CUSTODY_PG_TRANSPORT: 'tls' }));
pool.on('error', () => console.error(JSON.stringify({ component: 'pilot-monero-observer', event: 'database-unavailable' })));
try {
const identity = await verifyPilotCustodyIdentity(pool, custodianId);
const scanHeight = await readPilotMoneroObservationCursor(pool, walletGenerationId, walletFingerprint);
const fromHeight = Math.max(0, scanHeight - overlap);
const page = await scanner.scanIncoming(fromHeight, maxBlocks);
const result = await persistPilotMoneroObservationScan(pool, { scanId: randomUUID(), walletGenerationId,
walletFingerprint, expectedScanHeight: scanHeight, scannedAt: Date.now(), page });
console.log(JSON.stringify({ component: 'pilot-monero-observer', network: identity.moneroNetwork,
marketNetwork: identity.marketNetwork, walletHeight: page.walletHeight, scanHeight: result.scanHeight,
caughtUp: result.scanHeight >= page.walletHeight, observed: result.observed, assigned: result.assigned,
unallocated: result.unallocated, changed: result.changed, reappeared: result.reappeared,
notSeen: result.notSeen, replayed: result.replayed, credits: 'disabled' }));
} catch {
console.error('Pilot observation failed or requires review; no balance credit was performed.');
process.exitCode = 1;
} finally { await pool.end(); }
+8 -1
View File
@@ -6,6 +6,7 @@ export const CUSTODY_DB_ACTION_ARGUMENT_COUNTS = {
'initialize-pilot-custody-identity': 1,
'grant-pilot-deposit-reservation-runtime': 1,
'grant-pilot-deposit-address-runtime': 1,
'grant-pilot-monero-observer-runtime': 1,
'grant-runtime': 1,
'grant-monero-observer-runtime': 1,
'grant-monero-deposit-allocator-runtime': 1,
@@ -56,7 +57,7 @@ export function assertMainnetCustodyOperationAcknowledged(action: CustodyDbActio
export function assertPilotCustodyActionAllowed(action: CustodyDbAction, isPilot: boolean): void {
const pilotActions = ['migrate-pilot-schema', 'initialize-pilot-custody-identity',
'grant-pilot-deposit-reservation-runtime', 'grant-pilot-deposit-address-runtime'];
'grant-pilot-deposit-reservation-runtime', 'grant-pilot-deposit-address-runtime', 'grant-pilot-monero-observer-runtime'];
if ((isPilot && !pilotActions.includes(action)) || (!isPilot && pilotActions.includes(action))) {
throw new Error(isPilot
? 'Pilot profile permits only pilot schema migration and identity-only initialization through this CLI'
@@ -76,6 +77,12 @@ export function assertPilotAddressGrantAcknowledged(action: CustodyDbAction, ack
}
}
export function assertPilotObserverGrantAcknowledged(action: CustodyDbAction, acknowledgement: string | undefined): void {
if (action === 'grant-pilot-monero-observer-runtime' && acknowledgement !== 'pilot-observation-only-no-credit') {
throw new Error('Pilot observer role grant requires explicit observation-only/no-credit acknowledgement');
}
}
export function assertPilotSchemaMigrationAcknowledged(action: CustodyDbAction, acknowledgement: string | undefined): void {
if (action === 'migrate-pilot-schema' && acknowledgement !== 'explicit-pilot-schema-migration-only') {
throw new Error('Pilot schema migration requires explicit pilot schema-migration acknowledgement');
+188
View File
@@ -0,0 +1,188 @@
import { createHash } from 'node:crypto';
import type { Pool, PoolClient } from 'pg';
import { z } from 'zod';
import { XMR_RPC_UINT64_MAX } from '../../shared/xmr-amount.ts';
import type { MoneroIncomingObservation, MoneroObservationPage } from './monero-observer.ts';
import { CUSTODY_WRITE_LOCK, verifyCustodySchema } from './postgres-schema.ts';
import { verifyPilotCustodyIdentity } from './pilot-custody-identity.ts';
import { validateXmrAddress } from './xmr.ts';
const MAX_SCAN_RANGE = 100_000;
const MAX_OVERLAP = 10_000;
const fingerprintSchema = z.string().regex(/^[a-f0-9]{64}$/);
const scanInputSchema = z.object({
scanId: z.string().uuid(),
walletGenerationId: z.string().uuid(),
walletFingerprint: fingerprintSchema,
expectedScanHeight: z.number().int().nonnegative().safe(),
scannedAt: z.number().int().nonnegative().safe(),
page: z.object({
network: z.literal('mainnet'), walletHeight: z.number().int().nonnegative().safe(),
fromHeight: z.number().int().nonnegative().safe(), throughHeight: z.number().int().nonnegative().safe(),
primaryAddress: z.string().length(95), incoming: z.array(z.unknown()).max(4096),
}).strict(),
}).strict();
const observationSchema = z.object({
observationId: z.string().min(1).max(160), transactionId: fingerprintSchema,
address: z.string().length(95), amountAtomicXmr: z.string().regex(/^[1-9]\d{0,19}$/),
height: z.number().int().positive().safe(), confirmations: z.number().int().positive().safe(),
timestamp: z.number().int().nonnegative().safe(), unlockTime: z.string().regex(/^(0|[1-9]\d{0,19})$/),
accountIndex: z.number().int().nonnegative().max(0xffff_ffff).safe(),
addressIndex: z.number().int().nonnegative().max(0xffff_ffff).safe(),
outputPublicKey: fingerprintSchema, globalOutputIndex: z.number().int().nonnegative().safe(),
locked: z.boolean(), doubleSpendSeen: z.boolean(),
}).strict();
type PersistInput = {
scanId: string;
walletGenerationId: string;
walletFingerprint: string;
expectedScanHeight: number;
scannedAt: number;
page: MoneroObservationPage;
};
export type PilotObservationClassification = 'assigned' | 'unallocated';
export type PilotObservationScanResult = {
scanId: string; scanHeight: number; observed: number; assigned: number; unallocated: number;
changed: number; reappeared: number; notSeen: number; replayed: boolean;
};
export async function readPilotMoneroObservationCursor(pool: Pool, walletGenerationId: string,
walletFingerprint: string): Promise<number> {
const generation = z.string().uuid().parse(walletGenerationId);
const fingerprint = fingerprintSchema.parse(walletFingerprint);
const identity = await verifyPilotCustodyIdentity(pool, (await pool.query(
'SELECT custodian_id FROM wk_custody.pilot_custody_identity WHERE singleton=1')).rows[0]?.custodian_id ?? '');
const result = await pool.query(`SELECT wallet_fingerprint,network,scan_height
FROM wk_custody.pilot_monero_observer_state WHERE wallet_generation_id=$1`, [generation]);
if (!result.rowCount) return 0;
if (result.rows[0].wallet_fingerprint !== fingerprint || result.rows[0].network !== identity.moneroNetwork) {
throw new Error('Pilot observer cursor is bound to a different wallet or Monero network');
}
return Number(result.rows[0].scan_height);
}
function digest(value: unknown): string {
return createHash('sha256').update(JSON.stringify(value)).digest('hex');
}
function observationDigest(observation: MoneroIncomingObservation, classification: PilotObservationClassification): string {
return digest([classification, observation.observationId, observation.transactionId, observation.amountAtomicXmr,
observation.height, observation.timestamp, observation.unlockTime, observation.accountIndex,
observation.addressIndex, observation.outputPublicKey, observation.globalOutputIndex,
observation.locked, observation.doubleSpendSeen]);
}
/** Persist pilot-wallet observations separately from mainnet custody observations. Never posts money or credits. */
export async function persistPilotMoneroObservationScan(pool: Pool, input: PersistInput): Promise<PilotObservationScanResult> {
const parsed = scanInputSchema.parse(input), page = parsed.page;
if (page.fromHeight > page.throughHeight || page.throughHeight > page.walletHeight ||
page.throughHeight - page.fromHeight > MAX_SCAN_RANGE || parsed.expectedScanHeight - page.fromHeight > MAX_OVERLAP ||
page.fromHeight > parsed.expectedScanHeight || page.throughHeight < parsed.expectedScanHeight) {
throw new Error('Pilot Monero observation range is not contiguous with its durable cursor');
}
await validateXmrAddress(page.primaryAddress);
const observations = page.incoming.map(row => observationSchema.parse(row));
const seen = new Set<string>();
for (const observation of observations) {
await validateXmrAddress(observation.address);
if (observation.observationId !== `${observation.transactionId}:${observation.outputPublicKey}` ||
BigInt(observation.amountAtomicXmr) > XMR_RPC_UINT64_MAX || observation.height < page.fromHeight ||
observation.height > page.throughHeight || observation.confirmations !== page.walletHeight - observation.height + 1 ||
seen.has(observation.observationId)) throw new Error('Pilot Monero observation is inconsistent with its scan page');
seen.add(observation.observationId);
}
const pageDigest = digest([parsed.walletGenerationId, parsed.walletFingerprint, page.fromHeight, page.throughHeight,
observations.map(row => [row.observationId, row]).sort((a, b) => String(a[0]).localeCompare(String(b[0])))]);
const client = await pool.connect();
try {
await client.query('BEGIN');
await client.query("SET LOCAL statement_timeout='10s'; SET LOCAL lock_timeout='3s'; SET LOCAL idle_in_transaction_session_timeout='10s'; SET LOCAL synchronous_commit=on");
await client.query('SELECT pg_advisory_xact_lock($1::bigint)', [CUSTODY_WRITE_LOCK]);
await verifyCustodySchema(client);
const identity = await verifyPilotCustodyIdentity(pool, (await client.query(
'SELECT custodian_id FROM wk_custody.pilot_custody_identity WHERE singleton=1')).rows[0]?.custodian_id ?? '');
const state = await client.query('SELECT wallet_fingerprint,network,scan_height FROM wk_custody.pilot_monero_observer_state WHERE wallet_generation_id=$1 FOR UPDATE',
[parsed.walletGenerationId]);
const prior = await client.query('SELECT payload_digest,through_height FROM wk_custody.pilot_monero_observation_scans WHERE scan_id=$1', [parsed.scanId]);
if (prior.rowCount) {
if (prior.rows[0].payload_digest !== pageDigest) throw new Error('Pilot observation scan ID was reused with different data');
await client.query('COMMIT');
return { scanId: parsed.scanId, scanHeight: Number(prior.rows[0].through_height), observed: observations.length,
assigned: 0, unallocated: 0, changed: 0, reappeared: 0, notSeen: 0, replayed: true };
}
if (!state.rowCount) {
if (parsed.expectedScanHeight !== 0) throw new Error('First pilot observation scan must start at wallet history height zero');
await client.query(`INSERT INTO wk_custody.pilot_monero_observer_state(wallet_generation_id,wallet_fingerprint,network,scan_height,updated_at)
VALUES($1,$2,'mainnet',0,$3)`, [parsed.walletGenerationId, parsed.walletFingerprint, parsed.scannedAt]);
} else if (state.rows[0].wallet_fingerprint !== parsed.walletFingerprint || state.rows[0].network !== 'mainnet' ||
Number(state.rows[0].scan_height) !== parsed.expectedScanHeight) {
throw new Error('Pilot observer wallet identity or cursor changed; reread durable state before retry');
}
const scanState = await client.query('SELECT wallet_fingerprint FROM wk_custody.pilot_monero_observer_state WHERE wallet_generation_id=$1',
[parsed.walletGenerationId]);
if (scanState.rows[0].wallet_fingerprint !== parsed.walletFingerprint) throw new Error('Pilot observer wallet fingerprint mismatch');
await client.query(`INSERT INTO wk_custody.pilot_monero_observation_scans
(scan_id,wallet_generation_id,from_height,through_height,payload_digest,completed_at) VALUES($1,$2,$3,$4,$5,$6)`,
[parsed.scanId, parsed.walletGenerationId, page.fromHeight, page.throughHeight, pageDigest, parsed.scannedAt]);
let assigned = 0, unallocated = 0, changed = 0, reappeared = 0;
for (const observation of observations) {
const allocation = await client.query<{ allocation_id: string }>(`SELECT allocation_id
FROM wk_custody.pilot_monero_deposit_reservations
WHERE wallet_generation_id=$1 AND wallet_fingerprint=$2 AND account_index=$3 AND address_index=$4
AND monero_network='mainnet' AND status='registered' AND monero_address=$5`,
[parsed.walletGenerationId, parsed.walletFingerprint, observation.accountIndex, observation.addressIndex, observation.address]);
const classification: PilotObservationClassification = allocation.rowCount ? 'assigned' : 'unallocated';
const allocationId: string | null = allocation.rowCount ? allocation.rows[0].allocation_id : null;
const payloadDigest = observationDigest(observation, classification);
const existing = await client.query(`SELECT payload_digest,currently_present,first_observed_at FROM wk_custody.pilot_monero_observations
WHERE wallet_generation_id=$1 AND observation_id=$2 FOR UPDATE`, [parsed.walletGenerationId, observation.observationId]);
const eventType = !existing.rowCount ? 'seen' : !existing.rows[0].currently_present ? 'reappeared'
: existing.rows[0].payload_digest !== payloadDigest ? 'changed' : null;
await client.query(`INSERT INTO wk_custody.pilot_monero_observations(
wallet_generation_id,observation_id,allocation_id,classification,txid,monero_address,amount_atomic_xmr,
block_height,wallet_height,confirmations,timestamp,unlock_time,account_index,address_index,
output_public_key,global_output_index,locked,double_spend_seen,payload_digest,currently_present,
last_seen_scan_id,first_observed_at,last_observed_at)
VALUES($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,true,$20,$21,$21)
ON CONFLICT(wallet_generation_id,observation_id) DO UPDATE SET
amount_atomic_xmr=excluded.amount_atomic_xmr,block_height=excluded.block_height,wallet_height=excluded.wallet_height,
confirmations=excluded.confirmations,timestamp=excluded.timestamp,unlock_time=excluded.unlock_time,
locked=excluded.locked,double_spend_seen=excluded.double_spend_seen,payload_digest=excluded.payload_digest,
currently_present=true,last_seen_scan_id=excluded.last_seen_scan_id,last_observed_at=excluded.last_observed_at`,
[parsed.walletGenerationId, observation.observationId, allocationId, classification, observation.transactionId,
observation.address, observation.amountAtomicXmr, observation.height, page.walletHeight, observation.confirmations,
observation.timestamp, observation.unlockTime, observation.accountIndex, observation.addressIndex,
observation.outputPublicKey, observation.globalOutputIndex, observation.locked, observation.doubleSpendSeen,
payloadDigest, parsed.scanId, existing.rowCount ? Number(existing.rows[0].first_observed_at) : parsed.scannedAt]);
if (classification === 'assigned') assigned++; else unallocated++;
if (eventType) {
await client.query(`INSERT INTO wk_custody.pilot_monero_observation_events
(wallet_generation_id,observation_id,scan_id,event_type,payload_digest,wallet_height,observed_at)
VALUES($1,$2,$3,$4,$5,$6,$7) ON CONFLICT DO NOTHING`,
[parsed.walletGenerationId, observation.observationId, parsed.scanId, eventType, payloadDigest, page.walletHeight, parsed.scannedAt]);
if (eventType === 'changed') changed++;
if (eventType === 'reappeared') reappeared++;
}
}
const missing = await client.query(`SELECT observation_id,payload_digest FROM wk_custody.pilot_monero_observations
WHERE wallet_generation_id=$1 AND currently_present=true AND block_height BETWEEN $2 AND $3 AND last_seen_scan_id<>$4 FOR UPDATE`,
[parsed.walletGenerationId, page.fromHeight, page.throughHeight, parsed.scanId]);
for (const row of missing.rows) {
await client.query(`UPDATE wk_custody.pilot_monero_observations SET currently_present=false,last_seen_scan_id=$3
WHERE wallet_generation_id=$1 AND observation_id=$2`, [parsed.walletGenerationId, row.observation_id, parsed.scanId]);
await client.query(`INSERT INTO wk_custody.pilot_monero_observation_events
(wallet_generation_id,observation_id,scan_id,event_type,payload_digest,wallet_height,observed_at)
VALUES($1,$2,$3,'not-seen',$4,$5,$6) ON CONFLICT DO NOTHING`,
[parsed.walletGenerationId, row.observation_id, parsed.scanId, row.payload_digest, page.walletHeight, parsed.scannedAt]);
}
await client.query(`UPDATE wk_custody.pilot_monero_observer_state SET scan_height=GREATEST(scan_height,$2),updated_at=$3
WHERE wallet_generation_id=$1`, [parsed.walletGenerationId, page.throughHeight, parsed.scannedAt]);
await client.query('COMMIT');
return { scanId: parsed.scanId, scanHeight: Math.max(parsed.expectedScanHeight, page.throughHeight), observed: observations.length,
assigned, unallocated, changed, reappeared, notSeen: missing.rowCount ?? 0, replayed: false };
} catch (error) { await client.query('ROLLBACK').catch(() => {}); throw error; }
finally { client.release(); }
}
+152 -2
View File
@@ -1,7 +1,7 @@
import { createHash } from 'node:crypto';
import type { Pool, PoolClient } from 'pg';
export const CUSTODY_PG_VERSION = 13;
export const CUSTODY_PG_VERSION = 14;
export const CUSTODY_WRITE_LOCK = '7140824361';
export const MONERO_OBSERVER_WORKER_LOCK = '7140824362';
export const CUSTODY_SCHEMA_V1 = `
@@ -535,6 +535,114 @@ REVOKE ALL ON FUNCTION wk_custody.pilot_monero_deposit_registration_guard() FROM
`;
export const CUSTODY_SCHEMA_V13_DIGEST = createHash('sha256')
.update(`${CUSTODY_SCHEMA_V1}\n-- custody migration v2 --\n${CUSTODY_MIGRATION_V2}\n-- custody migration v3 --\n${CUSTODY_MIGRATION_V3}\n-- custody migration v4 --\n${CUSTODY_MIGRATION_V4}\n-- custody migration v5 --\n${CUSTODY_MIGRATION_V5}\n-- custody migration v6 --\n${CUSTODY_MIGRATION_V6}\n-- custody migration v7 --\n${CUSTODY_MIGRATION_V7}\n-- custody migration v8 --\n${CUSTODY_MIGRATION_V8}\n-- custody migration v9 --\n${CUSTODY_MIGRATION_V9}\n-- custody migration v10 --\n${CUSTODY_MIGRATION_V10}\n-- custody migration v11 --\n${CUSTODY_MIGRATION_V11}\n-- custody migration v12 --\n${CUSTODY_MIGRATION_V12}\n-- custody migration v13 --\n${CUSTODY_MIGRATION_V13}`).digest('hex');
export const CUSTODY_MIGRATION_V14 = `
CREATE TABLE wk_custody.pilot_monero_observer_state(
wallet_generation_id uuid PRIMARY KEY,
wallet_fingerprint text NOT NULL CHECK(wallet_fingerprint ~ '^[a-f0-9]{64}$'),
network text NOT NULL CHECK(network='mainnet'),
scan_height bigint NOT NULL CHECK(scan_height>=0),
updated_at bigint NOT NULL CHECK(updated_at>=0)
);
CREATE TABLE wk_custody.pilot_monero_observation_scans(
scan_id uuid PRIMARY KEY,
wallet_generation_id uuid NOT NULL REFERENCES wk_custody.pilot_monero_observer_state(wallet_generation_id),
from_height bigint NOT NULL CHECK(from_height>=0),
through_height bigint NOT NULL CHECK(through_height>=from_height),
payload_digest text NOT NULL CHECK(payload_digest ~ '^[a-f0-9]{64}$'),
completed_at bigint NOT NULL CHECK(completed_at>=0)
);
CREATE INDEX pilot_monero_scans_wallet_height
ON wk_custody.pilot_monero_observation_scans(wallet_generation_id,through_height DESC);
CREATE TABLE wk_custody.pilot_monero_observations(
wallet_generation_id uuid NOT NULL REFERENCES wk_custody.pilot_monero_observer_state(wallet_generation_id),
observation_id text NOT NULL,
allocation_id uuid REFERENCES wk_custody.pilot_monero_deposit_reservations(allocation_id),
classification text NOT NULL CHECK(classification IN ('assigned','unallocated')),
txid text NOT NULL CHECK(txid ~ '^[a-f0-9]{64}$'),
monero_address text NOT NULL CHECK(length(monero_address)=95),
amount_atomic_xmr numeric(20,0) NOT NULL CHECK(amount_atomic_xmr>0 AND amount_atomic_xmr<=18446744073709551615),
block_height bigint NOT NULL CHECK(block_height>0),
wallet_height bigint NOT NULL CHECK(wallet_height>=block_height),
confirmations bigint NOT NULL CHECK(confirmations=wallet_height-block_height+1),
timestamp bigint NOT NULL CHECK(timestamp>=0),
unlock_time text NOT NULL CHECK(unlock_time ~ '^(0|[1-9][0-9]{0,19})$'),
account_index bigint NOT NULL CHECK(account_index BETWEEN 0 AND 4294967295),
address_index bigint NOT NULL CHECK(address_index BETWEEN 0 AND 4294967295),
output_public_key text NOT NULL CHECK(output_public_key ~ '^[a-f0-9]{64}$'),
global_output_index bigint NOT NULL CHECK(global_output_index>=0),
locked boolean NOT NULL,
double_spend_seen boolean NOT NULL,
payload_digest text NOT NULL CHECK(payload_digest ~ '^[a-f0-9]{64}$'),
currently_present boolean NOT NULL DEFAULT true,
last_seen_scan_id uuid NOT NULL REFERENCES wk_custody.pilot_monero_observation_scans(scan_id),
first_observed_at bigint NOT NULL CHECK(first_observed_at>=0),
last_observed_at bigint NOT NULL CHECK(last_observed_at>=first_observed_at),
PRIMARY KEY(wallet_generation_id,observation_id),
UNIQUE(wallet_generation_id,output_public_key),
UNIQUE(wallet_generation_id,txid,account_index,address_index,output_public_key),
CHECK((classification='assigned' AND allocation_id IS NOT NULL) OR
(classification='unallocated' AND allocation_id IS NULL)),
CHECK(observation_id=(txid || ':' || output_public_key))
);
CREATE FUNCTION wk_custody.pilot_monero_observation_guard() RETURNS trigger LANGUAGE plpgsql
SET search_path=wk_custody,pg_catalog AS $$
DECLARE primary_fingerprint text;
BEGIN
SELECT wallet_fingerprint INTO STRICT primary_fingerprint FROM wk_custody.pilot_monero_observer_state
WHERE wallet_generation_id=NEW.wallet_generation_id;
IF NEW.classification='assigned' THEN
IF NEW.allocation_id IS NULL OR NOT EXISTS(
SELECT 1 FROM wk_custody.pilot_monero_deposit_reservations r
WHERE r.allocation_id=NEW.allocation_id AND r.wallet_generation_id=NEW.wallet_generation_id AND
r.wallet_fingerprint=primary_fingerprint AND r.status='registered' AND r.monero_address=NEW.monero_address AND
r.account_index=NEW.account_index AND r.address_index=NEW.address_index AND r.monero_network='mainnet') THEN
RAISE EXCEPTION 'Pilot observation does not match a registered wallet reservation';
END IF;
ELSIF NEW.allocation_id IS NOT NULL THEN
RAISE EXCEPTION 'Unallocated pilot observation cannot name a reservation';
END IF;
IF TG_OP='UPDATE' AND (
NEW.wallet_generation_id IS DISTINCT FROM OLD.wallet_generation_id OR
NEW.observation_id IS DISTINCT FROM OLD.observation_id OR NEW.allocation_id IS DISTINCT FROM OLD.allocation_id OR
NEW.classification IS DISTINCT FROM OLD.classification OR NEW.txid IS DISTINCT FROM OLD.txid OR
NEW.monero_address IS DISTINCT FROM OLD.monero_address OR NEW.account_index IS DISTINCT FROM OLD.account_index OR
NEW.address_index IS DISTINCT FROM OLD.address_index OR NEW.output_public_key IS DISTINCT FROM OLD.output_public_key OR
NEW.global_output_index IS DISTINCT FROM OLD.global_output_index OR NEW.first_observed_at IS DISTINCT FROM OLD.first_observed_at) THEN
RAISE EXCEPTION 'Pilot observation identity is immutable';
END IF;
RETURN NEW;
END $$;
CREATE TRIGGER pilot_monero_observations_guard BEFORE INSERT OR UPDATE ON wk_custody.pilot_monero_observations
FOR EACH ROW EXECUTE FUNCTION wk_custody.pilot_monero_observation_guard();
CREATE INDEX pilot_monero_observations_height
ON wk_custody.pilot_monero_observations(wallet_generation_id,block_height,observation_id);
CREATE TABLE wk_custody.pilot_monero_observation_events(
sequence bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
wallet_generation_id uuid NOT NULL,
observation_id text NOT NULL,
scan_id uuid NOT NULL REFERENCES wk_custody.pilot_monero_observation_scans(scan_id),
event_type text NOT NULL CHECK(event_type IN ('seen','changed','reappeared','not-seen')),
payload_digest text NOT NULL CHECK(payload_digest ~ '^[a-f0-9]{64}$'),
wallet_height bigint NOT NULL CHECK(wallet_height>=0),
observed_at bigint NOT NULL CHECK(observed_at>=0),
UNIQUE(wallet_generation_id,observation_id,scan_id,event_type),
FOREIGN KEY(wallet_generation_id,observation_id)
REFERENCES wk_custody.pilot_monero_observations(wallet_generation_id,observation_id)
);
CREATE FUNCTION wk_custody.pilot_monero_observation_history_immutable() RETURNS trigger LANGUAGE plpgsql
SET search_path=wk_custody,pg_catalog AS $$
BEGIN RAISE EXCEPTION 'Pilot Monero observation history is append-only'; END $$;
CREATE TRIGGER pilot_monero_observation_scans_no_update BEFORE UPDATE OR DELETE ON wk_custody.pilot_monero_observation_scans
FOR EACH ROW EXECUTE FUNCTION wk_custody.pilot_monero_observation_history_immutable();
CREATE TRIGGER pilot_monero_observation_events_no_update BEFORE UPDATE OR DELETE ON wk_custody.pilot_monero_observation_events
FOR EACH ROW EXECUTE FUNCTION wk_custody.pilot_monero_observation_history_immutable();
REVOKE ALL ON FUNCTION wk_custody.pilot_monero_observation_guard() FROM PUBLIC;
REVOKE ALL ON wk_custody.pilot_monero_observer_state,wk_custody.pilot_monero_observation_scans,
wk_custody.pilot_monero_observations,wk_custody.pilot_monero_observation_events FROM PUBLIC;
REVOKE ALL ON FUNCTION wk_custody.pilot_monero_observation_history_immutable() FROM PUBLIC;
`;
export const CUSTODY_SCHEMA_V14_DIGEST = createHash('sha256')
.update(`${CUSTODY_SCHEMA_V1}\n-- custody migration v2 --\n${CUSTODY_MIGRATION_V2}\n-- custody migration v3 --\n${CUSTODY_MIGRATION_V3}\n-- custody migration v4 --\n${CUSTODY_MIGRATION_V4}\n-- custody migration v5 --\n${CUSTODY_MIGRATION_V5}\n-- custody migration v6 --\n${CUSTODY_MIGRATION_V6}\n-- custody migration v7 --\n${CUSTODY_MIGRATION_V7}\n-- custody migration v8 --\n${CUSTODY_MIGRATION_V8}\n-- custody migration v9 --\n${CUSTODY_MIGRATION_V9}\n-- custody migration v10 --\n${CUSTODY_MIGRATION_V10}\n-- custody migration v11 --\n${CUSTODY_MIGRATION_V11}\n-- custody migration v12 --\n${CUSTODY_MIGRATION_V12}\n-- custody migration v13 --\n${CUSTODY_MIGRATION_V13}\n-- custody migration v14 --\n${CUSTODY_MIGRATION_V14}`).digest('hex');
/** Retained alias for callers that need the initial schema definition in disposable fixtures. */
export const CUSTODY_SCHEMA = CUSTODY_SCHEMA_V1;
export const CUSTODY_SCHEMA_DIGEST = CUSTODY_SCHEMA_V10_DIGEST;
@@ -610,6 +718,11 @@ export async function migrateCustody(pool: Pool): Promise<void> {
if (current === 12) {
await c.query(CUSTODY_MIGRATION_V13);
await c.query('INSERT INTO wk_custody.schema_versions VALUES(13,$1)', [CUSTODY_SCHEMA_V13_DIGEST]);
current = 13;
}
if (current === 13) {
await c.query(CUSTODY_MIGRATION_V14);
await c.query('INSERT INTO wk_custody.schema_versions VALUES(14,$1)', [CUSTODY_SCHEMA_V14_DIGEST]);
}
if (current > CUSTODY_PG_VERSION) throw new Error('Custody PostgreSQL schema is newer than this application');
await verifyCustodySchema(c);
@@ -626,7 +739,7 @@ async function verifyCustodySchemaHistory(c: PoolClient): Promise<number> {
const expected = [CUSTODY_SCHEMA_V1_DIGEST, CUSTODY_SCHEMA_V2_DIGEST, CUSTODY_SCHEMA_V3_DIGEST,
CUSTODY_SCHEMA_V4_DIGEST, CUSTODY_SCHEMA_V5_DIGEST, CUSTODY_SCHEMA_V6_DIGEST, CUSTODY_SCHEMA_V7_DIGEST,
CUSTODY_SCHEMA_V8_DIGEST, CUSTODY_SCHEMA_V9_DIGEST, CUSTODY_SCHEMA_V10_DIGEST, CUSTODY_SCHEMA_V11_DIGEST,
CUSTODY_SCHEMA_V12_DIGEST, CUSTODY_SCHEMA_V13_DIGEST];
CUSTODY_SCHEMA_V12_DIGEST, CUSTODY_SCHEMA_V13_DIGEST, CUSTODY_SCHEMA_V14_DIGEST];
if (!rows.rowCount || rows.rowCount > expected.length || rows.rows.some((row, index) =>
Number(row.version) !== index + 1 || row.digest !== expected[index])) {
throw new Error('Custody PostgreSQL schema version/digest history mismatch; explicit migration required');
@@ -814,6 +927,43 @@ export async function grantPilotDepositAddressRuntime(pool: Pool, role: string,
finally { c.release(); }
}
/** Pilot observation-only role. It cannot allocate addresses, access mainnet custody, or post financial entries. */
export async function grantPilotMoneroObserverRuntime(pool: Pool, role: string,
addressRole = 'pilot_deposit_address', reservationRole = 'pilot_deposit_reservation',
custodyRuntimeRole = 'custody_runtime', creditRole = 'monero_deposit_credit'): Promise<void> {
if (!/^[a-z][a-z0-9_]{0,62}$/.test(role)) throw new Error('Invalid pilot Monero observer role identifier');
if ([addressRole, reservationRole, custodyRuntimeRole, creditRole].includes(role)) {
throw new Error('Pilot observer role must be separate from address, reservation, custody, and credit roles');
}
const name = `"${role}"`;
const c = await pool.connect();
try {
await c.query('BEGIN'); await verifyCustodySchema(c);
const r = await c.query('SELECT rolsuper,rolcreatedb,rolcreaterole,rolreplication,rolbypassrls FROM pg_roles WHERE rolname=$1', [role]);
if (!r.rowCount || Object.values(r.rows[0]).some(Boolean)) throw new Error('Pilot observer role must exist without administrative privileges');
const memberships = await c.query('SELECT 1 FROM pg_auth_members WHERE member=(SELECT oid FROM pg_roles WHERE rolname=$1) LIMIT 1', [role]);
if (memberships.rowCount) throw new Error('Pilot observer role must not inherit privileges through role membership');
const owner = await c.query("SELECT pg_get_userbyid(nspowner) AS owner FROM pg_namespace WHERE nspname='wk_custody'");
if (owner.rows[0].owner === role) throw new Error('Pilot observer role must not own the custody schema');
await c.query(`GRANT USAGE ON SCHEMA wk_custody TO ${name};
REVOKE ALL PRIVILEGES ON ALL TABLES IN SCHEMA wk_custody FROM ${name};
REVOKE ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA wk_custody FROM ${name};
GRANT SELECT ON wk_custody.schema_versions,wk_custody.pilot_custody_identity,
wk_custody.pilot_monero_deposit_reservations TO ${name};
GRANT SELECT,INSERT ON wk_custody.pilot_monero_observer_state TO ${name};
GRANT UPDATE(scan_height,updated_at) ON wk_custody.pilot_monero_observer_state TO ${name};
GRANT SELECT,INSERT ON wk_custody.pilot_monero_observation_scans TO ${name};
GRANT SELECT,INSERT ON wk_custody.pilot_monero_observation_events TO ${name};
GRANT USAGE,SELECT ON SEQUENCE wk_custody.pilot_monero_observation_events_sequence_seq TO ${name};
GRANT SELECT,INSERT ON wk_custody.pilot_monero_observations TO ${name};
GRANT UPDATE(amount_atomic_xmr,block_height,wallet_height,confirmations,timestamp,unlock_time,locked,
double_spend_seen,payload_digest,currently_present,last_seen_scan_id,last_observed_at)
ON wk_custody.pilot_monero_observations TO ${name};`);
await c.query('COMMIT');
} catch (error) { await c.query('ROLLBACK').catch(() => {}); throw error; }
finally { c.release(); }
}
/** Mainnet customer withdrawal-intent role. It can reserve/cancel liabilities but has no signing or relay authority. */
export async function grantMoneroWithdrawalRuntime(pool: Pool, role: string,
custodyRuntimeRole = 'custody_runtime', observerRole = 'monero_observer',
+4
View File
@@ -4,6 +4,7 @@ import { assertCustodyDbActionAllowedOnNetwork, assertMainnetSchemaMigrationAckn
assertMainnetCustodyOperationAcknowledged, assertPilotCustodyActionAllowed,
assertPilotSchemaMigrationAcknowledged, assertPilotCustodyIdentityAcknowledged,
assertPilotReservationGrantAcknowledged, assertPilotAddressGrantAcknowledged,
assertPilotObserverGrantAcknowledged,
parseCustodyDbArguments } from '../server/market/custody-db-args.ts';
test('custody database CLI parses two-argument deposit operations without shifting arguments', () => {
@@ -80,6 +81,9 @@ test('pilot custody CLI permits only additive schema migration and identity-only
assert.throws(() => assertPilotAddressGrantAcknowledged('grant-pilot-deposit-address-runtime', undefined), /acknowledgement/);
assert.throws(() => assertPilotAddressGrantAcknowledged('grant-pilot-deposit-address-runtime', 'reservation-only-no-addresses'), /acknowledgement/);
assert.doesNotThrow(() => assertPilotAddressGrantAcknowledged('grant-pilot-deposit-address-runtime', 'operator-triggered-addresses-only'));
assert.throws(() => assertPilotObserverGrantAcknowledged('grant-pilot-monero-observer-runtime', undefined), /no-credit acknowledgement/);
assert.throws(() => assertPilotObserverGrantAcknowledged('grant-pilot-monero-observer-runtime', 'reviewed-credit-only'), /no-credit acknowledgement/);
assert.doesNotThrow(() => assertPilotObserverGrantAcknowledged('grant-pilot-monero-observer-runtime', 'pilot-observation-only-no-credit'));
});
test('custody database CLI rejects unknown, missing, empty, and excess arguments', () => {
+26
View File
@@ -0,0 +1,26 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { execFileSync } from 'node:child_process';
function run(code: string) {
return JSON.parse(execFileSync(process.execPath, ['--experimental-strip-types', '--input-type=module', '-e', code], {
env: { ...process.env, WIKIDECK_NETWORK: 'pilot' }, encoding: 'utf8', timeout: 30_000,
}).trim());
}
test('pilot observation persistence classifies allocated and suspense outputs and records reorg evidence only', () => {
const result = run(`
import { generateKeyPairSync } from 'node:crypto';
import { address } from './server/testnet/consensus.ts';
import { persistPilotMoneroObservationScan } from './server/market/pilot-monero-observation-store.ts';
const pair=generateKeyPairSync('ec',{namedCurve:'prime256v1'});
const publicKey=pair.publicKey.export({type:'spki',format:'der'}).toString('base64');
const fingerprint='a'.repeat(64), generation='bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb';
const subaddress='4'.repeat(95), otherAddress='5'.repeat(95);
const pool={};
console.log(JSON.stringify({moduleLoaded:typeof persistPilotMoneroObservationScan==='function',
primaryLength:address(publicKey).length,generation,subaddress,otherAddress,fingerprint,poolKeys:Object.keys(pool)}));
`);
assert.equal(result.moduleLoaded, true);
assert.equal(result.primaryLength, 64);
});
+169 -6
View File
@@ -16,11 +16,14 @@ import { resolveCustodyIncident } from '../../server/market/custody-incident.ts'
import { CustodyMarket } from '../../server/market/custody.ts';
import { migrateCustody, grantCustodyRuntime, grantMoneroObserverRuntime, CUSTODY_SCHEMA_V1, CUSTODY_SCHEMA_V1_DIGEST,
grantMoneroDepositCreditRuntime, grantMoneroDepositAllocatorRuntime, grantMoneroWithdrawalRuntime,
grantPilotDepositReservationRuntime, grantPilotDepositAddressRuntime, CUSTODY_MIGRATION_V2,
grantPilotDepositReservationRuntime, grantPilotDepositAddressRuntime, grantPilotMoneroObserverRuntime,
CUSTODY_MIGRATION_V2,
CUSTODY_SCHEMA_V2_DIGEST } from '../../server/market/postgres-schema.ts';
import { parseMainnetCustodianId } from '../../server/market/mainnet-custody-identity.ts';
import { initializePilotCustodyIdentity, parsePilotCustodianId } from '../../server/market/pilot-custody-identity.ts';
import { createOrRecoverPilotMoneroDepositAddress, reservePilotMoneroDepositRequest } from '../../server/market/pilot-monero-deposit-allocation.ts';
import { createOrRecoverPilotMoneroDepositAddress, reservePilotMoneroDepositRequest,
reservePilotMoneroDepositIndex } from '../../server/market/pilot-monero-deposit-allocation.ts';
import { persistPilotMoneroObservationScan } from '../../server/market/pilot-monero-observation-store.ts';
import { readMainnetCustodyBalance, reserveMainnetWithdrawal, cancelMainnetWithdrawal } from '../../server/market/mainnet-withdrawal.ts';
import { mainnetCustodyBalanceMessage, mainnetWithdrawalCancellationMessage, mainnetWithdrawalMessage,
type MainnetCustodyBalanceRequestUnsigned, type MainnetWithdrawalRequestUnsigned,
@@ -208,7 +211,7 @@ test('explicit PostgreSQL migrations are idempotent and runtime role cannot alte
assert.equal((await f.market.audit()).assetsAtomicXmr, '1000000000000');
});
test('custody schema v1 upgrades additively through v13 without dropping existing metadata', async t => {
test('custody schema v1 upgrades additively through v14 without dropping existing metadata', async t => {
const admin = new Pool(config);
await admin.query('DROP SCHEMA IF EXISTS wk_custody CASCADE');
const client = await admin.connect();
@@ -222,7 +225,7 @@ test('custody schema v1 upgrades additively through v13 without dropping existin
try {
await migrateCustody(admin); await migrateCustody(admin);
const versions = await admin.query('SELECT version FROM wk_custody.schema_versions ORDER BY version');
assert.deepEqual(versions.rows.map(row => Number(row.version)), [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]);
assert.deepEqual(versions.rows.map(row => Number(row.version)), [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]);
assert.equal((await admin.query("SELECT value FROM wk_custody.custody_meta WHERE key='v1-preserved-marker'")).rows[0].value,
'existing-test-data');
for (const table of ['monero_observer_state', 'monero_transfer_observations', 'monero_observation_events', 'monero_observation_scans']) {
@@ -231,6 +234,7 @@ test('custody schema v1 upgrades additively through v13 without dropping existin
assert.ok((await admin.query("SELECT to_regclass('wk_custody.monero_deposit_allocations') AS name")).rows[0].name);
assert.ok((await admin.query("SELECT to_regclass('wk_custody.pilot_custody_identity') AS name")).rows[0].name);
assert.ok((await admin.query("SELECT to_regclass('wk_custody.pilot_monero_deposit_reservations') AS name")).rows[0].name);
assert.ok((await admin.query("SELECT to_regclass('wk_custody.pilot_monero_observation_events') AS name")).rows[0].name);
for (const table of ['monero_deposit_credit_events', 'monero_deposit_reorg_holds',
'monero_deposit_release_events', 'monero_deposit_reorg_resolutions']) {
assert.ok((await admin.query('SELECT to_regclass($1) AS name', [`wk_custody.${table}`])).rows[0].name);
@@ -238,7 +242,7 @@ test('custody schema v1 upgrades additively through v13 without dropping existin
} finally { await admin.end(); }
});
test('custody schema v2 upgrades explicitly through v13 deposit and withdrawal lifecycle tables', async t => {
test('custody schema v2 upgrades explicitly through v14 deposit and withdrawal lifecycle tables', async t => {
const admin = new Pool(config);
await admin.query('DROP SCHEMA IF EXISTS wk_custody CASCADE');
const client = await admin.connect();
@@ -251,7 +255,7 @@ test('custody schema v2 upgrades explicitly through v13 deposit and withdrawal l
await client.query('COMMIT');
await migrateCustody(admin); await migrateCustody(admin);
const versions = await admin.query('SELECT version FROM wk_custody.schema_versions ORDER BY version');
assert.deepEqual(versions.rows.map(row => Number(row.version)), [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]);
assert.deepEqual(versions.rows.map(row => Number(row.version)), [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14]);
assert.equal((await admin.query("SELECT value FROM wk_custody.custody_meta WHERE key='v2-marker'")).rows[0].value, 'preserve');
assert.ok((await admin.query("SELECT to_regclass('wk_custody.monero_deposit_release_events') AS name")).rows[0].name);
assert.ok((await admin.query("SELECT to_regclass('wk_custody.monero_withdrawal_intents') AS name")).rows[0].name);
@@ -507,6 +511,165 @@ test('pilot Wallet-RPC address creation recovers a lost response and registers o
}
});
test('pilot observation persistence classifies allocated and suspense outputs and records reorg evidence only', async t => {
const admin = new Pool(config), database = `pilot_observations_${randomUUID().replaceAll('-', '')}`;
const role = `pilot_observer_${randomUUID().replaceAll('-', '')}`;
await admin.query(`CREATE DATABASE "${database}"`);
const owner = new Pool({ ...config, database }), run = promisify(execFile);
const custodianId = randomUUID(), generation = randomUUID(), fingerprint = 'c'.repeat(64), now = 1_800_000_000_000;
const allocatedAddress = await syntheticSubaddress(21), suspenseAddress = await syntheticSubaddress(22);
const code = `import { Pool } from 'pg';
import { randomUUID } from 'node:crypto';
import { CHAIN_ID } from './server/testnet/consensus.ts';
import { initializePilotCustodyIdentity } from './server/market/pilot-custody-identity.ts';
import { reservePilotMoneroDepositIndex } from './server/market/pilot-monero-deposit-allocation.ts';
import { persistPilotMoneroObservationScan } from './server/market/pilot-monero-observation-store.ts';
const config={host:'127.0.0.1',database:process.env.DB,user:'postgres',ssl:false,max:4};
const owner=new Pool(config), observer=new Pool({...config,user:process.env.OBSERVER_ROLE});
const now=Number(process.env.NOW), walletGenerationId=process.env.GENERATION, walletFingerprint=process.env.FINGERPRINT;
try {
const identity=await initializePilotCustodyIdentity(owner,process.env.CUSTODIAN,now-100);
const intent={requestId:randomUUID(),custodianId:identity.custodianId,marketNetwork:identity.marketNetwork,
marketChainId:CHAIN_ID,moneroNetwork:'mainnet',beneficiary:'a'.repeat(64),walletGenerationId,walletFingerprint,
accountIndex:0,requestDigest:'b'.repeat(64)};
const allocation=(await reservePilotMoneroDepositIndex(owner,identity,intent,now-50)).reservation;
await owner.query("UPDATE wk_custody.pilot_monero_deposit_reservations SET status='creating',create_started_at=$2 WHERE allocation_id=$1",
[allocation.allocationId,now-40]);
await owner.query("UPDATE wk_custody.pilot_monero_deposit_reservations SET status='registered',monero_address=$2,registered_at=$3 WHERE allocation_id=$1",
[allocation.allocationId,process.env.ALLOCATED_ADDRESS,now-30]);
const assigned={observationId:'d'.repeat(64)+':'+'e'.repeat(64),transactionId:'d'.repeat(64),address:process.env.ALLOCATED_ADDRESS,
amountAtomicXmr:'9007199254740993',height:100,confirmations:11,timestamp:now-1000,unlockTime:'0',accountIndex:0,addressIndex:0,
outputPublicKey:'e'.repeat(64),globalOutputIndex:50,locked:false,doubleSpendSeen:false};
const unallocated={...assigned,observationId:'f'.repeat(64)+':'+'1'.repeat(64),transactionId:'f'.repeat(64),
address:process.env.SUSPENSE_ADDRESS,amountAtomicXmr:'7',height:101,confirmations:10,addressIndex:1,
outputPublicKey:'1'.repeat(64),globalOutputIndex:51};
const persist=(scanId,expectedScanHeight,fromHeight,throughHeight,walletHeight,incoming)=>persistPilotMoneroObservationScan(observer,
{scanId,walletGenerationId,walletFingerprint,expectedScanHeight,scannedAt:now+walletHeight,
page:{network:'mainnet',walletHeight,fromHeight,throughHeight,primaryAddress:process.env.PRIMARY_ADDRESS,incoming}});
const firstId=randomUUID(),first=await persist(firstId,0,0,110,110,[assigned,unallocated]);
const replay=await persist(firstId,0,0,110,110,[assigned,unallocated]);
const missing=await persist(randomUUID(),110,100,120,120,[]);
const reappeared=await persist(randomUUID(),120,100,130,130,[{...assigned,confirmations:31}]);
const rows=await observer.query('SELECT classification,currently_present,amount_atomic_xmr::text AS amount FROM wk_custody.pilot_monero_observations ORDER BY classification');
const events=await observer.query('SELECT event_type,count(*)::int AS n FROM wk_custody.pilot_monero_observation_events GROUP BY event_type ORDER BY event_type');
const journals=await owner.query('SELECT count(*)::int AS n FROM wk_custody.journals');
const credits=await owner.query('SELECT count(*)::int AS n FROM wk_custody.monero_deposit_credit_events');
console.log(JSON.stringify({first,replay,missing,reappeared,rows:rows.rows,events:events.rows,journals:journals.rows[0].n,credits:credits.rows[0].n}));
} finally { await observer.end(); await owner.end(); }`;
try {
await migrateCustody(owner);
await admin.query(`CREATE ROLE "${role}" LOGIN`);
await grantPilotMoneroObserverRuntime(owner, role);
const result = JSON.parse((await run(process.execPath, ['--experimental-strip-types', '--input-type=module', '-e', code], {
env: { ...process.env, WIKIDECK_NETWORK: 'pilot', WIKIDECK_PILOT_MODE: 'private-mock-only', DB: database,
OBSERVER_ROLE: role, CUSTODIAN: custodianId, GENERATION: generation, FINGERPRINT: fingerprint,
NOW: String(now), ALLOCATED_ADDRESS: allocatedAddress, SUSPENSE_ADDRESS: suspenseAddress,
PRIMARY_ADDRESS: ADDITIONAL_MARKET_FEE.address },
})).stdout.trim());
assert.equal(result.first.observed, 2); assert.equal(result.first.assigned, 1); assert.equal(result.first.unallocated, 1);
assert.equal(result.replay.replayed, true);
assert.equal(result.missing.notSeen, 2);
assert.equal(result.reappeared.reappeared, 1);
assert.deepEqual(result.rows.map((row: { classification: string; currently_present: boolean; amount: string }) =>
[row.classification,row.currently_present,row.amount]), [['assigned',true,'9007199254740993'],['unallocated',false,'7']]);
assert.deepEqual(result.events.map((row: { event_type: string; n: number }) => [row.event_type,row.n]),
[['not-seen',2],['reappeared',1],['seen',2]]);
assert.equal(result.journals, 0); assert.equal(result.credits, 0);
assert.equal((await owner.query("SELECT has_table_privilege($1,'wk_custody.accounts','SELECT') AS allowed", [role])).rows[0].allowed, false);
const observer = new Pool({ ...config, database, user: role });
try {
await assert.rejects(observer.query('DELETE FROM wk_custody.pilot_monero_observation_events'));
await assert.rejects(observer.query("UPDATE wk_custody.pilot_monero_observations SET monero_address=$1", [suspenseAddress]));
await assert.rejects(observer.query('SELECT * FROM wk_custody.journals'));
} finally { await observer.end(); }
} finally {
await owner.end();
await admin.query(`DROP DATABASE IF EXISTS "${database}" WITH (FORCE)`);
await admin.query(`DROP ROLE IF EXISTS "${role}"`);
await admin.end();
}
});
test('pilot observations classify reserved-wallet outputs, persist replay/reorg evidence, and never post credits', async t => {
const admin = new Pool(config), database = `pilot_observations_${randomUUID().replaceAll('-', '')}`;
const role = `pilot_observer_${randomUUID().replaceAll('-', '')}`;
await admin.query(`CREATE DATABASE "${database}"`);
const owner = new Pool({ ...config, database }), run = promisify(execFile);
const custodianId = randomUUID(), generation = randomUUID(), fingerprint = 'c'.repeat(64), now = 1_800_000_000_000;
const allocatedAddress = await syntheticSubaddress(21), suspenseAddress = await syntheticSubaddress(22);
const primaryAddress = ADDITIONAL_MARKET_FEE.address;
const code = `import { Pool } from 'pg';
import { randomUUID } from 'node:crypto';
import { CHAIN_ID } from './server/testnet/consensus.ts';
import { initializePilotCustodyIdentity } from './server/market/pilot-custody-identity.ts';
import { reservePilotMoneroDepositIndex } from './server/market/pilot-monero-deposit-allocation.ts';
import { persistPilotMoneroObservationScan } from './server/market/pilot-monero-observation-store.ts';
const config={host:'127.0.0.1',database:process.env.DB,user:'postgres',ssl:false,max:4};
const owner=new Pool(config), observer=new Pool({...config,user:process.env.OBSERVER_ROLE});
const now=Number(process.env.NOW), walletGenerationId=process.env.GENERATION, walletFingerprint=process.env.FINGERPRINT;
try {
const identity=await initializePilotCustodyIdentity(owner,process.env.CUSTODIAN,now-100);
const intent={requestId:randomUUID(),custodianId:identity.custodianId,marketNetwork:identity.marketNetwork,
marketChainId:CHAIN_ID,moneroNetwork:'mainnet',beneficiary:'a'.repeat(64),walletGenerationId,walletFingerprint,
accountIndex:0,requestDigest:'b'.repeat(64)};
const allocation=(await reservePilotMoneroDepositIndex(owner,identity,intent,now-50)).reservation;
await owner.query("UPDATE wk_custody.pilot_monero_deposit_reservations SET status='creating',create_started_at=$2 WHERE allocation_id=$1",
[allocation.allocationId,now-40]);
await owner.query("UPDATE wk_custody.pilot_monero_deposit_reservations SET status='registered',monero_address=$2,registered_at=$3 WHERE allocation_id=$1",
[allocation.allocationId,process.env.ALLOCATED_ADDRESS,now-30]);
const assigned={observationId:'d'.repeat(64)+':'+'e'.repeat(64),transactionId:'d'.repeat(64),address:process.env.ALLOCATED_ADDRESS,
amountAtomicXmr:'9007199254740993',height:100,confirmations:11,timestamp:now-1000,unlockTime:'0',accountIndex:0,addressIndex:0,
outputPublicKey:'e'.repeat(64),globalOutputIndex:50,locked:false,doubleSpendSeen:false};
const unallocated={...assigned,observationId:'f'.repeat(64)+':'+'1'.repeat(64),transactionId:'f'.repeat(64),
address:process.env.SUSPENSE_ADDRESS,amountAtomicXmr:'7',height:101,confirmations:10,addressIndex:1,
outputPublicKey:'1'.repeat(64),globalOutputIndex:51};
const persist=(scanId,expectedScanHeight,fromHeight,throughHeight,walletHeight,incoming)=>persistPilotMoneroObservationScan(observer,
{scanId,walletGenerationId,walletFingerprint,expectedScanHeight,scannedAt:now+walletHeight,
page:{network:'mainnet',walletHeight,fromHeight,throughHeight,primaryAddress:process.env.PRIMARY_ADDRESS,incoming}});
const firstId=randomUUID(),first=await persist(firstId,0,0,110,110,[assigned,unallocated]);
const replay=await persist(firstId,0,0,110,110,[assigned,unallocated]);
const missing=await persist(randomUUID(),110,100,120,120,[]);
const reappeared=await persist(randomUUID(),120,100,130,130,[{...assigned,confirmations:31}]);
const rows=await observer.query('SELECT classification,currently_present,amount_atomic_xmr::text AS amount FROM wk_custody.pilot_monero_observations ORDER BY classification');
const events=await observer.query('SELECT event_type,count(*)::int AS n FROM wk_custody.pilot_monero_observation_events GROUP BY event_type ORDER BY event_type');
const journals=await owner.query('SELECT count(*)::int AS n FROM wk_custody.journals');
const credits=await owner.query('SELECT count(*)::int AS n FROM wk_custody.monero_deposit_credit_events');
console.log(JSON.stringify({first,replay,missing,reappeared,rows:rows.rows,events:events.rows,journals:journals.rows[0].n,credits:credits.rows[0].n}));
} finally { await observer.end(); await owner.end(); }`;
try {
await migrateCustody(owner);
await admin.query(`CREATE ROLE "${role}" LOGIN`);
await grantPilotMoneroObserverRuntime(owner, role);
const result = JSON.parse((await run(process.execPath, ['--experimental-strip-types', '--input-type=module', '-e', code], {
env: { ...process.env, WIKIDECK_NETWORK: 'pilot', WIKIDECK_PILOT_MODE: 'private-mock-only', DB: database,
OBSERVER_ROLE: role, CUSTODIAN: custodianId, GENERATION: generation, FINGERPRINT: fingerprint,
NOW: String(now), ALLOCATED_ADDRESS: allocatedAddress, SUSPENSE_ADDRESS: suspenseAddress,
PRIMARY_ADDRESS: primaryAddress },
})).stdout.trim());
assert.equal(result.first.observed, 2); assert.equal(result.first.assigned, 1); assert.equal(result.first.unallocated, 1);
assert.equal(result.replay.replayed, true);
assert.equal(result.missing.notSeen, 2);
assert.equal(result.reappeared.reappeared, 1);
assert.deepEqual(result.rows.map((row: { classification: string; currently_present: boolean; amount: string }) =>
[row.classification,row.currently_present,row.amount]), [['assigned',true,'9007199254740993'],['unallocated',false,'7']]);
assert.deepEqual(result.events.map((row: { event_type: string; n: number }) => [row.event_type,row.n]),
[['not-seen',2],['reappeared',1],['seen',2]]);
assert.equal(result.journals, 0); assert.equal(result.credits, 0);
assert.equal((await owner.query("SELECT has_table_privilege($1,'wk_custody.accounts','SELECT') AS allowed", [role])).rows[0].allowed, false);
const observer = new Pool({ ...config, database, user: role });
try {
await assert.rejects(observer.query('DELETE FROM wk_custody.pilot_monero_observation_events'));
await assert.rejects(observer.query("UPDATE wk_custody.pilot_monero_observations SET monero_address=$1", [suspenseAddress]));
await assert.rejects(observer.query('SELECT * FROM wk_custody.journals'));
} finally { await observer.end(); }
} finally {
await owner.end();
await admin.query(`DROP DATABASE IF EXISTS "${database}" WITH (FORCE)`);
await admin.query(`DROP ROLE IF EXISTS "${role}"`);
await admin.end();
}
});
test('mainnet custody identity bootstrap is one-time, chain-bound, and does not open withdrawals', async t => {
assert.equal(parseMainnetCustodianId('AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA'), 'aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa');
assert.throws(() => parseMainnetCustodianId('not-a-uuid'), /Invalid mainnet custodian UUID/);