diff --git a/README.md b/README.md index 52bc850..b14f998 100644 --- a/README.md +++ b/README.md @@ -109,6 +109,8 @@ This mines three real blocks using disposable identities. It changes the testnet ## Network and safety limits +[SHKeeper integration groundwork](docs/shkeeper-settlement.md) is an offline, fake-transport settlement prototype only. It tests signed invoice terms and durable seller/1% payout reconciliation; it is not connected to the running market, real processor, production backup or public APIs. Live payment activation remains unavailable. + [Deployment and recovery preparation](docs/operations-recovery.md) includes explicit trusted-proxy configuration, opt-in loopback-only diagnostics and `ops:check`, plus stopped-node paired `node:backup` create/verify/new-directory restore tooling. Backups contain sensitive market data and require external encryption/access control. Tools do not configure TLS/alerts or authorize production launch/restore. [Stage-1 node resource controls](docs/node-resources.md) bound global RPC admission, response bytes and canonical page sizes. The UI reuses paged history; legacy full-state responses stop beyond 128 blocks. A consent-gated loopback-only smoke-load script is available as `test:rpc-load`. This does not resolve long-chain/DAG storage limits or establish production capacity. diff --git a/deploy/nginx.conf.example b/deploy/nginx.conf.example index e8b601f..2c5220e 100755 --- a/deploy/nginx.conf.example +++ b/deploy/nginx.conf.example @@ -38,4 +38,4 @@ server { proxy_send_timeout 15s; proxy_read_timeout 30s; } -} \ No newline at end of file +} diff --git a/docs/marketplace.md b/docs/marketplace.md index 6be8915..63785c3 100755 --- a/docs/marketplace.md +++ b/docs/marketplace.md @@ -73,7 +73,7 @@ For **review**, preserve backups and inspect canonical blocks, pending transfers ## SHKeeper production boundary -The intended future live processor is **SHKeeper**, using the operator-provided instance at https://shkeeper.eom.dev/. **It is not contacted or configured by this sandbox; there is no live adapter in this release.** Do not supply payment credentials yet. No secret belongs in browser build variables, source, or version-controlled Compose files. +The intended future live processor is **SHKeeper**, using the operator-provided instance at https://shkeeper.eom.dev/. **It is not contacted or configured by this sandbox; no live adapter is enabled.** Stage 5 adds an [offline wire-contract and durable settlement prototype](shkeeper-settlement.md) with fake-transport tests, separate from the running marketplace. Its live factory always refuses activation. Do not supply payment credentials yet. No secret belongs in browser build variables, source, or version-controlled Compose files. According to [SHKeeper's API documentation](https://shkeeper.io/api/), creating an XMR invoice uses `POST /api/v1/XMR/payment_request` with a USD amount and an API key, which naturally supports USD listings with an XMR checkout quote. Seller payouts use a separate Basic-authenticated XMR payout endpoint. The merchant wallet receives the gross amount; paying the seller net means the operator temporarily **custodies funds** and retains the market fee. Network fees and any processor fees need an explicit, disclosed policy before enabling this path. diff --git a/docs/operations-recovery.md b/docs/operations-recovery.md index b578d17..0d30247 100755 --- a/docs/operations-recovery.md +++ b/docs/operations-recovery.md @@ -67,4 +67,4 @@ The native HTTP integration now cleanly stops its disposable nodes, backs up the Final Node 24 TypeScript/Vite and production runtime builds passed. The suite passed **98 tests, 0 failed, 0 skipped**. A separate packaged-runtime smoke drill ran with no external network, read-only root, dropped capabilities and a disposable 64 MiB tmpfs: monitoring correctly returned nonzero for isolation, empty chain, not-yet-reconciled market and low disk; clean stopped-node backup, verification and new-directory restore each exited zero. That empty-chain CLI drill reported proof revalidation as false, as designed; nonempty real-proof replay is covered by the integration drill above. No host volume or production wallet was used. -SEC-07/SEC-09 are only partially addressed: actual edge/source-IP/TLS checks, a production monitoring agent and verified alert delivery, off-host backup storage, long-history restore timing, filesystem/power-loss testing, independent peer/miner topology and named operators remain unverified. See [operator launch register](operator-launch.md) and [security review](security-review.md). No startup or payment approval is granted by these tools. \ No newline at end of file +SEC-07/SEC-09 are only partially addressed: actual edge/source-IP/TLS checks, a production monitoring agent and verified alert delivery, off-host backup storage, long-history restore timing, filesystem/power-loss testing, independent peer/miner topology and named operators remain unverified. See [operator launch register](operator-launch.md) and [security review](security-review.md). No startup or payment approval is granted by these tools. diff --git a/docs/operator-launch.md b/docs/operator-launch.md index f9065d6..f70836b 100755 --- a/docs/operator-launch.md +++ b/docs/operator-launch.md @@ -23,6 +23,8 @@ The runtime image can execute the entry point with `--launch-preflight` as well; ## 2. Evidence register — all required before production sign-off +[Stage-5 SHKeeper prototype](shkeeper-settlement.md) is fixture-only and disconnected from the public market. Production payment activation, real processor verification, custody/refund policy, journal/backup/monitor integration and operator reconciliation remain missing. It cannot justify closing the real-money hold point. + [Stage-4 deployment and recovery tooling](operations-recovery.md) adds explicit proxy trust, private loopback diagnostics/checks and stopped-node pair backup/verification/non-overwriting restore. A local native proof-replay restore drill is automated, but actual-host TLS/proxy verification, off-host recovery, alert delivery and operational ownership still require external evidence. [Stage-3 consensus calibration evidence](consensus-calibration.md) contains a bounded local native measurement and deterministic timestamp/retarget scenarios. It is not the missing representative network-hashpower or independent consensus review evidence. Do not populate launch approvals from its illustrative targets. diff --git a/docs/security-review.md b/docs/security-review.md index 4a73dbc..96331d5 100755 --- a/docs/security-review.md +++ b/docs/security-review.md @@ -12,6 +12,8 @@ Scope: current consensus/ledger/P2P/RandomX integration, HTTP entry point, brows ## Findings that block production +**Stage-5 follow-up:** [Offline SHKeeper/settlement prototype](shkeeper-settlement.md) adds wire-contract validation and durable signed-quote/payout failure-path tests, without connecting a real processor or changing mock-only runtime behavior. SEC-04/SEC-05 remain open; the prototype is not production accounting/backup/UI integration or an atomic/refund solution. + **Stage-4 follow-up:** [Operations/recovery tooling](operations-recovery.md) provides explicit trusted proxy IP/CIDRs, opt-in separate loopback diagnostics and aggregate monitoring checks, a proxy example (not installed), stopped-node SQLite pair snapshots/verification and new-directory restore. Native integration replays restored reference proofs and preserves marketplace identity/orders/receipts. SEC-07 and SEC-09 remain open pending intended-host deployment, off-host recovery and real alert/on-call evidence. **Stage-3 follow-up:** [Consensus calibration tools and evidence](consensus-calibration.md) add an offline native benchmark and deterministic actual-helper scenarios plus validator timestamp regressions. Initial target is 16 expected hashes; a short quota-limited sample is not a network calibration. SEC-02 remains open pending representative hashpower, adversarial/multi-host evidence and independent review; no consensus/genesis change was made. diff --git a/docs/shkeeper-settlement.md b/docs/shkeeper-settlement.md new file mode 100755 index 0000000..3454010 --- /dev/null +++ b/docs/shkeeper-settlement.md @@ -0,0 +1,73 @@ +# SHKeeper integration prototype — stage 5, disabled + +**This is an offline, fixture-only implementation, not live checkout.** The running marketplace remains mock-only on every profile. `paymentMode` still rejects non-mock settings, `createLiveProcessor()` always throws, and mainnet/production approval gates remain unchanged. No real invoice, callback, payout, refund or credential exchange was performed. The operator-provided SHKeeper instance was **not contacted**. Only the public API documentation was read. + +## What is implemented + +- [XMR validation](../server/market/xmr.ts): Monero block-Base58 decoding, mainnet standard/subaddress prefix and Keccak-256 checksum checking, integer atomic-unit amount parsing (12 decimal places), bounded USD decimal parsing. Integrated/payment-ID and testnet addresses are rejected. A valid address/checksum is not proof of curve-key correctness, ownership or recipient control. Monero has tail emission; no invented fixed-supply cap is used as a validator. +- [SHKeeper wire-contract harness](../server/market/shkeeper.ts): explicit injected transport only, fixed HTTPS-shaped processor origin, correct API-key versus Basic authentication, no redirects, five-second deadline, 256 KiB streamed response limit, bounded schemas/arrays and secret-safe request failures. There is no global/default `fetch`, production network transport, credential loader, environment bypass or usable callback route. Tests supply ephemeral fixture credentials and fake `Response` objects. The factory name deliberately says **Harness**. +- [Durable settlement lab](../server/market/settlement-lab.ts): a separate SQLite/WAL journal, serialized operations, strict network identity and exclusive single-process lock; seller authorization of payout address/sale terms; buyer authorization of the exact invoice/address/amount/expiry and both fee allocations; authoritative invoice polling; exact confirmed payment verification; trusted canonical-delivery-reader boundary; durable payout intents; separate seller and hardcoded 1% payout legs; task/transaction allocation uniqueness; partial-success and ambiguous-outcome holds; restart reconciliation. +- No web UI/action, callback handler, background worker or production `Market` path constructs the harness/lab. The prototype must not be connected to real money by simply injecting global `fetch` or changing an environment variable. + +## API contract evidence + +Read from [public SHKeeper API docs](https://shkeeper.io/api/) version **2.1.47** on 2026-10-20, including the embedded OpenAPI definitions for `Invoice`, `Transaction` and `PayoutResultObj`: + +| Operation | Contract used | +| --- | --- | +| Create invoice | `POST /api/v1/XMR/payment_request`, `X-Shkeeper-Api-Key`, USD `amount`, unique `external_id`, required `callback_url`. Response has `id`, XMR `amount`, `wallet`, `exchange_rate`, `recalculate_after`. | +| Invoice lookup | `GET /api/v1/invoices/{external_id}`, API key, `status: success`, `invoices` array. Invoice fields: `external_id`, `fiat`, `amount_fiat`, `balance_fiat`, `status` in UNPAID/PARTIAL/PAID/OVERPAID, `txs`. | +| Invoice transaction | `addr`, `amount`, `crypto`, `status` CONFIRMED/UNCONFIRMED, `txid`. No numeric confirmation count is documented in this schema. | +| XMR payout | `POST /api/v1/XMR/payout`, Basic authentication, `amount`, `destination`, `fee` priority 1–4. Prototype chooses 2. XMR does not support the multipayout endpoint. | +| Payout status | `GET /api/v1/XMR/task/{task_id}`, Basic authentication, PENDING/FAILURE/SUCCESS; result entries contain `amount`, `dest`, success/error, optional message and `txids`. | + +The harness accepts numeric/string invoice IDs as documented examples differ, but requires UUID payout task IDs. It accepts quote `recalculate_after` only 0 or "0"; a floating/recalculating quote is refused rather than silently repriced. The required callback URL uses the reserved **market.invalid** domain and has no receiving route; it is a deliberate fixture placeholder that makes the create operation unsuitable for live use. + +Documentation alone cannot prove the deployed processor returns identical shapes/semantics. The actual instance/version, XMR enablement, precision, quote locking, fees, confirmation policy, task success meaning and failure/partial-send behavior require an explicitly authorized future integration review. A processor's `CONFIRMED` flag is not proof of an agreed XMR confirmation depth until its configured policy is independently established. Callback signing was not established by these schemas; **callback payloads never authorize payment** in this prototype. + +## Authorization and accounting + +Seller signs the lab's domain-separated network/chain/order/card/sequence/buyer terms, USD price, hosting basis points, expiry, NFT delivery depth, payout public key/address, and fixed **1% recipient/rate**. This proof links their NFT identity to the requested XMR destination; it does not prove that they control its private spend key. Invalid checksum/network destinations fail before quote creation. + +After invoice creation, buyer signs the exact invoice ID, currency, address, XMR amount, exchange rate, USD terms, expiry, hosting retention, seller net/destination and fixed-fee amount/destination. Wrong buyer, modified quote and wrong signature are refused. This second authorization is distinct from the existing mock market's USD reservation command; the public marketplace has not been migrated to this protocol. + +Both fees are computed independently from gross XMR in integer atomic units, rounded down. Seller net = gross − hosting retention − 1% allocation. **Hosting stays in the merchant wallet**, as designed; it is not a missing third payout or burned funds. Seller net and 1% recipient require separate XMR payouts because XMR multipayout is unsupported. For 0.1 XMR and 2.5% hosting: 0.0025 retained, 0.0965 seller, 0.001 fixed recipient. A sub-atomic fixed fee rounds to zero and its leg is skipped, not inflated. + +Processor fees included in an invoice can alter actual gross/fiat semantics. The prototype requires exact expected fiat and crypto totals and will hold inconsistent fee-bearing cases rather than assuming compatibility. Network fees, overpayment handling, processor-fee policy and reserves must be designed/disclosed before connecting live. This is a custodial gross-receipt/net-payout design, **not** a trustless XMR/NFT exchange. + +## State machine and failure rules + +1. **Draft → invoice intent:** persist `invoice-submitting` before calling the fixture create endpoint. A response loss, invalid response or restart with unresolved intent enters review. The POST is not automatically repeated, even though invoice external IDs exist, because create/update behavior and outcome reconciliation must be reviewed. +2. **Quote → awaiting payment:** buyer explicitly authorizes the exact quote before payment can be recognized. Expired signatures/quotes fail. No public payable address is exposed by the running app. +3. **Payment polling:** only server-to-server-shaped lookup data is used; exactly one external-ID invoice is required. It must have the expected fiat amount, currency and transaction addresses/currency; no duplicate tx IDs; exact confirmed gross and fiat balance. Partial/unconfirmed payments wait. Overpaid, contradictory, unknown, changed or late observations enter review; there is no blind refund or automatic invoice reuse. +4. **Paid identity:** save the exact incoming tx-ID set and enforce global incoming allocation uniqueness in the lab journal. Later changes to the paid set or amounts suspend settlement. The same observed deposit cannot fund two lab orders. +5. **NFT delivery:** the reader is a trusted consensus-library boundary, never a buyer-supplied HTTP boolean. It must return the exact transfer identity (chain/card/from/to/sequence), required canonical depth and a well-formed tip. Before any payout, missing depth waits; after payout begins, changed delivery enters permanent review. Recheck invoice and delivery before each separate payout. These reads cannot make off-chain money and on-chain delivery atomic; a reorg can occur after any check. +6. **Durable payout intent:** mark a leg `submitting` and commit **before** its create POST. Save its task ID under a unique constraint. Transport timeout, malformed result, task-ID collision, interrupted persistence or restart with no acknowledged task ID becomes `ambiguous`/review. Never retry that payout automatically. +7. **Known tasks:** repeat status GETs only. PENDING waits. SUCCESS must contain exactly one successful result matching amount/destination, at least one well-formed distinct transaction ID and no transaction ID already allocated to another leg/order. FAILURE, malformed success, amount/address mismatch or duplicate evidence enters review. A known task query failure does not create another POST. +8. **Partial success:** seller completion is durably recorded before the fee leg. If fee fails, preserve the seller's success and hold the remainder. There is no rollback, re-payment of the seller or assumption that the entire sale was unpaid. Reorg after completion holds without paying again. + +`expiresAt` is an invoice/payment-authorization deadline, not a deadline that cancels a correctly paid sale while NFT delivery is being mined. Refusing payout solely because a legitimate invoice later expires would not be a safe general settlement policy. Delivery timeouts/disputes and reserves remain unimplemented live-policy requirements. + +## Journal, recovery and activation holds + +The lab uses a **separate settlement-lab.sqlite** and lock under a caller-supplied disposable directory. It is **not** inside the production two-database backup contract, not exposed by current operations diagnostics and not an authoritative production accounting system. Current node backups do not protect it. Test fixtures reopen the lab to verify intent/task persistence; this is not a production processor reconciliation tool. Do not delete a stale lab lock unless its owning process is verified stopped. + +There is no operator override/retry/refund endpoint. `review` is intentionally not auto-recovered. The lab retains privacy-sensitive participant/payout/transaction data and signatures but **not processor credentials**. Trusted local filesystem ownership and external encryption remain necessary. Schema inspection validates identity, but this is not tamper-resistant accounting against an operator who controls the DB. + +Before live integration can be reviewed/activated, the following remain required: + +- Production network readiness and independent custody/consensus/application review; explicit operator payment activation, separate from network startup. **No live approval variable is implemented in this release.** +- A production adapter with restricted server-only secret loading, approved fixed origin/DNS/egress controls, no redirects, TLS review, least-privilege credentials and a verified deployed API contract. Never put credentials in Vite/browser variables or code. +- Real-market journal/schema, backup/restore/monitoring integration and migrations; typed authenticity validation of persisted authorization/amount records; trusted canonical delivery lookup; exposure only through participant-authorized APIs, not direct lab methods. +- Actual quote presentation/authorization UX, seller payout-address registration/change authorization, minimum order/payout policy, exact fees and network-cost/reserve treatment. +- Verified XMR confirmation policy and task success/failure semantics; bounded callback hints only; late/partial/overpayment/refund and non-delivery rules. Previously signed NFT transfers may arrive after a proposed refund—no auto-refund atomicity claim is made. +- Durable operator reconciliation tooling for ambiguous create/payout requests, partial sends, known task failures, reorgs and external wallet/task history. Timeout is **not** failure/no-send evidence. Never erase an intent or re-submit to make tests/operations appear healthy. +- Controlled end-to-end processor tests only after specific approval; this work did not request secrets, create invoices, transfer XMR or access the configured instance. No fund movement is authorized by this document. + +## Tests + +Final Node 24 TypeScript/Vite build passed. The full suite ran in a Docker container with **external networking disabled**: **119 tests passed, 0 failed, 0 skipped**, including existing native RandomX loopback integration. No real SHKeeper endpoint or credential was used; only public documentation retrieval occurred outside those offline tests. + +Offline fake transports cover endpoint auth separation, bounded bytes/redirect/type failures, five-second abort, fixed quote policy, XMR checksum/network/decimal rejection, seller/buyer quote binding, confirmed exact payment/depth, duplicate inbound allocation, duplicate payout tasks/tx IDs, concurrent advancement, response-loss ambiguity, restart quarantine, known-task polling, seller-success/fee-failure, mismatched payouts, changed paid evidence and post-payout reorg hold. Existing mock marketplace/backup/lock/launch and native RandomX tests remain in the full suite. + +This is partial engineering groundwork for SEC-04/SEC-05, not closure of those production findings. Live payments remain hard-disabled. diff --git a/scripts/node-backup.ts b/scripts/node-backup.ts index a196c2f..3eb4b3b 100755 --- a/scripts/node-backup.ts +++ b/scripts/node-backup.ts @@ -9,4 +9,4 @@ const source=arg('--source'),destination=arg('--destination'); if(!source||source.startsWith('--'))throw new Error('Explicit --source required'); if(action!=='verify'&&(!args.includes('--consent')||!destination||destination.startsWith('--')))throw new Error('Explicit --consent and new --destination required; stop the node first'); const result=action==='create'?await createBackup(source,destination!):action==='restore'?await restoreBackup(source,destination!):await verifyBackup(source); -console.log(JSON.stringify(result,null,2)); \ No newline at end of file +console.log(JSON.stringify(result,null,2)); diff --git a/scripts/ops-check.ts b/scripts/ops-check.ts index 14d12ab..4d5f5b2 100755 --- a/scripts/ops-check.ts +++ b/scripts/ops-check.ts @@ -8,4 +8,4 @@ try{for(;;){const part=await reader.read();if(part.done)break;bytes+=part.value. finally{await reader.cancel();reader.releaseLock();} const joined=new Uint8Array(bytes);let offset=0;for(const chunk of chunks){joined.set(chunk,offset);offset+=chunk.length;} const report=checkOperations(JSON.parse(new TextDecoder('utf-8',{fatal:true}).decode(joined))); -console.log(JSON.stringify(report,null,2));if(!report.healthy)process.exitCode=1; \ No newline at end of file +console.log(JSON.stringify(report,null,2));if(!report.healthy)process.exitCode=1; diff --git a/server/market/settlement-lab.ts b/server/market/settlement-lab.ts new file mode 100755 index 0000000..9f3b970 --- /dev/null +++ b/server/market/settlement-lab.ts @@ -0,0 +1,206 @@ +import { DatabaseSync } from 'node:sqlite'; +import { mkdir,open,unlink,chmod } from 'node:fs/promises'; +import { join } from 'node:path'; +import { z } from 'zod'; +import { ADDITIONAL_MARKET_FEE } from '../../shared/market.ts'; +import { NETWORK } from '../../shared/network.ts'; +import { CHAIN_ID,sha256,verifySignature,transactionId } from '../testnet/consensus.ts'; +import { atomicXmr,fiatCents,validateXmrAddress } from './xmr.ts'; +import { xmrString } from './payments.ts'; +import type { SettlementProcessor,ProcessorQuote,InvoiceObservation } from './shkeeper.ts'; + +const hex=z.string().regex(/^[a-f0-9]{64}$/); +const termsSchema=z.object({id:z.string().uuid(),network:z.literal(NETWORK),chainId:z.literal(CHAIN_ID), + cardHash:hex,sequence:z.number().int().nonnegative().safe(),seller:hex,buyer:hex,sellerPublicKey:z.string().max(256), + buyerPublicKey:z.string().max(256),sellerXmr:z.string().length(95),priceCents:z.number().int().min(1).max(100_000_000), + feeBps:z.number().int().min(0).max(5000),expiresAt:z.number().int().nonnegative().safe(), + deliveryConfirmations:z.number().int().min(2).max(1000)}).strict(); +export type SettlementTerms=z.infer; +export function sellerSettlementMessage(t:SettlementTerms){return JSON.stringify([NETWORK,'settlement-lab-terms-v1', + t.chainId,t.id,t.cardHash,t.sequence,t.seller,t.buyer,t.sellerPublicKey,t.buyerPublicKey,t.sellerXmr,t.priceCents,t.feeBps, + t.expiresAt,t.deliveryConfirmations,ADDITIONAL_MARKET_FEE.bps,ADDITIONAL_MARKET_FEE.address]);} +type Leg={amount:string;destination:string;state:'ready'|'submitting'|'pending'|'complete'|'ambiguous'|'review'|'skipped';taskId?:string;txids?:string[]}; +export type LabSettlement={terms:SettlementTerms;sellerSignature:string; + status:'draft'|'invoice-submitting'|'quote'|'awaiting-payment'|'paid'|'paying'|'complete'|'review'; + quote?:ProcessorQuote;quoteDigest?:string;buyerSignature?:string;paidObservedAt?:number;paidTxids?:string[]; + hostingXmr?:string;seller?:Leg;fee?:Leg;reviewReason?:string}; +export function buyerInvoiceMessage(s:LabSettlement){ + if(!s.quote||!s.seller||!s.fee)throw new Error('No invoice quote to authorize'); + return JSON.stringify([NETWORK,'settlement-lab-invoice-v1',s.terms.chainId,s.terms.id,sha256(sellerSettlementMessage(s.terms)), + s.quote.id,'XMR','USD',s.terms.priceCents,s.quote.address,s.quote.amountXmr,s.quote.exchangeRate,s.terms.expiresAt, + s.hostingXmr,s.seller.amount,s.seller.destination,s.fee.amount,s.fee.destination]); +} +export type DeliveryEvidence={transferId:string|null;confirmations:number;tip:string}; +/** Trusted consensus-reader boundary, NOT a boolean submitted by an HTTP client. */ +export type DeliveryReader=(terms:SettlementTerms)=>Promise; + +/** Offline settlement research journal. NOT imported by app/Market and NOT part of production + * backup format. No public routes, worker, callback or CLI activation. Test fixtures only. */ +export class SettlementLab { + private readonly db:DatabaseSync; + private readonly processor:SettlementProcessor; + private readonly delivery:DeliveryReader; + private readonly now:()=>number; + private readonly release:()=>Promise; + private queue:Promise=Promise.resolve();private queued=0; + private constructor(db:DatabaseSync,processor:SettlementProcessor,delivery:DeliveryReader,now:()=>number,release:()=>Promise){ + this.db=db;this.processor=processor;this.delivery=delivery;this.now=now;this.release=release; + } + static async open(directory:string,processor:SettlementProcessor,delivery:DeliveryReader,now=Date.now){ + await mkdir(directory,{recursive:true,mode:0o700});const file=join(directory,'settlement-lab.sqlite'),lockPath=file+'.lock'; + const lock=await open(lockPath,'wx',0o600);let db:DatabaseSync|undefined; + try{ + db=new DatabaseSync(file); + db.exec(`PRAGMA journal_mode=WAL;PRAGMA synchronous=FULL;PRAGMA busy_timeout=5000; + CREATE TABLE IF NOT EXISTS lab_meta(key TEXT PRIMARY KEY,value TEXT NOT NULL); + CREATE TABLE IF NOT EXISTS lab_orders(id TEXT PRIMARY KEY,data TEXT NOT NULL); + CREATE TABLE IF NOT EXISTS lab_tasks(id TEXT PRIMARY KEY,order_id TEXT NOT NULL); + CREATE TABLE IF NOT EXISTS lab_txids(id TEXT PRIMARY KEY,order_id TEXT NOT NULL); + CREATE TABLE IF NOT EXISTS lab_incoming(id TEXT PRIMARY KEY,order_id TEXT NOT NULL);`); + const identity=JSON.stringify([NETWORK,CHAIN_ID,'offline-settlement-lab-v1']); + const row=db.prepare("SELECT value FROM lab_meta WHERE key='identity'").get() as {value:string}|undefined; + if(row&&row.value!==identity)throw new Error('Settlement lab network/schema mismatch'); + if(!row)db.prepare('INSERT INTO lab_meta VALUES(?,?)').run('identity',identity); + if(process.platform!=='win32')await chmod(file,0o600); + // A committed intent with no saved task ID must never be replayed after restart. + const update=db.prepare('UPDATE lab_orders SET data=? WHERE id=?'); + db.exec('BEGIN IMMEDIATE'); + try{ + for(const row of db.prepare('SELECT id,data FROM lab_orders').all() as {id:string;data:string}[]){ + const s=JSON.parse(row.data) as LabSettlement;let uncertain=s.status==='invoice-submitting'; + for(const leg of [s.seller,s.fee])if(leg?.state==='submitting'){leg.state='ambiguous';uncertain=true;} + if(uncertain){s.status='review';s.reviewReason='restart-after-unresolved-processor-intent';update.run(JSON.stringify(s),row.id);} + } + db.exec('COMMIT'); + }catch(error){db.exec('ROLLBACK');throw error;} + return new SettlementLab(db,processor,delivery,now,async()=>{await lock.close();await unlink(lockPath);}); + }catch(error){db?.close();await lock.close();await unlink(lockPath);throw error;} + } + async close(){await this.queue;this.db.close();await this.release();} + get(id:string):LabSettlement{ + z.string().uuid().parse(id);const row=this.db.prepare('SELECT data FROM lab_orders WHERE id=?').get(id) as {data:string}|undefined; + if(!row)throw new Error('Unknown lab settlement');return JSON.parse(row.data); + } + private save(s:LabSettlement){this.db.prepare('UPDATE lab_orders SET data=? WHERE id=?').run(JSON.stringify(s),s.terms.id);} + private hold(s:LabSettlement,reason:string){s.status='review';s.reviewReason=reason;this.save(s);return s;} + private serialize(fn:()=>Promise):Promise{ + if(this.queued>=32)return Promise.reject(new Error('Settlement lab busy')); + this.queued++;const run=this.queue.then(async()=>{try{return await fn();}finally{this.queued--;}}); + this.queue=run.catch(()=>undefined);return run; + } + create(raw:unknown,sellerSignature:string){ + const terms=termsSchema.parse(raw); + return this.serialize(async()=>{ + if(terms.seller===terms.buyer||terms.expiresAt<=this.now()||terms.expiresAt>this.now()+30*60000|| + !verifySignature(terms.sellerPublicKey,terms.seller,sellerSettlementMessage(terms),sellerSignature))throw new Error('Invalid seller terms or expiry'); + await validateXmrAddress(terms.sellerXmr);await validateXmrAddress(ADDITIONAL_MARKET_FEE.address); + if((this.db.prepare('SELECT COUNT(*) AS n FROM lab_orders').get() as {n:number}).n>=1000)throw new Error('Lab journal capacity reached'); + const s:LabSettlement={terms,sellerSignature,status:'draft'}; + this.db.prepare('INSERT INTO lab_orders VALUES(?,?)').run(terms.id,JSON.stringify(s));return s; + }); + } + issueInvoice(id:string){return this.serialize(async()=>{ + const s=this.get(id);if(s.status!=='draft')return s; + if(this.now()>=s.terms.expiresAt)return this.hold(s,'invoice-deadline'); + s.status='invoice-submitting';this.save(s); // Durable intent BEFORE network-shaped I/O, no automatic POST retry. + try{ + s.quote=await this.processor.createInvoice(id,s.terms.priceCents); + await validateXmrAddress(s.quote.address);const gross=atomicXmr(s.quote.amountXmr); + if(!s.quote.id||gross<=0n)throw new Error('Invalid quote'); + const hosting=gross*BigInt(s.terms.feeBps)/10000n,fee=gross*100n/10000n; + s.hostingXmr=xmrString(hosting); + s.seller={amount:xmrString(gross-hosting-fee),destination:s.terms.sellerXmr,state:'ready'}; + s.fee={amount:xmrString(fee),destination:ADDITIONAL_MARKET_FEE.address,state:fee===0n?'skipped':'ready'}; + s.quoteDigest=sha256(buyerInvoiceMessage(s));s.status='quote';this.save(s);return s; + }catch{return this.hold(s,'invoice-create-ambiguous-or-invalid');} + });} + approveQuote(id:string,signature:string){return this.serialize(async()=>{ + const s=this.get(id); + if(s.status!=='quote'||this.now()>=s.terms.expiresAt|| + !verifySignature(s.terms.buyerPublicKey,s.terms.buyer,buyerInvoiceMessage(s),signature))throw new Error('Invalid buyer invoice authorization'); + s.buyerSignature=signature;s.status='awaiting-payment';this.save(s);return s; + });} + private async observe(s:LabSettlement,observation?:InvoiceObservation):Promise<'paid'|'waiting'|'review'>{ + let o:InvoiceObservation; + try{o=observation??await this.processor.invoice(s.terms.id);}catch{return 'review';} + if(!s.quote||o.external_id!==s.terms.id||o.fiat!=='USD'||fiatCents(o.amount_fiat)!==s.terms.priceCents)return 'review'; + let confirmed=0n,all=0n;const seen=new Set(); + for(const tx of o.txs){ + if(tx.addr!==s.quote.address||tx.crypto!=='XMR'||seen.has(tx.txid))return 'review'; + seen.add(tx.txid);const amount=atomicXmr(tx.amount);all+=amount;if(tx.status==='CONFIRMED')confirmed+=amount; + } + const gross=atomicXmr(s.quote.amountXmr); + if(s.paidTxids&&JSON.stringify([...seen].sort())!==JSON.stringify([...s.paidTxids].sort()))return 'review'; + if(o.status==='OVERPAID'||all>gross||fiatCents(o.balance_fiat)>s.terms.priceCents)return 'review'; + if(o.status==='PAID'&&confirmed===gross&&all===gross&&fiatCents(o.balance_fiat)===s.terms.priceCents)return 'paid'; + if(o.status==='UNPAID'&&all===0n&&fiatCents(o.balance_fiat)===0)return 'waiting'; + if(o.status==='PARTIAL'&&all{ + const s=this.get(id);if(s.status!=='awaiting-payment')return s; + if(this.now()>=s.terms.expiresAt)return this.hold(s,'late-or-expired-payment-requires-review'); + let observation:InvoiceObservation; + try{observation=await this.processor.invoice(s.terms.id);}catch{return this.hold(s,'invoice-verification-failed');} + const observed=await this.observe(s,observation); + if(observed==='review')return this.hold(s,'invoice-verification-failed'); + if(this.now()>=s.terms.expiresAt)return this.hold(s,'payment-observed-after-deadline-requires-review'); + if(observed==='paid'){ + // Independent observations must not allocate the same Monero transaction to two orders. + this.db.exec('BEGIN IMMEDIATE'); + try{ + for(const tx of observation.txs)this.db.prepare('INSERT INTO lab_incoming VALUES(?,?)').run(tx.txid,s.terms.id); + s.status='paid';s.paidObservedAt=this.now();s.paidTxids=observation.txs.map(tx=>tx.txid);this.save(s);this.db.exec('COMMIT'); + }catch{this.db.exec('ROLLBACK');return this.hold(s,'incoming-transaction-already-allocated-or-storage-conflict');} + }return s; + });} + private expectedDelivery(t:SettlementTerms){return transactionId({network:NETWORK,chainId:CHAIN_ID,cardHash:t.cardHash, + sequence:t.sequence,from:t.seller,to:t.buyer,publicKey:'',signature:''});} + private async ready(s:LabSettlement){ + const evidence=await this.delivery(structuredClone(s.terms)); + return evidence.transferId===this.expectedDelivery(s.terms)&&Number.isSafeInteger(evidence.confirmations)&& + evidence.confirmations>=s.terms.deliveryConfirmations&&/^[a-f0-9]{64}$/.test(evidence.tip); + } + advance(id:string){return this.serialize(async()=>{ + const s=this.get(id);if(!['paid','paying','complete'].includes(s.status))return s; + // Reads before each payout are risk controls, not cross-chain atomicity or finality. + if(await this.observe(s)!=='paid')return this.hold(s,'paid-invoice-no-longer-verifiable'); + if(!await this.ready(s)){ + if(s.status==='paid')return s; // Payment confirmed; still waiting for adequate NFT depth. + return this.hold(s,'delivery-reorganized-or-insufficient'); + } + if(s.status==='complete')return s; + s.status='paying';this.save(s); + for(const leg of [s.seller!,s.fee!]){ + if(leg.state==='complete'||leg.state==='skipped')continue; + if(leg.state==='ambiguous'||leg.state==='review'||leg.state==='submitting')return this.hold(s,'payout-intent-requires-operator-reconciliation'); + if(leg.state==='ready'){ + if(!await this.ready(s)||await this.observe(s)!=='paid')return this.hold(s,'settlement-evidence-changed-before-payout'); + leg.state='submitting';this.save(s); + try{ + leg.taskId=await this.processor.payout(leg.amount,leg.destination);z.string().uuid().parse(leg.taskId); + this.db.exec('BEGIN IMMEDIATE'); + try{this.db.prepare('INSERT INTO lab_tasks VALUES(?,?)').run(leg.taskId,s.terms.id);leg.state='pending';this.save(s);this.db.exec('COMMIT');} + catch(error){this.db.exec('ROLLBACK');throw error;} + } + catch{leg.state='ambiguous';return this.hold(s,'payout-create-outcome-unknown-never-retry-automatically');} + } + let task; + try{task=await this.processor.task(leg.taskId!);}catch{return s;} // Known task GET can be polled, no new payout POST. + if(task.status==='PENDING')return s; + const result=task.result; + if(task.status!=='SUCCESS'||result.length!==1||result[0].status!=='success'||result[0].dest!==leg.destination|| + atomicXmr(result[0].amount)!==atomicXmr(leg.amount)||!result[0].txids?.length|| + result[0].txids.some(tx=>!(/^[a-f0-9]{64}$/.test(tx)))||new Set(result[0].txids).size!==result[0].txids.length){ + leg.state='review';return this.hold(s,'payout-task-failed-partial-or-mismatched'); + } + this.db.exec('BEGIN IMMEDIATE'); + try{ + for(const txid of result[0].txids)this.db.prepare('INSERT INTO lab_txids VALUES(?,?)').run(txid,s.terms.id); + leg.state='complete';leg.txids=result[0].txids;this.save(s);this.db.exec('COMMIT'); + }catch{this.db.exec('ROLLBACK');leg.state='review';return this.hold(s,'payout-transaction-already-allocated-or-storage-conflict');} + } + s.status='complete';this.save(s);return s; + });} +} diff --git a/server/market/shkeeper.ts b/server/market/shkeeper.ts new file mode 100755 index 0000000..1441381 --- /dev/null +++ b/server/market/shkeeper.ts @@ -0,0 +1,86 @@ +import { z } from 'zod'; +import { validateXmrAddress, atomicXmr, fiatCents } from './xmr.ts'; + +const identifier=z.union([z.string().min(1).max(128),z.number().int().nonnegative().safe()]).transform(String); +const decimal=z.string().min(1).max(32); +const external=z.string().uuid(); +const transaction=z.object({addr:z.string().length(95),amount:decimal,crypto:z.literal('XMR'), + status:z.enum(['CONFIRMED','UNCONFIRMED']),txid:z.string().regex(/^[a-f0-9]{64}$/)}); +export const invoiceObservation=z.object({external_id:external,fiat:z.literal('USD'),amount_fiat:decimal, + balance_fiat:decimal,status:z.enum(['UNPAID','PARTIAL','PAID','OVERPAID']),txs:z.array(transaction).max(256)}); +export type InvoiceObservation=z.infer; +export type ProcessorQuote={id:string;address:string;amountXmr:string;exchangeRate:string}; +export type PayoutResult={status:'PENDING'|'FAILURE'|'SUCCESS';result:{amount:string;dest:string;status:'success'|'error';txids?:string[]}[]}; +/** Explicit injection only. There is deliberately NO default/global fetch or credential-env loader. + * This interface exercises wire contracts with offline fake responses, not a live activation gate. */ +export type FixtureTransport=(url:string,init:RequestInit)=>Promise; +export interface SettlementProcessor { + createInvoice(id:string,priceCents:number):Promise; + invoice(id:string):Promise; + payout(amount:string,destination:string):Promise; + task(id:string):Promise; +} +export function createLiveProcessor():never{ + throw new Error('Live SHKeeper activation is not implemented or approved. Mock mode only.'); +} +export function createSHKeeperHarness(transport:FixtureTransport,credentials:{apiKey:string;username:string;password:string}):SettlementProcessor{ + // Ephemeral fixtures are supplied by tests. Application startup never imports/constructs this harness. + if(!transport||typeof transport!=='function')throw new Error('Explicit offline fixture transport required'); + for(const value of Object.values(credentials))if(!value||value.length>1024||/[\r\n\x00]/.test(value))throw new Error('Invalid fixture credentials'); + if(credentials.username.includes(':'))throw new Error('Invalid Basic username'); + const origin='https://shkeeper.eom.dev'; + async function request(path:string,basic:boolean,body?:unknown):Promise{ + const controller=new AbortController(),timer=setTimeout(()=>controller.abort(),5000); + const aborted=new Promise((_,reject)=>{controller.signal.addEventListener('abort',()=>reject(new Error('Processor request deadline exceeded')),{once:true});}); + const operation=(async()=>{ + try{ + const response=await transport(origin+path,{method:body===undefined?'GET':'POST',redirect:'error',signal:controller.signal, + headers:{Accept:'application/json',...(basic?{Authorization:`Basic ${Buffer.from(`${credentials.username}:${credentials.password}`).toString('base64')}`}:{'X-Shkeeper-Api-Key':credentials.apiKey}), + ...(body===undefined?{}:{'Content-Type':'application/json'})},...(body===undefined?{}:{body:JSON.stringify(body)})}); + if(!response.ok||response.redirected||!response.body||!/^application\/json(?:\s*;|$)/i.test(response.headers.get('content-type')??'')){ + await response.body?.cancel();throw new Error('Processor response contract rejected'); + } + const length=response.headers.get('content-length'); + if(length!==null&&(!/^\d+$/.test(length)||Number(length)>256*1024)){await response.body.cancel();throw new Error('Processor response limit');} + const reader=response.body.getReader(),chunks:Uint8Array[]=[];let size=0; + try{for(;;){const part=await reader.read();if(part.done)break;size+=part.value.byteLength;if(size>256*1024)throw new Error('Processor response limit');chunks.push(part.value);}} + finally{await reader.cancel();reader.releaseLock();} + return JSON.parse(new TextDecoder('utf-8',{fatal:true}).decode(Buffer.concat(chunks))); + }catch{throw new Error('Processor operation failed or response could not be verified');} // Never include credentials/upstream body. + })(); + try{return await Promise.race([operation,aborted]);}finally{clearTimeout(timer);} + } + return { + async createInvoice(id,priceCents){ + external.parse(id);if(!Number.isSafeInteger(priceCents)||priceCents<1||priceCents>100_000_000)throw new Error('Invalid invoice price'); + const amount=`${Math.floor(priceCents/100)}.${String(priceCents%100).padStart(2,'0')}`; + const raw=await request('/api/v1/XMR/payment_request',false,{external_id:id,fiat:'USD',amount, + callback_url:`https://market.invalid/disabled-callback/${id}`}); // No live callback route or token. + const q=z.object({status:z.literal('success'),id:identifier,wallet:z.string().length(95),amount:decimal, + exchange_rate:decimal,recalculate_after:z.union([z.literal(0),z.literal('0')])}).parse(raw); + await validateXmrAddress(q.wallet);if(atomicXmr(q.amount)<=0n||!/^\d{1,12}(\.\d{1,12})?$/.test(q.exchange_rate)||Number(q.exchange_rate)<=0)throw new Error('Invalid processor quote'); + return {id:q.id,address:q.wallet,amountXmr:q.amount,exchangeRate:q.exchange_rate}; + }, + async invoice(id){ + external.parse(id); + const data=z.object({status:z.literal('success'),invoices:z.array(invoiceObservation).max(2)}).parse(await request(`/api/v1/invoices/${encodeURIComponent(id)}`,false)); + if(data.invoices.length!==1||data.invoices[0].external_id!==id)throw new Error('Invoice identity is missing or ambiguous'); + const invoice=data.invoices[0];fiatCents(invoice.amount_fiat);fiatCents(invoice.balance_fiat); + const seen=new Set(); + for(const tx of invoice.txs){await validateXmrAddress(tx.addr);atomicXmr(tx.amount);if(seen.has(tx.txid))throw new Error('Duplicate invoice transaction');seen.add(tx.txid);} + return invoice; + }, + async payout(amount,destination){ + if(atomicXmr(amount)<=0n)throw new Error('Payout must be positive');await validateXmrAddress(destination); + const result=z.object({task_id:z.string().uuid()}).parse(await request('/api/v1/XMR/payout',true,{amount,destination,fee:'2'})); + return result.task_id; + }, + async task(id){ + z.string().uuid().parse(id); + const result=z.object({status:z.enum(['PENDING','FAILURE','SUCCESS']),result:z.array(z.object({amount:decimal,dest:z.string().length(95), + status:z.enum(['success','error']),txids:z.array(z.string().regex(/^[a-f0-9]{64}$/)).max(64).optional()})).max(2).default([])}).parse(await request(`/api/v1/XMR/task/${id}`,true)); + for(const row of result.result){atomicXmr(row.amount);await validateXmrAddress(row.dest);} + return result; + }, + }; +} diff --git a/server/market/xmr.ts b/server/market/xmr.ts new file mode 100755 index 0000000..937b2ab --- /dev/null +++ b/server/market/xmr.ts @@ -0,0 +1,30 @@ +import { keccak } from 'hash-wasm'; + +const alphabet='123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'; +const decodedSizes:Record={2:1,3:2,5:3,6:4,7:5,9:6,10:7,11:8}; +/** Support only mainnet standard/subaddresses. Integrated/payment-ID addresses need a separate policy. */ +export async function validateXmrAddress(address:string):Promise{ + if(typeof address!=='string'||address.length!==95)throw new Error('Expected a mainnet standard XMR address or subaddress'); + const bytes:number[]=[]; + for(let start=0;start(size).fill(0); + for(let i=size-1;i>=0;i--){output[i]=Number(value&255n);value>>=8n;} + if(value)throw new Error('XMR Base58 block overflow');bytes.push(...output); + } + if(bytes.length!==69||![18,42].includes(bytes[0]))throw new Error('XMR address network/type is not supported'); + const decoded=Uint8Array.from(bytes),expected=(await keccak(decoded.slice(0,-4),256)).slice(0,8); + if(Buffer.from(decoded.slice(-4)).toString('hex')!==expected)throw new Error('XMR checksum mismatch'); + return address; +} +export function atomicXmr(value:string):bigint{ + if(typeof value!=='string'||!/^(0|[1-9]\d{0,9})(\.\d{1,12})?$/.test(value))throw new Error('Invalid bounded XMR decimal'); + const [whole,fraction='']=value.split('.');return BigInt(whole)*1_000_000_000_000n+BigInt(fraction.padEnd(12,'0')); +} +export function fiatCents(value:string):number{ + if(typeof value!=='string'||!/^(0|[1-9]\d{0,6})(\.\d{1,2})?$/.test(value))throw new Error('Invalid USD decimal'); + const [whole,fraction='']=value.split('.');return Number(whole)*100+Number(fraction.padEnd(2,'0')); +} diff --git a/server/operations-check.ts b/server/operations-check.ts index f565c6c..9a55adc 100755 --- a/server/operations-check.ts +++ b/server/operations-check.ts @@ -23,4 +23,4 @@ export function checkOperations(raw:unknown,now=Date.now()){ else if(disk.data.availableBytes<512*1024*1024)warnings.push('disk-low'); return {healthy:warnings.length===0,warnings, note:'Local threshold hints only, not consensus readiness or launch approval. Reorg/rejection counters and native RSS need external time-series monitoring.'}; -} \ No newline at end of file +} diff --git a/server/operations.ts b/server/operations.ts index bc4a07c..2562e01 100755 --- a/server/operations.ts +++ b/server/operations.ts @@ -60,4 +60,4 @@ export function startOperations(port: number | undefined, snapshot: ()=>unknown, server.on('error',()=>{clearInterval(diskTimer);delay.disable();console.error('Private operations listener failed; check local monitoring configuration.');}); server.listen(port,'127.0.0.1'); return {close(){clearInterval(diskTimer);delay.disable();server.closeAllConnections();server.close();},address:()=>server.address() as AddressInfo|null}; -} \ No newline at end of file +} diff --git a/server/preflight.ts b/server/preflight.ts index b74439a..e945fd3 100755 --- a/server/preflight.ts +++ b/server/preflight.ts @@ -27,7 +27,7 @@ export function launchPreflight(env: NodeJS.ProcessEnv = process.env, now = Date { id: 'public-node-resource-controls', status: 'blocked', detail: 'Stage-1 global RPC budgets, bounded pages/client history and read caching exist. In-memory DAG growth, full-history UI/reconciliation and production-scale load/capacity evidence remain unresolved.' }, { id: 'consensus-launch-calibration', status: 'blocked', detail: 'Stage-3 bounded offline native benchmarks and deterministic retarget/timestamp analysis exist. Representative network hashrate, stochastic/multi-host adversarial evidence, reviewed launch rules/genesis and independent economic-security review remain missing.' }, { id: 'wallet-key-protection', status: 'blocked', detail: 'Stage-2 encrypted vault storage, explicit migration/unlock and lock-aware signing exist. Legacy plaintext copies, unlocked-page compromise, migration/lifecycle coverage and independent production wallet/recovery review remain unresolved.' }, - { id: 'live-market-settlement', status: 'blocked', detail: 'No live SHKeeper adapter or reviewed payout/refund/reorg system exists. Mock payment cannot constitute real payment for valuable cards.' }, + { id: 'live-market-settlement', status: 'blocked', detail: 'Stage-5 offline SHKeeper wire/settlement prototypes and fake-processor failure tests exist. No live transport, production journal/UI/backup integration, reviewed confirmation/refund/custody policy or payment activation is enabled. Mock payment is not real payment.' }, { id: 'release-security-review', status: 'unverified', detail: 'Independent consensus/application review and release-specific dependency/native/container scans require external evidence; an operator environment flag is not evidence.' }, { id: 'deployment-rehearsal', status: 'unverified', detail: 'Stage-4 explicit proxy trust, private diagnostics/checker and stopped-node paired backup/restore with local native replay tests exist. Actual HTTPS/proxy topology, independent hosts/miners, alert delivery, off-host recovery and on-call procedures remain unverified.' }, { id: 'fee-recipient-control', status: 'unverified', detail: 'Confirm the fixed public XMR fee address, checksum and recipient control out of band before any future live payout. No keys or proof are requested here.' }, diff --git a/server/recovery.ts b/server/recovery.ts index 605a88a..412e9b8 100755 --- a/server/recovery.ts +++ b/server/recovery.ts @@ -101,4 +101,4 @@ export async function restoreBackup(directory:string,destination:string){ return {network:NETWORK,chainId:CHAIN_ID,restored:true,proofsRevalidated:false, warning:'SQLite/hash checks only. Replay with the reference engine in an isolated reviewed rehearsal before production use.'}; }); -} \ No newline at end of file +} diff --git a/tests/operations.test.ts b/tests/operations.test.ts index 4ca193c..a3e26e8 100755 --- a/tests/operations.test.ts +++ b/tests/operations.test.ts @@ -46,4 +46,4 @@ test('operations checks identify isolation, stale reconciliation, disk pressure const report=checkOperations(bad,now); for(const warning of ['native-engine-unavailable','queue-pressure','peer-isolation','market-reconciliation-stale','market-operator-review','disk-sample-unavailable-or-stale'])assert.ok(report.warnings.includes(warning)); assert.throws(()=>checkOperations({})); -}); \ No newline at end of file +}); diff --git a/tests/recovery.test.ts b/tests/recovery.test.ts index b0109eb..2fa1147 100755 --- a/tests/recovery.test.ts +++ b/tests/recovery.test.ts @@ -41,4 +41,4 @@ test('backup verifier rejects tampering, sidecars, network mismatch and incomple await assert.rejects(verifyBackup(backup),/hash\/size/); const missing=join(root,'incomplete');await mkdir(missing);await assert.rejects(verifyBackup(missing)); await assert.rejects(access(join(source,'testnet-node.lock'))); // maintenance lock released, not silently left behind -}); \ No newline at end of file +}); diff --git a/tests/shkeeper.test.ts b/tests/shkeeper.test.ts new file mode 100755 index 0000000..96ee655 --- /dev/null +++ b/tests/shkeeper.test.ts @@ -0,0 +1,231 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { randomUUID,generateKeyPairSync,sign } from 'node:crypto'; +import { mkdtemp,rm } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { DatabaseSync } from 'node:sqlite'; +import { keccak } from 'hash-wasm'; +import { ADDITIONAL_MARKET_FEE } from '../shared/market.ts'; +import { NETWORK } from '../shared/network.ts'; +import { CHAIN_ID,address,sha256,transactionId } from '../server/testnet/consensus.ts'; +import { atomicXmr,validateXmrAddress } from '../server/market/xmr.ts'; +import { createLiveProcessor,createSHKeeperHarness,type InvoiceObservation,type PayoutResult,type FixtureTransport } from '../server/market/shkeeper.ts'; +import { SettlementLab,sellerSettlementMessage,buyerInvoiceMessage,type SettlementTerms,type DeliveryEvidence } from '../server/market/settlement-lab.ts'; + +const credentials={apiKey:'ephemeral-fixture-key',username:'fixture-user',password:'fixture-password-not-live'}; +const merchant=ADDITIONAL_MARKET_FEE.address; +async function fixtureAddress(prefix=18){ + const bytes=Uint8Array.from([prefix,...Array(64).fill(2)]),checksum=Buffer.from((await keccak(bytes,256)).slice(0,8),'hex'); + const data=Buffer.concat([bytes,checksum]),alphabet='123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz',sizes:Record={1:2,2:3,3:5,4:6,5:7,6:9,7:10,8:11}; + let result=''; + for(let i=0;isign('sha256',Buffer.from(s),{key:keys.privateKey,dsaEncoding:'ieee-p1363'}).toString('base64')}; +} +const seller=identity(),buyer=identity(); +function fakeProcessor(){ + const invoices=new Map(),tasks=new Map(); + let payoutMode:'success'|'pending'|'failure'|'mismatch'|'lost-response'='success'; + let duplicateTx=false,duplicateTask=false,recalculate=false,failSecond=false,loseInvoice=false; + const posts:{path:string;body:Record}[]=[]; + const transport:FixtureTransport=async(url,init)=>{ + assert.ok(url.startsWith('https://shkeeper.eom.dev/api/v1/'));assert.equal(init.redirect,'error');assert.ok(init.signal); + const headers=init.headers as Record,path=new URL(url).pathname; + const basic=path.includes('/payout')||path.includes('/task/'); + assert.equal(headers.Accept,'application/json'); + if(basic){assert.equal(headers.Authorization,`Basic ${Buffer.from(`${credentials.username}:${credentials.password}`).toString('base64')}`);assert.equal(headers['X-Shkeeper-Api-Key'],undefined);} + else{assert.equal(headers['X-Shkeeper-Api-Key'],credentials.apiKey);assert.equal(headers.Authorization,undefined);} + if(init.method==='POST'){ + const body=JSON.parse(init.body as string);posts.push({path,body}); + if(path.endsWith('/payment_request')){ + assert.equal(body.fiat,'USD');assert.equal(body.amount,'15.00'); + invoices.set(body.external_id,{external_id:body.external_id,fiat:'USD',amount_fiat:'15.00',balance_fiat:'0.00',status:'UNPAID',txs:[]}); + if(loseInvoice)throw new Error('created invoice, response lost'); + return Response.json({status:'success',id:randomUUID(),wallet:merchant,amount:'0.100000000000',exchange_rate:'150',recalculate_after:recalculate?60:0}); + } + assert.equal(body.fee,'2');const id=duplicateTask&&tasks.size?[...tasks.keys()][0]:randomUUID();tasks.set(id,{amount:body.amount,destination:body.destination}); + if(payoutMode==='lost-response')throw new Error(`Sensitive upstream detail ${credentials.password}`); + return Response.json({task_id:id}); + } + if(path.includes('/invoices/'))return Response.json({status:'success',invoices:[invoices.get(path.split('/').at(-1)!)]}); + const id=path.split('/').at(-1)!,task=tasks.get(id);assert.ok(task); + const failure=payoutMode==='failure'||(failSecond&&tasks.size===2&&id===[...tasks.keys()][1]); + const result:PayoutResult=payoutMode==='pending'?{status:'PENDING',result:[]}:{status:failure?'FAILURE':'SUCCESS',result:[{ + amount:payoutMode==='mismatch'?'0.2':task.amount,dest:task.destination,status:failure?'error':'success',txids:[duplicateTx?'f'.repeat(64):sha256(id)]}]}; + return Response.json(result); + }; + return {client:createSHKeeperHarness(transport,credentials),transport,invoices,tasks,posts, + mode:(value:typeof payoutMode)=>{payoutMode=value;},duplicateTx:()=>{duplicateTx=true;},duplicateTask:()=>{duplicateTask=true;},recalculate:()=>{recalculate=true;}, + failFee:()=>{failSecond=true;},loseInvoice:()=>{loseInvoice=true;}}; +} +async function fixture(t:{after(fn:()=>Promise):void}){ + const dir=await mkdtemp(join(tmpdir(),'wikideck-settlement-lab-')),processor=fakeProcessor(); + let now=1_800_000_000_000; + let evidence:DeliveryEvidence={transferId:null,confirmations:0,tip:'a'.repeat(64)}; + let journal=await SettlementLab.open(dir,processor.client,async()=>structuredClone(evidence),()=>now); + t.after(async()=>{await journal.close();await rm(dir,{recursive:true,force:true});}); + const terms:SettlementTerms={id:randomUUID(),network:NETWORK,chainId:CHAIN_ID,cardHash:'b'.repeat(64),sequence:0, + seller:seller.address,buyer:buyer.address,sellerPublicKey:seller.publicKey,buyerPublicKey:buyer.publicKey, + sellerXmr:await fixtureAddress(),priceCents:1500,feeBps:250,expiresAt:now+30*60000,deliveryConfirmations:10}; + const create=()=>journal.create(terms,seller.sign(sellerSettlementMessage(terms))); + const quote=async()=>{await create();return journal.issueInvoice(terms.id);}; + const approved=async()=>{const q=await quote();await journal.approveQuote(terms.id,buyer.sign(buyerInvoiceMessage(q)));return q;}; + const markPaid=()=>{processor.invoices.set(terms.id,{external_id:terms.id,fiat:'USD',amount_fiat:'15.00',balance_fiat:'15.00',status:'PAID', + txs:[{addr:merchant,amount:'0.1',crypto:'XMR',status:'CONFIRMED',txid:'c'.repeat(64)}]});}; + const deliver=(confirmations=10)=>{evidence={transferId:transactionId({network:NETWORK,chainId:CHAIN_ID,cardHash:terms.cardHash,sequence:0,from:seller.address,to:buyer.address,publicKey:'',signature:''}),confirmations,tip:'a'.repeat(64)};}; + return {dir,processor,terms,create,quote,approved,markPaid,deliver,get journal(){return journal;}, + advanceTime:(ms:number)=>{now+=ms;},loseDelivery:()=>{evidence={...evidence,transferId:null,confirmations:0};}, + restart:async()=>{await journal.close();journal=await SettlementLab.open(dir,processor.client,async()=>structuredClone(evidence),()=>now);}}; +} + +test('XMR checksum/network and amount validation reject malformed destinations and floating-point syntax',async()=>{ + assert.equal(await validateXmrAddress(merchant),merchant); + assert.equal(await validateXmrAddress(await fixtureAddress(42)),await fixtureAddress(42)); + await assert.rejects(validateXmrAddress(merchant.slice(0,-1)+'1'),/checksum/); + await assert.rejects(validateXmrAddress(await fixtureAddress(53)),/network/); + await assert.rejects(validateXmrAddress('https://merchant.invalid')); + for(const amount of ['-1','1e-3','0.0000000000001','Infinity',' 1','01'])assert.throws(()=>atomicXmr(amount)); + assert.equal(atomicXmr('0.000000000001'),1n); + assert.throws(()=>createLiveProcessor(),/not implemented or approved/); +}); +test('SHKeeper harness authenticates each documented endpoint and rejects unlocked quote rates',async()=>{ + const f=fakeProcessor(),id=randomUUID();const q=await f.client.createInvoice(id,1500); + assert.equal(q.amountXmr,'0.100000000000');assert.equal((await f.client.invoice(id)).status,'UNPAID'); + const task=await f.client.payout('0.001',merchant);assert.equal((await f.client.task(task)).status,'SUCCESS'); + f.recalculate();await assert.rejects(f.client.createInvoice(randomUUID(),1500)); + assert.equal(f.posts[0].body.callback_url.startsWith('https://market.invalid/'),true); +}); +test('processor responses are bounded, redirects/types/errors are refused without credential leakage',async()=>{ + for(const response of [new Response('{}',{status:302,headers:{'Content-Type':'application/json'}}),new Response(''), + new Response('{}',{headers:{'Content-Type':'application/json','Content-Length':'9999999'}}), + new Response('x'.repeat(256*1024+1),{headers:{'Content-Type':'application/json'}})]){ + const client=createSHKeeperHarness(async()=>response,credentials); + await assert.rejects(client.invoice(randomUUID()),error=>error instanceof Error&&!error.message.includes(credentials.password)); + } + const failing=createSHKeeperHarness(async()=>{throw new Error(credentials.apiKey+credentials.password);},credentials); + await assert.rejects(failing.payout('0.1',merchant),error=>error instanceof Error&&!error.message.includes(credentials.password)&&!error.message.includes(credentials.apiKey)); +}); +test('seller destination and buyer exact invoice quote require independent signatures',async t=>{ + const f=await fixture(t); + await assert.rejects(f.journal.create({...f.terms,sellerXmr:merchant},seller.sign(sellerSettlementMessage(f.terms))),/seller terms/); + const q=await f.quote();assert.equal(q.seller?.amount,'0.096500000000');assert.equal(q.fee?.amount,'0.001000000000'); + const altered=structuredClone(q);altered.quote!.amountXmr='0.2'; + await assert.rejects(f.journal.approveQuote(f.terms.id,buyer.sign(buyerInvoiceMessage(altered))),/buyer invoice/); + await assert.rejects(f.journal.approveQuote(f.terms.id,seller.sign(buyerInvoiceMessage(q))),/buyer invoice/); + assert.equal((await f.journal.approveQuote(f.terms.id,buyer.sign(buyerInvoiceMessage(q)))).status,'awaiting-payment'); + assert.equal(f.processor.posts.filter(p=>p.path.endsWith('/payout')).length,0); +}); +test('confirmed exact payment plus canonical depth settles seller and fixed fee separately once',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id); + f.deliver(9);assert.equal((await f.journal.advance(f.terms.id)).status,'paid');assert.equal(f.processor.tasks.size,0); + f.deliver();const done=await f.journal.advance(f.terms.id); + assert.equal(done.status,'complete');assert.equal(done.seller?.state,'complete');assert.equal(done.fee?.state,'complete'); + const payouts=f.processor.posts.filter(p=>p.path.endsWith('/payout')); + assert.deepEqual(payouts.map(p=>[p.body.destination,p.body.amount]),[[f.terms.sellerXmr,'0.096500000000'],[merchant,'0.001000000000']]); + await f.restart();await f.journal.advance(f.terms.id);assert.equal(f.processor.tasks.size,2); +}); +test('partial/unconfirmed payments wait; overpayments, identity mismatch and late receipt are held',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid(); + const o=f.processor.invoices.get(f.terms.id)!;o.txs[0].status='UNCONFIRMED'; + assert.equal((await f.journal.pollPayment(f.terms.id)).status,'awaiting-payment'); + o.status='PARTIAL';o.txs[0].amount='0.05';o.balance_fiat='7.50'; + assert.equal((await f.journal.pollPayment(f.terms.id)).status,'awaiting-payment'); + f.advanceTime(30*60000); + assert.equal((await f.journal.pollPayment(f.terms.id)).status,'review');assert.equal(f.processor.tasks.size,0); +}); +test('authoritative lookup rejects duplicate/mismatched transactions even with PAID status',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid(); + const o=f.processor.invoices.get(f.terms.id)!;o.txs.push({...o.txs[0]}); + assert.equal((await f.journal.pollPayment(f.terms.id)).status,'review');assert.equal(f.processor.tasks.size,0); +}); +test('processor PAID with overpayment or changed fiat does not authorize delivery settlement',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid(); + f.processor.invoices.get(f.terms.id)!.txs[0].amount='0.11'; + assert.equal((await f.journal.pollPayment(f.terms.id)).status,'review'); +}); +test('payout response loss remains ambiguous across retries/restart; no blind second POST',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver();f.processor.mode('lost-response'); + const failed=await f.journal.advance(f.terms.id);assert.equal(failed.status,'review');assert.equal(failed.seller?.state,'ambiguous');assert.equal(f.processor.tasks.size,1); + await f.journal.advance(f.terms.id);await f.restart();await f.journal.advance(f.terms.id); + assert.equal(f.processor.posts.filter(p=>p.path.endsWith('/payout')).length,1); +}); +test('known pending task is polled after restart without recreating payout',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver();f.processor.mode('pending'); + assert.equal((await f.journal.advance(f.terms.id)).seller?.state,'pending');await f.restart(); + await f.journal.advance(f.terms.id);assert.equal(f.processor.tasks.size,1); + f.processor.mode('success');assert.equal((await f.journal.advance(f.terms.id)).status,'complete');assert.equal(f.processor.tasks.size,2); +}); +test('duplicate payout transaction evidence is held after seller success, never allocated twice',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver(); + // First call: seller task remains pending. Next call confirms seller then submits fee in pending mode. + f.processor.mode('pending');await f.journal.advance(f.terms.id);f.processor.mode('success'); + f.processor.duplicateTx();const result=await f.journal.advance(f.terms.id); + assert.equal(result.status,'review');assert.equal(result.seller?.state,'complete');assert.equal(result.fee?.state,'review'); + await f.restart();await f.journal.advance(f.terms.id);assert.equal(f.processor.tasks.size,2); +}); +test('reorg after payout pauses settlement permanently instead of paying again',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver(); + assert.equal((await f.journal.advance(f.terms.id)).status,'complete');f.loseDelivery(); + assert.equal((await f.journal.advance(f.terms.id)).status,'review');f.deliver();await f.journal.advance(f.terms.id);assert.equal(f.processor.tasks.size,2); +}); +test('persisted submitting intent is quarantined at restart before any transport call',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id); + const s=f.journal.get(f.terms.id);s.status='paying';s.seller!.state='submitting'; + const db=new DatabaseSync(join(f.dir,'settlement-lab.sqlite')); + try{db.prepare('UPDATE lab_orders SET data=? WHERE id=?').run(JSON.stringify(s),f.terms.id);}finally{db.close();} + await f.restart();f.deliver();assert.equal((await f.journal.advance(f.terms.id)).seller?.state,'ambiguous');assert.equal(f.processor.tasks.size,0); +}); +test('seller success followed by fee task failure preserves partial success and refuses retry',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver();f.processor.failFee(); + const result=await f.journal.advance(f.terms.id); + assert.equal(result.status,'review');assert.equal(result.seller?.state,'complete');assert.equal(result.fee?.state,'review'); + await f.restart();await f.journal.advance(f.terms.id);assert.equal(f.processor.tasks.size,2); +}); +test('mismatched payout task amount and reused task IDs require review without replay',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver();f.processor.mode('mismatch'); + assert.equal((await f.journal.advance(f.terms.id)).status,'review');assert.equal(f.processor.tasks.size,1); +}); +test('reused processor task ID across seller and fee legs is quarantined',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver();f.processor.duplicateTask(); + const s=await f.journal.advance(f.terms.id);assert.equal(s.status,'review');assert.equal(s.seller?.state,'complete');assert.equal(s.fee?.state,'ambiguous'); + const calls=f.processor.posts.length;await f.restart();await f.journal.advance(f.terms.id);assert.equal(f.processor.posts.length,calls); +}); +test('invoice creation response loss is held and never repeated automatically',async t=>{ + const f=await fixture(t);await f.create();f.processor.loseInvoice(); + assert.equal((await f.journal.issueInvoice(f.terms.id)).status,'review'); + await f.restart();await f.journal.issueInvoice(f.terms.id); + assert.equal(f.processor.posts.filter(p=>p.path.endsWith('/payment_request')).length,1); +}); +test('changing authoritative paid transaction identity suspends payouts',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver(); + f.processor.invoices.get(f.terms.id)!.txs[0].txid='e'.repeat(64); + assert.equal((await f.journal.advance(f.terms.id)).status,'review');assert.equal(f.processor.tasks.size,0); +}); +test('concurrent settlement advances serialize and cannot duplicate payout POSTs',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id);f.deliver(); + const results=await Promise.all(Array.from({length:8},()=>f.journal.advance(f.terms.id))); + assert.ok(results.every(s=>s.status==='complete')); + assert.equal(f.processor.posts.filter(p=>p.path.endsWith('/payout')).length,2); +}); +test('same incoming transaction cannot fund two separate invoice orders',async t=>{ + const f=await fixture(t);await f.approved();f.markPaid();await f.journal.pollPayment(f.terms.id); + const second={...f.terms,id:randomUUID()}; + await f.journal.create(second,seller.sign(sellerSettlementMessage(second))); + const quote=await f.journal.issueInvoice(second.id);await f.journal.approveQuote(second.id,buyer.sign(buyerInvoiceMessage(quote))); + const observation=structuredClone(f.processor.invoices.get(f.terms.id)!);observation.external_id=second.id; + f.processor.invoices.set(second.id,observation); + assert.equal((await f.journal.pollPayment(second.id)).status,'review');assert.equal(f.processor.tasks.size,0); +}); +test('processor deadline aborts hung fixture transport without returning secret detail', {timeout:10000},async()=>{ + let aborted=false; + const transport:FixtureTransport=async(_url,init)=>new Promise((_resolve,reject)=>{ + init.signal!.addEventListener('abort',()=>{aborted=true;reject(new Error(credentials.password));},{once:true}); + }); + const processor=createSHKeeperHarness(transport,credentials); + await assert.rejects(processor.invoice(randomUUID()),error=>error instanceof Error&&!error.message.includes(credentials.password)); + assert.equal(aborted,true); +});