WikiDeck/wikideck
3
0
Fork 1
Code Issues 8 Pull Requests Actions Packages Projects 2 Releases Wiki Activity

Extended benchmark

Browse Source
This commit is contained in:
eric
2026-10-21 10:14:34 -04:00
parent 687b628424
commit 6adfda3cf0
3 changed files with 39 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/canonical-indexes.md
+1 -1
View File
@@ -46,7 +46,7 @@ Observed in the development Docker environment:
These numbers are a reproducible **baseline**, not acceptance SLOs: host CPU/storage and production concurrency were not selected, outputs include synthetic article payloads, warm caches apply, one transaction amortizes insertion fsync, and real validation/startup/HTTP/market workloads are excluded. They do not establish a supported production envelope.
The synthetic market-journal harness [benchmark](../scripts/benchmark-market-reconciliation.ts), available as `npm run benchmark:market -- 10000`, separately measures bounded reconciliation of terminal order/listing records in its own temporary SQLite directory. On the same development Docker environment, 10,000 orders plus 10,000 listings completed in 157 reconciliation passes in 772 ms (about 25,903 records/s), with pass-latency p50/p95/max of 4.57/9.13/20.04 ms and a 7,180,288-byte SQLite file. This dataset intentionally avoids active-order delivery and listing validation branches; it measures local keyset/SQLite overhead only. It is not a target-hardware result or throughput/SLO claim and excludes concurrent HTTP traffic, event-loop interference from other work, PostgreSQL, chain validation and adversarial workloads.
The synthetic market-journal harness [benchmark](../scripts/benchmark-market-reconciliation.ts), available as `npm run benchmark:market -- 10000 [terminal|confirmed-delivery]`, measures two bounded reconciliation paths in its own temporary SQLite directory. On the same development Docker environment, 10,000 terminal orders plus 10,000 listings completed in 776 ms (about 25,770 records/s), with pass-latency p50/p95/max of 4.56/8.03/17.73 ms, sampled event-loop-delay p95/max of 16.70/19.55 ms at 10ms resolution, and an 11,927,552-byte SQLite file. The synthetic confirmed-delivery scenario transitioned 10,000 paid orders and reserved listings to complete/sold in 2,031 ms (about 9,846 records/s), with pass-latency p50/p95/max of 12.27/24.75/35.59 ms, sampled event-loop-delay p95/max of 28.77/48.46 ms, and a 12,832,768-byte SQLite file. Its canonical transaction IDs come from an in-memory adapter fixture, not validated Ledger evidence. The event-loop samples demonstrate synchronous batch stalls locally; yielding between benchmark passes allows the monitor to sample but does not offload production work. These are not target-hardware throughput/SLO claims and exclude concurrent HTTP traffic, other server work, PostgreSQL, actual chain validation and adversarial workloads.
## Remaining gates
docs/production-plan.md
+1 -1
View File
@@ -63,7 +63,7 @@ Acceptance: exact-amount property/boundary tests, versioned signature tests, zer
Can run alongside P0/P1. Addresses SEC-01/SEC-06 rather than adding more fixed caps.
**Implemented M2 increments, 2026-10-20–21:** [canonical indexes and durable market observation](canonical-indexes.md). Canonical ownership/transaction SQL indexes now serve NFT, deck, custody availability and mock-market queries; per-block undo and anchored connect/disconnect events commit with ledger updates. Market checkpoint/reorg holds persist atomically and bounded catch-up blocks new mutations. Order/listing reconciliation now advances in durable, stable-ID keyset batches of at most 128 records per table, resumes after restart, re-fences when the chain tip/revision changes, and runs from an HTTP lifecycle poller rather than relying only on page loads. Final TypeScript/Vite build and **174 offline tests passed**; focused market tests passed 23/23 and the disposable PostgreSQL integration suite passed 31/31. A new synthetic local benchmark processed 10,000 terminal orders plus 10,000 listings in 772 ms on the development Docker host; this is a reproducible harness result, **not** a target-hardware SLO or complete M2 acceptance. Queue isolation, target-scale/concurrent throughput and latency, production financial storage/recovery, retained-DAG capacity and operator dataset/SLO targets remain open.
**Implemented M2 increments, 2026-10-20–21:** [canonical indexes and durable market observation](canonical-indexes.md). Canonical ownership/transaction SQL indexes now serve NFT, deck, custody availability and mock-market queries; per-block undo and anchored connect/disconnect events commit with ledger updates. Market checkpoint/reorg holds persist atomically and bounded catch-up blocks new mutations. Order/listing reconciliation now advances in durable, stable-ID keyset batches of at most 128 records per table, resumes after restart, re-fences when the chain tip/revision changes, and runs from an HTTP lifecycle poller rather than relying only on page loads. Final TypeScript/Vite build and **174 offline tests passed**; focused market tests passed 23/23 and the disposable PostgreSQL integration suite passed 31/31. Synthetic 10,000-order/listing runs processed terminal records in 776 ms (event-loop-delay max 19.55 ms) and paid orders with confirmed-delivery adapter fixtures in 2,031 ms (max 48.46 ms); these measurements show synchronous batches can block the local loop, but are not target-hardware SLOs or complete M2 acceptance. Queue isolation, target-scale/concurrent throughput and latency, production financial storage/recovery, retained-DAG capacity and operator dataset/SLO targets remain open.
**Next financial-storage increment:** [ADR 003 — PostgreSQL](postgres-financial-storage.md) selects and integrates the custody SQL backend with existing HTTP callers, exact constrained accounting, explicit migration/runtime roles, source-freezing SQLite import and dump/restore rehearsal. The adapter remains on the fictitious sandbox domain; it is not a production deposit/withdrawal worker. SQLite node-only backup deliberately refuses PostgreSQL-bound financial state. Production hardware/SLO tests, coordinated backup/PITR/failover, real payment schema/worker controls and independent review remain open, so this does not close M2/M4 or authorize funds.
scripts/benchmark-market-reconciliation.ts
+37 -18
View File
@@ -2,27 +2,31 @@ import { DatabaseSync } from 'node:sqlite';
import { mkdtemp, rm, stat } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { performance } from 'node:perf_hooks';
import { monitorEventLoopDelay, performance } from 'node:perf_hooks';
import { randomUUID } from 'node:crypto';
import { setImmediate as yieldToEventLoop } from 'node:timers/promises';
import { Market } from '../server/market/store.ts';
import { GENESIS_HASH } from '../server/testnet/consensus.ts';
import { CHAIN_ID, GENESIS_HASH, transactionId } from '../server/testnet/consensus.ts';
import { NETWORK } from '../shared/testnet.ts';
// Synthetic terminal-order journal only. It is isolated in a temporary SQLite database and
// never connects to a Ledger, peer, payment adapter, wallet, or production data directory.
// Synthetic market journal only. It is isolated in a temporary SQLite database and never
// connects to a Ledger, peer, payment adapter, wallet, or production data directory.
const text = process.argv[2] ?? '10000';
if (!/^[1-9]\d{0,5}$/.test(text) || Number(text) > 100000) throw new Error('Fixture size must be 1..100000 orders');
const count = Number(text);
const count = Number(text), scenario = process.argv[3] ?? 'terminal';
if (!['terminal', 'confirmed-delivery'].includes(scenario)) throw new Error('Scenario must be terminal or confirmed-delivery');
const directory = await mkdtemp(join(tmpdir(), 'wikideck-market-reconciliation-'));
const streamId = randomUUID(), tip = 'a'.repeat(64);
const deliveryRows = new Map<string, { height: number; blockHash: string }>();
const seller = '1'.repeat(64), buyer = '2'.repeat(64);
let market: Market | undefined;
try {
// Indexed-only adapter: the benchmark must not fall back to materializing chain snapshots.
const chain = {
snapshot: () => { throw new Error('Benchmark must not request a full chain snapshot'); },
summary: () => ({ tip, revision: 0, height: 1 }),
summary: () => ({ tip, revision: 0, height: 2 }),
nft: () => undefined,
canonicalTransaction: () => undefined,
canonicalTransaction: (id: string) => deliveryRows.get(id),
pendingTransactions: () => [],
canonicalHead: () => 0,
canonicalEvents: (after: number) => ({ streamId, previousAnchor: GENESIS_HASH, anchor: GENESIS_HASH,
@@ -34,15 +38,23 @@ try {
try {
const insertListing = db.prepare('INSERT INTO listings(id,card,status,data) VALUES(?,?,?,?)');
const insertOrder = db.prepare('INSERT INTO orders(id,data) VALUES(?,?)');
const seller = 'synthetic-seller';
db.exec('BEGIN IMMEDIATE');
try {
for (let i = 0; i < count; i++) {
const id = i.toString().padStart(12, '0');
const listingId = `listing-${id}`, orderId = `order-${id}`, cardHash = i.toString(16).padStart(64, '0');
const listing = { id: listingId, cardHash, seller, sequence: 0, kind: 'fixed', status: 'cancelled' };
const order = { id: orderId, listingId, seller, buyer: 'synthetic-buyer', cardHash, sequence: 0, status: 'refunded' };
insertListing.run(listingId, cardHash, 'cancelled', JSON.stringify(listing));
const terminal = scenario === 'terminal';
const listingStatus = terminal ? 'cancelled' : 'reserved';
const listing = { id: listingId, cardHash, seller, sequence: 0, kind: 'fixed', status: listingStatus };
const order = { id: orderId, listingId, seller, buyer, cardHash, sequence: 0,
status: terminal ? 'refunded' : 'paid', payout: null, deliveryConfirmations: 0,
requiredConfirmations: 2, createdAt: 1, payBy: 2, deliverBy: 3 };
if (!terminal) {
const delivery = { network: NETWORK, chainId: CHAIN_ID, cardHash, from: seller, to: buyer,
sequence: 0, publicKey: '', signature: '' };
deliveryRows.set(transactionId(delivery), { height: 1, blockHash: tip });
}
insertListing.run(listingId, cardHash, listingStatus, JSON.stringify(listing));
insertOrder.run(orderId, JSON.stringify(order));
}
db.exec('COMMIT');
@@ -51,23 +63,30 @@ try {
const latencies: number[] = [];
const maxPasses = Math.ceil(count / 128) * 2 + 4;
const eventLoopDelay = monitorEventLoopDelay({ resolution: 10 });
eventLoopDelay.enable();
const start = performance.now();
for (let pass = 0; pass < maxPasses && (pass === 0 || market.hasReconciliationBacklog()); pass++) {
const before = performance.now();
market.browse();
latencies.push(performance.now() - before);
}
try {
for (let pass = 0; pass < maxPasses && (pass === 0 || market.hasReconciliationBacklog()); pass++) {
const before = performance.now();
market.browse();
latencies.push(performance.now() - before);
await yieldToEventLoop(); // Permit the delay monitor to observe synchronous batch stalls.
}
} finally { eventLoopDelay.disable(); }
const elapsedMs = performance.now() - start;
if (market.hasReconciliationBacklog()) throw new Error(`Reconciliation did not finish within ${maxPasses} bounded passes`);
market.close(); market = undefined;
const bytes = (await stat(join(directory, NETWORK.includes('mainnet') ? 'mainnet' : 'testnet', 'market.sqlite'))).size;
latencies.sort((a, b) => a - b);
const percentile = (p: number) => latencies[Math.min(latencies.length - 1, Math.ceil(latencies.length * p) - 1)];
console.log(JSON.stringify({ fixture: 'synthetic-terminal-orders-and-listings', orders: count, listings: count,
console.log(JSON.stringify({ fixture: `synthetic-${scenario}-orders-and-listings`, orders: count, listings: count,
batchSizePerTable: 128, reconciliationPasses: latencies.length, elapsedMs,
rowsPerSecond: Math.round((count * 2 * 1000) / elapsedMs), passLatencyP50Ms: percentile(0.50),
passLatencyP95Ms: percentile(0.95), passLatencyMaxMs: latencies.at(-1), sqliteBytes: bytes,
warning: 'Synthetic terminal records on this development host only; excludes live chain validation, active-order logic, concurrent HTTP load, PostgreSQL, and production SLO certification.' }, null, 2));
eventLoopDelayResolutionMs: 10, eventLoopDelayP95Ms: eventLoopDelay.percentile(95) / 1e6,
eventLoopDelayMaxMs: eventLoopDelay.max / 1e6,
warning: 'Synthetic fixture on this development host only; canonical deliveries are adapter fixtures, not Ledger evidence. Excludes concurrent HTTP load, PostgreSQL, and production SLO certification.' }, null, 2));
} finally {
market?.close();
await rm(directory, { recursive: true, force: true });