Improvements
This commit is contained in:
Executable
+51
@@ -0,0 +1,51 @@
|
||||
# Node resource controls — stage 1
|
||||
|
||||
This stage bounds public RPC transport/admission and browser state loading without changing consensus, chain identity, launch approvals or the mock-only payment boundary. **It does not make the node production-ready.**
|
||||
|
||||
## Admission and transport
|
||||
|
||||
API and P2P traffic have separate process-local token buckets. API saturation does not spend P2P tokens. Both lanes still share the event loop, ledger mutation queue, native engine and host resources: this is admission separation, not CPU/process isolation. Budgets are global per process rather than per wallet/IP, so changing wallet or source IP cannot bypass the global limit. Existing per-IP limits remain in place; no permissive proxy trust was enabled.
|
||||
|
||||
Admission precedes JSON parsing, signature verification and route work. Excess traffic receives 503 plus `Retry-After: 1`; there is no unbounded retry queue. JSON request bodies remain capped at 68 KiB and compressed request inflation is disabled. Response JSON is capped before transmission. Byte budget is charged on serialized JSON bytes, not network/TLS framing. Small error responses are not charged recursively when rejecting a payload/budget; edge limits are still needed.
|
||||
|
||||
| Variable | Default | Accepted range |
|
||||
| --- | --- | --- |
|
||||
| `WIKIDECK_RPC_REQUESTS_PER_SECOND` | 120 | 1–1000 |
|
||||
| `WIKIDECK_RPC_WRITES_PER_SECOND` | 10 | 1–100 |
|
||||
| `WIKIDECK_RPC_MAX_ACTIVE` | 32 | 1–256 |
|
||||
| `WIKIDECK_RPC_RESPONSE_BYTES` | 2097152 (2 MiB) | 128 KiB–2 MiB |
|
||||
| `WIKIDECK_RPC_BYTES_PER_SECOND` | 8388608 (8 MiB/s) | 2–128 MiB/s |
|
||||
| `WIKIDECK_RPC_LEGACY_BLOCKS` | 128 | 0–128 |
|
||||
|
||||
Token bursts equal one second of the configured rate. P2P uses a separate fixed lane of 40 requests/s, 10 writes/s and 16 active HTTP responses, with the same configured byte limits. Active slots are released on response finish/connection close; asynchronous route work already admitted may still be completing after disconnection. Existing bounded native, ledger and Wikimedia queues/concurrency remain necessary.
|
||||
|
||||
HTTP has at most 256 accepted connections, 10-second headers timeout, 15-second request timeout, 5-second idle keepalive and a 30-second socket inactivity timeout. These are process transport protections, not a substitute for proxy timeouts/connection limits, global ingress limits, OS limits or load-balanced per-instance policy. Body parsing is no longer ahead of admission. Health bypasses the API lanes for liveness, but still shares server connections and CPU.
|
||||
|
||||
Configuration is validated before opening node resources. Lower limits can shed legitimate traffic, including sync; test settings against the intended topology before deploying. Do not raise limits merely to hide overload. Custom settings require container environment forwarding when deploying with Compose.
|
||||
|
||||
## Bounded state API
|
||||
|
||||
- `GET /api/summary`: network/chain/genesis, canonical tip/height/work, process-local revision, peers/maturity and bounded pending transactions. No full-chain copy. Revision is only a local cache invalidation counter, not consensus or a durable sequence.
|
||||
- `GET /api/canonical?tip=<hash>&offset=0&limit=16`: up to **16 canonical blocks**, pinned to the current tip. Tip changes return 409 so the caller restarts. Offset cannot exceed current height. Each page uses the cached canonical branch and clones only its slice. The first page after a new tip may rebuild that branch in O(height).
|
||||
- Legacy `/api/state` and `/api/chain` still return full snapshots only up to the configured legacy height and response-byte cap. Beyond it, they return 413 with paging instructions **before full snapshot cloning**. They never silently truncate ownership history. Legacy scripts/API clients must migrate to paging for longer chains.
|
||||
- `/p2p/status` and `/p2p/mempool` now obtain only the fields needed instead of cloning the chain. P2P block inventory semantics are unchanged.
|
||||
|
||||
The browser and trade signing refresh use bounded streaming response reads (2 MiB/page), pagination/network/tip/parent continuity checks, one restart on tip conflict, shared in-flight loads, and append-history cache reuse. A final summary confirms the loaded tip. These checks are **not independent PoW/consensus verification**: the UI still trusts the node.
|
||||
|
||||
The existing snapshot-based UI still assembles complete ownership history, capped at **10,000 blocks or 64 MiB of encoded history** with a 30-second fetch deadline. If exceeded, it fails explicitly rather than presenting partial ownership as complete. That is a client resource guard, not a solution for an archival-sized wallet/explorer. Full state objects and parsed JS memory cost more than encoded JSON. A truly indexed wallet and fully paged explorer remain required before closing SEC-01. An unchanged refresh requests two summaries and no block pages; append loading verifies the cached prefix and requests only new blocks.
|
||||
|
||||
## Marketplace read improvements
|
||||
|
||||
Public listing pages query 25 indexed matching rows for a 24-row page plus next-page detection instead of loading the entire listing table. Actor activity uses indexed seller/buyer expressions. Browse/activity reconciliation results are reused for at most one second while the production ledger tip/revision are unchanged. Any observed ledger revision invalidates reuse. Every state-changing market command still reconciles freshly before validating terms; no cached eligibility authorizes a sale.
|
||||
|
||||
Canonical transaction identities are indexed once per reconciliation pass rather than searched through the full chain for each order. Unchanged rows are not rewritten to SQLite. These reduce repeated work, but **reconciliation still clones/scans canonical history and all journal rows**, ownership checks still scan history, private activity is not paginated, and receipt/journal lifecycle needs further work. The response cap can reject an actor's oversized activity response; it is not silently truncated. These limitations keep SEC-06 open.
|
||||
|
||||
## Tests and safe load tooling
|
||||
|
||||
Final validation: **86 tests passed, 0 failed, 0 skipped** in the Node 24 Docker test image, including real RandomX HTTP integration. TypeScript/Vite build passed. The suite includes deterministic token refill/write/active/byte limits; a held-response 40-request loopback burst (4 admitted, 36 shed); health/P2P-lane availability during API saturation; canonical paging defensive copies/tip conflicts/legacy guards; client append caching, malformed pages/continuity, bounded history, single-flight loading and response size checks; non-recursive oversized-response errors; and paging against the real RandomX HTTP fixture.
|
||||
|
||||
`test:rpc-load` invokes the read-only [local smoke-load script](../scripts/rpc-load.ts). It requires `--consent`, accepts only **literal loopback IP origins** (not remote hosts or DNS names), at most 32 workers and 1,000 total requests. Options: `--node`, `--concurrency`, `--requests`. It reads summary/market endpoints and checks health, with response/time bounds. It never submits a block, bid, payment or transfer. Use only a disposable isolated node: even read traffic consumes CPU/IO. Output includes status counts, bytes and p50/p95 latency; rejected load is visible, not counted as successful capacity.
|
||||
|
||||
Initial smoke measurement on a disposable **empty-chain/empty-market**, loopback-only Docker node limited to **1 CPU / 1 GiB**: 100 read requests, 8 workers, 100 HTTP 200s, 40,348 JSON bytes, 1,837 ms total, p50 99 ms, p95 463 ms, final health HTTP 200. This is a single local run, not a performance guarantee; it does not measure hashing load, a populated journal, long-chain costs or hostile Internet traffic. The deterministic 40-request held-response test separately verifies overload shedding (4 admitted, 36 rejected) and peer/health-lane availability. Final test totals are recorded in the completion report.
|
||||
|
||||
Neither measurement establishes that a realistic long-chain public deployment meets its latency/throughput SLO. Production-scale chain/journal fixtures, slow clients, adversarial payloads, sustained concurrency, disk pressure and multi-host recovery measurements remain outstanding. No launch-readiness finding is marked resolved by this stage.
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
**Current decision: HOLD / NO-GO.** Review [the security findings](security-review.md) before scheduling production. The automated preflight intentionally exits nonzero while known production blockers remain. An environment variable, successful container startup or genesis timestamp cannot substitute for closing these findings.
|
||||
|
||||
[Stage-1 node resource controls and local load tooling](node-resources.md) are implemented. This is partial remediation, not capacity certification or closure of the public-node hold point.
|
||||
|
||||
## 1. Safe inspection — does not launch anything
|
||||
|
||||
Use Node.js 24 and a clean reviewed checkout. For mainnet assessment on a host, select `WIKIDECK_NETWORK=mainnet` in the process environment, then run the `launch:preflight` package script. It runs the node entry point with `--launch-preflight`, prints JSON and exits. `--network-manifest` remains available for identity inspection only.
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
## Decision: NO-GO for public production or valuable assets
|
||||
|
||||
**Stage-1 follow-up:** [Node resource controls](node-resources.md) adds global admission/byte budgets, bounded canonical pages, browser cache/history limits and marketplace read improvements. SEC-01 and SEC-06 remain open: full-history client/reconciliation, DAG/disk growth and production-scale capacity evidence are not solved. Original source line references below describe review-time locations and may shift as remediation proceeds.
|
||||
|
||||
This is an internal, source-assisted review with automated regression and dependency checks, **not an independent security audit, penetration test, formal consensus proof, or production certification**. The candidate has not been launched and no approval/attestation was set. Do not interpret passing tests, the existence of a mainnet profile, or the arrival of its candidate genesis date as permission to launch.
|
||||
|
||||
Scope: current consensus/ledger/P2P/RandomX integration, HTTP entry point, browser wallet/backups, marketplace authorization/accounting/reconciliation, Docker configuration and candidate startup gates. The operating hosts, real reverse proxy, public peers, SHKeeper instance, wallets and funds were **not** accessed or assessed. No active Internet-facing adversarial load test or independent native-code audit was performed.
|
||||
|
||||
@@ -79,6 +79,7 @@ Sync polls every **5 seconds**, checks network/chain ID, imports at most **32**
|
||||
| --- | --- |
|
||||
| `GET /healthz` | Process health/network, not sync or consensus security. |
|
||||
| `GET /api/state`, `/api/chain` | Canonical blocks, mined/pending transactions, tip/work and peer count. |
|
||||
| `GET /api/summary`, `/api/canonical?tip=<hash>&offset=0&limit=16` | Bounded state metadata and current-tip-pinned canonical pages. Full legacy snapshots are limited to 128 blocks and the response-byte budget. See [resource controls](node-resources.md). |
|
||||
| `GET /api/network` | Selected profile, pinned chain/genesis, candidate launch status and mock-payment boundary. |
|
||||
| `GET /api/market?offset=0`, `POST /api/market/command` | Node-local mock-payment marketplace; strict signed listing, bid, checkout, delivery and private activity commands. Not consensus escrow. See [marketplace operations](marketplace.md). |
|
||||
| `GET /api/nfts/:id`, `/api/peers` | Canonical NFT owner/sequence/confirmations, article and at-mint statistics; configured peer status. |
|
||||
|
||||
Reference in New Issue
Block a user