wikideck/docs/testnet-consensus.md

Experimental testnet consensus

Implementation reference for the current tree, not a public-mainnet specification or security certification. See README for setup/wallet commands and native-engine details for RandomX tooling. Consensus code: shared serialization, validation, DAG/persistence, peer transport, HTTP, miner.

This document is for operators and developers who need the exact current protocol rules. It is not a quick-start guide: consensus serialization and constants are implementation-specific, and changing them can create a different chain.

Contents

• Network, genesis and article selection
• Header, work and three distinct identities
• Targets, timestamps and epoch boundaries
• Native NFT transition and commitments
• Fork choice, reorgs and storage
• Fixed peers and HTTP surface

Network, genesis and article selection

• Network: wikideck-randomx-testnet-4, block version 4. Genesis has no blocks, NFTs or work; first mined height is 1. No privileged issuance/admin mint key. Version 4 has a new chain identity; prior-version ledger files and wallet backups are not compatible.
• GENESIS_TIME = 1792411200 (Unix seconds). Let S(x) mean lowercase hex SHA-256 of UTF-8 x, and J(x) mean compact JavaScript JSON.stringify(x); field order below is consensus-critical.
• MAX_TARGET is 0fff followed by 60 f digits. CHAIN_ID = S(J([NETWORK, CONSENSUS, GENESIS_TIME, MAX_TARGET])), using the constants object's insertion order in the implementation. CONSENSUS identifies the random Wikipedia source and mint-statistics snapshot format.
• GENESIS_HASH = S(NETWORK + ':genesis:' + CHAIN_ID); INITIAL_KEY = S(NETWORK + ':initial-key:' + CHAIN_ID). Nodes independently derive the same manifest and reject other network/chain IDs.
• Starter samples remain only as interface previews; they are not eligible for minting. Each mining-template request asks the fixed English Wikipedia API endpoint for up to ten random namespace-0 pages, with a plain-text extract and thumbnail. It makes at most three requests and returns a page with a usable extract and an HTTPS image on upload.wikimedia.org or thumb.wikimedia.org. A page may be minted repeatedly. There is no static fallback: if no pictured article or its statistics are available, template creation fails and the miner can retry.
• The selected Article snapshot and statsSnapshot are copied into the block header and covered by the miner's block signature and block ID. articleTuple = [pageId, revisionId, title, extract, image-or-empty-string, url, source, category]. The at-mint statistics contain capture time, metadata revision, page bytes, language-link count/truncation, six-month pageviews when available, and the derived Attention/Depth/Reach scores. Missing pageviews are represented as unavailable, never invented zeroes. The node forces a fresh enrichment request for this snapshot; it is captured when the mining template is prepared, since changing it after work begins would invalidate the proof. Current UI statistics are fetched separately and may reflect later edits.
• A repeated page still creates a distinct NFT: token identity includes the parent block, height, and owner. Validators check article/image and stats schemas, the page URL, Wikimedia HTTPS host, pageview summaries, derived scores, and block commitments, but do not constrain repeated page IDs.
• Validators do not contact Wikipedia during consensus validation. They verify the committed snapshot's structure and internal metric summaries, not the API provenance or ongoing existence of the page/image. A miner can construct a syntactically valid snapshot itself; this experimental design avoids making block acceptance dependent on an external service being online. The node's template endpoint fetches the article and fresh statistics for ordinary mining clients.

Header, work and three distinct identities

headerMessage(h) is J([NETWORK, version, chainId, height, previousHash, createdAt, articleTuple(article), statsSnapshotTuple, owner, publicKey, target, key, transactionsRoot, stateRoot]). statsSnapshotTuple commits capture time, revision, size/language counts, ordered monthly views and summary values, plus the three derived scores. It excludes the nonce. Hash its UTF-8 bytes with SHA-256 to obtain 32 raw bytes, then append the nonce encoded as 8 unsigned little-endian bytes. The resulting 40-byte mining blob has nonce offset 32 (not a 32-bit nonce). Node's numeric interface restricts nonces to nonnegative safe integers; the native line protocol supports uint64.

Identifier	Exact construction / meaning
Block ID (hash)	SHA-256 of the raw mining blob, lowercase hex. Does not include ECDSA signature bytes or the separate powHash field.
RandomX proof (powHash)	Native RandomX of the mining blob using the 32-byte epoch key. Determines work eligibility and game rarity/stats, not the NFT ID.
NFT ID (tokenId)	S(J([NETWORK, CHAIN_ID, 'mint', previousHash, height, owner, article.pageId, statsSnapshotTuple])). The capture time and page statistics help distinguish repeated mints of the same page. Independent of nonce/proof/signature, avoiding a circular dependency through state commitment.

Interpret RandomX output and target as unsigned big-endian integers; accept powHash <= target including equality. This differs from Monero's target convention. The initial/easiest threshold accepts exactly 1/16 of uniformly distributed hashes (expected 16 attempts). No fake or browser PoW is accepted by the production node.

The miner signs J([NETWORK, 'block-claim', blockID]) using ECDSA P-256/SHA-256, IEEE-P1363 encoding. Address is SHA-256 of canonical SPKI DER public-key bytes. Nodes check public-key/address consistency, signature, body schema, exact parent/height, article, target, key, roots, mint ID, block ID and recomputed native proof. Signatures remain required even though their bytes are excluded from block identity.

Targets, timestamps and epoch boundaries

• Desired spacing: 30 seconds; retarget interval: 10 ancestors. Heights 1–10 use the initial target; height 11 first retargets. Otherwise inherit the parent target.
• At each retarget, compare the last and first timestamps of the previous 10 blocks: expected span is 9 × 30 = 270 seconds. Clamp elapsed to floor(270/4) through 270*4, multiply the parent target by elapsed/270 with integer division, then clamp target to [1, MAX_TARGET]. Selection uses work, not height or wall-clock estimates.
• A timestamp must exceed the median of up to 11 ancestors (sorted upper-middle for even counts; genesis time if none) and must be at most 120 seconds ahead of validation time. Keep operator clocks synchronized.
• Epoch key source height is floor(max(0, height - 64) / 2048) * 2048. Zero means INITIAL_KEY; otherwise use the ancestor block hash at that source height. Thus heights 1–2111 use the initial key, 2112–4159 use height 2048's hash, and height 4160 switches to height 4096's hash. Forks derive keys from their own ancestors.

Native NFT transition and commitments

Each block executes its ordered transfers against its parent's NFT state, then mints one NFT to the block owner. State entries contain owner, sequence (initially 0), minted height and page ID. Transfers cannot spend this block's new issuance.

A transfer carries network, chain ID, cardHash (NFT token ID, despite the legacy field name), from, to, sequence, public key and signature. Its signed message is J([NETWORK, 'nft-transfer', chainId, cardHash, from, to, sequence]). Transfer ID is SHA-256 of that message, excluding ECDSA signature bytes to avoid signature-malleability identity changes. Require the current owner, matching public key, a different recipient, exact current sequence, valid signature, and inclusion height minus mint height >= 2; increment sequence on application. Replays and duplicate IDs within a block fail.

transactionsRoot commits the ordered transfer IDs. stateRoot commits the complete post-transfer, post-mint state: sort token IDs lexicographically; each input leaf is S(J([id, owner, sequence, mintedHeight, pageId])). merkleRoot returns S(NETWORK + ':empty') for no leaves. For nonempty input, hash each raw 32-byte input leaf with byte prefix 0x00; hash pairs of raw children with byte prefix 0x01, duplicating an odd final child at each layer. Finally return S(J([NETWORK, 'merkle', originalLeafCount, treeRootHex])). Count commitment prevents odd-padding ambiguity; leaf/node prefixes separate domains. These commitment semantics, the RandomX version, and big-endian proof convention are pinned in the genesis consensus manifest.

The mempool holds at most 256 ordered transfers, validating sequentially against the next inclusion height; a template takes the first 64. The protocol can accept chained transfers of the same NFT in order; the browser conservatively blocks a second pending transfer for a card. A 202 response changes no canonical ownership. Mint at height h is eligible for inclusion at h+2 (pool admission can happen at tip h+1). Confirmations are tipHeight - mintedHeight + 1, not a finality guarantee.

Fork choice, reorgs and storage

Per-block work is floor(2^256 / (target + 1)); accumulate from genesis work zero. Keep all valid known-parent branches. Adopt a tip only for strictly greater cumulative work; equal-work ties retain the current tip, including across replay of persisted insertion order. A shorter, harder branch can win.

On reorg, reconstruct ownership from the winning branch. Orphan mints disappear from canonical ownership; disconnected transfers are reconsidered before existing pending entries, in order, only if still valid and not already committed. Conflicting, immature or orphan-issuance transfers are discarded. No checkpoints/finality or rollback protection beyond observed work; an isolated node has no peer-backed security.

The ledger stores the entire DAG and pending pool in data/testnet/ledger.json, under the WIKIDECK_DATA_DIR testnet subdirectory. The file is created when the node first writes ledger data. Defaults use project data; Docker uses /app/data. Disk caps: 128 MiB, 10,000 total blocks including side branches; blocks: 64 KiB and 64 transfers. Each in-memory DAG node stores a full NFT-state snapshot: potentially quadratic memory growth, plus full-file rewrite cost. These are experimental stop limits, not scalable storage or a public DoS defense.

Mutations are serialized (queue cap 256); persist via a synchronized temporary file and atomic rename before exposing new state. Restart independently revalidates every saved proof and transaction; malformed, oversized or mismatched files fail closed without automatic reset. HTTP startup exclusively creates a data-directory lock named testnet-node.lock. It is removed on normal shutdown, not automatically after a crash: stop and verify all owning processes have exited before manual stale-lock removal. Never bypass a live lock or share a writer directory. Back up stopped-node volumes/files and wallet exports separately; destructive volume reset cannot recover assets from a wallet alone.

Fixed peers and HTTP surface

Peers are explicit operator-configured URL origins, up to 16, not discovered from network input. Default: HTTPS/public addresses only, checked and pinned during DNS lookup; no credentials, paths, queries, fragments or redirects. WIKIDECK_ALLOW_PRIVATE_PEERS=1 permits private/HTTP destinations only by explicit operator choice (used for local Docker tests). No bootnodes, DHT, automatic public deployment or claimed Sybil resistance.

Sync polls every 5 seconds, checks network/chain ID, imports at most 32 topologically ordered DAG blocks per peer/pass, and validates remote pool entries locally. Accepted blocks/transfers are gossiped to available configured peers; busy-peer gossip is dropped, with polling for catch-up. Responses are capped at 2 MiB with a 10-second deadline. Peer count is current outbound connectivity metadata, not consensus membership or a security score.

Route	Purpose
GET /healthz	Process health/network, not sync or consensus security.
GET /api/state, /api/chain	Canonical blocks, mined/pending transfers, tip/work and peer count.
GET /api/nfts/:id, /api/peers	Canonical NFT owner/sequence/confirmations, article and at-mint statistics; configured peer status.
GET /api/wikipedia/:pageId	Cached current enrichment for positive English Wikipedia page IDs: categories, views, size, languages and display-only scores. Never a consensus input.
POST /api/mining/template	Rate-limited public miner identity; fetches a random article with a picture plus fresh at-mint statistics. Returns 503 when either cannot be fetched.
POST /api/blocks	Independently validates a signed block, proof, article snapshot, required Wikimedia image URL, and statistics snapshot consistency. Repeated page IDs are accepted.
POST /api/transfer	Signed transfer admission; 202 means pending.
GET /p2p/status, /p2p/blocks?offset=0, /p2p/mempool	Manifest/size/tip, 32-block inventory pages, pending transactions.
POST /p2p/block, /p2p/transaction	Same validation as local block/transfer submission.

POSTs require JSON, enforce supplied browser origins and signatures where applicable; peers without an Origin header are not authenticated by this check. HTTP parsing is capped at 68 KiB; API/P2P rate limits are 300/minute and block/transfer POSTs additionally 30/minute. There are no fees, peer authentication or economic spam deterrents. Same-origin browser RPC trusts the node; it does not independently verify these commitments. Deploy no funds: majority-hashpower, eclipse, censorship, reorg and resource-exhaustion risks require independent audits and adversarial testing before any broader launch.