wikideck/docs/testnet-consensus.md

Experimental testnet consensus

Implementation reference for the current tree, not a public-mainnet specification or security certification. See README for setup/wallet commands and native-engine details for RandomX tooling. Consensus code: shared serialization, validation, DAG/persistence, peer transport, HTTP, miner.

This document is for operators and developers who need the exact current protocol rules. It is not a quick-start guide: consensus serialization and constants are implementation-specific, and changing them can create a different chain.

Contents

• Network, genesis and article selection
• Header, work and three distinct identities
• Targets, timestamps and epoch boundaries
• Native NFT transition and commitments
• Fork choice, reorgs and storage
• Fixed peers and HTTP surface

Network, genesis and article selection

• Network: wikideck-randomx-testnet-6, block version 6. Genesis has no blocks, NFTs or work; first mined height is 1. No privileged issuance/admin mint key. Version 6 adds atomic NFT trade bundles and has a new chain identity; prior-version ledger files and wallet backups are not compatible.
• GENESIS_TIME = 1792411200 (Unix seconds). Let S(x) mean lowercase hex SHA-256 of UTF-8 x, and J(x) mean compact JavaScript JSON.stringify(x); field order below is consensus-critical.
• MAX_TARGET is 0fff followed by 60 f digits. CHAIN_ID = S(J([NETWORK, CONSENSUS, GENESIS_TIME, MAX_TARGET])), using the constants object's insertion order in the implementation. CONSENSUS identifies the random Wikipedia source and mint-statistics snapshot format.
• GENESIS_HASH = S(NETWORK + ':genesis:' + CHAIN_ID); INITIAL_KEY = S(NETWORK + ':initial-key:' + CHAIN_ID). Nodes independently derive the same manifest and reject other network/chain IDs.
• Starter samples remain only as interface previews; they are not eligible for minting. Each mining-template request asks the fixed English Wikipedia API endpoint for up to ten random namespace-0 pages, with a plain-text extract and thumbnail. It makes at most three requests and returns a page with a usable extract and an HTTPS image on upload.wikimedia.org or thumb.wikimedia.org. A page may be minted repeatedly. There is no static fallback: if no pictured article or its statistics are available, template creation fails and the miner can retry.
• The selected Article snapshot and statsSnapshot are copied into the block header and covered by the miner's block signature and block ID. articleTuple = [pageId, revisionId, title, extract, image-or-empty-string, url, source, category]. The at-mint statistics contain capture time, metadata revision, page bytes, language-link count/truncation, six-month pageviews when available, and the derived Attention/Depth/Reach scores. Missing pageviews are represented as unavailable, never invented zeroes. The node forces a fresh enrichment request for this snapshot; it is captured when the mining template is prepared, since changing it after work begins would invalidate the proof. Current UI statistics are fetched separately and may reflect later edits.
• A repeated page still creates a distinct NFT: token identity includes the parent block, height, and owner. Validators check article/image and stats schemas, the page URL, Wikimedia HTTPS host, pageview summaries, derived scores, and block commitments, but do not constrain repeated page IDs.
• Validators do not contact Wikipedia during consensus validation. They verify the committed snapshot's structure and internal metric summaries, not the API provenance or ongoing existence of the page/image. A miner can construct a syntactically valid snapshot itself; this experimental design avoids making block acceptance dependent on an external service being online. The node's template endpoint fetches the article and fresh statistics for ordinary mining clients.

Header, work and three distinct identities

headerMessage(h) is J([NETWORK, version, chainId, height, previousHash, createdAt, articleTuple(article), statsSnapshotTuple, owner, publicKey, target, key, transactionsRoot, stateRoot]). statsSnapshotTuple commits capture time, revision, size/language counts, ordered monthly views and summary values, plus the three derived scores. It excludes the nonce. Hash its UTF-8 bytes with SHA-256 to obtain 32 raw bytes, then append the nonce encoded as 8 unsigned little-endian bytes. The resulting 40-byte mining blob has nonce offset 32 (not a 32-bit nonce). Node's numeric interface restricts nonces to nonnegative safe integers; the native line protocol supports uint64.

Identifier	Exact construction / meaning
Block ID (hash)	SHA-256 of the raw mining blob, lowercase hex. Does not include ECDSA signature bytes or the separate powHash field.
RandomX proof (powHash)	Native RandomX of the mining blob using the 32-byte epoch key. Determines work eligibility and game rarity/stats, not the NFT ID.
NFT ID (tokenId)	S(J([NETWORK, CHAIN_ID, 'mint', previousHash, height, owner, article.pageId, statsSnapshotTuple])). The capture time and page statistics help distinguish repeated mints of the same page. Independent of nonce/proof/signature, avoiding a circular dependency through state commitment.

Interpret RandomX output and target as unsigned big-endian integers; accept powHash <= target including equality. This differs from Monero's target convention. The initial/easiest threshold accepts exactly 1/16 of uniformly distributed hashes (expected 16 attempts). No fake or browser PoW is accepted by the production node.

The miner signs J([NETWORK, 'block-claim', blockID]) using ECDSA P-256/SHA-256, IEEE-P1363 encoding. Address is SHA-256 of canonical SPKI DER public-key bytes. Nodes check public-key/address consistency, signature, body schema, exact parent/height, article, target, key, roots, mint ID, block ID and recomputed native proof. Signatures remain required even though their bytes are excluded from block identity.

Targets, timestamps and epoch boundaries

• Desired spacing: 15 minutes (900 seconds); retarget interval: 10 ancestors. Heights 1–10 use the initial target; height 11 first retargets. Otherwise inherit the parent target.
• At each retarget, compare the last and first timestamps of the previous 10 blocks: expected span is 9 × 900 = 8,100 seconds. Clamp elapsed to floor(8,100/4) = 2,025 through 8,100*4 = 32,400 seconds, multiply the parent target by elapsed/8,100 with integer division, then clamp target to [1, MAX_TARGET]. Selection uses work, not height or wall-clock estimates.
• A timestamp must exceed the median of up to 11 ancestors (sorted upper-middle for even counts; genesis time if none) and must be at most 120 seconds ahead of validation time. Keep operator clocks synchronized.
• Epoch key source height is floor(max(0, height - 64) / 2048) * 2048. Zero means INITIAL_KEY; otherwise use the ancestor block hash at that source height. Thus heights 1–2111 use the initial key, 2112–4159 use height 2048's hash, and height 4160 switches to height 4096's hash. Forks derive keys from their own ancestors.

Native NFT transition and commitments

Each block executes its ordered transactions against its parent's NFT state, then mints one NFT to the block owner. State entries contain owner, sequence (initially 0), minted height and page ID. Transactions cannot spend this block's new issuance.

An individual transfer carries network, chain ID, cardHash (NFT token ID, despite the legacy field name), from, to, sequence, public key and signature. Its signed message is J([NETWORK, 'nft-transfer', chainId, cardHash, from, to, sequence]). Transfer ID is SHA-256 of that message, excluding ECDSA signature bytes to avoid signature-malleability identity changes. Require the current owner, matching public key, a different recipient, exact current sequence, valid signature, and inclusion height minus mint height >= 2; increment sequence on application. Replays and duplicate IDs within a block fail.

A trade is one type: 'trade' transaction with proposer and counterparty addresses, ordered offered/requested NFT legs (token ID plus current sequence), and one P-256 signature from each party over J([NETWORK, 'nft-trade', chainId, proposer, counterparty, offeredLegs, requestedLegs]), where each leg is [cardHash, sequence]. Both public keys must derive the respective party addresses. Each side must contain 1–8 distinct NFTs; duplicates across either side are rejected. At inclusion, validation checks all selected NFTs against the same pre-trade state, owners, sequences, maturity, and both signatures before writing any changes. If any leg is invalid, the entire trade is rejected; if valid, all offered ownerships move to the counterparty and all requested ownerships to the proposer, with each sequence incremented once. A trade counts as one mempool and block transaction and is committed by one transaction ID. It cannot be partially included or partially replayed. Offers are not reservations; stale offers fail validation.

transactionsRoot commits the ordered transaction IDs (individual transfer IDs or atomic trade IDs). stateRoot commits the complete post-transaction, post-mint state: sort token IDs lexicographically; each input leaf is S(J([id, owner, sequence, mintedHeight, pageId])). merkleRoot returns S(NETWORK + ':empty') for no leaves. For nonempty input, hash each raw 32-byte input leaf with byte prefix 0x00; hash pairs of raw children with byte prefix 0x01, duplicating an odd final child at each layer. Finally return S(J([NETWORK, 'merkle', originalLeafCount, treeRootHex])). Count commitment prevents odd-padding ambiguity; leaf/node prefixes separate domains. These commitment semantics, the RandomX version, and big-endian proof convention are pinned in the genesis consensus manifest.

The mempool holds at most 256 ordered transactions (transfers or trade bundles), validating sequentially against the next inclusion height; a template takes the first 64 transaction units. A 202 response changes no canonical ownership. Mint at height h is eligible for inclusion at h+2 (pool admission can happen at tip h+1). Confirmations are tipHeight - mintedHeight + 1, not a finality guarantee.

Fork choice, reorgs and storage

Per-block work is floor(2^256 / (target + 1)); accumulate from genesis work zero. Keep all valid known-parent branches. Adopt a tip only for strictly greater cumulative work; equal-work ties retain the current tip, including across replay of persisted insertion order. A shorter, harder branch can win.

On reorg, reconstruct ownership from the winning branch. Orphan mints disappear from canonical ownership; disconnected transactions are reconsidered before existing pending entries, in order, only if still valid and not already committed. A trade is requeued as one unit only when every leg remains valid; otherwise the complete bundle is discarded. No checkpoints/finality or rollback protection beyond observed work; an isolated node has no peer-backed security.

The ledger is stored in data/testnet/ledger.sqlite, under the WIKIDECK_DATA_DIR testnet subdirectory (Docker uses /app/data). SQLite WAL mode with full synchronous commits stores each accepted block once in an indexed table and replaces the bounded mempool in the same transaction; writes no longer serialize/rewrite the entire DAG, and there is no 128 MiB/10,000-block ledger cap. A matching v6 legacy ledger.json is schema-checked and imported transactionally once; the source JSON is left untouched as a recovery copy. Earlier-protocol ledgers are not compatible and fail closed. Back up the complete stopped-node data directory, including SQLite database/WAL files. Do not copy only ledger.sqlite while the node is running.

The in-memory DAG keeps block records and cumulative work, while each block's NFT state is a persistent overlay containing only that block's changes rather than a full state copy. State-root calculation still visits the complete NFT state, and startup still revalidates every stored proof; CPU and startup work therefore grow with chain size. This is materially more storage-efficient, but not yet an archival-scale chain database or a public DoS defense. Blocks remain individually limited to 64 KiB and 64 transaction units; the mempool remains capped at 256 transaction units. Mutations are serialized (queue cap 256). HTTP startup exclusively creates a data-directory lock named testnet-node.lock. It is removed on normal shutdown, not automatically after a crash: stop and verify all owning processes have exited before manual stale-lock removal. Never bypass a live lock or share a writer directory. Back up node data and wallet exports separately; deleting a volume cannot be recovered from a wallet alone.

Fixed peers and HTTP surface

Peers are explicit operator-configured URL origins, up to 16, not discovered from network input. Default: HTTPS/public addresses only, checked and pinned during DNS lookup; no credentials, paths, queries, fragments or redirects. WIKIDECK_ALLOW_PRIVATE_PEERS=1 permits private/HTTP destinations only by explicit operator choice (used for local Docker tests). No bootnodes, DHT, automatic public deployment or claimed Sybil resistance.

Sync polls every 5 seconds, checks network/chain ID, imports at most 32 topologically ordered DAG blocks per peer/pass, and validates remote pool entries locally. Accepted blocks and transactions (including complete trade bundles) are gossiped to configured peers; busy-peer gossip is dropped, with polling for catch-up. Responses are capped at 2 MiB with a 10-second deadline. Peer count is current outbound connectivity metadata, not consensus membership or a security score.

Route	Purpose
GET /healthz	Process health/network, not sync or consensus security.
GET /api/state, /api/chain	Canonical blocks, mined/pending transactions, tip/work and peer count.
GET /api/summary, /api/canonical?tip=<hash>&offset=0&limit=16	Bounded state metadata and current-tip-pinned canonical pages. Full legacy snapshots are limited to 128 blocks and the response-byte budget. See resource controls.
GET /api/network	Selected profile, pinned chain/genesis, and candidate launch status.
GET /api/nfts/:id, /api/peers	Canonical NFT owner/sequence/confirmations, article and at-mint statistics; configured peer status.
GET /api/decks/:address?offset=0&limit=24	Public canonical deck for a wallet address at one tip; paged, maximum 24 cards per response. This reveals public address holdings.
GET /api/wikipedia/:pageId	Cached current enrichment for positive English Wikipedia page IDs: categories, views, size, languages and display-only scores. Never a consensus input.
POST /api/mining/template	Rate-limited public miner identity; fetches a random article with a picture plus fresh at-mint statistics. Returns 503 when either cannot be fetched.
POST /api/blocks	Independently validates a signed block, proof, article snapshot, required Wikimedia image URL, and statistics snapshot consistency. Repeated page IDs are accepted.
POST /api/transfer	Signed single-card transfer admission; 202 means pending.
POST /api/trades	Validates and admits one signed, atomic multi-card trade bundle; 202 means the whole bundle is pending.
GET /p2p/status, /p2p/blocks?offset=0, /p2p/mempool	Manifest/size/tip, 32-block inventory pages, pending transactions.
POST /p2p/block, /p2p/transaction	Same validation as local block/transaction submission; transaction endpoint accepts a transfer or atomic trade.

POSTs require JSON, enforce supplied browser origins and signatures where applicable; peers without an Origin header are not authenticated by this check. HTTP parsing is capped at 68 KiB; API/P2P rate limits are 300/minute and block/transaction/trade submissions additionally 30/minute. There are no consensus fees, peer authentication, or economic spam deterrents. Same-origin browser RPC trusts the node; it does not independently verify these commitments. Deploy no valuable assets: majority-hashpower, eclipse, censorship, reorg and resource-exhaustion risks remain and are not externally audited under the operator's current policy.

Testnet storage and chain identities above remain unchanged. The gated mainnet candidate selects separate chain/signature domains, storage, lock and wallet identities; it is not a running or certified production network.