Native RandomX engine

This is the CPU proof engine used by the Wikideck testnet node and explicit opt-in native miner. Mining runs only when the operator starts the native CLI; the browser does not mine or launch native programs. The Docker image builds this pinned implementation and runs its known-vector test. There is no GPU path, simulated hash, or production fallback hash.

This document covers building the executable, its memory modes, its line protocol, and the TypeScript adapter. For the user-facing install and run instructions, start with the README.

Source and build
Modes and resource use
Line protocol
Node API
Real upstream known-vector check

Source and build

RandomX v1.2.1 is fetched by CMake FetchContent from the official HTTPS Git repository, pinned to 102f8acf90a7649ada410de5499a7ec62e49e1da (also the upstream v1.2.1 tag). The first configuration requires network access and Git; later builds reuse the checkout. Keep upstream copyright/license notices when distributing binaries. Do not override FetchContent sources with untrusted local code.

Requirements: CMake 3.24+, Git, a C/C++17 toolchain, and a 64-bit host for fast mode. MSVC needs its C++ and MASM tools in a developer terminal. MinGW needs its compiler and make on PATH; select the MinGW Makefiles generator. Linux/macOS need their normal C/C++ build tools. No administrator privileges or huge pages are required. On MSVC, upstream's pinned build invokes its own PowerShell assembly-generation script; review upstream build tooling before running under restrictive policies.

From the repository root:

cmake -S native -B native/build -DCMAKE_BUILD_TYPE=Release -DWIKIDECK_RANDOMX_TESTS=OFF
cmake --build native/build --config Release --target wikideck-randomx --parallel 2

Convenience scripts (two compiler jobs by default):

sh scripts/build-randomx.sh
sh scripts/build-randomx.sh --test

./scripts/build-randomx.ps1
./scripts/build-randomx.ps1 -Generator 'MinGW Makefiles' -Test

The scripts propagate configure/build/test failures. PowerShell execution policy must permit the script; do not bypass a managed policy. POSIX customization: JOBS=2, RANDOMX_BUILD_DIR=/absolute/build/path; PowerShell customization: -Jobs 2, -BuildDirectory <path>. Select a toolchain/generator through the normal CMake environment or a manual configuration. Don't switch generators in an existing build directory.

All generators place the executable directly under native/build (with .exe on Windows), including Release builds with multi-config generators. Upstream tests/benchmark/codegen are excluded from the default build: v1.2.1 does not provide a tests-disable option. No benchmark or dataset-heavy fast-mode test runs automatically.

Modes and resource use

Run wikideck-randomx with no arguments for light mode (256 MiB cache plus approximately 2 MiB VM scratchpad and runtime overhead). --fast adds the real approximately 2080 MiB dataset; the 256 MiB cache is retained for key changes, so budget over 2.3 GiB per process. Dataset initialization is single-threaded and can be expensive. Neither mode allocates RandomX memory until its first valid request. Cache, dataset, and VM are reused for the same key. A changed key reinitializes the cache and, in fast mode, the entire dataset before creating a fresh VM.

Flags start with randomx_get_flags() (CPU-supported AES/JIT/Argon2 optimization). RANDOMX_FLAG_SECURE is explicitly added for VM JIT W^X, and upstream is compiled with RANDOMX_FORCE_SECURE for cache JIT policy too. Platforms without JIT use the reference interpreter; allocation/JIT failures are fatal, never a fake PoW result. No large pages are requested. RAII releases VM, dataset, then cache on normal EOF and caught errors. Public PoW key/cache data is not secret and is released, not claimed to be securely erased from physical memory.

Line protocol

One ASCII command per LF-terminated line; CRLF is also accepted. Exactly one ASCII response line, flushed immediately, per valid command:

hash <key64hex> <blobhex>
<hash64hex>

mine <key64hex> <blobhex> <target64hex> <nonceOffsetDecimal> <maxAttemptsDecimal>
found <hash64hex> <nonceDecimal>
exhausted

found and exhausted are alternatives, not two responses. Input hex may use either case; output hashes are lowercase in the RandomX byte order. Keys and targets are exactly 32 bytes (64 hex characters); blobs are 1..8192 bytes. Tokens use exactly one ASCII space, without leading/trailing whitespace. Commands are capped at 16600 bytes before LF. Invalid characters, malformed/overflowing unsigned decimal, extra tokens, invalid bounds, or a partial final line cause a concise stderr error and nonzero exit, with no fabricated response on stdout. Empty stdin EOF exits successfully. There are no stdout banners or diagnostic logs.

Mining requires maxAttempts in 1..10000 and nonceOffset + 8 <= blob.size. The initial nonce is read from the blob's existing eight little-endian bytes. The first attempt uses that nonce, then increments by one and rewrites LE8 for each attempt. Only those eight bytes change. At uint64 maximum, search ends without wrapping. The native response contains an exact unsigned decimal nonce.

Target convention: interpret the 32 output bytes and target bytes as unsigned BIG-ENDIAN integers; accept hash <= target. This intentionally differs from Monero's convention. In particular, equal hashes pass. Never reverse the hash or target when using this interface.

Node API

RandomX exports:

interface PowEngine {
  hash(key: string, blob: Uint8Array): Promise<string>;
  close(): void;
}
class RandomX implements PowEngine {
  constructor(options?: { binary?: string; fast?: boolean });
  hash(key: string, blob: Uint8Array): Promise<string>;
  mine(key: string, blob: Uint8Array, target: string,
       nonceOffset: number, maxAttempts: number):
    Promise<{ hash: string; nonce: number } | null>;
  close(): void;
}

Binary precedence: options.binary, RANDOMX_BINARY, then the executable under the repository's native build directory (resolved relative to this module, independent of working directory). Relative explicit paths are resolved against the caller's working directory. Configure only a trusted executable: this is a native-code trust boundary, not a sandbox. The adapter uses spawn with a fixed argument array, shell: false, and pipe stdio; user data only enters stdin after strict validation. Missing/non-executable files throw in the constructor. Other spawn failures are retained and reject current/future requests.

Each instance owns one long-lived child, one active command, and at most 64 total active/queued requests. Calls are FIFO, including mixed hash/mine calls and key changes. Inputs are snapshotted before queuing. The 120-second deadline starts when each command is sent (includes cold cache/dataset initialization); queued requests wait for earlier work. Keep mining batches small enough to finish on the host CPU. The process is killed on timeout, malformed/unsolicited/oversized output, stream error, or crash, rejecting all outstanding work and subsequent calls. Responses and retained stderr are bounded. No automatic retry or process restart.

mine returns null for exhaustion and validates returned hash syntax, target, and nonce range (it does not independently recompute a native result). Because the API returns number, its entire requested nonce range must fit Number.MAX_SAFE_INTEGER; higher uint64 nonces are supported only by the direct native protocol. Invalid inputs reject without sending native work.

Always call close() in finally. It is idempotent, rejects pending/queued work, and sends EOF for graceful RAII cleanup; a busy child is forcibly killed after one second. A failed/closed instance cannot be reopened.

Real upstream known-vector check

The opt-in --self-test executable argument validates upstream's exact Hash test 1a/2a:

Key: ASCII test key 000 (12 bytes, not NUL-terminated or zero-padded).
Input: ASCII This is a test (14 bytes, no terminator).
Expected RandomX hash: 639183aae1bf4c9a35884cb46b09cad9175f04efd7684e7262a0ac1c2f0b4e3f.

The self-test uses an internal API so the line protocol retains its strict 32-byte-key requirement. Padding this upstream key to 32 bytes would change the hash and is not a valid reproduction of the vector. Self-test success is reported on stderr only, with exit zero; mismatch exits nonzero. --fast --self-test checks the same real vector in dataset mode, but requires the full memory allocation.

cmake -S native -B native/build -DCMAKE_BUILD_TYPE=Release -DWIKIDECK_RANDOMX_TESTS=ON
cmake --build native/build --config Release --target wikideck-randomx --parallel 2
ctest --test-dir native/build -C Release --output-on-failure

This test is OFF by default and is separate from the application's tests. Without a built native binary, no native engine or native test is run by the application.

9.4 KiB Raw Blame History