diff --git a/README.md b/README.md index 0c6c693..0be59d2 100644 --- a/README.md +++ b/README.md @@ -5,10 +5,13 @@ Deploy the Wikideck production container to an existing Kubernetes cluster using volume claim, a dedicated ServiceAccount, a ClusterIP Service, and a NetworkPolicy. An HTTPS Ingress is optional and disabled by default. -**Wikideck is a prototype, not a hardened public service.** This role does not add -application authentication, distributed consensus, a database, or wallet recovery. -Use test wallets and restrict access. Mining happens in the browser; the server -coordinates the chain and verifies proofs. +**Wikideck is an experimental RandomX testnet, not a production-ready public +mainnet.** The updated application has independent validating nodes, fork choice, +native NFTs, and encrypted wallet backup/recovery. It supplies no public bootnodes +or deployed public network. Use test wallets and no valuable assets. This role +deploys a **validator**, never a miner: mining is an explicitly started native CLI +process outside the browser. Private keys, backups and passwords must stay off +validator pods and out of Ansible variables. ## Requirements @@ -24,6 +27,10 @@ coordinates the chain and verifies proofs. - **A published image built from Wikideck's production Dockerfile.** There is no assumed official registry image. Set `wikideck_image` to your actual tagged or digest-pinned image. The role does not build/push images or install a cluster. + Rebuild from the current source: the image must include Node 24, libstdc++, and + the native RandomX executable, with `RANDOMX_BINARY` set by the image. An old + browser-mining image is incompatible. Build and verify for your worker node + architecture; do not assume a locally built image is multi-architecture. - For ingress: an ingress controller, DNS, and a trusted TLS Secret in the app namespace. Alternatively, configure cert-manager annotations and an existing issuer. The role does not install cert-manager or create an issuer. @@ -37,8 +44,8 @@ environment. Windows controllers must run Ansible through WSL or Linux container Install this checkout into your playbook's role search path as `roles/wikideck`, or install it using an Ansible Galaxy Git role requirement named `wikideck`. [examples/playbook.yml](examples/playbook.yml) shows an HTTPS deployment with -ingress-nginx and cert-manager. Replace the image, hostname, class, issuer, and -controller namespace with real values before running it. +ingress-nginx and cert-manager. Replace the image, hostname, peer URLs, class, +issuer, and controller namespace with real values before running it. ```yaml - name: Deploy a cluster-internal Wikideck instance @@ -67,6 +74,64 @@ for browser wallet cryptography. Configure HTTP-to-HTTPS redirect at your ingres controller; the example includes the ingress-nginx annotation. Ingress creation does not wait for DNS, certificate issuance, or an externally reachable endpoint. +### Connect validators + +Set `wikideck_peers` to a list of up to 16 compatible peer HTTPS origins, such as +`https://peer-a.example.org`. The role joins this list into `WIKIDECK_PEERS`. +An empty list deliberately starts an **isolated validator**. There is no automatic +discovery, bootnode list, or automatic reciprocal configuration: configure the +other operators' nodes too. All nodes must run the same network/genesis/consensus. + +Peer RPC uses `/p2p/*` on the **same port 3001** as the UI/API; no separate TCP/UDP +P2P Service or host networking is required. The existing `/` ingress path forwards +those routes. Keep peer JSON responses uncompressed and do not redirect HTTPS +peer requests: the client rejects redirects, non-JSON and compressed responses. +An interactive login gateway on `/p2p/*` will break the current unauthenticated +peer protocol; use deliberate source restrictions/abuse controls instead. + +Public peers must use HTTPS and public DNS/IP addresses. Wikideck validates every +connection's DNS answers and pins the connection to that validated result. Never +disable TLS verification to work around a certificate or split-horizon DNS error. +The default policy allows public TCP 443 only when peers are configured. For +HTTPS peer origins on another port, add that port to +`wikideck_network_policy_https_ports`, or supply narrowly scoped explicit egress +rules. Browser Wikimedia image fetches are client-side; validators no longer +fetch random Wikipedia articles. + +For an explicitly private testnet only, set `wikideck_allow_private_peers: true` +and provide `wikideck_network_policy_peer_egress` rules with destination selectors +and numeric TCP ports. This opt-in permits **both private addresses and HTTP** in +the application; it does not automatically open network policy. For example: + +```yaml +wikideck_peers: + - http://other-node.games.svc.cluster.local:80 +wikideck_allow_private_peers: true +wikideck_network_policy_https_to: [] +wikideck_network_policy_peer_egress: + - to: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: games + podSelector: + matchLabels: + app.kubernetes.io/instance: other-node + ports: + - protocol: TCP + port: 3001 +``` + +The receiving node must separately allow the sender through its ingress policy. +Service DNAT handling depends on the CNI; the example selects the destination pod +and its target port 3001 rather than opening all destinations on Service port 80. +Do not enable private-peer exceptions for public peer configuration. + +Deploy multiple validators as **separate role instances with different names and +PVCs** (and separate hostnames if using ingress), not by scaling one Deployment. +Sharing a machine/cluster/storage failure domain does not provide independent +operator security. Empty local chains can synchronize from configured peers; +never seed a second running node by sharing its writable PVC. + ## Variables All defaults are in [defaults/main.yml](defaults/main.yml). @@ -78,10 +143,12 @@ All defaults are in [defaults/main.yml](defaults/main.yml). | `wikideck_namespace` | `wikideck` | Namespace for all namespaced resources. | | `wikideck_create_namespace` | `true` | Create the namespace; disable for a pre-provisioned namespace. | | `wikideck_kubeconfig` / `wikideck_kube_context` | `null` | Override controller authentication/context. API TLS verification remains enabled. | -| `wikideck_wait` / `wikideck_wait_timeout` | `true` / `300` | Wait for Deployment readiness, timeout in seconds. | +| `wikideck_wait` / `wikideck_wait_timeout` | `true` / `900` | Wait for Deployment readiness, timeout in seconds. | +| `wikideck_startup_probe_failure_threshold` | `180` | Failed probes at 5-second intervals before startup restart; default budget 900 seconds. | | `wikideck_image_pull_policy` | `IfNotPresent` | `Always`, `IfNotPresent`, or `Never`. | | `wikideck_image_pull_secrets` | `[]` | Names of existing registry Secrets in this namespace. | -| `wikideck_offline` | `false` | Starter articles without live Wikipedia requests. | +| `wikideck_peers` | `[]` | Up to 16 static peer origins; empty means isolated. | +| `wikideck_allow_private_peers` | `false` | Explicit test-only private/HTTP peer permission. | | `wikideck_public_origin` | empty | Additional allowed HTTPS browser origin; required with ingress. | | `wikideck_ingress_enabled` | `false` | Create TLS ingress; disabling removes the role's named ingress. | | `wikideck_ingress_host` | empty | DNS hostname, without scheme, port, or path. | @@ -92,12 +159,19 @@ All defaults are in [defaults/main.yml](defaults/main.yml). | `wikideck_storage_size` | `1Gi` | Requested PVC capacity; resizing requires StorageClass support. | | `wikideck_storage_class` | `null` | `null`: cluster default; empty string: no class; otherwise a class name. | | `wikideck_storage_access_modes` | `[ReadWriteOnce]` | `ReadWriteOnce` or `ReadWriteOncePod`; never `ReadWriteMany`. | -| `wikideck_resources` | see defaults | CPU/memory requests `100m`/`128Mi`, limits `1`/`512Mi`. | +| `wikideck_resources` | see defaults | CPU/memory requests `250m`/`512Mi`, limits `1`/`1Gi`; small-validator starting point, not a capacity guarantee. | | `wikideck_node_selector` / `wikideck_tolerations` / `wikideck_affinity` | `{}` / `[]` / `{}` | Pod placement settings. | | `wikideck_network_policy_enabled` | `true` | Manage network isolation; disabling deletes this role's policy. | | `wikideck_network_policy_ingress_from` | same-namespace pods | List of NetworkPolicy peers allowed to reach TCP 3001; empty list denies all ingress. | | `wikideck_network_policy_dns_to` | `kube-system` namespace | DNS peers allowed on UDP/TCP 53; adapt for custom/NodeLocal DNS. | -| `wikideck_network_policy_https_to` | public IPv4/IPv6 ranges, private/link-local excluded | TCP 443 egress peers; omitted in offline mode. | +| `wikideck_network_policy_https_to` | public IPv4/IPv6 ranges, private/link-local excluded | Public peer destinations; omitted when there are no configured peers. | +| `wikideck_network_policy_https_ports` | `[443]` | Public peer TCP ports; extend for custom HTTPS origins. | +| `wikideck_network_policy_peer_egress` | `[]` | Additional explicit `to` + numeric TCP `ports` rules, emitted only with configured peers. | + +`wikideck_offline` / `WIKIDECK_OFFLINE` have been removed. The current catalog is +fixed and genesis-bound, not a live Wikipedia feed. Remove the old role variable; +it no longer controls networking. The image's packaged `RANDOMX_BINARY` and default +entrypoint are deliberately preserved; no production fake-PoW fallback is added. ## Storage and safe updates @@ -108,7 +182,11 @@ All defaults are in [defaults/main.yml](defaults/main.yml). `ReadWriteOncePod` if supported by your CSI driver. Do not force-delete an old pod on a partitioned node until it has been fenced; Recreate is not a distributed storage lock. -- `/app/data` contains the chain and signed transfer journal. The process runs +- `/app/data/testnet/ledger.json` stores the block DAG and pending transfers; + `/app/data/testnet-node.lock` exclusively locks the entire data directory. + The old demo chain is separate, not imported, and has **no migration** to the + new testnet. Back it up first; preferably use a fresh claim for the new network. + The process runs as UID/GID 1000 with `fsGroup: 1000`. Your storage driver must honor ownership settings, or the existing volume must already be writable by this identity. No privileged/root ownership-fixing init container is used. @@ -121,9 +199,37 @@ All defaults are in [defaults/main.yml](defaults/main.yml). backup or use your CSI provider's documented snapshot procedure. Encrypt disks and backups, define retention, and test restores. Deleting a namespace or PVC can still destroy data depending on the underlying reclaim policy. -- Chain verification runs at startup. For large chains, review resources, - startup probe budget (300 seconds), and readiness wait timeout. JSON storage - is unbounded; this role cannot solve Wikideck's persistence scalability limits. +- Full native RandomX revalidation runs before the server listens. Increase + `wikideck_startup_probe_failure_threshold` and `wikideck_wait_timeout` together + for your measured replay time, leaving scheduling/image-pull margin. A healthy + empty node does not exercise the native cache or prove peer synchronization. +- The app bounds the ledger to 128 MiB and 10,000 total DAG blocks, with 256 + pending transfers. Complete NFT states are retained per DAG node, so RAM may + grow quadratically despite disk bounds. Whole-ledger rewrites and full replay + are **not suitable for a long-lived public chain**; a larger PVC does not fix + those application limits. + +### Crash recovery and resource sizing + +The application removes its lock on graceful shutdown but deliberately does +**not** automatically recover a stale lock after an OOM kill, forced termination, +node loss, or interruption during startup replay. Kubernetes restarts can remain +in CrashLoopBackOff until an operator intervenes. This role never deletes the lock +in an init container or as an automatic "repair". + +To recover, first stop the Deployment and verify every former owner is gone; +fence an unreachable old node before mounting the claim elsewhere. Take a backup, +then remove only the confirmed stale lock using an approved maintenance pod or +storage procedure. Preserve the ledger, restore the Deployment to one replica, +and inspect replay and peer status. Do not automatically remove locks based on +their PID: PIDs are reused and isolated between containers. + +Each validator uses native **light-mode RandomX**, requiring a 256 MiB cache plus +scratchpad, Node heap, ledger states, and process overhead. The new 1 GiB limit +matches the updated upstream Compose starting budget, but must be measured against +your ledger, peer load and replay. Watch OOM events, CPU throttling, replay time, +and block propagation. Fast miners need **over 2.3 GiB per process** and separate +capacity/security planning; this role never launches one or mounts its wallet. ## Security and networking @@ -135,14 +241,14 @@ temporary directory are writable. All three HTTP probes target `/healthz` on por period accommodates the application's 25-second shutdown deadline. NetworkPolicy defaults allow same-namespace clients, cluster DNS in `kube-system`, -and public HTTPS for Wikipedia. With ingress, explicitly allow the ingress +and public HTTPS only when peer origins are configured. With ingress, allow the ingress controller's namespace/pods as shown in the example. Peer entries are additive OR alternatives: put a namespaceSelector and podSelector in the **same entry** to restrict pods within that namespace. Empty peer lists deny that traffic; empty peer objects are rejected to avoid accidentally allowing every source/destination. -Standard NetworkPolicy cannot allow Wikipedia by hostname; the default public -TCP 443 egress is broader than a domain allowlist. Use CNI FQDN policies or an +Standard NetworkPolicy cannot allow peer origins by hostname; public +TCP 443 egress is broader than the app's fixed peer allowlist. Narrow CIDRs or use CNI FQDN policies or an egress gateway if required. Policies are additive with any other policies in the namespace and have no effect without an enforcing CNI. NodeLocal DNS, host-network ingress controllers, service meshes, and private proxies may need peer changes. @@ -155,16 +261,28 @@ API certificate verification. The role does not embed secrets or implement its own cryptography; image signature verification and encrypted storage are cluster operator responsibilities. Scan and rebuild the application image regularly. +Native RandomX uses ordinary memory allocations and secure W^X JIT; the current +app does not request huge pages, host devices, executable files in `/tmp`, or +privileged capabilities. Keep the existing hardening. Verify actual hashing on +your worker architecture and runtime security profile; HTTP health alone cannot +detect every native engine failure. Do not grant privileged mode or disable +seccomp as an untested performance workaround. + Wikideck does not currently configure Express `trust proxy`. Clients behind one proxy can therefore share an IP-based application rate limit. Review the application before public exposure; do not try to fix this by blindly trusting -forwarded headers. The role adds no authentication, WAF, or abuse protection. +forwarded headers. The current app limits `/api` and `/p2p` together; default +forwarded-header behavior can also produce express-rate-limit validation errors +in logs. The role adds no authentication, WAF, or abuse protection. Eclipse, +majority-hashpower, spam/DoS and reorganization risks remain application/protocol +issues; deployment hardening does not make a public mainnet safe. ## Validation [tests/render.yml](tests/render.yml) runs entirely offline: it validates inputs, renders/parses every template across default, ingress, existing-claim, storage -class, offline, deny-all, and digest-image variants, and checks wiring/security +class, isolated/public/private peers, custom HTTPS ports, deny-all, longer startup, +and digest-image variants, and checks wiring/security invariants. It also checks that unsafe/invalid settings are rejected. Run it with `ansible-playbook -i tests/inventory tests/render.yml`. @@ -173,8 +291,19 @@ checkout. Syntax-check it with `ansible-playbook -i tests/inventory tests/test.y To deploy to a disposable test namespace, run it with `ansible-playbook -i tests/inventory tests/test.yml -e wikideck_image=YOUR_IMAGE -e wikideck_namespace=wikideck-test`. Run it a second time to verify idempotence. Check Pod readiness, `/healthz`, PVC -retention across updates, DNS/HTTPS egress, TLS, and browser mining/transfer flows -on your chosen cluster. Offline tests do not prove CSI/CNI/controller behavior. +retention across updates, DNS/peer egress, TLS, and wallet/transfer flows on your +chosen cluster. `/healthz` is only process health, not sync readiness. Inspect +`/api/peers` for connected peers and `/p2p/status` for matching network, chainId and +tip across independently deployed nodes. `/api/peers` exposes configuration; +apply your access policy deliberately. Offline tests do not prove CSI/CNI/controller +behavior or consensus safety. Mining a real block is a separate explicit-consent +test; it must not be part of routine deployment or health probes. + +[tests/runtime-smoke.mjs](tests/runtime-smoke.mjs) is a disposable image-level +check: it runs the native known-vector self-test (hashing, **not mining**), checks +node API health, and verifies graceful lock cleanup and restart durability. Mount +it read-only and invoke Node inside the current image with disposable writable +data and no network. It does not use or modify a real chain or wallet. Ansible check mode (`--check --diff`) is supported by the Kubernetes module, but still needs API access; creating a new namespace in check mode may require a diff --git a/defaults/main.yml b/defaults/main.yml index 6d617b2..eb29912 100644 --- a/defaults/main.yml +++ b/defaults/main.yml @@ -10,12 +10,17 @@ wikideck_create_namespace: true wikideck_kubeconfig: null wikideck_kube_context: null wikideck_wait: true -wikideck_wait_timeout: 300 +wikideck_wait_timeout: 900 +# Full native RandomX ledger revalidation happens before HTTP starts. +wikideck_startup_probe_failure_threshold: 180 wikideck_image_pull_policy: IfNotPresent # Existing registry Secret names in the application namespace. wikideck_image_pull_secrets: [] -wikideck_offline: false +# Static peer origins, at most 16. Empty means an isolated validator, not a network. +wikideck_peers: [] +# Test/private networks only; permits private addresses AND HTTP in the app. +wikideck_allow_private_peers: false # Exact browser origin, without a trailing slash. Required when using ingress. wikideck_public_origin: "" wikideck_ingress_enabled: false @@ -35,11 +40,11 @@ wikideck_storage_access_modes: wikideck_resources: requests: - cpu: 100m - memory: 128Mi + cpu: 250m + memory: 512Mi limits: cpu: "1" - memory: 512Mi + memory: 1Gi wikideck_node_selector: {} wikideck_tolerations: [] wikideck_affinity: {} @@ -54,7 +59,13 @@ wikideck_network_policy_dns_to: - namespaceSelector: matchLabels: kubernetes.io/metadata.name: kube-system -# Live article discovery needs HTTPS. Standard NetworkPolicy cannot select FQDNs. +# Public peer HTTPS egress, emitted only when wikideck_peers is nonempty. +# Standard NetworkPolicy cannot select FQDNs; narrow these CIDRs when practical. +wikideck_network_policy_https_ports: + - 443 +# Explicit additional NetworkPolicy egress rules for private/custom-port peers. +# Each rule must include nonempty 'to' peers and TCP 'ports'. No rules by default. +wikideck_network_policy_peer_egress: [] wikideck_network_policy_https_to: - ipBlock: cidr: 0.0.0.0/0 diff --git a/examples/playbook.yml b/examples/playbook.yml index f095b00..242a45b 100644 --- a/examples/playbook.yml +++ b/examples/playbook.yml @@ -8,6 +8,10 @@ vars: # Replace with your published, scanned image; preferably use @sha256:... wikideck_image: registry.example.com/games/wikideck:1.0.0 + # Replace with real compatible testnet peers. No bootnodes are supplied. + wikideck_peers: + - https://peer-a.example.org + - https://peer-b.example.org wikideck_ingress_enabled: true wikideck_ingress_host: wikideck.example.com wikideck_public_origin: https://wikideck.example.com diff --git a/tasks/validate.yml b/tasks/validate.yml index d1dc5d7..c44a1e4 100644 --- a/tasks/validate.yml +++ b/tasks/validate.yml @@ -11,6 +11,7 @@ - wikideck_namespace | length <= 63 - wikideck_image_pull_policy in ['Always', 'IfNotPresent', 'Never'] - wikideck_wait_timeout | string is match('^[1-9][0-9]*$') + - wikideck_startup_probe_failure_threshold is integer and wikideck_startup_probe_failure_threshold > 0 - wikideck_resources is mapping - wikideck_resources.requests is defined and wikideck_resources.requests is mapping - wikideck_resources.limits is defined and wikideck_resources.limits is mapping @@ -29,7 +30,7 @@ - wikideck_public_origin | length == 0 or wikideck_public_origin is match('^https://[a-z0-9.-]+(:[0-9]+)?$') - wikideck_ingress_enabled is boolean - wikideck_network_policy_enabled is boolean - - wikideck_offline is boolean + - wikideck_allow_private_peers is boolean - wikideck_create_namespace is boolean - wikideck_wait is boolean fail_msg: >- @@ -64,6 +65,56 @@ - wikideck_network_policy_ingress_from - wikideck_network_policy_dns_to - wikideck_network_policy_https_to + - wikideck_network_policy_https_ports + - wikideck_network_policy_peer_egress + - wikideck_peers + +- name: Validate the static peer allowlist + ansible.builtin.assert: + that: + - wikideck_peers | length <= 16 + fail_msg: Wikideck supports at most 16 configured peer origins. + +- name: Validate peer origins without credentials or paths + ansible.builtin.assert: + that: + - item is string + - item | length <= 2048 + - item is match('^https?://([a-zA-Z0-9]([a-zA-Z0-9.-]*[a-zA-Z0-9])?|\[[0-9a-fA-F:]+\])(:[0-9]{1,5})?$') + - wikideck_allow_private_peers or item is match('^https://') + fail_msg: >- + Peers must be HTTPS origins without credentials, paths, commas, queries or + fragments. Private/HTTP test peers require wikideck_allow_private_peers=true. + Wikideck additionally validates ports and public DNS/IP addresses at runtime. + loop: "{{ wikideck_peers }}" + +- name: Validate public peer egress ports + ansible.builtin.assert: + that: + - item is integer and item > 0 and item <= 65535 + fail_msg: HTTPS egress ports must be integers in the range 1..65535. + loop: "{{ wikideck_network_policy_https_ports }}" + +- name: Validate explicit peer egress rules + ansible.builtin.assert: + that: + - item is mapping + - item.to is defined and item.to is sequence and item.to is not string and item.to is not mapping + - item.to | length > 0 + - item.ports is defined and item.ports is sequence and item.ports is not string and item.ports is not mapping + - item.ports | length > 0 + fail_msg: Peer egress rules require explicit nonempty to and ports lists. + loop: "{{ wikideck_network_policy_peer_egress }}" + +- name: Validate explicit peer egress TCP ports + ansible.builtin.assert: + that: + - item is mapping + - item.protocol | default('TCP') == 'TCP' + - item.port is defined and item.port is integer and item.port > 0 and item.port <= 65535 + - item.endPort is not defined + fail_msg: Peer egress ports must be individual numeric TCP ports in the range 1..65535. + loop: "{{ wikideck_network_policy_peer_egress | map(attribute='ports') | flatten }}" - name: Require explicit network policy peers ansible.builtin.assert: @@ -73,5 +124,6 @@ fail_msg: Use an explicit podSelector, namespaceSelector, or ipBlock rather than an empty peer. loop: >- {{ wikideck_network_policy_ingress_from + wikideck_network_policy_dns_to - + wikideck_network_policy_https_to }} + + wikideck_network_policy_https_to + + (wikideck_network_policy_peer_egress | map(attribute='to') | flatten) }} when: wikideck_network_policy_enabled | bool diff --git a/templates/deployment.yml.j2 b/templates/deployment.yml.j2 index a777ee3..59fa9a8 100644 --- a/templates/deployment.yml.j2 +++ b/templates/deployment.yml.j2 @@ -9,7 +9,7 @@ metadata: app.kubernetes.io/instance: {{ wikideck_name | to_json }} app.kubernetes.io/managed-by: ansible spec: - # Never expose replicas as a variable: chain.json has no multi-writer locking. + # One validator per ledger/PVC. Other validators need separate role instances. replicas: 1 strategy: type: Recreate @@ -67,8 +67,10 @@ spec: value: "3001" - name: WIKIDECK_DATA_DIR value: /app/data - - name: WIKIDECK_OFFLINE - value: {{ ('1' if wikideck_offline else '0') | to_json }} + - name: WIKIDECK_PEERS + value: {{ (wikideck_peers | join(',')) | to_json }} + - name: WIKIDECK_ALLOW_PRIVATE_PEERS + value: {{ ('1' if wikideck_allow_private_peers else '0') | to_json }} {% if wikideck_public_origin | length > 0 %} - name: PUBLIC_ORIGIN value: {{ wikideck_public_origin | to_json }} @@ -80,7 +82,7 @@ spec: port: http periodSeconds: 5 timeoutSeconds: 5 - failureThreshold: 60 + failureThreshold: {{ wikideck_startup_probe_failure_threshold | int }} readinessProbe: httpGet: path: /healthz diff --git a/templates/networkpolicy.yml.j2 b/templates/networkpolicy.yml.j2 index 1c6cad8..60e9c1c 100644 --- a/templates/networkpolicy.yml.j2 +++ b/templates/networkpolicy.yml.j2 @@ -23,7 +23,8 @@ spec: {% else %} ingress: [] {% endif %} -{% if wikideck_network_policy_dns_to | length == 0 and (wikideck_offline or wikideck_network_policy_https_to | length == 0) %} +{% set public_egress = wikideck_peers | length > 0 and wikideck_network_policy_https_to | length > 0 and wikideck_network_policy_https_ports | length > 0 %} +{% if wikideck_network_policy_dns_to | length == 0 and not public_egress and (wikideck_peers | length == 0 or wikideck_network_policy_peer_egress | length == 0) %} egress: [] {% else %} egress: @@ -35,10 +36,17 @@ spec: - protocol: TCP port: 53 {% endif %} -{% if not wikideck_offline and wikideck_network_policy_https_to | length > 0 %} +{% if public_egress %} - to: {{ wikideck_network_policy_https_to | to_json }} ports: +{% for port in wikideck_network_policy_https_ports %} - protocol: TCP - port: 443 + port: {{ port | int }} +{% endfor %} +{% endif %} +{% if wikideck_peers | length > 0 %} +{% for rule in wikideck_network_policy_peer_egress %} + - {{ rule | to_json }} +{% endfor %} {% endif %} {% endif %} diff --git a/tests/invalid_case.yml b/tests/invalid_case.yml index 7a08eeb..f458c46 100644 --- a/tests/invalid_case.yml +++ b/tests/invalid_case.yml @@ -12,7 +12,11 @@ wikideck_ingress_host: "{{ invalid_case.host | default('wikideck.example.com') }}" wikideck_public_origin: "{{ invalid_case.origin | default('') }}" wikideck_resources: "{{ invalid_case.resources | default({'requests': {'cpu': '100m', 'memory': '128Mi'}, 'limits': {'cpu': '1', 'memory': '512Mi'}}) }}" - wikideck_offline: "{{ invalid_case.offline | default(false) }}" + wikideck_peers: "{{ invalid_case.peers | default([]) }}" + wikideck_allow_private_peers: "{{ invalid_case.allow_private | default(false) }}" + wikideck_startup_probe_failure_threshold: "{{ invalid_case.startup_threshold | default(180) }}" + wikideck_network_policy_https_ports: "{{ invalid_case.https_ports | default([443]) }}" + wikideck_network_policy_peer_egress: "{{ invalid_case.peer_egress | default([]) }}" wikideck_image_pull_secrets: "{{ invalid_case.pull_secrets | default([]) }}" wikideck_network_policy_ingress_from: "{{ invalid_case.ingress_from | default([{'podSelector': {}}]) }}" block: diff --git a/tests/render.yml b/tests/render.yml index d45b5ef..5faebda 100644 --- a/tests/render.yml +++ b/tests/render.yml @@ -13,7 +13,9 @@ wikideck_image: "{{ render_case.image | default('registry.example.com/wikideck:1.0.0') }}" wikideck_name: "{{ render_case.name | default('wikideck') }}" wikideck_namespace: "{{ render_case.namespace | default('wikideck') }}" - wikideck_offline: "{{ render_case.offline | default(false) }}" + wikideck_peers: "{{ render_case.peers | default([]) }}" + wikideck_allow_private_peers: "{{ render_case.allow_private | default(false) }}" + wikideck_startup_probe_failure_threshold: "{{ render_case.startup_threshold | default(180) }}" wikideck_ingress_enabled: "{{ render_case.ingress | default(false) }}" wikideck_ingress_host: wikideck.example.com wikideck_public_origin: "{{ 'https://wikideck.example.com' if render_case.ingress | default(false) else '' }}" @@ -25,9 +27,12 @@ wikideck_network_policy_ingress_from: "{{ render_case.ingress_from | default([{'podSelector': {}}]) }}" wikideck_network_policy_dns_to: "{{ render_case.dns_to | default([{'namespaceSelector': {}}]) }}" wikideck_network_policy_https_to: "{{ render_case.https_to | default([{'ipBlock': {'cidr': '0.0.0.0/0'}}]) }}" + wikideck_network_policy_https_ports: "{{ render_case.https_ports | default([443]) }}" + wikideck_network_policy_peer_egress: "{{ render_case.peer_egress | default([]) }}" loop: - - label: defaults + - label: isolated validator defaults - label: TLS ingress and existing storage + peers: [https://peer-a.example.org, https://peer-b.example.org] ingress: true ingress_class: nginx annotations: @@ -37,15 +42,35 @@ pull_secrets: [registry-credentials] name: custom-deck namespace: games - - label: offline and no storage class - offline: true + - label: isolated validator and no storage class storage_class: "" - label: default-deny all traffic ingress_from: [] dns_to: [] https_to: [] - - label: DNS-only offline mode - offline: true + - label: public peers with custom HTTPS port and long startup + peers: [https://peer.example.org:8443] + https_ports: [443, 8443] + startup_threshold: 360 + - label: private HTTP testnet with restricted egress + peers: [http://other-node.games.svc.cluster.local:80] + allow_private: true + https_to: [] + peer_egress: + - to: + - namespaceSelector: + matchLabels: + kubernetes.io/metadata.name: games + podSelector: + matchLabels: + app.kubernetes.io/instance: other-node + ports: + - protocol: TCP + port: 3001 + - label: public peer with all egress explicitly denied + peers: [https://peer.example.org] + dns_to: [] + https_to: [] - label: digest-pinned image image: registry.example.com/wikideck@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa loop_control: @@ -84,11 +109,43 @@ resources: requests: {cpu: 100m, memory: 128Mi} - label: ambiguous boolean - offline: "false" + allow_private: "false" - label: wrong list type pull_secrets: not-a-list - label: empty peer allows all traffic ingress_from: [{}] + - label: too many peers + peers: [https://a.example.org, https://b.example.org, https://c.example.org, https://d.example.org, https://e.example.org, https://f.example.org, https://g.example.org, https://h.example.org, https://i.example.org, https://j.example.org, https://k.example.org, https://l.example.org, https://m.example.org, https://n.example.org, https://o.example.org, https://p.example.org, https://q.example.org] + - label: peers must be a list + peers: https://peer.example.org + - label: HTTP peers need explicit opt-in + peers: [http://peer.example.org] + - label: credentials in peer URL + peers: [https://user:password@peer.example.org] + - label: path in peer URL + peers: [https://peer.example.org/p2p] + - label: query in peer URL + peers: ['https://peer.example.org?x=1'] + - label: comma can inject a second peer + peers: ['https://peer.example.org,https://another.example.org'] + - label: zero startup threshold + startup_threshold: 0 + - label: invalid HTTPS egress port + https_ports: [65536] + - label: egress rule without destinations + peer_egress: + - ports: [{protocol: TCP, port: 3001}] + - label: egress rule without ports + peer_egress: + - to: [{podSelector: {}}] + - label: egress rule with an empty peer + peer_egress: + - to: [{}] + ports: [{protocol: TCP, port: 3001}] + - label: non-TCP peer egress + peer_egress: + - to: [{podSelector: {}}] + ports: [{protocol: UDP, port: 3001}] loop_control: loop_var: invalid_case label: "{{ invalid_case.label }}" diff --git a/tests/render_case.yml b/tests/render_case.yml index 9c924ea..52ba6c4 100644 --- a/tests/render_case.yml +++ b/tests/render_case.yml @@ -36,6 +36,8 @@ - container.resources == wikideck_resources - container.ports[0].containerPort == 3001 - container.startupProbe.httpGet.path == '/healthz' + - container.startupProbe.failureThreshold == wikideck_startup_probe_failure_threshold + - container.startupProbe.periodSeconds == 5 - container.readinessProbe.httpGet.path == '/healthz' - container.livenessProbe.httpGet.path == '/healthz' - pod.terminationGracePeriodSeconds >= 30 @@ -44,7 +46,12 @@ - container.volumeMounts[0].mountPath == '/app/data' - env.HOST == '0.0.0.0' and env.PORT == '3001' - env.WIKIDECK_DATA_DIR == '/app/data' and env.NODE_ENV == 'production' - - env.WIKIDECK_OFFLINE == ('1' if wikideck_offline else '0') + - env.WIKIDECK_PEERS == wikideck_peers | join(',') + - env.WIKIDECK_ALLOW_PRIVATE_PEERS == ('1' if wikideck_allow_private_peers else '0') + - env.WIKIDECK_OFFLINE is not defined + - env.RANDOMX_BINARY is not defined + - container.command is not defined and container.args is not defined + - pod.containers | length == 1 - (env.PUBLIC_ORIGIN | default('')) == wikideck_public_origin - wikideck_test_manifests.pvc.spec.accessModes == wikideck_storage_access_modes - wikideck_test_manifests.ingress.spec.tls[0].secretName == wikideck_tls_secret_name @@ -56,6 +63,8 @@ - wikideck_test_manifests.networkpolicy.spec.policyTypes == ['Ingress', 'Egress'] - wikideck_test_manifests.networkpolicy.spec.ingress | length == (1 if wikideck_network_policy_ingress_from | length > 0 else 0) - wikideck_test_manifests.networkpolicy.spec.egress | length == expected_egress_count | int + - wikideck_resources.requests.memory == '512Mi' + - wikideck_resources.limits.memory == '1Gi' vars: deployment: "{{ wikideck_test_manifests.deployment }}" pod: "{{ deployment.spec.template.spec }}" @@ -64,7 +73,29 @@ expected_claim: "{{ wikideck_existing_claim if wikideck_existing_claim | length > 0 else wikideck_name ~ '-data' }}" expected_egress_count: >- {{ (1 if wikideck_network_policy_dns_to | length > 0 else 0) - + (1 if not wikideck_offline and wikideck_network_policy_https_to | length > 0 else 0) }} + + (1 if wikideck_peers | length > 0 and wikideck_network_policy_https_to | length > 0 + and wikideck_network_policy_https_ports | length > 0 else 0) + + (wikideck_network_policy_peer_egress | length if wikideck_peers | length > 0 else 0) }} + +- name: Check public peer egress destinations and ports + ansible.builtin.assert: + that: + - public_rule.to == wikideck_network_policy_https_to + - public_rule.ports | map(attribute='port') | list == wikideck_network_policy_https_ports + - public_rule.ports | map(attribute='protocol') | unique | list == ['TCP'] + vars: + public_rule: "{{ wikideck_test_manifests.networkpolicy.spec.egress[1 if wikideck_network_policy_dns_to | length > 0 else 0] }}" + when: + - wikideck_peers | length > 0 + - wikideck_network_policy_https_to | length > 0 + - wikideck_network_policy_https_ports | length > 0 + +- name: Check explicit peer egress survives unchanged + ansible.builtin.assert: + that: wikideck_test_manifests.networkpolicy.spec.egress[-(wikideck_network_policy_peer_egress | length):] == wikideck_network_policy_peer_egress + when: + - wikideck_peers | length > 0 + - wikideck_network_policy_peer_egress | length > 0 - name: Check omitted default StorageClass ansible.builtin.assert: diff --git a/tests/runtime-smoke.mjs b/tests/runtime-smoke.mjs new file mode 100644 index 0000000..50174f6 --- /dev/null +++ b/tests/runtime-smoke.mjs @@ -0,0 +1,67 @@ +// Image-level compatibility test only: no mining, wallets, peers, or live-chain writes. +import assert from 'node:assert/strict'; +import { spawn, execFileSync } from 'node:child_process'; +import { once } from 'node:events'; +import { readFile, access } from 'node:fs/promises'; + +assert.equal(process.env.WIKIDECK_DATA_DIR, '/app/data'); +assert.equal(process.env.WIKIDECK_PEERS, ''); +assert.ok(process.env.RANDOMX_BINARY, 'Use the updated production image'); +execFileSync(process.env.RANDOMX_BINARY, ['--self-test'], { + timeout: 120_000, + stdio: 'inherit', +}); + +async function runNode() { + const child = spawn(process.execPath, ['--experimental-strip-types', 'server/index.ts'], { + cwd: '/app', + env: process.env, + stdio: ['ignore', 'pipe', 'pipe'], + }); + const exited = once(child, 'exit'); + let output = ''; + const timeout = setTimeout(() => child.kill('SIGKILL'), 120_000); + try { + await new Promise((resolve, reject) => { + child.on('error', reject); + child.once('exit', (code) => reject(new Error(`Node exited before ready: ${code}\n${output}`))); + child.stderr.on('data', (chunk) => { output = (output + chunk).slice(-8192); }); + child.stdout.on('data', (chunk) => { + output = (output + chunk).slice(-8192); + if (output.includes('Wikideck RandomX testnet:')) resolve(); + }); + }); + for (const path of ['/healthz', '/p2p/status', '/api/peers']) { + const response = await fetch(`http://127.0.0.1:3001${path}`, { + signal: AbortSignal.timeout(10_000), + }); + assert.equal(response.status, 200, path); + const body = await response.json(); + if (path === '/healthz') { + assert.equal(body.status, 'ok'); + assert.equal(body.network, 'wikideck-randomx-testnet-2'); + } else if (path === '/p2p/status') { + assert.equal(body.network, 'wikideck-randomx-testnet-2'); + assert.equal(body.size, 0); + assert.match(body.chainId, /^[a-f0-9]{64}$/); + } else assert.deepEqual(body.peers, []); + } + await access('/app/data/testnet-node.lock'); + const ledger = JSON.parse(await readFile('/app/data/testnet/ledger.json', 'utf8')); + assert.equal(ledger.network, 'wikideck-randomx-testnet-2'); + assert.deepEqual(ledger.blocks, []); + child.kill('SIGTERM'); + assert.deepEqual(await exited, [0, null], output); + await assert.rejects(access('/app/data/testnet-node.lock'), { code: 'ENOENT' }); + } finally { + clearTimeout(timeout); + if (child.exitCode === null && child.signalCode === null) { + child.kill('SIGKILL'); + await exited; + } + } +} + +await runNode(); +await runNode(); +console.log('PASS: native RandomX, node API, graceful lock cleanup and empty-ledger restart');