p2p
This commit is contained in:
@@ -5,10 +5,13 @@ Deploy the Wikideck production container to an existing Kubernetes cluster using
|
||||
volume claim, a dedicated ServiceAccount, a ClusterIP Service, and a NetworkPolicy.
|
||||
An HTTPS Ingress is optional and disabled by default.
|
||||
|
||||
**Wikideck is a prototype, not a hardened public service.** This role does not add
|
||||
application authentication, distributed consensus, a database, or wallet recovery.
|
||||
Use test wallets and restrict access. Mining happens in the browser; the server
|
||||
coordinates the chain and verifies proofs.
|
||||
**Wikideck is an experimental RandomX testnet, not a production-ready public
|
||||
mainnet.** The updated application has independent validating nodes, fork choice,
|
||||
native NFTs, and encrypted wallet backup/recovery. It supplies no public bootnodes
|
||||
or deployed public network. Use test wallets and no valuable assets. This role
|
||||
deploys a **validator**, never a miner: mining is an explicitly started native CLI
|
||||
process outside the browser. Private keys, backups and passwords must stay off
|
||||
validator pods and out of Ansible variables.
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -24,6 +27,10 @@ coordinates the chain and verifies proofs.
|
||||
- **A published image built from Wikideck's production Dockerfile.** There is no
|
||||
assumed official registry image. Set `wikideck_image` to your actual tagged or
|
||||
digest-pinned image. The role does not build/push images or install a cluster.
|
||||
Rebuild from the current source: the image must include Node 24, libstdc++, and
|
||||
the native RandomX executable, with `RANDOMX_BINARY` set by the image. An old
|
||||
browser-mining image is incompatible. Build and verify for your worker node
|
||||
architecture; do not assume a locally built image is multi-architecture.
|
||||
- For ingress: an ingress controller, DNS, and a trusted TLS Secret in the app
|
||||
namespace. Alternatively, configure cert-manager annotations and an existing
|
||||
issuer. The role does not install cert-manager or create an issuer.
|
||||
@@ -37,8 +44,8 @@ environment. Windows controllers must run Ansible through WSL or Linux container
|
||||
Install this checkout into your playbook's role search path as `roles/wikideck`,
|
||||
or install it using an Ansible Galaxy Git role requirement named `wikideck`.
|
||||
[examples/playbook.yml](examples/playbook.yml) shows an HTTPS deployment with
|
||||
ingress-nginx and cert-manager. Replace the image, hostname, class, issuer, and
|
||||
controller namespace with real values before running it.
|
||||
ingress-nginx and cert-manager. Replace the image, hostname, peer URLs, class,
|
||||
issuer, and controller namespace with real values before running it.
|
||||
|
||||
```yaml
|
||||
- name: Deploy a cluster-internal Wikideck instance
|
||||
@@ -67,6 +74,64 @@ for browser wallet cryptography. Configure HTTP-to-HTTPS redirect at your ingres
|
||||
controller; the example includes the ingress-nginx annotation. Ingress creation
|
||||
does not wait for DNS, certificate issuance, or an externally reachable endpoint.
|
||||
|
||||
### Connect validators
|
||||
|
||||
Set `wikideck_peers` to a list of up to 16 compatible peer HTTPS origins, such as
|
||||
`https://peer-a.example.org`. The role joins this list into `WIKIDECK_PEERS`.
|
||||
An empty list deliberately starts an **isolated validator**. There is no automatic
|
||||
discovery, bootnode list, or automatic reciprocal configuration: configure the
|
||||
other operators' nodes too. All nodes must run the same network/genesis/consensus.
|
||||
|
||||
Peer RPC uses `/p2p/*` on the **same port 3001** as the UI/API; no separate TCP/UDP
|
||||
P2P Service or host networking is required. The existing `/` ingress path forwards
|
||||
those routes. Keep peer JSON responses uncompressed and do not redirect HTTPS
|
||||
peer requests: the client rejects redirects, non-JSON and compressed responses.
|
||||
An interactive login gateway on `/p2p/*` will break the current unauthenticated
|
||||
peer protocol; use deliberate source restrictions/abuse controls instead.
|
||||
|
||||
Public peers must use HTTPS and public DNS/IP addresses. Wikideck validates every
|
||||
connection's DNS answers and pins the connection to that validated result. Never
|
||||
disable TLS verification to work around a certificate or split-horizon DNS error.
|
||||
The default policy allows public TCP 443 only when peers are configured. For
|
||||
HTTPS peer origins on another port, add that port to
|
||||
`wikideck_network_policy_https_ports`, or supply narrowly scoped explicit egress
|
||||
rules. Browser Wikimedia image fetches are client-side; validators no longer
|
||||
fetch random Wikipedia articles.
|
||||
|
||||
For an explicitly private testnet only, set `wikideck_allow_private_peers: true`
|
||||
and provide `wikideck_network_policy_peer_egress` rules with destination selectors
|
||||
and numeric TCP ports. This opt-in permits **both private addresses and HTTP** in
|
||||
the application; it does not automatically open network policy. For example:
|
||||
|
||||
```yaml
|
||||
wikideck_peers:
|
||||
- http://other-node.games.svc.cluster.local:80
|
||||
wikideck_allow_private_peers: true
|
||||
wikideck_network_policy_https_to: []
|
||||
wikideck_network_policy_peer_egress:
|
||||
- to:
|
||||
- namespaceSelector:
|
||||
matchLabels:
|
||||
kubernetes.io/metadata.name: games
|
||||
podSelector:
|
||||
matchLabels:
|
||||
app.kubernetes.io/instance: other-node
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 3001
|
||||
```
|
||||
|
||||
The receiving node must separately allow the sender through its ingress policy.
|
||||
Service DNAT handling depends on the CNI; the example selects the destination pod
|
||||
and its target port 3001 rather than opening all destinations on Service port 80.
|
||||
Do not enable private-peer exceptions for public peer configuration.
|
||||
|
||||
Deploy multiple validators as **separate role instances with different names and
|
||||
PVCs** (and separate hostnames if using ingress), not by scaling one Deployment.
|
||||
Sharing a machine/cluster/storage failure domain does not provide independent
|
||||
operator security. Empty local chains can synchronize from configured peers;
|
||||
never seed a second running node by sharing its writable PVC.
|
||||
|
||||
## Variables
|
||||
|
||||
All defaults are in [defaults/main.yml](defaults/main.yml).
|
||||
@@ -78,10 +143,12 @@ All defaults are in [defaults/main.yml](defaults/main.yml).
|
||||
| `wikideck_namespace` | `wikideck` | Namespace for all namespaced resources. |
|
||||
| `wikideck_create_namespace` | `true` | Create the namespace; disable for a pre-provisioned namespace. |
|
||||
| `wikideck_kubeconfig` / `wikideck_kube_context` | `null` | Override controller authentication/context. API TLS verification remains enabled. |
|
||||
| `wikideck_wait` / `wikideck_wait_timeout` | `true` / `300` | Wait for Deployment readiness, timeout in seconds. |
|
||||
| `wikideck_wait` / `wikideck_wait_timeout` | `true` / `900` | Wait for Deployment readiness, timeout in seconds. |
|
||||
| `wikideck_startup_probe_failure_threshold` | `180` | Failed probes at 5-second intervals before startup restart; default budget 900 seconds. |
|
||||
| `wikideck_image_pull_policy` | `IfNotPresent` | `Always`, `IfNotPresent`, or `Never`. |
|
||||
| `wikideck_image_pull_secrets` | `[]` | Names of existing registry Secrets in this namespace. |
|
||||
| `wikideck_offline` | `false` | Starter articles without live Wikipedia requests. |
|
||||
| `wikideck_peers` | `[]` | Up to 16 static peer origins; empty means isolated. |
|
||||
| `wikideck_allow_private_peers` | `false` | Explicit test-only private/HTTP peer permission. |
|
||||
| `wikideck_public_origin` | empty | Additional allowed HTTPS browser origin; required with ingress. |
|
||||
| `wikideck_ingress_enabled` | `false` | Create TLS ingress; disabling removes the role's named ingress. |
|
||||
| `wikideck_ingress_host` | empty | DNS hostname, without scheme, port, or path. |
|
||||
@@ -92,12 +159,19 @@ All defaults are in [defaults/main.yml](defaults/main.yml).
|
||||
| `wikideck_storage_size` | `1Gi` | Requested PVC capacity; resizing requires StorageClass support. |
|
||||
| `wikideck_storage_class` | `null` | `null`: cluster default; empty string: no class; otherwise a class name. |
|
||||
| `wikideck_storage_access_modes` | `[ReadWriteOnce]` | `ReadWriteOnce` or `ReadWriteOncePod`; never `ReadWriteMany`. |
|
||||
| `wikideck_resources` | see defaults | CPU/memory requests `100m`/`128Mi`, limits `1`/`512Mi`. |
|
||||
| `wikideck_resources` | see defaults | CPU/memory requests `250m`/`512Mi`, limits `1`/`1Gi`; small-validator starting point, not a capacity guarantee. |
|
||||
| `wikideck_node_selector` / `wikideck_tolerations` / `wikideck_affinity` | `{}` / `[]` / `{}` | Pod placement settings. |
|
||||
| `wikideck_network_policy_enabled` | `true` | Manage network isolation; disabling deletes this role's policy. |
|
||||
| `wikideck_network_policy_ingress_from` | same-namespace pods | List of NetworkPolicy peers allowed to reach TCP 3001; empty list denies all ingress. |
|
||||
| `wikideck_network_policy_dns_to` | `kube-system` namespace | DNS peers allowed on UDP/TCP 53; adapt for custom/NodeLocal DNS. |
|
||||
| `wikideck_network_policy_https_to` | public IPv4/IPv6 ranges, private/link-local excluded | TCP 443 egress peers; omitted in offline mode. |
|
||||
| `wikideck_network_policy_https_to` | public IPv4/IPv6 ranges, private/link-local excluded | Public peer destinations; omitted when there are no configured peers. |
|
||||
| `wikideck_network_policy_https_ports` | `[443]` | Public peer TCP ports; extend for custom HTTPS origins. |
|
||||
| `wikideck_network_policy_peer_egress` | `[]` | Additional explicit `to` + numeric TCP `ports` rules, emitted only with configured peers. |
|
||||
|
||||
`wikideck_offline` / `WIKIDECK_OFFLINE` have been removed. The current catalog is
|
||||
fixed and genesis-bound, not a live Wikipedia feed. Remove the old role variable;
|
||||
it no longer controls networking. The image's packaged `RANDOMX_BINARY` and default
|
||||
entrypoint are deliberately preserved; no production fake-PoW fallback is added.
|
||||
|
||||
## Storage and safe updates
|
||||
|
||||
@@ -108,7 +182,11 @@ All defaults are in [defaults/main.yml](defaults/main.yml).
|
||||
`ReadWriteOncePod` if supported by your CSI driver. Do not force-delete an old
|
||||
pod on a partitioned node until it has been fenced; Recreate is not a
|
||||
distributed storage lock.
|
||||
- `/app/data` contains the chain and signed transfer journal. The process runs
|
||||
- `/app/data/testnet/ledger.json` stores the block DAG and pending transfers;
|
||||
`/app/data/testnet-node.lock` exclusively locks the entire data directory.
|
||||
The old demo chain is separate, not imported, and has **no migration** to the
|
||||
new testnet. Back it up first; preferably use a fresh claim for the new network.
|
||||
The process runs
|
||||
as UID/GID 1000 with `fsGroup: 1000`. Your storage driver must honor ownership
|
||||
settings, or the existing volume must already be writable by this identity.
|
||||
No privileged/root ownership-fixing init container is used.
|
||||
@@ -121,9 +199,37 @@ All defaults are in [defaults/main.yml](defaults/main.yml).
|
||||
backup or use your CSI provider's documented snapshot procedure. Encrypt disks
|
||||
and backups, define retention, and test restores. Deleting a namespace or PVC
|
||||
can still destroy data depending on the underlying reclaim policy.
|
||||
- Chain verification runs at startup. For large chains, review resources,
|
||||
startup probe budget (300 seconds), and readiness wait timeout. JSON storage
|
||||
is unbounded; this role cannot solve Wikideck's persistence scalability limits.
|
||||
- Full native RandomX revalidation runs before the server listens. Increase
|
||||
`wikideck_startup_probe_failure_threshold` and `wikideck_wait_timeout` together
|
||||
for your measured replay time, leaving scheduling/image-pull margin. A healthy
|
||||
empty node does not exercise the native cache or prove peer synchronization.
|
||||
- The app bounds the ledger to 128 MiB and 10,000 total DAG blocks, with 256
|
||||
pending transfers. Complete NFT states are retained per DAG node, so RAM may
|
||||
grow quadratically despite disk bounds. Whole-ledger rewrites and full replay
|
||||
are **not suitable for a long-lived public chain**; a larger PVC does not fix
|
||||
those application limits.
|
||||
|
||||
### Crash recovery and resource sizing
|
||||
|
||||
The application removes its lock on graceful shutdown but deliberately does
|
||||
**not** automatically recover a stale lock after an OOM kill, forced termination,
|
||||
node loss, or interruption during startup replay. Kubernetes restarts can remain
|
||||
in CrashLoopBackOff until an operator intervenes. This role never deletes the lock
|
||||
in an init container or as an automatic "repair".
|
||||
|
||||
To recover, first stop the Deployment and verify every former owner is gone;
|
||||
fence an unreachable old node before mounting the claim elsewhere. Take a backup,
|
||||
then remove only the confirmed stale lock using an approved maintenance pod or
|
||||
storage procedure. Preserve the ledger, restore the Deployment to one replica,
|
||||
and inspect replay and peer status. Do not automatically remove locks based on
|
||||
their PID: PIDs are reused and isolated between containers.
|
||||
|
||||
Each validator uses native **light-mode RandomX**, requiring a 256 MiB cache plus
|
||||
scratchpad, Node heap, ledger states, and process overhead. The new 1 GiB limit
|
||||
matches the updated upstream Compose starting budget, but must be measured against
|
||||
your ledger, peer load and replay. Watch OOM events, CPU throttling, replay time,
|
||||
and block propagation. Fast miners need **over 2.3 GiB per process** and separate
|
||||
capacity/security planning; this role never launches one or mounts its wallet.
|
||||
|
||||
## Security and networking
|
||||
|
||||
@@ -135,14 +241,14 @@ temporary directory are writable. All three HTTP probes target `/healthz` on por
|
||||
period accommodates the application's 25-second shutdown deadline.
|
||||
|
||||
NetworkPolicy defaults allow same-namespace clients, cluster DNS in `kube-system`,
|
||||
and public HTTPS for Wikipedia. With ingress, explicitly allow the ingress
|
||||
and public HTTPS only when peer origins are configured. With ingress, allow the ingress
|
||||
controller's namespace/pods as shown in the example. Peer entries are additive
|
||||
OR alternatives: put a namespaceSelector and podSelector in the **same entry** to
|
||||
restrict pods within that namespace. Empty peer lists deny that traffic; empty
|
||||
peer objects are rejected to avoid accidentally allowing every source/destination.
|
||||
|
||||
Standard NetworkPolicy cannot allow Wikipedia by hostname; the default public
|
||||
TCP 443 egress is broader than a domain allowlist. Use CNI FQDN policies or an
|
||||
Standard NetworkPolicy cannot allow peer origins by hostname; public
|
||||
TCP 443 egress is broader than the app's fixed peer allowlist. Narrow CIDRs or use CNI FQDN policies or an
|
||||
egress gateway if required. Policies are additive with any other policies in the
|
||||
namespace and have no effect without an enforcing CNI. NodeLocal DNS, host-network
|
||||
ingress controllers, service meshes, and private proxies may need peer changes.
|
||||
@@ -155,16 +261,28 @@ API certificate verification. The role does not embed secrets or implement its
|
||||
own cryptography; image signature verification and encrypted storage are cluster
|
||||
operator responsibilities. Scan and rebuild the application image regularly.
|
||||
|
||||
Native RandomX uses ordinary memory allocations and secure W^X JIT; the current
|
||||
app does not request huge pages, host devices, executable files in `/tmp`, or
|
||||
privileged capabilities. Keep the existing hardening. Verify actual hashing on
|
||||
your worker architecture and runtime security profile; HTTP health alone cannot
|
||||
detect every native engine failure. Do not grant privileged mode or disable
|
||||
seccomp as an untested performance workaround.
|
||||
|
||||
Wikideck does not currently configure Express `trust proxy`. Clients behind one
|
||||
proxy can therefore share an IP-based application rate limit. Review the
|
||||
application before public exposure; do not try to fix this by blindly trusting
|
||||
forwarded headers. The role adds no authentication, WAF, or abuse protection.
|
||||
forwarded headers. The current app limits `/api` and `/p2p` together; default
|
||||
forwarded-header behavior can also produce express-rate-limit validation errors
|
||||
in logs. The role adds no authentication, WAF, or abuse protection. Eclipse,
|
||||
majority-hashpower, spam/DoS and reorganization risks remain application/protocol
|
||||
issues; deployment hardening does not make a public mainnet safe.
|
||||
|
||||
## Validation
|
||||
|
||||
[tests/render.yml](tests/render.yml) runs entirely offline: it validates inputs,
|
||||
renders/parses every template across default, ingress, existing-claim, storage
|
||||
class, offline, deny-all, and digest-image variants, and checks wiring/security
|
||||
class, isolated/public/private peers, custom HTTPS ports, deny-all, longer startup,
|
||||
and digest-image variants, and checks wiring/security
|
||||
invariants. It also checks that unsafe/invalid settings are rejected. Run it with
|
||||
`ansible-playbook -i tests/inventory tests/render.yml`.
|
||||
|
||||
@@ -173,8 +291,19 @@ checkout. Syntax-check it with `ansible-playbook -i tests/inventory tests/test.y
|
||||
To deploy to a disposable test namespace, run it with
|
||||
`ansible-playbook -i tests/inventory tests/test.yml -e wikideck_image=YOUR_IMAGE -e wikideck_namespace=wikideck-test`.
|
||||
Run it a second time to verify idempotence. Check Pod readiness, `/healthz`, PVC
|
||||
retention across updates, DNS/HTTPS egress, TLS, and browser mining/transfer flows
|
||||
on your chosen cluster. Offline tests do not prove CSI/CNI/controller behavior.
|
||||
retention across updates, DNS/peer egress, TLS, and wallet/transfer flows on your
|
||||
chosen cluster. `/healthz` is only process health, not sync readiness. Inspect
|
||||
`/api/peers` for connected peers and `/p2p/status` for matching network, chainId and
|
||||
tip across independently deployed nodes. `/api/peers` exposes configuration;
|
||||
apply your access policy deliberately. Offline tests do not prove CSI/CNI/controller
|
||||
behavior or consensus safety. Mining a real block is a separate explicit-consent
|
||||
test; it must not be part of routine deployment or health probes.
|
||||
|
||||
[tests/runtime-smoke.mjs](tests/runtime-smoke.mjs) is a disposable image-level
|
||||
check: it runs the native known-vector self-test (hashing, **not mining**), checks
|
||||
node API health, and verifies graceful lock cleanup and restart durability. Mount
|
||||
it read-only and invoke Node inside the current image with disposable writable
|
||||
data and no network. It does not use or modify a real chain or wallet.
|
||||
|
||||
Ansible check mode (`--check --diff`) is supported by the Kubernetes module, but
|
||||
still needs API access; creating a new namespace in check mode may require a
|
||||
|
||||
Reference in New Issue
Block a user