Production

This commit is contained in:
2026-11-01 17:11:07 -05:00
parent 885f4160a7
commit dc194b8168
7 changed files with 362 additions and 40 deletions
+48 -3
View File
@@ -81,7 +81,7 @@ An empty list deliberately starts an **isolated validator**. There is no automat
discovery, bootnode list, or automatic reciprocal configuration: configure the
other operators' nodes too. All nodes must run the same network/genesis/consensus.
For mainnet, application startup additionally requires at least two distinct
public HTTPS peer origins, an arrived image-pinned genesis time, the exact chain-ID
peer origins (public HTTPS by default), an arrived image-pinned genesis time, the exact chain-ID
approval, and explicit unaudited-risk acceptance. These switches are not launch
approval from this role; the project currently records a NO-GO decision. The role
does not discover or verify peer independence.
@@ -102,7 +102,8 @@ HTTPS peer origins on another port, add that port to
rules. Browser Wikimedia image fetches are client-side; validators no longer
fetch random Wikipedia articles.
For an explicitly private testnet only, set `wikideck_allow_private_peers: true`
For an explicitly private testnet or operator-controlled mainnet candidate cluster,
set `wikideck_allow_private_peers: true`
and provide `wikideck_network_policy_peer_egress` rules with destination selectors
and numeric TCP ports. This opt-in permits **both private addresses and HTTP** in
the application; it does not automatically open network policy. For example:
@@ -128,7 +129,12 @@ wikideck_network_policy_peer_egress:
The receiving node must separately allow the sender through its ingress policy.
Service DNAT handling depends on the CNI; the example selects the destination pod
and its target port 3001 rather than opening all destinations on Service port 80.
Do not enable private-peer exceptions for public peer configuration.
Do not enable private-peer exceptions for public peer configuration. Updated
mainnet candidate images advertise `privatePeersSupported: true` in their manifest;
older images reject the private opt-in. Browser HTTPS, chain approval, risk
acknowledgment and arrived genesis remain mandatory. Private HTTP RPC is not
authenticated or encrypted; keep it within a trusted cluster boundary or add
independently configured mesh transport protection.
Deploy multiple validators as **separate role instances with different names and
PVCs** (and separate hostnames if using ingress), not by scaling one Deployment.
@@ -136,6 +142,45 @@ Sharing a machine/cluster/storage failure domain does not provide independent
operator security. Empty local chains can synchronize from configured peers;
never seed a second running node by sharing its writable PVC.
### Three-node latest candidate playbook
[playbooks/wikideck.yaml](playbooks/wikideck.yaml) configures the public `wikideck`
node plus `wikideck-peer-a` and `wikideck-peer-b` within the same namespace. Each
uses its own Service and 2 TiB PVC (6 TiB requested in total), reciprocal internal
DNS peer URLs, scoped validator-pod ingress/egress, and public TCP 443 egress for
Wikipedia article and Wikimedia statistics requests used by mining templates.
Standard NetworkPolicy cannot select FQDNs; private address ranges remain excluded
from that public HTTPS rule. Only the public node gets
the `wikideck.eom.dev` TLS ingress. All three are created before readiness waits.
This deploys validators, not miners, and does not establish independent operators.
The playbook uses this role checkout directly, not a separately installed older
Galaxy role. It defaults to `ericomeehan/wikideck:latest` with pull policy `Always`
and `wikideck_candidate_risk_ack: operator-accepts-unaudited-risk`. This explicit
candidate workflow is not the digest-pinned public-launch example. Before creating
ledger resources, a short-lived hardened Kubernetes Job runs the image's
`--network-manifest` command. The playbook generates `wikideck_candidate_chain_id`
from its output and rejects older images without private-peer support, non-mainnet
images, or a future image-baked genesis. The Job is removed even on failure; it
mounts no ledger, wallet secrets, or service-account token.
Rebuild and publish `latest` from the updated application before running this
playbook. The default Dockerfile genesis is November 1, 2026, 00:00 UTC
(`1793491200`), so a normal rebuild can start now without custom genesis arguments.
Older images retain their old genesis; publishing updated source is required.
Never change the chain identity of an existing ledger without a deliberate backup
and migration plan. Do not move `latest` while inspection/deployment is in progress:
all nodes must pull the same image. For reproducible public releases use the
digest-pinned [examples/playbook.yml](examples/playbook.yml) instead. A mutable tag
with `Always` is pulled when pods start. The playbook records the inspected
container image ID in a pod-template annotation, so rerunning after publishing a
new `latest` triggers a rollout; unchanged image IDs do not force a restart.
Preserve existing data and keep the same genesis for updates to an operating chain.
The controller needs Job creation/deletion and Pod/log reads in addition to the
role's permissions. Manifest inspection needs a live cluster, so this playbook
rejects check mode; the role's offline render tests require no cluster credentials.
## Variables
All defaults are in [defaults/main.yml](defaults/main.yml).
+3 -1
View File
@@ -19,7 +19,7 @@ wikideck_image_pull_secrets: []
# Static peer origins, at most 16. Empty means an isolated validator, not a network.
wikideck_peers: []
# Test/private networks only; permits private addresses AND HTTP in the app.
# Operator-controlled test/private candidate clusters; permits private addresses AND HTTP.
wikideck_allow_private_peers: false
# Explicit mainnet candidate startup gates. Never set these before launch approval.
wikideck_mainnet_launch: ""
@@ -51,6 +51,8 @@ wikideck_resources:
wikideck_node_selector: {}
wikideck_tolerations: []
wikideck_affinity: {}
# Pod-template annotations can trigger a rollout when a mutable image tag changes.
wikideck_pod_annotations: {}
# Trust only literal proxy source IPs/CIDRs whose edge overwrites forwarded headers.
wikideck_trusted_proxies: []
# Optional private operations listener; application binds it to container loopback.
+258 -35
View File
@@ -1,45 +1,268 @@
- name: Deploy Wikideck
- name: Deploy a three-node private Wikideck mainnet candidate
hosts: alpha-control-plane
become: true
vars_files:
- ~/Desktop/secrets.yaml
vars:
# This is an operator-controlled candidate, not an independent public network.
wikideck_release_image: ericomeehan/wikideck:latest
wikideck_candidate_risk_ack: operator-accepts-unaudited-risk
wikideck_cluster_nodes:
- wikideck
- wikideck-peer-a
- wikideck-peer-b
wikideck_namespace: wikideck
wikideck_image: "{{ wikideck_release_image }}"
wikideck_image_pull_policy: Always
# Roll out latest only when the inspected image changes, not on every run.
wikideck_pod_annotations:
wikideck.eom.dev/image-id: "{{ wikideck_manifest_image_id }}"
wikideck_allow_private_peers: true
wikideck_mainnet_launch: "{{ wikideck_candidate_chain_id }}"
wikideck_mainnet_risk_ack: "{{ wikideck_candidate_risk_ack }}"
# Internal validators share the browser origin, but only the first has ingress.
wikideck_public_origin: https://wikideck.eom.dev
wikideck_ingress_host: wikideck.eom.dev
wikideck_ingress_class_name: nginx
wikideck_ingress_annotations:
cert-manager.io/cluster-issuer: ca-issuer
nginx.ingress.kubernetes.io/ssl-redirect: "true"
wikideck_storage_size: 2Ti
wikideck_storage_class: nfs-client
# Populate only after confirming the real proxy source IPs/CIDRs.
wikideck_trusted_proxies: "{{ wikideck_ingress_proxy_sources | default([]) }}"
# Start every node before waiting; the first must not wait for absent peers.
wikideck_wait: false
wikideck_wait_timeout: 900
wikideck_network_policy_ingress_from:
- podSelector:
matchLabels:
app.kubernetes.io/name: wikideck
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-nginx
# Mining templates fetch Wikipedia articles and Wikimedia statistics over HTTPS.
# Standard NetworkPolicy cannot select FQDNs; permit public TCP 443, not private ranges.
wikideck_network_policy_https_to:
- ipBlock:
cidr: 0.0.0.0/0
except:
- 10.0.0.0/8
- 100.64.0.0/10
- 127.0.0.0/8
- 169.254.0.0/16
- 172.16.0.0/12
- 192.168.0.0/16
- ipBlock:
cidr: "::/0"
except:
- "::1/128"
- "fc00::/7"
- "fe80::/10"
wikideck_network_policy_peer_egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/name: wikideck
ports:
- protocol: TCP
port: 3001
pre_tasks:
- name: Require an explicitly approved, immutable mainnet release
- name: Validate candidate cluster inputs
ansible.builtin.assert:
that:
- wikideck_mainnet_deployment_approved | default(false) | bool
- wikideck_release_image is defined and wikideck_release_image is string and wikideck_release_image is match('^.+@sha256:[a-f0-9]{64}$')
- wikideck_mainnet_peers is defined and wikideck_mainnet_peers is sequence and wikideck_mainnet_peers is not string and wikideck_mainnet_peers | length >= 2
- wikideck_mainnet_peers | unique | list | length == wikideck_mainnet_peers | length
- wikideck_candidate_chain_id is defined and wikideck_candidate_chain_id is match('^[a-f0-9]{64}$')
- not ansible_check_mode
- wikideck_release_image is string and wikideck_release_image is match('^[^\s@]+:latest$')
- wikideck_cluster_nodes is sequence and wikideck_cluster_nodes is not string and wikideck_cluster_nodes is not mapping
- wikideck_cluster_nodes | length == 3
- wikideck_cluster_nodes | unique | list | length == 3
- wikideck_cluster_nodes | select('match', '^[a-z]([a-z0-9-]*[a-z0-9])?$') | list | length == 3
- wikideck_cluster_nodes | map('length') | max <= 50
- wikideck_candidate_risk_ack | default('') == 'operator-accepts-unaudited-risk'
- wikideck_release_genesis_time is defined and (wikideck_release_genesis_time | string) is match('^[1-9][0-9]{8,10}$')
- wikideck_release_genesis_time | int >= 1700000000 and wikideck_release_genesis_time | int <= 4102444800
- wikideck_release_genesis_time | int <= ansible_date_time.epoch | int
fail_msg: >-
Mainnet is currently NO-GO. Do not deploy until the operator has approved
a digest-pinned image with a verified frozen manifest, arrived genesis,
at least two independent public HTTPS peers, and explicit risk acceptance.
roles:
- role: ericomeehan.wikideck
This candidate playbook requires a latest-tag image, three unique node
names, and operator-accepts-unaudited-risk. Check mode cannot inspect the
image manifest; use the offline render tests instead.
run_once: true
- name: Initialize manifest registry Secret references
ansible.builtin.set_fact:
wikideck_manifest_pull_secrets: []
run_once: true
- name: Convert registry Secret names to Kubernetes references
ansible.builtin.set_fact:
wikideck_manifest_pull_secrets: "{{ wikideck_manifest_pull_secrets + [{'name': item}] }}"
loop: "{{ wikideck_image_pull_secrets | default([]) }}"
run_once: true
- name: Create namespace for read-only image inspection
kubernetes.core.k8s:
kubeconfig: "{{ wikideck_kubeconfig | default(omit, true) }}"
context: "{{ wikideck_kube_context | default(omit, true) }}"
state: present
apply: true
definition:
apiVersion: v1
kind: Namespace
metadata:
name: "{{ wikideck_namespace }}"
run_once: true
- name: Inspect latest without opening a ledger or starting a validator
run_once: true
block:
- name: Remove a stale manifest inspection Job
kubernetes.core.k8s:
kubeconfig: "{{ wikideck_kubeconfig | default(omit, true) }}"
context: "{{ wikideck_kube_context | default(omit, true) }}"
state: absent
api_version: batch/v1
kind: Job
namespace: "{{ wikideck_namespace }}"
name: wikideck-candidate-manifest
wait: true
- name: Run the image manifest command
kubernetes.core.k8s:
kubeconfig: "{{ wikideck_kubeconfig | default(omit, true) }}"
context: "{{ wikideck_kube_context | default(omit, true) }}"
state: present
wait: true
wait_timeout: 300
wait_condition:
type: Complete
status: "True"
definition:
apiVersion: batch/v1
kind: Job
metadata:
name: wikideck-candidate-manifest
namespace: "{{ wikideck_namespace }}"
spec:
backoffLimit: 0
activeDeadlineSeconds: 240
ttlSecondsAfterFinished: 600
template:
spec:
restartPolicy: Never
automountServiceAccountToken: false
enableServiceLinks: false
imagePullSecrets: "{{ wikideck_manifest_pull_secrets }}"
securityContext:
runAsNonRoot: true
runAsUser: 1000
runAsGroup: 1000
seccompProfile:
type: RuntimeDefault
containers:
- name: manifest
image: "{{ wikideck_release_image }}"
imagePullPolicy: Always
command:
- node
- --experimental-strip-types
- server/index.ts
- --network-manifest
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
capabilities:
drop: [ALL]
resources:
requests: {cpu: 100m, memory: 128Mi}
limits: {cpu: "1", memory: 512Mi}
- name: Locate the completed manifest pod
kubernetes.core.k8s_info:
kubeconfig: "{{ wikideck_kubeconfig | default(omit, true) }}"
context: "{{ wikideck_kube_context | default(omit, true) }}"
kind: Pod
namespace: "{{ wikideck_namespace }}"
label_selectors:
- job-name=wikideck-candidate-manifest
register: wikideck_manifest_pods
- name: Require exactly one completed manifest pod
ansible.builtin.assert:
that:
- wikideck_manifest_pods.resources | length == 1
- wikideck_manifest_pods.resources[0].status.phase == 'Succeeded'
- name: Read the public image manifest
kubernetes.core.k8s_log:
kubeconfig: "{{ wikideck_kubeconfig | default(omit, true) }}"
context: "{{ wikideck_kube_context | default(omit, true) }}"
namespace: "{{ wikideck_namespace }}"
name: "{{ wikideck_manifest_pods.resources[0].metadata.name }}"
container: manifest
register: wikideck_manifest_log
- name: Parse the image identity
ansible.builtin.set_fact:
wikideck_candidate_manifest: "{{ wikideck_manifest_log.log | from_json }}"
- name: Require a compatible image and an arrived image-baked genesis
ansible.builtin.assert:
that:
- wikideck_candidate_manifest.profile == 'mainnet'
- wikideck_candidate_manifest.network == 'wikideck-randomx-mainnet-1'
- wikideck_candidate_manifest.privatePeersSupported | default(false) == true
- wikideck_candidate_manifest.chainId is string and wikideck_candidate_manifest.chainId is match('^[a-f0-9]{64}$')
- wikideck_candidate_manifest.genesisTime is integer
- wikideck_candidate_manifest.genesisTime >= 1700000000 and wikideck_candidate_manifest.genesisTime <= 4102444800
- wikideck_candidate_manifest.genesisTime <= ansible_date_time.epoch | int
fail_msg: >-
Rebuild and publish latest from code supporting private candidate
peers, with a deliberately chosen, arrived mainnet genesis frozen
into the image. Runtime genesis overrides are not supported.
- name: Generate the candidate chain approval from this image
ansible.builtin.set_fact:
wikideck_candidate_chain_id: "{{ wikideck_candidate_manifest.chainId }}"
wikideck_release_genesis_time: "{{ wikideck_candidate_manifest.genesisTime }}"
wikideck_manifest_image_id: "{{ wikideck_manifest_pods.resources[0].status.containerStatuses[0].imageID }}"
always:
- name: Remove the manifest inspection Job and its pod
kubernetes.core.k8s:
kubeconfig: "{{ wikideck_kubeconfig | default(omit, true) }}"
context: "{{ wikideck_kube_context | default(omit, true) }}"
state: absent
api_version: batch/v1
kind: Job
namespace: "{{ wikideck_namespace }}"
name: wikideck-candidate-manifest
wait: true
tasks:
- name: Deploy each validator with its own service and persistent ledger
ansible.builtin.include_role:
# Use this updated checkout rather than an older installed Galaxy role.
name: "{{ playbook_dir }}/.." # noqa: role-name[path]
vars:
wikideck_image: "{{ wikideck_release_image }}"
wikideck_image_pull_policy: IfNotPresent
wikideck_peers: "{{ wikideck_mainnet_peers }}"
wikideck_mainnet_launch: "{{ wikideck_candidate_chain_id }}"
wikideck_mainnet_risk_ack: "{{ wikideck_candidate_risk_ack }}"
wikideck_public_origin: https://wikideck.eom.dev
wikideck_ingress_enabled: true
wikideck_ingress_host: wikideck.eom.dev
wikideck_ingress_class_name: nginx
wikideck_ingress_annotations:
cert-manager.io/cluster-issuer: ca-issuer
# Match the actual ingress controller namespace; add its podSelector if known.
wikideck_network_policy_ingress_from:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-nginx
wikideck_storage_size: 2Ti
wikideck_storage_class: nfs-client
# Populate only after confirming the real proxy source IPs/CIDRs.
wikideck_trusted_proxies: "{{ wikideck_ingress_proxy_sources | default([]) }}"
wikideck_name: "{{ wikideck_cluster_node }}"
wikideck_existing_claim: ""
wikideck_ingress_enabled: "{{ wikideck_cluster_node == wikideck_cluster_nodes[0] }}"
wikideck_peers: >-
{{ wikideck_cluster_nodes | reject('equalto', wikideck_cluster_node)
| map('regex_replace', '^', 'http://')
| map('regex_replace', '$', '.' ~ wikideck_namespace ~ '.svc.cluster.local:80') | list }}
loop: "{{ wikideck_cluster_nodes }}"
loop_control:
loop_var: wikideck_cluster_node
run_once: true
- name: Wait for all three validators after creating the full peer topology
kubernetes.core.k8s_info:
kubeconfig: "{{ wikideck_kubeconfig | default(omit, true) }}"
context: "{{ wikideck_kube_context | default(omit, true) }}"
api_version: apps/v1
kind: Deployment
namespace: "{{ wikideck_namespace }}"
name: "{{ item }}"
wait: true
wait_timeout: "{{ wikideck_wait_timeout | int }}"
wait_condition:
type: Available
status: "True"
loop: "{{ wikideck_cluster_nodes }}"
run_once: true
+2 -1
View File
@@ -19,6 +19,7 @@
- wikideck_resources.limits.cpu is defined and wikideck_resources.limits.memory is defined
- wikideck_node_selector is mapping
- wikideck_affinity is mapping
- wikideck_pod_annotations is mapping
- wikideck_ingress_annotations is mapping
- wikideck_storage_size is string and wikideck_storage_size | length > 0
- wikideck_storage_access_modes | length > 0
@@ -95,7 +96,7 @@
- wikideck_allow_private_peers or item is match('^https://')
fail_msg: >-
Peers must be HTTPS origins without credentials, paths, commas, queries or
fragments. Private/HTTP test peers require wikideck_allow_private_peers=true.
fragments. Private/HTTP cluster peers require wikideck_allow_private_peers=true.
Wikideck additionally validates ports and public DNS/IP addresses at runtime.
loop: "{{ wikideck_peers }}"
+1
View File
@@ -20,6 +20,7 @@ spec:
app.kubernetes.io/instance: {{ wikideck_name | to_json }}
template:
metadata:
annotations: {{ wikideck_pod_annotations | to_json }}
labels:
app.kubernetes.io/name: wikideck
app.kubernetes.io/instance: {{ wikideck_name | to_json }}
+48
View File
@@ -13,6 +13,8 @@
wikideck_image: "{{ render_case.image | default('registry.example.com/wikideck:1.0.0') }}"
wikideck_name: "{{ render_case.name | default('wikideck') }}"
wikideck_namespace: "{{ render_case.namespace | default('wikideck') }}"
wikideck_image_pull_policy: "{{ render_case.pull_policy | default('IfNotPresent') }}"
wikideck_pod_annotations: "{{ render_case.pod_annotations | default({}) }}"
wikideck_peers: "{{ render_case.peers | default([]) }}"
wikideck_allow_private_peers: "{{ render_case.allow_private | default(false) }}"
wikideck_mainnet_launch: "{{ render_case.mainnet_launch | default('') }}"
@@ -84,6 +86,52 @@
- label: exact mainnet launch configuration
mainnet_launch: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
risk_ack: operator-accepts-unaudited-risk
- label: latest private mainnet candidate public node
image: ericomeehan/wikideck:latest
pull_policy: Always
pod_annotations:
wikideck.eom.dev/image-id: docker-pullable://ericomeehan/wikideck@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
peers: [http://wikideck-peer-a.wikideck.svc.cluster.local:80, http://wikideck-peer-b.wikideck.svc.cluster.local:80]
allow_private: true
ingress: true
mainnet_launch: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
risk_ack: operator-accepts-unaudited-risk
# Public HTTPS is needed for Wikipedia mining templates and statistics.
https_to:
- ipBlock:
cidr: 0.0.0.0/0
except: [10.0.0.0/8, 100.64.0.0/10, 127.0.0.0/8, 169.254.0.0/16, 172.16.0.0/12, 192.168.0.0/16]
- ipBlock:
cidr: "::/0"
except: ["::1/128", "fc00::/7", "fe80::/10"]
ingress_from:
- podSelector:
matchLabels:
app.kubernetes.io/name: wikideck
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: ingress-nginx
peer_egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/name: wikideck
ports: [{protocol: TCP, port: 3001}]
- label: latest private mainnet candidate internal peer
image: ericomeehan/wikideck:latest
pull_policy: Always
name: wikideck-peer-a
peers: [http://wikideck.wikideck.svc.cluster.local:80, http://wikideck-peer-b.wikideck.svc.cluster.local:80]
allow_private: true
mainnet_launch: aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
risk_ack: operator-accepts-unaudited-risk
https_to: []
peer_egress:
- to:
- podSelector:
matchLabels:
app.kubernetes.io/name: wikideck
ports: [{protocol: TCP, port: 3001}]
loop_control:
loop_var: render_case
label: "{{ render_case.label }}"
+2
View File
@@ -18,6 +18,7 @@
that:
- deployment.spec.replicas == 1
- deployment.spec.strategy.type == 'Recreate'
- deployment.spec.template.metadata.annotations == wikideck_pod_annotations
- deployment.spec.selector.matchLabels == deployment.spec.template.metadata.labels
- wikideck_test_manifests.service.spec.selector == deployment.spec.selector.matchLabels
- wikideck_test_manifests.networkpolicy.spec.podSelector.matchLabels == deployment.spec.selector.matchLabels
@@ -33,6 +34,7 @@
- not container.securityContext.allowPrivilegeEscalation
- container.securityContext.capabilities.drop == ['ALL']
- container.image == wikideck_image
- container.imagePullPolicy == wikideck_image_pull_policy
- container.resources == wikideck_resources
- container.ports[0].containerPort == 3001
- container.startupProbe.httpGet.path == '/healthz'