wild-cloud/wild-pc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rename Castle -> Wild PC across the repo

Browse Source 
Repo-side rename only (Phases 1-3 of the migration plan); the live box
(~/.castle, systemd units, /data/castle, domains) is a separate cutover.

- Slug `castle` -> `wildpc`: CLI command, module names (wildpc_core/cli/api),
  dist names, entry point `wildpc = wildpc_cli.main:main`.
- Identifiers: CastleConfig/NATSClient/DirError/MDNS -> Wildpc*.
- Env/constants: CASTLE_* -> WILDPC_*; ~/.castle -> ~/.wildpc, castle.yaml ->
  wildpc.yaml, /data/castle -> /data/wildpc.
- Systemd UNIT_PREFIX castle- -> wildpc-; own programs castle-api/gateway/etc.
- Display prose "Castle" -> "Wild PC" in docs, agent-guide files, README, frontend.
- Package dirs and bootstrap yaml renamed via git mv; lockfiles regenerated;
  redundant nested uv.lock files dropped (workspace root lock is authoritative).

Tests: core 273, cli 47, wildpc-api 120 all pass. Frontend type-checks + builds.
Fixed a stale test fixture (secret_env_path kind arg) broken pre-rename.
This commit is contained in:
Paul Payne
2026-07-18 22:55:08 -07:00
parent 25d7a522cc
commit 05b28cb584
190 changed files with 2428 additions and 3337 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
docs/fleet-mesh-plan.md
View File

@@ -7,7 +7,7 @@ consumed service) — every underlying piece is verified.
## Context
Castle's mesh today is a deliberately minimal, read-only gossip: each node
Wild PC's mesh today is a deliberately minimal, read-only gossip: each node
publishes a secret-stripped `NodeRegistry` to Mosquitto MQTT (retained) + an
online/offline LWT, and peers aggregate it into `MeshStateManager`. It is
LAN-only, unauthenticated, carries no secrets, and does nothing with what it
@@ -26,7 +26,7 @@ Two adopted, genuinely-open components (see licensing analysis in chat history):
**TTL keys as presence** (a provider's key vanishes on death → the breaker
trip signal, no separate health plumbing).
- **OpenBao** (Linux Foundation / OpenSSF, MPL-2.0, Vault fork) — the secret
authority behind castle's existing `${secret:...}` mechanism.
authority behind wildpc's existing `${secret:...}` mechanism.
Design stance: **static single-writer `role` authority, no consensus** —
availability comes from followers running cached local state, not failover.
@@ -36,11 +36,11 @@ NATS single-node matches that topology. Consensus stays explicitly out of scope.
| Plane | Component | Primary code seam (verified) |
|-------|-----------|------------------------------|
| Transport + KV + presence | NATS (`castle-nats`) | `castle-api/.../mqtt_client.py` (whole file), `main.py` lifespan 62-77, `config.py` Settings |
| Transport + KV + presence | NATS (`wildpc-nats`) | `wildpc-api/.../mqtt_client.py` (whole file), `main.py` lifespan 62-77, `config.py` Settings |
| Shared config + registry + presence | NATS JetStream KV | `mesh.py` `MeshStateManager`; new KV buckets |
| Secret authority | OpenBao (`castle-openbao`) | `core/.../config.py:307` `_read_secret` (single chokepoint), `castle-api/.../secrets.py` CRUD |
| Secret authority | OpenBao (`wildpc-openbao`) | `core/.../config.py:307` `_read_secret` (single chokepoint), `wildpc-api/.../secrets.py` CRUD |
| Binding-by-purpose + breaker | Caddy gateway | `generators/caddyfile.py` `compute_routes` (`remote` kind + ignored `remote_registries` are pre-stubbed), `deploy.py:509-540` `_target_url`/`_requires_env` (local-only today) |
| Authority role | `NodeConfig.role` | `castle_core/registry.py` / `manifest.py` |
| Authority role | `NodeConfig.role` | `wildpc_core/registry.py` / `manifest.py` |
## Phased plan
@@ -48,47 +48,47 @@ Each phase is independently useful; the risky secret-transport work is last.
### Phase 0 — Stand up the two services (no behavior change)
- **`castle-nats`**: a `SystemdDeployment` + `RunContainer` (`nats:2` with
JetStream `-js`), data volume `/data/castle/castle-nats`, `reach: internal`
- **`wildpc-nats`**: a `SystemdDeployment` + `RunContainer` (`nats:2` with
JetStream `-js`), data volume `/data/wildpc/wildpc-nats`, `reach: internal`
(`nats.<domain>`). Follow the container YAML shape in `docs/tcp-exposure.md`
(postgres example). Replace the Mosquitto provisioning in `install.sh`
(`setup_mqtt`, `seed_mosquitto_config`, image/port consts) with NATS — or
better, promote it to a bootstrap deployment YAML (today the broker is only a
shell-provisioned container, not a declared deployment).
- **`castle-openbao`**: `RunContainer` (`openbao/openbao`), data volume, a file
- **`wildpc-openbao`**: `RunContainer` (`openbao/openbao`), data volume, a file
storage backend, `reach: internal`, **auto-unseal** (local transit/key) so the
node boots unattended (Decision 2).
### Phase 1 — NATS as the mesh transport (swap Mosquitto, behavior-preserving)
- New `nats_client.py` (async-native `nats-py`) replacing `mqtt_client.py`.
Registry → a **JetStream KV bucket `castle-registry`** keyed by hostname
(KV last-value replaces MQTT retained). Presence → a **`castle-presence`**
Registry → a **JetStream KV bucket `wildpc-registry`** keyed by hostname
(KV last-value replaces MQTT retained). Presence → a **`wildpc-presence`**
bucket with a per-node **TTL key** the node renews (replaces LWT + the 300s
`STALE_TTL` poll; note `prune_stale` is currently uncalled).
- Wire in `main.py` lifespan; rename `mqtt_*` → `nats_*` in `config.py` (keep
`CASTLE_API_` prefix). `MeshStateManager` logic is reused; the KV **watch**
`WILDPC_API_` prefix). `MeshStateManager` logic is reused; the KV **watch**
callback drives `update_node`/`set_offline` — and because `nats-py` is
asyncio-native, the `run_coroutine_threadsafe` cross-thread hop for
`broadcast("mesh", …)` **goes away**.
- **Preserve the secret-stripping invariant** from `_registry_to_json` (env /
run_cmd / castle_root never on the wire).
run_cmd / wildpc_root never on the wire).
- Rename MQTT-named fields: `models.py` `MeshStatus` (217-226) + frontend
`types/index.ts` (326-330), `MeshPanel.tsx`. Drop `paho-mqtt`; add `nats-py`.
Keep zeroconf peer-advert for LAN discovery **and** support explicit NATS seed
URLs for cross-network (Decision 1); drop the dangling unused `_mqtt._tcp`
browse. Rewrite `test_mqtt.py` → `test_nats.py`.
- Fixes a latent gap: today the registry is published **on-connect only** (no
periodic/on-change republish). KV writes on `castle apply` fix this naturally.
periodic/on-change republish). KV writes on `wildpc apply` fix this naturally.
### Phase 2 — Shared config + presence via JetStream KV
- Buckets: `castle-config` (shared LAN config, **authority-written only**),
`castle-registry` (per-node), `castle-presence` (TTL liveness).
- Add static **`role: authority | follower`** to `NodeConfig` (castle.yaml).
- Buckets: `wildpc-config` (shared LAN config, **authority-written only**),
`wildpc-registry` (per-node), `wildpc-presence` (TTL liveness).
- Add static **`role: authority | follower`** to `NodeConfig` (wildpc.yaml).
Pin **`civil` = authority** (Decision 3); only the authority may write
`castle-config`. Authority-down ⇒ shared state read-only, nodes serve cached.
- Followers **watch** `castle-config` and reconcile via `castle apply`. Presence
`wildpc-config`. Authority-down ⇒ shared state read-only, nodes serve cached.
- Followers **watch** `wildpc-config` and reconcile via `wildpc apply`. Presence
key renewal is the churn signal for Phase 3.
### Phase 3 — Cross-node `requires` resolution + gateway binding + breaker (keystone)
@@ -102,7 +102,7 @@ Each phase is independently useful; the risky secret-transport work is last.
write at `deploy.py:157` (currently passed nothing). Binding = a **stable local
subdomain route** the gateway re-targets to the remote node's URL — the program
sees one unchanging URL.
- **Circuit-breaker:** gate the remote route on the `castle-presence` key
- **Circuit-breaker:** gate the remote route on the `wildpc-presence` key
(regenerate + `_reload_gateway` when a provider appears/vanishes), backed by
Caddy passive health (`fail_duration` / `lb_try_duration`) on the
`reverse_proxy` line in `_host_matcher_block`. Consumer degraded-mode =
@@ -112,9 +112,9 @@ Each phase is independently useful; the risky secret-transport work is last.
### Phase 4 — OpenBao as the secret backend (last; needs the security foundation)
- Introduce a **`SecretBackend` seam** at the single read chokepoint
`_read_secret` (`config.py:307`) + the `castle-api/.../secrets.py` writer:
`_read_secret` (`config.py:307`) + the `wildpc-api/.../secrets.py` writer:
`FileSecretBackend` (default, unchanged) and `OpenBaoBackend`. `${secret:NAME}`
syntax is untouched; backend selected in castle.yaml or by env.
syntax is untouched; backend selected in wildpc.yaml or by env.
- Followers auth to OpenBao (token / AppRole); **need-to-know scoping** via
per-deployment policies. Keep the file backend as fallback; migrate secrets in.
@@ -129,8 +129,8 @@ Each phase is independently useful; the risky secret-transport work is last.
- **P1:** bring up two nodes; confirm registry + presence propagate over NATS and
the mesh view (`/mesh/status`, System Map) is identical to the MQTT behavior.
- **P2:** write a key to `castle-config` on the authority; observe a follower
watch fire and `castle apply` reconcile.
- **P2:** write a key to `wildpc-config` on the authority; observe a follower
watch fire and `wildpc apply` reconcile.
- **P3:** define a service on node A that `requires` a ref provided on node B;
confirm the gateway routes cross-node, then **kill node B** and confirm the
consumer fails fast (curl returns the degraded 503, not a hang) and **recovers**
@@ -145,17 +145,17 @@ Each phase is independently useful; the risky secret-transport work is last.
Both services stood up **alongside** the live MQTT mesh (no cutover yet):
- `castle-nats` (`nats:2`, JetStream) — deployment
`~/.castle/deployments/services/castle-nats.yaml`, config
`/data/castle/castle-nats/config/nats-server.conf`. Verified: container active,
- `wildpc-nats` (`nats:2`, JetStream) — deployment
`~/.wildpc/deployments/services/wildpc-nats.yaml`, config
`/data/wildpc/wildpc-nats/config/nats-server.conf`. Verified: container active,
`/healthz` ok, JetStream enabled (`/jsz` shows store dir + limits), and a real
**KV round-trip** — created `castle-registry` (put/get) and a TTL
`castle-presence` bucket (the presence primitive). Test buckets cleaned up.
- `castle-openbao` (`openbao/openbao:latest`, v2.5.5) — deployment
`~/.castle/deployments/services/castle-openbao.yaml`, config
`/data/castle/castle-openbao/config/openbao.hcl`. Verified: initialized (1
**KV round-trip** — created `wildpc-registry` (put/get) and a TTL
`wildpc-presence` bucket (the presence primitive). Test buckets cleaned up.
- `wildpc-openbao` (`openbao/openbao:latest`, v2.5.5) — deployment
`~/.wildpc/deployments/services/wildpc-openbao.yaml`, config
`/data/wildpc/wildpc-openbao/config/openbao.hcl`. Verified: initialized (1
share / threshold 1), unsealed, **KV-v2 secret put/get/delete** round-trip.
Unseal key + root token stored as castle secrets `OPENBAO_UNSEAL_KEY` /
Unseal key + root token stored as wildpc secrets `OPENBAO_UNSEAL_KEY` /
`OPENBAO_ROOT_TOKEN`.
**Consciously deferred (tracked):**
@@ -172,10 +172,10 @@ Both services stood up **alongside** the live MQTT mesh (no cutover yet):
MQTT/Mosquitto transport replaced by NATS JetStream KV.
- New `castle_api/mesh_wire.py` — transport-agnostic registry (de)serialization
- New `wildpc_api/mesh_wire.py` — transport-agnostic registry (de)serialization
(moved out of `mqtt_client.py`; preserves the secret-stripping invariant).
- New `castle_api/nats_client.py` — `CastleNATSClient`: connects, PUTs its
registry to the `castle-registry` KV bucket, seeds from existing keys, watches
- New `wildpc_api/nats_client.py` — `WildpcNATSClient`: connects, PUTs its
registry to the `wildpc-registry` KV bucket, seeds from existing keys, watches
for peer PUT/DELETE, heartbeats a re-PUT (crash liveness via the existing
stale-TTL), and DELETEs its key on graceful stop (immediate peer-offline).
Async-native → the paho cross-thread `run_coroutine_threadsafe` hop is gone.
@@ -184,54 +184,54 @@ MQTT/Mosquitto transport replaced by NATS JetStream KV.
- Frontend: `types/index.ts` `MeshStatus` + `MeshPanel.tsx` updated to the new
fields. `mqtt_client.py` + `test_mqtt.py` deleted; `test_mesh_wire.py` added.
`mdns.py` dead `_mqtt._tcp` broker-browse removed. `paho-mqtt` → `nats-py`.
- **Live cutover:** `castle-api.yaml` env → `CASTLE_API_NATS_*`, `requires:
castle-nats`; applied. castle-api healthy on NATS, `/mesh/status` connected,
- **Live cutover:** `wildpc-api.yaml` env → `WILDPC_API_NATS_*`, `requires:
wildpc-nats`; applied. wildpc-api healthy on NATS, `/mesh/status` connected,
registry (9.7 KiB, 48 deployments) published to KV, **no secrets on the wire**.
- Verification run: ruff clean; 89 api + 196 core tests pass; frontend `tsc`
clean; runtime KV round-trip confirmed against the live `castle-nats`.
clean; runtime KV round-trip confirmed against the live `wildpc-nats`.
**Pending second node:** two-node mesh-view parity (peer sees peer, offline on
departure) — needs the second LAN node running the NATS-enabled castle-api
pointed at civil's NATS (seed URL) or a clustered `castle-nats`. Revert path if
needed: restore `CASTLE_API_MQTT_*` env — but note `mqtt_client.py` is deleted,
so revert = `git checkout main -- castle-api` then re-apply. The old Mosquitto
departure) — needs the second LAN node running the NATS-enabled wildpc-api
pointed at civil's NATS (seed URL) or a clustered `wildpc-nats`. Revert path if
needed: restore `WILDPC_API_MQTT_*` env — but note `mqtt_client.py` is deleted,
so revert = `git checkout main -- wildpc-api` then re-apply. The old Mosquitto
`mqtt` service is left running (dormant) as a safety net; retire it once the
second node is proven on NATS.
### Phase 2 — DONE + single-node verified + tested (2026-07-07)
- **`role` field** on `NodeConfig` + `CastleConfig`, wired end-to-end:
castle.yaml top-level `role:` → config → `_node_config` → registry.yaml →
- **`role` field** on `NodeConfig` + `WildpcConfig`, wired end-to-end:
wildpc.yaml top-level `role:` → config → `_node_config` → registry.yaml →
mesh wire. **`civil` pinned `role: authority`**; default `follower`.
- **Presence** (`castle-presence`) — a TTL KV bucket each node renews on the
- **Presence** (`wildpc-presence`) — a TTL KV bucket each node renews on the
heartbeat; expiry = the node is gone. Delete-on-stop for immediate departure.
- **Shared config** (`castle-config`) — `get_shared_config` / `put_shared_config`
- **Shared config** (`wildpc-config`) — `get_shared_config` / `put_shared_config`
(authority-gated: followers raise `PermissionError`), plus a watch loop that
broadcasts a `config_changed` SSE (the follower `castle apply` reconcile hook
broadcasts a `config_changed` SSE (the follower `wildpc apply` reconcile hook
hangs off this — inert on one node).
- Hardened `stop()` to bound the NATS drain (can't hang systemd shutdown).
- Verified live: all three buckets present, `civil=online` presence,
`"role": "authority"` on the wire, authority write + follower-deny exercised
against the running `castle-nats`.
against the running `wildpc-nats`.
- **Tests added:** `core/tests/test_fleet_role.py` (role config load + registry
round-trip), `castle-api/tests/test_nats_client.py` (role gating, hermetic),
round-trip), `wildpc-api/tests/test_nats_client.py` (role gating, hermetic),
role round-trip in `test_mesh_wire.py`. Suites: **200 core + 93 api** pass.
**Pending second node:** the follower-side reconcile (watch `castle-config` →
`castle apply`) is wired as an SSE hook but only meaningful with a peer.
**Pending second node:** the follower-side reconcile (watch `wildpc-config` →
`wildpc apply`) is wired as an SSE hook but only meaningful with a peer.
### Phase 4 — secret-read backend DONE + verified + tested (2026-07-07)
- New `core/castle_core/secret_backends.py`: `SecretBackend` protocol,
- New `core/wildpc_core/secret_backends.py`: `SecretBackend` protocol,
`FileSecretBackend` (the historical behavior), `OpenBaoBackend` (KV-v2 read with
file fallback), and `build_backend()` selecting via `CASTLE_SECRET_BACKEND`
file fallback), and `build_backend()` selecting via `WILDPC_SECRET_BACKEND`
(default **file** — production is byte-for-byte unchanged until opted in).
- `_read_secret` (the single chokepoint, `config.py`) now delegates to the active
backend. `${secret:NAME}` syntax untouched.
- OpenBao token bootstraps from the file backend (it can't live in the vault it
unlocks); a missing key / auth failure / unreachable server all fall through to
file, so a partly-migrated vault keeps working.
- Verified live against the running `castle-openbao`: a secret stored in the vault
- Verified live against the running `wildpc-openbao`: a secret stored in the vault
resolves through `${secret:...}` in openbao mode; file-only secrets resolve via
fallback; missing → placeholder; **default file mode unchanged**.
- **Tests:** `core/tests/test_secret_backends.py` (file hit/miss, backend
@@ -239,12 +239,12 @@ second node is proven on NATS.
**Hardening:**
1. **Write path — DONE + verified live.** `SecretBackend` gained
`write`/`delete`/`list_names`; `castle-api/secrets.py` routes all CRUD through
`write`/`delete`/`list_names`; `wildpc-api/secrets.py` routes all CRUD through
the active backend, so in openbao mode the dashboard writes to the vault.
Verified: write→vault→read→list→delete round-trip against live OpenBao.
2. **Auto-unseal on boot — DONE + verified.** Added `exec_start_post` to the
systemd spec (general, `-`-prefixed so a hook failure never fails the unit);
`castle-openbao` runs `/data/castle/castle-openbao/unseal.sh` (polls up, unseals
`wildpc-openbao` runs `/data/wildpc/wildpc-openbao/unseal.sh` (polls up, unseals
with `OPENBAO_UNSEAL_KEY`). Verified: manual seal → restart → auto-unsealed.
3. **TLS hardening — DONE + verified across both nodes (2026-07-07).**
Reuses the existing ACME **wildcard cert** via `expose.tcp.tls` (the postgres
@@ -253,12 +253,12 @@ second node is proven on NATS.
custom CA to distribute**.
- **NATS:** `tls{}` (wildcard cert in `/tls`) + `authorization{token}`
(`$NATS_TOKEN` from the `NATS_TOKEN` secret). Clients connect to
`tls://castle-nats.civil.payne.io:4222` with the token; `CastleNATSClient`
`tls://wildpc-nats.civil.payne.io:4222` with the token; `WildpcNATSClient`
gained token support (a `tls://` URL → nats-py verifies via system CA).
**civil + primer both cut over**; plaintext now rejected (verified:
`certificate is valid for *.civil.payne.io, not localhost`).
- **OpenBao:** listener `tls_cert_file`/`tls_key_file` from `/tls`; reachable at
`https://castle-openbao.civil.payne.io:8200`. Auto-unseal + secret backend
`https://wildpc-openbao.civil.payne.io:8200`. Auto-unseal + secret backend
both use the HTTPS URL. Verified: HTTPS serves, **auto-unsealed over HTTPS**,
plaintext rejected, backend reads a secret over HTTPS.
- Coordinated cutover across the whole fleet (both nodes) with a brief,
@@ -267,7 +267,7 @@ second node is proven on NATS.
### Phase 3 — cross-node routing + breaker: logic DONE + verified against a real peer (2026-07-07)
**Real second node:** `primer` (192.168.8.129) migrated onto the NATS mesh
(branch checked out, castle-api pointed at `nats://civil:4222` via a systemd
(branch checked out, wildpc-api pointed at `nats://civil:4222` via a systemd
drop-in). civil ↔ primer mesh confirmed over the LAN — **Phase 1 two-node parity
done for real**, not simulated. (This also *restored* the civil↔primer mesh my
Phase 1 cutover had split — primer was still on MQTT→civil.)
@@ -280,12 +280,12 @@ Phase 1 cutover had split — primer was still on MQTT→civil.)
(2s dial timeout + passive health) for the there-but-wedged case; **presence
expiry removes the peer from the route set entirely** (gone → no route) — the
primary breaker.
- `castle_api/mesh_gateway.py` — on peer join/leave/change (+ startup) the API
- `wildpc_api/mesh_gateway.py` — on peer join/leave/change (+ startup) the API
re-renders the Caddyfile (same generator `apply` uses + remote routes for
online peers) and reloads the gateway **iff content changed**.
- **Verified:** 4 hermetic tests (route emitted / address fallback / breaker:
peer-absent → no route / unconsumed → no route); **resolution against primer's
real registry** pulled live from the mesh → `castle-api → primer:9020`; and the
real registry** pulled live from the mesh → `wildpc-api → primer:9020`; and the
live integration proven a **no-op** on civil (Caddyfile hash unchanged, gateway
healthy) since civil consumes nothing cross-node yet. Suites: **210 core + 93 api**.
@@ -309,6 +309,6 @@ primer is now a permanent mesh member on the branch (revert: remove the drop-in
as `role: authority`; all others are followers. **Confirmed:** when `civil` is
down, shared config/secrets go **read-only** fleet-wide and every node keeps
serving its own deployments from cached local state.
4. **NATS — single server now, cluster-ready.** Run one `castle-nats` today
4. **NATS — single server now, cluster-ready.** Run one `wildpc-nats` today
(matches the single-writer authority). Clustering is deferred but must be a
pure config addition (cluster/routes block + a few more nodes) — no re-architecture.