Orange terminal prompt icon with white stroke-based line art on rounded rect,
matching the shared visual identity across Wild Central, Cloud, and Directory.
A job's systemd unit is castle-<name>-job.service and its EnvironmentFile points
at castle-<name>-job.service.env, but _write_secret_env_file wrote
castle-<name>.service.env (kind defaulted to 'service'). Every secret-using job
failed to start (Failed to load environment files). Thread the deployment kind
through to secret_env_path so the filename matches the unit.
Writing a secret by hand meant knowing the active backend and its layout;
getting it wrong (a file write on an OpenBao fleet) left the value where the
resolver never reads it, so ${secret:NAME} silently degraded to the literal
<MISSING_SECRET:NAME> placeholder that a service then used as its credential.
- `castle secret {list|set|get|rm}` reads/writes the ACTIVE backend, so there's
no wrong store to pick. `set NAME` with no value reads a hidden prompt / stdin.
- `castle apply` (and --plan) now refuses to converge any deployment whose
${secret:...} refs don't resolve in the active backend: exits non-zero, writes
nothing, and names each deployment + secret + the `castle secret set` fix.
- New helpers in core/config.py: active_secret_backend(), active_backend_name(),
secret_refs(). Gate impl: deploy._unresolved_secrets() + ApplyResult.blocked.
- Tests: test_deploy_secret_gate.py, TestSecretRefs, test_secret.py. Docs: AGENTS.md.
Static (manager: caddy) deployments always emitted the SPA fallback
`try_files {path} /index.html`, which serves the root index.html for
every unmatched path. That's correct for React/Vite apps but swallows a
multi-page content site's in-page links back to the homepage, so a Hugo
site (payne.io) couldn't navigate off its landing page.
Add an explicit `spa: bool` on CaddyDeployment (default True, preserving
existing SPA behavior). When False, the gateway serves plain file_server
— resolving directory indexes (/posts/ -> /posts/index.html) and 404ing
missing paths. Threaded through the registry snapshot + mesh and the
route computation so both the internal wildcard and public apex blocks
honor it. `castle program create --stack hugo` now sets spa: false.
Stacks declare host toolchains (uv/pnpm/node/hugo/deno/psql) that can drift
from what's installed and, worse, from what's on a running service's PATH.
Make those dependencies first-class and visible.
Model (core):
- stacks.ToolRequirement + StackHandler.tools declare each stack's toolchains
(command, purpose, phase, install_hint); tools_for() is the single source.
- relations synthesizes them as `kind: tool` requirements so functional?/graph
account for them; checked runtime-env-aware (run-phase tools of a systemd
service are probed against the service's PATH, not the shell) via a shared
generators.systemd.runtime_path helper the unit generator also uses, so the
checker can't drift from the generator. hint_for() makes every unmet
requirement actionable.
- stack_status: the derived per-stack health the CLI/API/UI all render.
- config: add ~/.deno/bin to USER_TOOL_PATH_DIRS so deno (supabase edge fns)
is found by services and the check, same as ~/.local/bin and the pnpm dirs.
Surfaces:
- castle stack list|info (new resource) + GET /stacks/status, /stacks/{name}
(GET /stacks stays a bare name list for the create-form select).
- castle doctor gains a "Stacks & dependencies" section (FAIL for an enabled
deployment's missing tool, WARN otherwise, unused stacks skipped).
- castle apply preflight warns (advisory, like _acme_preflight) when a tool is
missing where a service runs; ConvergePanel renders those warnings.
- Dashboard Stacks page: per-stack tool checklist with versions + copyable
install hints, program links, and verb chips.
Tests: relations (drift + hints), doctor (ok/fail/skip), /stacks endpoints.
The nav label was already "System" and the page is SystemMap; only the URL
was stale. Move it to /system, add a back-compat redirect from /map, and update
the command-palette navigations. Unrelated graph node type "map" left as-is.
Register an existing repo as a program from the /programs page — the web
equivalent of `castle program add`. An inline form with an editable,
autocompleting path bar browses the *server's* filesystem (a browser's own
file dialog only sees the client machine) and also accepts a git URL.
- core: extract the adopt logic (target parsing, stack/command detection,
ProgramSpec build) into castle_core.adopt so the CLI and API share it; the
CLI's add.py now delegates to it.
- castle-api: GET /fs/browse (directory listing, per-entry stat errors skipped
so one unreadable child can't 403 the dir) and POST /programs/adopt.
- app: AddProgramForm + Add-program button on the Programs page; useBrowse /
useAdoptProgram hooks.
Also tidy ProgramCard: deployments render below the description, and the
deployment name label is hidden when it matches the program title.
Swap the canonical KIND_ICONS.program glyph from Package to SquareCode, and route the palette + System-map program affordances through it. (The Programs nav icon in Layout.tsx changes too, committed separately with its own WIP.)
The command palette searched deployments but not the program catalog. List programs too: a deployed program stays one row with a program-source jump; a source-only program gets its own row. Also swap the System-map jump to the Gauge icon and give each row a leading kind icon.
The path-tool lifecycle offered only Enable/Disable (keyed on enabled), so an enabled-but-not-on-PATH tool showed just "Disable" with no way to converge it. Add an Install/Uninstall (apply) action whenever desired (enabled) and actual (installed) disagree.
Every surface hand-mapped deployment kind → lucide icon, so they could (and did) drift. Add KIND_ICONS/kindIcon to lib/labels beside kindLabel, and route KindBadge (→ all detail headers/cards/sections), the Overview tiles, and the program links through it.
_on_path() led with a bare shutil.which(), which resolves against the calling process's PATH. When castle-api runs inside the repo venv (uv run), a tool merely colocated there — e.g. a stray editable a dev uv-pip-installed — read as "installed on PATH", so the UI showed installed and the converge planner reported no drift. Strip the running interpreter's own venv bin from the search path; tools installed anywhere else on PATH still count.
- service/static/tool/job detail pages now link back to their program;
backfill the program ref on registry-only static+job details (the
registry Deployment carries no program ref).
- new RelatedDeployments card surfaces a deployment's requires edges both
directions (Depends on / Required by) as links; references render inert.
- replace the service detail Subdomain/Served-at rows with a launch icon
in the header that opens the gateway URL in a new tab.
Both machines (civil, primer) are on the current on-disk layout, so the
read-compat for older formats is dead weight. Remove it:
- config loader: only deployments/<kind>/<name>.yaml (drop flat deployments/*.yaml
and the services/+jobs/ split); drop the run.runner→manager normalizer.
- registry loader: require composite <kind>/<name> keys + manager/kind (registry.yaml
is regenerated every apply, so it's always current).
- manifest: drop the proxy/public→reach legacy input and the component→program alias
(the derived proxy/public read accessors stay). reach field defaults already match.
- config_editor: parse incoming specs directly (frontend sends the current shape).
- save_config: drop the flat-file migration cleanup.
Bootstrap seeds move to bootstrap/deployments/<kind>/ and install.sh seeds that
per-kind tree directly. The legacy→current translation for terse test fixtures
moves into each test conftest (out of production). All suites green (362).
- endpoint-level: a deployment/program edit via /config/* leaves castle.yaml
globals byte-identical (the exact regression that shipped — the endpoints were
tested but nobody asserted globals survived)
- write_program_file leaves globals untouched (core)
- /secrets/info reports the backend
Still uncovered (tracked): castle mesh CLI, direct-read refactor (dns/stacks),
backend-aware doctor, and all frontend (no test runner configured).
Root-cause hardening for the dropped-globals bug: config edits went through the
full-document save_config, which rewrote castle.yaml globals (+ every resource
file) on every deployment/program edit — so an unmodeled global (role, secrets)
was dropped.
- add write_deployment_file / write_program_file: persist ONE resource file,
never touching globals or other resources
- config_editor deployment/program save/delete/enable now use those instead of
save_config, so a deployment edit can't rewrite castle.yaml
- save_config now preserves ANY unmanaged top-level global (not just secrets)
- _aggregate_yaml surfaces role + secrets (raw-yaml round-trip completeness)
- tests: scoped write leaves globals byte-identical; unmanaged global survives
Verified live: editing lakehouse via the endpoint leaves castle.yaml unchanged.
The deployment form now manages only the wiring (ENV_VAR -> secret name) with a
set/unset status badge and a link to the Secrets page for the value — it no
longer fetches or inline-edits secret values. A deployment only ever stores
${secret:NAME}; values live in the backend and are managed in one place (the
Secrets page). Removes the pre-vault holdover of pulling secret values into the
deployment-editing surface.
save_config rewrites castle.yaml from scratch and only re-emitted gateway/repo/
roots — so any config save (dashboard deployment edit, config_editor) silently
STRIPPED top-level 'role' and the 'secrets:' backend block. Effect: the node
reverted to a file-backend follower, the vault stopped resolving, and a
subsequent apply baked <MISSING_SECRET> into service env (hit castle-lakehouse).
Now re-emit role (from config) + preserve the secrets block (read from the
existing file, since it's not modeled on CastleConfig). Regression test added.
A bare `ssh host 'castle …'` / `'pnpm …'` couldn't find the toolchain: uv,
castle, pnpm, and nvm node live on the interactive PATH (~/.zsh.d/*.sh), which
non-interactive/non-login shells don't load. Add ensure_shell_path() to mirror
just those PATH entries into ~/.zshenv (which non-interactive zsh does load),
so every bootstrapped node can drive castle/pnpm over ssh. Idempotent, marker-
guarded, and skipped for non-zsh users.
Post-migration sweep of legacy file-based secret code:
- deploy.py + doctor.py checked for the token *file* → false 'secret not found'
warnings now that tokens live in the vault; use read_secret (backend-aware)
- drop now-unused SECRETS_DIR imports
- refresh stale docstring/prompt text (${secret} 'reads ~/.castle/secrets' →
'via the active backend'; SecretsEditor add-prompt)
Kept (legitimate file paths): the bootstrap tier (OPENBAO_* token/unseal,
cloudflared creds dir) + the rendered 0600 env files.
- new Secrets page (/secrets, nav entry): lists vault secrets, reveal/copy,
add/edit/delete, shows active backend + addr + role; read-only on followers
- GET /secrets/info (backend/addr/role/writable); fix /secrets CRUD to use the
castle.yaml-selected backend (was defaulting to the now-empty file backend)
- OpenBaoBackend.list_names drops folder entries (node-prefix groups)
- tests: conftest forces file backend so host castle.yaml can't leak into tests
app builds clean (tsc 0); endpoints verified live against the vault.
The System Map subsumes the read-only Graph diagnostic, so remove the page,
its route, and nav entry (the /graph API + useGraph stay — the map and palette
consume them).
Add a host switcher under the sidebar brand: it shows which castle host you're
driving (the is_local node) and, when peers are present, jumps to a peer's
dashboard at castle.<domain>, preserving the current view. Node summaries now
carry gateway_domain so the switcher can build each host's URL.
- castle mesh status|nodes|config (list/get/set) — hits the local castle-api;
surfaces the shared-config write path (authority-gated)
- API: GET/PUT /mesh/config/{key:path}, GET /mesh/config; nats_client gains
list_shared_config
- tests: NATS integration suite against a real broker via a docker fixture
(peer discovery, presence, graceful-offline, shared config, secrets-off-wire);
plus HTTP-layer /mesh/config tests. Closes the runtime-coverage gap.
- AGENTS.md §8 updated for NATS + the mesh CLI
100 api + 211 core tests pass.
NATS tls:// + token and OpenBao HTTPS via the ACME wildcard cert (expose.tcp.tls),
no custom CA. civil + primer both cut over; plaintext rejected on both. Server
config + unseal.sh are instance-side (/data/castle, ~/.castle).
- CastleNATSClient accepts a token; a tls:// server URL makes nats-py verify the
broker against the system CA (trusts the wildcard's Let's Encrypt issuer, no
custom CA needed)
- config: CASTLE_API_NATS_TOKEN; main wires it through
Server side (instance): nats-server.conf gains tls{} (wildcard cert via
expose.tcp.tls) + authorization{token}; clients use tls://castle-nats.<domain>.
- SecretBackend gains write/delete/list_names; FileSecretBackend + OpenBaoBackend
implement them (vault KV-v2 POST/DELETE/LIST); castle-api/secrets.py routes all
CRUD through the active backend (dashboard writes to the vault in openbao mode)
- add systemd exec_start_post (general; '-' prefix so a hook failure can't fail
the unit); castle-openbao runs an unseal.sh on boot using OPENBAO_UNSEAL_KEY
- tests: file write/read/list/delete round-trip
Verified live: write→vault→read→list→delete against OpenBao; seal→restart→
auto-unsealed. TLS hardening remains (deferred; gates cross-network/real secrets).
- secret_backends.py: SecretBackend protocol, FileSecretBackend (historical),
OpenBaoBackend (KV-v2 read + file fallback), build_backend() env-selected
- _read_secret delegates to the active backend; default is file, so production
is unchanged until CASTLE_SECRET_BACKEND=openbao
- OpenBao token bootstraps from the file backend; missing/unreachable falls back
- tests: file hit/miss, backend selection, unreachable + empty-token fallback
Read path verified live against castle-openbao. Write path, auto-unseal-on-boot,
and TLS hardening documented as remaining for full OpenBao production use.
- add static 'role' (authority|follower) to NodeConfig/CastleConfig, wired from
castle.yaml through the registry to the mesh wire; civil pinned authority
- castle-presence: TTL KV bucket each node renews (churn signal); delete-on-stop
- castle-config: authority-gated get/put + change-watch SSE (follower reconcile
hook hangs off it)
- bound the NATS drain on stop() so shutdown can't hang
- tests: role config/registry round-trip, wire round-trip, role write-gating
Single-node verified; follower reconcile pending the second node.
Phased plan to evolve the mesh into a purpose-driven fleet: NATS (JetStream KV +
presence) replacing Mosquitto, OpenBao as the secret authority, cross-node
requires resolution + gateway circuit-breaker, static civil=authority role.
Records resolved decisions and Phase 0/1 progress.
The program-level Capability model (provides/consumes) was a vestige of an
earlier design superseded by the deployment-level requires/Requirement graph.
It was read only into a diagnostic node and surfaced as an always-empty
'Capabilities' panel; nothing acted on it. Removed from manifest, relations,
the CLI re-export, and the System Map.
The `castle` CLI and the `castle-api` service are two independent in-process
drivers of `castle_core`. Each resolved DATA_DIR/REPOS_DIR at import time from
its own process env (default /data/castle), persisted nowhere — so they silently
diverged, and `apply`/dashboard-apply crashed on a non-existent /data.
Make the loaded CastleConfig the single source of truth:
- Resolve data_dir/repos_dir only in load_config (env > castle.yaml > default),
anchored to the config root; drop the DATA_DIR/REPOS_DIR module globals and the
import-time file read entirely — no global twin that can disagree with the file.
- Thread config.data_dir/repos_dir through ensure_dirs, _env_context, tls_dir_for
(now unified — deploy no longer inlines the tls path), and create/add/clone.
- ensure_dirs raises an actionable CastleDirError instead of a bare PermissionError;
the api surfaces it as 422.
- doctor: "data dir writable" check + WARN when CASTLE_DATA_DIR/REPOS_DIR env
overrides the file (the one remaining cross-process divergence vector).
- install.sh persists data_dir/repos_dir into castle.yaml (idempotent, non-default).
- Docs: registry.md globals + AGENTS.md roots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mesh payload didn't include a node's acme domain, so a peer had no way
to build launch URLs for another machine's exposed apps. Publish
gateway_domain in the MQTT registry payload and surface it per-deployment on
/mesh/deployments as `domain`. The dashboard palette and System Map now build
`https://<subdomain>.<domain>` for remote http-exposed apps (references use
their base_url), making primer's apps launchable from civil.
