Stacks declare host toolchains (uv/pnpm/node/hugo/deno/psql) that can drift
from what's installed and, worse, from what's on a running service's PATH.
Make those dependencies first-class and visible.
Model (core):
- stacks.ToolRequirement + StackHandler.tools declare each stack's toolchains
(command, purpose, phase, install_hint); tools_for() is the single source.
- relations synthesizes them as `kind: tool` requirements so functional?/graph
account for them; checked runtime-env-aware (run-phase tools of a systemd
service are probed against the service's PATH, not the shell) via a shared
generators.systemd.runtime_path helper the unit generator also uses, so the
checker can't drift from the generator. hint_for() makes every unmet
requirement actionable.
- stack_status: the derived per-stack health the CLI/API/UI all render.
- config: add ~/.deno/bin to USER_TOOL_PATH_DIRS so deno (supabase edge fns)
is found by services and the check, same as ~/.local/bin and the pnpm dirs.
Surfaces:
- castle stack list|info (new resource) + GET /stacks/status, /stacks/{name}
(GET /stacks stays a bare name list for the create-form select).
- castle doctor gains a "Stacks & dependencies" section (FAIL for an enabled
deployment's missing tool, WARN otherwise, unused stacks skipped).
- castle apply preflight warns (advisory, like _acme_preflight) when a tool is
missing where a service runs; ConvergePanel renders those warnings.
- Dashboard Stacks page: per-stack tool checklist with versions + copyable
install hints, program links, and verb chips.
Tests: relations (drift + hints), doctor (ok/fail/skip), /stacks endpoints.
The nav label was already "System" and the page is SystemMap; only the URL
was stale. Move it to /system, add a back-compat redirect from /map, and update
the command-palette navigations. Unrelated graph node type "map" left as-is.
Register an existing repo as a program from the /programs page — the web
equivalent of `castle program add`. An inline form with an editable,
autocompleting path bar browses the *server's* filesystem (a browser's own
file dialog only sees the client machine) and also accepts a git URL.
- core: extract the adopt logic (target parsing, stack/command detection,
ProgramSpec build) into castle_core.adopt so the CLI and API share it; the
CLI's add.py now delegates to it.
- castle-api: GET /fs/browse (directory listing, per-entry stat errors skipped
so one unreadable child can't 403 the dir) and POST /programs/adopt.
- app: AddProgramForm + Add-program button on the Programs page; useBrowse /
useAdoptProgram hooks.
Also tidy ProgramCard: deployments render below the description, and the
deployment name label is hidden when it matches the program title.
Swap the canonical KIND_ICONS.program glyph from Package to SquareCode, and route the palette + System-map program affordances through it. (The Programs nav icon in Layout.tsx changes too, committed separately with its own WIP.)
The command palette searched deployments but not the program catalog. List programs too: a deployed program stays one row with a program-source jump; a source-only program gets its own row. Also swap the System-map jump to the Gauge icon and give each row a leading kind icon.
The path-tool lifecycle offered only Enable/Disable (keyed on enabled), so an enabled-but-not-on-PATH tool showed just "Disable" with no way to converge it. Add an Install/Uninstall (apply) action whenever desired (enabled) and actual (installed) disagree.
Every surface hand-mapped deployment kind → lucide icon, so they could (and did) drift. Add KIND_ICONS/kindIcon to lib/labels beside kindLabel, and route KindBadge (→ all detail headers/cards/sections), the Overview tiles, and the program links through it.
- service/static/tool/job detail pages now link back to their program;
backfill the program ref on registry-only static+job details (the
registry Deployment carries no program ref).
- new RelatedDeployments card surfaces a deployment's requires edges both
directions (Depends on / Required by) as links; references render inert.
- replace the service detail Subdomain/Served-at rows with a launch icon
in the header that opens the gateway URL in a new tab.
The deployment form now manages only the wiring (ENV_VAR -> secret name) with a
set/unset status badge and a link to the Secrets page for the value — it no
longer fetches or inline-edits secret values. A deployment only ever stores
${secret:NAME}; values live in the backend and are managed in one place (the
Secrets page). Removes the pre-vault holdover of pulling secret values into the
deployment-editing surface.
Post-migration sweep of legacy file-based secret code:
- deploy.py + doctor.py checked for the token *file* → false 'secret not found'
warnings now that tokens live in the vault; use read_secret (backend-aware)
- drop now-unused SECRETS_DIR imports
- refresh stale docstring/prompt text (${secret} 'reads ~/.castle/secrets' →
'via the active backend'; SecretsEditor add-prompt)
Kept (legitimate file paths): the bootstrap tier (OPENBAO_* token/unseal,
cloudflared creds dir) + the rendered 0600 env files.
- new Secrets page (/secrets, nav entry): lists vault secrets, reveal/copy,
add/edit/delete, shows active backend + addr + role; read-only on followers
- GET /secrets/info (backend/addr/role/writable); fix /secrets CRUD to use the
castle.yaml-selected backend (was defaulting to the now-empty file backend)
- OpenBaoBackend.list_names drops folder entries (node-prefix groups)
- tests: conftest forces file backend so host castle.yaml can't leak into tests
app builds clean (tsc 0); endpoints verified live against the vault.
The System Map subsumes the read-only Graph diagnostic, so remove the page,
its route, and nav entry (the /graph API + useGraph stay — the map and palette
consume them).
Add a host switcher under the sidebar brand: it shows which castle host you're
driving (the is_local node) and, when peers are present, jumps to a peer's
dashboard at castle.<domain>, preserving the current view. Node summaries now
carry gateway_domain so the switcher can build each host's URL.
The program-level Capability model (provides/consumes) was a vestige of an
earlier design superseded by the deployment-level requires/Requirement graph.
It was read only into a diagnostic node and surfaced as an always-empty
'Capabilities' panel; nothing acted on it. Removed from manifest, relations,
the CLI re-export, and the System Map.
The mesh payload didn't include a node's acme domain, so a peer had no way
to build launch URLs for another machine's exposed apps. Publish
gateway_domain in the MQTT registry payload and surface it per-deployment on
/mesh/deployments as `domain`. The dashboard palette and System Map now build
`https://<subdomain>.<domain>` for remote http-exposed apps (references use
their base_url), making primer's apps launchable from civil.
- Mesh endpoints now gate the http port on `subdomain` (the registry's "exposed"
signal), matching the local reach-gated derivation — so a remote reach:off
service (e.g. primer's castle-gateway) no longer shows a phantom :9000.
- The focus/resync effect no longer rebuilds the node set while a connection is
being dragged; a background refetch mid-drag was silently cancelling the
connection, so drag-to-expose/-require intermittently "did nothing".
The keyboard twin of the map's inspect panel: ⌘K (or the "Launch…" nav button)
from any page. Empty query = the "Start Menu" (browsable launchable apps, local +
remote, grouped by machine badge); typing searches every deployment. Enter
launches the app in a new tab; per-result actions jump to it on the map
(/map?focus=<id>, which the map now honors by selecting + centering) or open its
Castle details. Navigation/launch only — no mutation — for v1.
- "Start Menu": frontends and http-exposed services get a launch ↗ that opens
the running app (https://<name>.<domain>) in a new tab — on the node (hover)
and in the panel header.
- The right-click / kebab NodeMenu is gone; its actions (Restart, expose, Delete)
now live in the inspect panel as an Actions row, so one gesture (click a node)
is the whole control surface. Panel header separates Launch (the app) from
Castle details (the management page). Remote nodes stay view-only.
- Local node's deployments render on top (with a "⬡ <host> · this node" label and
the lane headers), then a labeled "other machines" divider, then each remote
machine as a band below — same kind-columns, so cross-node edges align vertically.
- Remote nodes get full UX parity: draggable (persisted), selectable, and a unified
focus adjacency (consumes/consumed-by over local + remote + reference ids) drives
identical highlight + inspect-panel behavior whether a node is local or remote.
The panel shows a machine badge on cross-node deps and disables edit on remotes.
Remote machines' deployments now sit in the same kind-columns as local (so a
cross-node edge runs straight down its column), are draggable with persisted
positions, and the lane headers lift above the topmost band so they head every
machine's columns.
Carry each deployment's `requires` (deployment refs, no secrets) from config →
registry → the mesh MQTT payload, and expose it on /mesh/deployments. The System
Map resolves a remote deployment's ref against the provider set across nodes
(same machine → local provider → other machine) and draws a dashed cross-node
edge. This is the multi-node payoff: e.g. primer's castle-api → civil's mqtt.
- Loading spinner while the graph query is in flight.
- Deployment cards drop the "consumed by N" line and external-ref chips — both
now live in the inspect panel, so cards are just name/port/reach.
- A node's external (reference) edges are drawn only while it's selected, then
removed — no permanent web of long cross-map lines.
- LAN exposure edges animate like the Internet ones (animation = live traffic).
Reframes the map around consumes / consumed_by and adds:
- Inspect on select: dim the rest, light a node's consumes/consumed-by, and a
side panel listing both (removable/navigable) with protocol chips.
- Protocol-typed dependency edges (http/pg/bolt/mqtt) + reach badges; TCP infra
now shows its port.
- External resources: a registry lane of reference nodes with an "add external"
form; consumption of one renders as a chip on the consumer (no cross-map line).
- Suggested (undeclared) consumption as dashed amber edges — click to accept.
- Capability/candidates in the panel (alternative providers of the same protocol).
- A "Machines" band rendering other mesh nodes' deployments (read-only).
Adds useSuggestions / useMeshDeployments / useSaveReference hooks and the
GraphEndpoint / GraphSuggestion / MeshDeployment types.
A spatial control surface for the whole node. Tools, jobs, services, and
frontends sit in lanes with program-source grips; edges show requires
(dependency), same-program siblings, and exposure to LAN/Internet targets.
Direct manipulation, each backed by the existing config PUT + apply:
- drag a node to LAN/Internet to set reach (internal/public)
- delete an exposure line or a node (with confirm) to unexpose/remove
- right-click or the per-node ⋯ menu: open, restart, expose, delete
- click the source grip to open the backing program
Nodes are draggable with Shift-/toggle box-select; layout persists to
localStorage with a reset control. Adds useSetReach/useDeleteDeployment
hooks, the @xyflow/react dep, and the nav entry + route.
Model:
- Drop `requires` from ProgramSpec; requires is now deployment-only.
Collapse Requirement to {ref, bind} with kind defaulting to "deployment"
(drop the unused `version`). System-package preconditions stay on the
program as `system_dependencies`, synthesized into {kind: system} for the
functional check. relations.py/deploy.py read only the deployment's requires.
- create.py seeds a stack's substrate dependency onto the deployment, not the
program. Docs + tests updated.
UI:
- Add a `requires` editor (useRequires) on service, job, and static detail
pages — edit service→service deps (ref + optional bind env var).
- Restore the gateway route table "Open" column: a kind-aware in-app link to
each route's deployment.
- Reformat the reach control: name baked into each option
(localhost:PORT (local) / <name>.<domain> (internal) / <name>.<pubdomain> (public)).
- Services page: sort statics in with systemd services by name, add a
search + kind-filter bar mirroring the programs page.
API: /config/{services,jobs,tools,static}/{name} now pin the twin they
target — _save_deployment/_delete_deployment take an explicit kind so a
patch to a 'backup' service can't bleed into a 'backup' job/tool. Add
/tools and /static endpoints; keep /deployments/{name} kind-agnostic.
New test_kind_twins proves per-kind save/delete isolation on disk.
UI: ConfigPanel and CreateDeploymentForm write to the kind-scoped
resource; GatewayPanel/NodeDetail/DeploymentsSection link via a shared
detailPath(name, kind) helper instead of the ambiguous /deployment/:name.
Includes incidental ruff-format reflow of untouched api files.
Deployment/program edits could silently drop spec fields:
- GET /deployments/{name} served the runtime view (no reach/program/root/expose)
for deployed deployments, so the dashboard round-tripped a lossy manifest and
stripped fields on save (broke astro). Now serves the editable castle.yaml spec
when the deployment is in config.
- _save_deployment and save_program replaced the whole object with client input.
Now shallow-merge over the existing spec (omit = preserve, null = clear), so a
partial/lossy save can't drop untouched fields.
- save_yaml rebuilt GatewayConfig from `port` only, wiping tls/domain/tunnel/
cert_hook on a whole-file save. Now uses a shared parse_gateway() (also used by
load_config) so gateway fields can't drift between the two.
Dashboard forms (ServiceFields/StaticFields) send explicit null to clear, per the
merge contract; adds exposure host-label helpers. Coverage: detail-serves-spec,
save round-trip, partial-patch-preserves, null-clears, program partial-patch,
parse_gateway, and config save/load round-trip.
Frontend builds triggered from the castle web app run inside the castle-api
systemd service, whose PATH omits nvm's versioned node dir — so `pnpm build`
died with `node: not found` even though it worked from an interactive shell.
Introduce a per-program node convention: a program declares its version the
ecosystem-standard way (.node-version / .nvmrc / package.json engines.node),
and castle_core.toolchains.resolve_node_bin() maps it to a concrete nvm bin dir
(CASTLE_NODE_VERSIONS_DIR, default ~/.nvm/versions/node; newest match wins).
The same resolver feeds both sites that run a program's node:
- build time: stacks._build_env() prepends the pinned node for the dev-verb
subprocess (keyed on the source dir), so `castle program build` uses the
program's node regardless of caller.
- run time: deploy._build_deployed() stores it in Deployment.path_prepend,
which the systemd generator puts ahead of the default unit PATH — so a
`launcher: node` service runs its program's node.
A pinned-but-uninstalled version fails loud with an `nvm install` hint instead
of a cryptic `node: not found`. Unpinned → no injection (no guessing).
Also: the systemd generator now honors an explicit PATH in defaults.env as a
full override instead of clobbering it with a trailing Environment=PATH line
(systemd's last-assignment-wins rule had silently defeated the documented
escape hatch).
Pins app/.node-version and documents the convention in the react-vite stack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A derived, mostly-computed model of how programs, deployments, and repos relate,
plus the git-sync surfaces that motivated it. See docs/relationships.md.
Core:
- `requires: [{kind, ref, version?, bind?}]` on programs + deployments — one
precondition relation; `system_dependencies` is its `{kind: system}` alias.
kind fixes meaning + check (system=installed, deployment=exists).
- relations.py: derives repos (git toplevel / monorepo), fan-in, and the
predicates functional?/fresh?/deployed? — nothing stored.
- env is generated FROM a `{kind: deployment, bind}` requirement (target URL →
consumer env), never scraped back into one; explicit defaults.env still wins.
- git.py: working-copy status/pull, repo toplevel + remote url.
Surfaces:
- `castle graph` + GET /graph — the relationship diagnostic.
- GET /repos, /repos/{key}/git|sync — repo-scoped sync (a repo is the sync unit;
a monorepo backs several programs). GET /programs/{name}/git|sync + repo context.
- Dashboard: Graph screen, program-page git status + repo-aware Sync, and a
monorepo banner on Programs.
Governing principle: predicates are derived; encode only the non-derivable, as a
node or edge property. Pull-only sync — converge stays a separate step.
Expose raw-TCP services (postgres) by name at <name>.<domain>:<port> and
cut the gateway's ACME wildcard cert onto them so they present a trusted
cert. Replaces the proxy/public booleans with a single `reach` enum
(off|internal|public); legacy input still parses via derived accessors.
Core:
- expose.tcp{port,tls} + TlsSpec(material: pair|combined|off, reload)
- tls.py: materialize cert files from Caddy's wildcard, reconcile on
renewal; `castle tls` CLI; optional cert_obtained events-exec hook
(gateway.cert_hook, gated so a plugin-less Caddy still parses)
- apply waits (bounded) for the wildcard to issue before materializing so
a fresh-node TLS service starts with its cert in place, then scopes
materialization to the deployments being applied
- reach: internal|public requires an expose block (no silent no-op);
public raw-TCP guarded until tunnel support lands
- chain.pem (${tls_ca}) is the issuer chain (leaf stripped), a real CA
bundle distinct from cert.pem
- one shared ${...} resolver (resolve_placeholders) for env and container
run fields; run.env now expands like volumes/args; $$ escapes a literal
- validate the generated Caddyfile before reloading the gateway so an
invalid config never degrades routing
Docs: docs/tcp-exposure.md. Tests cover reach/expose validation,
placeholder expansion + escape, issuer-chain material, TLS materialize.
The gateway is a deployment (castle-gateway) — its lifecycle is the same
convergence as everything else, so the bespoke start/stop/reload verbs
(which still routed through the retired enable/disable path) are gone.
- CLI: `castle gateway` is now inspection only — bare or `status` shows
the route table. Start/stop/reload it via `castle apply` (render routes
+ reload) or `castle restart castle-gateway`. Deletes the last users of
`_service_enable`/`_service_disable`, so those are removed too.
- API: retire `POST /gateway/reload` (making routes live is `POST /apply`);
`PUT /gateway/config` message + docstring now say apply, not deploy.
- UI: GatewayPanel's "Reload" button → "Apply" (useApply); drop the
useGatewayReload hook; GatewaySettings points at `castle apply`.
- docs: `castle gateway reload|status` → `castle gateway` (status lens).
core 129 / cli 31 / api 59 pass; dashboard builds clean; live `castle
gateway` shows routes with no start/stop/reload; /gateway/reload removed
from the API.
Rework the dashboard around convergence:
- ServiceControls / ServiceCard / JobCard: replace start/restart/stop
with a Power toggle (set enabled → apply, i.e. activate/deactivate)
plus Restart (the one imperative bounce).
- ToolDetail: install/uninstall becomes an Enable/Disable toggle;
`installed` stays the live state, `enabled` the desired one.
- ConfigPanel Apply and CreateDeploymentForm now POST /apply (one
converge that renders + restarts only what changed) instead of
/deploy + a separate start/restart.
- New ConvergePanel on Overview: a terraform-style Preview (POST /apply
plan=true) showing would-activate/restart/deactivate, then Apply.
- useApply + useSetEnabled hooks; `enabled` added to Service/Job/
Deployment types; both handle the self-restart connection drop.
Backend: PUT /config/deployments/{name}/enabled sets desired on/off
(edit config; caller runs apply) — the declarative toggle the UI uses.
Dashboard builds clean (tsc + vite); 59 API tests pass; live /apply and
/services?enabled verified after an api restart.
- Mount xterm as an absolute inset-0 fill so it always has an explicit box and
fit() measures the real size when the dock expands (was collapsing via the
flex/percentage height chain).
- Acquire the pty slave as the child's controlling terminal (setsid + TIOCSCTTY)
so the kernel delivers SIGWINCH on a browser resize. Previously the initial
size was read once but live resizes were dropped, so the pane stayed at its
startup size — content never grew with the window.
- Run agent sessions under an isolated tmux server (its own systemd scope, with a
keepalive session) so they survive a castle-api restart and stay rediscoverable;
each WebSocket is a tmux attach client. Falls back to the in-memory pty backend
when tmux is absent or CASTLE_API_AGENT_BACKEND=memory.
- Refactor the session layer behind one Attachment interface (create/list/attach/
close), keeping resume-by-id, agent-native history, and continue modes.
- Fix the terminal not refilling when the dock is maximized (explicit refit signal
+ instant size transition so xterm measures the final box).
- On narrow screens, go full-bleed and strip chrome (header/border/padding) when
maximized.
The field showing a deployment's launcher + target is now labelled 'Launch'
(consistent with launcher/manager vocabulary) on ServiceDetail and the Service/
Job/Create field editors.
ProgramList gains a filterable prop; the Programs page enables it. Chips filter by
whether a program has a deployment of that kind (membership over program.deployments),
so a tool-and-job program shows under both Tool and Job. Counts are programs-per-kind.
Tools page stays unfiltered (already scoped).
A static showed both 'Served at' (from the static block) and 'Subdomain' (the
general proxied-service field) — same URL twice. Gate 'Subdomain' to non-static;
a static's gateway URL is already 'Served at'.
- Tool detail showed installed tools as 'Not installed': /deployments/{name}
populated 'installed' for path deployments but left 'active' null, and
ToolDetail keyed off 'active'. Now the endpoint sets active=installed for path,
and ToolDetail reads 'installed' directly (verified: on-PATH ⇒ True, else False).
- A deployment now inherits its program's description by default: copied at
creation (CLI create + API _save_deployment on new deployments) + a one-time
patchup of existing ones (29 deployments).
- Tools page cards no longer list deployments (ProgramCard showDeployments=false) —
the lens already scopes to the tool; the deployment list is a Programs-catalog thing.
ProgramCard/ProgramList take an optional linkBase (default /programs); the Tools
page passes /tools so a tool tile opens /tools/<name> (its deployment detail),
not /programs/<name>.
- Nav: Services/Scheduled/Tools nested under a 'Deployments' group (expandable;
flattened to icons when the sidebar is collapsed). Programs stays top-level.
- Overview: Tools tile (Wrench, count of tool programs).
- New /tools/:name tool detail page (useDeployment): Install/Uninstall lifecycle,
a link up to its program, and an editable ToolFields (description + env).
- ServiceDetail is static-aware: a caddy deployment shows served-by-gateway +
URL + root, no start/stop, and edits via a new StaticFields (root + public + env)
so a launcher is never injected into a caddy deployment.
- Program detail's Deployment section is now a clean list of links — every
deployment (tool→/tools, service/static→/services, job→/jobs) links to its
detail page; lifecycle controls moved onto those pages.
clean pnpm build + tsc.
- Programs page: each card lists the program's deployments (name · kind) instead
of a single kind badge; kind-filter chips removed (nav is the kind lens now).
protonmail honestly shows tool + job.
- New Tools nav page (Wrench) — programs with a tool deployment (/programs?kind=tool).
- Services page now shows statics too: /services returns service + static, and a
static renders as a caddy-served card (KindBadge, served URL, no start/stop).
- Program detail: DeploymentsSection + deleteConfirm rebuilt over program.deployments;
header drops the (now nonexistent) program kind.
- ServiceFields / JobFields: launcher is now an editable select (python|command|
container|compose|node), persisted into run.launcher via the normal save.
clean pnpm build + tsc --noEmit.
- /services was returning every non-job deployment (tools, statics, remotes);
filter it to kind==service so the Services page shows only services.
- Programs page gains kind-filter chips (All / Service / Job / Tool / Static /
Reference) with per-kind counts, colored to match KindBadge.
- Replace window.confirm with a reusable ConfirmModal (styled, danger variant,
Esc/Enter, preserves bulleted bodies); FormFooter's delete uses it, so program/
service/job deletion now confirms in-app with the enumerated cascade list.
castle-api 58 passed; /services live drops from 37 to 15 (services only).
The confirm dialog (window.confirm in FormFooter) now spells out exactly what
cascade will do — uninstall the tool from PATH / drop the static route / stop,
disable and remove named service+job deployments, and remove the catalog entry —
each as its own bullet, ending with the reassurance that source on disk is kept.
A program and its 1:1 tool/static deployment are one thing to the user, so
'Delete program' now just works instead of refusing. DELETE /config/programs/
{name}?cascade=true tears down each referencing deployment (manager-aware:
uninstall a tool from PATH, stop+disable a service/job, drop a static route),
removes them and the program, then converges the runtime (prune orphan units,
regenerate the Caddyfile, reload the gateway). Without cascade it still 409s
(safe API default). The dashboard drops the 'can't remove while deployed' block,
labels the action 'Delete program', and the confirm names exactly what will go.
Verified live: throwaway tool → cascade delete removes program + path deployment,
real config untouched; plain delete of a referenced program still 409s.
castle-api 57 passed.
Install/Uninstall no longer live in the program's action cluster (they were a
tool's PATH-deployment lifecycle masquerading as a program action). ProgramActions
is now dev-verbs-only (build/test/lint/type-check/check). The program detail's
Deployment section owns lifecycle, keyed on kind: a tool shows Installed-on-PATH
with an Install/Uninstall toggle, a static shows served-by-gateway + its URL,
services/jobs link to their own pages. Program list cards drop the quick toggle
for a plain active dot. 'Remove program' stays as catalog deletion — no longer a
dead-end now that the blocking deployment is visible in the Deployment section.
