_target_url only resolved http_exposed deployments, so a `requires` with a
`bind` pointing at a `manager: none` reference (an external resource) injected
nothing. Return the reference's base_url, so binding an external endpoint into a
consumer's env works — e.g. litellm's vllm-payne reference → VLLM_API_BASE.
Graph model gains, all derived (GET /graph auto-exposes via asdict):
- endpoints[{protocol,port}] + reach per node — reusing http_exposed/tcp_port,
so raw-TCP infra (postgres/neo4j/mqtt) is finally visible and edges can be
protocol-typed by the target's socket.
- base_url for reference (external) nodes.
- provides/consumes capability types, activating the dormant ProgramSpec fields.
New surfaces:
- core/audit.py + GET /graph/suggestions: SUGGESTS undeclared `requires` by
matching a deployment's env endpoint values against provider sockets. Never
writes; the graph stays declaration-derived (docs/relationships.md).
- kind-scoped PUT/DELETE /config/references/{name} for external resources.
- GET /mesh/deployments (flattened remote deployments + derived endpoints), and
the mesh MQTT payload now carries tcp_port + base_url (forward-compatible,
still no secrets) so peers can resolve cross-node endpoints.
_reload_gateway validated the generated Caddyfile in castle-api's own
process env, which lacks CLOUDFLARE_API_TOKEN. Under acme the Caddyfile
references {env.CLOUDFLARE_API_TOKEN}, so `caddy validate` failed on an
empty token and the reload was silently skipped — every API-driven apply
left the running Caddy stale (route changes never took effect until a
manual reload). Validate with the gateway service's own resolved env so
the DNS-provider token is present.
Model:
- Drop `requires` from ProgramSpec; requires is now deployment-only.
Collapse Requirement to {ref, bind} with kind defaulting to "deployment"
(drop the unused `version`). System-package preconditions stay on the
program as `system_dependencies`, synthesized into {kind: system} for the
functional check. relations.py/deploy.py read only the deployment's requires.
- create.py seeds a stack's substrate dependency onto the deployment, not the
program. Docs + tests updated.
UI:
- Add a `requires` editor (useRequires) on service, job, and static detail
pages — edit service→service deps (ref + optional bind env var).
- Restore the gateway route table "Open" column: a kind-aware in-app link to
each route's deployment.
- Reformat the reach control: name baked into each option
(localhost:PORT (local) / <name>.<domain> (internal) / <name>.<pubdomain> (public)).
- Services page: sort statics in with systemd services by name, add a
search + kind-filter bar mirroring the programs page.
A deployment's identity becomes (name, kind), so a tool, service, and job can
share a name (a `backup` trio). Kind is structural — which per-kind store / dir —
not a derived discriminator threaded around.
- CastleConfig holds per-kind stores (services/jobs/tools/statics/references);
all_deployments()/deployments_named()/deployment(kind,name) replace the single
flat `deployments` dict. A construction-only `deployments` InitVar routes a flat
map into the stores by kind_for.
- Storage: deployments/<store>/<name>.yaml (per-kind dirs), with read-compat for
the old flat + services/+jobs/ layouts.
- Registry keyed by composite "<kind>/<name>" + name field; get()/all()/named()/put().
- Job units carry a -job marker (castle-<name>-job.{service,timer}); services and
everything else unchanged — so a service and a job can coexist.
- deploy/apply/lifecycle thread kind through unit naming, activate/deactivate,
is_active; caddyfile/tunnel/relations iterate via all_deployments/registry.all.
- Subdomain uniqueness validated across kinds (a service and a static can't share one).
182 core tests pass, incl. a new same-name-trio suite (round-trip, distinct units,
subdomain guard). API/CLI/UI updated in follow-up phases.
Deployment/program edits could silently drop spec fields:
- GET /deployments/{name} served the runtime view (no reach/program/root/expose)
for deployed deployments, so the dashboard round-tripped a lossy manifest and
stripped fields on save (broke astro). Now serves the editable castle.yaml spec
when the deployment is in config.
- _save_deployment and save_program replaced the whole object with client input.
Now shallow-merge over the existing spec (omit = preserve, null = clear), so a
partial/lossy save can't drop untouched fields.
- save_yaml rebuilt GatewayConfig from `port` only, wiping tls/domain/tunnel/
cert_hook on a whole-file save. Now uses a shared parse_gateway() (also used by
load_config) so gateway fields can't drift between the two.
Dashboard forms (ServiceFields/StaticFields) send explicit null to clear, per the
merge contract; adds exposure host-label helpers. Coverage: detail-serves-spec,
save round-trip, partial-patch-preserves, null-clears, program partial-patch,
parse_gateway, and config save/load round-trip.
Frontend builds triggered from the castle web app run inside the castle-api
systemd service, whose PATH omits nvm's versioned node dir — so `pnpm build`
died with `node: not found` even though it worked from an interactive shell.
Introduce a per-program node convention: a program declares its version the
ecosystem-standard way (.node-version / .nvmrc / package.json engines.node),
and castle_core.toolchains.resolve_node_bin() maps it to a concrete nvm bin dir
(CASTLE_NODE_VERSIONS_DIR, default ~/.nvm/versions/node; newest match wins).
The same resolver feeds both sites that run a program's node:
- build time: stacks._build_env() prepends the pinned node for the dev-verb
subprocess (keyed on the source dir), so `castle program build` uses the
program's node regardless of caller.
- run time: deploy._build_deployed() stores it in Deployment.path_prepend,
which the systemd generator puts ahead of the default unit PATH — so a
`launcher: node` service runs its program's node.
A pinned-but-uninstalled version fails loud with an `nvm install` hint instead
of a cryptic `node: not found`. Unpinned → no injection (no guessing).
Also: the systemd generator now honors an explicit PATH in defaults.env as a
full override instead of clobbering it with a trailing Environment=PATH line
(systemd's last-assignment-wins rule had silently defeated the documented
escape hatch).
Pins app/.node-version and documents the convention in the react-vite stack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
`system_dependencies` holds apt PACKAGE names; checking `which <name>` only
coincides with "installed" when the name equals its command (pandoc, rsync). Names
like poppler-utils / texlive-latex-base / docker-compose-plugin have no binary of
that name, so the functional? predicate wrongly reported them unmet. Fall back to
`dpkg -s` (the real "installed" check) after the PATH fast-path.
A derived, mostly-computed model of how programs, deployments, and repos relate,
plus the git-sync surfaces that motivated it. See docs/relationships.md.
Core:
- `requires: [{kind, ref, version?, bind?}]` on programs + deployments — one
precondition relation; `system_dependencies` is its `{kind: system}` alias.
kind fixes meaning + check (system=installed, deployment=exists).
- relations.py: derives repos (git toplevel / monorepo), fan-in, and the
predicates functional?/fresh?/deployed? — nothing stored.
- env is generated FROM a `{kind: deployment, bind}` requirement (target URL →
consumer env), never scraped back into one; explicit defaults.env still wins.
- git.py: working-copy status/pull, repo toplevel + remote url.
Surfaces:
- `castle graph` + GET /graph — the relationship diagnostic.
- GET /repos, /repos/{key}/git|sync — repo-scoped sync (a repo is the sync unit;
a monorepo backs several programs). GET /programs/{name}/git|sync + repo context.
- Dashboard: Graph screen, program-page git status + repo-aware Sync, and a
monorepo banner on Programs.
Governing principle: predicates are derived; encode only the non-derivable, as a
node or edge property. Pull-only sync — converge stays a separate step.
Expose raw-TCP services (postgres) by name at <name>.<domain>:<port> and
cut the gateway's ACME wildcard cert onto them so they present a trusted
cert. Replaces the proxy/public booleans with a single `reach` enum
(off|internal|public); legacy input still parses via derived accessors.
Core:
- expose.tcp{port,tls} + TlsSpec(material: pair|combined|off, reload)
- tls.py: materialize cert files from Caddy's wildcard, reconcile on
renewal; `castle tls` CLI; optional cert_obtained events-exec hook
(gateway.cert_hook, gated so a plugin-less Caddy still parses)
- apply waits (bounded) for the wildcard to issue before materializing so
a fresh-node TLS service starts with its cert in place, then scopes
materialization to the deployments being applied
- reach: internal|public requires an expose block (no silent no-op);
public raw-TCP guarded until tunnel support lands
- chain.pem (${tls_ca}) is the issuer chain (leaf stripped), a real CA
bundle distinct from cert.pem
- one shared ${...} resolver (resolve_placeholders) for env and container
run fields; run.env now expands like volumes/args; $$ escapes a literal
- validate the generated Caddyfile before reloading the gateway so an
invalid config never degrades routing
Docs: docs/tcp-exposure.md. Tests cover reach/expose validation,
placeholder expansion + escape, issuer-chain material, TLS materialize.
Cross-checked every doc against the code and fixed the stragglers the
apply/gateway/subdomain refactors left behind:
- Retired `tls: internal` mode: dropped the misleading reference in
registry.md and the stale comment in config.py GatewayConfig (only
off|acme exist; the dns-and-tls.md "was removed" note is kept as
accurate history).
- Subdomain-only routing: fixed "path-based routing" claims in design.md
(proxy list + mesh), "path prefix or host" → host, and the "path-prefix
routes stay on :port" line in registry.md.
- Frontends serve at their subdomain root (VITE_BASE=/), not /<name>/:
fixed supabase.md (x2), the scaffold.py docstring, the stacks.py build
comment, and design.md's "serve prefix baked in" line.
README, AGENTS, developing-castle, dns-and-tls audited clean. Verified
against code; core 129 / cli 31 pass; ruff clean.
Reconcile the running system to declared config in a single operation,
replacing the old deploy && start plus the per-kind enable/disable/
install/uninstall verbs. The mechanism varies by manager (systemd unit /
PATH install / gateway route); the verb never does.
Core (castle_core.deploy.apply):
- render (units, Caddyfile, tunnel) then reconcile: activate every
enabled deployment that's down, restart any whose unit bytes changed,
deactivate the disabled ones. Returns ApplyResult (the enacted diff).
- --plan computes the diff and touches nothing.
- restart-on-change is exact: _render_unit_files is the single source of
truth for unit bytes, used both to write units and to predict restarts,
so the plan can't drift from what deploy writes.
Desired state:
- DeploymentBase.enabled (default True) — the declarative on/off. apply
converges to it; a disabled deployment is defined but not running and
gets no gateway route (else it would 502). Registry omits enabled when
True, keeping existing registries byte-identical.
CLI:
- add `castle apply [name] [--plan]`; keep `restart` as the one
imperative bounce, plus status/doctor/logs/gateway/program.
- retire deploy, start, stop (top-level); service/job deploy|enable|
disable|start|stop; tool install|uninstall; program install|uninstall.
- next-step strings + doctor hints now say `castle apply`.
Verified: core 129 + cli 31 green; live `castle apply` on a converged
node reports "nothing to do" with no spurious restarts; --plan shows an
empty diff; gateway + api stay up.
castle deploy now manages the public zone's tunnel CNAMEs to match the current
set of public: true services — creating missing records and deleting removed
ones — instead of only printing manual 'cloudflared tunnel route dns' hints.
New core/generators/dns.py reconciles via the Cloudflare API (stdlib only). It
only ever touches records whose content is <tunnel_id>.cfargotunnel.com, so
hand-managed records in the same zone are never altered. deploy._write_tunnel_config
calls it in both branches (delete of the last public service cleans up its CNAME).
Without a token it's a no-op and the manual route-once hints are surfaced as before.
Token: ~/.castle/secrets/CLOUDFLARE_PUBLIC_DNS_TOKEN, a single DNS:Edit permission
scoped to the public zone (the 'Edit zone DNS' template) — confirmed sufficient:
that one permission resolves the zone by name and edits records, no separate
Zone:Read needed. Separate from CLOUDFLARE_API_TOKEN (ACME, internal zone).
castle doctor gains a read-only probe that WARNs when the token is absent (CNAMEs
stay manual) and FAILs when it can't reach the zone/records.
Rework the supabase stack so every app owns an isolated Postgres schema
(= program name) instead of sharing `public` with a table_prefix. This
gives a clean, knowable teardown and drops migration version tracking to
per-app, so version tokens can no longer collide across apps.
- stacks: add StackHandler.owns_data + teardown() (default no-op). Supabase
build now creates+grants the app schema, tracks migrations in
<schema>.schema_migrations, and applies each with search_path set to the
schema (write unqualified names). teardown = drop schema <app> cascade.
Fix _substrate_db_url default port 5432 -> 5433 (substrate's direct PG).
- deploy: derive ${supabase_app_schemas} from registered apps; substrate maps
PGRST_DB_SCHEMAS: public,storage,graphql_public${supabase_app_schemas} so
PostgREST exposes each app schema (restart substrate after add/remove).
- delete: generic owns_data -> teardown path behind new --purge-data flag,
replacing the hardcoded supabase remnant note.
- scaffold + docs: new apps born schema-isolated (unqualified tables,
db:{schema}, schema: in supabase.app.yaml).
_TUNNEL_NAME was "tunnel", but the deployment is named castle-tunnel, so
unit_name() produced castle-tunnel.service — a unit that doesn't exist (the
real one is castle-castle-tunnel.service). The is-active check never matched,
so deploy printed "Tunnel not running" and skipped the ingress reload. New
public: true services then 404'd until a manual tunnel restart.
Set _TUNNEL_NAME = "castle-tunnel" to match the deployment name, mirroring
_GATEWAY_NAME. Deploy now restarts cloudflared on ingress changes.
A program's package name can differ from the console script it exposes
(litellm-intent-router installs the 'intent-router' executable), so the
which(<program>) install-check missed it: the tool installed fine but castle kept
reporting 'not installed', making the Install button look dead. _on_path now also
consults 'uv tool list' by package name (2s-cached), and the API's install-checks
use the shared tool_installed() helper. Verified litellm-intent-router now reads
installed/active True.
A program has no single kind — it HAS deployments, each with its own kind (a
program can be a tool AND a job, e.g. protonmail). Remove program-level kind:
- core: drop ProgramSpec.kind; CastleConfig.kind_of → deployments_of(name) →
[(deployment-name, kind)]; tools property derives from a tool deployment.
- api: ProgramSummary drops kind/services/jobs → deployments: [{name, kind}];
/programs?kind= filters by deployment-kind membership; a program's legacy
DeploymentSummary carries kind=None.
- cli: list/info show a program's set of deployment kinds; --kind filters by
membership.
Also: Services page now covers statics — /services returns kind in
{service, static} (both are exposed, URL-reachable 'services', caddy vs systemd),
and ServiceSummary gains kind + manager to distinguish them.
Suites: core 124, cli 25, castle-api 58.
Replace the conflated `runner` axis with two orthogonal ones: `manager`
(systemd|caddy|path|none) — who supervises/realizes a deployment — and, for
systemd only, a nested `launcher` (python|command|container|compose|node) — how
the process starts. ServiceSpec and JobSpec collapse into one manager-
discriminated DeploymentSpec union (Systemd/Caddy/Path/Remote); the services/
and jobs/ config dirs collapse into one deployments/ dir. The human "kind"
(service|job|tool|static|reference) is fully derived (kind_for), never stored —
the frontend kind is renamed static. behavior is gone.
- core: DeploymentSpec union + LaunchSpec + kind_for; legacy-aware loader
normalizes old runner shapes; CastleConfig.deployments with derived
services/jobs/tools views; registry.Deployment carries manager/launcher/kind.
- cli: service/job/tool as filtered views + a deployment group; --behavior→--kind,
create --runner→--launcher; lifecycle dispatches over config.deployments.
- castle-api: /deployments primary with /services,/jobs as views; summaries
derive kind; PUT/DELETE /config/deployments/{name} (services/jobs aliased).
- app: KindBadge, frontend→static everywhere, pick-a-kind creation wizard,
per-kind config editors.
- docs: single deployments/ layout, manager/launcher, static kind throughout.
Live migration verified byte-identical: regenerated Caddyfile and every unit
ExecStart line unchanged, so nothing restarted. Suites: core 124, cli 25,
castle-api 55; dashboard build + type-check clean.
behavior (tool/daemon/frontend) is now a derived display label computed
from a program's deployments — path->tool, static->frontend, process->daemon
(behavior_for_runner). load_config populates it so every reader gets the live
value; _spec_to_yaml_dict excludes it so it is never written to disk.
_summary_from_service derives it from the runner instead of hardcoding daemon.
Fixtures gain path/python deployments so tool/daemon behaviors derive.
Surfaces the DNS/Cloudflare/public state that previously required editing
castle.yaml + curl:
- API: GET /gateway now returns domain, public_domain, tunnel_id,
tunnel_connected, and a public_url per public route; new PUT /gateway/config
edits tls/domain/public_domain/tunnel_id in castle.yaml (deploy to apply).
save_config now persists public_domain/tunnel_id (previously dropped).
- UI: a GatewaySettings row in the Gateway panel shows TLS / domain / public
(tunnel up/down + public_domain) with inline edit; each public route gets a
green "public" badge linking to its <name>.<public_domain> URL.
- A static service is inherently exposed, so `public: true` no longer trips the
"requires proxy" validator — frontends can be public via the tunnel for free
(the original friction that started this refactor, now resolved).
- docs/registry.md: the manager model (systemd/caddy/path/none), the runner→manager
table incl. static/path, and behavior as a derived display label.
- manifest: RunPath (runner: path) — a CLI on PATH via `uv tool install`, manager
= path, no unit/route (like remote/static).
- lifecycle/service: a path deployment's start/stop = install/uninstall (routed
through activate/deactivate, which skip the reinstall when already on PATH).
`castle service start/stop <tool>` and status now dispatch by manager; status
shows the manager tag [systemd|caddy|path|none].
- config.tools() now derives from `runner: path` services, not the behavior label.
- castle-api: the "installed" flag derives from manager==path, not behavior==tool.
- create: a python-cli stack emits a `runner: path` service.
`behavior` no longer drives any runtime logic — it survives only as a display
label + `list --behavior` filter. Verified live: html2text uninstall→reinstall
round-trip via `castle service stop/start`.
- manifest: `manager_for(runner)` — the single source of truth mapping a runner to
its manager (systemd | caddy | path | none). deploy's `managed` now derives from it.
- lifecycle: is_active/activate/deactivate dispatch by manager instead of behavior
(removed the stale `_is_static_frontend`, which broke once frontends became
services). caddy → gateway reload; path → install/uninstall; none → no-op.
- cli/service: start/stop/restart and enable/disable route through the manager
(systemctl only for systemd; static reloads the gateway; remote is a no-op);
platform restart skips non-systemd runners.
- create: a frontend stack (react-vite/supabase) now emits a `runner: static`
service, not a bare frontend program.
Two features built this session, committed together as a checkpoint before the
deployment-model refactor (they share files, so kept as one rollback point):
- Public exposure via Cloudflare tunnel: a `public` toggle on services (requires
proxy), gateway.public_domain/tunnel_id config, a cloudflared ingress generator
(generators/tunnel.py) that maps <name>.<public_domain> to the gateway's
internal host, deploy wiring, and docs/tunnel-setup.md.
- Frontends as `static`-runner services: a RunStatic runner served by Caddy
(file_server), no systemd unit (like `remote`); the Caddyfile generator now
derives static routes from services instead of a behavior==frontend branch.
Services often need their own public origin — the URL a browser loads them
at through the gateway — to populate CORS/WebSocket/secure-context allowlists.
Previously this had to be hardcoded in the app's own config, which silently
broke when the subdomain-routing migration changed the address (openclaw's
Control UI rejected its new origin).
${public_url} resolves at deploy to the service's gateway-facing base URL:
https://<name>.<gateway.domain> when exposed under tls: acme, else the
node-local http://localhost:<port>. It tracks gateway.domain, so a domain
change needs no per-app edit. Jobs (not exposed) don't get it.
Matches the TLS mode as the raw lowercased string == "acme", the same way the
Caddyfile generator does (the TLSMode enum is stale).
The proxy->bool refactor (770574f) missed the shared CLI test fixture, which
still used proxy: {caddy: {path_prefix: ...}} — failing ~25 CLI tests with
'Input should be a valid boolean'. Set it to proxy: True.
Also drop the now-dead 'caddy' entry from _STRUCTURAL_KEYS (no nested caddy
key exists anymore) and refresh a stale proxy.caddy docstring reference.
The dashboard's stack dropdown hardcoded its options (python-cli/fastapi/
react-vite), so a supabase program showed "(none)" — the value had no matching
option. Make the backend the single source of truth instead:
- core: available_stacks() returns sorted(HANDLERS) — the stacks castle has
handlers for.
- castle-api: GET /stacks exposes it.
- cli: --stack choices derive from available_stacks() (drops the duplicated list).
- app: ProgramFields fetches /stacks (useStacks) and renders options from it,
always including the current value so it never silently blanks; StackBadge
gets a Supabase label.
A new backend stack now appears everywhere without a frontend edit.
TLSMode (off/internal/letsencrypt) was only referenced by HttpPublic.tls,
and HttpPublic (the expose.http.public field, still carrying a path_prefix
and a tls defaulting to the deleted internal mode) was referenced by no
config, generator, deploy code, or test — a leftover from the pre-proxy:bool
exposure design. Drop the enum, the model, the public field, and both CLI
shim re-exports.
CaddySpec had only `enable` left (path_prefix/host removed; extra_snippets was
unused), so `proxy: { caddy: {} }` was three levels of nesting for a checkbox —
and it invited drift (supabase silently reverted to proxy.caddy.host). Replace it
with a plain `ServiceSpec.proxy: bool`.
- Model: remove ProxySpec/CaddySpec; `proxy: bool = False`. Readers simplified to
`bool(svc.proxy)` (generator, castle-api summaries, CLI info).
- CLI create / dashboard editor + create form: write `proxy: true` / a checkbox.
- Configs migrated to `proxy: true` (and supabase's stale proxy.caddy.host block
removed).
- Docs (registry, dns-and-tls, design, stack guides) + tests/fixtures updated;
also fixed lingering path-model staleness in the stack guides.
The dashboard is now served at castle.<domain> (was castle-app.<domain>). Since
the subdomain is the program name, this is just a program rename plus the one
special-case constant.
- Generator: _DASHBOARD = "castle" (the :9000 redirect target and off-mode root).
- Program file renamed (runtime): programs/castle-app.yaml -> programs/castle.yaml
(source unchanged: repo:app).
- Dashboard UI: ProgramDetail shows a frontend's subdomain (via subdomainUrl),
dropping the old castle-app-at-root special-case; comment refs updated.
- Docs: registry.md/design.md/react-vite.md updated to the subdomain model and the
`castle` name (react-vite.md also corrected — frontends serve at their subdomain
root with VITE_BASE=/, and builds are manual via `castle program build`, not
deploy).
Collapse the two proxy shapes (path_prefix + host) to a single checkbox: a
service is either port-only, or exposed at <service-name>.<gateway.domain>. Path
routes and their prefix-stripping foot-guns are gone; the subdomain is always the
service name (rename the service to change it).
- Schema: CaddySpec is just `enable` (presence = expose). service_proxy_targets
returns (expose, port, base_url); Deployment carries `subdomain` (was
proxy_path/proxy_host). Registry/deploy/mesh/CLI updated in lockstep.
- Generator: compute_routes emits one host route per exposed service/frontend
(address = name). acme mode = one *.<domain> site with a reverse_proxy matcher
per service + a file_server matcher per frontend; the :<port> site redirects to
the dashboard. off mode = a HTTP control plane on :<port> (dashboard at / +
/api → castle-api); other services are port-only.
- Dashboard/API: castle-app and castle-api are ordinary subdomains. The dashboard
derives the API base at runtime (castle-api.<domain> on a subdomain, else /api)
and calls it cross-origin — castle-api already allows CORS *. Frontends build
with VITE_BASE=/ (served at their subdomain root).
- CLI: `service create` drops --path/--host; --no-proxy makes it port-only.
- Docs/tests reworked for the model; docs use generic placeholders only (no
personal host/domain/IP details).
acme mode fully replaced the internal-CA path (which required installing a private
CA on every device — the exact pain, esp. on Android, that acme avoids). Remove
`internal` entirely and simplify how services declare host routes.
- Generator (caddyfile.py): drop the `tls internal` branch — modes are now
`off` | `acme`. In acme mode the published subdomain is the **first DNS label**
of `proxy.caddy.host` (a bare `claw`, or a legacy `claw.civil.lan`, both →
`claw.<domain>`), so services stay domain-agnostic and the declared value is
authoritative again (no more silent service-name override). Shared
`_host_matcher_block` reused by off-mode and the acme wildcard site.
- castle-api: delete `GET /gateway/ca.crt`, `_gateway_ca_pem`, `_ca_fingerprint`
and the now-unused imports; drop `ca_fingerprint` from `GatewayInfo` (keep
`tls`).
- Dashboard: remove the CA-cert download button + unused imports; drop
`ca_fingerprint` from the `GatewayInfo` type.
- Tests: replace TestCaddyfileTlsInternal with an off-mode class (keeps the
runner-agnostic host-route coverage); acme tests assert first-label derivation
incl. label-wins-over-service-name; drop the castle-api CA-endpoint test.
- Docs: registry.md + dns-and-tls.md — two-mode tables (off|acme), remove the
internal sections/CA-download, document the bare-label host convention; note a
domain-less node stays on `off`.
internal-CA HTTPS forces every device to trust a private root, which breaks down
on some clients (Android browsers). acme mode instead obtains a real Let's Encrypt
wildcard cert (*.<domain>) via a DNS-01 challenge, so browsers trust it with zero
CA install — while services stay internal-only (no inbound exposure, no public A
records; only a transient _acme-challenge TXT touches the public zone, and LAN DNS
resolves the names to a private IP).
- Config: GatewayConfig/NodeConfig gain domain, acme_email, acme_dns_provider
(plumbed through load/save + registry + deploy, omitted-when-empty for
round-trip stability). tls stays the discriminator: off | internal | acme.
- Caddyfile generator: acme branch emits a global {email, acme_dns <provider>
{env.TOKEN}} block + one *.<domain> wildcard site with host matchers derived
from the service name (<name>.<domain>); path/static stay on :<port>. Shared
_host_matcher_block helper (reused by off-mode). CASTLE_ACME_STAGING=1 toggles
the LE staging CA; acme-without-domain falls back to HTTP with a warning.
- deploy(): _acme_preflight warns (never writes) when domain, the gateway-service
token env, or the CLOUDFLARE_API_TOKEN secret is missing.
- install.sh: opt-in --with-dns-plugin[=provider] builds a DNS-plugin Caddy via
xcaddy (pinned) to /usr/local/bin/caddy, which the gateway picks up on deploy.
- Tests: TestCaddyfileTlsAcme (global block, wildcard site, derived host matcher,
path routes on :port, staging toggle, no-domain fallback). Docs: gateway.tls
modes table + full acme/DNS-01 section (setup, wild-central LAN DNS, staging).
Adds "a stack whose default is a substrate": a shared self-hosted Supabase
backend plus a `--stack supabase` that scaffolds per-app projects (migrations +
edge functions + static UI) which deploy against it. Apps own their code and
stay repo-durable; only their rows/blobs live on the shared substrate.
- compose runner: new RunCompose supervises a multi-container stack as one
systemd unit (ExecStart=`compose up`, generated ExecStop=`compose down`);
secrets via EnvironmentFile. Reusable beyond Supabase. Deployment.stop_cmd
carries the teardown command through the registry.
- supabase stack: CLI --stack choice, STACK_DEFAULTS→frontend with
build.outputs=[public], _scaffold_supabase() (migrations/RLS/functions/
static UI/app manifest), and SupabaseHandler with a forward-only idempotent
migration runner (plan_migrations + psql build) and deno dev-verbs.
- docs: docs/stacks/supabase.md, registered in CLAUDE.md; compose runner
documented in registry.md.
- tests: compose run/stop cmd, systemd ExecStop, compose host-route TLS,
migration planner, supabase verb resolution, create --stack supabase.
The gateway was HTTP-only by design, but a plain-HTTP origin is not a browser
"secure context", so apps that need WebCrypto (device identity, E2E crypto)
can't run when proxied — only localhost or HTTPS qualifies.
Add an opt-in `gateway.tls: internal`. When set, each host route
(proxy.caddy.host) is emitted as its own `<host> { tls internal; reverse_proxy }`
site, served over HTTPS by Caddy's local CA (Caddy binds :443 and redirects :80).
Path-prefix/static routes stay on the HTTP :<port> site — give a service a host
route to put it on HTTPS. Default (unset/off) keeps the existing HTTP-only
Caddyfile (single :<port> block, auto_https off, host matchers inline).
Plumbing mirrors gateway_port: GatewayConfig.tls (castle.yaml load/save) →
NodeConfig.gateway_tls (registry load/save, set in deploy) → the Caddyfile
generator. Tests cover both modes; docs/registry.md documents the option plus
the two operational requirements (binding 443/80 via the unprivileged-port
sysctl, and trusting Caddy's root CA on clients) and the host-route-vs-prefix
guidance that motivates it.
Editing a service's routing (port, path_prefix, host) and then hitting
"apply" or "gateway reload" — rather than a full deploy — left the running
Caddyfile reflecting the old routing. The Caddyfile was generated from the
derived registry.yaml snapshot, which only `deploy()` rebuilds from
castle.yaml; apply/reload regenerated from that stale snapshot.
Make castle.yaml the single source of truth for local routes:
- caddyfile.py: new shared `service_proxy_targets()` derives a service's
(proxy_path, proxy_host, port, base_url) from its spec. `compute_routes`
now builds local routes from `config.services` when castle.yaml is
loadable, falling back to `registry.deployed` only when it isn't. Since
deploy/apply/reload all funnel through `compute_routes`, every regeneration
path now tracks the spec. Mesh/remote and static-frontend routes unchanged.
- deploy.py: `_build_deployed_service` uses the same shared deriver, so the
written registry and the computed routes can't diverge.
- config_editor.py: `/config/apply` now runs a full `deploy()` (rebuilding
units + Caddyfile and reloading the gateway) before restarting services, so
changed unit ExecStarts actually take effect. Scheduled jobs are no longer
restarted on apply. Drops the now-unused direct caddy-reload path.
- Regression tests: a stale registry's old port/path loses to castle.yaml;
registry fallback still works when config is unavailable.
- Introduced `_write_castle_config` function to scatter nested config into separate YAML files for programs, services, and jobs.
- Updated `castle_root` fixture to utilize the new config writing method.
- Added tests for configuration aggregation and scattering in `test_health.py`.
- Removed legacy unit handling from `manifest.py` and refactored related code in `config.py`.
- Updated documentation to reflect new configuration directory structure.
- Removed obsolete unit tests related to unit expansion.
A python-runner service ran from a tool venv (uv tool install --editable),
separate from the project venv that uv sync manages. The editable link kept
source live, but dependencies were resolved independently in each venv, so a
new dependency landed in the project venv and never reached the running
service — leaving no clean rollout for it.
Deploy now emits ExecStart=uv run --project <source> --no-dev <program>,
running the service in place from its own project venv. uv run syncs the env
to the lockfile before launching, so a single restart picks up both code and
dependency changes; the ExecStart is deterministic from source and never goes
stale. The tool-venv install is gated to the command runner only; uv tool
install is reserved for tool-behavior programs, where PATH is the point. A
python service with no resolvable source falls back to a PATH lookup.
pnpm 10+ runs a deps-status check before `pnpm <script>` that wants to
purge and reinstall node_modules, aborting in a non-TTY build with
ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY. It is a false positive for a
build verb (which builds, not installs), so ReactViteHandler now prepends
--config.verify-deps-before-run=false to every pnpm call, building against
the existing modules. CI=true is kept as a general non-interactive guard.
Generated systemd units hardcoded a thin PATH (~/.local/bin + system dirs)
while dev-verb builds used a separate _EXTRA_PATH_DIRS list, so a service
could be built and run with different tools on PATH. Hoist a single
USER_TOOL_PATH_DIRS in config.py, consumed by both stacks._build_env
(build) and the systemd unit generator (runtime).
Also prefer pnpm's modern $PNPM_HOME/bin shim over the bare $PNPM_HOME dir
(older installs leave a stale version wrapper there) and add
/usr/local/go/bin so Go-based builds/services resolve their toolchain.
The check composite returned ActionResult with no output on success, so the
dashboard's 'Check All' (and the CLI) showed nothing when checks passed — only
a failing sub-verb ever produced output. It now accumulates each sub-verb's
output with a ✓/✗ marker (lint / type-check / test), on both success and
failure, so the output panel always shows what ran.
Also fixes lan-info's type-check: socket.getaddrinfo sockaddr is loosely typed,
so str(infos[0][4][0]) makes the address an unambiguous str.
core 94 green; ruff clean; verified POST /programs/lan-info/check returns output.
handle_path /foo/* matches /foo/ but not the bare /foo, so the no-trailing-
slash form fell through to the root catch-all (castle-app) — which then 404'd
because its router has no such route. Hitting http://gateway:9000/power-graph-app
served the dashboard's 404 instead of Power Graph; same for every proxied
service accessed without the slash.
The generator now emits 'redir /foo /foo/' for each path-prefix route (static,
proxy, remote), so the no-slash form canonicalizes to the slash form that
handle_path matches. Caddy's bare path matcher is exact, so /foo/bar is
unaffected. Host routes and the root '/' are unaffected.
Verified: /power-graph-app → 302 → Power Graph; caddy validate clean; core 94 /
api 52 green.
The gateway does two things — reverse-proxy services and serve static
frontends — but the dashboard/API route table only ever showed the proxy
routes, so 'serving a frontend' and 'proxying a service' looked like unrelated
features and castle-app/power-graph-app were invisible in the route view.
Now there's one concept: a route maps an address (path '/foo' or host 'foo.lan')
to a target of one kind — static (a built dist served by file_server), proxy
(a local service port), or remote (a service on another node).
- core: caddyfile.py gains compute_routes() — the single source of truth for
the route list; generate_caddyfile_from_registry renders it (output byte-for-
byte identical, verified against the live Caddyfile). Adds a GatewayRoute
dataclass.
- api: GatewayRoute model → {address, kind, target, name, node}; GET /gateway
builds from compute_routes (incl. config for static frontends + mesh for
remote), so the table matches what Caddy actually does.
- app: Gateway panel shows Address · Kind · Target for every route (static
frontends + host routes now appear); program detail shows 'Reachable at
/foo/ · served (static)' for static frontends.
- cli: 'castle gateway status' prints the full route table.
- docs: registry.md/design.md/CLAUDE.md describe routes as one concept with
three target kinds.
core 94 / cli 24 / api 52 green; ruff + app build clean; Caddyfile unchanged.
A service/job's env is now exactly its defaults.env — castle injects no hidden
convention vars. Values support ${port}/${data_dir}/${name} placeholders
(resolved at deploy, alongside ${secret:…}), so a program's own env var names
map to castle's computed values without hardcoding.
Why: the auto-injected <PREFIX>_PORT/<PREFIX>_DATA_DIR were a guess at the
program's env names — right for castle-scaffolded services, dead weight for
adopted ones (lakehouse carried two dead vars; notification-bridge/backup jobs
too). They also weren't visible in the config editor (computed at deploy), which
was the source of the 'four env vars but the UI shows none' mystery.
- core: resolve_env_vars gains a context (${port}/${data_dir}/${name});
deploy builds env from defaults.env only — no <PREFIX>_* injection, no
port_env. Removed the port_env field and the dead _env_prefix helper.
- cli: 'service/job create' gains repeatable --env KEY=VALUE (replaces
--port-env); 'program create' scaffolds <PREFIX>_PORT/_DATA_DIR: ${…} for new
daemons.
- app: removed the 'Port env' field; the Environment editor (defaults.env) is
the single place, with a placeholder hint.
- live migration: central-context/castle-api/power-graph/protonmail mapped their
real vars explicitly; lakehouse → just LAKEHOUSED_DAEMON_PORT: ${port}, data
stays in ~/.lakehoused. Verified all services healthy on their ports, dead
vars gone, zero failed units.
- docs: registry.md/design.md/stack guides + findings updated to the explicit
model.
core 94 / cli 24 / api 52 green; ruff + app build clean.
Names collide across resource types (a program and a service can share a name),
so the CLI no longer resolves bare names. Operations live under the resource
they act on; platform lifecycle and the cross-resource overview stay top-level.
castle program list|info|create|add|clone|delete|run|install|uninstall|
build|test|lint|format|type-check|check
castle service list|info|create|delete|deploy|enable|disable|start|stop|restart|logs
castle job <same verbs> (create takes --schedule)
castle gateway start|stop|reload|status
castle list | status | deploy [name] # cross-cutting
castle start | stop | restart # whole platform (systemd verbs)
Key changes:
- info/delete/list/run are resource-scoped (no more cross-section name match).
- delete only removes the named resource's entry; program delete still blocks
when a deployment references it.
- per-resource start/stop/restart (systemctl on the unit/timer); top-level
start/stop/restart act on the whole platform.
- expose → 'service create' (more general: --program ref, --runner); gained
'job create' (the CLI never had one).
- dropped up/down — bringing things online is the honest 'deploy && start'.
Docs (CLAUDE.md, registry.md, stack guides) and user-facing messages updated.
core 94 / cli 24 / api 52 green; ruff clean. Verified live: program/service/job
list+info scope correctly, service create+scoped-delete round-trips.
- ProgramActions: install/uninstall (activate) shown only for tools and
no-service frontends. Daemons activate via a service/job, so the program
page no longer offers a misleading Install/Uninstall (which secretly
enabled/disabled the service).
- Program page: new Deployments section listing the services/jobs that run
the program, as links (a program → 0-N services/jobs).
- Service page: 'Runs' row from the run_target summary field (the old code
dug into manifest.run, which was empty pre-Stage-1), a Host row for
proxy_host, and a convenience link to the referenced program.
- lifecycle.activate on a daemon with no service now errors toward
'castle expose' instead of silently installing its binary to PATH.
core 94 green; ruff + app build clean.
Driven by adopting lakehouse (an existing daemon that bundles its own SPA) —
see docs/findings-lakehouse.md.
#3 deploy reloads the gateway. castle deploy regenerated the Caddyfile but
left the running Caddy on the old config, so new proxy routes were silently
dead. Deploy now reloads the gateway when it's running.
#1 castle expose <program>. Turns an adopted program into a service
(run/port/health/proxy/systemd) in one command — the missing
daemon-to-service step. Flags: --port --health --path --run --port-env
--host --no-proxy.
#2 port_env mapping. A service can declare expose.http.internal.port_env so
castle sets the env var the program actually reads (e.g. lakehoused reads
LAKEHOUSED_DAEMON_PORT, not castle's convention LAKEHOUSE_PORT). Castle now
genuinely drives an adopted daemon's bind port.
#4a auto-base for react-vite. The build passes VITE_BASE = the gateway serve
prefix (/<name>/, or / for castle-app); vite.config reads it. A castle-built
frontend now works at its subpath with no hand-tuned base. castle-app's
vite.config updated as the reference.
#4b host-based routing. proxy.caddy.host routes a whole hostname to the backend
root via a host matcher inside the :9000 site, so a root-based SPA (base="/")
serves unchanged — the fix for proxying an app castle can't rebuild. Caddyfile
now emits 'auto_https off' (HTTP-only gateway on a non-standard port).
Nit: activate skips the editable reinstall when the tool is already on PATH.
Tests: core 94, cli 24, api 52; ruff + app build clean. Verified live: lakehouse
runs under systemd, API at /lakehouse, full UI (SPA boots) via host routing.
