`try_files {path} {path}/index.html /index.html` for SPA sites and the
dashboard: a URL like /docs/setup/ serves that directory's own index.html
before falling back to the root one, so a multi-page static site works at its
sub-paths while single-page apps still get client-side routing.
Found uncommitted in the checkout, author unknown, and already live: wildpc
runs from this tree, so every Caddyfile generated since used it.
docs/mesh.md says what runs where (castle-nats, castle-openbao, each node's
wildpc-api), how to tell it is healthy, what to do when a node goes stale, the
vault is sealed or NATS is down, and the routine tasks -- written from the live
config, not the plan. The plan is done and goes; its history is in git, and its
unfinished items are tasks in Wild Life.
AGENTS.md used the services' old names (wildpc-nats, wildpc-openbao) and
pointed at the plan for operations; it now names castle-* and points here.
For services with no authentication of their own -- a trace viewer, an admin
UI, an OTLP receiver -- which must not be open to the LAN just because they run
here.
- `expose.http.basic_auth: {user, password_hash}` has the gateway ask for HTTP
Basic auth (Caddy's `basic_auth`, bcrypt from `caddy hash-password`) before
proxying, on the internal name and any public one. It is recorded in the
registry too, so a Caddyfile generated from the registry alone still asks
rather than serving the route open. Checked against caddy 2.11: 401 without
credentials, through with them.
- A container's `ports` may give the host side an address, `4318:
127.0.0.1:4318` -> `-p 127.0.0.1:4318:4318`. A bare port still publishes on
every interface, as before, and the docs now say that this means the LAN.
A `{ref, bind}` to a TCP-exposed deployment (e.g. postgres) projected
nothing, so consumers hardcoded the docker bridge IP. Project the address
it's already documented as reachable at.
- container launcher: `gpus:` passes through as docker `--gpus`.
- requires: `path:` is appended to the bound URL, for consumers that want a
sub-path as their base (e.g. an OpenAI-compatible server's /v1).
- container services now get `ExecStop=-docker stop wildpc-<name>`. SIGTERM
only reached the `docker run` client, so a stop/restart left the container
running unsupervised and holding its name, and the next start failed with
a name conflict until systemd gave up.
The dashboard's deployment kept the pre-rename name `wildpc`, so it was
published at wildpc.<domain> while everything else had moved to Wild PC
naming. Rename the deployment and its program to `wild-pc` (the subdomain
is the deployment name) and follow the `_DASHBOARD` constant in both the
Caddyfile generator and doctor — that constant also drives the `:9000`
redirect and doctor's `wildpc program build` hint, which is why the
program is renamed alongside the deployment.
Browsing the old name exposed a second problem: in acme mode every route
lives inside one `*.<domain>` wildcard site, so an unrouted host still
matches the site and completes its TLS handshake. Caddy then ran off the
end of the block and returned a bare 200 with an empty body — a typo'd or
renamed subdomain looked *up* in a browser while rendering nothing. Each
wildcard site now ends with an unmatched `handle` that responds 404.
The API subdomain is untouched: the dashboard derives its API origin by
stripping the first label, so wild-pc.<domain> still resolves to
wildpc-api.<domain>, and the WebSocket origin gate matches on
gateway.domain rather than a hostname.
Also seed the bootstrap dashboard deployment with the `requires:
wildpc-api` block the live deployment already carried, so a fresh install
declares the dependency.
Static (manager: caddy) deployments always emitted the SPA fallback
`try_files {path} /index.html`, which serves the root index.html for
every unmatched path. That's correct for React/Vite apps but swallows a
multi-page content site's in-page links back to the homepage, so a Hugo
site (payne.io) couldn't navigate off its landing page.
Add an explicit `spa: bool` on CaddyDeployment (default True, preserving
existing SPA behavior). When False, the gateway serves plain file_server
— resolving directory indexes (/posts/ -> /posts/index.html) and 404ing
missing paths. Threaded through the registry snapshot + mesh and the
route computation so both the internal wildcard and public apex blocks
honor it. `castle program create --stack hugo` now sets spa: false.
NATS tls:// + token and OpenBao HTTPS via the ACME wildcard cert (expose.tcp.tls),
no custom CA. civil + primer both cut over; plaintext rejected on both. Server
config + unseal.sh are instance-side (/data/castle, ~/.castle).
- SecretBackend gains write/delete/list_names; FileSecretBackend + OpenBaoBackend
implement them (vault KV-v2 POST/DELETE/LIST); castle-api/secrets.py routes all
CRUD through the active backend (dashboard writes to the vault in openbao mode)
- add systemd exec_start_post (general; '-' prefix so a hook failure can't fail
the unit); castle-openbao runs an unseal.sh on boot using OPENBAO_UNSEAL_KEY
- tests: file write/read/list/delete round-trip
Verified live: write→vault→read→list→delete against OpenBao; seal→restart→
auto-unsealed. TLS hardening remains (deferred; gates cross-network/real secrets).
- secret_backends.py: SecretBackend protocol, FileSecretBackend (historical),
OpenBaoBackend (KV-v2 read + file fallback), build_backend() env-selected
- _read_secret delegates to the active backend; default is file, so production
is unchanged until CASTLE_SECRET_BACKEND=openbao
- OpenBao token bootstraps from the file backend; missing/unreachable falls back
- tests: file hit/miss, backend selection, unreachable + empty-token fallback
Read path verified live against castle-openbao. Write path, auto-unseal-on-boot,
and TLS hardening documented as remaining for full OpenBao production use.
- add static 'role' (authority|follower) to NodeConfig/CastleConfig, wired from
castle.yaml through the registry to the mesh wire; civil pinned authority
- castle-presence: TTL KV bucket each node renews (churn signal); delete-on-stop
- castle-config: authority-gated get/put + change-watch SSE (follower reconcile
hook hangs off it)
- bound the NATS drain on stop() so shutdown can't hang
- tests: role config/registry round-trip, wire round-trip, role write-gating
Single-node verified; follower reconcile pending the second node.
Phased plan to evolve the mesh into a purpose-driven fleet: NATS (JetStream KV +
presence) replacing Mosquitto, OpenBao as the secret authority, cross-node
requires resolution + gateway circuit-breaker, static civil=authority role.
Records resolved decisions and Phase 0/1 progress.
The `castle` CLI and the `castle-api` service are two independent in-process
drivers of `castle_core`. Each resolved DATA_DIR/REPOS_DIR at import time from
its own process env (default /data/castle), persisted nowhere — so they silently
diverged, and `apply`/dashboard-apply crashed on a non-existent /data.
Make the loaded CastleConfig the single source of truth:
- Resolve data_dir/repos_dir only in load_config (env > castle.yaml > default),
anchored to the config root; drop the DATA_DIR/REPOS_DIR module globals and the
import-time file read entirely — no global twin that can disagree with the file.
- Thread config.data_dir/repos_dir through ensure_dirs, _env_context, tls_dir_for
(now unified — deploy no longer inlines the tls path), and create/add/clone.
- ensure_dirs raises an actionable CastleDirError instead of a bare PermissionError;
the api surfaces it as 422.
- doctor: "data dir writable" check + WARN when CASTLE_DATA_DIR/REPOS_DIR env
overrides the file (the one remaining cross-process divergence vector).
- install.sh persists data_dir/repos_dir into castle.yaml (idempotent, non-default).
- Docs: registry.md globals + AGENTS.md roots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Model:
- Drop `requires` from ProgramSpec; requires is now deployment-only.
Collapse Requirement to {ref, bind} with kind defaulting to "deployment"
(drop the unused `version`). System-package preconditions stay on the
program as `system_dependencies`, synthesized into {kind: system} for the
functional check. relations.py/deploy.py read only the deployment's requires.
- create.py seeds a stack's substrate dependency onto the deployment, not the
program. Docs + tests updated.
UI:
- Add a `requires` editor (useRequires) on service, job, and static detail
pages — edit service→service deps (ref + optional bind env var).
- Restore the gateway route table "Open" column: a kind-aware in-app link to
each route's deployment.
- Reformat the reach control: name baked into each option
(localhost:PORT (local) / <name>.<domain> (internal) / <name>.<pubdomain> (public)).
- Services page: sort statics in with systemd services by name, add a
search + kind-filter bar mirroring the programs page.
Frontend builds triggered from the castle web app run inside the castle-api
systemd service, whose PATH omits nvm's versioned node dir — so `pnpm build`
died with `node: not found` even though it worked from an interactive shell.
Introduce a per-program node convention: a program declares its version the
ecosystem-standard way (.node-version / .nvmrc / package.json engines.node),
and castle_core.toolchains.resolve_node_bin() maps it to a concrete nvm bin dir
(CASTLE_NODE_VERSIONS_DIR, default ~/.nvm/versions/node; newest match wins).
The same resolver feeds both sites that run a program's node:
- build time: stacks._build_env() prepends the pinned node for the dev-verb
subprocess (keyed on the source dir), so `castle program build` uses the
program's node regardless of caller.
- run time: deploy._build_deployed() stores it in Deployment.path_prepend,
which the systemd generator puts ahead of the default unit PATH — so a
`launcher: node` service runs its program's node.
A pinned-but-uninstalled version fails loud with an `nvm install` hint instead
of a cryptic `node: not found`. Unpinned → no injection (no guessing).
Also: the systemd generator now honors an explicit PATH in defaults.env as a
full override instead of clobbering it with a trailing Environment=PATH line
(systemd's last-assignment-wins rule had silently defeated the documented
escape hatch).
Pins app/.node-version and documents the convention in the react-vite stack.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A derived, mostly-computed model of how programs, deployments, and repos relate,
plus the git-sync surfaces that motivated it. See docs/relationships.md.
Core:
- `requires: [{kind, ref, version?, bind?}]` on programs + deployments — one
precondition relation; `system_dependencies` is its `{kind: system}` alias.
kind fixes meaning + check (system=installed, deployment=exists).
- relations.py: derives repos (git toplevel / monorepo), fan-in, and the
predicates functional?/fresh?/deployed? — nothing stored.
- env is generated FROM a `{kind: deployment, bind}` requirement (target URL →
consumer env), never scraped back into one; explicit defaults.env still wins.
- git.py: working-copy status/pull, repo toplevel + remote url.
Surfaces:
- `castle graph` + GET /graph — the relationship diagnostic.
- GET /repos, /repos/{key}/git|sync — repo-scoped sync (a repo is the sync unit;
a monorepo backs several programs). GET /programs/{name}/git|sync + repo context.
- Dashboard: Graph screen, program-page git status + repo-aware Sync, and a
monorepo banner on Programs.
Governing principle: predicates are derived; encode only the non-derivable, as a
node or edge property. Pull-only sync — converge stays a separate step.
Expose raw-TCP services (postgres) by name at <name>.<domain>:<port> and
cut the gateway's ACME wildcard cert onto them so they present a trusted
cert. Replaces the proxy/public booleans with a single `reach` enum
(off|internal|public); legacy input still parses via derived accessors.
Core:
- expose.tcp{port,tls} + TlsSpec(material: pair|combined|off, reload)
- tls.py: materialize cert files from Caddy's wildcard, reconcile on
renewal; `castle tls` CLI; optional cert_obtained events-exec hook
(gateway.cert_hook, gated so a plugin-less Caddy still parses)
- apply waits (bounded) for the wildcard to issue before materializing so
a fresh-node TLS service starts with its cert in place, then scopes
materialization to the deployments being applied
- reach: internal|public requires an expose block (no silent no-op);
public raw-TCP guarded until tunnel support lands
- chain.pem (${tls_ca}) is the issuer chain (leaf stripped), a real CA
bundle distinct from cert.pem
- one shared ${...} resolver (resolve_placeholders) for env and container
run fields; run.env now expands like volumes/args; $$ escapes a literal
- validate the generated Caddyfile before reloading the gateway so an
invalid config never degrades routing
Docs: docs/tcp-exposure.md. Tests cover reach/expose validation,
placeholder expansion + escape, issuer-chain material, TLS materialize.
Cross-checked every doc against the code and fixed the stragglers the
apply/gateway/subdomain refactors left behind:
- Retired `tls: internal` mode: dropped the misleading reference in
registry.md and the stale comment in config.py GatewayConfig (only
off|acme exist; the dns-and-tls.md "was removed" note is kept as
accurate history).
- Subdomain-only routing: fixed "path-based routing" claims in design.md
(proxy list + mesh), "path prefix or host" → host, and the "path-prefix
routes stay on :port" line in registry.md.
- Frontends serve at their subdomain root (VITE_BASE=/), not /<name>/:
fixed supabase.md (x2), the scaffold.py docstring, the stacks.py build
comment, and design.md's "serve prefix baked in" line.
README, AGENTS, developing-castle, dns-and-tls audited clean. Verified
against code; core 129 / cli 31 pass; ruff clean.
The apply/converge + gateway refactors let the docs collapse a lot:
- design.md §Component Lifecycle: the build→install→deploy three-step
("enables and starts services") becomes two moves — develop, then
`castle apply` (activation polymorphic over the manager). Filesystem
layout de-duped and corrected (program source is /data/repos/<name>,
not the long-removed ~/.castle/code).
- registry.md §Lifecycle: three near-identical per-kind flows (service/
tool/job) collapse into one flow + a "what apply does per manager"
table. Manager table's "start/stop" column → "how apply activates".
Fixed stale paths-table rows (code/<name>) and a stale `service run`.
- developing-castle.md endpoint list: `POST /apply` is the one lifecycle
endpoint; dropped retired `/deploy`; service actions are restart-only;
added `/config/deployments/{name}/enabled`.
Net −37 lines; no retired-verb or stale-path references remain in docs.
The gateway is a deployment (castle-gateway) — its lifecycle is the same
convergence as everything else, so the bespoke start/stop/reload verbs
(which still routed through the retired enable/disable path) are gone.
- CLI: `castle gateway` is now inspection only — bare or `status` shows
the route table. Start/stop/reload it via `castle apply` (render routes
+ reload) or `castle restart castle-gateway`. Deletes the last users of
`_service_enable`/`_service_disable`, so those are removed too.
- API: retire `POST /gateway/reload` (making routes live is `POST /apply`);
`PUT /gateway/config` message + docstring now say apply, not deploy.
- UI: GatewayPanel's "Reload" button → "Apply" (useApply); drop the
useGatewayReload hook; GatewaySettings points at `castle apply`.
- docs: `castle gateway reload|status` → `castle gateway` (status lens).
core 129 / cli 31 / api 59 pass; dashboard builds clean; live `castle
gateway` shows routes with no start/stop/reload; /gateway/reload removed
from the API.
Update the README, AGENTS.md, install.sh, and docs/ for the converge
model: one `castle apply [name] [--plan]` replaces `deploy && start` and
the per-kind enable/disable/install/start/stop verbs. CLI references now
list apply + restart (the imperative bounce) + logs; recipes use
`castle apply`; "turn it off" is documented as `enabled: false` + apply.
install.sh's next-steps collapse to a single `castle apply`.
Verified: no retired-verb command instructions remain in any markdown;
install.sh syntax OK; live `castle apply --plan` reports converged;
doctor all-green; core 129 / cli 31 / api 59 pass.
castle deploy now manages the public zone's tunnel CNAMEs to match the current
set of public: true services — creating missing records and deleting removed
ones — instead of only printing manual 'cloudflared tunnel route dns' hints.
New core/generators/dns.py reconciles via the Cloudflare API (stdlib only). It
only ever touches records whose content is <tunnel_id>.cfargotunnel.com, so
hand-managed records in the same zone are never altered. deploy._write_tunnel_config
calls it in both branches (delete of the last public service cleans up its CNAME).
Without a token it's a no-op and the manual route-once hints are surfaced as before.
Token: ~/.castle/secrets/CLOUDFLARE_PUBLIC_DNS_TOKEN, a single DNS:Edit permission
scoped to the public zone (the 'Edit zone DNS' template) — confirmed sufficient:
that one permission resolves the zone by name and edits records, no separate
Zone:Read needed. Separate from CLOUDFLARE_API_TOKEN (ACME, internal zone).
castle doctor gains a read-only probe that WARNs when the token is absent (CNAMEs
stay manual) and FAILs when it can't reach the zone/records.
Rework the supabase stack so every app owns an isolated Postgres schema
(= program name) instead of sharing `public` with a table_prefix. This
gives a clean, knowable teardown and drops migration version tracking to
per-app, so version tokens can no longer collide across apps.
- stacks: add StackHandler.owns_data + teardown() (default no-op). Supabase
build now creates+grants the app schema, tracks migrations in
<schema>.schema_migrations, and applies each with search_path set to the
schema (write unqualified names). teardown = drop schema <app> cascade.
Fix _substrate_db_url default port 5432 -> 5433 (substrate's direct PG).
- deploy: derive ${supabase_app_schemas} from registered apps; substrate maps
PGRST_DB_SCHEMAS: public,storage,graphql_public${supabase_app_schemas} so
PostgREST exposes each app schema (restart substrate after add/remove).
- delete: generic owns_data -> teardown path behind new --purge-data flag,
replacing the hardcoded supabase remnant note.
- scaffold + docs: new apps born schema-isolated (unqualified tables,
db:{schema}, schema: in supabase.app.yaml).
The tools lens coding assistants need. 'castle tool list [--json]' lists tools
(programs with a manager:path deployment) surfacing each tool's actual EXECUTABLE
— resolved from pyproject [project.scripts], which can differ from the program
name (litellm-intent-router → intent-router) — plus description and install state.
'castle tool info <name> [--json]' details one; 'castle tool install|uninstall'.
--json is a clean machine-readable catalog for building context.
Sweep: main.py descriptions/help mention the tool lens; 'frontend'→'static' in
install help; add.py._detect drops the dead behavior return (add adopts source
only); stale comments/help fixed. Docs (CLAUDE.md/README/registry) document
'castle tool'. Tests: cli/tests/test_tool.py (list/info/json).
Suites: core 124, cli 29, castle-api 58.
Replace the conflated `runner` axis with two orthogonal ones: `manager`
(systemd|caddy|path|none) — who supervises/realizes a deployment — and, for
systemd only, a nested `launcher` (python|command|container|compose|node) — how
the process starts. ServiceSpec and JobSpec collapse into one manager-
discriminated DeploymentSpec union (Systemd/Caddy/Path/Remote); the services/
and jobs/ config dirs collapse into one deployments/ dir. The human "kind"
(service|job|tool|static|reference) is fully derived (kind_for), never stored —
the frontend kind is renamed static. behavior is gone.
- core: DeploymentSpec union + LaunchSpec + kind_for; legacy-aware loader
normalizes old runner shapes; CastleConfig.deployments with derived
services/jobs/tools views; registry.Deployment carries manager/launcher/kind.
- cli: service/job/tool as filtered views + a deployment group; --behavior→--kind,
create --runner→--launcher; lifecycle dispatches over config.deployments.
- castle-api: /deployments primary with /services,/jobs as views; summaries
derive kind; PUT/DELETE /config/deployments/{name} (services/jobs aliased).
- app: KindBadge, frontend→static everywhere, pick-a-kind creation wizard,
per-kind config editors.
- docs: single deployments/ layout, manager/launcher, static kind throughout.
Live migration verified byte-identical: regenerated Caddyfile and every unit
ExecStart line unchanged, so nothing restarted. Suites: core 124, cli 25,
castle-api 55; dashboard build + type-check clean.
- A static service is inherently exposed, so `public: true` no longer trips the
"requires proxy" validator — frontends can be public via the tunnel for free
(the original friction that started this refactor, now resolved).
- docs/registry.md: the manager model (systemd/caddy/path/none), the runner→manager
table incl. static/path, and behavior as a derived display label.
Two features built this session, committed together as a checkpoint before the
deployment-model refactor (they share files, so kept as one rollback point):
- Public exposure via Cloudflare tunnel: a `public` toggle on services (requires
proxy), gateway.public_domain/tunnel_id config, a cloudflared ingress generator
(generators/tunnel.py) that maps <name>.<public_domain> to the gateway's
internal host, deploy wiring, and docs/tunnel-setup.md.
- Frontends as `static`-runner services: a RunStatic runner served by Caddy
(file_server), no systemd unit (like `remote`); the Caddyfile generator now
derives static routes from services instead of a behavior==frontend branch.
Services often need their own public origin — the URL a browser loads them
at through the gateway — to populate CORS/WebSocket/secure-context allowlists.
Previously this had to be hardcoded in the app's own config, which silently
broke when the subdomain-routing migration changed the address (openclaw's
Control UI rejected its new origin).
${public_url} resolves at deploy to the service's gateway-facing base URL:
https://<name>.<gateway.domain> when exposed under tls: acme, else the
node-local http://localhost:<port>. It tracks gateway.domain, so a domain
change needs no per-app edit. Jobs (not exposed) don't get it.
Matches the TLS mode as the raw lowercased string == "acme", the same way the
Caddyfile generator does (the TLSMode enum is stale).
CaddySpec had only `enable` left (path_prefix/host removed; extra_snippets was
unused), so `proxy: { caddy: {} }` was three levels of nesting for a checkbox —
and it invited drift (supabase silently reverted to proxy.caddy.host). Replace it
with a plain `ServiceSpec.proxy: bool`.
- Model: remove ProxySpec/CaddySpec; `proxy: bool = False`. Readers simplified to
`bool(svc.proxy)` (generator, castle-api summaries, CLI info).
- CLI create / dashboard editor + create form: write `proxy: true` / a checkbox.
- Configs migrated to `proxy: true` (and supabase's stale proxy.caddy.host block
removed).
- Docs (registry, dns-and-tls, design, stack guides) + tests/fixtures updated;
also fixed lingering path-model staleness in the stack guides.
The dashboard is now served at castle.<domain> (was castle-app.<domain>). Since
the subdomain is the program name, this is just a program rename plus the one
special-case constant.
- Generator: _DASHBOARD = "castle" (the :9000 redirect target and off-mode root).
- Program file renamed (runtime): programs/castle-app.yaml -> programs/castle.yaml
(source unchanged: repo:app).
- Dashboard UI: ProgramDetail shows a frontend's subdomain (via subdomainUrl),
dropping the old castle-app-at-root special-case; comment refs updated.
- Docs: registry.md/design.md/react-vite.md updated to the subdomain model and the
`castle` name (react-vite.md also corrected — frontends serve at their subdomain
root with VITE_BASE=/, and builds are manual via `castle program build`, not
deploy).
Collapse the two proxy shapes (path_prefix + host) to a single checkbox: a
service is either port-only, or exposed at <service-name>.<gateway.domain>. Path
routes and their prefix-stripping foot-guns are gone; the subdomain is always the
service name (rename the service to change it).
- Schema: CaddySpec is just `enable` (presence = expose). service_proxy_targets
returns (expose, port, base_url); Deployment carries `subdomain` (was
proxy_path/proxy_host). Registry/deploy/mesh/CLI updated in lockstep.
- Generator: compute_routes emits one host route per exposed service/frontend
(address = name). acme mode = one *.<domain> site with a reverse_proxy matcher
per service + a file_server matcher per frontend; the :<port> site redirects to
the dashboard. off mode = a HTTP control plane on :<port> (dashboard at / +
/api → castle-api); other services are port-only.
- Dashboard/API: castle-app and castle-api are ordinary subdomains. The dashboard
derives the API base at runtime (castle-api.<domain> on a subdomain, else /api)
and calls it cross-origin — castle-api already allows CORS *. Frontends build
with VITE_BASE=/ (served at their subdomain root).
- CLI: `service create` drops --path/--host; --no-proxy makes it port-only.
- Docs/tests reworked for the model; docs use generic placeholders only (no
personal host/domain/IP details).
acme mode fully replaced the internal-CA path (which required installing a private
CA on every device — the exact pain, esp. on Android, that acme avoids). Remove
`internal` entirely and simplify how services declare host routes.
- Generator (caddyfile.py): drop the `tls internal` branch — modes are now
`off` | `acme`. In acme mode the published subdomain is the **first DNS label**
of `proxy.caddy.host` (a bare `claw`, or a legacy `claw.civil.lan`, both →
`claw.<domain>`), so services stay domain-agnostic and the declared value is
authoritative again (no more silent service-name override). Shared
`_host_matcher_block` reused by off-mode and the acme wildcard site.
- castle-api: delete `GET /gateway/ca.crt`, `_gateway_ca_pem`, `_ca_fingerprint`
and the now-unused imports; drop `ca_fingerprint` from `GatewayInfo` (keep
`tls`).
- Dashboard: remove the CA-cert download button + unused imports; drop
`ca_fingerprint` from the `GatewayInfo` type.
- Tests: replace TestCaddyfileTlsInternal with an off-mode class (keeps the
runner-agnostic host-route coverage); acme tests assert first-label derivation
incl. label-wins-over-service-name; drop the castle-api CA-endpoint test.
- Docs: registry.md + dns-and-tls.md — two-mode tables (off|acme), remove the
internal sections/CA-download, document the bare-label host convention; note a
domain-less node stays on `off`.
A conceptual companion to registry.md's field-level gateway reference. Covers the
two questions (resolve / trust) an internal-only LAN service must answer:
- DNS: gateway-as-ingress, path-prefix vs host routes, LAN name resolution,
wildcard records, split-horizon (internal names, no public A records), and the
private-TLD vs public-subdomain zone styles with the worked router/wild-central
topology.
- TLS: the three gateway.tls modes (off / internal / acme) with when-to-use, the
internal-CA distribution pain vs the acme DNS-01 "no client setup" win, the
secure-context (WebCrypto) rationale, the put-a-service-on-HTTPS recipe,
operational prerequisites (443/80 bind, plugin caddy, DNS token, staging), and a
decision table.
Linked from CLAUDE.md's docs list.
internal-CA HTTPS forces every device to trust a private root, which breaks down
on some clients (Android browsers). acme mode instead obtains a real Let's Encrypt
wildcard cert (*.<domain>) via a DNS-01 challenge, so browsers trust it with zero
CA install — while services stay internal-only (no inbound exposure, no public A
records; only a transient _acme-challenge TXT touches the public zone, and LAN DNS
resolves the names to a private IP).
- Config: GatewayConfig/NodeConfig gain domain, acme_email, acme_dns_provider
(plumbed through load/save + registry + deploy, omitted-when-empty for
round-trip stability). tls stays the discriminator: off | internal | acme.
- Caddyfile generator: acme branch emits a global {email, acme_dns <provider>
{env.TOKEN}} block + one *.<domain> wildcard site with host matchers derived
from the service name (<name>.<domain>); path/static stay on :<port>. Shared
_host_matcher_block helper (reused by off-mode). CASTLE_ACME_STAGING=1 toggles
the LE staging CA; acme-without-domain falls back to HTTP with a warning.
- deploy(): _acme_preflight warns (never writes) when domain, the gateway-service
token env, or the CLOUDFLARE_API_TOKEN secret is missing.
- install.sh: opt-in --with-dns-plugin[=provider] builds a DNS-plugin Caddy via
xcaddy (pinned) to /usr/local/bin/caddy, which the gateway picks up on deploy.
- Tests: TestCaddyfileTlsAcme (global block, wildcard site, derived host matcher,
path routes on :port, staging toggle, no-domain fallback). Docs: gateway.tls
modes table + full acme/DNS-01 section (setup, wild-central LAN DNS, staging).
Adds "a stack whose default is a substrate": a shared self-hosted Supabase
backend plus a `--stack supabase` that scaffolds per-app projects (migrations +
edge functions + static UI) which deploy against it. Apps own their code and
stay repo-durable; only their rows/blobs live on the shared substrate.
- compose runner: new RunCompose supervises a multi-container stack as one
systemd unit (ExecStart=`compose up`, generated ExecStop=`compose down`);
secrets via EnvironmentFile. Reusable beyond Supabase. Deployment.stop_cmd
carries the teardown command through the registry.
- supabase stack: CLI --stack choice, STACK_DEFAULTS→frontend with
build.outputs=[public], _scaffold_supabase() (migrations/RLS/functions/
static UI/app manifest), and SupabaseHandler with a forward-only idempotent
migration runner (plan_migrations + psql build) and deno dev-verbs.
- docs: docs/stacks/supabase.md, registered in CLAUDE.md; compose runner
documented in registry.md.
- tests: compose run/stop cmd, systemd ExecStop, compose host-route TLS,
migration planner, supabase verb resolution, create --stack supabase.
When the gateway serves internal-CA HTTPS (gateway.tls=internal), other devices
must trust Caddy's root CA to accept *.lan HTTPS. Surface that from the dashboard
instead of hand-copying root.crt around.
- API: GET /gateway/ca.crt returns the public root cert (PEM) as a download,
sourced from Caddy's admin API (/pki/ca/local — matches the running gateway)
with an on-disk root.crt fallback. 404 unless tls=internal. GatewayInfo gains
`tls` and `ca_fingerprint` (SHA-256, for out-of-band verification). The CA
private key is never exposed — only the public root.
- UI: a "CA cert" button in the Gateway panel, shown only when tls=internal,
linking to the download with the fingerprint in its tooltip.
- Tests for the tls=off defaults and the 404-without-internal-tls path; docs note
the button + endpoint.
The gateway was HTTP-only by design, but a plain-HTTP origin is not a browser
"secure context", so apps that need WebCrypto (device identity, E2E crypto)
can't run when proxied — only localhost or HTTPS qualifies.
Add an opt-in `gateway.tls: internal`. When set, each host route
(proxy.caddy.host) is emitted as its own `<host> { tls internal; reverse_proxy }`
site, served over HTTPS by Caddy's local CA (Caddy binds :443 and redirects :80).
Path-prefix/static routes stay on the HTTP :<port> site — give a service a host
route to put it on HTTPS. Default (unset/off) keeps the existing HTTP-only
Caddyfile (single :<port> block, auto_https off, host matchers inline).
Plumbing mirrors gateway_port: GatewayConfig.tls (castle.yaml load/save) →
NodeConfig.gateway_tls (registry load/save, set in deploy) → the Caddyfile
generator. Tests cover both modes; docs/registry.md documents the option plus
the two operational requirements (binding 443/80 via the unprivileged-port
sysctl, and trusting Caddy's root CA on clients) and the host-route-vs-prefix
guidance that motivates it.
- Introduced `_write_castle_config` function to scatter nested config into separate YAML files for programs, services, and jobs.
- Updated `castle_root` fixture to utilize the new config writing method.
- Added tests for configuration aggregation and scattering in `test_health.py`.
- Removed legacy unit handling from `manifest.py` and refactored related code in `config.py`.
- Updated documentation to reflect new configuration directory structure.
- Removed obsolete unit tests related to unit expansion.
A python-runner service ran from a tool venv (uv tool install --editable),
separate from the project venv that uv sync manages. The editable link kept
source live, but dependencies were resolved independently in each venv, so a
new dependency landed in the project venv and never reached the running
service — leaving no clean rollout for it.
Deploy now emits ExecStart=uv run --project <source> --no-dev <program>,
running the service in place from its own project venv. uv run syncs the env
to the lockfile before launching, so a single restart picks up both code and
dependency changes; the ExecStart is deterministic from source and never goes
stale. The tool-venv install is gated to the command runner only; uv tool
install is reserved for tool-behavior programs, where PATH is the point. A
python service with no resolvable source falls back to a PATH lookup.
The gateway does two things — reverse-proxy services and serve static
frontends — but the dashboard/API route table only ever showed the proxy
routes, so 'serving a frontend' and 'proxying a service' looked like unrelated
features and castle-app/power-graph-app were invisible in the route view.
Now there's one concept: a route maps an address (path '/foo' or host 'foo.lan')
to a target of one kind — static (a built dist served by file_server), proxy
(a local service port), or remote (a service on another node).
- core: caddyfile.py gains compute_routes() — the single source of truth for
the route list; generate_caddyfile_from_registry renders it (output byte-for-
byte identical, verified against the live Caddyfile). Adds a GatewayRoute
dataclass.
- api: GatewayRoute model → {address, kind, target, name, node}; GET /gateway
builds from compute_routes (incl. config for static frontends + mesh for
remote), so the table matches what Caddy actually does.
- app: Gateway panel shows Address · Kind · Target for every route (static
frontends + host routes now appear); program detail shows 'Reachable at
/foo/ · served (static)' for static frontends.
- cli: 'castle gateway status' prints the full route table.
- docs: registry.md/design.md/CLAUDE.md describe routes as one concept with
three target kinds.
core 94 / cli 24 / api 52 green; ruff + app build clean; Caddyfile unchanged.
A service/job's env is now exactly its defaults.env — castle injects no hidden
convention vars. Values support ${port}/${data_dir}/${name} placeholders
(resolved at deploy, alongside ${secret:…}), so a program's own env var names
map to castle's computed values without hardcoding.
Why: the auto-injected <PREFIX>_PORT/<PREFIX>_DATA_DIR were a guess at the
program's env names — right for castle-scaffolded services, dead weight for
adopted ones (lakehouse carried two dead vars; notification-bridge/backup jobs
too). They also weren't visible in the config editor (computed at deploy), which
was the source of the 'four env vars but the UI shows none' mystery.
- core: resolve_env_vars gains a context (${port}/${data_dir}/${name});
deploy builds env from defaults.env only — no <PREFIX>_* injection, no
port_env. Removed the port_env field and the dead _env_prefix helper.
- cli: 'service/job create' gains repeatable --env KEY=VALUE (replaces
--port-env); 'program create' scaffolds <PREFIX>_PORT/_DATA_DIR: ${…} for new
daemons.
- app: removed the 'Port env' field; the Environment editor (defaults.env) is
the single place, with a placeholder hint.
- live migration: central-context/castle-api/power-graph/protonmail mapped their
real vars explicitly; lakehouse → just LAKEHOUSED_DAEMON_PORT: ${port}, data
stays in ~/.lakehoused. Verified all services healthy on their ports, dead
vars gone, zero failed units.
- docs: registry.md/design.md/stack guides + findings updated to the explicit
model.
core 94 / cli 24 / api 52 green; ruff + app build clean.
Sweep of accumulated drift unrelated to the CLI reorg:
- Source paths: ~/.castle/code/<name> → /data/repos/<name> (REPOS_DIR), and
source: code/X → absolute /data/repos/X. registry.md source-resolution table
and 'how programs get in' section rewritten for the /data/repos layout.
- Old field name component: → program: in all YAML examples.
- Old term 'component' → 'program' throughout design.md and the stack guides.
- Frontend serving: corrected the copy-to-artifacts/content model to serve-in-
place from the repo (<source>/<dist>), matching Phase 2b; dropped the dead
_copy_app_static reference and the 'runtime never references the source tree'
claim (frontends are the deliberate exception).
- Stale endpoints/commands: /components → /deployments; castle create/add →
castle program create/add in the create.py/add.py docstrings.
CLAUDE.md was already current. cli 24 green; ruff clean.
Names collide across resource types (a program and a service can share a name),
so the CLI no longer resolves bare names. Operations live under the resource
they act on; platform lifecycle and the cross-resource overview stay top-level.
castle program list|info|create|add|clone|delete|run|install|uninstall|
build|test|lint|format|type-check|check
castle service list|info|create|delete|deploy|enable|disable|start|stop|restart|logs
castle job <same verbs> (create takes --schedule)
castle gateway start|stop|reload|status
castle list | status | deploy [name] # cross-cutting
castle start | stop | restart # whole platform (systemd verbs)
Key changes:
- info/delete/list/run are resource-scoped (no more cross-section name match).
- delete only removes the named resource's entry; program delete still blocks
when a deployment references it.
- per-resource start/stop/restart (systemctl on the unit/timer); top-level
start/stop/restart act on the whole platform.
- expose → 'service create' (more general: --program ref, --runner); gained
'job create' (the CLI never had one).
- dropped up/down — bringing things online is the honest 'deploy && start'.
Docs (CLAUDE.md, registry.md, stack guides) and user-facing messages updated.
core 94 / cli 24 / api 52 green; ruff clean. Verified live: program/service/job
list+info scope correctly, service create+scoped-delete round-trips.
