The gateway was HTTP-only by design, but a plain-HTTP origin is not a browser
"secure context", so apps that need WebCrypto (device identity, E2E crypto)
can't run when proxied — only localhost or HTTPS qualifies.
Add an opt-in `gateway.tls: internal`. When set, each host route
(proxy.caddy.host) is emitted as its own `<host> { tls internal; reverse_proxy }`
site, served over HTTPS by Caddy's local CA (Caddy binds :443 and redirects :80).
Path-prefix/static routes stay on the HTTP :<port> site — give a service a host
route to put it on HTTPS. Default (unset/off) keeps the existing HTTP-only
Caddyfile (single :<port> block, auto_https off, host matchers inline).
Plumbing mirrors gateway_port: GatewayConfig.tls (castle.yaml load/save) →
NodeConfig.gateway_tls (registry load/save, set in deploy) → the Caddyfile
generator. Tests cover both modes; docs/registry.md documents the option plus
the two operational requirements (binding 443/80 via the unprivileged-port
sysctl, and trusting Caddy's root CA on clients) and the host-route-vs-prefix
guidance that motivates it.
Editing a service's routing (port, path_prefix, host) and then hitting
"apply" or "gateway reload" — rather than a full deploy — left the running
Caddyfile reflecting the old routing. The Caddyfile was generated from the
derived registry.yaml snapshot, which only `deploy()` rebuilds from
castle.yaml; apply/reload regenerated from that stale snapshot.
Make castle.yaml the single source of truth for local routes:
- caddyfile.py: new shared `service_proxy_targets()` derives a service's
(proxy_path, proxy_host, port, base_url) from its spec. `compute_routes`
now builds local routes from `config.services` when castle.yaml is
loadable, falling back to `registry.deployed` only when it isn't. Since
deploy/apply/reload all funnel through `compute_routes`, every regeneration
path now tracks the spec. Mesh/remote and static-frontend routes unchanged.
- deploy.py: `_build_deployed_service` uses the same shared deriver, so the
written registry and the computed routes can't diverge.
- config_editor.py: `/config/apply` now runs a full `deploy()` (rebuilding
units + Caddyfile and reloading the gateway) before restarting services, so
changed unit ExecStarts actually take effect. Scheduled jobs are no longer
restarted on apply. Drops the now-unused direct caddy-reload path.
- Regression tests: a stale registry's old port/path loses to castle.yaml;
registry fallback still works when config is unavailable.
- Introduced `_write_castle_config` function to scatter nested config into separate YAML files for programs, services, and jobs.
- Updated `castle_root` fixture to utilize the new config writing method.
- Added tests for configuration aggregation and scattering in `test_health.py`.
- Removed legacy unit handling from `manifest.py` and refactored related code in `config.py`.
- Updated documentation to reflect new configuration directory structure.
- Removed obsolete unit tests related to unit expansion.
A python-runner service ran from a tool venv (uv tool install --editable),
separate from the project venv that uv sync manages. The editable link kept
source live, but dependencies were resolved independently in each venv, so a
new dependency landed in the project venv and never reached the running
service — leaving no clean rollout for it.
Deploy now emits ExecStart=uv run --project <source> --no-dev <program>,
running the service in place from its own project venv. uv run syncs the env
to the lockfile before launching, so a single restart picks up both code and
dependency changes; the ExecStart is deterministic from source and never goes
stale. The tool-venv install is gated to the command runner only; uv tool
install is reserved for tool-behavior programs, where PATH is the point. A
python service with no resolvable source falls back to a PATH lookup.
pnpm 10+ runs a deps-status check before `pnpm <script>` that wants to
purge and reinstall node_modules, aborting in a non-TTY build with
ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY. It is a false positive for a
build verb (which builds, not installs), so ReactViteHandler now prepends
--config.verify-deps-before-run=false to every pnpm call, building against
the existing modules. CI=true is kept as a general non-interactive guard.
Generated systemd units hardcoded a thin PATH (~/.local/bin + system dirs)
while dev-verb builds used a separate _EXTRA_PATH_DIRS list, so a service
could be built and run with different tools on PATH. Hoist a single
USER_TOOL_PATH_DIRS in config.py, consumed by both stacks._build_env
(build) and the systemd unit generator (runtime).
Also prefer pnpm's modern $PNPM_HOME/bin shim over the bare $PNPM_HOME dir
(older installs leave a stale version wrapper there) and add
/usr/local/go/bin so Go-based builds/services resolve their toolchain.
The check composite returned ActionResult with no output on success, so the
dashboard's 'Check All' (and the CLI) showed nothing when checks passed — only
a failing sub-verb ever produced output. It now accumulates each sub-verb's
output with a ✓/✗ marker (lint / type-check / test), on both success and
failure, so the output panel always shows what ran.
Also fixes lan-info's type-check: socket.getaddrinfo sockaddr is loosely typed,
so str(infos[0][4][0]) makes the address an unambiguous str.
core 94 green; ruff clean; verified POST /programs/lan-info/check returns output.
handle_path /foo/* matches /foo/ but not the bare /foo, so the no-trailing-
slash form fell through to the root catch-all (castle-app) — which then 404'd
because its router has no such route. Hitting http://gateway:9000/power-graph-app
served the dashboard's 404 instead of Power Graph; same for every proxied
service accessed without the slash.
The generator now emits 'redir /foo /foo/' for each path-prefix route (static,
proxy, remote), so the no-slash form canonicalizes to the slash form that
handle_path matches. Caddy's bare path matcher is exact, so /foo/bar is
unaffected. Host routes and the root '/' are unaffected.
Verified: /power-graph-app → 302 → Power Graph; caddy validate clean; core 94 /
api 52 green.
The gateway does two things — reverse-proxy services and serve static
frontends — but the dashboard/API route table only ever showed the proxy
routes, so 'serving a frontend' and 'proxying a service' looked like unrelated
features and castle-app/power-graph-app were invisible in the route view.
Now there's one concept: a route maps an address (path '/foo' or host 'foo.lan')
to a target of one kind — static (a built dist served by file_server), proxy
(a local service port), or remote (a service on another node).
- core: caddyfile.py gains compute_routes() — the single source of truth for
the route list; generate_caddyfile_from_registry renders it (output byte-for-
byte identical, verified against the live Caddyfile). Adds a GatewayRoute
dataclass.
- api: GatewayRoute model → {address, kind, target, name, node}; GET /gateway
builds from compute_routes (incl. config for static frontends + mesh for
remote), so the table matches what Caddy actually does.
- app: Gateway panel shows Address · Kind · Target for every route (static
frontends + host routes now appear); program detail shows 'Reachable at
/foo/ · served (static)' for static frontends.
- cli: 'castle gateway status' prints the full route table.
- docs: registry.md/design.md/CLAUDE.md describe routes as one concept with
three target kinds.
core 94 / cli 24 / api 52 green; ruff + app build clean; Caddyfile unchanged.
A service/job's env is now exactly its defaults.env — castle injects no hidden
convention vars. Values support ${port}/${data_dir}/${name} placeholders
(resolved at deploy, alongside ${secret:…}), so a program's own env var names
map to castle's computed values without hardcoding.
Why: the auto-injected <PREFIX>_PORT/<PREFIX>_DATA_DIR were a guess at the
program's env names — right for castle-scaffolded services, dead weight for
adopted ones (lakehouse carried two dead vars; notification-bridge/backup jobs
too). They also weren't visible in the config editor (computed at deploy), which
was the source of the 'four env vars but the UI shows none' mystery.
- core: resolve_env_vars gains a context (${port}/${data_dir}/${name});
deploy builds env from defaults.env only — no <PREFIX>_* injection, no
port_env. Removed the port_env field and the dead _env_prefix helper.
- cli: 'service/job create' gains repeatable --env KEY=VALUE (replaces
--port-env); 'program create' scaffolds <PREFIX>_PORT/_DATA_DIR: ${…} for new
daemons.
- app: removed the 'Port env' field; the Environment editor (defaults.env) is
the single place, with a placeholder hint.
- live migration: central-context/castle-api/power-graph/protonmail mapped their
real vars explicitly; lakehouse → just LAKEHOUSED_DAEMON_PORT: ${port}, data
stays in ~/.lakehoused. Verified all services healthy on their ports, dead
vars gone, zero failed units.
- docs: registry.md/design.md/stack guides + findings updated to the explicit
model.
core 94 / cli 24 / api 52 green; ruff + app build clean.
Names collide across resource types (a program and a service can share a name),
so the CLI no longer resolves bare names. Operations live under the resource
they act on; platform lifecycle and the cross-resource overview stay top-level.
castle program list|info|create|add|clone|delete|run|install|uninstall|
build|test|lint|format|type-check|check
castle service list|info|create|delete|deploy|enable|disable|start|stop|restart|logs
castle job <same verbs> (create takes --schedule)
castle gateway start|stop|reload|status
castle list | status | deploy [name] # cross-cutting
castle start | stop | restart # whole platform (systemd verbs)
Key changes:
- info/delete/list/run are resource-scoped (no more cross-section name match).
- delete only removes the named resource's entry; program delete still blocks
when a deployment references it.
- per-resource start/stop/restart (systemctl on the unit/timer); top-level
start/stop/restart act on the whole platform.
- expose → 'service create' (more general: --program ref, --runner); gained
'job create' (the CLI never had one).
- dropped up/down — bringing things online is the honest 'deploy && start'.
Docs (CLAUDE.md, registry.md, stack guides) and user-facing messages updated.
core 94 / cli 24 / api 52 green; ruff clean. Verified live: program/service/job
list+info scope correctly, service create+scoped-delete round-trips.
- ProgramActions: install/uninstall (activate) shown only for tools and
no-service frontends. Daemons activate via a service/job, so the program
page no longer offers a misleading Install/Uninstall (which secretly
enabled/disabled the service).
- Program page: new Deployments section listing the services/jobs that run
the program, as links (a program → 0-N services/jobs).
- Service page: 'Runs' row from the run_target summary field (the old code
dug into manifest.run, which was empty pre-Stage-1), a Host row for
proxy_host, and a convenience link to the referenced program.
- lifecycle.activate on a daemon with no service now errors toward
'castle expose' instead of silently installing its binary to PATH.
core 94 green; ruff + app build clean.
Driven by adopting lakehouse (an existing daemon that bundles its own SPA) —
see docs/findings-lakehouse.md.
#3 deploy reloads the gateway. castle deploy regenerated the Caddyfile but
left the running Caddy on the old config, so new proxy routes were silently
dead. Deploy now reloads the gateway when it's running.
#1 castle expose <program>. Turns an adopted program into a service
(run/port/health/proxy/systemd) in one command — the missing
daemon-to-service step. Flags: --port --health --path --run --port-env
--host --no-proxy.
#2 port_env mapping. A service can declare expose.http.internal.port_env so
castle sets the env var the program actually reads (e.g. lakehoused reads
LAKEHOUSED_DAEMON_PORT, not castle's convention LAKEHOUSE_PORT). Castle now
genuinely drives an adopted daemon's bind port.
#4a auto-base for react-vite. The build passes VITE_BASE = the gateway serve
prefix (/<name>/, or / for castle-app); vite.config reads it. A castle-built
frontend now works at its subpath with no hand-tuned base. castle-app's
vite.config updated as the reference.
#4b host-based routing. proxy.caddy.host routes a whole hostname to the backend
root via a host matcher inside the :9000 site, so a root-based SPA (base="/")
serves unchanged — the fix for proxying an app castle can't rebuild. Caddyfile
now emits 'auto_https off' (HTTP-only gateway on a non-standard port).
Nit: activate skips the editable reinstall when the tool is already on PATH.
Tests: core 94, cli 24, api 52; ruff + app build clean. Verified live: lakehouse
runs under systemd, API at /lakehouse, full UI (SPA boots) via host routing.
#1 Positional dest unified to `name` across info + dev verbs (was `project`).
#2 install/uninstall help now states the activate semantics (tool→PATH,
service/job→systemd, frontend→served) rather than 'to PATH'.
#4 `service status` (a plural op) moved to `services status`; the singular
`service` group is enable/disable only.
#5 `list --behavior` now filters the program *catalog* by its real behavior
field. behavior (what a program is) and service/job (how it's deployed) are
separate axes: `list` shows a Programs catalog plus Services/Jobs deployment
views; a behavior filter scopes the catalog only.
#6 `list` uses lifecycle.is_active uniformly for the status dot (tool on PATH /
service running / job timer / static frontend served) instead of registry
presence.
#7 New `castle restart <name>` — restart a service (unit) or job (timer).
#8 New `castle status` — unified services + jobs + programs activation view.
#9 New `castle format [name]` dev verb (ruff format / pnpm format); wired into
stacks DEV_ACTIONS + handlers.
#10 New `castle up` — deploy + start everything in one shot.
Docs: CLAUDE.md command reference refreshed. Tests updated to the two-axis
list model (+ a daemon-behavior program fixture). core 92 · cli 24 · api 52
green; ruff clean.
The Caddyfile generator now emits static-frontend routes that root directly at
<source>/<build.outputs[0]> (e.g. /data/repos/castle/app/dist) instead of a copy
under ~/.castle/artifacts/content. Removes _copy_app_static and the central
content-dir staging entirely; rebuilds are live without a copy step.
- caddyfile.py: manifest-driven static routes (castle-app at /, others at /<name>)
- ReactViteHandler.install just builds in place (no copy); uninstall is a no-op
- is_active(static frontend) = repo dist exists
- test isolation updated for the generator's config use
Dashboard + power-graph-app verified serving from repo dist; 168 tests pass.
Convergent deploy (prefix-based prune): castle deploy now removes orphaned
castle-* systemd units/timers no longer in the registry. Full deploy only;
partial (--name) deploys preserve siblings. Fixes the orphaned-timer / stale-
path drift class.
Container runner: derive --name from the service name (castle-<name>) instead
of the image name; skip the <PREFIX>_DATA_DIR env injection for containers
(avoids colliding with image env namespaces like NEO4J_*).
Unified active lifecycle (core/lifecycle.py): install/uninstall keep their
names but become activate/deactivate, dispatching by behavior — tool→PATH,
daemon/self-serving-frontend/job→systemd, static frontend→served. is_active
reports a uniform state (PATH-independent for tools). The CLI install/uninstall
and service enable/disable route through the extracted core.
API + dashboard: surface a uniform `active` state on programs alongside
`installed`; ProgramDetail shows active/inactive.
Tests: test_deploy_prune (6), test_lifecycle (5). 168 tests pass; lint clean.
- Extract all deploy logic from CLI into castle-core/deploy.py as a reusable deploy() function
* Resolves services/jobs to DeployedComponents
* Generates systemd units and Caddyfile
* Copies frontend assets and reloads systemd
* Returns DeployResult with deployed count and messages
- Simplify CLI deploy command to a thin wrapper calling castle_core.deploy()
- Add POST /deploy endpoint to castle-api for remote deployment management
* Supports optional {name} body param to deploy one or all services
* Returns deployment result as JSON
- Register deploy router in castle-api main.py
- Update README with new deploy endpoint documentation
This enables full remote management: build, test, lint, and deploy any castle
node over HTTP (e.g. POST http://node:9000/api/deploy)
- Added base_url field to DeployedComponent for external URL proxying
- Updated Caddyfile generator to proxy to base_url when present instead of requiring local port
- Added runner: remote handling in deploy command (returns empty run_cmd, marks as unmanaged)
- Reformatted long ternary expression in registry.py for readability
This enables castle to manage services that proxy to external endpoints without requiring a local process.
Move all instance state into ~/.castle/ with clear separation:
- code/ — user project source (was components/)
- artifacts/ — generated specs, built content (was generated/, static/)
- data/ — service runtime data (was /data/castle/)
- secrets/ — credentials
castle.yaml moves to ~/.castle/castle.yaml as the canonical location.
Source paths use repo: prefix for git repo programs (castle-api, app)
and relative paths for user projects (code/central-context).
Add idempotent install.sh for bootstrapping infrastructure (Docker,
Caddy, MQTT, Postgres, Neo4j) with interactive service detection.
Remove components/ from repo (now in ~/.castle/code/), deinit
submodules, remove .gitmodules.
- Updated the terminology from "components" to "programs" across the codebase, including in config loading, saving, and manifest specifications.
- Introduced a new `stacks.py` file to handle lifecycle actions for development stacks, implementing handlers for Python and React Vite stacks.
- Adjusted tests to reflect the new program structure and ensure proper functionality.
- Revised documentation to align with the new terminology and structure, ensuring clarity on the purpose and configuration of programs, services, and jobs.
