Update the README, AGENTS.md, install.sh, and docs/ for the converge
model: one `castle apply [name] [--plan]` replaces `deploy && start` and
the per-kind enable/disable/install/start/stop verbs. CLI references now
list apply + restart (the imperative bounce) + logs; recipes use
`castle apply`; "turn it off" is documented as `enabled: false` + apply.
install.sh's next-steps collapse to a single `castle apply`.
Verified: no retired-verb command instructions remain in any markdown;
install.sh syntax OK; live `castle apply --plan` reports converged;
doctor all-green; core 129 / cli 31 / api 59 pass.
Rework the dashboard around convergence:
- ServiceControls / ServiceCard / JobCard: replace start/restart/stop
with a Power toggle (set enabled → apply, i.e. activate/deactivate)
plus Restart (the one imperative bounce).
- ToolDetail: install/uninstall becomes an Enable/Disable toggle;
`installed` stays the live state, `enabled` the desired one.
- ConfigPanel Apply and CreateDeploymentForm now POST /apply (one
converge that renders + restarts only what changed) instead of
/deploy + a separate start/restart.
- New ConvergePanel on Overview: a terraform-style Preview (POST /apply
plan=true) showing would-activate/restart/deactivate, then Apply.
- useApply + useSetEnabled hooks; `enabled` added to Service/Job/
Deployment types; both handle the self-restart connection drop.
Backend: PUT /config/deployments/{name}/enabled sets desired on/off
(edit config; caller runs apply) — the declarative toggle the UI uses.
Dashboard builds clean (tsc + vite); 59 API tests pass; live /apply and
/services?enabled verified after an api restart.
- Replace POST /deploy with POST /apply (name + plan), returning the
enacted diff (activated/restarted/deactivated/unchanged). /config/apply
now delegates to core apply() too, so there's one convergence path.
- Retire /services/{name}/start and /stop (keep /restart as the
imperative bounce). Drop install/uninstall from program actions — tool/
static activation is convergence now.
- Surface `enabled` on ServiceSummary/JobSummary/DeploymentSummary so the
UI can show and toggle desired state.
- conftest: drop the obsolete config_editor.get_registry patch (apply_config
no longer reads the registry directly).
59 API tests pass; route table shows /apply (no /deploy), /restart only.
Reconcile the running system to declared config in a single operation,
replacing the old deploy && start plus the per-kind enable/disable/
install/uninstall verbs. The mechanism varies by manager (systemd unit /
PATH install / gateway route); the verb never does.
Core (castle_core.deploy.apply):
- render (units, Caddyfile, tunnel) then reconcile: activate every
enabled deployment that's down, restart any whose unit bytes changed,
deactivate the disabled ones. Returns ApplyResult (the enacted diff).
- --plan computes the diff and touches nothing.
- restart-on-change is exact: _render_unit_files is the single source of
truth for unit bytes, used both to write units and to predict restarts,
so the plan can't drift from what deploy writes.
Desired state:
- DeploymentBase.enabled (default True) — the declarative on/off. apply
converges to it; a disabled deployment is defined but not running and
gets no gateway route (else it would 502). Registry omits enabled when
True, keeping existing registries byte-identical.
CLI:
- add `castle apply [name] [--plan]`; keep `restart` as the one
imperative bounce, plus status/doctor/logs/gateway/program.
- retire deploy, start, stop (top-level); service/job deploy|enable|
disable|start|stop; tool install|uninstall; program install|uninstall.
- next-step strings + doctor hints now say `castle apply`.
Verified: core 129 + cli 31 green; live `castle apply` on a converged
node reports "nothing to do" with no spurious restarts; --plan shows an
empty diff; gateway + api stay up.
castle deploy now manages the public zone's tunnel CNAMEs to match the current
set of public: true services — creating missing records and deleting removed
ones — instead of only printing manual 'cloudflared tunnel route dns' hints.
New core/generators/dns.py reconciles via the Cloudflare API (stdlib only). It
only ever touches records whose content is <tunnel_id>.cfargotunnel.com, so
hand-managed records in the same zone are never altered. deploy._write_tunnel_config
calls it in both branches (delete of the last public service cleans up its CNAME).
Without a token it's a no-op and the manual route-once hints are surfaced as before.
Token: ~/.castle/secrets/CLOUDFLARE_PUBLIC_DNS_TOKEN, a single DNS:Edit permission
scoped to the public zone (the 'Edit zone DNS' template) — confirmed sufficient:
that one permission resolves the zone by name and edits records, no separate
Zone:Read needed. Separate from CLOUDFLARE_API_TOKEN (ACME, internal zone).
castle doctor gains a read-only probe that WARNs when the token is absent (CNAMEs
stay manual) and FAILs when it can't reach the zone/records.
The README/docs explained how the system works but not how to get it up
and running, and there was no way to tell a healthy node from a
half-configured one. Two additions close that gap.
castle doctor — a read-only preflight/postflight check. It inspects
setup *and* runtime (CLI on PATH, uv, lingering; repo:, control plane
registered, dashboard built; gateway/api running + listening, specs
generated; and — under tls=acme — DNS-plugin Caddy, provider token,
:443 bind; tunnel config for public services) and, for anything not
green, prints the exact next command. Exit 0 when nothing failed
(warnings allowed), 1 otherwise, so it doubles as a scriptable smoke
test after install/deploy. Agents can lean on it the same way they use
`castle tool list`.
Docs — the quick start now leads with prerequisites and one command
(install.sh installs the CLI + registers the control plane, so the old
manual `uv tool install` step is gone), adds a `castle doctor` verify
step, and gains an "exposure ladder" table framing the three rungs
(localhost → LAN HTTPS → public) that link the deep DNS/TLS/tunnel docs.
install.sh's closing summary and the AGENTS.md CLI reference mention
doctor too.
Rework the supabase stack so every app owns an isolated Postgres schema
(= program name) instead of sharing `public` with a table_prefix. This
gives a clean, knowable teardown and drops migration version tracking to
per-app, so version tokens can no longer collide across apps.
- stacks: add StackHandler.owns_data + teardown() (default no-op). Supabase
build now creates+grants the app schema, tracks migrations in
<schema>.schema_migrations, and applies each with search_path set to the
schema (write unqualified names). teardown = drop schema <app> cascade.
Fix _substrate_db_url default port 5432 -> 5433 (substrate's direct PG).
- deploy: derive ${supabase_app_schemas} from registered apps; substrate maps
PGRST_DB_SCHEMAS: public,storage,graphql_public${supabase_app_schemas} so
PostgREST exposes each app schema (restart substrate after add/remove).
- delete: generic owns_data -> teardown path behind new --purge-data flag,
replacing the hardcoded supabase remnant note.
- scaffold + docs: new apps born schema-isolated (unqualified tables,
db:{schema}, schema: in supabase.app.yaml).
- Mount xterm as an absolute inset-0 fill so it always has an explicit box and
fit() measures the real size when the dock expands (was collapsing via the
flex/percentage height chain).
- Acquire the pty slave as the child's controlling terminal (setsid + TIOCSCTTY)
so the kernel delivers SIGWINCH on a browser resize. Previously the initial
size was read once but live resizes were dropped, so the pane stayed at its
startup size — content never grew with the window.
Server-side counterpart to the global commit-msg hook (which blocks the same word
locally, but can be bypassed with --no-verify or via GitHub UI edits). Checks only
message text, never file contents.
- Run agent sessions under an isolated tmux server (its own systemd scope, with a
keepalive session) so they survive a castle-api restart and stay rediscoverable;
each WebSocket is a tmux attach client. Falls back to the in-memory pty backend
when tmux is absent or CASTLE_API_AGENT_BACKEND=memory.
- Refactor the session layer behind one Attachment interface (create/list/attach/
close), keeping resume-by-id, agent-native history, and continue modes.
- Fix the terminal not refilling when the dock is maximized (explicit refit signal
+ instant size transition so xterm measures the final box).
- On narrow screens, go full-bleed and strip chrome (header/border/padding) when
maximized.
_TUNNEL_NAME was "tunnel", but the deployment is named castle-tunnel, so
unit_name() produced castle-tunnel.service — a unit that doesn't exist (the
real one is castle-castle-tunnel.service). The is-active check never matched,
so deploy printed "Tunnel not running" and skipped the ingress reload. New
public: true services then 404'd until a manual tunnel restart.
Set _TUNNEL_NAME = "castle-tunnel" to match the deployment name, mirroring
_GATEWAY_NAME. Deploy now restarts cloudflared on ingress changes.
A fresh clone + ./install.sh set up the infra (Docker, Caddy, MQTT,
Postgres) but left the registry empty — so `castle deploy && castle
start` brought up nothing. install.sh now gets a person all the way to
a running Castle:
- ensure_uv + install_cli: install uv and the `castle` CLI (editable
from ./cli) so the command exists on a fresh machine.
- seed_control_plane: register castle-gateway, castle-api, and the
castle dashboard from a new bootstrap/ seed dir, and add `repo:` to
castle.yaml so `source: repo:<name>` resolves. Never clobbers
existing entries.
- build_dashboard: build app/dist/ so the gateway has a UI to serve
(best-effort; warns if pnpm is absent).
The gateway seed uses a `__SPECS_DIR__` placeholder (no machine-specific
paths in the repo), substituted with this machine's specs dir at seed
time. .gitignore negates bootstrap/**/castle.yaml so the seeds are
tracked despite the user-registry ignore rule.
Verified: seeded config loads, derives kinds service/service/static,
and renders an off-mode Caddyfile serving the dashboard at :9000/ with
/api/* proxied to castle-api.
Neo4j was auto-provisioned for everyone; a graph DB isn't core, so don't foist it
on new users. Gate it behind maybe_setup_neo4j: enable with --with-neo4j, an
interactive yes, or a previous run's choice (recorded in infra.conf, so a decline
isn't re-prompted). MQTT and Postgres stay default-on. Existing installs with
NEO4J=container keep running it unchanged.
install.sh provisioned mqtt/postgres/neo4j data + their docker -v mounts under
~/.castle/data, but castle_core.config and every container deployment use
/data/castle — so a fresh install put the data in the wrong place. Resolve DATA_DIR
the same way config.py does (${CASTLE_DATA_DIR:-/data/castle}); since that's outside
$HOME, create it with a sudo+chown fallback when the parent isn't user-writable.
Also add REPOS_DIR and fix the summary (Code→Repos, 'castle services start'→'castle
start').
Moves the ~/.castle/{programs,deployments} dir creation and the minimal
castle.yaml (gateway.port: 9000) into install.sh's create_directories — seeded
only when absent, never clobbering an existing config. Drops the stale 'code/'
dir (program source lives in /data/repos now). README quick-start no longer
does the manual mkdir/printf; install.sh handles it.
Rebuild the README on the 'personal software estate' framing and bring it current:
programs vs deployments (manager → derived kind), the service/job/tool lenses,
'castle tool list --json' for assistants, and the new agent-launching mechanism
(castle.yaml agents: + the dashboard terminal dock). Fix stale claims — subdomain
routing (no path prefixes), defaults.env is explicit (no auto-injected convention
vars), /data/repos + /data/castle paths, castle deploy && castle start. Trim the
sprawling/stale API table to a pointer; link docs + AGENTS.md + CLAUDE.md.
The field showing a deployment's launcher + target is now labelled 'Launch'
(consistent with launcher/manager vocabulary) on ServiceDetail and the Service/
Job/Create field editors.
Tools and statics only appeared as labels in the Programs catalog; now they get
their own sections alongside Services and Jobs (deployment views over
config.deployments filtered by kind), in both the human and --json output.
The tools lens coding assistants need. 'castle tool list [--json]' lists tools
(programs with a manager:path deployment) surfacing each tool's actual EXECUTABLE
— resolved from pyproject [project.scripts], which can differ from the program
name (litellm-intent-router → intent-router) — plus description and install state.
'castle tool info <name> [--json]' details one; 'castle tool install|uninstall'.
--json is a clean machine-readable catalog for building context.
Sweep: main.py descriptions/help mention the tool lens; 'frontend'→'static' in
install help; add.py._detect drops the dead behavior return (add adopts source
only); stale comments/help fixed. Docs (CLAUDE.md/README/registry) document
'castle tool'. Tests: cli/tests/test_tool.py (list/info/json).
Suites: core 124, cli 29, castle-api 58.
A program's package name can differ from the console script it exposes
(litellm-intent-router installs the 'intent-router' executable), so the
which(<program>) install-check missed it: the tool installed fine but castle kept
reporting 'not installed', making the Install button look dead. _on_path now also
consults 'uv tool list' by package name (2s-cached), and the API's install-checks
use the shared tool_installed() helper. Verified litellm-intent-router now reads
installed/active True.
ProgramList gains a filterable prop; the Programs page enables it. Chips filter by
whether a program has a deployment of that kind (membership over program.deployments),
so a tool-and-job program shows under both Tool and Job. Counts are programs-per-kind.
Tools page stays unfiltered (already scoped).
A static showed both 'Served at' (from the static block) and 'Subdomain' (the
general proxied-service field) — same URL twice. Gate 'Subdomain' to non-static;
a static's gateway URL is already 'Served at'.
- Tool detail showed installed tools as 'Not installed': /deployments/{name}
populated 'installed' for path deployments but left 'active' null, and
ToolDetail keyed off 'active'. Now the endpoint sets active=installed for path,
and ToolDetail reads 'installed' directly (verified: on-PATH ⇒ True, else False).
- A deployment now inherits its program's description by default: copied at
creation (CLI create + API _save_deployment on new deployments) + a one-time
patchup of existing ones (29 deployments).
- Tools page cards no longer list deployments (ProgramCard showDeployments=false) —
the lens already scopes to the tool; the deployment list is a Programs-catalog thing.
ProgramCard/ProgramList take an optional linkBase (default /programs); the Tools
page passes /tools so a tool tile opens /tools/<name> (its deployment detail),
not /programs/<name>.
- Nav: Services/Scheduled/Tools nested under a 'Deployments' group (expandable;
flattened to icons when the sidebar is collapsed). Programs stays top-level.
- Overview: Tools tile (Wrench, count of tool programs).
- New /tools/:name tool detail page (useDeployment): Install/Uninstall lifecycle,
a link up to its program, and an editable ToolFields (description + env).
- ServiceDetail is static-aware: a caddy deployment shows served-by-gateway +
URL + root, no start/stop, and edits via a new StaticFields (root + public + env)
so a launcher is never injected into a caddy deployment.
- Program detail's Deployment section is now a clean list of links — every
deployment (tool→/tools, service/static→/services, job→/jobs) links to its
detail page; lifecycle controls moved onto those pages.
clean pnpm build + tsc.
- Programs page: each card lists the program's deployments (name · kind) instead
of a single kind badge; kind-filter chips removed (nav is the kind lens now).
protonmail honestly shows tool + job.
- New Tools nav page (Wrench) — programs with a tool deployment (/programs?kind=tool).
- Services page now shows statics too: /services returns service + static, and a
static renders as a caddy-served card (KindBadge, served URL, no start/stop).
- Program detail: DeploymentsSection + deleteConfirm rebuilt over program.deployments;
header drops the (now nonexistent) program kind.
- ServiceFields / JobFields: launcher is now an editable select (python|command|
container|compose|node), persisted into run.launcher via the normal save.
clean pnpm build + tsc --noEmit.
A program has no single kind — it HAS deployments, each with its own kind (a
program can be a tool AND a job, e.g. protonmail). Remove program-level kind:
- core: drop ProgramSpec.kind; CastleConfig.kind_of → deployments_of(name) →
[(deployment-name, kind)]; tools property derives from a tool deployment.
- api: ProgramSummary drops kind/services/jobs → deployments: [{name, kind}];
/programs?kind= filters by deployment-kind membership; a program's legacy
DeploymentSummary carries kind=None.
- cli: list/info show a program's set of deployment kinds; --kind filters by
membership.
Also: Services page now covers statics — /services returns kind in
{service, static} (both are exposed, URL-reachable 'services', caddy vs systemd),
and ServiceSummary gains kind + manager to distinguish them.
Suites: core 124, cli 25, castle-api 58.
- /services was returning every non-job deployment (tools, statics, remotes);
filter it to kind==service so the Services page shows only services.
- Programs page gains kind-filter chips (All / Service / Job / Tool / Static /
Reference) with per-kind counts, colored to match KindBadge.
- Replace window.confirm with a reusable ConfirmModal (styled, danger variant,
Esc/Enter, preserves bulleted bodies); FormFooter's delete uses it, so program/
service/job deletion now confirms in-app with the enumerated cascade list.
castle-api 58 passed; /services live drops from 37 to 15 (services only).
The confirm dialog (window.confirm in FormFooter) now spells out exactly what
cascade will do — uninstall the tool from PATH / drop the static route / stop,
disable and remove named service+job deployments, and remove the catalog entry —
each as its own bullet, ending with the reassurance that source on disk is kept.
A program and its 1:1 tool/static deployment are one thing to the user, so
'Delete program' now just works instead of refusing. DELETE /config/programs/
{name}?cascade=true tears down each referencing deployment (manager-aware:
uninstall a tool from PATH, stop+disable a service/job, drop a static route),
removes them and the program, then converges the runtime (prune orphan units,
regenerate the Caddyfile, reload the gateway). Without cascade it still 409s
(safe API default). The dashboard drops the 'can't remove while deployed' block,
labels the action 'Delete program', and the confirm names exactly what will go.
Verified live: throwaway tool → cascade delete removes program + path deployment,
real config untouched; plain delete of a referenced program still 409s.
castle-api 57 passed.
Install/Uninstall no longer live in the program's action cluster (they were a
tool's PATH-deployment lifecycle masquerading as a program action). ProgramActions
is now dev-verbs-only (build/test/lint/type-check/check). The program detail's
Deployment section owns lifecycle, keyed on kind: a tool shows Installed-on-PATH
with an Install/Uninstall toggle, a static shows served-by-gateway + its URL,
services/jobs link to their own pages. Program list cards drop the quick toggle
for a plain active dot. 'Remove program' stays as catalog deletion — no longer a
dead-end now that the blocking deployment is visible in the Deployment section.
The API's ProgramSummary no longer emits runner (a program has no launcher —
that lives on its deployment). Remove the stale UI type field and the
never-rendered Launcher row on the program detail page; a program's launcher
already shows on its deployment in the Deployments section.
Replace the conflated `runner` axis with two orthogonal ones: `manager`
(systemd|caddy|path|none) — who supervises/realizes a deployment — and, for
systemd only, a nested `launcher` (python|command|container|compose|node) — how
the process starts. ServiceSpec and JobSpec collapse into one manager-
discriminated DeploymentSpec union (Systemd/Caddy/Path/Remote); the services/
and jobs/ config dirs collapse into one deployments/ dir. The human "kind"
(service|job|tool|static|reference) is fully derived (kind_for), never stored —
the frontend kind is renamed static. behavior is gone.
- core: DeploymentSpec union + LaunchSpec + kind_for; legacy-aware loader
normalizes old runner shapes; CastleConfig.deployments with derived
services/jobs/tools views; registry.Deployment carries manager/launcher/kind.
- cli: service/job/tool as filtered views + a deployment group; --behavior→--kind,
create --runner→--launcher; lifecycle dispatches over config.deployments.
- castle-api: /deployments primary with /services,/jobs as views; summaries
derive kind; PUT/DELETE /config/deployments/{name} (services/jobs aliased).
- app: KindBadge, frontend→static everywhere, pick-a-kind creation wizard,
per-kind config editors.
- docs: single deployments/ layout, manager/launcher, static kind throughout.
Live migration verified byte-identical: regenerated Caddyfile and every unit
ExecStart line unchanged, so nothing restarted. Suites: core 124, cli 25,
castle-api 55; dashboard build + type-check clean.
behavior (tool/daemon/frontend) is now a derived display label computed
from a program's deployments — path->tool, static->frontend, process->daemon
(behavior_for_runner). load_config populates it so every reader gets the live
value; _spec_to_yaml_dict excludes it so it is never written to disk.
_summary_from_service derives it from the runner instead of hardcoding daemon.
Fixtures gain path/python deployments so tool/daemon behaviors derive.
- Gateway route table: a public route's address now links to its public URL
(green, with the tunnel icon) instead of the internal one.
- Program detail: remove the editable `behavior` selector — behavior is a derived
label now and drives no logic, so editing it was meaningless (the stored value
is preserved on save via the manifest spread).
Surfaces the DNS/Cloudflare/public state that previously required editing
castle.yaml + curl:
- API: GET /gateway now returns domain, public_domain, tunnel_id,
tunnel_connected, and a public_url per public route; new PUT /gateway/config
edits tls/domain/public_domain/tunnel_id in castle.yaml (deploy to apply).
save_config now persists public_domain/tunnel_id (previously dropped).
- UI: a GatewaySettings row in the Gateway panel shows TLS / domain / public
(tunnel up/down + public_domain) with inline edit; each public route gets a
green "public" badge linking to its <name>.<public_domain> URL.
- A static service is inherently exposed, so `public: true` no longer trips the
"requires proxy" validator — frontends can be public via the tunnel for free
(the original friction that started this refactor, now resolved).
- docs/registry.md: the manager model (systemd/caddy/path/none), the runner→manager
table incl. static/path, and behavior as a derived display label.
- manifest: RunPath (runner: path) — a CLI on PATH via `uv tool install`, manager
= path, no unit/route (like remote/static).
- lifecycle/service: a path deployment's start/stop = install/uninstall (routed
through activate/deactivate, which skip the reinstall when already on PATH).
`castle service start/stop <tool>` and status now dispatch by manager; status
shows the manager tag [systemd|caddy|path|none].
- config.tools() now derives from `runner: path` services, not the behavior label.
- castle-api: the "installed" flag derives from manager==path, not behavior==tool.
- create: a python-cli stack emits a `runner: path` service.
`behavior` no longer drives any runtime logic — it survives only as a display
label + `list --behavior` filter. Verified live: html2text uninstall→reinstall
round-trip via `castle service stop/start`.
- manifest: `manager_for(runner)` — the single source of truth mapping a runner to
its manager (systemd | caddy | path | none). deploy's `managed` now derives from it.
- lifecycle: is_active/activate/deactivate dispatch by manager instead of behavior
(removed the stale `_is_static_frontend`, which broke once frontends became
services). caddy → gateway reload; path → install/uninstall; none → no-op.
- cli/service: start/stop/restart and enable/disable route through the manager
(systemctl only for systemd; static reloads the gateway; remote is a no-op);
platform restart skips non-systemd runners.
- create: a frontend stack (react-vite/supabase) now emits a `runner: static`
service, not a bare frontend program.
Two features built this session, committed together as a checkpoint before the
deployment-model refactor (they share files, so kept as one rollback point):
- Public exposure via Cloudflare tunnel: a `public` toggle on services (requires
proxy), gateway.public_domain/tunnel_id config, a cloudflared ingress generator
(generators/tunnel.py) that maps <name>.<public_domain> to the gateway's
internal host, deploy wiring, and docs/tunnel-setup.md.
- Frontends as `static`-runner services: a RunStatic runner served by Caddy
(file_server), no systemd unit (like `remote`); the Caddyfile generator now
derives static routes from services instead of a behavior==frontend branch.
The auto-scroll effect used bottomRef.scrollIntoView(), which walks the
entire scroll-ancestor chain up to the window and scrolls the page to the
log's bottom on every new line. Scroll the <pre> container directly via
scrollTop instead, so only the log pane moves. Drop the now-unused
bottom marker div and ref.
Use the stretched-link pattern: the card is relative and the title link
gets an after:absolute after:inset-0 overlay so clicking anywhere on the
card navigates to the detail page. Interactive controls (action buttons,
subdomain/Docs links, ProgramActions) are raised with relative z-10 so they
stay independently clickable. Cards also get a hover border for affordance.
Reorder the sidebar so Gateway sits right under Overview. Replace the
Scheduled and Program tables with card grids matching Services: add JobCard
and ProgramCard, convert ScheduledSection to a JobCard grid, and turn the
former ProgramTable into a filterable ProgramCard grid (renamed ProgramList;
kept the search box, dropped the sortable table). Remove the now-orphaned
SortHeader component.
Introduce a Layout shell with a collapsible left sidebar (state persisted
to localStorage) that becomes a hamburger drawer under md. Break the single
Dashboard into routed pages — Overview (summary tiles + node bar), Services,
Scheduled, Programs, Gateway, Mesh — nested under the shared Layout via an
Outlet. Centralize useEventStream in Layout (drop the per-page duplicates),
repoint detail-page back links at their section pages, and remove the now-dead
SectionHeader component / SECTION_HEADERS map (each page owns its header).
