Commit Graph

177 Commits

Author SHA1 Message Date
Paul Payne
46778b7f2e feat(app): system map — consumption model, inspect mode, externals, mesh
Reframes the map around consumes / consumed_by and adds:
- Inspect on select: dim the rest, light a node's consumes/consumed-by, and a
  side panel listing both (removable/navigable) with protocol chips.
- Protocol-typed dependency edges (http/pg/bolt/mqtt) + reach badges; TCP infra
  now shows its port.
- External resources: a registry lane of reference nodes with an "add external"
  form; consumption of one renders as a chip on the consumer (no cross-map line).
- Suggested (undeclared) consumption as dashed amber edges — click to accept.
- Capability/candidates in the panel (alternative providers of the same protocol).
- A "Machines" band rendering other mesh nodes' deployments (read-only).

Adds useSuggestions / useMeshDeployments / useSaveReference hooks and the
GraphEndpoint / GraphSuggestion / MeshDeployment types.
2026-07-06 16:09:12 -07:00
Paul Payne
800f539ef6 feat(core+api): sockets, consumption audit, capabilities, mesh endpoints
Graph model gains, all derived (GET /graph auto-exposes via asdict):
- endpoints[{protocol,port}] + reach per node — reusing http_exposed/tcp_port,
  so raw-TCP infra (postgres/neo4j/mqtt) is finally visible and edges can be
  protocol-typed by the target's socket.
- base_url for reference (external) nodes.
- provides/consumes capability types, activating the dormant ProgramSpec fields.

New surfaces:
- core/audit.py + GET /graph/suggestions: SUGGESTS undeclared `requires` by
  matching a deployment's env endpoint values against provider sockets. Never
  writes; the graph stays declaration-derived (docs/relationships.md).
- kind-scoped PUT/DELETE /config/references/{name} for external resources.
- GET /mesh/deployments (flattened remote deployments + derived endpoints), and
  the mesh MQTT payload now carries tcp_port + base_url (forward-compatible,
  still no secrets) so peers can resolve cross-node endpoints.
2026-07-06 16:09:12 -07:00
Paul Payne
8086a09bf7 feat(app): interactive system map (/map)
A spatial control surface for the whole node. Tools, jobs, services, and
frontends sit in lanes with program-source grips; edges show requires
(dependency), same-program siblings, and exposure to LAN/Internet targets.

Direct manipulation, each backed by the existing config PUT + apply:
- drag a node to LAN/Internet to set reach (internal/public)
- delete an exposure line or a node (with confirm) to unexpose/remove
- right-click or the per-node ⋯ menu: open, restart, expose, delete
- click the source grip to open the backing program

Nodes are draggable with Shift-/toggle box-select; layout persists to
localStorage with a reset control. Adds useSetReach/useDeleteDeployment
hooks, the @xyflow/react dep, and the nav entry + route.
2026-07-06 13:12:15 -07:00
Paul Payne
a018a21587 fix(core): reload gateway on apply under acme+cloudflare
_reload_gateway validated the generated Caddyfile in castle-api's own
process env, which lacks CLOUDFLARE_API_TOKEN. Under acme the Caddyfile
references {env.CLOUDFLARE_API_TOKEN}, so `caddy validate` failed on an
empty token and the reload was silently skipped — every API-driven apply
left the running Caddy stale (route changes never took effect until a
manual reload). Validate with the gateway service's own resolved env so
the DNS-provider token is present.
2026-07-06 13:12:15 -07:00
Paul Payne
e438f45b54 feat: deployment-scoped requires + UI editors, gateway nav, services filter
Model:
- Drop `requires` from ProgramSpec; requires is now deployment-only.
  Collapse Requirement to {ref, bind} with kind defaulting to "deployment"
  (drop the unused `version`). System-package preconditions stay on the
  program as `system_dependencies`, synthesized into {kind: system} for the
  functional check. relations.py/deploy.py read only the deployment's requires.
- create.py seeds a stack's substrate dependency onto the deployment, not the
  program. Docs + tests updated.

UI:
- Add a `requires` editor (useRequires) on service, job, and static detail
  pages — edit service→service deps (ref + optional bind env var).
- Restore the gateway route table "Open" column: a kind-aware in-app link to
  each route's deployment.
- Reformat the reach control: name baked into each option
  (localhost:PORT (local) / <name>.<domain> (internal) / <name>.<pubdomain> (public)).
- Services page: sort statics in with systemd services by name, add a
  search + kind-filter bar mirroring the programs page.
2026-07-06 04:00:24 -07:00
Paul Payne
20bf78caf1 api+ui: kind-scoped save/delete endpoints and links
API: /config/{services,jobs,tools,static}/{name} now pin the twin they
target — _save_deployment/_delete_deployment take an explicit kind so a
patch to a 'backup' service can't bleed into a 'backup' job/tool. Add
/tools and /static endpoints; keep /deployments/{name} kind-agnostic.
New test_kind_twins proves per-kind save/delete isolation on disk.

UI: ConfigPanel and CreateDeploymentForm write to the kind-scoped
resource; GatewayPanel/NodeDetail/DeploymentsSection link via a shared
detailPath(name, kind) helper instead of the ambiguous /deployment/:name.

Includes incidental ruff-format reflow of untouched api files.
2026-07-06 02:51:08 -07:00
Paul Payne
0cb41851cf cli: resolve deployments by (name, kind), not bare name
Thread kind through every CLI resolver so a tool, service, and job can
share a name. Reads go through config.deployment(kind,name) /
deployments_named(name) / all_deployments(); writes target the per-kind
store (config.services/jobs/tools/statics). delete cascades over
(kind,name) tuples; run/info resolve via registry.named().
2026-07-06 02:44:25 -07:00
Paul Payne
315afe3f5f feat(api): kind-scoped deployment resolution
Resolve deployments by (name, kind) via the core per-kind stores / registry.get:
- get_service resolves a service-or-static; get_job a job; get_component returns
  the first kind for a name (the /services|/jobs|/tools/{name} endpoints are the
  unambiguous addresses). _save_deployment stores into the kind store, preferring
  the existing same-named deployment on a partial patch (can't derive kind from a
  partial payload). delete/set-enabled act across a name's kinds.
- All iterate-all loops use registry.all(); lookups use registry.get/named.
- services.py/logs.py resolve the managed deployment and use kind-aware unit names.
- mqtt registry (de)serialization keyed by composite "<kind>/<name>".
API + core suites green (85 + 182).
2026-07-06 02:35:20 -07:00
Paul Payne
c5cf3a1561 feat(core): kind-scoped deployment identity (name, kind)
A deployment's identity becomes (name, kind), so a tool, service, and job can
share a name (a `backup` trio). Kind is structural — which per-kind store / dir —
not a derived discriminator threaded around.

- CastleConfig holds per-kind stores (services/jobs/tools/statics/references);
  all_deployments()/deployments_named()/deployment(kind,name) replace the single
  flat `deployments` dict. A construction-only `deployments` InitVar routes a flat
  map into the stores by kind_for.
- Storage: deployments/<store>/<name>.yaml (per-kind dirs), with read-compat for
  the old flat + services/+jobs/ layouts.
- Registry keyed by composite "<kind>/<name>" + name field; get()/all()/named()/put().
- Job units carry a -job marker (castle-<name>-job.{service,timer}); services and
  everything else unchanged — so a service and a job can coexist.
- deploy/apply/lifecycle thread kind through unit naming, activate/deactivate,
  is_active; caddyfile/tunnel/relations iterate via all_deployments/registry.all.
- Subdomain uniqueness validated across kinds (a service and a static can't share one).

182 core tests pass, incl. a new same-name-trio suite (round-trip, distinct units,
subdomain guard). API/CLI/UI updated in follow-up phases.
2026-07-06 02:25:05 -07:00
Paul Payne
b4b3db5327 fix(api): serve editable spec on /services/{name} + public_url for statics
- /services/{name} served the runtime view for a STATIC (caddy) deployment
  (it's not in config.services), so its manifest lacked reach/root/program and
  the dashboard's reach dropdown defaulted to "internal". Now serves the editable
  spec whenever the deployment is in config.deployments — the twin of the earlier
  /deployments/{name} fix.
- gateway route public_url scanned only config.services, so public statics got
  public_url=None. Now scans all deployments.

Coverage: parametrized invariant sweep (every kind × every detail endpoint serves
the editable spec) + a /deployments-vs-/services consistency test that fails the
instant the two paths diverge; public/internal/static public_url derivation and
raw-TCP route exclusion.
2026-07-06 00:04:24 -07:00
Paul Payne
07c02aa151 fix(api): editable-spec detail + PATCH-merge saves + shared gateway parser
Deployment/program edits could silently drop spec fields:

- GET /deployments/{name} served the runtime view (no reach/program/root/expose)
  for deployed deployments, so the dashboard round-tripped a lossy manifest and
  stripped fields on save (broke astro). Now serves the editable castle.yaml spec
  when the deployment is in config.
- _save_deployment and save_program replaced the whole object with client input.
  Now shallow-merge over the existing spec (omit = preserve, null = clear), so a
  partial/lossy save can't drop untouched fields.
- save_yaml rebuilt GatewayConfig from `port` only, wiping tls/domain/tunnel/
  cert_hook on a whole-file save. Now uses a shared parse_gateway() (also used by
  load_config) so gateway fields can't drift between the two.

Dashboard forms (ServiceFields/StaticFields) send explicit null to clear, per the
merge contract; adds exposure host-label helpers. Coverage: detail-serves-spec,
save round-trip, partial-patch-preserves, null-clears, program partial-patch,
parse_gateway, and config save/load round-trip.
2026-07-05 23:23:59 -07:00
payneio
eeaa65f7ce feat: resolve a program's pinned node version for build + runtime
Frontend builds triggered from the castle web app run inside the castle-api
systemd service, whose PATH omits nvm's versioned node dir — so `pnpm build`
died with `node: not found` even though it worked from an interactive shell.

Introduce a per-program node convention: a program declares its version the
ecosystem-standard way (.node-version / .nvmrc / package.json engines.node),
and castle_core.toolchains.resolve_node_bin() maps it to a concrete nvm bin dir
(CASTLE_NODE_VERSIONS_DIR, default ~/.nvm/versions/node; newest match wins).
The same resolver feeds both sites that run a program's node:

- build time: stacks._build_env() prepends the pinned node for the dev-verb
  subprocess (keyed on the source dir), so `castle program build` uses the
  program's node regardless of caller.
- run time: deploy._build_deployed() stores it in Deployment.path_prepend,
  which the systemd generator puts ahead of the default unit PATH — so a
  `launcher: node` service runs its program's node.

A pinned-but-uninstalled version fails loud with an `nvm install` hint instead
of a cryptic `node: not found`. Unpinned → no injection (no guessing).

Also: the systemd generator now honors an explicit PATH in defaults.env as a
full override instead of clobbering it with a trailing Environment=PATH line
(systemd's last-assignment-wins rule had silently defeated the documented
escape hatch).

Pins app/.node-version and documents the convention in the react-vite stack.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-05 19:17:23 -07:00
Paul Payne
f641e7f8d3 style: fix lint errors (line length, import sort) 2026-07-05 15:52:41 -07:00
Paul Payne
d14b9eafa7 feat: supabase stack seeds a requires on the substrate at create
A stack's substrate dependency now becomes a real encoded `requires` at
`castle program create` time (STACK_REQUIRES), so the relationship graph shows it.
This keeps `stack` uncoupled from the runtime model — the stack declares the edge
once at creation; the graph only ever reads the encoded `requires`, never the stack.
2026-07-05 15:24:18 -07:00
Paul Payne
b9d38be707 fix: system requirement check verifies the package, not a PATH command
`system_dependencies` holds apt PACKAGE names; checking `which <name>` only
coincides with "installed" when the name equals its command (pandoc, rsync). Names
like poppler-utils / texlive-latex-base / docker-compose-plugin have no binary of
that name, so the functional? predicate wrongly reported them unmet. Fall back to
`dpkg -s` (the real "installed" check) after the PATH fast-path.
2026-07-05 15:17:03 -07:00
Paul Payne
add356dcf2 feat: relationship model (requires/repos/predicates) + git sync
A derived, mostly-computed model of how programs, deployments, and repos relate,
plus the git-sync surfaces that motivated it. See docs/relationships.md.

Core:
- `requires: [{kind, ref, version?, bind?}]` on programs + deployments — one
  precondition relation; `system_dependencies` is its `{kind: system}` alias.
  kind fixes meaning + check (system=installed, deployment=exists).
- relations.py: derives repos (git toplevel / monorepo), fan-in, and the
  predicates functional?/fresh?/deployed? — nothing stored.
- env is generated FROM a `{kind: deployment, bind}` requirement (target URL →
  consumer env), never scraped back into one; explicit defaults.env still wins.
- git.py: working-copy status/pull, repo toplevel + remote url.

Surfaces:
- `castle graph` + GET /graph — the relationship diagnostic.
- GET /repos, /repos/{key}/git|sync — repo-scoped sync (a repo is the sync unit;
  a monorepo backs several programs). GET /programs/{name}/git|sync + repo context.
- Dashboard: Graph screen, program-page git status + repo-aware Sync, and a
  monorepo banner on Programs.

Governing principle: predicates are derived; encode only the non-derivable, as a
node or edge property. Pull-only sync — converge stays a separate step.
2026-07-05 10:55:42 -07:00
Paul Payne
3c566540aa feat: raw-TCP exposure with castle-managed TLS + reach model
Expose raw-TCP services (postgres) by name at <name>.<domain>:<port> and
cut the gateway's ACME wildcard cert onto them so they present a trusted
cert. Replaces the proxy/public booleans with a single `reach` enum
(off|internal|public); legacy input still parses via derived accessors.

Core:
- expose.tcp{port,tls} + TlsSpec(material: pair|combined|off, reload)
- tls.py: materialize cert files from Caddy's wildcard, reconcile on
  renewal; `castle tls` CLI; optional cert_obtained events-exec hook
  (gateway.cert_hook, gated so a plugin-less Caddy still parses)
- apply waits (bounded) for the wildcard to issue before materializing so
  a fresh-node TLS service starts with its cert in place, then scopes
  materialization to the deployments being applied
- reach: internal|public requires an expose block (no silent no-op);
  public raw-TCP guarded until tunnel support lands
- chain.pem (${tls_ca}) is the issuer chain (leaf stripped), a real CA
  bundle distinct from cert.pem
- one shared ${...} resolver (resolve_placeholders) for env and container
  run fields; run.env now expands like volumes/args; $$ escapes a literal
- validate the generated Caddyfile before reloading the gateway so an
  invalid config never degrades routing

Docs: docs/tcp-exposure.md. Tests cover reach/expose validation,
placeholder expansion + escape, issuer-chain material, TLS materialize.
2026-07-04 23:04:26 -07:00
Paul Payne
0e8bf2571f docs: fix stale TLS/routing references (audit sweep)
Cross-checked every doc against the code and fixed the stragglers the
apply/gateway/subdomain refactors left behind:

- Retired `tls: internal` mode: dropped the misleading reference in
  registry.md and the stale comment in config.py GatewayConfig (only
  off|acme exist; the dns-and-tls.md "was removed" note is kept as
  accurate history).
- Subdomain-only routing: fixed "path-based routing" claims in design.md
  (proxy list + mesh), "path prefix or host" → host, and the "path-prefix
  routes stay on :port" line in registry.md.
- Frontends serve at their subdomain root (VITE_BASE=/), not /<name>/:
  fixed supabase.md (x2), the scaffold.py docstring, the stacks.py build
  comment, and design.md's "serve prefix baked in" line.

README, AGENTS, developing-castle, dns-and-tls audited clean. Verified
against code; core 129 / cli 31 pass; ruff clean.
2026-07-02 13:56:41 -07:00
Paul Payne
43a6ad76bb docs: simplify lifecycle sections for the converge model
The apply/converge + gateway refactors let the docs collapse a lot:

- design.md §Component Lifecycle: the build→install→deploy three-step
  ("enables and starts services") becomes two moves — develop, then
  `castle apply` (activation polymorphic over the manager). Filesystem
  layout de-duped and corrected (program source is /data/repos/<name>,
  not the long-removed ~/.castle/code).
- registry.md §Lifecycle: three near-identical per-kind flows (service/
  tool/job) collapse into one flow + a "what apply does per manager"
  table. Manager table's "start/stop" column → "how apply activates".
  Fixed stale paths-table rows (code/<name>) and a stale `service run`.
- developing-castle.md endpoint list: `POST /apply` is the one lifecycle
  endpoint; dropped retired `/deploy`; service actions are restart-only;
  added `/config/deployments/{name}/enabled`.

Net −37 lines; no retired-verb or stale-path references remain in docs.
2026-07-02 12:20:33 -07:00
Paul Payne
7410680f57 Merge refactor/gateway-converge: gateway converges via castle apply 2026-07-02 12:15:33 -07:00
Paul Payne
ef588cb806 gateway: converge, don't start/stop/reload
The gateway is a deployment (castle-gateway) — its lifecycle is the same
convergence as everything else, so the bespoke start/stop/reload verbs
(which still routed through the retired enable/disable path) are gone.

- CLI: `castle gateway` is now inspection only — bare or `status` shows
  the route table. Start/stop/reload it via `castle apply` (render routes
  + reload) or `castle restart castle-gateway`. Deletes the last users of
  `_service_enable`/`_service_disable`, so those are removed too.
- API: retire `POST /gateway/reload` (making routes live is `POST /apply`);
  `PUT /gateway/config` message + docstring now say apply, not deploy.
- UI: GatewayPanel's "Reload" button → "Apply" (useApply); drop the
  useGatewayReload hook; GatewaySettings points at `castle apply`.
- docs: `castle gateway reload|status` → `castle gateway` (status lens).

core 129 / cli 31 / api 59 pass; dashboard builds clean; live `castle
gateway` shows routes with no start/stop/reload; /gateway/reload removed
from the API.
2026-07-02 12:12:00 -07:00
Paul Payne
ca32e6f29f Merge pull request #1 from payneio/refactor/apply-converge
castle apply — one declarative converge verb
2026-07-02 12:06:23 -07:00
Paul Payne
9abef4ac34 docs+install: sweep deploy/start → castle apply
Update the README, AGENTS.md, install.sh, and docs/ for the converge
model: one `castle apply [name] [--plan]` replaces `deploy && start` and
the per-kind enable/disable/install/start/stop verbs. CLI references now
list apply + restart (the imperative bounce) + logs; recipes use
`castle apply`; "turn it off" is documented as `enabled: false` + apply.
install.sh's next-steps collapse to a single `castle apply`.

Verified: no retired-verb command instructions remain in any markdown;
install.sh syntax OK; live `castle apply --plan` reports converged;
doctor all-green; core 129 / cli 31 / api 59 pass.
2026-07-02 11:54:31 -07:00
Paul Payne
f422c1879d ui: Apply everywhere — power toggles, plan preview, no start/stop/install
Rework the dashboard around convergence:
- ServiceControls / ServiceCard / JobCard: replace start/restart/stop
  with a Power toggle (set enabled → apply, i.e. activate/deactivate)
  plus Restart (the one imperative bounce).
- ToolDetail: install/uninstall becomes an Enable/Disable toggle;
  `installed` stays the live state, `enabled` the desired one.
- ConfigPanel Apply and CreateDeploymentForm now POST /apply (one
  converge that renders + restarts only what changed) instead of
  /deploy + a separate start/restart.
- New ConvergePanel on Overview: a terraform-style Preview (POST /apply
  plan=true) showing would-activate/restart/deactivate, then Apply.
- useApply + useSetEnabled hooks; `enabled` added to Service/Job/
  Deployment types; both handle the self-restart connection drop.

Backend: PUT /config/deployments/{name}/enabled sets desired on/off
(edit config; caller runs apply) — the declarative toggle the UI uses.

Dashboard builds clean (tsc + vite); 59 API tests pass; live /apply and
/services?enabled verified after an api restart.
2026-07-02 11:49:54 -07:00
Paul Payne
5d15d18e1d api: POST /apply converge endpoint; enabled in summaries
- Replace POST /deploy with POST /apply (name + plan), returning the
  enacted diff (activated/restarted/deactivated/unchanged). /config/apply
  now delegates to core apply() too, so there's one convergence path.
- Retire /services/{name}/start and /stop (keep /restart as the
  imperative bounce). Drop install/uninstall from program actions — tool/
  static activation is convergence now.
- Surface `enabled` on ServiceSummary/JobSummary/DeploymentSummary so the
  UI can show and toggle desired state.
- conftest: drop the obsolete config_editor.get_registry patch (apply_config
  no longer reads the registry directly).

59 API tests pass; route table shows /apply (no /deploy), /restart only.
2026-07-02 11:41:00 -07:00
Paul Payne
d0a206f7d6 core+cli: castle apply — one converge verb replaces deploy/start/enable/…
Reconcile the running system to declared config in a single operation,
replacing the old deploy && start plus the per-kind enable/disable/
install/uninstall verbs. The mechanism varies by manager (systemd unit /
PATH install / gateway route); the verb never does.

Core (castle_core.deploy.apply):
- render (units, Caddyfile, tunnel) then reconcile: activate every
  enabled deployment that's down, restart any whose unit bytes changed,
  deactivate the disabled ones. Returns ApplyResult (the enacted diff).
- --plan computes the diff and touches nothing.
- restart-on-change is exact: _render_unit_files is the single source of
  truth for unit bytes, used both to write units and to predict restarts,
  so the plan can't drift from what deploy writes.

Desired state:
- DeploymentBase.enabled (default True) — the declarative on/off. apply
  converges to it; a disabled deployment is defined but not running and
  gets no gateway route (else it would 502). Registry omits enabled when
  True, keeping existing registries byte-identical.

CLI:
- add `castle apply [name] [--plan]`; keep `restart` as the one
  imperative bounce, plus status/doctor/logs/gateway/program.
- retire deploy, start, stop (top-level); service/job deploy|enable|
  disable|start|stop; tool install|uninstall; program install|uninstall.
- next-step strings + doctor hints now say `castle apply`.

Verified: core 129 + cli 31 green; live `castle apply` on a converged
node reports "nothing to do" with no spurious restarts; --plan shows an
empty diff; gateway + api stay up.
2026-07-02 11:35:34 -07:00
Paul Payne
9028d15ec6 deploy: auto-reconcile public DNS CNAMEs via Cloudflare
castle deploy now manages the public zone's tunnel CNAMEs to match the current
set of public: true services — creating missing records and deleting removed
ones — instead of only printing manual 'cloudflared tunnel route dns' hints.

New core/generators/dns.py reconciles via the Cloudflare API (stdlib only). It
only ever touches records whose content is <tunnel_id>.cfargotunnel.com, so
hand-managed records in the same zone are never altered. deploy._write_tunnel_config
calls it in both branches (delete of the last public service cleans up its CNAME).
Without a token it's a no-op and the manual route-once hints are surfaced as before.

Token: ~/.castle/secrets/CLOUDFLARE_PUBLIC_DNS_TOKEN, a single DNS:Edit permission
scoped to the public zone (the 'Edit zone DNS' template) — confirmed sufficient:
that one permission resolves the zone by name and edits records, no separate
Zone:Read needed. Separate from CLOUDFLARE_API_TOKEN (ACME, internal zone).

castle doctor gains a read-only probe that WARNs when the token is absent (CNAMEs
stay manual) and FAILs when it can't reach the zone/records.
2026-07-02 11:02:34 -07:00
Paul Payne
2adc073863 Add castle doctor + getting-started docs
The README/docs explained how the system works but not how to get it up
and running, and there was no way to tell a healthy node from a
half-configured one. Two additions close that gap.

castle doctor — a read-only preflight/postflight check. It inspects
setup *and* runtime (CLI on PATH, uv, lingering; repo:, control plane
registered, dashboard built; gateway/api running + listening, specs
generated; and — under tls=acme — DNS-plugin Caddy, provider token,
:443 bind; tunnel config for public services) and, for anything not
green, prints the exact next command. Exit 0 when nothing failed
(warnings allowed), 1 otherwise, so it doubles as a scriptable smoke
test after install/deploy. Agents can lean on it the same way they use
`castle tool list`.

Docs — the quick start now leads with prerequisites and one command
(install.sh installs the CLI + registers the control plane, so the old
manual `uv tool install` step is gone), adds a `castle doctor` verify
step, and gains an "exposure ladder" table framing the three rungs
(localhost → LAN HTTPS → public) that link the deep DNS/TLS/tunnel docs.
install.sh's closing summary and the AGENTS.md CLI reference mention
doctor too.
2026-07-02 10:07:58 -07:00
Paul Payne
b917db5e00 feat(supabase): isolate each app in its own Postgres schema + stack teardown
Rework the supabase stack so every app owns an isolated Postgres schema
(= program name) instead of sharing `public` with a table_prefix. This
gives a clean, knowable teardown and drops migration version tracking to
per-app, so version tokens can no longer collide across apps.

- stacks: add StackHandler.owns_data + teardown() (default no-op). Supabase
  build now creates+grants the app schema, tracks migrations in
  <schema>.schema_migrations, and applies each with search_path set to the
  schema (write unqualified names). teardown = drop schema <app> cascade.
  Fix _substrate_db_url default port 5432 -> 5433 (substrate's direct PG).
- deploy: derive ${supabase_app_schemas} from registered apps; substrate maps
  PGRST_DB_SCHEMAS: public,storage,graphql_public${supabase_app_schemas} so
  PostgREST exposes each app schema (restart substrate after add/remove).
- delete: generic owns_data -> teardown path behind new --purge-data flag,
  replacing the hardcoded supabase remnant note.
- scaffold + docs: new apps born schema-isolated (unqualified tables,
  db:{schema}, schema: in supabase.app.yaml).
2026-07-02 07:39:02 -07:00
Paul Payne
4c12882cc7 fix(agents): terminal fills the panel on maximize
- Mount xterm as an absolute inset-0 fill so it always has an explicit box and
  fit() measures the real size when the dock expands (was collapsing via the
  flex/percentage height chain).
- Acquire the pty slave as the child's controlling terminal (setsid + TIOCSCTTY)
  so the kernel delivers SIGWINCH on a browser resize. Previously the initial
  size was read once but live resizes were dropped, so the pane stayed at its
  startup size — content never grew with the window.
2026-07-02 04:02:18 -07:00
Paul Payne
373ddcaca8 ci: reject PRs that reference the assistant by name in title/body/commits
Server-side counterpart to the global commit-msg hook (which blocks the same word
locally, but can be bypassed with --no-verify or via GitHub UI edits). Checks only
message text, never file contents.
2026-07-02 03:42:04 -07:00
Paul Payne
bd644ea905 feat(agents): durable tmux-backed terminal sessions + fit/chrome fixes
- Run agent sessions under an isolated tmux server (its own systemd scope, with a
  keepalive session) so they survive a castle-api restart and stay rediscoverable;
  each WebSocket is a tmux attach client. Falls back to the in-memory pty backend
  when tmux is absent or CASTLE_API_AGENT_BACKEND=memory.
- Refactor the session layer behind one Attachment interface (create/list/attach/
  close), keeping resume-by-id, agent-native history, and continue modes.
- Fix the terminal not refilling when the dock is maximized (explicit refit signal
  + instant size transition so xterm measures the final box).
- On narrow screens, go full-bleed and strip chrome (header/border/padding) when
  maximized.
2026-07-02 03:41:42 -07:00
Paul Payne
25e2275999 deploy: fix tunnel unit name so deploy auto-reloads cloudflared
_TUNNEL_NAME was "tunnel", but the deployment is named castle-tunnel, so
unit_name() produced castle-tunnel.service — a unit that doesn't exist (the
real one is castle-castle-tunnel.service). The is-active check never matched,
so deploy printed "Tunnel not running" and skipped the ingress reload. New
public: true services then 404'd until a manual tunnel restart.

Set _TUNNEL_NAME = "castle-tunnel" to match the deployment name, mirroring
_GATEWAY_NAME. Deploy now restarts cloudflared on ingress changes.
2026-07-01 16:54:27 -07:00
Paul Payne
d34ff9220d install.sh: bootstrap Castle's own control plane
A fresh clone + ./install.sh set up the infra (Docker, Caddy, MQTT,
Postgres) but left the registry empty — so `castle deploy && castle
start` brought up nothing. install.sh now gets a person all the way to
a running Castle:

- ensure_uv + install_cli: install uv and the `castle` CLI (editable
  from ./cli) so the command exists on a fresh machine.
- seed_control_plane: register castle-gateway, castle-api, and the
  castle dashboard from a new bootstrap/ seed dir, and add `repo:` to
  castle.yaml so `source: repo:<name>` resolves. Never clobbers
  existing entries.
- build_dashboard: build app/dist/ so the gateway has a UI to serve
  (best-effort; warns if pnpm is absent).

The gateway seed uses a `__SPECS_DIR__` placeholder (no machine-specific
paths in the repo), substituted with this machine's specs dir at seed
time. .gitignore negates bootstrap/**/castle.yaml so the seeds are
tracked despite the user-registry ignore rule.

Verified: seeded config loads, derives kinds service/service/static,
and renders an off-mode Caddyfile serving the dashboard at :9000/ with
/api/* proxied to castle-api.
2026-07-01 16:33:42 -07:00
Paul Payne
442b1692d9 install.sh: Neo4j is optional, off by default
Neo4j was auto-provisioned for everyone; a graph DB isn't core, so don't foist it
on new users. Gate it behind maybe_setup_neo4j: enable with --with-neo4j, an
interactive yes, or a previous run's choice (recorded in infra.conf, so a decline
isn't re-prompted). MQTT and Postgres stay default-on. Existing installs with
NEO4J=container keep running it unchanged.
2026-07-01 16:21:12 -07:00
Paul Payne
643d7591d2 Remove discussion.md to streamline documentation and focus on core concepts for castle's design. 2026-07-01 15:40:33 -07:00
Paul Payne
043b5028f0 Update AGENTS.md and CLAUDE.md for clarity; add developing-castle.md for development guidance 2026-07-01 15:39:39 -07:00
Paul Payne
9d6ea60294 install.sh: DATA_DIR = /data/castle (match runtime + deployments)
install.sh provisioned mqtt/postgres/neo4j data + their docker -v mounts under
~/.castle/data, but castle_core.config and every container deployment use
/data/castle — so a fresh install put the data in the wrong place. Resolve DATA_DIR
the same way config.py does (${CASTLE_DATA_DIR:-/data/castle}); since that's outside
$HOME, create it with a sudo+chown fallback when the parent isn't user-writable.
Also add REPOS_DIR and fix the summary (Code→Repos, 'castle services start'→'castle
start').
2026-07-01 15:39:19 -07:00
Paul Payne
d1076995a6 install.sh: create programs/ + deployments/ and seed a default castle.yaml
Moves the ~/.castle/{programs,deployments} dir creation and the minimal
castle.yaml (gateway.port: 9000) into install.sh's create_directories — seeded
only when absent, never clobbering an existing config. Drops the stale 'code/'
dir (program source lives in /data/repos now). README quick-start no longer
does the manual mkdir/printf; install.sh handles it.
2026-07-01 15:29:48 -07:00
Paul Payne
06b301550a Rewrite README: current model, tools lens, agents, accurate gateway/env
Rebuild the README on the 'personal software estate' framing and bring it current:
programs vs deployments (manager → derived kind), the service/job/tool lenses,
'castle tool list --json' for assistants, and the new agent-launching mechanism
(castle.yaml agents: + the dashboard terminal dock). Fix stale claims — subdomain
routing (no path prefixes), defaults.env is explicit (no auto-injected convention
vars), /data/repos + /data/castle paths, castle deploy && castle start. Trim the
sprawling/stale API table to a pointer; link docs + AGENTS.md + CLAUDE.md.
2026-07-01 15:21:54 -07:00
Paul Payne
ba2001df49 Adds assistants. 2026-07-01 15:11:45 -07:00
Paul Payne
6929bd89d3 Detail pages: rename the 'Runs' field to 'Launch' (matches the launcher concept)
The field showing a deployment's launcher + target is now labelled 'Launch'
(consistent with launcher/manager vocabulary) on ServiceDetail and the Service/
Job/Create field editors.
2026-07-01 13:35:02 -07:00
Paul Payne
776e1235d7 castle list: add Tools and Static deployment sections
Tools and statics only appeared as labels in the Programs catalog; now they get
their own sections alongside Services and Jobs (deployment views over
config.deployments filtered by kind), in both the human and --json output.
2026-07-01 13:24:26 -07:00
Paul Payne
416ec043fc CLI: add a 'castle tool' lens + sweep for model alignment
The tools lens coding assistants need. 'castle tool list [--json]' lists tools
(programs with a manager:path deployment) surfacing each tool's actual EXECUTABLE
— resolved from pyproject [project.scripts], which can differ from the program
name (litellm-intent-router → intent-router) — plus description and install state.
'castle tool info <name> [--json]' details one; 'castle tool install|uninstall'.
--json is a clean machine-readable catalog for building context.

Sweep: main.py descriptions/help mention the tool lens; 'frontend'→'static' in
install help; add.py._detect drops the dead behavior return (add adopts source
only); stale comments/help fixed. Docs (CLAUDE.md/README/registry) document
'castle tool'. Tests: cli/tests/test_tool.py (list/info/json).

Suites: core 124, cli 29, castle-api 58.
2026-07-01 13:21:05 -07:00
Paul Payne
bcd040edfd Detect tools whose executable name differs from the program name
A program's package name can differ from the console script it exposes
(litellm-intent-router installs the 'intent-router' executable), so the
which(<program>) install-check missed it: the tool installed fine but castle kept
reporting 'not installed', making the Install button look dead. _on_path now also
consults 'uv tool list' by package name (2s-cached), and the API's install-checks
use the shared tool_installed() helper. Verified litellm-intent-router now reads
installed/active True.
2026-07-01 13:13:32 -07:00
Paul Payne
60cbb8a844 Programs page: deployment-kind filter chips (opt-in)
ProgramList gains a filterable prop; the Programs page enables it. Chips filter by
whether a program has a deployment of that kind (membership over program.deployments),
so a tool-and-job program shows under both Tool and Job. Counts are programs-per-kind.
Tools page stays unfiltered (already scoped).
2026-07-01 13:10:13 -07:00
Paul Payne
05031a1395 Static service detail: drop the duplicate Subdomain field
A static showed both 'Served at' (from the static block) and 'Subdomain' (the
general proxied-service field) — same URL twice. Gate 'Subdomain' to non-static;
a static's gateway URL is already 'Served at'.
2026-07-01 13:08:46 -07:00
Paul Payne
0463cd91f1 Fix tool install-state, inherit program description, declutter Tools cards
- Tool detail showed installed tools as 'Not installed': /deployments/{name}
  populated 'installed' for path deployments but left 'active' null, and
  ToolDetail keyed off 'active'. Now the endpoint sets active=installed for path,
  and ToolDetail reads 'installed' directly (verified: on-PATH ⇒ True, else False).
- A deployment now inherits its program's description by default: copied at
  creation (CLI create + API _save_deployment on new deployments) + a one-time
  patchup of existing ones (29 deployments).
- Tools page cards no longer list deployments (ProgramCard showDeployments=false) —
  the lens already scopes to the tool; the deployment list is a Programs-catalog thing.
2026-07-01 13:07:00 -07:00
Paul Payne
183c589b63 Tools page cards link to the tool detail page, not the program
ProgramCard/ProgramList take an optional linkBase (default /programs); the Tools
page passes /tools so a tool tile opens /tools/<name> (its deployment detail),
not /programs/<name>.
2026-07-01 12:59:31 -07:00
Paul Payne
86d4552fdc Dashboard: Deployments nav group, tool detail page, deployment links, static-aware detail
- Nav: Services/Scheduled/Tools nested under a 'Deployments' group (expandable;
  flattened to icons when the sidebar is collapsed). Programs stays top-level.
- Overview: Tools tile (Wrench, count of tool programs).
- New /tools/:name tool detail page (useDeployment): Install/Uninstall lifecycle,
  a link up to its program, and an editable ToolFields (description + env).
- ServiceDetail is static-aware: a caddy deployment shows served-by-gateway +
  URL + root, no start/stop, and edits via a new StaticFields (root + public + env)
  so a launcher is never injected into a caddy deployment.
- Program detail's Deployment section is now a clean list of links — every
  deployment (tool→/tools, service/static→/services, job→/jobs) links to its
  detail page; lifecycle controls moved onto those pages.

clean pnpm build + tsc.
2026-07-01 12:57:25 -07:00