Commit Graph

55 Commits

Author SHA1 Message Date
Paul Payne
ba2001df49 Adds assistants. 2026-07-01 15:11:45 -07:00
Paul Payne
bcd040edfd Detect tools whose executable name differs from the program name
A program's package name can differ from the console script it exposes
(litellm-intent-router installs the 'intent-router' executable), so the
which(<program>) install-check missed it: the tool installed fine but castle kept
reporting 'not installed', making the Install button look dead. _on_path now also
consults 'uv tool list' by package name (2s-cached), and the API's install-checks
use the shared tool_installed() helper. Verified litellm-intent-router now reads
installed/active True.
2026-07-01 13:13:32 -07:00
Paul Payne
10a86d0b6f kind is a deployment property, not a program property
A program has no single kind — it HAS deployments, each with its own kind (a
program can be a tool AND a job, e.g. protonmail). Remove program-level kind:

- core: drop ProgramSpec.kind; CastleConfig.kind_of → deployments_of(name) →
  [(deployment-name, kind)]; tools property derives from a tool deployment.
- api: ProgramSummary drops kind/services/jobs → deployments: [{name, kind}];
  /programs?kind= filters by deployment-kind membership; a program's legacy
  DeploymentSummary carries kind=None.
- cli: list/info show a program's set of deployment kinds; --kind filters by
  membership.

Also: Services page now covers statics — /services returns kind in
{service, static} (both are exposed, URL-reachable 'services', caddy vs systemd),
and ServiceSummary gains kind + manager to distinguish them.

Suites: core 124, cli 25, castle-api 58.
2026-07-01 12:25:54 -07:00
Paul Payne
317232ca6a Manager-first deployment model: split runner, merge service/job, frontend→static
Replace the conflated `runner` axis with two orthogonal ones: `manager`
(systemd|caddy|path|none) — who supervises/realizes a deployment — and, for
systemd only, a nested `launcher` (python|command|container|compose|node) — how
the process starts. ServiceSpec and JobSpec collapse into one manager-
discriminated DeploymentSpec union (Systemd/Caddy/Path/Remote); the services/
and jobs/ config dirs collapse into one deployments/ dir. The human "kind"
(service|job|tool|static|reference) is fully derived (kind_for), never stored —
the frontend kind is renamed static. behavior is gone.

- core: DeploymentSpec union + LaunchSpec + kind_for; legacy-aware loader
  normalizes old runner shapes; CastleConfig.deployments with derived
  services/jobs/tools views; registry.Deployment carries manager/launcher/kind.
- cli: service/job/tool as filtered views + a deployment group; --behavior→--kind,
  create --runner→--launcher; lifecycle dispatches over config.deployments.
- castle-api: /deployments primary with /services,/jobs as views; summaries
  derive kind; PUT/DELETE /config/deployments/{name} (services/jobs aliased).
- app: KindBadge, frontend→static everywhere, pick-a-kind creation wizard,
  per-kind config editors.
- docs: single deployments/ layout, manager/launcher, static kind throughout.

Live migration verified byte-identical: regenerated Caddyfile and every unit
ExecStart line unchanged, so nothing restarted. Suites: core 124, cli 25,
castle-api 55; dashboard build + type-check clean.
2026-07-01 10:23:03 -07:00
Paul Payne
00e8d58c6a Derive program behavior from deployments; stop persisting it
behavior (tool/daemon/frontend) is now a derived display label computed
from a program's deployments — path->tool, static->frontend, process->daemon
(behavior_for_runner). load_config populates it so every reader gets the live
value; _spec_to_yaml_dict excludes it so it is never written to disk.
_summary_from_service derives it from the runner instead of hardcoding daemon.
Fixtures gain path/python deployments so tool/daemon behaviors derive.
2026-07-01 08:46:54 -07:00
Paul Payne
e9b197756b Dashboard: editable gateway settings + public-exposure panel
Surfaces the DNS/Cloudflare/public state that previously required editing
castle.yaml + curl:

- API: GET /gateway now returns domain, public_domain, tunnel_id,
  tunnel_connected, and a public_url per public route; new PUT /gateway/config
  edits tls/domain/public_domain/tunnel_id in castle.yaml (deploy to apply).
  save_config now persists public_domain/tunnel_id (previously dropped).
- UI: a GatewaySettings row in the Gateway panel shows TLS / domain / public
  (tunnel up/down + public_domain) with inline edit; each public route gets a
  green "public" badge linking to its <name>.<public_domain> URL.
2026-07-01 08:29:36 -07:00
Paul Payne
fc83ce40c1 Public toggle composes for static frontends; document the deployment model
- A static service is inherently exposed, so `public: true` no longer trips the
  "requires proxy" validator — frontends can be public via the tunnel for free
  (the original friction that started this refactor, now resolved).
- docs/registry.md: the manager model (systemd/caddy/path/none), the runner→manager
  table incl. static/path, and behavior as a derived display label.
2026-07-01 07:01:37 -07:00
Paul Payne
7eca3cc217 Tools as PATH deployments; demote behavior from lifecycle logic
- manifest: RunPath (runner: path) — a CLI on PATH via `uv tool install`, manager
  = path, no unit/route (like remote/static).
- lifecycle/service: a path deployment's start/stop = install/uninstall (routed
  through activate/deactivate, which skip the reinstall when already on PATH).
  `castle service start/stop <tool>` and status now dispatch by manager; status
  shows the manager tag [systemd|caddy|path|none].
- config.tools() now derives from `runner: path` services, not the behavior label.
- castle-api: the "installed" flag derives from manager==path, not behavior==tool.
- create: a python-cli stack emits a `runner: path` service.

`behavior` no longer drives any runtime logic — it survives only as a display
label + `list --behavior` filter. Verified live: html2text uninstall→reinstall
round-trip via `castle service stop/start`.
2026-07-01 06:59:40 -07:00
Paul Payne
57861e58c1 Manager abstraction + manager-dispatched lifecycle; frontends create static services
- manifest: `manager_for(runner)` — the single source of truth mapping a runner to
  its manager (systemd | caddy | path | none). deploy's `managed` now derives from it.
- lifecycle: is_active/activate/deactivate dispatch by manager instead of behavior
  (removed the stale `_is_static_frontend`, which broke once frontends became
  services). caddy → gateway reload; path → install/uninstall; none → no-op.
- cli/service: start/stop/restart and enable/disable route through the manager
  (systemctl only for systemd; static reloads the gateway; remote is a no-op);
  platform restart skips non-systemd runners.
- create: a frontend stack (react-vite/supabase) now emits a `runner: static`
  service, not a bare frontend program.
2026-07-01 06:51:31 -07:00
Paul Payne
5f6afbf007 Checkpoint: Cloudflare tunnel + public toggle, and static-runner frontends
Two features built this session, committed together as a checkpoint before the
deployment-model refactor (they share files, so kept as one rollback point):

- Public exposure via Cloudflare tunnel: a `public` toggle on services (requires
  proxy), gateway.public_domain/tunnel_id config, a cloudflared ingress generator
  (generators/tunnel.py) that maps <name>.<public_domain> to the gateway's
  internal host, deploy wiring, and docs/tunnel-setup.md.
- Frontends as `static`-runner services: a RunStatic runner served by Caddy
  (file_server), no systemd unit (like `remote`); the Caddyfile generator now
  derives static routes from services instead of a behavior==frontend branch.
2026-07-01 06:44:23 -07:00
Paul Payne
c28198dab6 Add ${public_url} defaults.env placeholder for gateway-facing origin
Services often need their own public origin — the URL a browser loads them
at through the gateway — to populate CORS/WebSocket/secure-context allowlists.
Previously this had to be hardcoded in the app's own config, which silently
broke when the subdomain-routing migration changed the address (openclaw's
Control UI rejected its new origin).

${public_url} resolves at deploy to the service's gateway-facing base URL:
https://<name>.<gateway.domain> when exposed under tls: acme, else the
node-local http://localhost:<port>. It tracks gateway.domain, so a domain
change needs no per-app edit. Jobs (not exposed) don't get it.

Matches the TLS mode as the raw lowercased string == "acme", the same way the
Caddyfile generator does (the TLSMode enum is stale).
2026-06-30 22:28:50 -07:00
Paul Payne
d0075300e3 Fix proxy:bool refactor drift in CLI fixture + dead structural key
The proxy->bool refactor (770574f) missed the shared CLI test fixture, which
still used proxy: {caddy: {path_prefix: ...}} — failing ~25 CLI tests with
'Input should be a valid boolean'. Set it to proxy: True.

Also drop the now-dead 'caddy' entry from _STRUCTURAL_KEYS (no nested caddy
key exists anymore) and refresh a stale proxy.caddy docstring reference.
2026-06-30 21:45:44 -07:00
Paul Payne
14145c9f15 Populate the program stack select from castle-api
The dashboard's stack dropdown hardcoded its options (python-cli/fastapi/
react-vite), so a supabase program showed "(none)" — the value had no matching
option. Make the backend the single source of truth instead:

- core: available_stacks() returns sorted(HANDLERS) — the stacks castle has
  handlers for.
- castle-api: GET /stacks exposes it.
- cli: --stack choices derive from available_stacks() (drops the duplicated list).
- app: ProgramFields fetches /stacks (useStacks) and renders options from it,
  always including the current value so it never silently blanks; StackBadge
  gets a Supabase label.

A new backend stack now appears everywhere without a frontend edit.
2026-06-30 21:43:11 -07:00
Paul Payne
bd9ea76d6a Remove stale TLSMode/HttpPublic exposure vestige
TLSMode (off/internal/letsencrypt) was only referenced by HttpPublic.tls,
and HttpPublic (the expose.http.public field, still carrying a path_prefix
and a tls defaulting to the deleted internal mode) was referenced by no
config, generator, deploy code, or test — a leftover from the pre-proxy:bool
exposure design. Drop the enum, the model, the public field, and both CLI
shim re-exports.
2026-06-30 21:38:24 -07:00
Paul Payne
770574f575 Flatten proxy config to a bool (drop the caddy key)
CaddySpec had only `enable` left (path_prefix/host removed; extra_snippets was
unused), so `proxy: { caddy: {} }` was three levels of nesting for a checkbox —
and it invited drift (supabase silently reverted to proxy.caddy.host). Replace it
with a plain `ServiceSpec.proxy: bool`.

- Model: remove ProxySpec/CaddySpec; `proxy: bool = False`. Readers simplified to
  `bool(svc.proxy)` (generator, castle-api summaries, CLI info).
- CLI create / dashboard editor + create form: write `proxy: true` / a checkbox.
- Configs migrated to `proxy: true` (and supabase's stale proxy.caddy.host block
  removed).
- Docs (registry, dns-and-tls, design, stack guides) + tests/fixtures updated;
  also fixed lingering path-model staleness in the stack guides.
2026-06-30 21:23:14 -07:00
Paul Payne
4ff6af8f4d Rename the dashboard program castle-app -> castle
The dashboard is now served at castle.<domain> (was castle-app.<domain>). Since
the subdomain is the program name, this is just a program rename plus the one
special-case constant.

- Generator: _DASHBOARD = "castle" (the :9000 redirect target and off-mode root).
- Program file renamed (runtime): programs/castle-app.yaml -> programs/castle.yaml
  (source unchanged: repo:app).
- Dashboard UI: ProgramDetail shows a frontend's subdomain (via subdomainUrl),
  dropping the old castle-app-at-root special-case; comment refs updated.
- Docs: registry.md/design.md/react-vite.md updated to the subdomain model and the
  `castle` name (react-vite.md also corrected — frontends serve at their subdomain
  root with VITE_BASE=/, and builds are manual via `castle program build`, not
  deploy).
2026-06-30 21:04:18 -07:00
Paul Payne
43ef1cc588 Subdomain-only gateway routing; drop path prefixes
Collapse the two proxy shapes (path_prefix + host) to a single checkbox: a
service is either port-only, or exposed at <service-name>.<gateway.domain>. Path
routes and their prefix-stripping foot-guns are gone; the subdomain is always the
service name (rename the service to change it).

- Schema: CaddySpec is just `enable` (presence = expose). service_proxy_targets
  returns (expose, port, base_url); Deployment carries `subdomain` (was
  proxy_path/proxy_host). Registry/deploy/mesh/CLI updated in lockstep.
- Generator: compute_routes emits one host route per exposed service/frontend
  (address = name). acme mode = one *.<domain> site with a reverse_proxy matcher
  per service + a file_server matcher per frontend; the :<port> site redirects to
  the dashboard. off mode = a HTTP control plane on :<port> (dashboard at / +
  /api → castle-api); other services are port-only.
- Dashboard/API: castle-app and castle-api are ordinary subdomains. The dashboard
  derives the API base at runtime (castle-api.<domain> on a subdomain, else /api)
  and calls it cross-origin — castle-api already allows CORS *. Frontends build
  with VITE_BASE=/ (served at their subdomain root).
- CLI: `service create` drops --path/--host; --no-proxy makes it port-only.
- Docs/tests reworked for the model; docs use generic placeholders only (no
  personal host/domain/IP details).
2026-06-30 20:47:03 -07:00
Paul Payne
a022a136fe Retire gateway.tls=internal; host routes use a bare subdomain label
acme mode fully replaced the internal-CA path (which required installing a private
CA on every device — the exact pain, esp. on Android, that acme avoids). Remove
`internal` entirely and simplify how services declare host routes.

- Generator (caddyfile.py): drop the `tls internal` branch — modes are now
  `off` | `acme`. In acme mode the published subdomain is the **first DNS label**
  of `proxy.caddy.host` (a bare `claw`, or a legacy `claw.civil.lan`, both →
  `claw.<domain>`), so services stay domain-agnostic and the declared value is
  authoritative again (no more silent service-name override). Shared
  `_host_matcher_block` reused by off-mode and the acme wildcard site.
- castle-api: delete `GET /gateway/ca.crt`, `_gateway_ca_pem`, `_ca_fingerprint`
  and the now-unused imports; drop `ca_fingerprint` from `GatewayInfo` (keep
  `tls`).
- Dashboard: remove the CA-cert download button + unused imports; drop
  `ca_fingerprint` from the `GatewayInfo` type.
- Tests: replace TestCaddyfileTlsInternal with an off-mode class (keeps the
  runner-agnostic host-route coverage); acme tests assert first-label derivation
  incl. label-wins-over-service-name; drop the castle-api CA-endpoint test.
- Docs: registry.md + dns-and-tls.md — two-mode tables (off|acme), remove the
  internal sections/CA-download, document the bare-label host convention; note a
  domain-less node stays on `off`.
2026-06-30 19:55:17 -07:00
Paul Payne
121f970f14 Add gateway.tls=acme: Let's Encrypt wildcard cert via DNS-01
internal-CA HTTPS forces every device to trust a private root, which breaks down
on some clients (Android browsers). acme mode instead obtains a real Let's Encrypt
wildcard cert (*.<domain>) via a DNS-01 challenge, so browsers trust it with zero
CA install — while services stay internal-only (no inbound exposure, no public A
records; only a transient _acme-challenge TXT touches the public zone, and LAN DNS
resolves the names to a private IP).

- Config: GatewayConfig/NodeConfig gain domain, acme_email, acme_dns_provider
  (plumbed through load/save + registry + deploy, omitted-when-empty for
  round-trip stability). tls stays the discriminator: off | internal | acme.
- Caddyfile generator: acme branch emits a global {email, acme_dns <provider>
  {env.TOKEN}} block + one *.<domain> wildcard site with host matchers derived
  from the service name (<name>.<domain>); path/static stay on :<port>. Shared
  _host_matcher_block helper (reused by off-mode). CASTLE_ACME_STAGING=1 toggles
  the LE staging CA; acme-without-domain falls back to HTTP with a warning.
- deploy(): _acme_preflight warns (never writes) when domain, the gateway-service
  token env, or the CLOUDFLARE_API_TOKEN secret is missing.
- install.sh: opt-in --with-dns-plugin[=provider] builds a DNS-plugin Caddy via
  xcaddy (pinned) to /usr/local/bin/caddy, which the gateway picks up on deploy.
- Tests: TestCaddyfileTlsAcme (global block, wildcard site, derived host matcher,
  path routes on :port, staging toggle, no-domain fallback). Docs: gateway.tls
  modes table + full acme/DNS-01 section (setup, wild-central LAN DNS, staging).
2026-06-30 18:45:13 -07:00
Paul Payne
b726e79c23 Add supabase stack + general compose runner
Adds "a stack whose default is a substrate": a shared self-hosted Supabase
backend plus a `--stack supabase` that scaffolds per-app projects (migrations +
edge functions + static UI) which deploy against it. Apps own their code and
stay repo-durable; only their rows/blobs live on the shared substrate.

- compose runner: new RunCompose supervises a multi-container stack as one
  systemd unit (ExecStart=`compose up`, generated ExecStop=`compose down`);
  secrets via EnvironmentFile. Reusable beyond Supabase. Deployment.stop_cmd
  carries the teardown command through the registry.
- supabase stack: CLI --stack choice, STACK_DEFAULTS→frontend with
  build.outputs=[public], _scaffold_supabase() (migrations/RLS/functions/
  static UI/app manifest), and SupabaseHandler with a forward-only idempotent
  migration runner (plan_migrations + psql build) and deno dev-verbs.
- docs: docs/stacks/supabase.md, registered in CLAUDE.md; compose runner
  documented in registry.md.
- tests: compose run/stop cmd, systemd ExecStop, compose host-route TLS,
  migration planner, supabase verb resolution, create --stack supabase.
2026-06-30 18:18:47 -07:00
Paul Payne
0d9c1ae6c2 Add gateway.tls=internal: HTTPS for host routes via Caddy's local CA
The gateway was HTTP-only by design, but a plain-HTTP origin is not a browser
"secure context", so apps that need WebCrypto (device identity, E2E crypto)
can't run when proxied — only localhost or HTTPS qualifies.

Add an opt-in `gateway.tls: internal`. When set, each host route
(proxy.caddy.host) is emitted as its own `<host> { tls internal; reverse_proxy }`
site, served over HTTPS by Caddy's local CA (Caddy binds :443 and redirects :80).
Path-prefix/static routes stay on the HTTP :<port> site — give a service a host
route to put it on HTTPS. Default (unset/off) keeps the existing HTTP-only
Caddyfile (single :<port> block, auto_https off, host matchers inline).

Plumbing mirrors gateway_port: GatewayConfig.tls (castle.yaml load/save) →
NodeConfig.gateway_tls (registry load/save, set in deploy) → the Caddyfile
generator. Tests cover both modes; docs/registry.md documents the option plus
the two operational requirements (binding 443/80 via the unprivileged-port
sysctl, and trusting Caddy's root CA on clients) and the host-route-vs-prefix
guidance that motivates it.
2026-06-29 08:13:17 -07:00
Paul Payne
b105487790 Fix gateway-route drift when a service is edited
Editing a service's routing (port, path_prefix, host) and then hitting
"apply" or "gateway reload" — rather than a full deploy — left the running
Caddyfile reflecting the old routing. The Caddyfile was generated from the
derived registry.yaml snapshot, which only `deploy()` rebuilds from
castle.yaml; apply/reload regenerated from that stale snapshot.

Make castle.yaml the single source of truth for local routes:

- caddyfile.py: new shared `service_proxy_targets()` derives a service's
  (proxy_path, proxy_host, port, base_url) from its spec. `compute_routes`
  now builds local routes from `config.services` when castle.yaml is
  loadable, falling back to `registry.deployed` only when it isn't. Since
  deploy/apply/reload all funnel through `compute_routes`, every regeneration
  path now tracks the spec. Mesh/remote and static-frontend routes unchanged.
- deploy.py: `_build_deployed_service` uses the same shared deriver, so the
  written registry and the computed routes can't diverge.
- config_editor.py: `/config/apply` now runs a full `deploy()` (rebuilding
  units + Caddyfile and reloading the gateway) before restarting services, so
  changed unit ExecStarts actually take effect. Scheduled jobs are no longer
  restarted on apply. Drops the now-unused direct caddy-reload path.
- Regression tests: a stale registry's old port/path loses to castle.yaml;
  registry fallback still works when config is unavailable.
2026-06-29 07:04:06 -07:00
Paul Payne
5987f01749 Refactor configuration handling to use directory-per-resource layout
- Introduced `_write_castle_config` function to scatter nested config into separate YAML files for programs, services, and jobs.
- Updated `castle_root` fixture to utilize the new config writing method.
- Added tests for configuration aggregation and scattering in `test_health.py`.
- Removed legacy unit handling from `manifest.py` and refactored related code in `config.py`.
- Updated documentation to reflect new configuration directory structure.
- Removed obsolete unit tests related to unit expansion.
2026-06-28 14:01:18 -07:00
Paul Payne
48c3915e4d feat: Enhance node runner to execute scripts in source directory and update tests 2026-06-28 13:18:28 -07:00
Paul Payne
3f1b2e6f56 feat: Implement secret management for environment variables and enhance systemd unit generation 2026-06-27 12:47:05 -07:00
Paul Payne
26a00c4073 refactor: Run python services in place via uv run
A python-runner service ran from a tool venv (uv tool install --editable),
separate from the project venv that uv sync manages. The editable link kept
source live, but dependencies were resolved independently in each venv, so a
new dependency landed in the project venv and never reached the running
service — leaving no clean rollout for it.

Deploy now emits ExecStart=uv run --project <source> --no-dev <program>,
running the service in place from its own project venv. uv run syncs the env
to the lockfile before launching, so a single restart picks up both code and
dependency changes; the ExecStart is deterministic from source and never goes
stale. The tool-venv install is gated to the command runner only; uv tool
install is reserved for tool-behavior programs, where PATH is the point. A
python service with no resolvable source falls back to a PATH lookup.
2026-06-17 11:03:11 -07:00
Paul Payne
cd0a3bc9e2 fix: Make react-vite builds robust to pnpm deps-status check
pnpm 10+ runs a deps-status check before `pnpm <script>` that wants to
purge and reinstall node_modules, aborting in a non-TTY build with
ERR_PNPM_ABORTED_REMOVE_MODULES_DIR_NO_TTY. It is a false positive for a
build verb (which builds, not installs), so ReactViteHandler now prepends
--config.verify-deps-before-run=false to every pnpm call, building against
the existing modules. CI=true is kept as a general non-interactive guard.
2026-06-17 11:03:02 -07:00
Paul Payne
290bcb6383 fix(core): unify build-time and runtime PATH via USER_TOOL_PATH_DIRS
Generated systemd units hardcoded a thin PATH (~/.local/bin + system dirs)
while dev-verb builds used a separate _EXTRA_PATH_DIRS list, so a service
could be built and run with different tools on PATH. Hoist a single
USER_TOOL_PATH_DIRS in config.py, consumed by both stacks._build_env
(build) and the systemd unit generator (runtime).

Also prefer pnpm's modern $PNPM_HOME/bin shim over the bare $PNPM_HOME dir
(older installs leave a stale version wrapper there) and add
/usr/local/go/bin so Go-based builds/services resolve their toolchain.
2026-06-16 17:47:49 -07:00
Paul Payne
392d9039d8 fix: 'check' returns combined output (was silent on success)
The check composite returned ActionResult with no output on success, so the
dashboard's 'Check All' (and the CLI) showed nothing when checks passed — only
a failing sub-verb ever produced output. It now accumulates each sub-verb's
output with a ✓/✗ marker (lint / type-check / test), on both success and
failure, so the output panel always shows what ran.

Also fixes lan-info's type-check: socket.getaddrinfo sockaddr is loosely typed,
so str(infos[0][4][0]) makes the address an unambiguous str.

core 94 green; ruff clean; verified POST /programs/lan-info/check returns output.
2026-06-15 09:40:46 -07:00
Paul Payne
32e2ac1f8e fix(gateway): redirect /foo → /foo/ for path routes
handle_path /foo/* matches /foo/ but not the bare /foo, so the no-trailing-
slash form fell through to the root catch-all (castle-app) — which then 404'd
because its router has no such route. Hitting http://gateway:9000/power-graph-app
served the dashboard's 404 instead of Power Graph; same for every proxied
service accessed without the slash.

The generator now emits 'redir /foo /foo/' for each path-prefix route (static,
proxy, remote), so the no-slash form canonicalizes to the slash form that
handle_path matches. Caddy's bare path matcher is exact, so /foo/bar is
unaffected. Host routes and the root '/' are unaffected.

Verified: /power-graph-app → 302 → Power Graph; caddy validate clean; core 94 /
api 52 green.
2026-06-14 22:26:00 -07:00
Paul Payne
c40f84104d feat: unified gateway route table (static · proxy · remote)
The gateway does two things — reverse-proxy services and serve static
frontends — but the dashboard/API route table only ever showed the proxy
routes, so 'serving a frontend' and 'proxying a service' looked like unrelated
features and castle-app/power-graph-app were invisible in the route view.

Now there's one concept: a route maps an address (path '/foo' or host 'foo.lan')
to a target of one kind — static (a built dist served by file_server), proxy
(a local service port), or remote (a service on another node).

- core: caddyfile.py gains compute_routes() — the single source of truth for
  the route list; generate_caddyfile_from_registry renders it (output byte-for-
  byte identical, verified against the live Caddyfile). Adds a GatewayRoute
  dataclass.
- api: GatewayRoute model → {address, kind, target, name, node}; GET /gateway
  builds from compute_routes (incl. config for static frontends + mesh for
  remote), so the table matches what Caddy actually does.
- app: Gateway panel shows Address · Kind · Target for every route (static
  frontends + host routes now appear); program detail shows 'Reachable at
  /foo/ · served (static)' for static frontends.
- cli: 'castle gateway status' prints the full route table.
- docs: registry.md/design.md/CLAUDE.md describe routes as one concept with
  three target kinds.

core 94 / cli 24 / api 52 green; ruff + app build clean; Caddyfile unchanged.
2026-06-14 17:48:35 -07:00
Paul Payne
7314b5cddb refactor(env): explicit defaults.env with placeholders; drop auto-injection
A service/job's env is now exactly its defaults.env — castle injects no hidden
convention vars. Values support ${port}/${data_dir}/${name} placeholders
(resolved at deploy, alongside ${secret:…}), so a program's own env var names
map to castle's computed values without hardcoding.

Why: the auto-injected <PREFIX>_PORT/<PREFIX>_DATA_DIR were a guess at the
program's env names — right for castle-scaffolded services, dead weight for
adopted ones (lakehouse carried two dead vars; notification-bridge/backup jobs
too). They also weren't visible in the config editor (computed at deploy), which
was the source of the 'four env vars but the UI shows none' mystery.

- core: resolve_env_vars gains a context (${port}/${data_dir}/${name});
  deploy builds env from defaults.env only — no <PREFIX>_* injection, no
  port_env. Removed the port_env field and the dead _env_prefix helper.
- cli: 'service/job create' gains repeatable --env KEY=VALUE (replaces
  --port-env); 'program create' scaffolds <PREFIX>_PORT/_DATA_DIR: ${…} for new
  daemons.
- app: removed the 'Port env' field; the Environment editor (defaults.env) is
  the single place, with a placeholder hint.
- live migration: central-context/castle-api/power-graph/protonmail mapped their
  real vars explicitly; lakehouse → just LAKEHOUSED_DAEMON_PORT: ${port}, data
  stays in ~/.lakehoused. Verified all services healthy on their ports, dead
  vars gone, zero failed units.
- docs: registry.md/design.md/stack guides + findings updated to the explicit
  model.

core 94 / cli 24 / api 52 green; ruff + app build clean.
2026-06-14 17:06:46 -07:00
Paul Payne
5c6d8ec6f1 refactor(cli): resource-first command surface
Names collide across resource types (a program and a service can share a name),
so the CLI no longer resolves bare names. Operations live under the resource
they act on; platform lifecycle and the cross-resource overview stay top-level.

  castle program  list|info|create|add|clone|delete|run|install|uninstall|
                  build|test|lint|format|type-check|check
  castle service  list|info|create|delete|deploy|enable|disable|start|stop|restart|logs
  castle job      <same verbs>   (create takes --schedule)
  castle gateway  start|stop|reload|status
  castle list | status | deploy [name]        # cross-cutting
  castle start | stop | restart               # whole platform (systemd verbs)

Key changes:
- info/delete/list/run are resource-scoped (no more cross-section name match).
- delete only removes the named resource's entry; program delete still blocks
  when a deployment references it.
- per-resource start/stop/restart (systemctl on the unit/timer); top-level
  start/stop/restart act on the whole platform.
- expose → 'service create' (more general: --program ref, --runner); gained
  'job create' (the CLI never had one).
- dropped up/down — bringing things online is the honest 'deploy && start'.

Docs (CLAUDE.md, registry.md, stack guides) and user-facing messages updated.
core 94 / cli 24 / api 52 green; ruff clean. Verified live: program/service/job
list+info scope correctly, service create+scoped-delete round-trips.
2026-06-14 16:25:34 -07:00
Paul Payne
dea585b15b Stage 3: App — program/deployment model & controls
- ProgramActions: install/uninstall (activate) shown only for tools and
  no-service frontends. Daemons activate via a service/job, so the program
  page no longer offers a misleading Install/Uninstall (which secretly
  enabled/disabled the service).
- Program page: new Deployments section listing the services/jobs that run
  the program, as links (a program → 0-N services/jobs).
- Service page: 'Runs' row from the run_target summary field (the old code
  dug into manifest.run, which was empty pre-Stage-1), a Host row for
  proxy_host, and a convenience link to the referenced program.
- lifecycle.activate on a daemon with no service now errors toward
  'castle expose' instead of silently installing its binary to PATH.

core 94 green; ruff + app build clean.
2026-06-14 15:37:20 -07:00
Paul Payne
4840cbe72a feat: castle UX fixes from the lakehouse adoption test
Driven by adopting lakehouse (an existing daemon that bundles its own SPA) —
see docs/findings-lakehouse.md.

#3 deploy reloads the gateway. castle deploy regenerated the Caddyfile but
   left the running Caddy on the old config, so new proxy routes were silently
   dead. Deploy now reloads the gateway when it's running.

#1 castle expose <program>. Turns an adopted program into a service
   (run/port/health/proxy/systemd) in one command — the missing
   daemon-to-service step. Flags: --port --health --path --run --port-env
   --host --no-proxy.

#2 port_env mapping. A service can declare expose.http.internal.port_env so
   castle sets the env var the program actually reads (e.g. lakehoused reads
   LAKEHOUSED_DAEMON_PORT, not castle's convention LAKEHOUSE_PORT). Castle now
   genuinely drives an adopted daemon's bind port.

#4a auto-base for react-vite. The build passes VITE_BASE = the gateway serve
   prefix (/<name>/, or / for castle-app); vite.config reads it. A castle-built
   frontend now works at its subpath with no hand-tuned base. castle-app's
   vite.config updated as the reference.

#4b host-based routing. proxy.caddy.host routes a whole hostname to the backend
   root via a host matcher inside the :9000 site, so a root-based SPA (base="/")
   serves unchanged — the fix for proxying an app castle can't rebuild. Caddyfile
   now emits 'auto_https off' (HTTP-only gateway on a non-standard port).

Nit: activate skips the editable reinstall when the tool is already on PATH.

Tests: core 94, cli 24, api 52; ruff + app build clean. Verified live: lakehouse
runs under systemd, API at /lakehouse, full UI (SPA boots) via host routing.
2026-06-14 13:16:34 -07:00
Paul Payne
a5dcb6b346 feat: CLI consistency pass — unified args, verbs, and status
#1 Positional dest unified to `name` across info + dev verbs (was `project`).
#2 install/uninstall help now states the activate semantics (tool→PATH,
   service/job→systemd, frontend→served) rather than 'to PATH'.
#4 `service status` (a plural op) moved to `services status`; the singular
   `service` group is enable/disable only.
#5 `list --behavior` now filters the program *catalog* by its real behavior
   field. behavior (what a program is) and service/job (how it's deployed) are
   separate axes: `list` shows a Programs catalog plus Services/Jobs deployment
   views; a behavior filter scopes the catalog only.
#6 `list` uses lifecycle.is_active uniformly for the status dot (tool on PATH /
   service running / job timer / static frontend served) instead of registry
   presence.
#7 New `castle restart <name>` — restart a service (unit) or job (timer).
#8 New `castle status` — unified services + jobs + programs activation view.
#9 New `castle format [name]` dev verb (ruff format / pnpm format); wired into
   stacks DEV_ACTIONS + handlers.
#10 New `castle up` — deploy + start everything in one shot.

Docs: CLAUDE.md command reference refreshed. Tests updated to the two-axis
list model (+ a daemon-behavior program fixture). core 92 · cli 24 · api 52
green; ruff clean.
2026-06-14 11:57:58 -07:00
Paul Payne
3f3b88f17b refactor: Align vocabulary — component → program / deployment
Canonical terms (see docs/registry.md glossary):
- program: any catalog entry (tool/daemon/frontend) — the software
- service / job: how a program is deployed (systemd .service / .timer)
- deployment: umbrella for the unified service+job+program view

Changes:
- core: ServiceSpec/JobSpec component: → program: (component accepted as
  back-compat validation_alias); DeployedComponent → Deployment.
- api: ComponentSummary/Detail → DeploymentSummary/Detail; GET /components
  → /deployments; GatewayRoute.component → program; GatewayInfo
  .component_count → deployment_count; ServiceActionResponse/action keys
  component → program.
- app: matching type renames, /components → /deployments hook, route
  /component/:name → /deployment/:name, GatewayRoute.program.
- docs: component-registry.md → registry.md (+ canonical glossary);
  CLAUDE.md endpoint list refreshed (drop removed /tools, add typed
  program/service/job + config routes).

Tests: core 92, cli 24, api 52 green; app build clean; ruff clean.
2026-06-14 11:05:22 -07:00
Paul Payne
482524bfe5 Phase 2b: serve static frontends in place from repo dist
The Caddyfile generator now emits static-frontend routes that root directly at
<source>/<build.outputs[0]> (e.g. /data/repos/castle/app/dist) instead of a copy
under ~/.castle/artifacts/content. Removes _copy_app_static and the central
content-dir staging entirely; rebuilds are live without a copy step.

- caddyfile.py: manifest-driven static routes (castle-app at /, others at /<name>)
- ReactViteHandler.install just builds in place (no copy); uninstall is a no-op
- is_active(static frontend) = repo dist exists
- test isolation updated for the generator's config use

Dashboard + power-graph-app verified serving from repo dist; 168 tests pass.
2026-06-13 18:16:09 -07:00
Paul Payne
2953aea548 Runtime half: convergent deploy + unified active lifecycle
Convergent deploy (prefix-based prune): castle deploy now removes orphaned
castle-* systemd units/timers no longer in the registry. Full deploy only;
partial (--name) deploys preserve siblings. Fixes the orphaned-timer / stale-
path drift class.

Container runner: derive --name from the service name (castle-<name>) instead
of the image name; skip the <PREFIX>_DATA_DIR env injection for containers
(avoids colliding with image env namespaces like NEO4J_*).

Unified active lifecycle (core/lifecycle.py): install/uninstall keep their
names but become activate/deactivate, dispatching by behavior — tool→PATH,
daemon/self-serving-frontend/job→systemd, static frontend→served. is_active
reports a uniform state (PATH-independent for tools). The CLI install/uninstall
and service enable/disable route through the extracted core.

API + dashboard: surface a uniform `active` state on programs alongside
`installed`; ProgramDetail shows active/inactive.

Tests: test_deploy_prune (6), test_lifecycle (5). 168 tests pass; lint clean.
2026-06-13 18:08:54 -07:00
Paul Payne
400e0b253b Less stack-centric and location-centric model. 2026-06-13 17:26:49 -07:00
Paul Payne
74a902ee62 Clean up. 2026-06-13 12:24:41 -07:00
payneio
691216b04e refactor: Extract deploy logic to castle-core and add deploy API endpoint
- Extract all deploy logic from CLI into castle-core/deploy.py as a reusable deploy() function
  * Resolves services/jobs to DeployedComponents
  * Generates systemd units and Caddyfile
  * Copies frontend assets and reloads systemd
  * Returns DeployResult with deployed count and messages
- Simplify CLI deploy command to a thin wrapper calling castle_core.deploy()
- Add POST /deploy endpoint to castle-api for remote deployment management
  * Supports optional {name} body param to deploy one or all services
  * Returns deployment result as JSON
- Register deploy router in castle-api main.py
- Update README with new deploy endpoint documentation

This enables full remote management: build, test, lint, and deploy any castle
node over HTTP (e.g. POST http://node:9000/api/deploy)
2026-04-27 22:01:39 -07:00
payneio
a553525c71 feat: Add support for remote services
- Added base_url field to DeployedComponent for external URL proxying
- Updated Caddyfile generator to proxy to base_url when present instead of requiring local port
- Added runner: remote handling in deploy command (returns empty run_cmd, marks as unmanaged)
- Reformatted long ternary expression in registry.py for readability

This enables castle to manage services that proxy to external endpoints without requiring a local process.
2026-04-27 20:48:25 -07:00
Paul Payne
01a09d6a61 feat: Implement unit specification and expansion logic in configuration 2026-04-08 16:57:19 -07:00
Paul Payne
8be129259c refactor: Restructure ~/.castle/ as the instance directory
Move all instance state into ~/.castle/ with clear separation:
- code/        — user project source (was components/)
- artifacts/   — generated specs, built content (was generated/, static/)
- data/        — service runtime data (was /data/castle/)
- secrets/     — credentials

castle.yaml moves to ~/.castle/castle.yaml as the canonical location.
Source paths use repo: prefix for git repo programs (castle-api, app)
and relative paths for user projects (code/central-context).

Add idempotent install.sh for bootstrapping infrastructure (Docker,
Caddy, MQTT, Postgres, Neo4j) with interactive service detection.

Remove components/ from repo (now in ~/.castle/code/), deinit
submodules, remove .gitmodules.
2026-03-09 22:03:37 -07:00
Paul Payne
99e70f8543 install_extras 2026-03-05 10:01:26 -08:00
Paul Payne
efab2a7893 refactor: Update component specifications to use 'program' and 'behavior' attributes, enhancing clarity and consistency across the application 2026-02-23 22:56:18 -08:00
Paul Payne
0d36e4f72a Refactor component terminology to programs in config and manifest
- Updated the terminology from "components" to "programs" across the codebase, including in config loading, saving, and manifest specifications.
- Introduced a new `stacks.py` file to handle lifecycle actions for development stacks, implementing handlers for Python and React Vite stacks.
- Adjusted tests to reflect the new program structure and ensure proper functionality.
- Revised documentation to align with the new terminology and structure, ensuring clarity on the purpose and configuration of programs, services, and jobs.
2026-02-23 22:09:41 -08:00
Paul Payne
56b9c8ddf1 refactor: Update component structure to use behavior and stack attributes, enhancing clarity and functionality across services, tools, and frontends 2026-02-23 16:32:55 -08:00
Paul Payne
3343e955fd feat: Implement multi-node support with MQTT and mDNS for service discovery and coordination 2026-02-23 02:30:12 -08:00