Commit Graph

15 Commits

Author SHA1 Message Date
Paul Payne
770574f575 Flatten proxy config to a bool (drop the caddy key)
CaddySpec had only `enable` left (path_prefix/host removed; extra_snippets was
unused), so `proxy: { caddy: {} }` was three levels of nesting for a checkbox —
and it invited drift (supabase silently reverted to proxy.caddy.host). Replace it
with a plain `ServiceSpec.proxy: bool`.

- Model: remove ProxySpec/CaddySpec; `proxy: bool = False`. Readers simplified to
  `bool(svc.proxy)` (generator, castle-api summaries, CLI info).
- CLI create / dashboard editor + create form: write `proxy: true` / a checkbox.
- Configs migrated to `proxy: true` (and supabase's stale proxy.caddy.host block
  removed).
- Docs (registry, dns-and-tls, design, stack guides) + tests/fixtures updated;
  also fixed lingering path-model staleness in the stack guides.
2026-06-30 21:23:14 -07:00
Paul Payne
4ff6af8f4d Rename the dashboard program castle-app -> castle
The dashboard is now served at castle.<domain> (was castle-app.<domain>). Since
the subdomain is the program name, this is just a program rename plus the one
special-case constant.

- Generator: _DASHBOARD = "castle" (the :9000 redirect target and off-mode root).
- Program file renamed (runtime): programs/castle-app.yaml -> programs/castle.yaml
  (source unchanged: repo:app).
- Dashboard UI: ProgramDetail shows a frontend's subdomain (via subdomainUrl),
  dropping the old castle-app-at-root special-case; comment refs updated.
- Docs: registry.md/design.md/react-vite.md updated to the subdomain model and the
  `castle` name (react-vite.md also corrected — frontends serve at their subdomain
  root with VITE_BASE=/, and builds are manual via `castle program build`, not
  deploy).
2026-06-30 21:04:18 -07:00
Paul Payne
43ef1cc588 Subdomain-only gateway routing; drop path prefixes
Collapse the two proxy shapes (path_prefix + host) to a single checkbox: a
service is either port-only, or exposed at <service-name>.<gateway.domain>. Path
routes and their prefix-stripping foot-guns are gone; the subdomain is always the
service name (rename the service to change it).

- Schema: CaddySpec is just `enable` (presence = expose). service_proxy_targets
  returns (expose, port, base_url); Deployment carries `subdomain` (was
  proxy_path/proxy_host). Registry/deploy/mesh/CLI updated in lockstep.
- Generator: compute_routes emits one host route per exposed service/frontend
  (address = name). acme mode = one *.<domain> site with a reverse_proxy matcher
  per service + a file_server matcher per frontend; the :<port> site redirects to
  the dashboard. off mode = a HTTP control plane on :<port> (dashboard at / +
  /api → castle-api); other services are port-only.
- Dashboard/API: castle-app and castle-api are ordinary subdomains. The dashboard
  derives the API base at runtime (castle-api.<domain> on a subdomain, else /api)
  and calls it cross-origin — castle-api already allows CORS *. Frontends build
  with VITE_BASE=/ (served at their subdomain root).
- CLI: `service create` drops --path/--host; --no-proxy makes it port-only.
- Docs/tests reworked for the model; docs use generic placeholders only (no
  personal host/domain/IP details).
2026-06-30 20:47:03 -07:00
Paul Payne
a022a136fe Retire gateway.tls=internal; host routes use a bare subdomain label
acme mode fully replaced the internal-CA path (which required installing a private
CA on every device — the exact pain, esp. on Android, that acme avoids). Remove
`internal` entirely and simplify how services declare host routes.

- Generator (caddyfile.py): drop the `tls internal` branch — modes are now
  `off` | `acme`. In acme mode the published subdomain is the **first DNS label**
  of `proxy.caddy.host` (a bare `claw`, or a legacy `claw.civil.lan`, both →
  `claw.<domain>`), so services stay domain-agnostic and the declared value is
  authoritative again (no more silent service-name override). Shared
  `_host_matcher_block` reused by off-mode and the acme wildcard site.
- castle-api: delete `GET /gateway/ca.crt`, `_gateway_ca_pem`, `_ca_fingerprint`
  and the now-unused imports; drop `ca_fingerprint` from `GatewayInfo` (keep
  `tls`).
- Dashboard: remove the CA-cert download button + unused imports; drop
  `ca_fingerprint` from the `GatewayInfo` type.
- Tests: replace TestCaddyfileTlsInternal with an off-mode class (keeps the
  runner-agnostic host-route coverage); acme tests assert first-label derivation
  incl. label-wins-over-service-name; drop the castle-api CA-endpoint test.
- Docs: registry.md + dns-and-tls.md — two-mode tables (off|acme), remove the
  internal sections/CA-download, document the bare-label host convention; note a
  domain-less node stays on `off`.
2026-06-30 19:55:17 -07:00
Paul Payne
121f970f14 Add gateway.tls=acme: Let's Encrypt wildcard cert via DNS-01
internal-CA HTTPS forces every device to trust a private root, which breaks down
on some clients (Android browsers). acme mode instead obtains a real Let's Encrypt
wildcard cert (*.<domain>) via a DNS-01 challenge, so browsers trust it with zero
CA install — while services stay internal-only (no inbound exposure, no public A
records; only a transient _acme-challenge TXT touches the public zone, and LAN DNS
resolves the names to a private IP).

- Config: GatewayConfig/NodeConfig gain domain, acme_email, acme_dns_provider
  (plumbed through load/save + registry + deploy, omitted-when-empty for
  round-trip stability). tls stays the discriminator: off | internal | acme.
- Caddyfile generator: acme branch emits a global {email, acme_dns <provider>
  {env.TOKEN}} block + one *.<domain> wildcard site with host matchers derived
  from the service name (<name>.<domain>); path/static stay on :<port>. Shared
  _host_matcher_block helper (reused by off-mode). CASTLE_ACME_STAGING=1 toggles
  the LE staging CA; acme-without-domain falls back to HTTP with a warning.
- deploy(): _acme_preflight warns (never writes) when domain, the gateway-service
  token env, or the CLOUDFLARE_API_TOKEN secret is missing.
- install.sh: opt-in --with-dns-plugin[=provider] builds a DNS-plugin Caddy via
  xcaddy (pinned) to /usr/local/bin/caddy, which the gateway picks up on deploy.
- Tests: TestCaddyfileTlsAcme (global block, wildcard site, derived host matcher,
  path routes on :port, staging toggle, no-domain fallback). Docs: gateway.tls
  modes table + full acme/DNS-01 section (setup, wild-central LAN DNS, staging).
2026-06-30 18:45:13 -07:00
Paul Payne
b726e79c23 Add supabase stack + general compose runner
Adds "a stack whose default is a substrate": a shared self-hosted Supabase
backend plus a `--stack supabase` that scaffolds per-app projects (migrations +
edge functions + static UI) which deploy against it. Apps own their code and
stay repo-durable; only their rows/blobs live on the shared substrate.

- compose runner: new RunCompose supervises a multi-container stack as one
  systemd unit (ExecStart=`compose up`, generated ExecStop=`compose down`);
  secrets via EnvironmentFile. Reusable beyond Supabase. Deployment.stop_cmd
  carries the teardown command through the registry.
- supabase stack: CLI --stack choice, STACK_DEFAULTS→frontend with
  build.outputs=[public], _scaffold_supabase() (migrations/RLS/functions/
  static UI/app manifest), and SupabaseHandler with a forward-only idempotent
  migration runner (plan_migrations + psql build) and deno dev-verbs.
- docs: docs/stacks/supabase.md, registered in CLAUDE.md; compose runner
  documented in registry.md.
- tests: compose run/stop cmd, systemd ExecStop, compose host-route TLS,
  migration planner, supabase verb resolution, create --stack supabase.
2026-06-30 18:18:47 -07:00
Paul Payne
0d9c1ae6c2 Add gateway.tls=internal: HTTPS for host routes via Caddy's local CA
The gateway was HTTP-only by design, but a plain-HTTP origin is not a browser
"secure context", so apps that need WebCrypto (device identity, E2E crypto)
can't run when proxied — only localhost or HTTPS qualifies.

Add an opt-in `gateway.tls: internal`. When set, each host route
(proxy.caddy.host) is emitted as its own `<host> { tls internal; reverse_proxy }`
site, served over HTTPS by Caddy's local CA (Caddy binds :443 and redirects :80).
Path-prefix/static routes stay on the HTTP :<port> site — give a service a host
route to put it on HTTPS. Default (unset/off) keeps the existing HTTP-only
Caddyfile (single :<port> block, auto_https off, host matchers inline).

Plumbing mirrors gateway_port: GatewayConfig.tls (castle.yaml load/save) →
NodeConfig.gateway_tls (registry load/save, set in deploy) → the Caddyfile
generator. Tests cover both modes; docs/registry.md documents the option plus
the two operational requirements (binding 443/80 via the unprivileged-port
sysctl, and trusting Caddy's root CA on clients) and the host-route-vs-prefix
guidance that motivates it.
2026-06-29 08:13:17 -07:00
Paul Payne
b105487790 Fix gateway-route drift when a service is edited
Editing a service's routing (port, path_prefix, host) and then hitting
"apply" or "gateway reload" — rather than a full deploy — left the running
Caddyfile reflecting the old routing. The Caddyfile was generated from the
derived registry.yaml snapshot, which only `deploy()` rebuilds from
castle.yaml; apply/reload regenerated from that stale snapshot.

Make castle.yaml the single source of truth for local routes:

- caddyfile.py: new shared `service_proxy_targets()` derives a service's
  (proxy_path, proxy_host, port, base_url) from its spec. `compute_routes`
  now builds local routes from `config.services` when castle.yaml is
  loadable, falling back to `registry.deployed` only when it isn't. Since
  deploy/apply/reload all funnel through `compute_routes`, every regeneration
  path now tracks the spec. Mesh/remote and static-frontend routes unchanged.
- deploy.py: `_build_deployed_service` uses the same shared deriver, so the
  written registry and the computed routes can't diverge.
- config_editor.py: `/config/apply` now runs a full `deploy()` (rebuilding
  units + Caddyfile and reloading the gateway) before restarting services, so
  changed unit ExecStarts actually take effect. Scheduled jobs are no longer
  restarted on apply. Drops the now-unused direct caddy-reload path.
- Regression tests: a stale registry's old port/path loses to castle.yaml;
  registry fallback still works when config is unavailable.
2026-06-29 07:04:06 -07:00
Paul Payne
4840cbe72a feat: castle UX fixes from the lakehouse adoption test
Driven by adopting lakehouse (an existing daemon that bundles its own SPA) —
see docs/findings-lakehouse.md.

#3 deploy reloads the gateway. castle deploy regenerated the Caddyfile but
   left the running Caddy on the old config, so new proxy routes were silently
   dead. Deploy now reloads the gateway when it's running.

#1 castle expose <program>. Turns an adopted program into a service
   (run/port/health/proxy/systemd) in one command — the missing
   daemon-to-service step. Flags: --port --health --path --run --port-env
   --host --no-proxy.

#2 port_env mapping. A service can declare expose.http.internal.port_env so
   castle sets the env var the program actually reads (e.g. lakehoused reads
   LAKEHOUSED_DAEMON_PORT, not castle's convention LAKEHOUSE_PORT). Castle now
   genuinely drives an adopted daemon's bind port.

#4a auto-base for react-vite. The build passes VITE_BASE = the gateway serve
   prefix (/<name>/, or / for castle-app); vite.config reads it. A castle-built
   frontend now works at its subpath with no hand-tuned base. castle-app's
   vite.config updated as the reference.

#4b host-based routing. proxy.caddy.host routes a whole hostname to the backend
   root via a host matcher inside the :9000 site, so a root-based SPA (base="/")
   serves unchanged — the fix for proxying an app castle can't rebuild. Caddyfile
   now emits 'auto_https off' (HTTP-only gateway on a non-standard port).

Nit: activate skips the editable reinstall when the tool is already on PATH.

Tests: core 94, cli 24, api 52; ruff + app build clean. Verified live: lakehouse
runs under systemd, API at /lakehouse, full UI (SPA boots) via host routing.
2026-06-14 13:16:34 -07:00
Paul Payne
3f3b88f17b refactor: Align vocabulary — component → program / deployment
Canonical terms (see docs/registry.md glossary):
- program: any catalog entry (tool/daemon/frontend) — the software
- service / job: how a program is deployed (systemd .service / .timer)
- deployment: umbrella for the unified service+job+program view

Changes:
- core: ServiceSpec/JobSpec component: → program: (component accepted as
  back-compat validation_alias); DeployedComponent → Deployment.
- api: ComponentSummary/Detail → DeploymentSummary/Detail; GET /components
  → /deployments; GatewayRoute.component → program; GatewayInfo
  .component_count → deployment_count; ServiceActionResponse/action keys
  component → program.
- app: matching type renames, /components → /deployments hook, route
  /component/:name → /deployment/:name, GatewayRoute.program.
- docs: component-registry.md → registry.md (+ canonical glossary);
  CLAUDE.md endpoint list refreshed (drop removed /tools, add typed
  program/service/job + config routes).

Tests: core 92, cli 24, api 52 green; app build clean; ruff clean.
2026-06-14 11:05:22 -07:00
Paul Payne
482524bfe5 Phase 2b: serve static frontends in place from repo dist
The Caddyfile generator now emits static-frontend routes that root directly at
<source>/<build.outputs[0]> (e.g. /data/repos/castle/app/dist) instead of a copy
under ~/.castle/artifacts/content. Removes _copy_app_static and the central
content-dir staging entirely; rebuilds are live without a copy step.

- caddyfile.py: manifest-driven static routes (castle-app at /, others at /<name>)
- ReactViteHandler.install just builds in place (no copy); uninstall is a no-op
- is_active(static frontend) = repo dist exists
- test isolation updated for the generator's config use

Dashboard + power-graph-app verified serving from repo dist; 168 tests pass.
2026-06-13 18:16:09 -07:00
Paul Payne
8be129259c refactor: Restructure ~/.castle/ as the instance directory
Move all instance state into ~/.castle/ with clear separation:
- code/        — user project source (was components/)
- artifacts/   — generated specs, built content (was generated/, static/)
- data/        — service runtime data (was /data/castle/)
- secrets/     — credentials

castle.yaml moves to ~/.castle/castle.yaml as the canonical location.
Source paths use repo: prefix for git repo programs (castle-api, app)
and relative paths for user projects (code/central-context).

Add idempotent install.sh for bootstrapping infrastructure (Docker,
Caddy, MQTT, Postgres, Neo4j) with interactive service detection.

Remove components/ from repo (now in ~/.castle/code/), deinit
submodules, remove .gitmodules.
2026-03-09 22:03:37 -07:00
Paul Payne
3343e955fd feat: Implement multi-node support with MQTT and mDNS for service discovery and coordination 2026-02-23 02:30:12 -08:00
Paul Payne
72d35f2641 refactor: Enhance component info and list commands with deployed state from registry 2026-02-23 00:07:42 -08:00
Paul Payne
41d4c574cb refactor: Migrate CLI and API components to use castle-core for configuration and manifest handling 2026-02-22 22:54:38 -08:00