A bare `ssh host 'castle …'` / `'pnpm …'` couldn't find the toolchain: uv,
castle, pnpm, and nvm node live on the interactive PATH (~/.zsh.d/*.sh), which
non-interactive/non-login shells don't load. Add ensure_shell_path() to mirror
just those PATH entries into ~/.zshenv (which non-interactive zsh does load),
so every bootstrapped node can drive castle/pnpm over ssh. Idempotent, marker-
guarded, and skipped for non-zsh users.
The `castle` CLI and the `castle-api` service are two independent in-process
drivers of `castle_core`. Each resolved DATA_DIR/REPOS_DIR at import time from
its own process env (default /data/castle), persisted nowhere — so they silently
diverged, and `apply`/dashboard-apply crashed on a non-existent /data.
Make the loaded CastleConfig the single source of truth:
- Resolve data_dir/repos_dir only in load_config (env > castle.yaml > default),
anchored to the config root; drop the DATA_DIR/REPOS_DIR module globals and the
import-time file read entirely — no global twin that can disagree with the file.
- Thread config.data_dir/repos_dir through ensure_dirs, _env_context, tls_dir_for
(now unified — deploy no longer inlines the tls path), and create/add/clone.
- ensure_dirs raises an actionable CastleDirError instead of a bare PermissionError;
the api surfaces it as 422.
- doctor: "data dir writable" check + WARN when CASTLE_DATA_DIR/REPOS_DIR env
overrides the file (the one remaining cross-process divergence vector).
- install.sh persists data_dir/repos_dir into castle.yaml (idempotent, non-default).
- Docs: registry.md globals + AGENTS.md roots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Expose raw-TCP services (postgres) by name at <name>.<domain>:<port> and
cut the gateway's ACME wildcard cert onto them so they present a trusted
cert. Replaces the proxy/public booleans with a single `reach` enum
(off|internal|public); legacy input still parses via derived accessors.
Core:
- expose.tcp{port,tls} + TlsSpec(material: pair|combined|off, reload)
- tls.py: materialize cert files from Caddy's wildcard, reconcile on
renewal; `castle tls` CLI; optional cert_obtained events-exec hook
(gateway.cert_hook, gated so a plugin-less Caddy still parses)
- apply waits (bounded) for the wildcard to issue before materializing so
a fresh-node TLS service starts with its cert in place, then scopes
materialization to the deployments being applied
- reach: internal|public requires an expose block (no silent no-op);
public raw-TCP guarded until tunnel support lands
- chain.pem (${tls_ca}) is the issuer chain (leaf stripped), a real CA
bundle distinct from cert.pem
- one shared ${...} resolver (resolve_placeholders) for env and container
run fields; run.env now expands like volumes/args; $$ escapes a literal
- validate the generated Caddyfile before reloading the gateway so an
invalid config never degrades routing
Docs: docs/tcp-exposure.md. Tests cover reach/expose validation,
placeholder expansion + escape, issuer-chain material, TLS materialize.
Update the README, AGENTS.md, install.sh, and docs/ for the converge
model: one `castle apply [name] [--plan]` replaces `deploy && start` and
the per-kind enable/disable/install/start/stop verbs. CLI references now
list apply + restart (the imperative bounce) + logs; recipes use
`castle apply`; "turn it off" is documented as `enabled: false` + apply.
install.sh's next-steps collapse to a single `castle apply`.
Verified: no retired-verb command instructions remain in any markdown;
install.sh syntax OK; live `castle apply --plan` reports converged;
doctor all-green; core 129 / cli 31 / api 59 pass.
The README/docs explained how the system works but not how to get it up
and running, and there was no way to tell a healthy node from a
half-configured one. Two additions close that gap.
castle doctor — a read-only preflight/postflight check. It inspects
setup *and* runtime (CLI on PATH, uv, lingering; repo:, control plane
registered, dashboard built; gateway/api running + listening, specs
generated; and — under tls=acme — DNS-plugin Caddy, provider token,
:443 bind; tunnel config for public services) and, for anything not
green, prints the exact next command. Exit 0 when nothing failed
(warnings allowed), 1 otherwise, so it doubles as a scriptable smoke
test after install/deploy. Agents can lean on it the same way they use
`castle tool list`.
Docs — the quick start now leads with prerequisites and one command
(install.sh installs the CLI + registers the control plane, so the old
manual `uv tool install` step is gone), adds a `castle doctor` verify
step, and gains an "exposure ladder" table framing the three rungs
(localhost → LAN HTTPS → public) that link the deep DNS/TLS/tunnel docs.
install.sh's closing summary and the AGENTS.md CLI reference mention
doctor too.
A fresh clone + ./install.sh set up the infra (Docker, Caddy, MQTT,
Postgres) but left the registry empty — so `castle deploy && castle
start` brought up nothing. install.sh now gets a person all the way to
a running Castle:
- ensure_uv + install_cli: install uv and the `castle` CLI (editable
from ./cli) so the command exists on a fresh machine.
- seed_control_plane: register castle-gateway, castle-api, and the
castle dashboard from a new bootstrap/ seed dir, and add `repo:` to
castle.yaml so `source: repo:<name>` resolves. Never clobbers
existing entries.
- build_dashboard: build app/dist/ so the gateway has a UI to serve
(best-effort; warns if pnpm is absent).
The gateway seed uses a `__SPECS_DIR__` placeholder (no machine-specific
paths in the repo), substituted with this machine's specs dir at seed
time. .gitignore negates bootstrap/**/castle.yaml so the seeds are
tracked despite the user-registry ignore rule.
Verified: seeded config loads, derives kinds service/service/static,
and renders an off-mode Caddyfile serving the dashboard at :9000/ with
/api/* proxied to castle-api.
Neo4j was auto-provisioned for everyone; a graph DB isn't core, so don't foist it
on new users. Gate it behind maybe_setup_neo4j: enable with --with-neo4j, an
interactive yes, or a previous run's choice (recorded in infra.conf, so a decline
isn't re-prompted). MQTT and Postgres stay default-on. Existing installs with
NEO4J=container keep running it unchanged.
install.sh provisioned mqtt/postgres/neo4j data + their docker -v mounts under
~/.castle/data, but castle_core.config and every container deployment use
/data/castle — so a fresh install put the data in the wrong place. Resolve DATA_DIR
the same way config.py does (${CASTLE_DATA_DIR:-/data/castle}); since that's outside
$HOME, create it with a sudo+chown fallback when the parent isn't user-writable.
Also add REPOS_DIR and fix the summary (Code→Repos, 'castle services start'→'castle
start').
Moves the ~/.castle/{programs,deployments} dir creation and the minimal
castle.yaml (gateway.port: 9000) into install.sh's create_directories — seeded
only when absent, never clobbering an existing config. Drops the stale 'code/'
dir (program source lives in /data/repos now). README quick-start no longer
does the manual mkdir/printf; install.sh handles it.
internal-CA HTTPS forces every device to trust a private root, which breaks down
on some clients (Android browsers). acme mode instead obtains a real Let's Encrypt
wildcard cert (*.<domain>) via a DNS-01 challenge, so browsers trust it with zero
CA install — while services stay internal-only (no inbound exposure, no public A
records; only a transient _acme-challenge TXT touches the public zone, and LAN DNS
resolves the names to a private IP).
- Config: GatewayConfig/NodeConfig gain domain, acme_email, acme_dns_provider
(plumbed through load/save + registry + deploy, omitted-when-empty for
round-trip stability). tls stays the discriminator: off | internal | acme.
- Caddyfile generator: acme branch emits a global {email, acme_dns <provider>
{env.TOKEN}} block + one *.<domain> wildcard site with host matchers derived
from the service name (<name>.<domain>); path/static stay on :<port>. Shared
_host_matcher_block helper (reused by off-mode). CASTLE_ACME_STAGING=1 toggles
the LE staging CA; acme-without-domain falls back to HTTP with a warning.
- deploy(): _acme_preflight warns (never writes) when domain, the gateway-service
token env, or the CLOUDFLARE_API_TOKEN secret is missing.
- install.sh: opt-in --with-dns-plugin[=provider] builds a DNS-plugin Caddy via
xcaddy (pinned) to /usr/local/bin/caddy, which the gateway picks up on deploy.
- Tests: TestCaddyfileTlsAcme (global block, wildcard site, derived host matcher,
path routes on :port, staging toggle, no-domain fallback). Docs: gateway.tls
modes table + full acme/DNS-01 section (setup, wild-central LAN DNS, staging).
Move all instance state into ~/.castle/ with clear separation:
- code/ — user project source (was components/)
- artifacts/ — generated specs, built content (was generated/, static/)
- data/ — service runtime data (was /data/castle/)
- secrets/ — credentials
castle.yaml moves to ~/.castle/castle.yaml as the canonical location.
Source paths use repo: prefix for git repo programs (castle-api, app)
and relative paths for user projects (code/central-context).
Add idempotent install.sh for bootstrapping infrastructure (Docker,
Caddy, MQTT, Postgres, Neo4j) with interactive service detection.
Remove components/ from repo (now in ~/.castle/code/), deinit
submodules, remove .gitmodules.
