The `castle` CLI and the `castle-api` service are two independent in-process
drivers of `castle_core`. Each resolved DATA_DIR/REPOS_DIR at import time from
its own process env (default /data/castle), persisted nowhere — so they silently
diverged, and `apply`/dashboard-apply crashed on a non-existent /data.
Make the loaded CastleConfig the single source of truth:
- Resolve data_dir/repos_dir only in load_config (env > castle.yaml > default),
anchored to the config root; drop the DATA_DIR/REPOS_DIR module globals and the
import-time file read entirely — no global twin that can disagree with the file.
- Thread config.data_dir/repos_dir through ensure_dirs, _env_context, tls_dir_for
(now unified — deploy no longer inlines the tls path), and create/add/clone.
- ensure_dirs raises an actionable CastleDirError instead of a bare PermissionError;
the api surfaces it as 422.
- doctor: "data dir writable" check + WARN when CASTLE_DATA_DIR/REPOS_DIR env
overrides the file (the one remaining cross-process divergence vector).
- install.sh persists data_dir/repos_dir into castle.yaml (idempotent, non-default).
- Docs: registry.md globals + AGENTS.md roots.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Model:
- Drop `requires` from ProgramSpec; requires is now deployment-only.
Collapse Requirement to {ref, bind} with kind defaulting to "deployment"
(drop the unused `version`). System-package preconditions stay on the
program as `system_dependencies`, synthesized into {kind: system} for the
functional check. relations.py/deploy.py read only the deployment's requires.
- create.py seeds a stack's substrate dependency onto the deployment, not the
program. Docs + tests updated.
UI:
- Add a `requires` editor (useRequires) on service, job, and static detail
pages — edit service→service deps (ref + optional bind env var).
- Restore the gateway route table "Open" column: a kind-aware in-app link to
each route's deployment.
- Reformat the reach control: name baked into each option
(localhost:PORT (local) / <name>.<domain> (internal) / <name>.<pubdomain> (public)).
- Services page: sort statics in with systemd services by name, add a
search + kind-filter bar mirroring the programs page.
Thread kind through every CLI resolver so a tool, service, and job can
share a name. Reads go through config.deployment(kind,name) /
deployments_named(name) / all_deployments(); writes target the per-kind
store (config.services/jobs/tools/statics). delete cascades over
(kind,name) tuples; run/info resolve via registry.named().
A stack's substrate dependency now becomes a real encoded `requires` at
`castle program create` time (STACK_REQUIRES), so the relationship graph shows it.
This keeps `stack` uncoupled from the runtime model — the stack declares the edge
once at creation; the graph only ever reads the encoded `requires`, never the stack.
Reconcile the running system to declared config in a single operation,
replacing the old deploy && start plus the per-kind enable/disable/
install/uninstall verbs. The mechanism varies by manager (systemd unit /
PATH install / gateway route); the verb never does.
Core (castle_core.deploy.apply):
- render (units, Caddyfile, tunnel) then reconcile: activate every
enabled deployment that's down, restart any whose unit bytes changed,
deactivate the disabled ones. Returns ApplyResult (the enacted diff).
- --plan computes the diff and touches nothing.
- restart-on-change is exact: _render_unit_files is the single source of
truth for unit bytes, used both to write units and to predict restarts,
so the plan can't drift from what deploy writes.
Desired state:
- DeploymentBase.enabled (default True) — the declarative on/off. apply
converges to it; a disabled deployment is defined but not running and
gets no gateway route (else it would 502). Registry omits enabled when
True, keeping existing registries byte-identical.
CLI:
- add `castle apply [name] [--plan]`; keep `restart` as the one
imperative bounce, plus status/doctor/logs/gateway/program.
- retire deploy, start, stop (top-level); service/job deploy|enable|
disable|start|stop; tool install|uninstall; program install|uninstall.
- next-step strings + doctor hints now say `castle apply`.
Verified: core 129 + cli 31 green; live `castle apply` on a converged
node reports "nothing to do" with no spurious restarts; --plan shows an
empty diff; gateway + api stay up.
The README/docs explained how the system works but not how to get it up
and running, and there was no way to tell a healthy node from a
half-configured one. Two additions close that gap.
castle doctor — a read-only preflight/postflight check. It inspects
setup *and* runtime (CLI on PATH, uv, lingering; repo:, control plane
registered, dashboard built; gateway/api running + listening, specs
generated; and — under tls=acme — DNS-plugin Caddy, provider token,
:443 bind; tunnel config for public services) and, for anything not
green, prints the exact next command. Exit 0 when nothing failed
(warnings allowed), 1 otherwise, so it doubles as a scriptable smoke
test after install/deploy. Agents can lean on it the same way they use
`castle tool list`.
Docs — the quick start now leads with prerequisites and one command
(install.sh installs the CLI + registers the control plane, so the old
manual `uv tool install` step is gone), adds a `castle doctor` verify
step, and gains an "exposure ladder" table framing the three rungs
(localhost → LAN HTTPS → public) that link the deep DNS/TLS/tunnel docs.
install.sh's closing summary and the AGENTS.md CLI reference mention
doctor too.
The tools lens coding assistants need. 'castle tool list [--json]' lists tools
(programs with a manager:path deployment) surfacing each tool's actual EXECUTABLE
— resolved from pyproject [project.scripts], which can differ from the program
name (litellm-intent-router → intent-router) — plus description and install state.
'castle tool info <name> [--json]' details one; 'castle tool install|uninstall'.
--json is a clean machine-readable catalog for building context.
Sweep: main.py descriptions/help mention the tool lens; 'frontend'→'static' in
install help; add.py._detect drops the dead behavior return (add adopts source
only); stale comments/help fixed. Docs (CLAUDE.md/README/registry) document
'castle tool'. Tests: cli/tests/test_tool.py (list/info/json).
Suites: core 124, cli 29, castle-api 58.
A program has no single kind — it HAS deployments, each with its own kind (a
program can be a tool AND a job, e.g. protonmail). Remove program-level kind:
- core: drop ProgramSpec.kind; CastleConfig.kind_of → deployments_of(name) →
[(deployment-name, kind)]; tools property derives from a tool deployment.
- api: ProgramSummary drops kind/services/jobs → deployments: [{name, kind}];
/programs?kind= filters by deployment-kind membership; a program's legacy
DeploymentSummary carries kind=None.
- cli: list/info show a program's set of deployment kinds; --kind filters by
membership.
Also: Services page now covers statics — /services returns kind in
{service, static} (both are exposed, URL-reachable 'services', caddy vs systemd),
and ServiceSummary gains kind + manager to distinguish them.
Suites: core 124, cli 25, castle-api 58.
Replace the conflated `runner` axis with two orthogonal ones: `manager`
(systemd|caddy|path|none) — who supervises/realizes a deployment — and, for
systemd only, a nested `launcher` (python|command|container|compose|node) — how
the process starts. ServiceSpec and JobSpec collapse into one manager-
discriminated DeploymentSpec union (Systemd/Caddy/Path/Remote); the services/
and jobs/ config dirs collapse into one deployments/ dir. The human "kind"
(service|job|tool|static|reference) is fully derived (kind_for), never stored —
the frontend kind is renamed static. behavior is gone.
- core: DeploymentSpec union + LaunchSpec + kind_for; legacy-aware loader
normalizes old runner shapes; CastleConfig.deployments with derived
services/jobs/tools views; registry.Deployment carries manager/launcher/kind.
- cli: service/job/tool as filtered views + a deployment group; --behavior→--kind,
create --runner→--launcher; lifecycle dispatches over config.deployments.
- castle-api: /deployments primary with /services,/jobs as views; summaries
derive kind; PUT/DELETE /config/deployments/{name} (services/jobs aliased).
- app: KindBadge, frontend→static everywhere, pick-a-kind creation wizard,
per-kind config editors.
- docs: single deployments/ layout, manager/launcher, static kind throughout.
Live migration verified byte-identical: regenerated Caddyfile and every unit
ExecStart line unchanged, so nothing restarted. Suites: core 124, cli 25,
castle-api 55; dashboard build + type-check clean.
behavior (tool/daemon/frontend) is now a derived display label computed
from a program's deployments — path->tool, static->frontend, process->daemon
(behavior_for_runner). load_config populates it so every reader gets the live
value; _spec_to_yaml_dict excludes it so it is never written to disk.
_summary_from_service derives it from the runner instead of hardcoding daemon.
Fixtures gain path/python deployments so tool/daemon behaviors derive.
- manifest: RunPath (runner: path) — a CLI on PATH via `uv tool install`, manager
= path, no unit/route (like remote/static).
- lifecycle/service: a path deployment's start/stop = install/uninstall (routed
through activate/deactivate, which skip the reinstall when already on PATH).
`castle service start/stop <tool>` and status now dispatch by manager; status
shows the manager tag [systemd|caddy|path|none].
- config.tools() now derives from `runner: path` services, not the behavior label.
- castle-api: the "installed" flag derives from manager==path, not behavior==tool.
- create: a python-cli stack emits a `runner: path` service.
`behavior` no longer drives any runtime logic — it survives only as a display
label + `list --behavior` filter. Verified live: html2text uninstall→reinstall
round-trip via `castle service stop/start`.
- manifest: `manager_for(runner)` — the single source of truth mapping a runner to
its manager (systemd | caddy | path | none). deploy's `managed` now derives from it.
- lifecycle: is_active/activate/deactivate dispatch by manager instead of behavior
(removed the stale `_is_static_frontend`, which broke once frontends became
services). caddy → gateway reload; path → install/uninstall; none → no-op.
- cli/service: start/stop/restart and enable/disable route through the manager
(systemctl only for systemd; static reloads the gateway; remote is a no-op);
platform restart skips non-systemd runners.
- create: a frontend stack (react-vite/supabase) now emits a `runner: static`
service, not a bare frontend program.
The proxy->bool refactor (770574f) missed the shared CLI test fixture, which
still used proxy: {caddy: {path_prefix: ...}} — failing ~25 CLI tests with
'Input should be a valid boolean'. Set it to proxy: True.
Also drop the now-dead 'caddy' entry from _STRUCTURAL_KEYS (no nested caddy
key exists anymore) and refresh a stale proxy.caddy docstring reference.
Adds "a stack whose default is a substrate": a shared self-hosted Supabase
backend plus a `--stack supabase` that scaffolds per-app projects (migrations +
edge functions + static UI) which deploy against it. Apps own their code and
stay repo-durable; only their rows/blobs live on the shared substrate.
- compose runner: new RunCompose supervises a multi-container stack as one
systemd unit (ExecStart=`compose up`, generated ExecStop=`compose down`);
secrets via EnvironmentFile. Reusable beyond Supabase. Deployment.stop_cmd
carries the teardown command through the registry.
- supabase stack: CLI --stack choice, STACK_DEFAULTS→frontend with
build.outputs=[public], _scaffold_supabase() (migrations/RLS/functions/
static UI/app manifest), and SupabaseHandler with a forward-only idempotent
migration runner (plan_migrations + psql build) and deno dev-verbs.
- docs: docs/stacks/supabase.md, registered in CLAUDE.md; compose runner
documented in registry.md.
- tests: compose run/stop cmd, systemd ExecStop, compose host-route TLS,
migration planner, supabase verb resolution, create --stack supabase.
- Introduced `_write_castle_config` function to scatter nested config into separate YAML files for programs, services, and jobs.
- Updated `castle_root` fixture to utilize the new config writing method.
- Added tests for configuration aggregation and scattering in `test_health.py`.
- Removed legacy unit handling from `manifest.py` and refactored related code in `config.py`.
- Updated documentation to reflect new configuration directory structure.
- Removed obsolete unit tests related to unit expansion.
Names collide across resource types (a program and a service can share a name),
so the CLI no longer resolves bare names. Operations live under the resource
they act on; platform lifecycle and the cross-resource overview stay top-level.
castle program list|info|create|add|clone|delete|run|install|uninstall|
build|test|lint|format|type-check|check
castle service list|info|create|delete|deploy|enable|disable|start|stop|restart|logs
castle job <same verbs> (create takes --schedule)
castle gateway start|stop|reload|status
castle list | status | deploy [name] # cross-cutting
castle start | stop | restart # whole platform (systemd verbs)
Key changes:
- info/delete/list/run are resource-scoped (no more cross-section name match).
- delete only removes the named resource's entry; program delete still blocks
when a deployment references it.
- per-resource start/stop/restart (systemctl on the unit/timer); top-level
start/stop/restart act on the whole platform.
- expose → 'service create' (more general: --program ref, --runner); gained
'job create' (the CLI never had one).
- dropped up/down — bringing things online is the honest 'deploy && start'.
Docs (CLAUDE.md, registry.md, stack guides) and user-facing messages updated.
core 94 / cli 24 / api 52 green; ruff clean. Verified live: program/service/job
list+info scope correctly, service create+scoped-delete round-trips.
#1 Positional dest unified to `name` across info + dev verbs (was `project`).
#2 install/uninstall help now states the activate semantics (tool→PATH,
service/job→systemd, frontend→served) rather than 'to PATH'.
#4 `service status` (a plural op) moved to `services status`; the singular
`service` group is enable/disable only.
#5 `list --behavior` now filters the program *catalog* by its real behavior
field. behavior (what a program is) and service/job (how it's deployed) are
separate axes: `list` shows a Programs catalog plus Services/Jobs deployment
views; a behavior filter scopes the catalog only.
#6 `list` uses lifecycle.is_active uniformly for the status dot (tool on PATH /
service running / job timer / static frontend served) instead of registry
presence.
#7 New `castle restart <name>` — restart a service (unit) or job (timer).
#8 New `castle status` — unified services + jobs + programs activation view.
#9 New `castle format [name]` dev verb (ruff format / pnpm format); wired into
stacks DEV_ACTIONS + handlers.
#10 New `castle up` — deploy + start everything in one shot.
Docs: CLAUDE.md command reference refreshed. Tests updated to the two-axis
list model (+ a daemon-behavior program fixture). core 92 · cli 24 · api 52
green; ruff clean.
- Updated the terminology from "components" to "programs" across the codebase, including in config loading, saving, and manifest specifications.
- Introduced a new `stacks.py` file to handle lifecycle actions for development stacks, implementing handlers for Python and React Vite stacks.
- Adjusted tests to reflect the new program structure and ensure proper functionality.
- Revised documentation to align with the new terminology and structure, ensuring clarity on the purpose and configuration of programs, services, and jobs.
- Introduced category tools support in the `castle create` command.
- Added detailed guides for creating components in CLAUDE.md.
- Implemented new API endpoints for listing and retrieving tool details.
- Updated component and tool models to include additional metadata.
- Improved error handling and response structures in service actions.
- Enhanced documentation for component registry and web APIs.
- Introduced `uv.lock` for dependency management with various packages including `pytest`, `colorama`, and `pluggy`.
- Added `pyrightconfig.json` for Python type checking configuration.
- Expanded `recommendations.md` with detailed scaling recommendations and project management strategies.
- Created shared `ruff.toml` for consistent linting across projects.
- Developed `Castle Tools` with various utilities including Android backup, browser automation, document conversion, and search tools.
- Implemented `backup-collect` and `schedule` tools for system administration tasks.
- Enhanced `search` functionality with indexing and querying capabilities using Tantivy.
- Added comprehensive documentation for each tool, including usage examples and installation instructions.
