From 043b5028f0c3914be81d2bdab5c8b9e254df3c07 Mon Sep 17 00:00:00 2001 From: Paul Payne Date: Wed, 1 Jul 2026 15:39:39 -0700 Subject: [PATCH] Update AGENTS.md and CLAUDE.md for clarity; add developing-castle.md for development guidance --- AGENTS.md | 314 ++++++++++++++++++++++++++++++++++---- CLAUDE.md | 244 +---------------------------- docs/developing-castle.md | 70 +++++++++ 3 files changed, 354 insertions(+), 274 deletions(-) create mode 100644 docs/developing-castle.md diff --git a/AGENTS.md b/AGENTS.md index 464e419..6e478d0 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,48 +1,296 @@ # AGENTS.md — Castle -You are working in the **Castle** repository — Paul's personal software platform, -a monorepo of independent programs (services, tools, libraries, frontends) managed -by the `castle` CLI. +You are working in **Castle**, a personal software platform. Castle is a +monorepo of independent programs managed by the `castle` CLI. From this directory +you can **manage all the software on this box from source** — create programs, +deploy them as services, jobs, tools, or static frontends, route them through the +gateway, expose them over TLS or a public tunnel, and coordinate across nodes. -## What Castle does +This file is the canonical, agent-agnostic guide. For exhaustive detail every +section links to a doc under `docs/`. Read those before non-trivial changes. -Castle turns a source repo into something running on this node. The two halves: +--- -- **programs/** — the software catalog: what software exists (`source`, `stack`, - `build`, `system_dependencies`). -- **deployments/** — how a program is realized here, discriminated on its - **`manager`**: `systemd` (a service, or a job with a `schedule`), `caddy` (a - static frontend), `path` (a CLI on your PATH), or `none` (an external - reference). The human-facing **kind** (service/job/tool/static/reference) is - *derived* from the manager, never stored. +## 1. Mental model — two layers -## Using the `castle` CLI +Castle splits every piece of software into **what it is** and **how it runs here**: -It's resource-first — operations live under the resource they act on: +- **`programs/.yaml`** — the software *catalog*: source, stack, build, + system dependencies. "What software exists." +- **`deployments/.yaml`** — how a program is *realized on this node*. + Discriminated on **`manager`**: `systemd` | `caddy` | `path` | `none`. + +The human-facing **kind** is **derived** from the manager (+ `schedule`), never +stored: + +| manager | + schedule? | derived **kind** | what it is | +|---------|-------------|------------------|------------| +| `systemd` | no | **service** | a long-running daemon | +| `systemd` | yes | **job** | a scheduled task (`.timer`) | +| `path` | — | **tool** | a CLI installed on your `PATH` | +| `caddy` | — | **static** | a built frontend served by the gateway | +| `none` | — | **reference** | an external service on another node | + +A program may have **no** deployment (just source you develop), or one/more +deployments. Global settings live in **`castle.yaml`** (`gateway`, `repo`, +`agents`). Config root defaults to `~/.castle/`. + +**Prime directive:** regular programs must **never depend on castle**. They take +standard config (data dir, port, URLs) via **env vars**; only castle's own +programs (CLI, gateway, api) know castle internals. When you scaffold or adopt a +program, wire it with env vars — not castle imports. + +→ Full reference: **`docs/registry.md`** (castle.yaml structure, every field, +manifest models, lifecycle). Architecture rationale: **`docs/design.md`**. + +--- + +## 2. The `castle` CLI + +Resource-first: operations live under the resource they act on. Names can collide +across resource types, so the resource is explicit. ```bash -castle program create --stack -castle program add # adopt an existing repo -castle service create --program

--port -castle service deploy && castle service enable -castle gateway reload # update reverse-proxy routes -castle deploy && castle start # apply config, then start everything -castle status # unified status -castle list --json # all deployments, machine-readable +# Programs — the software catalog +castle program list [--kind service] [--stack python-cli] [--json] +castle program info [--json] +castle program create [--stack ...] [--description ...] # scaffold NEW code +castle program add [--name ...] # adopt EXISTING repo +castle program clone [name] # provision repo: source +castle program delete [--source] [-y] +castle program run [args...] # declared run command +castle program install|uninstall [name] # activate tools/statics +castle program build|test|lint|format|type-check|check [name] # dev verbs + +# Services — daemons (manager: systemd, no schedule) +castle service list|info [--json] +castle service create [--program P] [--port N] [--health ...] [--launcher ...] +castle service deploy # generate unit + gateway route +castle service enable|disable # systemd enable/disable (+ boot) +castle service start|stop|restart +castle service logs [-f] [-n 50] + +# Jobs — scheduled tasks (manager: systemd + schedule). Same verbs; create takes --schedule +castle job create [--program P] --schedule "0 2 * * *" [--launcher ...] +castle job ... + +# Tools — CLIs on PATH (manager: path) +castle tool list [--json] # each tool's executable + description + install state +castle tool info [--json] +castle tool install|uninstall + +# Platform-wide +castle list [--kind ...] [--stack ...] [--json] # all deployments +castle status # unified health/status +castle deploy [name] # apply config → units + Caddyfile +castle start | stop | restart # all services (+ gateway) +castle gateway start|stop|reload|status # the Caddy gateway ``` -Expose a service on the network by setting `proxy: true` (→ -`.`); add `public: true` for the Cloudflare tunnel. +`castle service`/`job`/`tool` are **views** over the one deployment set, filtered +by derived kind. Bringing everything online is the two honest steps +**`castle deploy && castle start`** (apply config, then start). -## Read this first +**Dev verbs resolve per-program:** a declared `commands:` entry (or `build:`) +overrides the program's stack default, else the stack handler, else the verb is +unavailable — so a wired-in repo with **no stack** works if it declares its +commands. All projects use **uv** (Python) / **pnpm** (frontends). -- **@CLAUDE.md** — full architecture, the complete `castle` verb list, registry - model, DNS/TLS, and code-style conventions. This is the authoritative guide. -- `docs/registry.md`, `docs/dns-and-tls.md`, and `docs/stacks/*.md` — deep dives. +--- -**Key principle:** regular programs must never depend on castle. They take -standard config (data dir, port, URLs) via env vars; only castle's own programs -(CLI, gateway, api) know castle internals. +## 3. Recipes -Broader context about this machine and Paul's other projects lives at -`/data/.lakehouse/AGENTS.md`. +### Create a new service (HTTP daemon) + +```bash +castle program create my-service --stack python-fastapi --description "Does X" +cd /data/repos/my-service && uv sync # implement it +castle program test my-service +castle service create my-service --program my-service --port 9001 +castle service deploy my-service && castle service enable my-service +castle gateway reload +``` + +The service reads its port/data dir from env vars that `deployments/my-service.yaml` +maps via placeholders (see §6). Stack guide: **`docs/stacks/python-fastapi.md`**. + +### Create a CLI tool + +```bash +castle program create my-tool --stack python-cli --description "Does Y" +cd /data/repos/my-tool && uv sync +castle tool install my-tool # uv tool install → on PATH +``` + +`castle tool list --json` is the machine-readable tool catalog (each tool's real +**executable**, which may differ from the program name). Stack: +**`docs/stacks/python-cli.md`**. + +### Create a scheduled job + +A job is a `manager: systemd` deployment with a `schedule` (cron). Generates a +`.service` (Type=oneshot) + a `.timer`. + +```bash +castle job create nightly --program my-tool --schedule "0 2 * * *" --launcher command +castle job deploy nightly && castle job enable nightly +``` + +### Create a static frontend + +```bash +# scaffold a Vite/React app under /data/repos/my-frontend (see docs/stacks/react-vite.md) +castle program build my-frontend # produces dist/ +# deployments/my-frontend.yaml → manager: caddy, root: dist +castle deploy && castle gateway reload # served at my-frontend. +``` + +The gateway serves the build **in place** from `/` — no copy, no +Node process. Stack: **`docs/stacks/react-vite.md`**. Database-backed apps on the +shared Supabase substrate: **`docs/stacks/supabase.md`**. + +### Adopt an existing repo (no stack needed) + +```bash +castle program add ~/projects/some-rust-tool # local path +castle program add https://github.com/me/widget.git --name widget +``` + +Castle detects dev-verb commands (pyproject→uv/ruff/pytest, Cargo.toml→cargo, …) +or you declare them under `commands:` in `programs/.yaml`. + +--- + +## 4. The gateway — routing & exposure + +The **Caddy gateway** (port 9000) is the single ingress. It's both a reverse +proxy (to local services) and a static file server (for built frontends). It maps +a public **address** (always a subdomain, `.`) to a **target**: + +| target kind | is | declared by | +|-------------|----|-------------| +| **proxy** | a local service on a port | a service's `proxy: true` | +| **static** | a built frontend's `dist/` | a `manager: caddy` deployment's `root:` | +| **remote** | a service on another node | mesh discovery | + +Exposure is a **checkbox** on a service: + +```yaml +proxy: true # expose at . +public: true # ALSO expose to the internet via Cloudflare tunnel (requires proxy) +``` + +- `proxy: false`/omitted → reachable only at its own `host:port`. +- The subdomain is always the **service name** (rename the service to change it). +- There are **no path-prefix routes** — a whole subdomain maps to the backend + root, so root-relative assets and `window.location` WebSocket URLs just work. + +Inspect routes: `castle gateway status` (or `GET /gateway`). Regenerate + reload: +`castle gateway reload`. → Field-level detail: **`docs/registry.md`**. + +--- + +## 5. DNS & TLS — making names resolve and be trusted + +Two orthogonal questions for `https://foo./` to work from a LAN browser: +**resolve** (DNS) and **trust** (TLS). Castle doesn't run DNS — you add one +**wildcard** record on the LAN's DNS server (usually the router): +`address=//` (dnsmasq) pinned with a DHCP reservation. + +`gateway.tls` in `castle.yaml` picks the trust mode: + +| `gateway.tls` | serves | client setup | when | +|---------------|--------|--------------|------| +| `off` *(default)* | plain HTTP on `:9000` | none | no HTTPS needed / no domain | +| `acme` | **real Let's Encrypt wildcard** `*.` via DNS-01 | **nothing** | you own a domain; any device | + +`acme` gets a publicly-trusted cert with **no CA to install**, while services stay +**internal-only** (DNS-01 writes a TXT to the public zone; only LAN DNS resolves +the names — the public zone has no A records). HTTPS also unlocks **secure +context** (`crypto.subtle`, service workers), which plain-HTTP LAN hosts lack. + +`acme` operational prerequisites (castle can't do these for you): +- **DNS-plugin Caddy** at `/usr/local/bin/caddy` — `./install.sh --with-dns-plugin=cloudflare`. +- **Provider token** stored as a secret and mapped into the gateway service env + (`CLOUDFLARE_API_TOKEN`); `castle deploy` warns if missing. +- **Bind :443/:80** — lower the floor once: `net.ipv4.ip_unprivileged_port_start=80` + in `/etc/sysctl.d/` (beats `setcap`, which `NoNewPrivileges` would void). +- **Stage first**: `CASTLE_ACME_STAGING=1` at deploy, verify issuance, then unset + and redeploy for a production cert. + +→ Full conceptual + step-by-step guide: **`docs/dns-and-tls.md`**. Read the actual +values for *this* node in `~/.castle/castle.yaml` (`gateway.domain`, `tls`, etc.). + +--- + +## 6. Environment, secrets, data, placeholders + +`defaults.env` in a deployment is the **single explicit source** of the env a +service/job runs with — castle injects nothing implicitly. Map the vars your +program reads to castle's computed values with placeholders: + +```yaml +expose: { http: { internal: { port: 9001 }, health_path: /health } } +defaults: + env: + MY_SERVICE_PORT: ${port} # = expose.http.internal.port + MY_SERVICE_DATA_DIR: ${data_dir} # = $CASTLE_DATA_DIR/ + PUBLIC_URL: ${public_url} # gateway origin (CORS/allowlists) + API_KEY: ${secret:MY_API_KEY} # reads ~/.castle/secrets/MY_API_KEY +``` + +| placeholder | expands to | +|-------------|-----------| +| `${port}` | the service's `expose.http.internal.port` | +| `${data_dir}` | `$CASTLE_DATA_DIR/` (default `/data/castle/`) | +| `${name}` | the deployment name | +| `${public_url}` | `https://.` under acme, else `http://localhost:` | +| `${secret:NAME}` | contents of `~/.castle/secrets/NAME` (mode 700) | + +**Never** put secrets in `castle.yaml` or project dirs — use `${secret:…}`. +Roots: **`CASTLE_HOME`** (config/code/artifacts/secrets, default `~/.castle`) and +**`CASTLE_DATA_DIR`** (program data, default `/data/castle`) — both env-overridable. + +--- + +## 7. Public exposure — Cloudflare tunnel + +`public: true` (requires `proxy: true`) projects a service to the internet at +`.` (a **separate** zone, so internal subdomain names +stay out of public DNS). `castle deploy` generates the cloudflared ingress from the +set of public services. Needs `gateway.public_domain` + `gateway.tunnel_id` set and +the `castle-tunnel` service running. → One-time setup: **`docs/tunnel-setup.md`**. + +Routing only moves bytes — it does **not** supply a backend's own auth. Do not make +a service public unless it authenticates or is meant to be open. + +--- + +## 8. Mesh — multi-node coordination (opt-in) + +Disabled by default. Enable via env on `castle-api`: +`CASTLE_API_MQTT_ENABLED=true` (+ `_MQTT_HOST`/`_PORT`), `CASTLE_API_MDNS_ENABLED=true`. +Nodes then advertise/discover over MQTT (Mosquitto, `castle-mqtt`) + mDNS; remote +deployments surface as `manager: none` **reference** kinds. Inspect: +`GET /mesh/status`, `GET /nodes`. Modules: `castle_api.mesh`, `.mqtt_client`, `.mdns`. + +--- + +## 9. Where to read more + +| Topic | Doc | +|-------|-----| +| Registry model, `castle.yaml`, every field, lifecycle | **`docs/registry.md`** | +| Why castle is shaped this way | **`docs/design.md`** | +| DNS resolution + the two TLS modes, acme recipe | **`docs/dns-and-tls.md`** | +| Public exposure (cloudflared) one-time setup | **`docs/tunnel-setup.md`** | +| Writing FastAPI services | **`docs/stacks/python-fastapi.md`** | +| Writing CLI tools | **`docs/stacks/python-cli.md`** | +| Writing React/Vite frontends | **`docs/stacks/react-vite.md`** | +| Database-backed apps (shared Supabase) | **`docs/stacks/supabase.md`** | +| **Developing Castle itself** (CLI/core/api/app, key files, endpoints) | **`docs/developing-castle.md`** | + +Castle's own programs live in this repo (`source: repo:` → cli, core, +castle-api, app). Your programs live under `/data/repos//` with an absolute +`source:`. When in doubt about *this* node's actual config, read +`~/.castle/castle.yaml` and `castle status`. diff --git a/CLAUDE.md b/CLAUDE.md index 06fb5b1..c22dce2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,243 +1,5 @@ # CLAUDE.md -This file provides guidance to Claude Code (claude.ai/code) when working -with code in this repository. - -## Overview - -Castle is a personal software platform — a monorepo of independent projects -(services, tools, libraries) managed by the `castle` CLI. The registry config is split into two directories under your config root: - -- **`programs/`** — Software catalog (source, stack, system_dependencies, build) -- **`deployments/`** — How a program is realized on this node (manager, run, expose, proxy, schedule, systemd) - -Each program has a **stack** (development toolchain: python-fastapi, -python-cli, react-vite). Each deployment is discriminated on its **`manager`** -(`systemd` | `caddy` | `path` | `none`) — who supervises or realizes it. The -human-facing **kind** (service, job, tool, static, reference) is *derived* from -the manager (+ `schedule`), never stored. Scheduling, systemd management, and -proxying are orthogonal operations. A deployment references a program via -`program:` for description fallthrough. - -**Key principle:** Regular projects must never depend on castle. They accept standard -configuration (data dir, port, URLs) via env vars. Only castle programs (CLI, gateway) -know about castle internals. - -## Programs: create new, or adopt existing - -A **program** is a source repo castle knows how to work with (dev verbs) and -deploy. There are two ways to get one: - -- **`castle program create`** — scaffold *new* code from a **stack** (a - creation-time template). Stacks are guidance for how new code is written; they - are NOT required at runtime. -- **`castle program add`** — adopt an *existing* repo (a local path or git URL), - wherever it lives. No stack needed — castle detects sensible dev-verb commands - (pyproject→uv/ruff/pytest, Cargo.toml→cargo, etc.) or you declare them. - -**Stacks vs programs** are decoupled: a stack seeds a new program's default -verb commands and scaffold, but a program stands on its own via its declared -`commands:` (and `source:`/`repo:`). A program may have no stack at all. - -Stack guides (for writing *new* code, AI-facing): - -- @docs/registry.md — Registry architecture, castle.yaml structure, lifecycle -- @docs/dns-and-tls.md — DNS + TLS approach: gateway routing, LAN name resolution, the three `gateway.tls` modes (off/internal/acme) -- @docs/stacks/python-fastapi.md — FastAPI service patterns (config, routes, models, testing) -- @docs/stacks/python-cli.md — CLI tool patterns (argparse, stdin/stdout, piping, testing) -- @docs/stacks/react-vite.md — React/Vite/TypeScript frontend patterns -- @docs/stacks/supabase.md — Database-backed apps on the shared Supabase substrate (migrations, RLS, edge functions) - -### Quick start - -```bash -# New daemon, scaffolded from a stack -castle program create my-service --stack python-fastapi --description "Does something" -cd /data/repos/my-service && uv sync -castle service create my-service --program my-service --port 9001 # declare the deployment (manager: systemd) -castle service deploy my-service && castle service enable my-service # unit + start -castle gateway reload # update reverse proxy routes - -# New tool from a stack -castle program create my-tool --stack python-cli --description "Does something" - -# Adopt an existing repo (no stack required) -castle program add ~/projects/some-rust-tool -castle program add https://github.com/me/widget.git --name widget -``` - -`castle program create` scaffolds under `/data/repos/` (override with -`CASTLE_REPOS_DIR`) and registers the program under `programs/.yaml` with an absolute -`source:`. `castle program add` registers an existing repo in place under `programs/.yaml` (or records -its `repo:` URL for `castle program clone`). A program's deployment (if any) is -recorded separately under `deployments/.yaml`. - -## Castle CLI - -The CLI lives in `cli/` and is installed via `uv tool install --editable cli/`. - -The CLI is **resource-first**: operations live under the resource they act on -(`program`, `service`, `job`, `gateway`). Names can collide across resource -types (a program and a service may share a name), so the resource is explicit. -Platform-wide lifecycle and the cross-resource overview are top-level. - -```bash -# Programs — the software catalog -castle program list [--kind service] [--stack python-cli] [--json] -castle program info [--json] -castle program create [--stack ...] [--description ...] # scaffold new -castle program add [--name ...] # adopt existing repo -castle program clone [name] # clone repo: source -castle program delete [--source] [-y] -castle program run [args...] # declared run command -castle program install|uninstall [name] # activate tools/statics -castle program build|test|lint|format|type-check|check [name] # dev verbs - -# Services — long-running daemons (deployments with manager: systemd, no schedule) -castle service list [--json] -castle service info [--json] -castle service create [--program P] [--port N] [--health ...] \ - [--path ...] [--host ...] [--port-env ...] [--launcher ...] -castle service delete [-y] -castle service deploy # generate unit + route -castle service enable|disable # systemd enable/disable -castle service start|stop|restart # systemd lifecycle (one) -castle service logs [-f] [-n 50] - -# Jobs — scheduled tasks (manager: systemd + schedule; same verbs, create takes --schedule) -castle job create [--program P] --schedule "0 2 * * *" [--launcher ...] -castle job ... - -# Tools — CLIs on your PATH (deployments with manager: path) -castle tool list [--json] # each tool's executable + description + install state -castle tool info [--json] # a tool's executable, description, source, installed? -castle tool install|uninstall - -# Platform-wide (top-level) -castle list [--kind ...] [--stack ...] [--json] # all deployments (services, jobs, tools, statics) -castle status # unified status -castle deploy [name] # apply config → units + Caddyfile -castle start | stop | restart # all services (+ gateway) -castle gateway start|stop|reload|status # the Caddy gateway -``` - -Bringing everything online is the two honest steps `castle deploy && castle -start` (apply config, then start) — there is no bundled `up`. - -`castle service`, `castle job`, and `castle tool` are **views** over the single -deployment set, filtered by derived kind (systemd-no-schedule → service, -systemd+schedule → job, `manager: path` → tool). `castle tool list --json` is the -machine-readable tool catalog for assistants — it surfaces each tool's actual -**executable** (which can differ from the program name — `litellm-intent-router` -installs `intent-router`), its description, and install state. Statics -(`manager: caddy`) surface via `castle list --kind static`. - -**Dev verbs** resolve per-program: a declared `commands:` entry (or `build:`) -overrides the stack default, falling back to the program's stack handler, else -the verb is unavailable. So a wired-in repo with **no `stack`** works as long as -it declares its commands. Tools are reached via `castle tool list`. - -## Infrastructure - -Castle uses two roots, each overridable by an env var: `CASTLE_HOME` (config, -code, artifacts, secrets; default `~/.castle`) and `CASTLE_DATA_DIR` (program -data I/O on a dedicated volume; default `/data/castle`). Paths below use -`$CASTLE_HOME` and `$CASTLE_DATA_DIR` accordingly. - -- **Gateway**: Caddy at port 9000 — both a reverse proxy (to local/remote - services) and a static file server (for built frontends, served in place from - `/`). Config generated from `castle.yaml` into - `$CASTLE_HOME/artifacts/specs/Caddyfile`. A route maps an address (path or - host) to a target of kind static/proxy/remote; `castle gateway status` lists - them. Dashboard (castle-app) served at root. -- **Systemd**: User units generated under `~/.config/systemd/user/castle-*.service`. - Use drop-in overrides (`*.service.d/*.conf`) for extra env vars that `castle deploy` - shouldn't overwrite (e.g., `CASTLE_API_MQTT_ENABLED`). -- **Containers**: `manager: systemd` deployments with `run: { launcher: container }` - use Docker (preferred on this system due to rootless podman UID mapping issues). - Deploy resolves the runtime via `shutil.which("docker")`. -- **MQTT**: Mosquitto broker runs as `castle-mqtt` (Docker container on port 1883). - Data in `$CASTLE_DATA_DIR/castle-mqtt/`, config in `$CASTLE_DATA_DIR/castle-mqtt/config/`. -- **Data**: Service data lives in `$CASTLE_DATA_DIR//` (default - `/data/castle//`), passed via the generated `_DATA_DIR` env var. -- **Secrets**: `$CASTLE_HOME/secrets/` — never in project directories. - -## API Endpoints (castle-api, port 9020) - -Core: -- `GET /health` — Health check -- `GET /stream` — SSE stream (health, service-action, mesh events) - -Deployments (the unified view of services + jobs + programs): -- `GET /deployments` — List all (add `?include_remote=true` for cross-node) -- `GET /deployments/{name}` — Deployment detail -- `GET /status` — Live health for all services - -Programs / Services / Jobs (typed views + editing): -- `GET /programs`, `GET /programs/{name}` — Program catalog (`?kind=tool` to filter) -- `POST /programs/{name}/{action}` — Run a program verb (install/uninstall/build/…) -- `PUT|DELETE /programs/{name}` — Edit or remove a program entry -- `GET /services`, `GET /services/{name}`, `PUT|DELETE /services/{name}` -- `GET /jobs`, `GET /jobs/{name}`, `PUT|DELETE /jobs/{name}` - -Config: -- `GET /` — Full registry; `PUT /` — Save registry -- `POST /apply` — Apply registry changes; `POST /deploy` — Deploy to runtime - -Gateway: -- `GET /gateway` — Gateway info + full route table (every route tagged kind=static|proxy|remote, with its address and target) -- `GET /gateway/caddyfile` — Generated Caddyfile content -- `POST /gateway/reload` — Regenerate Caddyfile and reload Caddy - -Mesh: -- `GET /mesh/status` — MQTT connection state, broker info, peer list -- `GET /nodes` — All known nodes (local + discovered remote) -- `GET /nodes/{hostname}` — Node detail with deployed components - -Service actions: -- `POST /services/{name}/{action}` — start/stop/restart -- `GET /services/{name}/unit` — Systemd unit content - -## Mesh Coordination (opt-in) - -Multi-node discovery is disabled by default. Enable via env vars: - -```bash -CASTLE_API_MQTT_ENABLED=true # Connect to MQTT broker -CASTLE_API_MQTT_HOST=localhost # Broker address -CASTLE_API_MQTT_PORT=1883 # Broker port -CASTLE_API_MDNS_ENABLED=true # Advertise/discover via mDNS -``` - -Key modules: `castle_api.mesh` (MeshStateManager), `castle_api.mqtt_client` -(paho-mqtt wrapper), `castle_api.mdns` (python-zeroconf wrapper). - -## Per-Project Commands - -All projects use **uv**. Commands run from each project's directory: - -```bash -uv sync # Install deps -uv run pytest tests/ -v # Run tests -uv run ruff check . # Lint -uv run ruff format . # Format -``` - -Services also support: `uv run ` to start. - -## Code Style - -- **Linting/formatting**: ruff — shared `ruff.toml` at repo root (100-char lines) -- **Type checking**: pyright — shared `pyrightconfig.json` at repo root -- **Testing**: pytest, pytest-asyncio for async tests -- **Python**: 3.13 for services, 3.11+ minimum for tools/libraries - -## Key Files - -- `castle.yaml` — Global settings (gateway, repo); resources live under `programs/` and `deployments/` -- `core/src/castle_core/manifest.py` — Pydantic models (ProgramSpec, DeploymentSpec, LaunchSpec) -- `core/src/castle_core/config.py` — Config loader (castle.yaml → CastleConfig) -- `core/src/castle_core/generators/` — Systemd unit and Caddyfile generation -- `cli/src/castle_cli/templates/scaffold.py` — Project scaffolding templates -- `pyproject.toml` — uv workspace root (core, cli, castle-api) -- `ruff.toml` / `pyrightconfig.json` — Shared lint/type config +See @AGENTS.md — the single, agent-agnostic guide for this repo: how to use +Castle (create/deploy/expose/manage software) and how to develop Castle's own +code. Everything lives there. diff --git a/docs/developing-castle.md b/docs/developing-castle.md new file mode 100644 index 0000000..288b00f --- /dev/null +++ b/docs/developing-castle.md @@ -0,0 +1,70 @@ +# Developing Castle + +How to work on **Castle's own code** — the CLI, core library, control-plane API, +and dashboard. For *using* Castle to manage software (create/deploy/expose +programs), see the operator guide in [`AGENTS.md`](../AGENTS.md). + +## The Castle monorepo + +Castle's own programs live in **this git repo** (`source: repo:`) — distinct +from the programs you manage, which live under `/data/repos//`: + +- `cli/` — the `castle` CLI (installed via `uv tool install --editable cli/`) +- `core/` — `castle_core`: manifest models, config loader, generators +- `castle-api/` — the FastAPI control-plane service (port 9020) +- `app/` — the dashboard frontend (React/Vite; program name `castle`, served at + `castle.`) + +The root `pyproject.toml` is the **uv workspace** (core, cli, castle-api). + +## Key files + +- `core/src/castle_core/manifest.py` — Pydantic models: `ProgramSpec`, + `DeploymentSpec` (discriminated union on `manager`), `LaunchSpec`, `AgentSpec`, … +- `core/src/castle_core/config.py` — config loader (`castle.yaml` + `programs/` + + `deployments/` → `CastleConfig`), the two roots, `source:` resolution +- `core/src/castle_core/generators/` — systemd unit/timer + Caddyfile generation +- `cli/src/castle_cli/` — resource-first CLI commands; `templates/scaffold.py` +- `castle-api/src/castle_api/` — routes, health polling, SSE `stream.py`, mesh, + and the agent terminal UX (`agents.py`, `pty_session.py`, `agent_sessions.py`, + `agent_registry.py`) +- `ruff.toml` / `pyrightconfig.json` — shared lint/type config + +## Commands (all projects use uv) + +```bash +uv sync +uv run pytest tests/ -v +uv run ruff check . && uv run ruff format . +uv run pyright +``` + +Running `uv sync` from a member subdir trims the shared workspace venv — resync +from the repo root to restore all members. Code style: **ruff** (100-char lines), +**pyright**, **pytest** + **pytest-asyncio**; Python **3.13** for services, **3.11+** +for tools/libraries. + +## castle-api endpoints (port 9020) + +- Core: `GET /health`, `GET /stream` (SSE: health, service-action, mesh) +- Deployments: `GET /deployments[/{name}]`, `GET /status` +- Catalog + editing: `GET /programs[/{name}]`, `POST /programs/{name}/{action}`, + `PUT|DELETE /programs/{name}`; likewise `/services`, `/jobs` +- Config: `GET|PUT /`, `POST /apply`, `POST /deploy` +- Gateway: `GET /gateway`, `GET /gateway/caddyfile`, `POST /gateway/reload` +- Mesh: `GET /mesh/status`, `GET /nodes[/{hostname}]` +- Agents (dashboard terminal UX): `GET /agents`, `GET /agents/sessions`, + `GET /agents/history`, `DELETE /agents/sessions/{id}`, `WS /agents/{name}/session` +- Service actions: `POST /services/{name}/{action}`, `GET /services/{name}/unit` + +## Infrastructure internals + +- Generated Caddyfile at `~/.castle/artifacts/specs/Caddyfile`; a **plugin Caddy** + at `/usr/local/bin/caddy` when `tls: acme`. +- Systemd user units at `~/.config/systemd/user/castle-*.service` (+ `.timer`); + the unit for program `X` is `castle-X.service`. Use drop-in `*.service.d/*.conf` + for extra env `castle deploy` shouldn't overwrite. +- The `container` launcher resolves docker via `shutil.which("docker")` (preferred + over rootless podman on this box). +- Service data at `$CASTLE_DATA_DIR//`; secrets at `~/.castle/secrets/` + (mode 700) — never in project directories.