From 74a902ee62958095ca6a94ab4876162be67852c4 Mon Sep 17 00:00:00 2001 From: Paul Payne Date: Sat, 13 Jun 2026 12:24:41 -0700 Subject: [PATCH] Clean up. --- CLAUDE.md | 24 +++++--- README.md | 6 ++ cli/src/castle_cli/commands/create.py | 12 ++-- cli/tests/test_create.py | 4 +- core/src/castle_core/config.py | 29 ++++++++- core/src/castle_core/deploy.py | 16 +++-- docs/component-registry.md | 87 ++++++++++++++++++++++----- docs/design.md | 82 ++++++++++++++----------- docs/stacks/python-cli.md | 12 ++-- docs/stacks/python-fastapi.md | 7 ++- docs/stacks/react-vite.md | 29 ++++++--- 11 files changed, 217 insertions(+), 91 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8f53aae..3974f5b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -37,18 +37,19 @@ When creating a new service, tool, or frontend, follow the detailed guides: ```bash # Daemon (python-fastapi) castle create my-service --stack python-fastapi --description "Does something" -cd programs/my-service && uv sync +cd ~/.castle/code/my-service && uv sync uv run my-service # starts on auto-assigned port castle service enable my-service # register with systemd castle gateway reload # update reverse proxy routes # Tool (python-cli) castle create my-tool --stack python-cli --description "Does something" -cd programs/my-tool && uv sync +cd ~/.castle/code/my-tool && uv sync ``` -The `castle create` command scaffolds the project under `programs/`, -generates a CLAUDE.md, and registers it in `castle.yaml`. +The `castle create` command scaffolds the project under `~/.castle/code/`, +generates a CLAUDE.md, and registers it in `castle.yaml` with +`source: code/`. ## Castle CLI @@ -63,7 +64,6 @@ castle create --stack python-fastapi # Scaffold new project castle deploy [name] # Deploy to runtime (registry + systemd + Caddyfile) castle test [project] # Run tests (one or all) castle lint [project] # Run linter (one or all) -castle sync # Update submodules + uv sync all castle run # Run service in foreground castle logs [-f] [-n 50] # View service/job logs castle tool list # List all tools @@ -76,8 +76,13 @@ castle services start|stop # Start/stop everything ## Infrastructure +Castle uses two roots, each overridable by an env var: `CASTLE_HOME` (config, +code, artifacts, secrets; default `~/.castle`) and `CASTLE_DATA_DIR` (program +data I/O on a dedicated volume; default `/data/castle`). Paths below use +`$CASTLE_HOME` and `$CASTLE_DATA_DIR` accordingly. + - **Gateway**: Caddy reverse proxy at port 9000, config generated from `castle.yaml` - into `~/.castle/generated/Caddyfile`. Dashboard served at root. + into `$CASTLE_HOME/artifacts/specs/Caddyfile`. Dashboard served at root. - **Systemd**: User units generated under `~/.config/systemd/user/castle-*.service`. Use drop-in overrides (`*.service.d/*.conf`) for extra env vars that `castle deploy` shouldn't overwrite (e.g., `CASTLE_API_MQTT_ENABLED`). @@ -85,9 +90,10 @@ castle services start|stop # Start/stop everything due to rootless podman UID mapping issues). Deploy resolves the runtime via `shutil.which("docker")`. - **MQTT**: Mosquitto broker runs as `castle-mqtt` (Docker container on port 1883). - Data in `/data/castle/castle-mqtt/`, config in `/data/castle/castle-mqtt/config/`. -- **Data**: Service data lives in `/data/castle//`, passed via env var. -- **Secrets**: `~/.castle/secrets/` — never in project directories. + Data in `$CASTLE_DATA_DIR/castle-mqtt/`, config in `$CASTLE_DATA_DIR/castle-mqtt/config/`. +- **Data**: Service data lives in `$CASTLE_DATA_DIR//` (default + `/data/castle//`), passed via the generated `_DATA_DIR` env var. +- **Secrets**: `$CASTLE_HOME/secrets/` — never in project directories. ## API Endpoints (castle-api, port 9020) diff --git a/README.md b/README.md index 74aa8e1..5e6e2b0 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,12 @@ A personal software platform. Castle manages independent services, tools, and frontends from a single CLI, with a unified gateway, systemd integration, and a web dashboard. +Historically, applications have been developed by third parties, distributed through app stores, and installed on user devices. + +With the advent of AI-assisted software development, users can write the software they need directly, eliminating the need for packaging and distribution. This makes many classes of software simpler, too. Oftentimes all a user needs are simple scripts or configurations of existing tools. But no matter how simple your script or application, it needed to be tailored and packaged for specific distribution channels. Castle provides a unified environment for developing, managing, deploying, and advertising these simple applications. + +Castle _stacks_ are pre-configured development environments that provide a starting point for building applications. They include everything needed to get started, from the programming language and framework to the necessary dependencies and tools. This is a design intended for coding assistants to generate castle programs with a level of consistency and to ensure that they are properly configured and ready to use with Castle. If your coding assistant knows about Castle, it can help you create and manage your custom applications more efficiently. + ## Quick Start ```bash diff --git a/cli/src/castle_cli/commands/create.py b/cli/src/castle_cli/commands/create.py index e57963d..7d6920b 100644 --- a/cli/src/castle_cli/commands/create.py +++ b/cli/src/castle_cli/commands/create.py @@ -53,11 +53,11 @@ def run_create(args: argparse.Namespace) -> int: print(f"Error: '{name}' already exists in castle.yaml") return 1 - programs_dir = config.root / "programs" - programs_dir.mkdir(exist_ok=True) - project_dir = programs_dir / name + code_dir = config.root / "code" + code_dir.mkdir(exist_ok=True) + project_dir = code_dir / name if project_dir.exists(): - print(f"Error: directory 'programs/{name}' already exists") + print(f"Error: directory 'code/{name}' already exists") return 1 # Determine port for daemons @@ -82,7 +82,7 @@ def run_create(args: argparse.Namespace) -> int: config.programs[name] = ProgramSpec( id=name, description=args.description or f"A castle {stack} program", - source=f"programs/{name}", + source=f"code/{name}", stack=stack, behavior=behavior, ) @@ -108,7 +108,7 @@ def run_create(args: argparse.Namespace) -> int: print(f" Port: {port}") print(" Registered in castle.yaml") print("\nNext steps:") - print(f" cd programs/{name}") + print(f" cd ~/.castle/code/{name}") print(" uv sync") if behavior == "daemon": print(f" uv run {name} # starts on port {port}") diff --git a/cli/tests/test_create.py b/cli/tests/test_create.py index 4390004..74e6e29 100644 --- a/cli/tests/test_create.py +++ b/cli/tests/test_create.py @@ -32,7 +32,7 @@ class TestCreateCommand: result = run_create(args) assert result == 0 - project_dir = castle_root / "programs" / "my-api" + project_dir = castle_root / "code" / "my-api" assert project_dir.exists() assert (project_dir / "pyproject.toml").exists() assert (project_dir / "src" / "my_api" / "main.py").exists() @@ -64,7 +64,7 @@ class TestCreateCommand: result = run_create(args) assert result == 0 - project_dir = castle_root / "programs" / "my-tool2" + project_dir = castle_root / "code" / "my-tool2" assert project_dir.exists() assert (project_dir / "src" / "my_tool2" / "main.py").exists() assert (project_dir / "CLAUDE.md").exists() diff --git a/core/src/castle_core/config.py b/core/src/castle_core/config.py index 2264953..43cce5c 100644 --- a/core/src/castle_core/config.py +++ b/core/src/castle_core/config.py @@ -28,12 +28,37 @@ from castle_core.manifest import ( ) -CASTLE_HOME = Path.home() / ".castle" +def _resolve_castle_home() -> Path: + """Resolve the castle home directory (config, code, artifacts, secrets). + + Defaults to ~/.castle. Override with the CASTLE_HOME environment variable + (supports ~ and relative paths, which are expanded and made absolute). + """ + override = os.environ.get("CASTLE_HOME") + if override: + return Path(override).expanduser().resolve() + return Path.home() / ".castle" + + +def _resolve_data_dir() -> Path: + """Resolve the program data directory (service/program data I/O). + + Decoupled from CASTLE_HOME so bulk data can live on a dedicated volume. + Defaults to /data/castle. Override with the CASTLE_DATA_DIR environment + variable (supports ~ and relative paths, which are expanded and made absolute). + """ + override = os.environ.get("CASTLE_DATA_DIR") + if override: + return Path(override).expanduser().resolve() + return Path("/data/castle") + + +CASTLE_HOME = _resolve_castle_home() CODE_DIR = CASTLE_HOME / "code" ARTIFACTS_DIR = CASTLE_HOME / "artifacts" SPECS_DIR = ARTIFACTS_DIR / "specs" CONTENT_DIR = ARTIFACTS_DIR / "content" -DATA_DIR = CASTLE_HOME / "data" +DATA_DIR = _resolve_data_dir() SECRETS_DIR = CASTLE_HOME / "secrets" # Backwards-compat aliases (used by existing imports) diff --git a/core/src/castle_core/deploy.py b/core/src/castle_core/deploy.py index 16d59d6..9bc1003 100644 --- a/core/src/castle_core/deploy.py +++ b/core/src/castle_core/deploy.py @@ -146,7 +146,12 @@ def _build_deployed_service( ) -> DeployedComponent: """Build a DeployedComponent from a ServiceSpec.""" run = svc.run - prefix = _env_prefix(name) + # Env prefix and data dir are keyed by the program (component) the service + # runs, not the service name — that's the prefix the program's settings read + # (e.g. job `protonmail-sync` runs program `protonmail` → PROTONMAIL_DATA_DIR). + # Falls back to the service name when no component is referenced. + config_key = svc.component or name + prefix = _env_prefix(config_key) env: dict[str, str] = {} # Data dir convention (for managed services) @@ -154,7 +159,7 @@ def _build_deployed_service( if svc.manage and svc.manage.systemd and not svc.manage.systemd.enable: managed = False if managed: - env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / name) + env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / config_key) # Port convention (if exposed) port = None @@ -210,10 +215,13 @@ def _build_deployed_job( ) -> DeployedComponent: """Build a DeployedComponent from a JobSpec.""" run = job.run - prefix = _env_prefix(name) + # Keyed by the program (component) the job runs, not the job name — see + # _build_deployed_service for rationale. Falls back to the job name. + config_key = job.component or name + prefix = _env_prefix(config_key) env: dict[str, str] = {} - env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / name) + env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / config_key) if job.defaults and job.defaults.env: env.update(job.defaults.env) diff --git a/docs/component-registry.md b/docs/component-registry.md index 92a12c1..b25ad1d 100644 --- a/docs/component-registry.md +++ b/docs/component-registry.md @@ -6,7 +6,8 @@ architecture. ## castle.yaml -The single source of truth. Lives at the repo root. Three top-level sections: +The single source of truth. Lives at `~/.castle/castle.yaml`. Three top-level +sections: ```yaml gateway: @@ -15,7 +16,7 @@ gateway: programs: my-tool: description: Does something useful - source: programs/my-tool + source: code/my-tool stack: python-cli behavior: tool system_dependencies: [pandoc] @@ -76,10 +77,24 @@ Explicit declaration of how the program is used: ### `source` — Where the source lives ```yaml -source: programs/my-tool +source: code/my-tool # your programs, under ~/.castle/code/ +source: repo:castle-api # castle's own programs, inside the git repo ``` -Relative path from repo root to the project directory. +The `source` path is resolved one of three ways (`core/src/castle_core/config.py`): + +| `source:` value | Resolves to | Used for | +|-----------------|-------------|----------| +| `code/my-tool` *(relative)* | `$CASTLE_HOME/code/my-tool` | Your own programs | +| `repo:castle-api` | `/castle-api` (via the top-level `repo:` field) | Castle's built-in programs | +| `/abs/path` | as-is | Anything explicitly absolute | + +Relative sources resolve against the castle home (`$CASTLE_HOME`, default +`~/.castle` — see [Infrastructure paths](#infrastructure-paths)). Most programs +you create live under **`$CASTLE_HOME/code/`** and are recorded as +`source: code/`. Castle's own programs (CLI, core, castle-api, app) live in +the git repo and use the `repo:` prefix. When `castle deploy` writes `castle.yaml` +back out, it rewrites absolute paths into these relative forms. ### `stack` — Development toolchain @@ -215,28 +230,50 @@ Castle generates a systemd `.timer` file alongside the `.service` unit. Jobs also support `run` (required), `manage`, and `defaults` — same semantics as services. +## How programs get into `~/.castle/code/` + +Every program's source lives in one place — `~/.castle/code//`. It can +arrive there a few ways: + +1. **Scaffold a new one** with `castle create` — writes the project into + `~/.castle/code//` and registers it in `castle.yaml` with + `source: code/`. +2. **Clone an existing project** — `git clone ~/.castle/code/`, + then add a `programs:` entry pointing at `source: code/`. +3. **Drop files in directly** — a `code//` directory is just a working + tree; it doesn't have to be under version control to be registered and run. + +`~/.castle/` and `~/.castle/code/` are **not** themselves git repos, and there +are no submodules. Each program directory manages its own version control (or +none) independently — some are standalone git clones, others are just loose +files. + +Castle's own programs (CLI, core, castle-api, app) are the exception: they live +inside the castle git repo and are referenced with `source: repo:`. + ## Registering a new program ### Via `castle create` (recommended) ```bash -# Service — scaffolds project, assigns port, registers in castle.yaml +# Service — scaffolds into ~/.castle/code/, assigns port, registers in castle.yaml castle create my-service --stack python-fastapi --description "Does something" -# Tool — scaffolds under programs/ +# Tool — scaffolds into ~/.castle/code/ castle create my-tool --stack python-cli --description "Does something" ``` ### Manually -Add entries to the appropriate sections of `castle.yaml`: +Clone or create the project under `~/.castle/code/`, then add entries to the +appropriate sections of `castle.yaml`: ```yaml # Tool — only needs a program entry programs: my-tool: description: Does something useful - source: programs/my-tool + source: code/my-tool stack: python-cli behavior: tool @@ -244,7 +281,7 @@ programs: programs: my-service: description: Does something useful - source: programs/my-service + source: code/my-service stack: python-fastapi behavior: daemon @@ -270,7 +307,7 @@ services: ```bash castle create my-service --stack python-fastapi # 1. Scaffold + register -cd programs/my-service && uv sync # 2. Install deps +cd ~/.castle/code/my-service && uv sync # 2. Install deps # ... implement ... castle test my-service # 3. Run tests castle service enable my-service # 4. Generate systemd unit, start @@ -290,10 +327,10 @@ castle service disable my-service # Stop and remove systemd unit ```bash castle create my-tool --stack python-cli # 1. Scaffold + register -cd programs/my-tool && uv sync # 2. Install deps +cd ~/.castle/code/my-tool && uv sync # 2. Install deps # ... implement ... castle test my-tool # 3. Run tests -uv tool install --editable programs/my-tool/ # 4. Install to PATH +uv tool install --editable ~/.castle/code/my-tool/ # 4. Install to PATH ``` ### Job lifecycle @@ -317,15 +354,33 @@ and a `.timer` file. ## Infrastructure paths +Castle uses **two** independent roots, each overridable by an environment +variable (both expand `~` and resolve relative paths): + +- **`CASTLE_HOME`** — config, code, artifacts, and secrets. Default `~/.castle`. +- **`CASTLE_DATA_DIR`** — program/service data I/O (potentially large; lives on a + dedicated volume). Default `/data/castle`. Decoupled from `CASTLE_HOME` on + purpose so bulk data doesn't sit in the home directory. + | What | Where | |------|-------| -| Registry | `castle.yaml` (repo root) | -| Service data | `/data/castle//` | -| Secrets | `~/.castle/secrets/` | -| Generated Caddyfile | `~/.castle/generated/Caddyfile` | +| Castle home | `$CASTLE_HOME` (default `~/.castle`) | +| Registry | `$CASTLE_HOME/castle.yaml` | +| Program source (yours) | `$CASTLE_HOME/code//` | +| Program source (castle's) | `/` (via `source: repo:`) | +| Secrets | `$CASTLE_HOME/secrets/` | +| Generated Caddyfile | `$CASTLE_HOME/artifacts/specs/Caddyfile` | +| Built frontends | `$CASTLE_HOME/artifacts/content//` | +| **Service data** | **`$CASTLE_DATA_DIR//` (default `/data/castle//`)** | | Systemd units | `~/.config/systemd/user/castle-*.service` | | Systemd timers | `~/.config/systemd/user/castle-*.timer` | +Defined in `core/src/castle_core/config.py`: `CASTLE_HOME` (with derived +`CODE_DIR`, `SECRETS_DIR`, `SPECS_DIR`, `CONTENT_DIR`) and the independent +`DATA_DIR` (`CASTLE_DATA_DIR`). `castle deploy` passes each service its data +path via the generated `_DATA_DIR` env var. Systemd unit/timer paths are +fixed by systemd's user-unit convention. + ## Manifest models The Pydantic models live in `core/src/castle_core/manifest.py`. Key classes: diff --git a/docs/design.md b/docs/design.md index deae8db..571a4f4 100644 --- a/docs/design.md +++ b/docs/design.md @@ -31,8 +31,9 @@ is self-sufficient. The mesh is optional. operates above the line. 5. **Separate source from runtime.** The repo is for development. The - runtime lives in standard Unix locations (`~/.castle/`, `/data/castle/`, - systemd units). Nothing running should point into the source tree. + runtime lives in standard Unix locations (`$CASTLE_HOME`, default + `~/.castle/`, plus systemd units). Nothing running should point into the + source tree. 6. **AI-manageable.** The CLI and API exist so that AI assistants can discover, create, and manage components programmatically. Humans @@ -113,9 +114,10 @@ Manages running processes using standard Linux infrastructure. - TLS termination **Filesystem** handles storage: -- Service data: `/data/castle//` -- Secrets: `~/.castle/secrets/` -- Generated config: `~/.castle/generated/` +- Service data: `$CASTLE_DATA_DIR//` (default `/data/castle/`, on a + dedicated volume) +- Secrets: `$CASTLE_HOME/secrets/` (default `~/.castle/secrets/`) +- Generated config: `$CASTLE_HOME/artifacts/specs/` (Caddyfile, registry.yaml) Castle generates systemd unit files and Caddyfile entries from the registry. It doesn't run a daemon itself — it configures OS-level @@ -123,8 +125,8 @@ infrastructure and gets out of the way. Critically, the runtime layer references only standard paths — never the source tree. Systemd units point to installed binaries (on PATH -or in `~/.castle/bin/`), not to repo subdirectories. Caddy serves -from `~/.castle/static/`, not from build output directories in the repo. +or in `~/.local/bin/`), not to repo subdirectories. Caddy serves +from `$CASTLE_HOME/artifacts/content/`, not from build output directories in the repo. ### Registry Layer @@ -156,7 +158,7 @@ These map to two files: programs: central-context: description: Content storage API - source: components/central-context + source: code/central-context services: central-context: @@ -197,7 +199,7 @@ Convention-based env vars (`_PORT`, `_DATA_DIR`) are generated automatically during deploy. Only non-convention values need `defaults.env`. -**`~/.castle/registry.yaml`** (per-node, not in the repo) — Node config: +**`$CASTLE_HOME/artifacts/specs/registry.yaml`** (per-node, not in the repo, generated by `castle deploy`) — Node config: ```yaml node: @@ -209,7 +211,7 @@ deployed: runner: python run_cmd: [/home/user/.local/bin/central-context] env: - CENTRAL_CONTEXT_DATA_DIR: /data/castle/central-context + CENTRAL_CONTEXT_DATA_DIR: /home/user/.castle/data/central-context CENTRAL_CONTEXT_PORT: "9001" behavior: daemon stack: python-fastapi @@ -302,7 +304,7 @@ Local paths always take precedence. ### Dashboard The web dashboard (`castle-app`) is a React SPA served by Caddy from -`~/.castle/static/castle-app/`. It talks to `castle-api` via the +`$CASTLE_HOME/artifacts/content/castle-app/`. It talks to `castle-api` via the gateway proxy at `/api`. **Layout:** @@ -417,9 +419,10 @@ Each step is distinct: execute them implicitly. 2. **Install** — Makes the artifact available on the system. For tools: - `uv tool install` or compiled binary placed in `~/.castle/bin/`. For + `uv tool install` or compiled binary placed in `~/.local/bin/`. For services: same — the binary or entry point is on PATH or in a known - location. For frontends: built assets copied to `~/.castle/static/`. + location. For frontends: built assets copied to + `$CASTLE_HOME/artifacts/content/`. 3. **Deploy** — Materializes the runtime configuration. Reads the component spec, merges with node config, generates systemd units @@ -427,28 +430,32 @@ Each step is distinct: the source tree. Enables and starts services. For compiled languages (Rust, Go), build produces a standalone binary -and install is just placing it in `~/.castle/bin/`. For interpreted +and install is just placing it in `~/.local/bin/`. For interpreted languages (Python, Node), the runtime wrapper (uv, node) handles finding the installed artifact. ## Runtime Filesystem Layout -What already exists and what the target looks like: +Two roots, each overridable by an env var: `$CASTLE_HOME` (config, code, +artifacts, secrets; default `~/.castle`) and `$CASTLE_DATA_DIR` (bulk program +data; default `/data/castle`, kept on a dedicated volume): ``` -~/.castle/ ← Castle runtime home -├── registry.yaml ← Node config (what's deployed here) -├── generated/ ← Generated Caddyfile -│ └── Caddyfile -├── secrets/ ← Secret files (NAME → value) -│ └── PROTONMAIL_API_KEY -├── bin/ ← Compiled binaries, shims -│ └── my-go-tool -└── static/ ← Built frontend assets - └── castle-app/ - └── dist/ +$CASTLE_HOME/ ← Config & artifacts (default ~/.castle) +├── castle.yaml ← Registry spec (programs, services, jobs) +├── infra.conf ← Infrastructure install choices +├── code/ ← Program source (your programs) +│ └── / +├── artifacts/ +│ ├── specs/ ← Generated by `castle deploy` +│ │ ├── Caddyfile +│ │ └── registry.yaml ← Node config (what's deployed here) +│ └── content/ ← Built frontend assets +│ └── castle-app/ ← (index.html + assets, served at root) +└── secrets/ ← Secret files (NAME → value) + └── PROTONMAIL_API_KEY -/data/castle/ ← Persistent service data +$CASTLE_DATA_DIR/ ← Persistent service data (default /data/castle) └── / ~/.config/systemd/user/ ← Systemd units (standard location) @@ -458,8 +465,11 @@ What already exists and what the target looks like: └── ... ``` +Compiled-language tools (Rust, Go — planned) install their binaries to the +standard `~/.local/bin/`, not under `$CASTLE_HOME`. + Source (the repo) is referenced only during build and install. Everything -the runtime touches lives in `~/.castle/`, `/data/castle/`, or standard +the runtime touches lives under `$CASTLE_HOME`, `$CASTLE_DATA_DIR`, or standard systemd paths. ## OTP as Design Guide @@ -470,7 +480,7 @@ Castle's architecture parallels Erlang/OTP, mapped onto Unix: |-------------|------------------| | Application | Component (independent, self-contained) | | Application resource file | Component spec in `castle.yaml` | -| Release config (sys.config) | Node config in `~/.castle/registry.yaml` | +| Release config (sys.config) | Node config in `$CASTLE_HOME/artifacts/specs/registry.yaml` | | Release assembly | `castle deploy` (spec + node config → runtime) | | Supervisor | systemd (restart policies, ordering) | | Process | Running service/worker/job | @@ -508,17 +518,17 @@ What exists today: - **Three packages** — `castle-core` (models, config, generators), `castle-cli` (commands), `castle-api` (HTTP API) - **Source/runtime split** — `castle.yaml` (spec) → `castle deploy` → - `~/.castle/registry.yaml` (node config). Systemd units and Caddyfile - generated from registry with fully resolved paths. No repo references + `$CASTLE_HOME/artifacts/specs/registry.yaml` (node config). Systemd units and + Caddyfile generated from registry with fully resolved paths. No repo references in runtime artifacts. - **Convention-based env generation** — `castle deploy` auto-generates - `_DATA_DIR=/data/castle/` and `_PORT` from + `_DATA_DIR=$CASTLE_DATA_DIR/` and `_PORT` from the manifest. Only non-convention values need `defaults.env`. - **Gateway** — Caddy on port 9000, Caddyfile generated from registry - **API** — `castle-api` on port 9020, reads from registry (optional castle.yaml fallback for non-deployed components) - **Dashboard** — `castle-app` React/Vite frontend, static assets - served from `~/.castle/static/castle-app/` + served from `$CASTLE_HOME/artifacts/content/castle-app/` - **Services** — central-context (content storage), notification-bridge (desktop notification forwarder) - **Jobs** — protonmail (email sync every 5 min), backup-collect (nightly), @@ -531,7 +541,7 @@ What exists today: Opt-in via `CASTLE_API_MQTT_ENABLED` / `CASTLE_API_MDNS_ENABLED`. - **MQTT broker** — Mosquitto running as `castle-mqtt` Docker container on port 1883, managed by systemd. Config and data in - `/data/castle/castle-mqtt/`. + `$CASTLE_DATA_DIR/castle-mqtt/`. - **Node API** — `GET /mesh/status`, `GET /nodes`, `GET /nodes/{hostname}`. `GET /components?include_remote=true` for cross-node component listing. - **Gateway panel** — Dedicated UI showing route table, health per route, @@ -559,7 +569,7 @@ What doesn't exist yet: | Process supervision | systemd (user units) | Active | | HTTP routing | Caddy (port 9000) | Active | | Component specs | castle.yaml + Pydantic models | Active | -| Node config | `~/.castle/registry.yaml` | Active | +| Node config | `$CASTLE_HOME/artifacts/specs/registry.yaml` | Active | | CLI | castle (Python, uv) | Active | | API | castle-api (FastAPI) | Active | | Dashboard | castle-app (React, Vite, shadcn/ui) | Active | @@ -569,7 +579,7 @@ What doesn't exist yet: | Type checking | pyright (Python), tsc (TS) | Active | | Testing | pytest (Python), Vitest (TS) | Active | | Secrets | `~/.castle/secrets/` file-based | Active | -| Data storage | Filesystem (`/data/castle/`) | Active | +| Data storage | Filesystem (`$CASTLE_DATA_DIR/`, default `/data/castle/`) | Active | | Messaging | MQTT (paho-mqtt client, Mosquitto broker) | Active (opt-in) | | Node discovery | mDNS (python-zeroconf) + MQTT | Active (opt-in) | | Rust packaging | cargo | Planned | diff --git a/docs/stacks/python-cli.md b/docs/stacks/python-cli.md index b064e17..98ffae5 100644 --- a/docs/stacks/python-cli.md +++ b/docs/stacks/python-cli.md @@ -24,10 +24,10 @@ How to build CLI tools following Unix philosophy. ## Project layout -Each tool is an independent project under `components/` with its own `pyproject.toml`: +Each tool is an independent project under `~/.castle/code/` with its own `pyproject.toml`: ``` -components/my-tool/ +~/.castle/code/my-tool/ ├── src/my_tool/ │ ├── __init__.py │ └── main.py # Entry point @@ -37,13 +37,13 @@ components/my-tool/ └── CLAUDE.md ``` -Examples: `components/pdf2md/`, `components/gpt/`, `components/protonmail/` +Examples: `code/pdf2md/`, `code/gpt/`, `code/protonmail/` ## Creating a new tool ```bash castle create my-tool --stack python-cli --description "Does something" -cd components/my-tool && uv sync +cd ~/.castle/code/my-tool && uv sync ``` This scaffolds the project and registers it in `castle.yaml`. @@ -317,7 +317,7 @@ uv run ruff format . # Format programs: my-tool: description: Does something useful - source: components/my-tool + source: code/my-tool stack: python-cli behavior: tool ``` @@ -328,7 +328,7 @@ Tools with system dependencies declare them directly on the program: programs: pdf2md: description: Convert PDF files to Markdown - source: components/pdf2md + source: code/pdf2md stack: python-cli behavior: tool system_dependencies: [pandoc, poppler-utils] diff --git a/docs/stacks/python-fastapi.md b/docs/stacks/python-fastapi.md index a581a44..15999d8 100644 --- a/docs/stacks/python-fastapi.md +++ b/docs/stacks/python-fastapi.md @@ -102,7 +102,7 @@ Castle passes config via env vars in castle.yaml: programs: my-service: description: Does something useful - source: components/my-service + source: code/my-service stack: python-fastapi behavior: daemon @@ -292,12 +292,15 @@ Mapping convention: Castle services use filesystem storage with JSON metadata sidecars: ``` -/data/castle/my-service/ +$CASTLE_DATA_DIR/my-service/ # default /data/castle/my-service/ └── bucket/ ├── item-name └── item-name.meta.json ``` +The service receives this path via the generated `_DATA_DIR` env var — +it never hardcodes it. Use the `data_dir` setting from your config. + ```python # storage.py import hashlib diff --git a/docs/stacks/react-vite.md b/docs/stacks/react-vite.md index 735b9c7..bd35be4 100644 --- a/docs/stacks/react-vite.md +++ b/docs/stacks/react-vite.md @@ -20,7 +20,7 @@ How to build, serve, and manage web frontends as castle components. ```bash # Create the project -mkdir components/my-frontend && cd components/my-frontend +mkdir ~/.castle/code/my-frontend && cd ~/.castle/code/my-frontend pnpm create vite . --template react-ts # Core dependencies @@ -116,7 +116,7 @@ handles serving directly from the build output. programs: my-frontend: description: Web dashboard - source: components/my-frontend + source: code/my-frontend build: commands: - ["pnpm", "build"] @@ -124,7 +124,8 @@ programs: - dist/ ``` -For production, Caddy serves the static `dist/` output directly — no +For production, `castle deploy` copies the build output to +`~/.castle/artifacts/content//` and Caddy serves it from there — no Node process needed. See [Serving with Caddy](#serving-with-caddy) below. For development with Vite's dev server, add a service entry: @@ -148,12 +149,23 @@ See @docs/component-registry.md for the full registry reference. ## Serving with Caddy -For production, serve the static `dist/` output directly from Caddy rather than -running a Node process. The gateway Caddyfile can serve the files: +For production, the static build output is served by Caddy rather than a Node +process. You do **not** write this block by hand — `castle deploy` generates it. +The flow: + +1. `castle deploy` runs the program's `build.commands`, then copies each + `build.outputs` directory from `~/.castle/code//` into + `~/.castle/artifacts/content//` (`core/src/castle_core/deploy.py`, + `_copy_app_static`). +2. The Caddyfile generator scans `~/.castle/artifacts/content/`; any directory + containing an `index.html` is served as a SPA at a path prefix matching its + name. `castle-app` is special-cased to serve at the root `/`. + +The generated block looks like this (written to `~/.castle/artifacts/specs/Caddyfile`): ```caddyfile -handle_path /app/* { - root * /data/repos/castle/my-frontend/dist +handle_path /my-frontend/* { + root * /home/payne/.castle/artifacts/content/my-frontend try_files {path} /index.html file_server } @@ -161,7 +173,8 @@ handle_path /app/* { The `try_files {path} /index.html` directive is essential for SPA routing — it falls back to `index.html` for any path that doesn't match a static file, -letting React Router handle client-side routes. +letting React Router handle client-side routes. The serving prefix is derived +from the program name, not hand-configured. ## API integration