Clean up.

This commit is contained in:
2026-06-13 12:24:41 -07:00
parent 9fe95f6d1e
commit 74a902ee62
11 changed files with 217 additions and 91 deletions

View File

@@ -37,18 +37,19 @@ When creating a new service, tool, or frontend, follow the detailed guides:
```bash ```bash
# Daemon (python-fastapi) # Daemon (python-fastapi)
castle create my-service --stack python-fastapi --description "Does something" castle create my-service --stack python-fastapi --description "Does something"
cd programs/my-service && uv sync cd ~/.castle/code/my-service && uv sync
uv run my-service # starts on auto-assigned port uv run my-service # starts on auto-assigned port
castle service enable my-service # register with systemd castle service enable my-service # register with systemd
castle gateway reload # update reverse proxy routes castle gateway reload # update reverse proxy routes
# Tool (python-cli) # Tool (python-cli)
castle create my-tool --stack python-cli --description "Does something" castle create my-tool --stack python-cli --description "Does something"
cd programs/my-tool && uv sync cd ~/.castle/code/my-tool && uv sync
``` ```
The `castle create` command scaffolds the project under `programs/`, The `castle create` command scaffolds the project under `~/.castle/code/`,
generates a CLAUDE.md, and registers it in `castle.yaml`. generates a CLAUDE.md, and registers it in `castle.yaml` with
`source: code/<name>`.
## Castle CLI ## Castle CLI
@@ -63,7 +64,6 @@ castle create <name> --stack python-fastapi # Scaffold new project
castle deploy [name] # Deploy to runtime (registry + systemd + Caddyfile) castle deploy [name] # Deploy to runtime (registry + systemd + Caddyfile)
castle test [project] # Run tests (one or all) castle test [project] # Run tests (one or all)
castle lint [project] # Run linter (one or all) castle lint [project] # Run linter (one or all)
castle sync # Update submodules + uv sync all
castle run <name> # Run service in foreground castle run <name> # Run service in foreground
castle logs <name> [-f] [-n 50] # View service/job logs castle logs <name> [-f] [-n 50] # View service/job logs
castle tool list # List all tools castle tool list # List all tools
@@ -76,8 +76,13 @@ castle services start|stop # Start/stop everything
## Infrastructure ## Infrastructure
Castle uses two roots, each overridable by an env var: `CASTLE_HOME` (config,
code, artifacts, secrets; default `~/.castle`) and `CASTLE_DATA_DIR` (program
data I/O on a dedicated volume; default `/data/castle`). Paths below use
`$CASTLE_HOME` and `$CASTLE_DATA_DIR` accordingly.
- **Gateway**: Caddy reverse proxy at port 9000, config generated from `castle.yaml` - **Gateway**: Caddy reverse proxy at port 9000, config generated from `castle.yaml`
into `~/.castle/generated/Caddyfile`. Dashboard served at root. into `$CASTLE_HOME/artifacts/specs/Caddyfile`. Dashboard served at root.
- **Systemd**: User units generated under `~/.config/systemd/user/castle-*.service`. - **Systemd**: User units generated under `~/.config/systemd/user/castle-*.service`.
Use drop-in overrides (`*.service.d/*.conf`) for extra env vars that `castle deploy` Use drop-in overrides (`*.service.d/*.conf`) for extra env vars that `castle deploy`
shouldn't overwrite (e.g., `CASTLE_API_MQTT_ENABLED`). shouldn't overwrite (e.g., `CASTLE_API_MQTT_ENABLED`).
@@ -85,9 +90,10 @@ castle services start|stop # Start/stop everything
due to rootless podman UID mapping issues). Deploy resolves the runtime via due to rootless podman UID mapping issues). Deploy resolves the runtime via
`shutil.which("docker")`. `shutil.which("docker")`.
- **MQTT**: Mosquitto broker runs as `castle-mqtt` (Docker container on port 1883). - **MQTT**: Mosquitto broker runs as `castle-mqtt` (Docker container on port 1883).
Data in `/data/castle/castle-mqtt/`, config in `/data/castle/castle-mqtt/config/`. Data in `$CASTLE_DATA_DIR/castle-mqtt/`, config in `$CASTLE_DATA_DIR/castle-mqtt/config/`.
- **Data**: Service data lives in `/data/castle/<service-name>/`, passed via env var. - **Data**: Service data lives in `$CASTLE_DATA_DIR/<service-name>/` (default
- **Secrets**: `~/.castle/secrets/` — never in project directories. `/data/castle/<name>/`), passed via the generated `<PREFIX>_DATA_DIR` env var.
- **Secrets**: `$CASTLE_HOME/secrets/` — never in project directories.
## API Endpoints (castle-api, port 9020) ## API Endpoints (castle-api, port 9020)

View File

@@ -2,6 +2,12 @@
A personal software platform. Castle manages independent services, tools, and frontends from a single CLI, with a unified gateway, systemd integration, and a web dashboard. A personal software platform. Castle manages independent services, tools, and frontends from a single CLI, with a unified gateway, systemd integration, and a web dashboard.
Historically, applications have been developed by third parties, distributed through app stores, and installed on user devices.
With the advent of AI-assisted software development, users can write the software they need directly, eliminating the need for packaging and distribution. This makes many classes of software simpler, too. Oftentimes all a user needs are simple scripts or configurations of existing tools. But no matter how simple your script or application, it needed to be tailored and packaged for specific distribution channels. Castle provides a unified environment for developing, managing, deploying, and advertising these simple applications.
Castle _stacks_ are pre-configured development environments that provide a starting point for building applications. They include everything needed to get started, from the programming language and framework to the necessary dependencies and tools. This is a design intended for coding assistants to generate castle programs with a level of consistency and to ensure that they are properly configured and ready to use with Castle. If your coding assistant knows about Castle, it can help you create and manage your custom applications more efficiently.
## Quick Start ## Quick Start
```bash ```bash

View File

@@ -53,11 +53,11 @@ def run_create(args: argparse.Namespace) -> int:
print(f"Error: '{name}' already exists in castle.yaml") print(f"Error: '{name}' already exists in castle.yaml")
return 1 return 1
programs_dir = config.root / "programs" code_dir = config.root / "code"
programs_dir.mkdir(exist_ok=True) code_dir.mkdir(exist_ok=True)
project_dir = programs_dir / name project_dir = code_dir / name
if project_dir.exists(): if project_dir.exists():
print(f"Error: directory 'programs/{name}' already exists") print(f"Error: directory 'code/{name}' already exists")
return 1 return 1
# Determine port for daemons # Determine port for daemons
@@ -82,7 +82,7 @@ def run_create(args: argparse.Namespace) -> int:
config.programs[name] = ProgramSpec( config.programs[name] = ProgramSpec(
id=name, id=name,
description=args.description or f"A castle {stack} program", description=args.description or f"A castle {stack} program",
source=f"programs/{name}", source=f"code/{name}",
stack=stack, stack=stack,
behavior=behavior, behavior=behavior,
) )
@@ -108,7 +108,7 @@ def run_create(args: argparse.Namespace) -> int:
print(f" Port: {port}") print(f" Port: {port}")
print(" Registered in castle.yaml") print(" Registered in castle.yaml")
print("\nNext steps:") print("\nNext steps:")
print(f" cd programs/{name}") print(f" cd ~/.castle/code/{name}")
print(" uv sync") print(" uv sync")
if behavior == "daemon": if behavior == "daemon":
print(f" uv run {name} # starts on port {port}") print(f" uv run {name} # starts on port {port}")

View File

@@ -32,7 +32,7 @@ class TestCreateCommand:
result = run_create(args) result = run_create(args)
assert result == 0 assert result == 0
project_dir = castle_root / "programs" / "my-api" project_dir = castle_root / "code" / "my-api"
assert project_dir.exists() assert project_dir.exists()
assert (project_dir / "pyproject.toml").exists() assert (project_dir / "pyproject.toml").exists()
assert (project_dir / "src" / "my_api" / "main.py").exists() assert (project_dir / "src" / "my_api" / "main.py").exists()
@@ -64,7 +64,7 @@ class TestCreateCommand:
result = run_create(args) result = run_create(args)
assert result == 0 assert result == 0
project_dir = castle_root / "programs" / "my-tool2" project_dir = castle_root / "code" / "my-tool2"
assert project_dir.exists() assert project_dir.exists()
assert (project_dir / "src" / "my_tool2" / "main.py").exists() assert (project_dir / "src" / "my_tool2" / "main.py").exists()
assert (project_dir / "CLAUDE.md").exists() assert (project_dir / "CLAUDE.md").exists()

View File

@@ -28,12 +28,37 @@ from castle_core.manifest import (
) )
CASTLE_HOME = Path.home() / ".castle" def _resolve_castle_home() -> Path:
"""Resolve the castle home directory (config, code, artifacts, secrets).
Defaults to ~/.castle. Override with the CASTLE_HOME environment variable
(supports ~ and relative paths, which are expanded and made absolute).
"""
override = os.environ.get("CASTLE_HOME")
if override:
return Path(override).expanduser().resolve()
return Path.home() / ".castle"
def _resolve_data_dir() -> Path:
"""Resolve the program data directory (service/program data I/O).
Decoupled from CASTLE_HOME so bulk data can live on a dedicated volume.
Defaults to /data/castle. Override with the CASTLE_DATA_DIR environment
variable (supports ~ and relative paths, which are expanded and made absolute).
"""
override = os.environ.get("CASTLE_DATA_DIR")
if override:
return Path(override).expanduser().resolve()
return Path("/data/castle")
CASTLE_HOME = _resolve_castle_home()
CODE_DIR = CASTLE_HOME / "code" CODE_DIR = CASTLE_HOME / "code"
ARTIFACTS_DIR = CASTLE_HOME / "artifacts" ARTIFACTS_DIR = CASTLE_HOME / "artifacts"
SPECS_DIR = ARTIFACTS_DIR / "specs" SPECS_DIR = ARTIFACTS_DIR / "specs"
CONTENT_DIR = ARTIFACTS_DIR / "content" CONTENT_DIR = ARTIFACTS_DIR / "content"
DATA_DIR = CASTLE_HOME / "data" DATA_DIR = _resolve_data_dir()
SECRETS_DIR = CASTLE_HOME / "secrets" SECRETS_DIR = CASTLE_HOME / "secrets"
# Backwards-compat aliases (used by existing imports) # Backwards-compat aliases (used by existing imports)

View File

@@ -146,7 +146,12 @@ def _build_deployed_service(
) -> DeployedComponent: ) -> DeployedComponent:
"""Build a DeployedComponent from a ServiceSpec.""" """Build a DeployedComponent from a ServiceSpec."""
run = svc.run run = svc.run
prefix = _env_prefix(name) # Env prefix and data dir are keyed by the program (component) the service
# runs, not the service name — that's the prefix the program's settings read
# (e.g. job `protonmail-sync` runs program `protonmail` → PROTONMAIL_DATA_DIR).
# Falls back to the service name when no component is referenced.
config_key = svc.component or name
prefix = _env_prefix(config_key)
env: dict[str, str] = {} env: dict[str, str] = {}
# Data dir convention (for managed services) # Data dir convention (for managed services)
@@ -154,7 +159,7 @@ def _build_deployed_service(
if svc.manage and svc.manage.systemd and not svc.manage.systemd.enable: if svc.manage and svc.manage.systemd and not svc.manage.systemd.enable:
managed = False managed = False
if managed: if managed:
env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / name) env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / config_key)
# Port convention (if exposed) # Port convention (if exposed)
port = None port = None
@@ -210,10 +215,13 @@ def _build_deployed_job(
) -> DeployedComponent: ) -> DeployedComponent:
"""Build a DeployedComponent from a JobSpec.""" """Build a DeployedComponent from a JobSpec."""
run = job.run run = job.run
prefix = _env_prefix(name) # Keyed by the program (component) the job runs, not the job name — see
# _build_deployed_service for rationale. Falls back to the job name.
config_key = job.component or name
prefix = _env_prefix(config_key)
env: dict[str, str] = {} env: dict[str, str] = {}
env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / name) env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / config_key)
if job.defaults and job.defaults.env: if job.defaults and job.defaults.env:
env.update(job.defaults.env) env.update(job.defaults.env)

View File

@@ -6,7 +6,8 @@ architecture.
## castle.yaml ## castle.yaml
The single source of truth. Lives at the repo root. Three top-level sections: The single source of truth. Lives at `~/.castle/castle.yaml`. Three top-level
sections:
```yaml ```yaml
gateway: gateway:
@@ -15,7 +16,7 @@ gateway:
programs: programs:
my-tool: my-tool:
description: Does something useful description: Does something useful
source: programs/my-tool source: code/my-tool
stack: python-cli stack: python-cli
behavior: tool behavior: tool
system_dependencies: [pandoc] system_dependencies: [pandoc]
@@ -76,10 +77,24 @@ Explicit declaration of how the program is used:
### `source` — Where the source lives ### `source` — Where the source lives
```yaml ```yaml
source: programs/my-tool source: code/my-tool # your programs, under ~/.castle/code/
source: repo:castle-api # castle's own programs, inside the git repo
``` ```
Relative path from repo root to the project directory. The `source` path is resolved one of three ways (`core/src/castle_core/config.py`):
| `source:` value | Resolves to | Used for |
|-----------------|-------------|----------|
| `code/my-tool` *(relative)* | `$CASTLE_HOME/code/my-tool` | Your own programs |
| `repo:castle-api` | `<repo>/castle-api` (via the top-level `repo:` field) | Castle's built-in programs |
| `/abs/path` | as-is | Anything explicitly absolute |
Relative sources resolve against the castle home (`$CASTLE_HOME`, default
`~/.castle` — see [Infrastructure paths](#infrastructure-paths)). Most programs
you create live under **`$CASTLE_HOME/code/`** and are recorded as
`source: code/<name>`. Castle's own programs (CLI, core, castle-api, app) live in
the git repo and use the `repo:` prefix. When `castle deploy` writes `castle.yaml`
back out, it rewrites absolute paths into these relative forms.
### `stack` — Development toolchain ### `stack` — Development toolchain
@@ -215,28 +230,50 @@ Castle generates a systemd `.timer` file alongside the `.service` unit.
Jobs also support `run` (required), `manage`, and `defaults` — same Jobs also support `run` (required), `manage`, and `defaults` — same
semantics as services. semantics as services.
## How programs get into `~/.castle/code/`
Every program's source lives in one place — `~/.castle/code/<name>/`. It can
arrive there a few ways:
1. **Scaffold a new one** with `castle create` — writes the project into
`~/.castle/code/<name>/` and registers it in `castle.yaml` with
`source: code/<name>`.
2. **Clone an existing project**`git clone <url> ~/.castle/code/<name>`,
then add a `programs:` entry pointing at `source: code/<name>`.
3. **Drop files in directly** — a `code/<name>/` directory is just a working
tree; it doesn't have to be under version control to be registered and run.
`~/.castle/` and `~/.castle/code/` are **not** themselves git repos, and there
are no submodules. Each program directory manages its own version control (or
none) independently — some are standalone git clones, others are just loose
files.
Castle's own programs (CLI, core, castle-api, app) are the exception: they live
inside the castle git repo and are referenced with `source: repo:<name>`.
## Registering a new program ## Registering a new program
### Via `castle create` (recommended) ### Via `castle create` (recommended)
```bash ```bash
# Service — scaffolds project, assigns port, registers in castle.yaml # Service — scaffolds into ~/.castle/code/, assigns port, registers in castle.yaml
castle create my-service --stack python-fastapi --description "Does something" castle create my-service --stack python-fastapi --description "Does something"
# Tool — scaffolds under programs/ # Tool — scaffolds into ~/.castle/code/
castle create my-tool --stack python-cli --description "Does something" castle create my-tool --stack python-cli --description "Does something"
``` ```
### Manually ### Manually
Add entries to the appropriate sections of `castle.yaml`: Clone or create the project under `~/.castle/code/`, then add entries to the
appropriate sections of `castle.yaml`:
```yaml ```yaml
# Tool — only needs a program entry # Tool — only needs a program entry
programs: programs:
my-tool: my-tool:
description: Does something useful description: Does something useful
source: programs/my-tool source: code/my-tool
stack: python-cli stack: python-cli
behavior: tool behavior: tool
@@ -244,7 +281,7 @@ programs:
programs: programs:
my-service: my-service:
description: Does something useful description: Does something useful
source: programs/my-service source: code/my-service
stack: python-fastapi stack: python-fastapi
behavior: daemon behavior: daemon
@@ -270,7 +307,7 @@ services:
```bash ```bash
castle create my-service --stack python-fastapi # 1. Scaffold + register castle create my-service --stack python-fastapi # 1. Scaffold + register
cd programs/my-service && uv sync # 2. Install deps cd ~/.castle/code/my-service && uv sync # 2. Install deps
# ... implement ... # ... implement ...
castle test my-service # 3. Run tests castle test my-service # 3. Run tests
castle service enable my-service # 4. Generate systemd unit, start castle service enable my-service # 4. Generate systemd unit, start
@@ -290,10 +327,10 @@ castle service disable my-service # Stop and remove systemd unit
```bash ```bash
castle create my-tool --stack python-cli # 1. Scaffold + register castle create my-tool --stack python-cli # 1. Scaffold + register
cd programs/my-tool && uv sync # 2. Install deps cd ~/.castle/code/my-tool && uv sync # 2. Install deps
# ... implement ... # ... implement ...
castle test my-tool # 3. Run tests castle test my-tool # 3. Run tests
uv tool install --editable programs/my-tool/ # 4. Install to PATH uv tool install --editable ~/.castle/code/my-tool/ # 4. Install to PATH
``` ```
### Job lifecycle ### Job lifecycle
@@ -317,15 +354,33 @@ and a `.timer` file.
## Infrastructure paths ## Infrastructure paths
Castle uses **two** independent roots, each overridable by an environment
variable (both expand `~` and resolve relative paths):
- **`CASTLE_HOME`** — config, code, artifacts, and secrets. Default `~/.castle`.
- **`CASTLE_DATA_DIR`** — program/service data I/O (potentially large; lives on a
dedicated volume). Default `/data/castle`. Decoupled from `CASTLE_HOME` on
purpose so bulk data doesn't sit in the home directory.
| What | Where | | What | Where |
|------|-------| |------|-------|
| Registry | `castle.yaml` (repo root) | | Castle home | `$CASTLE_HOME` (default `~/.castle`) |
| Service data | `/data/castle/<name>/` | | Registry | `$CASTLE_HOME/castle.yaml` |
| Secrets | `~/.castle/secrets/<NAME>` | | Program source (yours) | `$CASTLE_HOME/code/<name>/` |
| Generated Caddyfile | `~/.castle/generated/Caddyfile` | | Program source (castle's) | `<repo>/<name>` (via `source: repo:<name>`) |
| Secrets | `$CASTLE_HOME/secrets/<NAME>` |
| Generated Caddyfile | `$CASTLE_HOME/artifacts/specs/Caddyfile` |
| Built frontends | `$CASTLE_HOME/artifacts/content/<name>/` |
| **Service data** | **`$CASTLE_DATA_DIR/<name>/` (default `/data/castle/<name>/`)** |
| Systemd units | `~/.config/systemd/user/castle-*.service` | | Systemd units | `~/.config/systemd/user/castle-*.service` |
| Systemd timers | `~/.config/systemd/user/castle-*.timer` | | Systemd timers | `~/.config/systemd/user/castle-*.timer` |
Defined in `core/src/castle_core/config.py`: `CASTLE_HOME` (with derived
`CODE_DIR`, `SECRETS_DIR`, `SPECS_DIR`, `CONTENT_DIR`) and the independent
`DATA_DIR` (`CASTLE_DATA_DIR`). `castle deploy` passes each service its data
path via the generated `<PREFIX>_DATA_DIR` env var. Systemd unit/timer paths are
fixed by systemd's user-unit convention.
## Manifest models ## Manifest models
The Pydantic models live in `core/src/castle_core/manifest.py`. Key classes: The Pydantic models live in `core/src/castle_core/manifest.py`. Key classes:

View File

@@ -31,8 +31,9 @@ is self-sufficient. The mesh is optional.
operates above the line. operates above the line.
5. **Separate source from runtime.** The repo is for development. The 5. **Separate source from runtime.** The repo is for development. The
runtime lives in standard Unix locations (`~/.castle/`, `/data/castle/`, runtime lives in standard Unix locations (`$CASTLE_HOME`, default
systemd units). Nothing running should point into the source tree. `~/.castle/`, plus systemd units). Nothing running should point into the
source tree.
6. **AI-manageable.** The CLI and API exist so that AI assistants can 6. **AI-manageable.** The CLI and API exist so that AI assistants can
discover, create, and manage components programmatically. Humans discover, create, and manage components programmatically. Humans
@@ -113,9 +114,10 @@ Manages running processes using standard Linux infrastructure.
- TLS termination - TLS termination
**Filesystem** handles storage: **Filesystem** handles storage:
- Service data: `/data/castle/<name>/` - Service data: `$CASTLE_DATA_DIR/<name>/` (default `/data/castle/`, on a
- Secrets: `~/.castle/secrets/` dedicated volume)
- Generated config: `~/.castle/generated/` - Secrets: `$CASTLE_HOME/secrets/` (default `~/.castle/secrets/`)
- Generated config: `$CASTLE_HOME/artifacts/specs/` (Caddyfile, registry.yaml)
Castle generates systemd unit files and Caddyfile entries from the Castle generates systemd unit files and Caddyfile entries from the
registry. It doesn't run a daemon itself — it configures OS-level registry. It doesn't run a daemon itself — it configures OS-level
@@ -123,8 +125,8 @@ infrastructure and gets out of the way.
Critically, the runtime layer references only standard paths — never Critically, the runtime layer references only standard paths — never
the source tree. Systemd units point to installed binaries (on PATH the source tree. Systemd units point to installed binaries (on PATH
or in `~/.castle/bin/`), not to repo subdirectories. Caddy serves or in `~/.local/bin/`), not to repo subdirectories. Caddy serves
from `~/.castle/static/`, not from build output directories in the repo. from `$CASTLE_HOME/artifacts/content/`, not from build output directories in the repo.
### Registry Layer ### Registry Layer
@@ -156,7 +158,7 @@ These map to two files:
programs: programs:
central-context: central-context:
description: Content storage API description: Content storage API
source: components/central-context source: code/central-context
services: services:
central-context: central-context:
@@ -197,7 +199,7 @@ Convention-based env vars (`<PREFIX>_PORT`, `<PREFIX>_DATA_DIR`) are
generated automatically during deploy. Only non-convention values need generated automatically during deploy. Only non-convention values need
`defaults.env`. `defaults.env`.
**`~/.castle/registry.yaml`** (per-node, not in the repo) — Node config: **`$CASTLE_HOME/artifacts/specs/registry.yaml`** (per-node, not in the repo, generated by `castle deploy`) — Node config:
```yaml ```yaml
node: node:
@@ -209,7 +211,7 @@ deployed:
runner: python runner: python
run_cmd: [/home/user/.local/bin/central-context] run_cmd: [/home/user/.local/bin/central-context]
env: env:
CENTRAL_CONTEXT_DATA_DIR: /data/castle/central-context CENTRAL_CONTEXT_DATA_DIR: /home/user/.castle/data/central-context
CENTRAL_CONTEXT_PORT: "9001" CENTRAL_CONTEXT_PORT: "9001"
behavior: daemon behavior: daemon
stack: python-fastapi stack: python-fastapi
@@ -302,7 +304,7 @@ Local paths always take precedence.
### Dashboard ### Dashboard
The web dashboard (`castle-app`) is a React SPA served by Caddy from The web dashboard (`castle-app`) is a React SPA served by Caddy from
`~/.castle/static/castle-app/`. It talks to `castle-api` via the `$CASTLE_HOME/artifacts/content/castle-app/`. It talks to `castle-api` via the
gateway proxy at `/api`. gateway proxy at `/api`.
**Layout:** **Layout:**
@@ -417,9 +419,10 @@ Each step is distinct:
execute them implicitly. execute them implicitly.
2. **Install** — Makes the artifact available on the system. For tools: 2. **Install** — Makes the artifact available on the system. For tools:
`uv tool install` or compiled binary placed in `~/.castle/bin/`. For `uv tool install` or compiled binary placed in `~/.local/bin/`. For
services: same — the binary or entry point is on PATH or in a known services: same — the binary or entry point is on PATH or in a known
location. For frontends: built assets copied to `~/.castle/static/`. location. For frontends: built assets copied to
`$CASTLE_HOME/artifacts/content/`.
3. **Deploy** — Materializes the runtime configuration. Reads the 3. **Deploy** — Materializes the runtime configuration. Reads the
component spec, merges with node config, generates systemd units component spec, merges with node config, generates systemd units
@@ -427,28 +430,32 @@ Each step is distinct:
the source tree. Enables and starts services. the source tree. Enables and starts services.
For compiled languages (Rust, Go), build produces a standalone binary For compiled languages (Rust, Go), build produces a standalone binary
and install is just placing it in `~/.castle/bin/`. For interpreted and install is just placing it in `~/.local/bin/`. For interpreted
languages (Python, Node), the runtime wrapper (uv, node) handles languages (Python, Node), the runtime wrapper (uv, node) handles
finding the installed artifact. finding the installed artifact.
## Runtime Filesystem Layout ## Runtime Filesystem Layout
What already exists and what the target looks like: Two roots, each overridable by an env var: `$CASTLE_HOME` (config, code,
artifacts, secrets; default `~/.castle`) and `$CASTLE_DATA_DIR` (bulk program
data; default `/data/castle`, kept on a dedicated volume):
``` ```
~/.castle/ ← Castle runtime home $CASTLE_HOME/ ← Config & artifacts (default ~/.castle)
├── registry.yaml ← Node config (what's deployed here) ├── castle.yaml ← Registry spec (programs, services, jobs)
├── generated/Generated Caddyfile ├── infra.confInfrastructure install choices
│ └── Caddyfile ├── code/ ← Program source (your programs)
├── secrets/ ← Secret files (NAME → value) │ └── <name>/
│ └── PROTONMAIL_API_KEY ├── artifacts/
├── bin/ ← Compiled binaries, shims │ ├── specs/ ← Generated by `castle deploy`
└── my-go-tool │ ├── Caddyfile
└── static/ ← Built frontend assets │ │ └── registry.yaml ← Node config (what's deployed here)
└── castle-app/ └── content/ ← Built frontend assets
└── dist/ └── castle-app/ ← (index.html + assets, served at root)
└── secrets/ ← Secret files (NAME → value)
└── PROTONMAIL_API_KEY
/data/castle/ ← Persistent service data $CASTLE_DATA_DIR/ ← Persistent service data (default /data/castle)
└── <name>/ └── <name>/
~/.config/systemd/user/ ← Systemd units (standard location) ~/.config/systemd/user/ ← Systemd units (standard location)
@@ -458,8 +465,11 @@ What already exists and what the target looks like:
└── ... └── ...
``` ```
Compiled-language tools (Rust, Go — planned) install their binaries to the
standard `~/.local/bin/`, not under `$CASTLE_HOME`.
Source (the repo) is referenced only during build and install. Everything Source (the repo) is referenced only during build and install. Everything
the runtime touches lives in `~/.castle/`, `/data/castle/`, or standard the runtime touches lives under `$CASTLE_HOME`, `$CASTLE_DATA_DIR`, or standard
systemd paths. systemd paths.
## OTP as Design Guide ## OTP as Design Guide
@@ -470,7 +480,7 @@ Castle's architecture parallels Erlang/OTP, mapped onto Unix:
|-------------|------------------| |-------------|------------------|
| Application | Component (independent, self-contained) | | Application | Component (independent, self-contained) |
| Application resource file | Component spec in `castle.yaml` | | Application resource file | Component spec in `castle.yaml` |
| Release config (sys.config) | Node config in `~/.castle/registry.yaml` | | Release config (sys.config) | Node config in `$CASTLE_HOME/artifacts/specs/registry.yaml` |
| Release assembly | `castle deploy` (spec + node config → runtime) | | Release assembly | `castle deploy` (spec + node config → runtime) |
| Supervisor | systemd (restart policies, ordering) | | Supervisor | systemd (restart policies, ordering) |
| Process | Running service/worker/job | | Process | Running service/worker/job |
@@ -508,17 +518,17 @@ What exists today:
- **Three packages** — `castle-core` (models, config, generators), - **Three packages** — `castle-core` (models, config, generators),
`castle-cli` (commands), `castle-api` (HTTP API) `castle-cli` (commands), `castle-api` (HTTP API)
- **Source/runtime split** — `castle.yaml` (spec) → `castle deploy` - **Source/runtime split** — `castle.yaml` (spec) → `castle deploy`
`~/.castle/registry.yaml` (node config). Systemd units and Caddyfile `$CASTLE_HOME/artifacts/specs/registry.yaml` (node config). Systemd units and
generated from registry with fully resolved paths. No repo references Caddyfile generated from registry with fully resolved paths. No repo references
in runtime artifacts. in runtime artifacts.
- **Convention-based env generation** — `castle deploy` auto-generates - **Convention-based env generation** — `castle deploy` auto-generates
`<PREFIX>_DATA_DIR=/data/castle/<name>` and `<PREFIX>_PORT` from `<PREFIX>_DATA_DIR=$CASTLE_DATA_DIR/<name>` and `<PREFIX>_PORT` from
the manifest. Only non-convention values need `defaults.env`. the manifest. Only non-convention values need `defaults.env`.
- **Gateway** — Caddy on port 9000, Caddyfile generated from registry - **Gateway** — Caddy on port 9000, Caddyfile generated from registry
- **API** — `castle-api` on port 9020, reads from registry (optional - **API** — `castle-api` on port 9020, reads from registry (optional
castle.yaml fallback for non-deployed components) castle.yaml fallback for non-deployed components)
- **Dashboard** — `castle-app` React/Vite frontend, static assets - **Dashboard** — `castle-app` React/Vite frontend, static assets
served from `~/.castle/static/castle-app/` served from `$CASTLE_HOME/artifacts/content/castle-app/`
- **Services** — central-context (content storage), notification-bridge - **Services** — central-context (content storage), notification-bridge
(desktop notification forwarder) (desktop notification forwarder)
- **Jobs** — protonmail (email sync every 5 min), backup-collect (nightly), - **Jobs** — protonmail (email sync every 5 min), backup-collect (nightly),
@@ -531,7 +541,7 @@ What exists today:
Opt-in via `CASTLE_API_MQTT_ENABLED` / `CASTLE_API_MDNS_ENABLED`. Opt-in via `CASTLE_API_MQTT_ENABLED` / `CASTLE_API_MDNS_ENABLED`.
- **MQTT broker** — Mosquitto running as `castle-mqtt` Docker container - **MQTT broker** — Mosquitto running as `castle-mqtt` Docker container
on port 1883, managed by systemd. Config and data in on port 1883, managed by systemd. Config and data in
`/data/castle/castle-mqtt/`. `$CASTLE_DATA_DIR/castle-mqtt/`.
- **Node API** — `GET /mesh/status`, `GET /nodes`, `GET /nodes/{hostname}`. - **Node API** — `GET /mesh/status`, `GET /nodes`, `GET /nodes/{hostname}`.
`GET /components?include_remote=true` for cross-node component listing. `GET /components?include_remote=true` for cross-node component listing.
- **Gateway panel** — Dedicated UI showing route table, health per route, - **Gateway panel** — Dedicated UI showing route table, health per route,
@@ -559,7 +569,7 @@ What doesn't exist yet:
| Process supervision | systemd (user units) | Active | | Process supervision | systemd (user units) | Active |
| HTTP routing | Caddy (port 9000) | Active | | HTTP routing | Caddy (port 9000) | Active |
| Component specs | castle.yaml + Pydantic models | Active | | Component specs | castle.yaml + Pydantic models | Active |
| Node config | `~/.castle/registry.yaml` | Active | | Node config | `$CASTLE_HOME/artifacts/specs/registry.yaml` | Active |
| CLI | castle (Python, uv) | Active | | CLI | castle (Python, uv) | Active |
| API | castle-api (FastAPI) | Active | | API | castle-api (FastAPI) | Active |
| Dashboard | castle-app (React, Vite, shadcn/ui) | Active | | Dashboard | castle-app (React, Vite, shadcn/ui) | Active |
@@ -569,7 +579,7 @@ What doesn't exist yet:
| Type checking | pyright (Python), tsc (TS) | Active | | Type checking | pyright (Python), tsc (TS) | Active |
| Testing | pytest (Python), Vitest (TS) | Active | | Testing | pytest (Python), Vitest (TS) | Active |
| Secrets | `~/.castle/secrets/` file-based | Active | | Secrets | `~/.castle/secrets/` file-based | Active |
| Data storage | Filesystem (`/data/castle/`) | Active | | Data storage | Filesystem (`$CASTLE_DATA_DIR/`, default `/data/castle/`) | Active |
| Messaging | MQTT (paho-mqtt client, Mosquitto broker) | Active (opt-in) | | Messaging | MQTT (paho-mqtt client, Mosquitto broker) | Active (opt-in) |
| Node discovery | mDNS (python-zeroconf) + MQTT | Active (opt-in) | | Node discovery | mDNS (python-zeroconf) + MQTT | Active (opt-in) |
| Rust packaging | cargo | Planned | | Rust packaging | cargo | Planned |

View File

@@ -24,10 +24,10 @@ How to build CLI tools following Unix philosophy.
## Project layout ## Project layout
Each tool is an independent project under `components/` with its own `pyproject.toml`: Each tool is an independent project under `~/.castle/code/` with its own `pyproject.toml`:
``` ```
components/my-tool/ ~/.castle/code/my-tool/
├── src/my_tool/ ├── src/my_tool/
│ ├── __init__.py │ ├── __init__.py
│ └── main.py # Entry point │ └── main.py # Entry point
@@ -37,13 +37,13 @@ components/my-tool/
└── CLAUDE.md └── CLAUDE.md
``` ```
Examples: `components/pdf2md/`, `components/gpt/`, `components/protonmail/` Examples: `code/pdf2md/`, `code/gpt/`, `code/protonmail/`
## Creating a new tool ## Creating a new tool
```bash ```bash
castle create my-tool --stack python-cli --description "Does something" castle create my-tool --stack python-cli --description "Does something"
cd components/my-tool && uv sync cd ~/.castle/code/my-tool && uv sync
``` ```
This scaffolds the project and registers it in `castle.yaml`. This scaffolds the project and registers it in `castle.yaml`.
@@ -317,7 +317,7 @@ uv run ruff format . # Format
programs: programs:
my-tool: my-tool:
description: Does something useful description: Does something useful
source: components/my-tool source: code/my-tool
stack: python-cli stack: python-cli
behavior: tool behavior: tool
``` ```
@@ -328,7 +328,7 @@ Tools with system dependencies declare them directly on the program:
programs: programs:
pdf2md: pdf2md:
description: Convert PDF files to Markdown description: Convert PDF files to Markdown
source: components/pdf2md source: code/pdf2md
stack: python-cli stack: python-cli
behavior: tool behavior: tool
system_dependencies: [pandoc, poppler-utils] system_dependencies: [pandoc, poppler-utils]

View File

@@ -102,7 +102,7 @@ Castle passes config via env vars in castle.yaml:
programs: programs:
my-service: my-service:
description: Does something useful description: Does something useful
source: components/my-service source: code/my-service
stack: python-fastapi stack: python-fastapi
behavior: daemon behavior: daemon
@@ -292,12 +292,15 @@ Mapping convention:
Castle services use filesystem storage with JSON metadata sidecars: Castle services use filesystem storage with JSON metadata sidecars:
``` ```
/data/castle/my-service/ $CASTLE_DATA_DIR/my-service/ # default /data/castle/my-service/
└── bucket/ └── bucket/
├── item-name ├── item-name
└── item-name.meta.json └── item-name.meta.json
``` ```
The service receives this path via the generated `<PREFIX>_DATA_DIR` env var —
it never hardcodes it. Use the `data_dir` setting from your config.
```python ```python
# storage.py # storage.py
import hashlib import hashlib

View File

@@ -20,7 +20,7 @@ How to build, serve, and manage web frontends as castle components.
```bash ```bash
# Create the project # Create the project
mkdir components/my-frontend && cd components/my-frontend mkdir ~/.castle/code/my-frontend && cd ~/.castle/code/my-frontend
pnpm create vite . --template react-ts pnpm create vite . --template react-ts
# Core dependencies # Core dependencies
@@ -116,7 +116,7 @@ handles serving directly from the build output.
programs: programs:
my-frontend: my-frontend:
description: Web dashboard description: Web dashboard
source: components/my-frontend source: code/my-frontend
build: build:
commands: commands:
- ["pnpm", "build"] - ["pnpm", "build"]
@@ -124,7 +124,8 @@ programs:
- dist/ - dist/
``` ```
For production, Caddy serves the static `dist/` output directly — no For production, `castle deploy` copies the build output to
`~/.castle/artifacts/content/<name>/` and Caddy serves it from there — no
Node process needed. See [Serving with Caddy](#serving-with-caddy) below. Node process needed. See [Serving with Caddy](#serving-with-caddy) below.
For development with Vite's dev server, add a service entry: For development with Vite's dev server, add a service entry:
@@ -148,12 +149,23 @@ See @docs/component-registry.md for the full registry reference.
## Serving with Caddy ## Serving with Caddy
For production, serve the static `dist/` output directly from Caddy rather than For production, the static build output is served by Caddy rather than a Node
running a Node process. The gateway Caddyfile can serve the files: process. You do **not** write this block by hand — `castle deploy` generates it.
The flow:
1. `castle deploy` runs the program's `build.commands`, then copies each
`build.outputs` directory from `~/.castle/code/<name>/` into
`~/.castle/artifacts/content/<name>/` (`core/src/castle_core/deploy.py`,
`_copy_app_static`).
2. The Caddyfile generator scans `~/.castle/artifacts/content/`; any directory
containing an `index.html` is served as a SPA at a path prefix matching its
name. `castle-app` is special-cased to serve at the root `/`.
The generated block looks like this (written to `~/.castle/artifacts/specs/Caddyfile`):
```caddyfile ```caddyfile
handle_path /app/* { handle_path /my-frontend/* {
root * /data/repos/castle/my-frontend/dist root * /home/payne/.castle/artifacts/content/my-frontend
try_files {path} /index.html try_files {path} /index.html
file_server file_server
} }
@@ -161,7 +173,8 @@ handle_path /app/* {
The `try_files {path} /index.html` directive is essential for SPA routing — The `try_files {path} /index.html` directive is essential for SPA routing —
it falls back to `index.html` for any path that doesn't match a static file, it falls back to `index.html` for any path that doesn't match a static file,
letting React Router handle client-side routes. letting React Router handle client-side routes. The serving prefix is derived
from the program name, not hand-configured.
## API integration ## API integration