Clean up.
This commit is contained in:
24
CLAUDE.md
24
CLAUDE.md
@@ -37,18 +37,19 @@ When creating a new service, tool, or frontend, follow the detailed guides:
|
||||
```bash
|
||||
# Daemon (python-fastapi)
|
||||
castle create my-service --stack python-fastapi --description "Does something"
|
||||
cd programs/my-service && uv sync
|
||||
cd ~/.castle/code/my-service && uv sync
|
||||
uv run my-service # starts on auto-assigned port
|
||||
castle service enable my-service # register with systemd
|
||||
castle gateway reload # update reverse proxy routes
|
||||
|
||||
# Tool (python-cli)
|
||||
castle create my-tool --stack python-cli --description "Does something"
|
||||
cd programs/my-tool && uv sync
|
||||
cd ~/.castle/code/my-tool && uv sync
|
||||
```
|
||||
|
||||
The `castle create` command scaffolds the project under `programs/`,
|
||||
generates a CLAUDE.md, and registers it in `castle.yaml`.
|
||||
The `castle create` command scaffolds the project under `~/.castle/code/`,
|
||||
generates a CLAUDE.md, and registers it in `castle.yaml` with
|
||||
`source: code/<name>`.
|
||||
|
||||
## Castle CLI
|
||||
|
||||
@@ -63,7 +64,6 @@ castle create <name> --stack python-fastapi # Scaffold new project
|
||||
castle deploy [name] # Deploy to runtime (registry + systemd + Caddyfile)
|
||||
castle test [project] # Run tests (one or all)
|
||||
castle lint [project] # Run linter (one or all)
|
||||
castle sync # Update submodules + uv sync all
|
||||
castle run <name> # Run service in foreground
|
||||
castle logs <name> [-f] [-n 50] # View service/job logs
|
||||
castle tool list # List all tools
|
||||
@@ -76,8 +76,13 @@ castle services start|stop # Start/stop everything
|
||||
|
||||
## Infrastructure
|
||||
|
||||
Castle uses two roots, each overridable by an env var: `CASTLE_HOME` (config,
|
||||
code, artifacts, secrets; default `~/.castle`) and `CASTLE_DATA_DIR` (program
|
||||
data I/O on a dedicated volume; default `/data/castle`). Paths below use
|
||||
`$CASTLE_HOME` and `$CASTLE_DATA_DIR` accordingly.
|
||||
|
||||
- **Gateway**: Caddy reverse proxy at port 9000, config generated from `castle.yaml`
|
||||
into `~/.castle/generated/Caddyfile`. Dashboard served at root.
|
||||
into `$CASTLE_HOME/artifacts/specs/Caddyfile`. Dashboard served at root.
|
||||
- **Systemd**: User units generated under `~/.config/systemd/user/castle-*.service`.
|
||||
Use drop-in overrides (`*.service.d/*.conf`) for extra env vars that `castle deploy`
|
||||
shouldn't overwrite (e.g., `CASTLE_API_MQTT_ENABLED`).
|
||||
@@ -85,9 +90,10 @@ castle services start|stop # Start/stop everything
|
||||
due to rootless podman UID mapping issues). Deploy resolves the runtime via
|
||||
`shutil.which("docker")`.
|
||||
- **MQTT**: Mosquitto broker runs as `castle-mqtt` (Docker container on port 1883).
|
||||
Data in `/data/castle/castle-mqtt/`, config in `/data/castle/castle-mqtt/config/`.
|
||||
- **Data**: Service data lives in `/data/castle/<service-name>/`, passed via env var.
|
||||
- **Secrets**: `~/.castle/secrets/` — never in project directories.
|
||||
Data in `$CASTLE_DATA_DIR/castle-mqtt/`, config in `$CASTLE_DATA_DIR/castle-mqtt/config/`.
|
||||
- **Data**: Service data lives in `$CASTLE_DATA_DIR/<service-name>/` (default
|
||||
`/data/castle/<name>/`), passed via the generated `<PREFIX>_DATA_DIR` env var.
|
||||
- **Secrets**: `$CASTLE_HOME/secrets/` — never in project directories.
|
||||
|
||||
## API Endpoints (castle-api, port 9020)
|
||||
|
||||
|
||||
@@ -2,6 +2,12 @@
|
||||
|
||||
A personal software platform. Castle manages independent services, tools, and frontends from a single CLI, with a unified gateway, systemd integration, and a web dashboard.
|
||||
|
||||
Historically, applications have been developed by third parties, distributed through app stores, and installed on user devices.
|
||||
|
||||
With the advent of AI-assisted software development, users can write the software they need directly, eliminating the need for packaging and distribution. This makes many classes of software simpler, too. Oftentimes all a user needs are simple scripts or configurations of existing tools. But no matter how simple your script or application, it needed to be tailored and packaged for specific distribution channels. Castle provides a unified environment for developing, managing, deploying, and advertising these simple applications.
|
||||
|
||||
Castle _stacks_ are pre-configured development environments that provide a starting point for building applications. They include everything needed to get started, from the programming language and framework to the necessary dependencies and tools. This is a design intended for coding assistants to generate castle programs with a level of consistency and to ensure that they are properly configured and ready to use with Castle. If your coding assistant knows about Castle, it can help you create and manage your custom applications more efficiently.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
|
||||
@@ -53,11 +53,11 @@ def run_create(args: argparse.Namespace) -> int:
|
||||
print(f"Error: '{name}' already exists in castle.yaml")
|
||||
return 1
|
||||
|
||||
programs_dir = config.root / "programs"
|
||||
programs_dir.mkdir(exist_ok=True)
|
||||
project_dir = programs_dir / name
|
||||
code_dir = config.root / "code"
|
||||
code_dir.mkdir(exist_ok=True)
|
||||
project_dir = code_dir / name
|
||||
if project_dir.exists():
|
||||
print(f"Error: directory 'programs/{name}' already exists")
|
||||
print(f"Error: directory 'code/{name}' already exists")
|
||||
return 1
|
||||
|
||||
# Determine port for daemons
|
||||
@@ -82,7 +82,7 @@ def run_create(args: argparse.Namespace) -> int:
|
||||
config.programs[name] = ProgramSpec(
|
||||
id=name,
|
||||
description=args.description or f"A castle {stack} program",
|
||||
source=f"programs/{name}",
|
||||
source=f"code/{name}",
|
||||
stack=stack,
|
||||
behavior=behavior,
|
||||
)
|
||||
@@ -108,7 +108,7 @@ def run_create(args: argparse.Namespace) -> int:
|
||||
print(f" Port: {port}")
|
||||
print(" Registered in castle.yaml")
|
||||
print("\nNext steps:")
|
||||
print(f" cd programs/{name}")
|
||||
print(f" cd ~/.castle/code/{name}")
|
||||
print(" uv sync")
|
||||
if behavior == "daemon":
|
||||
print(f" uv run {name} # starts on port {port}")
|
||||
|
||||
@@ -32,7 +32,7 @@ class TestCreateCommand:
|
||||
result = run_create(args)
|
||||
|
||||
assert result == 0
|
||||
project_dir = castle_root / "programs" / "my-api"
|
||||
project_dir = castle_root / "code" / "my-api"
|
||||
assert project_dir.exists()
|
||||
assert (project_dir / "pyproject.toml").exists()
|
||||
assert (project_dir / "src" / "my_api" / "main.py").exists()
|
||||
@@ -64,7 +64,7 @@ class TestCreateCommand:
|
||||
result = run_create(args)
|
||||
|
||||
assert result == 0
|
||||
project_dir = castle_root / "programs" / "my-tool2"
|
||||
project_dir = castle_root / "code" / "my-tool2"
|
||||
assert project_dir.exists()
|
||||
assert (project_dir / "src" / "my_tool2" / "main.py").exists()
|
||||
assert (project_dir / "CLAUDE.md").exists()
|
||||
|
||||
@@ -28,12 +28,37 @@ from castle_core.manifest import (
|
||||
)
|
||||
|
||||
|
||||
CASTLE_HOME = Path.home() / ".castle"
|
||||
def _resolve_castle_home() -> Path:
|
||||
"""Resolve the castle home directory (config, code, artifacts, secrets).
|
||||
|
||||
Defaults to ~/.castle. Override with the CASTLE_HOME environment variable
|
||||
(supports ~ and relative paths, which are expanded and made absolute).
|
||||
"""
|
||||
override = os.environ.get("CASTLE_HOME")
|
||||
if override:
|
||||
return Path(override).expanduser().resolve()
|
||||
return Path.home() / ".castle"
|
||||
|
||||
|
||||
def _resolve_data_dir() -> Path:
|
||||
"""Resolve the program data directory (service/program data I/O).
|
||||
|
||||
Decoupled from CASTLE_HOME so bulk data can live on a dedicated volume.
|
||||
Defaults to /data/castle. Override with the CASTLE_DATA_DIR environment
|
||||
variable (supports ~ and relative paths, which are expanded and made absolute).
|
||||
"""
|
||||
override = os.environ.get("CASTLE_DATA_DIR")
|
||||
if override:
|
||||
return Path(override).expanduser().resolve()
|
||||
return Path("/data/castle")
|
||||
|
||||
|
||||
CASTLE_HOME = _resolve_castle_home()
|
||||
CODE_DIR = CASTLE_HOME / "code"
|
||||
ARTIFACTS_DIR = CASTLE_HOME / "artifacts"
|
||||
SPECS_DIR = ARTIFACTS_DIR / "specs"
|
||||
CONTENT_DIR = ARTIFACTS_DIR / "content"
|
||||
DATA_DIR = CASTLE_HOME / "data"
|
||||
DATA_DIR = _resolve_data_dir()
|
||||
SECRETS_DIR = CASTLE_HOME / "secrets"
|
||||
|
||||
# Backwards-compat aliases (used by existing imports)
|
||||
|
||||
@@ -146,7 +146,12 @@ def _build_deployed_service(
|
||||
) -> DeployedComponent:
|
||||
"""Build a DeployedComponent from a ServiceSpec."""
|
||||
run = svc.run
|
||||
prefix = _env_prefix(name)
|
||||
# Env prefix and data dir are keyed by the program (component) the service
|
||||
# runs, not the service name — that's the prefix the program's settings read
|
||||
# (e.g. job `protonmail-sync` runs program `protonmail` → PROTONMAIL_DATA_DIR).
|
||||
# Falls back to the service name when no component is referenced.
|
||||
config_key = svc.component or name
|
||||
prefix = _env_prefix(config_key)
|
||||
env: dict[str, str] = {}
|
||||
|
||||
# Data dir convention (for managed services)
|
||||
@@ -154,7 +159,7 @@ def _build_deployed_service(
|
||||
if svc.manage and svc.manage.systemd and not svc.manage.systemd.enable:
|
||||
managed = False
|
||||
if managed:
|
||||
env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / name)
|
||||
env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / config_key)
|
||||
|
||||
# Port convention (if exposed)
|
||||
port = None
|
||||
@@ -210,10 +215,13 @@ def _build_deployed_job(
|
||||
) -> DeployedComponent:
|
||||
"""Build a DeployedComponent from a JobSpec."""
|
||||
run = job.run
|
||||
prefix = _env_prefix(name)
|
||||
# Keyed by the program (component) the job runs, not the job name — see
|
||||
# _build_deployed_service for rationale. Falls back to the job name.
|
||||
config_key = job.component or name
|
||||
prefix = _env_prefix(config_key)
|
||||
env: dict[str, str] = {}
|
||||
|
||||
env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / name)
|
||||
env[f"{prefix}_DATA_DIR"] = str(DATA_DIR / config_key)
|
||||
|
||||
if job.defaults and job.defaults.env:
|
||||
env.update(job.defaults.env)
|
||||
|
||||
@@ -6,7 +6,8 @@ architecture.
|
||||
|
||||
## castle.yaml
|
||||
|
||||
The single source of truth. Lives at the repo root. Three top-level sections:
|
||||
The single source of truth. Lives at `~/.castle/castle.yaml`. Three top-level
|
||||
sections:
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
@@ -15,7 +16,7 @@ gateway:
|
||||
programs:
|
||||
my-tool:
|
||||
description: Does something useful
|
||||
source: programs/my-tool
|
||||
source: code/my-tool
|
||||
stack: python-cli
|
||||
behavior: tool
|
||||
system_dependencies: [pandoc]
|
||||
@@ -76,10 +77,24 @@ Explicit declaration of how the program is used:
|
||||
### `source` — Where the source lives
|
||||
|
||||
```yaml
|
||||
source: programs/my-tool
|
||||
source: code/my-tool # your programs, under ~/.castle/code/
|
||||
source: repo:castle-api # castle's own programs, inside the git repo
|
||||
```
|
||||
|
||||
Relative path from repo root to the project directory.
|
||||
The `source` path is resolved one of three ways (`core/src/castle_core/config.py`):
|
||||
|
||||
| `source:` value | Resolves to | Used for |
|
||||
|-----------------|-------------|----------|
|
||||
| `code/my-tool` *(relative)* | `$CASTLE_HOME/code/my-tool` | Your own programs |
|
||||
| `repo:castle-api` | `<repo>/castle-api` (via the top-level `repo:` field) | Castle's built-in programs |
|
||||
| `/abs/path` | as-is | Anything explicitly absolute |
|
||||
|
||||
Relative sources resolve against the castle home (`$CASTLE_HOME`, default
|
||||
`~/.castle` — see [Infrastructure paths](#infrastructure-paths)). Most programs
|
||||
you create live under **`$CASTLE_HOME/code/`** and are recorded as
|
||||
`source: code/<name>`. Castle's own programs (CLI, core, castle-api, app) live in
|
||||
the git repo and use the `repo:` prefix. When `castle deploy` writes `castle.yaml`
|
||||
back out, it rewrites absolute paths into these relative forms.
|
||||
|
||||
### `stack` — Development toolchain
|
||||
|
||||
@@ -215,28 +230,50 @@ Castle generates a systemd `.timer` file alongside the `.service` unit.
|
||||
Jobs also support `run` (required), `manage`, and `defaults` — same
|
||||
semantics as services.
|
||||
|
||||
## How programs get into `~/.castle/code/`
|
||||
|
||||
Every program's source lives in one place — `~/.castle/code/<name>/`. It can
|
||||
arrive there a few ways:
|
||||
|
||||
1. **Scaffold a new one** with `castle create` — writes the project into
|
||||
`~/.castle/code/<name>/` and registers it in `castle.yaml` with
|
||||
`source: code/<name>`.
|
||||
2. **Clone an existing project** — `git clone <url> ~/.castle/code/<name>`,
|
||||
then add a `programs:` entry pointing at `source: code/<name>`.
|
||||
3. **Drop files in directly** — a `code/<name>/` directory is just a working
|
||||
tree; it doesn't have to be under version control to be registered and run.
|
||||
|
||||
`~/.castle/` and `~/.castle/code/` are **not** themselves git repos, and there
|
||||
are no submodules. Each program directory manages its own version control (or
|
||||
none) independently — some are standalone git clones, others are just loose
|
||||
files.
|
||||
|
||||
Castle's own programs (CLI, core, castle-api, app) are the exception: they live
|
||||
inside the castle git repo and are referenced with `source: repo:<name>`.
|
||||
|
||||
## Registering a new program
|
||||
|
||||
### Via `castle create` (recommended)
|
||||
|
||||
```bash
|
||||
# Service — scaffolds project, assigns port, registers in castle.yaml
|
||||
# Service — scaffolds into ~/.castle/code/, assigns port, registers in castle.yaml
|
||||
castle create my-service --stack python-fastapi --description "Does something"
|
||||
|
||||
# Tool — scaffolds under programs/
|
||||
# Tool — scaffolds into ~/.castle/code/
|
||||
castle create my-tool --stack python-cli --description "Does something"
|
||||
```
|
||||
|
||||
### Manually
|
||||
|
||||
Add entries to the appropriate sections of `castle.yaml`:
|
||||
Clone or create the project under `~/.castle/code/`, then add entries to the
|
||||
appropriate sections of `castle.yaml`:
|
||||
|
||||
```yaml
|
||||
# Tool — only needs a program entry
|
||||
programs:
|
||||
my-tool:
|
||||
description: Does something useful
|
||||
source: programs/my-tool
|
||||
source: code/my-tool
|
||||
stack: python-cli
|
||||
behavior: tool
|
||||
|
||||
@@ -244,7 +281,7 @@ programs:
|
||||
programs:
|
||||
my-service:
|
||||
description: Does something useful
|
||||
source: programs/my-service
|
||||
source: code/my-service
|
||||
stack: python-fastapi
|
||||
behavior: daemon
|
||||
|
||||
@@ -270,7 +307,7 @@ services:
|
||||
|
||||
```bash
|
||||
castle create my-service --stack python-fastapi # 1. Scaffold + register
|
||||
cd programs/my-service && uv sync # 2. Install deps
|
||||
cd ~/.castle/code/my-service && uv sync # 2. Install deps
|
||||
# ... implement ...
|
||||
castle test my-service # 3. Run tests
|
||||
castle service enable my-service # 4. Generate systemd unit, start
|
||||
@@ -290,10 +327,10 @@ castle service disable my-service # Stop and remove systemd unit
|
||||
|
||||
```bash
|
||||
castle create my-tool --stack python-cli # 1. Scaffold + register
|
||||
cd programs/my-tool && uv sync # 2. Install deps
|
||||
cd ~/.castle/code/my-tool && uv sync # 2. Install deps
|
||||
# ... implement ...
|
||||
castle test my-tool # 3. Run tests
|
||||
uv tool install --editable programs/my-tool/ # 4. Install to PATH
|
||||
uv tool install --editable ~/.castle/code/my-tool/ # 4. Install to PATH
|
||||
```
|
||||
|
||||
### Job lifecycle
|
||||
@@ -317,15 +354,33 @@ and a `.timer` file.
|
||||
|
||||
## Infrastructure paths
|
||||
|
||||
Castle uses **two** independent roots, each overridable by an environment
|
||||
variable (both expand `~` and resolve relative paths):
|
||||
|
||||
- **`CASTLE_HOME`** — config, code, artifacts, and secrets. Default `~/.castle`.
|
||||
- **`CASTLE_DATA_DIR`** — program/service data I/O (potentially large; lives on a
|
||||
dedicated volume). Default `/data/castle`. Decoupled from `CASTLE_HOME` on
|
||||
purpose so bulk data doesn't sit in the home directory.
|
||||
|
||||
| What | Where |
|
||||
|------|-------|
|
||||
| Registry | `castle.yaml` (repo root) |
|
||||
| Service data | `/data/castle/<name>/` |
|
||||
| Secrets | `~/.castle/secrets/<NAME>` |
|
||||
| Generated Caddyfile | `~/.castle/generated/Caddyfile` |
|
||||
| Castle home | `$CASTLE_HOME` (default `~/.castle`) |
|
||||
| Registry | `$CASTLE_HOME/castle.yaml` |
|
||||
| Program source (yours) | `$CASTLE_HOME/code/<name>/` |
|
||||
| Program source (castle's) | `<repo>/<name>` (via `source: repo:<name>`) |
|
||||
| Secrets | `$CASTLE_HOME/secrets/<NAME>` |
|
||||
| Generated Caddyfile | `$CASTLE_HOME/artifacts/specs/Caddyfile` |
|
||||
| Built frontends | `$CASTLE_HOME/artifacts/content/<name>/` |
|
||||
| **Service data** | **`$CASTLE_DATA_DIR/<name>/` (default `/data/castle/<name>/`)** |
|
||||
| Systemd units | `~/.config/systemd/user/castle-*.service` |
|
||||
| Systemd timers | `~/.config/systemd/user/castle-*.timer` |
|
||||
|
||||
Defined in `core/src/castle_core/config.py`: `CASTLE_HOME` (with derived
|
||||
`CODE_DIR`, `SECRETS_DIR`, `SPECS_DIR`, `CONTENT_DIR`) and the independent
|
||||
`DATA_DIR` (`CASTLE_DATA_DIR`). `castle deploy` passes each service its data
|
||||
path via the generated `<PREFIX>_DATA_DIR` env var. Systemd unit/timer paths are
|
||||
fixed by systemd's user-unit convention.
|
||||
|
||||
## Manifest models
|
||||
|
||||
The Pydantic models live in `core/src/castle_core/manifest.py`. Key classes:
|
||||
|
||||
@@ -31,8 +31,9 @@ is self-sufficient. The mesh is optional.
|
||||
operates above the line.
|
||||
|
||||
5. **Separate source from runtime.** The repo is for development. The
|
||||
runtime lives in standard Unix locations (`~/.castle/`, `/data/castle/`,
|
||||
systemd units). Nothing running should point into the source tree.
|
||||
runtime lives in standard Unix locations (`$CASTLE_HOME`, default
|
||||
`~/.castle/`, plus systemd units). Nothing running should point into the
|
||||
source tree.
|
||||
|
||||
6. **AI-manageable.** The CLI and API exist so that AI assistants can
|
||||
discover, create, and manage components programmatically. Humans
|
||||
@@ -113,9 +114,10 @@ Manages running processes using standard Linux infrastructure.
|
||||
- TLS termination
|
||||
|
||||
**Filesystem** handles storage:
|
||||
- Service data: `/data/castle/<name>/`
|
||||
- Secrets: `~/.castle/secrets/`
|
||||
- Generated config: `~/.castle/generated/`
|
||||
- Service data: `$CASTLE_DATA_DIR/<name>/` (default `/data/castle/`, on a
|
||||
dedicated volume)
|
||||
- Secrets: `$CASTLE_HOME/secrets/` (default `~/.castle/secrets/`)
|
||||
- Generated config: `$CASTLE_HOME/artifacts/specs/` (Caddyfile, registry.yaml)
|
||||
|
||||
Castle generates systemd unit files and Caddyfile entries from the
|
||||
registry. It doesn't run a daemon itself — it configures OS-level
|
||||
@@ -123,8 +125,8 @@ infrastructure and gets out of the way.
|
||||
|
||||
Critically, the runtime layer references only standard paths — never
|
||||
the source tree. Systemd units point to installed binaries (on PATH
|
||||
or in `~/.castle/bin/`), not to repo subdirectories. Caddy serves
|
||||
from `~/.castle/static/`, not from build output directories in the repo.
|
||||
or in `~/.local/bin/`), not to repo subdirectories. Caddy serves
|
||||
from `$CASTLE_HOME/artifacts/content/`, not from build output directories in the repo.
|
||||
|
||||
### Registry Layer
|
||||
|
||||
@@ -156,7 +158,7 @@ These map to two files:
|
||||
programs:
|
||||
central-context:
|
||||
description: Content storage API
|
||||
source: components/central-context
|
||||
source: code/central-context
|
||||
|
||||
services:
|
||||
central-context:
|
||||
@@ -197,7 +199,7 @@ Convention-based env vars (`<PREFIX>_PORT`, `<PREFIX>_DATA_DIR`) are
|
||||
generated automatically during deploy. Only non-convention values need
|
||||
`defaults.env`.
|
||||
|
||||
**`~/.castle/registry.yaml`** (per-node, not in the repo) — Node config:
|
||||
**`$CASTLE_HOME/artifacts/specs/registry.yaml`** (per-node, not in the repo, generated by `castle deploy`) — Node config:
|
||||
|
||||
```yaml
|
||||
node:
|
||||
@@ -209,7 +211,7 @@ deployed:
|
||||
runner: python
|
||||
run_cmd: [/home/user/.local/bin/central-context]
|
||||
env:
|
||||
CENTRAL_CONTEXT_DATA_DIR: /data/castle/central-context
|
||||
CENTRAL_CONTEXT_DATA_DIR: /home/user/.castle/data/central-context
|
||||
CENTRAL_CONTEXT_PORT: "9001"
|
||||
behavior: daemon
|
||||
stack: python-fastapi
|
||||
@@ -302,7 +304,7 @@ Local paths always take precedence.
|
||||
### Dashboard
|
||||
|
||||
The web dashboard (`castle-app`) is a React SPA served by Caddy from
|
||||
`~/.castle/static/castle-app/`. It talks to `castle-api` via the
|
||||
`$CASTLE_HOME/artifacts/content/castle-app/`. It talks to `castle-api` via the
|
||||
gateway proxy at `/api`.
|
||||
|
||||
**Layout:**
|
||||
@@ -417,9 +419,10 @@ Each step is distinct:
|
||||
execute them implicitly.
|
||||
|
||||
2. **Install** — Makes the artifact available on the system. For tools:
|
||||
`uv tool install` or compiled binary placed in `~/.castle/bin/`. For
|
||||
`uv tool install` or compiled binary placed in `~/.local/bin/`. For
|
||||
services: same — the binary or entry point is on PATH or in a known
|
||||
location. For frontends: built assets copied to `~/.castle/static/`.
|
||||
location. For frontends: built assets copied to
|
||||
`$CASTLE_HOME/artifacts/content/`.
|
||||
|
||||
3. **Deploy** — Materializes the runtime configuration. Reads the
|
||||
component spec, merges with node config, generates systemd units
|
||||
@@ -427,28 +430,32 @@ Each step is distinct:
|
||||
the source tree. Enables and starts services.
|
||||
|
||||
For compiled languages (Rust, Go), build produces a standalone binary
|
||||
and install is just placing it in `~/.castle/bin/`. For interpreted
|
||||
and install is just placing it in `~/.local/bin/`. For interpreted
|
||||
languages (Python, Node), the runtime wrapper (uv, node) handles
|
||||
finding the installed artifact.
|
||||
|
||||
## Runtime Filesystem Layout
|
||||
|
||||
What already exists and what the target looks like:
|
||||
Two roots, each overridable by an env var: `$CASTLE_HOME` (config, code,
|
||||
artifacts, secrets; default `~/.castle`) and `$CASTLE_DATA_DIR` (bulk program
|
||||
data; default `/data/castle`, kept on a dedicated volume):
|
||||
|
||||
```
|
||||
~/.castle/ ← Castle runtime home
|
||||
├── registry.yaml ← Node config (what's deployed here)
|
||||
├── generated/ ← Generated Caddyfile
|
||||
│ └── Caddyfile
|
||||
├── secrets/ ← Secret files (NAME → value)
|
||||
│ └── PROTONMAIL_API_KEY
|
||||
├── bin/ ← Compiled binaries, shims
|
||||
│ └── my-go-tool
|
||||
└── static/ ← Built frontend assets
|
||||
└── castle-app/
|
||||
└── dist/
|
||||
$CASTLE_HOME/ ← Config & artifacts (default ~/.castle)
|
||||
├── castle.yaml ← Registry spec (programs, services, jobs)
|
||||
├── infra.conf ← Infrastructure install choices
|
||||
├── code/ ← Program source (your programs)
|
||||
│ └── <name>/
|
||||
├── artifacts/
|
||||
│ ├── specs/ ← Generated by `castle deploy`
|
||||
│ │ ├── Caddyfile
|
||||
│ │ └── registry.yaml ← Node config (what's deployed here)
|
||||
│ └── content/ ← Built frontend assets
|
||||
│ └── castle-app/ ← (index.html + assets, served at root)
|
||||
└── secrets/ ← Secret files (NAME → value)
|
||||
└── PROTONMAIL_API_KEY
|
||||
|
||||
/data/castle/ ← Persistent service data
|
||||
$CASTLE_DATA_DIR/ ← Persistent service data (default /data/castle)
|
||||
└── <name>/
|
||||
|
||||
~/.config/systemd/user/ ← Systemd units (standard location)
|
||||
@@ -458,8 +465,11 @@ What already exists and what the target looks like:
|
||||
└── ...
|
||||
```
|
||||
|
||||
Compiled-language tools (Rust, Go — planned) install their binaries to the
|
||||
standard `~/.local/bin/`, not under `$CASTLE_HOME`.
|
||||
|
||||
Source (the repo) is referenced only during build and install. Everything
|
||||
the runtime touches lives in `~/.castle/`, `/data/castle/`, or standard
|
||||
the runtime touches lives under `$CASTLE_HOME`, `$CASTLE_DATA_DIR`, or standard
|
||||
systemd paths.
|
||||
|
||||
## OTP as Design Guide
|
||||
@@ -470,7 +480,7 @@ Castle's architecture parallels Erlang/OTP, mapped onto Unix:
|
||||
|-------------|------------------|
|
||||
| Application | Component (independent, self-contained) |
|
||||
| Application resource file | Component spec in `castle.yaml` |
|
||||
| Release config (sys.config) | Node config in `~/.castle/registry.yaml` |
|
||||
| Release config (sys.config) | Node config in `$CASTLE_HOME/artifacts/specs/registry.yaml` |
|
||||
| Release assembly | `castle deploy` (spec + node config → runtime) |
|
||||
| Supervisor | systemd (restart policies, ordering) |
|
||||
| Process | Running service/worker/job |
|
||||
@@ -508,17 +518,17 @@ What exists today:
|
||||
- **Three packages** — `castle-core` (models, config, generators),
|
||||
`castle-cli` (commands), `castle-api` (HTTP API)
|
||||
- **Source/runtime split** — `castle.yaml` (spec) → `castle deploy` →
|
||||
`~/.castle/registry.yaml` (node config). Systemd units and Caddyfile
|
||||
generated from registry with fully resolved paths. No repo references
|
||||
`$CASTLE_HOME/artifacts/specs/registry.yaml` (node config). Systemd units and
|
||||
Caddyfile generated from registry with fully resolved paths. No repo references
|
||||
in runtime artifacts.
|
||||
- **Convention-based env generation** — `castle deploy` auto-generates
|
||||
`<PREFIX>_DATA_DIR=/data/castle/<name>` and `<PREFIX>_PORT` from
|
||||
`<PREFIX>_DATA_DIR=$CASTLE_DATA_DIR/<name>` and `<PREFIX>_PORT` from
|
||||
the manifest. Only non-convention values need `defaults.env`.
|
||||
- **Gateway** — Caddy on port 9000, Caddyfile generated from registry
|
||||
- **API** — `castle-api` on port 9020, reads from registry (optional
|
||||
castle.yaml fallback for non-deployed components)
|
||||
- **Dashboard** — `castle-app` React/Vite frontend, static assets
|
||||
served from `~/.castle/static/castle-app/`
|
||||
served from `$CASTLE_HOME/artifacts/content/castle-app/`
|
||||
- **Services** — central-context (content storage), notification-bridge
|
||||
(desktop notification forwarder)
|
||||
- **Jobs** — protonmail (email sync every 5 min), backup-collect (nightly),
|
||||
@@ -531,7 +541,7 @@ What exists today:
|
||||
Opt-in via `CASTLE_API_MQTT_ENABLED` / `CASTLE_API_MDNS_ENABLED`.
|
||||
- **MQTT broker** — Mosquitto running as `castle-mqtt` Docker container
|
||||
on port 1883, managed by systemd. Config and data in
|
||||
`/data/castle/castle-mqtt/`.
|
||||
`$CASTLE_DATA_DIR/castle-mqtt/`.
|
||||
- **Node API** — `GET /mesh/status`, `GET /nodes`, `GET /nodes/{hostname}`.
|
||||
`GET /components?include_remote=true` for cross-node component listing.
|
||||
- **Gateway panel** — Dedicated UI showing route table, health per route,
|
||||
@@ -559,7 +569,7 @@ What doesn't exist yet:
|
||||
| Process supervision | systemd (user units) | Active |
|
||||
| HTTP routing | Caddy (port 9000) | Active |
|
||||
| Component specs | castle.yaml + Pydantic models | Active |
|
||||
| Node config | `~/.castle/registry.yaml` | Active |
|
||||
| Node config | `$CASTLE_HOME/artifacts/specs/registry.yaml` | Active |
|
||||
| CLI | castle (Python, uv) | Active |
|
||||
| API | castle-api (FastAPI) | Active |
|
||||
| Dashboard | castle-app (React, Vite, shadcn/ui) | Active |
|
||||
@@ -569,7 +579,7 @@ What doesn't exist yet:
|
||||
| Type checking | pyright (Python), tsc (TS) | Active |
|
||||
| Testing | pytest (Python), Vitest (TS) | Active |
|
||||
| Secrets | `~/.castle/secrets/` file-based | Active |
|
||||
| Data storage | Filesystem (`/data/castle/`) | Active |
|
||||
| Data storage | Filesystem (`$CASTLE_DATA_DIR/`, default `/data/castle/`) | Active |
|
||||
| Messaging | MQTT (paho-mqtt client, Mosquitto broker) | Active (opt-in) |
|
||||
| Node discovery | mDNS (python-zeroconf) + MQTT | Active (opt-in) |
|
||||
| Rust packaging | cargo | Planned |
|
||||
|
||||
@@ -24,10 +24,10 @@ How to build CLI tools following Unix philosophy.
|
||||
|
||||
## Project layout
|
||||
|
||||
Each tool is an independent project under `components/` with its own `pyproject.toml`:
|
||||
Each tool is an independent project under `~/.castle/code/` with its own `pyproject.toml`:
|
||||
|
||||
```
|
||||
components/my-tool/
|
||||
~/.castle/code/my-tool/
|
||||
├── src/my_tool/
|
||||
│ ├── __init__.py
|
||||
│ └── main.py # Entry point
|
||||
@@ -37,13 +37,13 @@ components/my-tool/
|
||||
└── CLAUDE.md
|
||||
```
|
||||
|
||||
Examples: `components/pdf2md/`, `components/gpt/`, `components/protonmail/`
|
||||
Examples: `code/pdf2md/`, `code/gpt/`, `code/protonmail/`
|
||||
|
||||
## Creating a new tool
|
||||
|
||||
```bash
|
||||
castle create my-tool --stack python-cli --description "Does something"
|
||||
cd components/my-tool && uv sync
|
||||
cd ~/.castle/code/my-tool && uv sync
|
||||
```
|
||||
|
||||
This scaffolds the project and registers it in `castle.yaml`.
|
||||
@@ -317,7 +317,7 @@ uv run ruff format . # Format
|
||||
programs:
|
||||
my-tool:
|
||||
description: Does something useful
|
||||
source: components/my-tool
|
||||
source: code/my-tool
|
||||
stack: python-cli
|
||||
behavior: tool
|
||||
```
|
||||
@@ -328,7 +328,7 @@ Tools with system dependencies declare them directly on the program:
|
||||
programs:
|
||||
pdf2md:
|
||||
description: Convert PDF files to Markdown
|
||||
source: components/pdf2md
|
||||
source: code/pdf2md
|
||||
stack: python-cli
|
||||
behavior: tool
|
||||
system_dependencies: [pandoc, poppler-utils]
|
||||
|
||||
@@ -102,7 +102,7 @@ Castle passes config via env vars in castle.yaml:
|
||||
programs:
|
||||
my-service:
|
||||
description: Does something useful
|
||||
source: components/my-service
|
||||
source: code/my-service
|
||||
stack: python-fastapi
|
||||
behavior: daemon
|
||||
|
||||
@@ -292,12 +292,15 @@ Mapping convention:
|
||||
Castle services use filesystem storage with JSON metadata sidecars:
|
||||
|
||||
```
|
||||
/data/castle/my-service/
|
||||
$CASTLE_DATA_DIR/my-service/ # default /data/castle/my-service/
|
||||
└── bucket/
|
||||
├── item-name
|
||||
└── item-name.meta.json
|
||||
```
|
||||
|
||||
The service receives this path via the generated `<PREFIX>_DATA_DIR` env var —
|
||||
it never hardcodes it. Use the `data_dir` setting from your config.
|
||||
|
||||
```python
|
||||
# storage.py
|
||||
import hashlib
|
||||
|
||||
@@ -20,7 +20,7 @@ How to build, serve, and manage web frontends as castle components.
|
||||
|
||||
```bash
|
||||
# Create the project
|
||||
mkdir components/my-frontend && cd components/my-frontend
|
||||
mkdir ~/.castle/code/my-frontend && cd ~/.castle/code/my-frontend
|
||||
pnpm create vite . --template react-ts
|
||||
|
||||
# Core dependencies
|
||||
@@ -116,7 +116,7 @@ handles serving directly from the build output.
|
||||
programs:
|
||||
my-frontend:
|
||||
description: Web dashboard
|
||||
source: components/my-frontend
|
||||
source: code/my-frontend
|
||||
build:
|
||||
commands:
|
||||
- ["pnpm", "build"]
|
||||
@@ -124,7 +124,8 @@ programs:
|
||||
- dist/
|
||||
```
|
||||
|
||||
For production, Caddy serves the static `dist/` output directly — no
|
||||
For production, `castle deploy` copies the build output to
|
||||
`~/.castle/artifacts/content/<name>/` and Caddy serves it from there — no
|
||||
Node process needed. See [Serving with Caddy](#serving-with-caddy) below.
|
||||
|
||||
For development with Vite's dev server, add a service entry:
|
||||
@@ -148,12 +149,23 @@ See @docs/component-registry.md for the full registry reference.
|
||||
|
||||
## Serving with Caddy
|
||||
|
||||
For production, serve the static `dist/` output directly from Caddy rather than
|
||||
running a Node process. The gateway Caddyfile can serve the files:
|
||||
For production, the static build output is served by Caddy rather than a Node
|
||||
process. You do **not** write this block by hand — `castle deploy` generates it.
|
||||
The flow:
|
||||
|
||||
1. `castle deploy` runs the program's `build.commands`, then copies each
|
||||
`build.outputs` directory from `~/.castle/code/<name>/` into
|
||||
`~/.castle/artifacts/content/<name>/` (`core/src/castle_core/deploy.py`,
|
||||
`_copy_app_static`).
|
||||
2. The Caddyfile generator scans `~/.castle/artifacts/content/`; any directory
|
||||
containing an `index.html` is served as a SPA at a path prefix matching its
|
||||
name. `castle-app` is special-cased to serve at the root `/`.
|
||||
|
||||
The generated block looks like this (written to `~/.castle/artifacts/specs/Caddyfile`):
|
||||
|
||||
```caddyfile
|
||||
handle_path /app/* {
|
||||
root * /data/repos/castle/my-frontend/dist
|
||||
handle_path /my-frontend/* {
|
||||
root * /home/payne/.castle/artifacts/content/my-frontend
|
||||
try_files {path} /index.html
|
||||
file_server
|
||||
}
|
||||
@@ -161,7 +173,8 @@ handle_path /app/* {
|
||||
|
||||
The `try_files {path} /index.html` directive is essential for SPA routing —
|
||||
it falls back to `index.html` for any path that doesn't match a static file,
|
||||
letting React Router handle client-side routes.
|
||||
letting React Router handle client-side routes. The serving prefix is derived
|
||||
from the program name, not hand-configured.
|
||||
|
||||
## API integration
|
||||
|
||||
|
||||
Reference in New Issue
Block a user