refactor: Decouple roles.
This commit is contained in:
@@ -1,19 +1,29 @@
|
||||
# Component Registry
|
||||
|
||||
How castle tracks, configures, and manages components. This is the central
|
||||
reference for `castle.yaml` structure and the manifest architecture.
|
||||
reference for `castle.yaml` structure and the registry architecture.
|
||||
|
||||
## castle.yaml
|
||||
|
||||
The single source of truth for all components. Lives at the repo root.
|
||||
Three top-level sections:
|
||||
|
||||
```yaml
|
||||
gateway:
|
||||
port: 9000
|
||||
|
||||
components:
|
||||
my-service:
|
||||
my-tool:
|
||||
description: Does something useful
|
||||
source: components/my-tool
|
||||
install:
|
||||
path: { alias: my-tool }
|
||||
tool:
|
||||
system_dependencies: [pandoc]
|
||||
|
||||
services:
|
||||
my-service:
|
||||
component: my-service
|
||||
run:
|
||||
runner: python
|
||||
tool: my-service
|
||||
@@ -25,16 +35,84 @@ components:
|
||||
caddy: { path_prefix: /my-service }
|
||||
manage:
|
||||
systemd: {}
|
||||
|
||||
jobs:
|
||||
my-job:
|
||||
component: my-tool
|
||||
run:
|
||||
runner: command
|
||||
argv: [my-tool, sync]
|
||||
schedule: "0 2 * * *"
|
||||
manage:
|
||||
systemd: {}
|
||||
```
|
||||
|
||||
## Manifest blocks
|
||||
### Section semantics
|
||||
|
||||
Each component declares **what it does** through these optional blocks:
|
||||
| Section | Purpose | Category |
|
||||
|---------|---------|----------|
|
||||
| `components:` | Software catalog — what exists | tool, frontend, component |
|
||||
| `services:` | Long-running daemons — how they run | service |
|
||||
| `jobs:` | Scheduled tasks — when they run | job |
|
||||
|
||||
### `run` — How to start it
|
||||
Services and jobs can reference a component via `component:` for description
|
||||
fallthrough and source code linking. They can also exist independently
|
||||
(e.g., `castle-gateway` runs Caddy — not our software).
|
||||
|
||||
Discriminated union on `runner`. The runner encodes both the language/toolchain
|
||||
(used by `castle sync`) and the deployment resolution (used by `castle deploy`):
|
||||
## Component blocks
|
||||
|
||||
Components define **what software exists** — identity, source, tools, builds.
|
||||
|
||||
### `source` — Where the source lives
|
||||
|
||||
```yaml
|
||||
source: components/my-tool
|
||||
```
|
||||
|
||||
Relative path from repo root to the project directory.
|
||||
|
||||
### `install` — How to install it
|
||||
|
||||
```yaml
|
||||
install:
|
||||
path:
|
||||
alias: my-tool # Command name in PATH
|
||||
```
|
||||
|
||||
Creates a shim so the tool is available system-wide after
|
||||
`uv tool install --editable .`.
|
||||
|
||||
### `tool` — Tool metadata
|
||||
|
||||
```yaml
|
||||
tool:
|
||||
version: "1.0.0"
|
||||
system_dependencies: [pandoc, poppler-utils]
|
||||
```
|
||||
|
||||
This block provides metadata for `castle tool list` and the dashboard.
|
||||
It's separate from `install` (which handles PATH registration). The source
|
||||
directory is set via the top-level `source` field on the component, not here.
|
||||
|
||||
### `build` — How to build it
|
||||
|
||||
```yaml
|
||||
build:
|
||||
commands:
|
||||
- ["pnpm", "build"]
|
||||
outputs:
|
||||
- dist/
|
||||
```
|
||||
|
||||
Components with build outputs are categorized as **frontends** in the UI.
|
||||
|
||||
## Service blocks
|
||||
|
||||
Services define **how long-running daemons are deployed**.
|
||||
|
||||
### `run` — How to start it (required)
|
||||
|
||||
Discriminated union on `runner`:
|
||||
|
||||
| Runner | Sync | Deploy | Key fields |
|
||||
|--------|------|--------|------------|
|
||||
@@ -44,38 +122,22 @@ Discriminated union on `runner`. The runner encodes both the language/toolchain
|
||||
| `node` | `package_manager install` | `package_manager run script` | `script`, `package_manager` |
|
||||
| `remote` | *(none)* | *(none — no local process)* | `base_url`, `health_url` |
|
||||
|
||||
**Services** use `python`:
|
||||
```yaml
|
||||
run:
|
||||
runner: python
|
||||
tool: my-service # name in [project.scripts]
|
||||
```
|
||||
|
||||
**Tools invoked by castle** (jobs, scheduled tasks) use `command`:
|
||||
```yaml
|
||||
run:
|
||||
runner: command
|
||||
argv: ["protonmail", "sync"]
|
||||
cwd: protonmail
|
||||
env:
|
||||
PROTONMAIL_USERNAME: user@example.com
|
||||
```
|
||||
|
||||
**Standalone tools** that users invoke directly often have no `run` block at
|
||||
all — castle just installs them to PATH.
|
||||
|
||||
### `expose` — What it exposes
|
||||
|
||||
```yaml
|
||||
expose:
|
||||
http:
|
||||
internal:
|
||||
port: 9001 # Required for services
|
||||
port: 9001 # Required for HTTP services
|
||||
health_path: /health # Used by health polling
|
||||
```
|
||||
|
||||
Having `expose.http` gives the component the **service** role.
|
||||
|
||||
### `proxy` — How to proxy it
|
||||
|
||||
```yaml
|
||||
@@ -95,7 +157,7 @@ manage:
|
||||
```
|
||||
|
||||
Enables `castle service enable/disable` and `castle logs`. An empty `{}`
|
||||
uses defaults (enable=true, restart=on-failure, restart_sec=5).
|
||||
uses defaults (enable=true, restart=on-failure, restart_sec=2).
|
||||
|
||||
Full options:
|
||||
```yaml
|
||||
@@ -107,91 +169,39 @@ manage:
|
||||
no_new_privileges: true
|
||||
after: [network.target, castle-other.service]
|
||||
wanted_by: [default.target]
|
||||
exec_reload: "caddy reload ..."
|
||||
```
|
||||
|
||||
### `install` — How to install it
|
||||
### `defaults` — Default environment
|
||||
|
||||
```yaml
|
||||
install:
|
||||
path:
|
||||
alias: my-tool # Command name in PATH
|
||||
```
|
||||
|
||||
Creates a shim so the tool is available system-wide after
|
||||
`uv tool install --editable .`.
|
||||
|
||||
### `tool` — Tool metadata
|
||||
|
||||
```yaml
|
||||
tool:
|
||||
source: components/my-tool/ # Source directory
|
||||
version: "1.0.0"
|
||||
system_dependencies: [pandoc, poppler-utils]
|
||||
```
|
||||
|
||||
This block provides metadata for `castle tool list` and the dashboard.
|
||||
It's separate from `install` (which handles PATH registration) and `run`
|
||||
(which handles execution).
|
||||
|
||||
The install method (uv tool install vs symlink) is inferred from the source
|
||||
directory: if `pyproject.toml` exists, it's a Python package; if the source
|
||||
is a file, it's symlinked.
|
||||
|
||||
### `build` — How to build it
|
||||
|
||||
```yaml
|
||||
build:
|
||||
commands:
|
||||
- ["pnpm", "build"]
|
||||
outputs:
|
||||
- dist/
|
||||
```
|
||||
|
||||
Having build outputs gives the component the **frontend** role.
|
||||
|
||||
### `triggers` — What triggers it
|
||||
|
||||
```yaml
|
||||
triggers:
|
||||
- type: schedule
|
||||
cron: "*/5 * * * *"
|
||||
timezone: America/Los_Angeles # default
|
||||
```
|
||||
|
||||
Having a schedule trigger gives the component the **job** role.
|
||||
Castle generates a systemd .timer file alongside the .service unit.
|
||||
|
||||
Other trigger types: `manual`, `event` (source + topic), `request` (protocol).
|
||||
|
||||
### `env` with secrets
|
||||
|
||||
Environment variables can reference secrets stored in `~/.castle/secrets/`:
|
||||
|
||||
```yaml
|
||||
run:
|
||||
defaults:
|
||||
env:
|
||||
CENTRAL_CONTEXT_URL: http://localhost:9001
|
||||
API_KEY: ${secret:MY_API_KEY}
|
||||
```
|
||||
|
||||
Castle resolves `${secret:NAME}` by reading `~/.castle/secrets/NAME`.
|
||||
Never store secrets in castle.yaml or project directories.
|
||||
|
||||
## Role derivation
|
||||
## Job blocks
|
||||
|
||||
Roles are **computed** from manifest declarations, never set manually:
|
||||
Jobs define **how scheduled tasks run**. Same blocks as services plus
|
||||
`schedule` and `timezone`.
|
||||
|
||||
| Role | Derived when |
|
||||
|------|-------------|
|
||||
| **service** | Has `expose.http` |
|
||||
| **tool** | Has `install.path` or has `tool` spec (fallback) |
|
||||
| **worker** | Has `manage.systemd` but no `expose.http` |
|
||||
| **job** | Has trigger with `type: schedule` |
|
||||
| **frontend** | Has `build` with outputs or commands |
|
||||
| **containerized** | Runner is `container` |
|
||||
| **remote** | Runner is `remote` |
|
||||
### `schedule` — Cron expression (required)
|
||||
|
||||
A component can have multiple roles. For example, `protonmail` is both a
|
||||
**tool** (installed to PATH) and a **job** (runs on a cron schedule).
|
||||
```yaml
|
||||
schedule: "*/5 * * * *"
|
||||
timezone: America/Los_Angeles # default
|
||||
```
|
||||
|
||||
Castle generates a systemd `.timer` file alongside the `.service` unit.
|
||||
|
||||
### Other blocks
|
||||
|
||||
Jobs also support `run` (required), `manage`, and `defaults` — same
|
||||
semantics as services.
|
||||
|
||||
## Registering a new component
|
||||
|
||||
@@ -207,17 +217,38 @@ castle create my-tool --type tool --description "Does something"
|
||||
|
||||
### Manually
|
||||
|
||||
Add an entry to the `components:` section of `castle.yaml`:
|
||||
Add entries to the appropriate sections of `castle.yaml`:
|
||||
|
||||
```yaml
|
||||
# Tool — only needs a component entry
|
||||
components:
|
||||
my-tool:
|
||||
description: Does something useful
|
||||
tool:
|
||||
source: components/my-tool/
|
||||
source: components/my-tool
|
||||
install:
|
||||
path:
|
||||
alias: my-tool
|
||||
|
||||
# Service — needs both component and service entries
|
||||
components:
|
||||
my-service:
|
||||
description: Does something useful
|
||||
source: components/my-service
|
||||
|
||||
services:
|
||||
my-service:
|
||||
component: my-service
|
||||
run:
|
||||
runner: python
|
||||
tool: my-service
|
||||
expose:
|
||||
http:
|
||||
internal: { port: 9001 }
|
||||
health_path: /health
|
||||
proxy:
|
||||
caddy: { path_prefix: /my-service }
|
||||
manage:
|
||||
systemd: {}
|
||||
```
|
||||
|
||||
## Lifecycle
|
||||
@@ -254,21 +285,18 @@ uv tool install --editable components/my-tool/ # 4. Install to PATH
|
||||
|
||||
### Job lifecycle
|
||||
|
||||
Jobs are tools or services with a schedule trigger. They need both `run`
|
||||
(so castle knows how to execute them) and `manage.systemd` (so systemd
|
||||
handles the timer):
|
||||
Jobs are defined in the `jobs:` section with a `run` spec and `schedule`:
|
||||
|
||||
```yaml
|
||||
my-job:
|
||||
description: Runs nightly
|
||||
run:
|
||||
runner: command
|
||||
argv: ["my-job"]
|
||||
triggers:
|
||||
- type: schedule
|
||||
cron: "0 2 * * *"
|
||||
manage:
|
||||
systemd: {}
|
||||
jobs:
|
||||
my-job:
|
||||
description: Runs nightly
|
||||
run:
|
||||
runner: command
|
||||
argv: ["my-job"]
|
||||
schedule: "0 2 * * *"
|
||||
manage:
|
||||
systemd: {}
|
||||
```
|
||||
|
||||
`castle service enable my-job` generates both a `.service` (Type=oneshot)
|
||||
@@ -289,14 +317,16 @@ and a `.timer` file.
|
||||
|
||||
The Pydantic models live in `core/src/castle_core/manifest.py`. Key classes:
|
||||
|
||||
- `ComponentManifest` — top-level model, has `roles` computed property
|
||||
- `ComponentSpec` — software catalog entry (source, install, tool, build)
|
||||
- `ServiceSpec` — long-running daemon (run, expose, proxy, manage, defaults)
|
||||
- `JobSpec` — scheduled task (run, schedule, manage, defaults)
|
||||
- `RunSpec` — discriminated union (RunPython, RunCommand, RunContainer, RunNode, RunRemote)
|
||||
- `TriggerSpec` — union (TriggerSchedule, TriggerManual, TriggerEvent, TriggerRequest)
|
||||
- `ExposeSpec`, `ProxySpec`, `ManageSpec`, `InstallSpec`, `ToolSpec`, `BuildSpec`
|
||||
- `CaddySpec`, `SystemdSpec`, `HttpExposeSpec`, `HttpInternal`
|
||||
|
||||
Config loading: `core/src/castle_core/config.py` — `load_config()` parses
|
||||
castle.yaml into `CastleConfig` with typed `components` dict.
|
||||
castle.yaml into `CastleConfig` with typed `components`, `services`, and
|
||||
`jobs` dicts.
|
||||
|
||||
Infrastructure generators: `core/src/castle_core/generators/` — systemd unit/timer
|
||||
generation (`systemd.py`) and Caddyfile generation (`caddyfile.py`).
|
||||
|
||||
@@ -20,9 +20,9 @@ is self-sufficient. The mesh is optional.
|
||||
Castle service is just a well-behaved Unix daemon that happens to
|
||||
be registered in a manifest.
|
||||
|
||||
3. **Declare capabilities, derive roles.** Components say what they
|
||||
do (expose HTTP, run on a schedule, install to PATH). Castle infers
|
||||
what they are (service, job, tool, frontend). No role labels.
|
||||
3. **Section is category.** Components, services, and jobs live in
|
||||
separate sections of `castle.yaml`. The section determines the
|
||||
category — no role derivation needed.
|
||||
|
||||
4. **Language-agnostic above the build line.** Below the build line,
|
||||
every language is different (uv, pnpm, cargo, go). Above it,
|
||||
@@ -149,13 +149,17 @@ answers: "is it working?"
|
||||
|
||||
These map to two files:
|
||||
|
||||
**`castle.yaml`** (in the repo, version-controlled) — Component specs:
|
||||
**`castle.yaml`** (in the repo, version-controlled) — Three sections:
|
||||
|
||||
```yaml
|
||||
components:
|
||||
central-context:
|
||||
description: Content storage API
|
||||
source: components/central-context
|
||||
|
||||
services:
|
||||
central-context:
|
||||
component: central-context
|
||||
run:
|
||||
runner: python
|
||||
tool: central-context
|
||||
@@ -168,12 +172,29 @@ components:
|
||||
path_prefix: /central-context
|
||||
manage:
|
||||
systemd: {}
|
||||
|
||||
jobs:
|
||||
backup-collect:
|
||||
component: backup-collect
|
||||
run:
|
||||
runner: command
|
||||
argv: [backup-collect]
|
||||
schedule: "0 2 * * *"
|
||||
manage:
|
||||
systemd: {}
|
||||
```
|
||||
|
||||
The spec says what the component *is* and what it *needs* — a port, a
|
||||
data directory, HTTP exposure. Convention-based env vars (`<PREFIX>_PORT`,
|
||||
`<PREFIX>_DATA_DIR`) are generated automatically during deploy. Only
|
||||
non-convention values need `defaults.env`.
|
||||
Components define *what software exists* (identity, source, install, tools).
|
||||
Services define *how daemons run* (run config, expose, proxy, systemd).
|
||||
Jobs define *how scheduled tasks run* (run config, cron schedule, systemd).
|
||||
|
||||
Services and jobs can reference a component via `component:` for description
|
||||
fallthrough and source code linking. They can also exist independently
|
||||
(e.g., `castle-gateway` runs Caddy — not our software).
|
||||
|
||||
Convention-based env vars (`<PREFIX>_PORT`, `<PREFIX>_DATA_DIR`) are
|
||||
generated automatically during deploy. Only non-convention values need
|
||||
`defaults.env`.
|
||||
|
||||
**`~/.castle/registry.yaml`** (per-node, not in the repo) — Node config:
|
||||
|
||||
@@ -189,7 +210,7 @@ deployed:
|
||||
env:
|
||||
CENTRAL_CONTEXT_DATA_DIR: /data/castle/central-context
|
||||
CENTRAL_CONTEXT_PORT: "9001"
|
||||
roles: [service]
|
||||
category: service
|
||||
port: 9001
|
||||
health_path: /health
|
||||
proxy_path: /central-context
|
||||
|
||||
@@ -317,28 +317,27 @@ uv run ruff format . # Format
|
||||
components:
|
||||
my-tool:
|
||||
description: Does something useful
|
||||
tool:
|
||||
source: components/my-tool/
|
||||
source: components/my-tool
|
||||
install:
|
||||
path:
|
||||
alias: my-tool
|
||||
```
|
||||
|
||||
Tools with system dependencies declare them in the manifest:
|
||||
Tools with system dependencies declare them in the component:
|
||||
|
||||
```yaml
|
||||
components:
|
||||
pdf2md:
|
||||
description: Convert PDF files to Markdown
|
||||
tool:
|
||||
source: components/pdf2md/
|
||||
system_dependencies: [pandoc, poppler-utils]
|
||||
source: components/pdf2md
|
||||
install:
|
||||
path:
|
||||
alias: pdf2md
|
||||
tool:
|
||||
system_dependencies: [pandoc, poppler-utils]
|
||||
```
|
||||
|
||||
Tools with `install.path` get the **tool** role. They don't need `expose`,
|
||||
`proxy`, or `manage` blocks unless castle also runs them (e.g., scheduled jobs).
|
||||
Tools live in the `components:` section. If a tool also runs on a schedule,
|
||||
add a separate entry in the `jobs:` section referencing the component.
|
||||
|
||||
See @docs/component-registry.md for the full manifest reference.
|
||||
See @docs/component-registry.md for the full registry reference.
|
||||
|
||||
@@ -101,7 +101,12 @@ Castle passes config via env vars in castle.yaml:
|
||||
```yaml
|
||||
components:
|
||||
my-service:
|
||||
source: my-service
|
||||
description: Does something useful
|
||||
source: components/my-service
|
||||
|
||||
services:
|
||||
my-service:
|
||||
component: my-service
|
||||
run:
|
||||
runner: python
|
||||
tool: my-service
|
||||
|
||||
@@ -107,8 +107,8 @@ The `build` output is a static SPA in `dist/` — just HTML, JS, and CSS files.
|
||||
|
||||
## Registering as a castle component
|
||||
|
||||
A frontend component has a `build` spec (produces static output) and optionally
|
||||
a `proxy` spec (Caddy serves the built files). No `run` block needed if Caddy
|
||||
A frontend component has a `build` spec (produces static output). Register it
|
||||
in the `components:` section of `castle.yaml`. No `run` block needed if Caddy
|
||||
handles serving directly from the build output.
|
||||
|
||||
```yaml
|
||||
@@ -116,45 +116,33 @@ handles serving directly from the build output.
|
||||
components:
|
||||
my-frontend:
|
||||
description: Web dashboard
|
||||
source: my-frontend
|
||||
build:
|
||||
commands:
|
||||
- ["pnpm", "build"]
|
||||
outputs:
|
||||
- dist/
|
||||
proxy:
|
||||
caddy:
|
||||
path_prefix: /app
|
||||
```
|
||||
|
||||
For development with Vite's dev server, add a `run` block:
|
||||
For production, Caddy serves the static files — add a service entry with a
|
||||
proxy spec:
|
||||
|
||||
```yaml
|
||||
components:
|
||||
services:
|
||||
my-frontend:
|
||||
description: Web dashboard
|
||||
component: my-frontend
|
||||
run:
|
||||
runner: node
|
||||
script: dev
|
||||
package_manager: pnpm
|
||||
cwd: my-frontend
|
||||
build:
|
||||
commands:
|
||||
- ["pnpm", "build"]
|
||||
outputs:
|
||||
- dist/
|
||||
expose:
|
||||
http:
|
||||
internal: { port: 5173 }
|
||||
proxy:
|
||||
caddy:
|
||||
path_prefix: /app
|
||||
caddy: { path_prefix: /app }
|
||||
```
|
||||
|
||||
This gives the component both the `frontend` role (from `build`) and the
|
||||
`service` role (from `expose.http`) during development.
|
||||
|
||||
See @docs/component-registry.md for the full manifest reference and role
|
||||
derivation rules.
|
||||
See @docs/component-registry.md for the full registry reference.
|
||||
|
||||
## Serving with Caddy
|
||||
|
||||
|
||||
Reference in New Issue
Block a user