refactor: Update component structure to use behavior and stack attributes, enhancing clarity and functionality across services, tools, and frontends

This commit is contained in:
2026-02-23 16:32:55 -08:00
parent 3343e955fd
commit 56b9c8ddf1
45 changed files with 698 additions and 667 deletions

343
docs/stacks/python-cli.md Normal file
View File

@@ -0,0 +1,343 @@
# Python Tools in Castle
How to build CLI tools following Unix philosophy.
## Principles
- Each tool does one thing well
- Read from stdin or file argument, write to stdout
- Compose via pipes: `pdf2md doc.pdf | gpt "summarize this"`
- Status messages go to stderr (don't interfere with piping)
- Exit 0 on success, 1 on error
## Stack
| Layer | Choice |
|-------|--------|
| **CLI** | argparse |
| **Package manager** | uv (never pip) |
| **Build** | hatchling |
| **Testing** | pytest |
| **Linting** | ruff (shared `ruff.toml` at repo root) |
| **Type checking** | pyright (shared `pyrightconfig.json` at repo root) |
| **Python** | 3.11+ minimum |
## Project layout
Each tool is an independent project under `components/` with its own `pyproject.toml`:
```
components/my-tool/
├── src/my_tool/
│ ├── __init__.py
│ └── main.py # Entry point
├── tests/
│ └── test_main.py
├── pyproject.toml
└── CLAUDE.md
```
Examples: `components/pdf2md/`, `components/gpt/`, `components/protonmail/`
## Creating a new tool
```bash
castle create my-tool --stack python-cli --description "Does something"
cd components/my-tool && uv sync
```
This scaffolds the project and registers it in `castle.yaml`.
## pyproject.toml
```toml
[project]
name = "my-tool"
version = "0.1.0"
description = "Does something useful"
requires-python = ">=3.11"
dependencies = []
[project.scripts]
my-tool = "my_tool.main:main"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/my_tool"]
[dependency-groups]
dev = ["pytest>=7.0.0"]
[tool.ruff.lint.isort]
known-first-party = ["my_tool"]
```
After `uv tool install --editable .`, the command is in PATH.
## Tool implementation patterns
### Simple tool: stdin/stdout
The most common pattern. Read from a file argument or stdin, process, write
to stdout.
```python
#!/usr/bin/env python3
"""
my-tool: Brief description
Usage:
my-tool [options] [FILE]
cat input.txt | my-tool
Examples:
my-tool input.txt
my-tool input.txt -o output.txt
cat input.txt | my-tool > output.txt
"""
import argparse
import sys
def process(data: str) -> str:
"""Core logic — pure function, easy to test."""
return data.upper()
def main() -> int:
parser = argparse.ArgumentParser(
description="Brief description",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument("input", nargs="?", help="Input file (default: stdin)")
parser.add_argument("-o", "--output", help="Output file (default: stdout)")
parser.add_argument("--version", action="version", version="my-tool 1.0.0")
args = parser.parse_args()
# Read
if args.input:
with open(args.input) as f:
data = f.read()
else:
data = sys.stdin.read()
# Process
result = process(data)
# Write
if args.output:
with open(args.output, "w") as f:
f.write(result)
print(f"Wrote to {args.output}", file=sys.stderr)
else:
print(result, end="")
return 0
if __name__ == "__main__":
sys.exit(main())
```
### Tool with subcommands
For complex tools with multiple operations:
```python
def cmd_init(args: argparse.Namespace) -> int:
"""Initialize a collection."""
directory = args.directory or "."
# ...
print(f"Initialized in {directory}")
return 0
def cmd_query(args: argparse.Namespace) -> int:
"""Search a collection."""
# ...
return 0
def main() -> int:
parser = argparse.ArgumentParser(description="Manage collections")
parser.add_argument("--version", action="version", version="1.0.0")
subparsers = parser.add_subparsers(dest="command")
init_p = subparsers.add_parser("init", help="Initialize")
init_p.add_argument("directory", nargs="?")
init_p.add_argument("--name", help="Collection name")
init_p.add_argument("--force", action="store_true")
query_p = subparsers.add_parser("query", help="Search")
query_p.add_argument("query", help="Search query")
query_p.add_argument("--limit", type=int, default=20)
args = parser.parse_args()
if args.command == "init":
return cmd_init(args)
elif args.command == "query":
return cmd_query(args)
else:
parser.print_help()
return 1
```
### Tool with external processes
When wrapping system commands:
```python
import subprocess
import sys
def convert(input_file: str, output_file: str) -> int:
try:
subprocess.run(
["pandoc", input_file, "-o", output_file],
check=True,
capture_output=True,
text=True,
)
return 0
except subprocess.CalledProcessError as e:
print(f"Error: {e}", file=sys.stderr)
if e.stderr:
print(e.stderr, file=sys.stderr)
return 1
except FileNotFoundError:
print("Error: pandoc not found. Install with: apt install pandoc",
file=sys.stderr)
return 1
```
Always use `check=True` and `capture_output=True` with subprocess. Handle
`FileNotFoundError` for missing system dependencies.
## Error handling
```python
def main() -> int:
try:
result = do_work()
print(result)
return 0
except SpecificError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except FileNotFoundError as e:
print(f"Error: file not found: {e}", file=sys.stderr)
return 1
```
Rules:
- Normal output goes to **stdout** (enables piping)
- Error messages go to **stderr**
- Status/progress messages go to **stderr**
- Return **0** for success, **1** for error
- Entry point: `sys.exit(main())`
## Piping
Tools compose naturally via Unix pipes:
```bash
# Convert and summarize
pdf2md paper.pdf | gpt "summarize this"
# Process a batch
for f in *.pdf; do pdf2md "$f" > "${f%.pdf}.md"; done
# Chain extractors
cat doc.txt | text-extractor | jq .content
```
## Testing
Tests use pytest. For standalone tools, test via subprocess to exercise the
real CLI interface:
```python
import subprocess
import sys
class TestCLI:
def test_version(self) -> None:
result = subprocess.run(
[sys.executable, "-m", "my_tool.main", "--version"],
capture_output=True,
text=True,
)
assert "my-tool" in result.stdout
def test_stdin(self) -> None:
result = subprocess.run(
[sys.executable, "-m", "my_tool.main"],
input="hello\n",
capture_output=True,
text=True,
)
assert result.returncode == 0
assert "hello" in result.stdout
def test_file_input(self, tmp_path) -> None:
input_file = tmp_path / "input.txt"
input_file.write_text("test data")
result = subprocess.run(
[sys.executable, "-m", "my_tool.main", str(input_file)],
capture_output=True,
text=True,
)
assert result.returncode == 0
assert "test data" in result.stdout
```
For unit testing core logic, import the function directly and test it as a
pure function.
## Commands
```bash
uv sync # Install deps
uv run my-tool --help # Run the tool
uv run pytest tests/ -v # Run tests
uv run ruff check . # Lint
uv run ruff format . # Format
```
## Registering in castle.yaml
```yaml
components:
my-tool:
description: Does something useful
source: components/my-tool
install:
path:
alias: my-tool
```
Tools with system dependencies declare them in the component:
```yaml
components:
pdf2md:
description: Convert PDF files to Markdown
source: components/pdf2md
install:
path:
alias: pdf2md
tool:
system_dependencies: [pandoc, poppler-utils]
```
Tools live in the `components:` section. If a tool also runs on a schedule,
add a separate entry in the `jobs:` section referencing the component.
See @docs/component-registry.md for the full registry reference.

View File

@@ -0,0 +1,440 @@
# Web APIs in Castle
How to build Python web APIs as castle service components. Based on the
patterns used in [wild-cloud/api](https://github.com/civilsociety-dev/wild-cloud)
and existing castle services (central-context, notification-bridge, event-bus).
## Stack
| Layer | Choice |
|-------|--------|
| **Framework** | FastAPI |
| **Server** | uvicorn |
| **Config** | pydantic-settings (env vars) |
| **Validation** | Pydantic models |
| **HTTP client** | httpx (async) |
| **Testing** | pytest + FastAPI TestClient |
| **Python** | 3.13+ for services |
## Project layout
```
my-service/
├── src/my_service/
│ ├── __init__.py # Package version
│ ├── main.py # FastAPI app, lifespan, entry point
│ ├── config.py # pydantic-settings
│ ├── models.py # Request/response Pydantic models
│ ├── routes.py # APIRouter with endpoints
│ └── storage.py # Domain logic (no FastAPI imports)
├── tests/
│ ├── conftest.py # Fixtures (client, temp dirs)
│ ├── test_api.py # Endpoint integration tests
│ └── test_storage.py # Domain unit tests
├── pyproject.toml
└── CLAUDE.md
```
Separation of concerns: routes handle HTTP, storage/core handles logic,
models define schemas. Domain code never imports FastAPI.
## pyproject.toml
```toml
[project]
name = "my-service"
version = "0.1.0"
description = "Does something useful"
requires-python = ">=3.13"
dependencies = [
"fastapi>=0.115.0",
"uvicorn>=0.34.0",
"pydantic-settings>=2.0.0",
]
[project.scripts]
my-service = "my_service.main:run"
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[tool.hatch.build.targets.wheel]
packages = ["src/my_service"]
[dependency-groups]
dev = [
"pytest>=8.0.0",
"httpx>=0.28.0",
]
```
## Configuration
Use pydantic-settings with an env prefix matching the service name:
```python
# config.py
from pathlib import Path
from pydantic_settings import BaseSettings
class Settings(BaseSettings):
data_dir: Path = Path("./data")
host: str = "0.0.0.0"
port: int = 9001
model_config = {
"env_prefix": "MY_SERVICE_",
"env_file": ".env",
}
def ensure_data_dir(self) -> None:
self.data_dir.mkdir(parents=True, exist_ok=True)
settings = Settings()
```
Castle passes config via env vars in castle.yaml:
```yaml
components:
my-service:
description: Does something useful
source: components/my-service
services:
my-service:
component: my-service
run:
runner: python
tool: my-service
expose:
http:
internal: { port: 9001 }
health_path: /health
proxy:
caddy: { path_prefix: /my-service }
manage:
systemd: {}
```
Convention-based env vars (`MY_SERVICE_DATA_DIR`, `MY_SERVICE_PORT`) are
generated automatically by `castle deploy`. Only non-convention values
need `defaults.env`:
```yaml
defaults:
env:
CENTRAL_CONTEXT_URL: http://localhost:9001
```
## Application entry point
```python
# main.py
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
import uvicorn
from fastapi import FastAPI
from my_service.config import settings
from my_service.routes import router
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
settings.ensure_data_dir()
yield
app = FastAPI(
title="my-service",
description="Does something useful",
version="0.1.0",
lifespan=lifespan,
)
app.include_router(router)
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok"}
def run() -> None:
uvicorn.run(
"my_service.main:app",
host=settings.host,
port=settings.port,
reload=False,
)
```
For services with async resources (HTTP clients, connections), initialize
them in the lifespan and clean up after yield:
```python
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
settings.ensure_data_dir()
async with httpx.AsyncClient(timeout=10.0) as client:
app.state.http_client = client
yield
```
## Routes
Use APIRouter with a prefix and tags. Map domain exceptions to HTTP status codes.
```python
# routes.py
from fastapi import APIRouter, HTTPException, status
from my_service.models import ItemCreate, ItemResponse
from my_service.storage import (
ItemExistsError,
ItemNotFoundError,
create_item,
get_item,
list_items,
)
router = APIRouter(prefix="/items", tags=["items"])
@router.post("", response_model=ItemResponse, status_code=status.HTTP_201_CREATED)
def create(request: ItemCreate) -> ItemResponse:
try:
return create_item(request)
except ItemExistsError as e:
raise HTTPException(status_code=status.HTTP_409_CONFLICT, detail=str(e))
@router.get("/{item_id}", response_model=ItemResponse)
def get(item_id: str) -> ItemResponse:
try:
return get_item(item_id)
except ItemNotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
@router.get("", response_model=list[ItemResponse])
def list_all() -> list[ItemResponse]:
return list_items()
```
## Request/response models
Separate create models (what the client sends) from response models (what
comes back). Use inheritance to avoid repetition.
```python
# models.py
from datetime import datetime
from pydantic import BaseModel, Field
class ItemCreate(BaseModel):
name: str = Field(..., description="Item name")
content: str = Field(..., description="Item content")
description: str | None = Field(default=None)
class ItemResponse(BaseModel):
name: str
description: str | None
created_at: datetime
size_bytes: int
checksum: str
class ItemWithBody(ItemResponse):
content: str
```
## Error handling
Define domain exceptions in the storage/core layer. Map them to HTTP status
codes in the route layer.
```python
# storage.py
class StorageError(Exception):
pass
class ItemExistsError(StorageError):
pass
class ItemNotFoundError(StorageError):
pass
class InvalidNameError(StorageError):
pass
```
Mapping convention:
| Exception | HTTP Status |
|-----------|-------------|
| `NotFoundError` | 404 |
| `ExistsError` / conflict | 409 |
| `InvalidError` / bad input | 400 |
| Unexpected | 500 (FastAPI default) |
## Storage
Castle services use filesystem storage with JSON metadata sidecars:
```
/data/castle/my-service/
└── bucket/
├── item-name
└── item-name.meta.json
```
```python
# storage.py
import hashlib
import json
from datetime import datetime
from pathlib import Path
from my_service.config import settings
from my_service.models import ItemCreate, ItemResponse
def create_item(request: ItemCreate) -> ItemResponse:
checksum = hashlib.sha256(request.content.encode()).hexdigest()
path = settings.data_dir / request.name
meta_path = path.with_suffix(path.suffix + ".meta.json")
if path.exists():
raise ItemExistsError(f"'{request.name}' already exists")
path.parent.mkdir(parents=True, exist_ok=True)
now = datetime.now()
metadata = ItemResponse(
name=request.name,
description=request.description,
created_at=now,
size_bytes=len(request.content.encode()),
checksum=checksum,
)
path.write_text(request.content, encoding="utf-8")
meta_path.write_text(
json.dumps(metadata.model_dump(), default=str, indent=2)
)
return metadata
```
## Testing
### Fixtures
```python
# tests/conftest.py
from collections.abc import Generator
from pathlib import Path
import pytest
from fastapi.testclient import TestClient
from my_service import config
from my_service.main import app
@pytest.fixture
def temp_data_dir(tmp_path: Path) -> Generator[Path, None, None]:
data_dir = tmp_path / "data"
data_dir.mkdir()
original = config.settings.data_dir
config.settings.data_dir = data_dir
yield data_dir
config.settings.data_dir = original
@pytest.fixture
def client(temp_data_dir: Path) -> Generator[TestClient, None, None]:
with TestClient(app) as client:
yield client
```
### Endpoint tests
```python
# tests/test_api.py
from fastapi import status
from fastapi.testclient import TestClient
class TestHealth:
def test_health(self, client: TestClient) -> None:
response = client.get("/health")
assert response.status_code == status.HTTP_200_OK
assert response.json() == {"status": "ok"}
class TestCreateItem:
def test_create(self, client: TestClient) -> None:
response = client.post(
"/items",
json={"name": "test", "content": "hello"},
)
assert response.status_code == status.HTTP_201_CREATED
data = response.json()
assert data["name"] == "test"
assert "checksum" in data
def test_duplicate_returns_409(self, client: TestClient) -> None:
payload = {"name": "dup", "content": "hello"}
client.post("/items", json=payload)
response = client.post("/items", json=payload)
assert response.status_code == status.HTTP_409_CONFLICT
```
### Domain unit tests
```python
# tests/test_storage.py
from pathlib import Path
from my_service.models import ItemCreate
from my_service.storage import create_item
class TestCreateItem:
def test_creates_files(self, temp_data_dir: Path) -> None:
request = ItemCreate(name="test", content="hello")
metadata = create_item(request)
assert metadata.name == "test"
assert (temp_data_dir / "test").exists()
assert (temp_data_dir / "test.meta.json").exists()
```
## Commands
```bash
uv sync # Install deps
uv run my-service # Run service
uv run pytest tests/ -v # Run tests
uv run ruff check . # Lint
uv run ruff format . # Format
```
## Scaffolding
`castle create` generates all of this automatically:
```bash
castle create my-service --stack python-fastapi --description "Does something useful"
```
See @docs/component-registry.md for manifest fields, castle.yaml structure,
and the full service lifecycle (enable, logs, gateway reload).

324
docs/stacks/react-vite.md Normal file
View File

@@ -0,0 +1,324 @@
# Web Frontends in Castle
How to build, serve, and manage web frontends as castle components.
## Stack
- Build: Vite 6
- Language: TypeScript 5.8 (strict)
- Framework: React 19
- Routing: React Router 7
- Styling: Tailwind CSS 4 (`@tailwindcss/vite` plugin)
- Components: shadcn/ui (new-york style) + Radix UI primitives
- Icons: Lucide React
- Server state: TanStack React Query 5
- Forms: React Hook Form + Zod validation
- Testing: Vitest + Testing Library
- Package manager: pnpm
## Scaffolding a new frontend
```bash
# Create the project
mkdir components/my-frontend && cd components/my-frontend
pnpm create vite . --template react-ts
# Core dependencies
pnpm add react-router react-router-dom \
@tanstack/react-query \
tailwindcss @tailwindcss/vite \
class-variance-authority clsx tailwind-merge \
lucide-react sonner zod \
react-hook-form @hookform/resolvers
# Dev dependencies
pnpm add -D vitest jsdom @testing-library/react @testing-library/jest-dom \
@testing-library/user-event
# shadcn/ui
pnpm dlx shadcn@latest init
```
## Vite config
```ts
// vite.config.ts
import path from "path"
import tailwindcss from "@tailwindcss/vite"
import { defineConfig } from "vite"
import react from "@vitejs/plugin-react"
export default defineConfig({
plugins: [react(), tailwindcss()],
resolve: {
alias: {
"@": path.resolve(__dirname, "./src"),
},
},
})
```
## Project layout
```
my-frontend/
├── src/
│ ├── main.tsx # ReactDOM.createRoot mount
│ ├── App.tsx # Root: QueryClientProvider + RouterProvider
│ ├── index.css # Tailwind imports + CSS custom properties
│ ├── router/
│ │ ├── index.tsx # createBrowserRouter
│ │ └── routes.tsx # Route tree
│ ├── components/
│ │ └── ui/ # shadcn/ui components
│ ├── services/api/
│ │ ├── client.ts # Typed fetch wrapper
│ │ └── hooks/ # React Query hooks per resource
│ ├── hooks/ # App-level hooks
│ ├── contexts/ # React context providers
│ ├── lib/
│ │ ├── utils.ts # cn() helper
│ │ └── queryClient.ts # Query client config
│ ├── types/ # Shared TypeScript types
│ └── schemas/ # Zod schemas
├── public/ # Static assets (favicon, manifest.json)
├── index.html # SPA entry point
├── package.json
├── vite.config.ts
├── tsconfig.json
├── vitest.config.ts
├── components.json # shadcn/ui config
└── .env # VITE_API_BASE_URL etc.
```
## Build commands
```bash
pnpm run dev # Vite dev server (:5173), HMR
pnpm run build # tsc -b && vite build → dist/
pnpm run preview # Serve production build locally
pnpm run type-check # tsc --noEmit
pnpm run lint # ESLint
pnpm run test # Vitest
pnpm run check # lint + type-check + test
```
The `build` output is a static SPA in `dist/` — just HTML, JS, and CSS files.
## Registering as a castle component
A frontend component has a `build` spec (produces static output). Register it
in the `components:` section of `castle.yaml`. No `run` block needed if Caddy
handles serving directly from the build output.
```yaml
# castle.yaml
components:
my-frontend:
description: Web dashboard
source: components/my-frontend
build:
commands:
- ["pnpm", "build"]
outputs:
- dist/
```
For production, Caddy serves the static `dist/` output directly — no
Node process needed. See [Serving with Caddy](#serving-with-caddy) below.
For development with Vite's dev server, add a service entry:
```yaml
services:
my-frontend:
component: my-frontend
run:
runner: node
script: dev
package_manager: pnpm
expose:
http:
internal: { port: 5173 }
proxy:
caddy: { path_prefix: /app }
```
See @docs/component-registry.md for the full registry reference.
## Serving with Caddy
For production, serve the static `dist/` output directly from Caddy rather than
running a Node process. The gateway Caddyfile can serve the files:
```caddyfile
handle_path /app/* {
root * /data/repos/castle/my-frontend/dist
try_files {path} /index.html
file_server
}
```
The `try_files {path} /index.html` directive is essential for SPA routing —
it falls back to `index.html` for any path that doesn't match a static file,
letting React Router handle client-side routes.
## API integration
Frontends talk to castle services via environment variables injected at build
time. Vite exposes variables prefixed with `VITE_`:
```bash
# .env
VITE_API_BASE_URL=http://localhost:9001
```
```ts
// src/services/api/client.ts
const BASE_URL = import.meta.env.VITE_API_BASE_URL ?? "http://localhost:9001"
class ApiClient {
private baseUrl: string
constructor(baseUrl = BASE_URL) {
this.baseUrl = baseUrl
}
async get<T>(path: string): Promise<T> {
const resp = await fetch(`${this.baseUrl}${path}`)
if (!resp.ok) throw new ApiError(resp.status, await resp.text())
return resp.json()
}
// post<T>, put<T>, delete<T>, etc.
}
export const apiClient = new ApiClient()
```
When served behind the castle gateway, the API base URL can use the gateway's
proxy paths (e.g., `/central-context/`) instead of direct ports, avoiding CORS.
## React Query setup
```ts
// src/lib/queryClient.ts
import { QueryClient } from "@tanstack/react-query"
export const queryClient = new QueryClient({
defaultOptions: {
queries: {
staleTime: 5 * 60 * 1000, // 5 minutes
gcTime: 10 * 60 * 1000, // 10 minutes
retry: 1,
},
},
})
```
```ts
// src/services/api/hooks/useThings.ts
import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query"
import { apiClient } from "../client"
export function useThings() {
return useQuery({
queryKey: ["things"],
queryFn: () => apiClient.get<Thing[]>("/things"),
})
}
export function useCreateThing() {
const qc = useQueryClient()
return useMutation({
mutationFn: (data: CreateThing) => apiClient.post<Thing>("/things", data),
onSuccess: () => qc.invalidateQueries({ queryKey: ["things"] }),
})
}
```
## shadcn/ui setup
Initialize with the `components.json` config:
```json
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "new-york",
"rsc": false,
"tsx": true,
"tailwind": {
"config": "",
"css": "src/index.css",
"baseColor": "neutral",
"cssVariables": true
},
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
"ui": "@/components/ui",
"lib": "@/lib",
"hooks": "@/hooks"
},
"iconLibrary": "lucide"
}
```
Add components as needed:
```bash
pnpm dlx shadcn@latest add button card dialog sidebar
```
Components are copied into `src/components/ui/` as source files you own and can
modify. They use Radix UI primitives underneath, with Tailwind for styling.
## Dark mode
Use CSS custom properties with a `.dark` class on `<html>`:
```css
/* src/index.css */
@import "tailwindcss";
:root {
--background: oklch(1 0 0);
--foreground: oklch(0.145 0 0);
--primary: oklch(0.205 0 0);
/* ... */
}
.dark {
--background: oklch(0.145 0 0);
--foreground: oklch(0.985 0 0);
--primary: oklch(0.922 0 0);
/* ... */
}
```
Toggle via a React context that persists the preference to `localStorage`.
## Testing
```ts
// vitest.config.ts
import { defineConfig } from "vitest/config"
import path from "path"
export default defineConfig({
test: {
environment: "jsdom",
setupFiles: ["src/test/setup.ts"],
exclude: ["dist/**"],
},
resolve: {
alias: { "@": path.resolve(__dirname, "./src") },
},
})
```
```bash
pnpm run test # Single run
pnpm run test:ui # Interactive UI
pnpm run test:coverage # With coverage report
```