wild-cloud/wild-directory
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reorganized for new stable/waypoint versioning design.

Browse Source
This commit is contained in:
Paul Payne
2026-05-24 18:28:47 +00:00
parent 945d2225a2
commit bc7a168851
352 changed files with 1264 additions and 294 deletions
Download Patch File Download Diff File Expand all files Collapse all files

338
ADDING-APPS.md
View File

@@ -6,26 +6,65 @@ This guide is for contributors and maintainers who want to create or modify Wild
Wild Cloud apps are Kubernetes applications packaged as Kustomize configurations with standardized conventions for configuration management, secrets handling, and deployment.
## Directory Structure
Each app has a two-level structure: an `app.yaml` meta file at the root, and version-specific files inside `versions/`. Version directories are named by **slot** (typically the major version), not by the full version string. The actual version lives in `manifest.yaml` inside the slot.
```
myapp/
├── app.yaml # App identity, latest slot pointer, upgrade routing
└── versions/
├── 2/ # Current latest slot (manifest.yaml has version: 2.3.1)
│ ├── manifest.yaml # Version-specific config (requires, defaultConfig, etc.)
│ ├── kustomization.yaml
│ └── *.yaml # Kubernetes resource templates
└── 1/ # Waypoint slot (only if upgrade routing needs it)
├── manifest.yaml
├── kustomization.yaml
└── *.yaml
```
Most apps have **one** version directory. A second appears only when a waypoint is needed for upgrade routing.
## Required Files
Each app directory must contain:
1. **`manifest.yaml`** - App metadata and configuration schema
2. **`kustomization.yaml`** - Kustomize configuration with Wild Cloud labels
3. **Resource files** - Kubernetes manifests (deployments, services, ingresses, etc.)
1. **`app.yaml`** - App identity, latest slot pointer, and upgrade routing rules
2. **`versions/{slot}/manifest.yaml`** - Version-specific configuration schema
3. **`versions/{slot}/kustomization.yaml`** - Kustomize configuration with Wild Cloud labels
4. **`versions/{slot}/*.yaml`** - Kubernetes resource templates
## App Manifest (`manifest.yaml`)
## App Meta (`app.yaml`)
The manifest defines the app's metadata, dependencies, configuration schema, and secret requirements.
This is the contents of an example `manifest.yaml` file for an app named "immich":
The `app.yaml` file at the app root defines identity, display info, and upgrade routing. These fields are version-independent.
```yaml
name: immich
is: immich
description: Immich is a self-hosted photo and video backup solution that allows you to store, manage, and share your media files securely.
version: 1.0.0
icon: https://immich.app/assets/images/logo.png
latest: "1"
```
### App Meta Fields
| Field | Required | Description |
|-------|----------|-------------|
| `name` | Yes | App identifier (must match directory name) |
| `is` | Yes | Unique id for this app. Used for `requires` mapping |
| `description` | Yes | Brief app description shown in listings |
| `icon` | No | URL to app icon for UI display |
| `category` | No | Category (e.g., `infrastructure`) |
| `latest` | Yes | Slot name -- directory name under `versions/` (not a version string) |
| `upgrade` | No | Upgrade routing rules (see Upgrade Metadata below) |
## Version Manifest (`versions/{slot}/manifest.yaml`)
Each version slot contains a `manifest.yaml` with version-specific installation details: dependencies, configuration schema, and secret requirements.
```yaml
version: 1.135.3-1
requires:
- name: pg
alias: db # Use a different reference name in templates
@@ -46,21 +85,17 @@ defaultConfig:
defaultSecrets:
- key: password # Random value will be generated if empty
- key: dbUrl
default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?pool=30" # Can reference secrets and config as long as they have been defined before this line. Reference config with {{ .app.? }} and secrets with {{ .secrets.? }}
default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?pool=30"
requiredSecrets:
- db.password # References postgres app via 'db' alias
- redis.auth # References redis app via 'redis' name (no alias)
```
### Manifest Fields
### Version Manifest Fields
| Field | Required | Description |
|-------|----------|-------------|
| `name` | Yes | App identifier (must match directory name) |
| `is` | Yes | Unique id for this app. Used for `requires` mapping |
| `description` | Yes | Brief app description shown in listings |
| `version` | Yes | App version (see Versioning Convention below) |
| `icon` | No | URL to app icon for UI display |
| `requires` | No | List of dependency apps with optional aliases |
| `defaultConfig` | Yes | Default configuration values merged into operator's `config.yaml` |
| `defaultSecrets` | No | This app's secrets (no 'default' = auto-generated) |
@@ -85,6 +120,27 @@ Wild Cloud uses a two-part version scheme inspired by Debian packaging: `<upstre
The web UI uses version comparison to detect available updates. If the deployed version differs from the wild-directory version, operators see an update indicator and can apply it from the app detail panel.
### Slot Naming Convention
Version directory names are **slot names**, not version strings. The slot is a stable label; the actual version lives in `manifest.yaml` inside the slot.
**Rules:**
- Use the **major version** as the slot name (e.g., `1`, `2`, `5`, `v3`)
- Preserve the `v` prefix if the upstream project uses it (e.g., `v1` for cert-manager)
- **Never** put packaging revisions (`-1`, `-2`) in directory names
- **Never** put minor/patch versions in directory names unless creating a waypoint that needs to be distinct from another slot at the same major version
**Examples:**
| App | Slot name | Version in manifest |
|-----|-----------|-------------------|
| Ghost 5.118.1-2 | `5` | `5.118.1-2` |
| cert-manager v1.17.2 | `v1` | `v1.17.2` |
| Immich 1.135.3-1 | `1` | `1.135.3-1` |
| Traefik v3.4 | `v3` | `v3.4` |
When bumping versions (upstream or packaging), update files inside the existing slot. Only create a new directory when you need a new waypoint.
### Upgrade Metadata
Most apps can upgrade from any version to any other version directly — no special metadata is needed. The `upgrade` field is **optional** and only required when an app has breaking changes that need controlled upgrade paths.
@@ -93,76 +149,100 @@ Most apps can upgrade from any version to any other version directly — no spec
**When you need `upgrade:`** Apps with breaking database schema changes, incompatible config formats, or upstream requirements for sequential version upgrades (e.g., Discourse requires stepping through major versions).
#### The `upgrade` block
#### The `upgrade` block in `app.yaml`
Upgrade routing rules live in `app.yaml`, centralized for all versions. The system iteratively re-evaluates these rules after each waypoint step.
```yaml
# app.yaml
name: myapp
latest: "3"
upgrade:
from:
- version: ">=3.5.0" # Can upgrade directly from 3.5.x
- version: ">=3.4.0"
via: "3.5.3-1" # Must pass through 3.5.x first
via: "2" # Must pass through slot "2" first (a waypoint)
- version: "<3.4.0"
blocked: true
notes: "Requires sequential major upgrades. See upstream docs."
preUpgrade:
backup: required # "none", "recommended", or "required"
```
Note: `latest` and `via` are **slot names** (directory names), not version strings. The system reads the actual version from the manifest inside each slot.
Version-specific upgrade behavior (migrations, configMigrations) lives in the version's `manifest.yaml`:
```yaml
# versions/3/manifest.yaml
version: 3.6.0
upgrade:
migrations:
pre:
- migrations/pre-deploy.yaml # K8s Job YAML paths relative to app dir
- migrations/pre-deploy.yaml # K8s Job YAML paths relative to version dir
post:
- migrations/post-deploy.yaml
configMigrations:
oldKeyName: newKeyName # Renames config keys automatically
```
**Fields:**
**`app.yaml` upgrade fields:**
| Field | Description |
|-------|-------------|
| `from` | List of version constraint rules, evaluated in order (first match wins) |
| `from[].version` | Version constraint: `>=`, `>`, `<=`, `<`, `=`, or `>0` (matches any) |
| `from[].via` | Waypoint version in `.versions/` — upgrade must pass through this version first |
| `from[].via` | Waypoint slot name in `versions/` — upgrade must pass through this slot first |
| `from[].blocked` | If true, upgrade is blocked with an error message |
| `from[].notes` | Human-readable message shown when blocked or as context |
| `preUpgrade.backup` | Backup requirement: `"required"` blocks upgrade until backup is done, `"recommended"` shows a warning |
| `migrations.pre` | K8s Job YAMLs to run before deploying each version step |
| `migrations.post` | K8s Job YAMLs to run after deploying each version step |
**Version `manifest.yaml` upgrade fields:**
| Field | Description |
|-------|-------------|
| `migrations.pre` | K8s Job YAMLs to run before deploying this version step |
| `migrations.post` | K8s Job YAMLs to run after deploying this version step |
| `configMigrations` | Map of old config key → new config key for automatic renaming |
#### Waypoint versions (`.versions/` directory)
#### Waypoint versions
When an upgrade requires passing through an intermediate version, store that version's files in a `.versions/` subdirectory:
When an upgrade requires passing through an intermediate version, add that version's files as a new slot in the `versions/` directory alongside the latest:
```
myapp/
├── manifest.yaml # Latest version (e.g., 3.6.0)
├── kustomization.yaml
├── *.yaml
└── .versions/
└── 3.5.3-1/ # Waypoint version
├── manifest.yaml # version: 3.5.3-1 (with its own upgrade rules)
├── app.yaml # Routing rules + latest pointer
└── versions/
├── 3/ # Latest slot (version: 3.6.0)
│ ├── manifest.yaml
│ ├── kustomization.yaml
│ └── *.yaml
└── 2/ # Waypoint slot (version: 2.8.0)
├── manifest.yaml
├── kustomization.yaml
└── *.yaml
```
Each waypoint is a complete app package. The system computes a chain automatically — for example, upgrading from 3.4.0 to 3.6.0 might produce: `3.4.0 → 3.5.3-1 → 3.6.0`.
Each waypoint is a complete app package. The system computes a chain automatically — for example, upgrading from 2.3.0 to 3.6.0 might produce: `2.3.0 → 2.8.0 (slot "2") → 3.6.0 (slot "3")`.
**Creating a waypoint:**
**Creating a waypoint:** The current latest slot becomes the waypoint (leave it in place), then create a new slot for the new major version:
```bash
mkdir -p wild-directory/myapp/.versions
rsync -a --exclude='.versions' wild-directory/myapp/ wild-directory/myapp/.versions/3.5.3-1/
# Now update wild-directory/myapp/manifest.yaml to the new version + upgrade rules
# Current slot "2" (with version 2.8.0) stays as a waypoint
# Create the new slot for the next major version
mkdir -p wild-directory/myapp/versions/3
# ... add manifest.yaml, kustomization.yaml, *.yaml for 3.0.0 ...
# Update app.yaml: set latest to "3", add upgrade routing rules with via: "2"
```
#### Migration jobs
Migration jobs are K8s Job manifests that run database migrations or other one-time tasks during an upgrade step. They must be **idempotent** (safe to re-run) since a failed upgrade might be retried.
Place migration job files in the waypoint or app directory and reference them from the `migrations` field:
Place migration job files in the version slot directory and reference them from that version's `manifest.yaml`:
```yaml
# migrations/db-migrate.yaml
# versions/3/migrations/db-migrate.yaml
apiVersion: batch/v1
kind: Job
metadata:
@@ -177,17 +257,18 @@ spec:
command: ["bundle", "exec", "rake", "db:migrate"]
```
Each migration step belongs to the version that introduces the breaking change. If version 3.6.0 requires a schema migration, the migration lives in the 3.6.0 manifest (or its waypoint), not on 3.5.x.
Each migration step belongs to the version that introduces the breaking change. If version 3.6.0 requires a schema migration, the migration lives in the slot `3/` directory.
#### Example: simple app with waypoint
```yaml
# myapp/manifest.yaml (version 2.0.0)
version: 2.0.0
# myapp/app.yaml
name: myapp
latest: "2"
upgrade:
from:
- version: ">=1.0.0"
via: "1.0.0-1"
via: "1"
- version: "<1.0.0"
blocked: true
notes: "Versions before 1.0.0 are not supported"
@@ -195,7 +276,171 @@ upgrade:
backup: recommended
```
This creates a 2-step upgrade path: `1.x → 1.0.0-1 → 2.0.0`. The waypoint at `.versions/1.0.0-1/` has no `upgrade` block, so it accepts any version directly.
This creates a 2-step upgrade path: `1.x → slot "1" (e.g., version 1.0.0-1) → slot "2" (e.g., version 2.0.0)`. The waypoint at `versions/1/` is a complete app package used as an intermediate step.
### Adding a New Version
When an upstream app releases a new version, you update the Wild Directory package to track it. The process depends on whether the new version has breaking changes.
#### Simple version bump (no breaking changes)
Most version updates are simple — update the container image tag, adjust any changed config, and update the version in `manifest.yaml`. No directory rename or `app.yaml` change needed.
```bash
# 1. Update files inside the existing slot
# - Bump version in manifest.yaml (e.g., 1.2.0 → 1.3.0)
# - Update container image tags in deployment YAMLs
# - Adjust defaultConfig if the new version adds/changes config
vi wild-directory/myapp/versions/1/manifest.yaml
vi wild-directory/myapp/versions/1/deployment.yaml
# 2. app.yaml doesn't change — latest still points to slot "1"
# 3. Test
wild app add myapp && wild app deploy myapp
```
The directory structure stays the same:
```
myapp/
├── app.yaml # latest: "1" (unchanged)
└── versions/
└── 1/
├── manifest.yaml # version: 1.3.0 (bumped)
└── *.yaml
```
#### Version bump with breaking changes (waypoint required)
When the new version can't safely upgrade from all previous versions — e.g., a database schema change requires stepping through an intermediate version — create a new slot for the new major version, keep the old slot as a waypoint, and add routing rules.
```bash
# 1. The current slot (2/) becomes a waypoint — leave it in place
# 2. Create a new slot for the new major version
mkdir -p wild-directory/myapp/versions/3
# ... add new version files (manifest.yaml, kustomization.yaml, *.yaml) ...
# 3. Update app.yaml: point latest to new slot, add upgrade routing rules
```
```yaml
# app.yaml
name: myapp
latest: "3"
upgrade:
from:
- version: ">=2.5.0" # 2.5.x can upgrade directly
- version: ">=2.0.0"
via: "2" # Older 2.x must pass through slot 2 first
- version: "<2.0.0"
blocked: true
notes: "Upgrade to 2.x first. See upstream migration guide."
preUpgrade:
backup: recommended
```
The resulting directory:
```
myapp/
├── app.yaml # latest: "3", upgrade routing rules
└── versions/
├── 3/ # New latest (manifest.yaml has version: 3.0.0)
│ ├── manifest.yaml
│ └── *.yaml
└── 2/ # Waypoint (manifest.yaml has version: 2.5.0)
├── manifest.yaml
└── *.yaml
```
#### Version bump with database migrations
When the new version requires a schema migration (e.g., `ALTER TABLE`, new indexes, data transformations), add migration job files to the slot directory and reference them from the version's `manifest.yaml`. Since this is a minor/patch update within the same major version, update files in-place in the existing slot.
```bash
# 1. Update files inside the existing slot
# - Bump version in manifest.yaml (e.g., 2.0.0 → 2.1.0)
# - Update container image tags in deployment YAMLs
vi wild-directory/myapp/versions/2/manifest.yaml
vi wild-directory/myapp/versions/2/deployment.yaml
# 2. Add migration job files
mkdir -p wild-directory/myapp/versions/2/migrations
```
Create the migration job:
```yaml
# versions/2/migrations/pre-deploy.yaml
apiVersion: batch/v1
kind: Job
metadata:
name: myapp-migrate-2-1-0
namespace: myapp
spec:
backoffLimit: 3
template:
spec:
restartPolicy: OnFailure
securityContext:
runAsNonRoot: true
runAsUser: 999
seccompProfile:
type: RuntimeDefault
containers:
- name: migrate
image: myapp:2.1.0
command: ["bundle", "exec", "rake", "db:migrate"]
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop: [ALL]
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: myapp-secrets
key: dbUrl
```
Reference the migration in the version manifest:
```yaml
# versions/2/manifest.yaml
version: 2.1.0
upgrade:
migrations:
pre:
- migrations/pre-deploy.yaml
defaultConfig:
# ...
```
`app.yaml` doesn't change — `latest` still points to slot `"2"`.
Migration jobs must be **idempotent** — safe to re-run if an upgrade is retried after a partial failure. Use `CREATE IF NOT EXISTS`, `ALTER TABLE IF NOT EXISTS`, etc.
**Pre vs post migrations:**
- `pre` — runs before deploying the new version's manifests (schema changes that the new code needs)
- `post` — runs after deploying (data backfills, cleanup that the old code didn't need)
#### Version bump with config key renames
When a version renames config keys (e.g., `dbHost` → `db.host`), use `configMigrations` to automatically rename them during upgrade:
```yaml
# versions/2/manifest.yaml
version: 2.1.0
upgrade:
configMigrations:
dbHost: db.host
dbPort: db.port
dbName: db.name
defaultConfig:
db:
host: "{{ .apps.pg.host }}"
port: "5432"
name: myapp
```
The system renames the keys in the instance's `config.yaml` before recompiling templates with the new version.
### Dependency Configuration
@@ -666,9 +911,16 @@ labels:
Before submitting a new or modified app, verify:
- [ ] **Manifest**
- [ ] **App Meta (`app.yaml`)**
- [ ] `name` matches directory name
- [ ] All required fields present (`name`, `description`, `version`, `defaultConfig`)
- [ ] `latest` points to a valid version in `versions/`
- [ ] `description` present
- [ ] `upgrade` rules correct (if applicable)
- [ ] **Version Manifest (`versions/{slot}/manifest.yaml`)**
- [ ] `version` field present with full version string (e.g., `1.135.3-1`)
- [ ] Slot directory follows naming convention (major version, e.g., `1`, `v1`)
- [ ] All required fields present (`version`, `defaultConfig`)
- [ ] All template variables defined in `defaultConfig`
- [ ] `defaultSecrets` uses maps with 'key' and 'default' attributes
- [ ] `requiredSecrets` references use `<app-ref>.<key>` format