wild-directory/ADDING-APPS.md

# Adding Wild Cloud Apps

For contributors and maintainers creating or modifying Wild Cloud app packages. For using existing apps, see [README.md](README.md).

## Directory Structure

Each app has a two-level structure: an `app.yaml` meta file at the root, and version-specific files inside `versions/`. Name version directories by **slot** (typically the major version), not the full version string. The actual version lives in `manifest.yaml` inside the slot.

```
myapp/
├── app.yaml               # App identity, latest slot pointer, upgrade routing
├── notes.md               # App-specific operational notes (optional but encouraged)
└── versions/
    ├── 2/                  # Current latest slot (manifest.yaml has version: 2.3.1)
    │   ├── manifest.yaml   # Version-specific config (requires, defaultConfig, etc.)
    │   ├── kustomization.yaml
    │   └── *.yaml          # Kubernetes resource templates
    └── 1/                  # Waypoint slot (only if upgrade routing needs it)
        ├── manifest.yaml
        ├── kustomization.yaml
        └── *.yaml
```

Most apps have **one** version directory. Add a second only when a waypoint is needed for upgrade routing.

## Required Files

Every app directory must contain:

1. **`app.yaml`** — App identity, latest slot pointer, and upgrade routing rules
2. **`versions/{slot}/manifest.yaml`** — Version-specific configuration schema
3. **`versions/{slot}/kustomization.yaml`** — Kustomize configuration with Wild Cloud labels
4. **`versions/{slot}/*.yaml`** — Kubernetes resource templates

## App Meta (`app.yaml`)

```yaml
name: immich
is: immich
description: Immich is a self-hosted photo and video backup solution that allows you to store, manage, and share your media files securely.
icon: https://immich.app/assets/images/logo.png
latest: "1"
```

| Field | Required | Description |
|-------|----------|-------------|
| `name` | Yes | App identifier — must match directory name |
| `is` | Yes | Unique id used for `requires` mapping |
| `description` | Yes | Brief description shown in listings |
| `icon` | Yes | URL to app icon for UI display |
| `category` | Yes | Category for filtering in the App Directory |
| `latest` | Yes | Slot name — directory name under `versions/` (not a version string) |
| `upgrade` | No | Upgrade routing rules (see Upgrade Metadata below) |

### Categories

Use an established category when one fits. Introduce a new one only when an app clearly doesn't belong anywhere existing.

| Category | Use for |
|----------|---------|
| `services` | Wild Cloud infrastructure (traefik, metallb, cert-manager, etc.) |
| `database` | Database engines (postgres, mysql, redis, memcached) |
| `productivity` | Wikis, project management, file sharing, collaboration |
| `business` | ERP, CRM, accounting, forms, marketing automation |
| `communication` | Chat, messaging, video conferencing, email |
| `social` | Social networks and fediverse apps |
| `community` | Community organizing, civic tech, decision-making |
| `education` | Learning management systems |
| `developer` | Dev tools, code hosting, package management |
| `security` | VPN, password management |
| `ai` | AI/ML inference and interfaces |
| `media` | Photo and video management |

## Version Manifest (`versions/{slot}/manifest.yaml`)

```yaml
version: 1.135.3-1
requires:
  - name: pg
    alias: db        # Use a different reference name in templates
  - name: redis      # 'alias' and 'installedAs' default to 'name' value
defaultConfig:
  namespace: immich
  externalDnsDomain: "{{ .cloud.domain }}"
  storage: 250Gi
  cacheStorage: 10Gi
  domain: immich.{{ .cloud.domain }}
  tlsSecretName: wildcard-wild-cloud-tls
  db:
    host: "{{ .apps.pg.host }}"
    name: immich
    user: immich
  redis:
    host: "{{ .apps.redis.host }}"
defaultSecrets:
  - key: password
  - key: dbUrl
    default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?pool=30"
requiredSecrets:
  - db.password
  - redis.auth
```

| Field | Required | Description |
|-------|----------|-------------|
| `version` | Yes | App version (see Versioning Convention below) |
| `requires` | No | Dependency apps with optional aliases |
| `defaultConfig` | Yes | Default configuration merged into the operator's `config.yaml` |
| `defaultSecrets` | No | This app's secrets — no `default` value means auto-generated |
| `requiredSecrets` | No | Secrets from dependency apps — format: `<app-ref>.<key>` `[WC-DOT]` |

### Versioning Convention

Use the two-part scheme `<upstream>-<revision>`:

- **Upstream version** tracks the third-party software (e.g., `1.120.2`, `v4.0.18`)
- **Packaging revision** tracks Wild Cloud packaging changes with no upstream change

**Examples:**
- `v4.0.18` — initial packaging
- `v4.0.18-1` — first packaging fix, no upstream change
- `v4.0.19` — upstream version bump, revision resets

**Bump the packaging revision** for: manifest field changes, template improvements, security context fixes, label corrections, etc.

**Bump the upstream version** when updating the container image tag.

### Slot Naming Convention

- Use the **major version** as the slot name (`1`, `2`, `v3`)
- Keep the `v` prefix if upstream uses it (`v1` for cert-manager)
- **Never** put packaging revisions (`-1`, `-2`) in directory names
- **Never** put minor/patch versions in directory names unless you need a distinct waypoint

| App | Slot name | Version in manifest |
|-----|-----------|-------------------|
| Ghost 5.118.1-2 | `5` | `5.118.1-2` |
| cert-manager v1.17.2 | `v1` | `v1.17.2` |
| Immich 1.135.3-1 | `1` | `1.135.3-1` |

When bumping versions, update files inside the existing slot. Only create a new directory when you need a waypoint.

### Upgrade Metadata

The `upgrade` field is **optional** — omit it for apps where any version can upgrade directly to any other. That covers 90% of apps.

Add `upgrade:` only when an app has breaking changes that need a controlled path (e.g., Discourse requires stepping through major versions).

#### The `upgrade` block in `app.yaml`

```yaml
name: myapp
latest: "3"
upgrade:
  from:
    - version: ">=3.5.0"
    - version: ">=3.4.0"
      via: "2"                  # Must pass through slot "2" first
    - version: "<3.4.0"
      blocked: true
      notes: "Requires sequential major upgrades. See upstream docs."
  preUpgrade:
    backup: required            # "none", "recommended", or "required"
```

`latest` and `via` are **slot names** (directory names), not version strings.

| Field | Description |
|-------|-------------|
| `from[].version` | Version constraint: `>=`, `>`, `<=`, `<`, `=`, or `>0` |
| `from[].via` | Waypoint slot to pass through first |
| `from[].blocked` | Block upgrade with an error message |
| `from[].notes` | Message shown when blocked or as context |
| `preUpgrade.backup` | `"required"` blocks until backup is done; `"recommended"` warns |

Version-specific migration behavior goes in the version's `manifest.yaml`:

```yaml
# versions/3/manifest.yaml
version: 3.6.0
upgrade:
  migrations:
    pre:
      - migrations/pre-deploy.yaml
    post:
      - migrations/post-deploy.yaml
  configMigrations:
    oldKeyName: newKeyName
```

| Field | Description |
|-------|-------------|
| `migrations.pre` | K8s Job YAMLs to run before deploying this version |
| `migrations.post` | K8s Job YAMLs to run after deploying |
| `configMigrations` | Map of old config key → new config key for automatic renaming |

#### Waypoint versions

When an upgrade requires an intermediate step, add the waypoint as a new slot alongside `latest`:

```
myapp/
├── app.yaml               # latest: "3", routing rules
└── versions/
    ├── 3/                  # Latest (version: 3.6.0)
    └── 2/                  # Waypoint (version: 2.8.0)
```

To create a waypoint: leave the current latest slot in place and create a new directory for the new major version. Update `app.yaml` to point `latest` at the new slot and add routing rules.

#### Migration jobs

Migration jobs are K8s Job manifests. Write them **idempotent** — safe to re-run if an upgrade is retried.

Place migration files in the version slot and reference them from `manifest.yaml`:

```yaml
# versions/3/migrations/db-migrate.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: myapp-db-migrate
spec:
  template:
    spec:
      restartPolicy: OnFailure
      containers:
        - name: migrate
          image: myapp:3.6.0
          command: ["bundle", "exec", "rake", "db:migrate"]
```

- **`pre`** — schema changes the new code needs before it starts
- **`post`** — backfills or cleanup after the new version is running

#### Config key renames

Use `configMigrations` to automatically rename config keys during upgrade:

```yaml
# versions/2/manifest.yaml
version: 2.1.0
upgrade:
  configMigrations:
    dbHost: db.host
    dbPort: db.port
```

### Adding a New Version

#### Simple version bump (no breaking changes)

Update files inside the existing slot. Do not rename directories or change `app.yaml`.

```bash
vi wild-directory/myapp/versions/1/manifest.yaml   # bump version
vi wild-directory/myapp/versions/1/deployment.yaml # update image tag
wild app add myapp && wild app deploy myapp
```

#### Breaking changes (waypoint required)

```bash
mkdir -p wild-directory/myapp/versions/3
# add manifest.yaml, kustomization.yaml, *.yaml for the new version
# update app.yaml: set latest: "3", add upgrade routing rules with via: "2"
```

#### Schema migrations

Add migration job files to the slot directory and reference them from `manifest.yaml`:

```yaml
# versions/2/manifest.yaml
version: 2.1.0
upgrade:
  migrations:
    pre:
      - migrations/pre-deploy.yaml
```

Migration jobs must be idempotent. Use `CREATE IF NOT EXISTS`, `ALTER TABLE IF NOT EXISTS`, etc.

### Dependencies

```yaml
requires:
  - name: postgres
    alias: db
  - name: redis
```

- `name` — the app to depend on (any installed app with a matching `is` field satisfies it)
- `alias` — reference name in templates (defaults to `name`)

At installation time, the operator maps each dependency to an actual installed app instance. The system tracks this mapping via `installedAs` in the local manifest.

### Template Variables

All template variables referenced in resource files must be defined in `defaultConfig`.

#### Variable namespaces

1. `{{ .cloud.* }}`, `{{ .cluster.* }}`, `{{ .operator.* }}` — standard Wild Cloud variables
2. `{{ .app.* }}` — this app's config
3. `{{ .apps.<ref>.* }}` — dependency app config, resolved via alias or name
4. `{{ .secrets.* }}` — this app's secrets (in `defaultSecrets` only)

#### Available configuration variables

##### operator
- `operator.email`

##### cloud
- `cloud.domain`, `cloud.baseDomain`, `cloud.internalDomain`
- `cloud.dnsmasq.ip`, `cloud.dnsmasq.interface`
- `cloud.router.ip`, `cloud.router.dynamicDns`, `cloud.dhcpRange`
- `cloud.nfs.host`, `cloud.nfs.mediaPath`, `cloud.nfs.storageCapacity`
- `cloud.dockerRegistryHost`
- `cloud.backup.root`

##### cluster
- `cluster.name`, `cluster.hostnamePrefix`
- `cluster.nodes.talos.version`, `cluster.nodes.talos.schematicId`
- `cluster.nodes.control.vip`
- `cluster.ipAddressPool`, `cluster.loadBalancerIp`
- `cluster.certManager.cloudflare.domain`, `cluster.certManager.cloudflare.zoneID`
- `cluster.externalDns.ownerId`
- `cluster.dockerRegistry.storage`

##### apps
Each deployed app gets a section under `apps.<name>.*`. Common fields:
- `apps.<name>.namespace`, `domain`, `externalDnsDomain`, `tlsSecretName`
- `apps.<name>.db.host`, `db.port`, `db.name`, `db.user`
- `apps.<name>.redis.host`
- `apps.<name>.smtp.host`, `smtp.port`, `smtp.user`, `smtp.from`, `smtp.tls`, `smtp.startTls`

#### Dependency reference resolution

`{{ .apps.<ref>.* }}` resolves by checking `alias` first, then `name`. The system uses the `installedAs` value to find the actual app config.

### Kustomization (`kustomization.yaml`)

```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: immich
labels:
  - includeSelectors: true
    pairs:
      app: immich
      managedBy: kustomize
      partOf: wild-cloud
resources:
  - deployment-server.yaml
  - ingress.yaml
  - namespace.yaml
  - pvc.yaml
  - service.yaml
  - db-init-job.yaml
```

- Set `namespace` to the app name `[WC-NS]`
- Include standard Wild Cloud labels with `includeSelectors: true` `[WC-SEL]`
- List every resource file under `resources:` `[WC-RES]`

`includeSelectors: true` automatically injects the standard labels into resource selectors. Individual resources can use simple component labels (`component: web`) — Kustomize expands them.

**Do not use Helm-style labels** (`app.kubernetes.io/name`, etc.). Use `component: web`, `component: worker`, etc. `[WC-HELM]`

## Configuration Templates

Resource files are gomplate templates compiled when operators add apps. Only variables defined in `defaultConfig` are available to resource templates.

### Ingress

Use this as the standard ingress template:

```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp
  annotations:
    external-dns.alpha.kubernetes.io/target: {{ .externalDnsDomain }}
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"
    external-dns.alpha.kubernetes.io/ttl: "60"
spec:
  ingressClassName: traefik
  rules:
  - host: {{ .domain }}
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: myapp
            port:
              number: 80
  tls:
  - hosts:
    - {{ .domain }}
    secretName: {{ .tlsSecretName }}
```

- Always use `spec.ingressClassName: traefik` — do not use the deprecated `kubernetes.io/ingress.class` annotation `[WC-ING-ANN]` `[WC-ING-CLS]`
- Do not add `cert-manager.io/cluster-issuer` — the wildcard TLS cert is pre-distributed to namespaces; this annotation overwrites it with a wrong per-app cert `[WC-CERT]`
- Do not add Traefik redirect annotations — HTTPS redirect is handled globally
- Define `externalDnsDomain`, `domain`, and `tlsSecretName` in `defaultConfig`

The `external-dns.alpha.kubernetes.io/target` annotation creates a CNAME from the app subdomain to the cluster domain. Always set `cloudflare-proxied: "false"` since Wild Cloud manages its own TLS. `[WC-DNS]`

## Database Setup

### Database Initialization Jobs

Include a `db-init-job.yaml` for every app that uses a database. The job must:

- Create the application database if it doesn't exist
- Create/update the application user with the correct credentials
- Grant permissions
- Install any required database extensions (`vector`, `bloom`, `pg_trgm`, etc.)
- Use `restartPolicy: OnFailure`
- Be listed in `kustomization.yaml` resources
- Use the appropriate security context (`runAsUser: 999` for PostgreSQL)
- Be **idempotent** — safe to re-run after a redeploy

See `immich`, `gitea`, or `openproject` for reference implementations.

### Database URLs

When apps need a database URL with embedded credentials, use a dedicated `dbUrl` secret — do not construct URLs in-place:

```yaml
# Wrong: Kustomize cannot do runtime env var substitution
- name: DB_URL
  value: "postgresql://user:$(DB_PASSWORD)@host/db"

# Correct: use a secret with the full URL
- name: DB_URL
  valueFrom:
    secretKeyRef:
      name: myapp-secrets
      key: dbUrl
```

Add `dbUrl` to `defaultSecrets` with the URL template as the `default` value.

### Database Env Var Naming

Name database-related env vars so the backup system can identify them:

- **Database name**: include `DATABASE`, `DB_NAME`, `DBNAME`, or `__DATABASE` in the env var name
- **Database URLs**: the value must contain `://`
- **Usernames**: include `USER` in the name — these are not patched even if the value matches the DB name

```yaml
- name: DB_NAME     # patched on restore
  value: myapp
- name: DB_USER     # not patched (contains USER)
  value: myapp
- name: DATABASE_URL  # patched (value contains ://)
  value: "postgresql://myapp:secret@postgres/myapp"
```

## Deployment Strategy `[WC-RWO]`

Set `strategy: type: Recreate` on any Deployment that uses a `ReadWriteOnce` PVC. RWO volumes attach to one pod at a time — `RollingUpdate` causes Multi-Attach errors when the new pod tries to mount while the old one still holds the volume.

```yaml
spec:
  replicas: 1
  strategy:
    type: Recreate
```

## Security Contexts `[WC-SC-POD]` `[WC-SC-CTR]`

Set security contexts at both pod and container levels on all workloads:

```yaml
spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 999
        runAsGroup: 999
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: app
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: [ALL]
          readOnlyRootFilesystem: false
```

Common UIDs: PostgreSQL `999`, Redis `999`. Consult the image docs for others.

For images that must run as root (s6-overlay, linuxserver.io), omit `runAsNonRoot` from the pod context and suppress `[WC-SC-POD]` / `[WC-SC-CTR]` with `ignoreRules` if the upstream manifest doesn't support it.

## Secrets

Define app secrets in `defaultSecrets`. Reference secrets from dependencies in `requiredSecrets`.

```yaml
defaultSecrets:
  - key: dbPassword
  - key: apiKey
requiredSecrets:
  - db.password      # from postgres (aliased as 'db')
  - redis.auth
```

Reference them in resources as:

```yaml
- name: DB_PASSWORD
  valueFrom:
    secretKeyRef:
      name: myapp-secrets
      key: dbPassword       # own secret
- name: POSTGRES_PASSWORD
  valueFrom:
    secretKeyRef:
      name: myapp-secrets
      key: db.password      # required secret
```

Never hardcode secret values in templates. Never commit `secrets.yaml`.

## Converting Helm Charts

When an official Helm chart exists, convert it rather than writing manifests from scratch.

```bash
helm fetch --untar --untardir charts repo/chart-name
helm template --output-dir base --namespace myapp --values values.yaml myapp charts/chart-name
```

Then convert to Wild Cloud format:

- Replace hardcoded values with gomplate variables (`{{ .cloud.domain }}`, etc.)
- Replace Helm labels with Wild Cloud standard labels
- Add `includeSelectors: true` to kustomization
- Use simple component labels (`component: web`, not `app.kubernetes.io/name`)
- Add security contexts
- Add external-dns annotations to ingresses
- Update secrets to use dotted-path convention

**Helm style → Wild Cloud style:**

```yaml
# Remove:
labels:
  app.kubernetes.io/name: myapp
  app.kubernetes.io/instance: release-name

# Add to kustomization.yaml:
labels:
  - includeSelectors: true
    pairs:
      app: myapp
      managedBy: kustomize
      partOf: wild-cloud

# In individual resources:
labels:
  component: server
```

## Post-Deploy Scripts

Some apps require a management command after first deploy to create an admin account, register a node, or invite the first user. Package these as shell scripts in `scripts/` alongside the kustomize files.

Register every script in `manifest.yaml` — the web UI shows a parameter form for each one:

```yaml
scripts:
  - name: create-superuser
    path: scripts/create-superuser.sh
    description: "Create the initial admin account."
    params:
      - name: EMAIL
        required: true
      - name: PASSWORD
        description: Leave blank to generate a random one
```

Follow `synapse/versions/v1/scripts/create-user.sh` as the reference implementation. See [docs/scripts.md](docs/scripts.md) for the full conventions and common commands.

## Common Gotchas

Keep app-specific operational notes in a `notes.md` file inside the app directory.

### Image availability
Before packaging, confirm the image pulls without authentication (`docker pull <image>`). Bitnami images moved off Docker Hub; ghcr.io images often require auth even for public repos. If no public image exists, add `ignoreRules: [WC-IMG]` with a comment.

### After template changes
After editing any file in wild-directory, re-run `wild app add <app>` before deploying — compiled templates in the instance directory are not auto-updated.

### PostgreSQL
If using PostgreSQL, see [docs/database.md](docs/database.md) — `sslmode=disable`, db-init job pattern, URL construction, env var naming.

### MySQL
If using MySQL, see [docs/database.md](docs/database.md) — db-init idempotency (`CREATE USER` + `ALTER USER`), required secret references.

### Redis
If using Redis, see [docs/redis.md](docs/redis.md) — authenticated URL pattern, apps that don't support passwords.

### Runtime memory, startup delays, health probes
If the app uses Node.js, Python, PHP, Ruby, JVM, nginx, or linuxserver.io images, see [docs/](docs/).

### Hex-format secrets
If an app requires a hex secret (e.g. exactly 64 chars for Outline), derive one with `crypto.SHA256`:
```yaml
defaultSecrets:
  - key: utilsSecret
  - key: secretKey
    default: '{{ crypto.SHA256 .secrets.utilsSecret }}'
```
Referenced secrets must appear earlier in the list.

### nginx proxy DNS `[WC-DKRDNS]`
If the app uses an nginx proxy config from Docker, see [docs/nginx.md](docs/nginx.md) — `127.0.0.11` doesn't exist in Kubernetes.

### Traefik "No available server" `[WC-SEL]`
If Traefik returns "No available server" after a redeploy, see [docs/traefik.md](docs/traefik.md) — immutable selector fix.

### Same-origin iframes
If the app embeds its own sub-pages in iframes and they break, see [docs/traefik.md](docs/traefik.md) — `X-Frame-Options` interaction.


## Validation

Run the validation script before submitting any new or modified app:

```bash
# From the wild-directory root:
admin/scripts/validate-app.sh <app-name>

# Or from inside your app directory:
../../admin/scripts/validate-app.sh
```

**Errors must be fixed before submitting.** Warnings are strong suggestions — fix them unless you have a specific documented reason not to.

For the full list of rule IDs, severities, and fix instructions:

```bash
admin/scripts/validate-apps.py --list-rules
```

**Suppressing rules:**

If your app has a legitimate reason to deviate (e.g. packaging upstream Helm-rendered manifests), add `ignoreRules` to `app.yaml` with a comment:

```yaml
ignoreRules:
  - WC-HELM  # upstream Helm-rendered manifests
  - WC-SEL   # upstream manifests; selectors are set by the Helm chart
```

Only suppress rules when genuinely necessary.

## Submission Checklist

Run the validator first — it catches most issues automatically:

```bash
admin/scripts/validate-app.sh <app-name>
```

**Errors must be fixed. Warnings must be fixed or suppressed with a comment in `ignoreRules`.**

Then verify the things the validator can't check:

- [ ] Docker images pull without authentication (`docker pull <image>`)
- [ ] App deploys and runs correctly in the `test-cloud` instance
- [ ] Dependencies (database, redis, etc.) work correctly end-to-end

## Contributing

1. Fork the repository
2. Create a new app directory following the structure above
3. Test your app in `test-cloud`
4. Submit a pull request with a description of the app and any special notes

## Notice: Third-Party Software

The Kubernetes manifests and Kustomize files in this directory are designed to deploy **third-party software**.

Unless otherwise stated, the software deployed by these manifests **is not authored or maintained** by this project. All copyrights, licenses, and responsibilities for that software remain with the respective upstream authors.

These files are provided solely for convenience and automation. Users are responsible for reviewing and complying with the licenses of the software they deploy.

This project is licensed under the GNU AGPLv3 or later, but this license does **not apply** to the third-party software being deployed.