Files

Paul Payne 434769ac7a Adds is attribute to manifests.

2025-12-31 08:15:17 +00:00

20 KiB

Raw Blame History

Adding Wild Cloud Apps

This guide is for contributors and maintainers who want to create or modify Wild Cloud apps. If you're looking to use existing apps, see README.md.

Overview

Wild Cloud apps are Kubernetes applications packaged as Kustomize configurations with standardized conventions for configuration management, secrets handling, and deployment.

Required Files

Each app directory must contain:

manifest.yaml - App metadata and configuration schema
kustomization.yaml - Kustomize configuration with Wild Cloud labels
Resource files - Kubernetes manifests (deployments, services, ingresses, etc.)

App Manifest (`manifest.yaml`)

The manifest defines the app's metadata, dependencies, configuration schema, and secret requirements.

This is the contents of an example manifest.yaml file for an app named "immich":

name: immich
is: immich
description: Immich is a self-hosted photo and video backup solution that allows you to store, manage, and share your media files securely.
version: 1.0.0
icon: https://immich.app/assets/images/logo.png
requires:
  - name: pg
    alias: db        # Use a different reference name in templates
  - name: redis      # 'alias' and 'installedAs' default to 'name' value
defaultConfig:
  serverImage: ghcr.io/immich-app/immich-server:release
  mlImage: ghcr.io/immich-app/immich-machine-learning:release
  timezone: UTC
  serverPort: 2283
  mlPort: 3003
  storage: 250Gi
  cacheStorage: 10Gi
  redisHostname: "{{ .apps.redis.host }}" # Can reference 'requires' app configurations
  dbHostname: "{{ .apps.pg.host }}"
  db: # Configuration can be nested
    name: immich
    user: immich
    host: "{{ .apps.pg.host }}"
    port: "{{ .apps.pg.port }}"
  domain: immich.{{ .cloud.domain }}
defaultSecrets:
  - key: password      # Random value will be generated if empty
  - key: dbUrl
    default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?pool=30" # Can reference secrets and config as long as they have been defined before this line. Reference config with {{ .app.? }} and secrets with {{ .secrets.? }}
requiredSecrets:
  - db.password   # References postgres app via 'db' alias
  - redis.auth    # References redis app via 'redis' name (no alias)

Manifest Fields

Field	Required	Description
`name`	Yes	App identifier (must match directory name)
`is`	Yes	Unique id for this app. Used for `requires` mapping
`description`	Yes	Brief app description shown in listings
`version`	Yes	App version (follow upstream versioning)
`icon`	No	URL to app icon for UI display
`requires`	No	List of dependency apps with optional aliases
`defaultConfig`	Yes	Default configuration values merged into operator's `config.yaml`
`defaultSecrets`	No	This app's secrets (no 'default' = auto-generated)
`requiredSecrets`	No	List of secrets from dependency apps (format: `<app-ref>.<key>`)

Dependency Configuration

Each dependency in requires can have:
- name: The app name to depend on (any app with a matching is field can satisfy this requirement)
- alias: Optional reference name for templates (defaults to name)

Manifest Template Variables (configuration and secrets)

Manifest Template Variable Sources

Standard Wild Cloud variables: {{ .cloud.* }}, {{ .cluster.* }}, {{ .operator.* }}
App-specific variables: {{ .app.* }} - resolved from current app's config
Dependency variables: {{ .apps.<ref>.* }} - resolved using app reference mapping
App-specific secrets (in 'defaultSecrets' ONLY): {{ secrets.* }}

Available Configuration Variiables

Here's a comprehensive rundown of all config variables that get set during cluster and service setup in config.yaml:

operator (Set during initial setup)

operator.email - Email for cluster operator/admin

cloud (Infrastructure-level settings)

DNS Configuration:

cloud.dns.ip - IP address of the DNS server (Wild Central)
cloud.dns.externalResolver - External DNS resolver (e.g., 1.1.1.1, 8.8.8.8)

Network Configuration:

cloud.router.ip - Router gateway IP
cloud.router.dynamicDns - Dynamic DNS hostname (optional)
cloud.dhcpRange - DHCP range for the network (e.g., "192.168.8.34,192.168.8.79")
cloud.dnsmasq.interface - Network interface for dnsmasq

Domain Configuration:

cloud.baseDomain - Base domain for the cloud (e.g., "payne.io")
cloud.domain - Full cloud domain (e.g., "cloud2.payne.io")
cloud.internalDomain - Internal cluster domain (e.g., "internal.cloud2.payne.io")

Storage Configuration (NFS Service):

cloud.nfs.host - NFS server hostname/IP
cloud.nfs.mediaPath - NFS export path for media storage
cloud.nfs.storageCapacity - NFS storage capacity (e.g., "50Gi", "1Ti")

Registry Configuration (Docker Registry Service):

cloud.dockerRegistryHost - Docker registry hostname (e.g., "registry.internal.cloud2.payne.io")

SMTP Configuration (SMTP Service):

cloud.smtp.host - SMTP server hostname
cloud.smtp.port - SMTP port (typically "465" or "587")
cloud.smtp.user - SMTP username
cloud.smtp.from - Default 'from' email address
cloud.smtp.tls - Enable TLS (true/false)
cloud.smtp.startTls - Enable STARTTLS (true/false)

Backup Configuration:

cloud.backup.root - Root path for backups

cluster (Kubernetes cluster settings)

Basic Cluster Info:

cluster.name - Cluster name identifier
cluster.hostnamePrefix - Prefix for node hostnames

Node Configuration:

cluster.nodes.talos.version - Talos Linux version (e.g., "v1.11.5")
cluster.nodes.talos.schematicId - Talos Image Factory schematic ID
cluster.nodes.control.vip - Virtual IP for control plane
cluster.nodes.active.* - Individual node configurations with:
- role - "controlplane" or "worker"
- interface - Network interface name
- disk - Disk device path
- currentIp - Current IP address
- targetIp - Target IP address
- configured - Configuration status
- applied - Applied status
- maintenance - Maintenance mode
- schematicId - Node-specific schematic ID
- version - Node-specific Talos version

MetalLB Service:

cluster.ipAddressPool - IP range for MetalLB (e.g., "192.168.8.80-192.168.8.89")
cluster.loadBalancerIp - Primary load balancer IP (e.g., "192.168.8.80")

Cert-Manager Service:

cluster.certManager.cloudflare.domain - Cloudflare domain for DNS-01 challenge
cluster.certManager.cloudflare.zoneID - Cloudflare zone ID

ExternalDNS Service:

cluster.externalDns.ownerId - Unique identifier for this cluster's DNS records

Docker Registry Service:

cluster.dockerRegistry.storage - Storage size for registry (e.g., "10Gi")

apps (Application configurations)

Each app added to the cluster gets its own section under apps. with app-specific configuration from the app's manifest. Common patterns include:

Standard app fields:

apps..namespace - Kubernetes namespace
apps..domain - App domain (e.g., "ghost.cloud2.payne.io")
apps..externalDnsDomain - Domain for external DNS
apps..tlsSecretName - TLS certificate secret name
apps..image - Container image
apps..port - Service port
apps..storage - Persistent volume size
apps..timezone - Timezone setting

Database-dependent apps:

apps..dbHost / dbHostname - Database hostname
apps..dbPort - Database port
apps..dbName - Database name
apps..dbUser / dbUsername - Database user

SMTP-enabled apps:

apps..smtp.host - SMTP server
apps..smtp.port - SMTP port
apps..smtp.user - SMTP username
apps..smtp.from - From address
apps..smtp.tls - TLS enabled
apps..smtp.startTls - STARTTLS enabled

Configuration Flow

Initial Setup: operator.email, basic cloud.* settings
Cluster Bootstrap: cluster.name, cluster.nodes.* settings
Infrastructure Services: Each service prompts for its serviceConfig from its manifest
- MetalLB → cluster.ipAddressPool, cluster.loadBalancerIp
- Cert-Manager → cluster.certManager.*
- ExternalDNS → cluster.externalDns.ownerId
- NFS → cloud.nfs.*
- Docker Registry → cloud.dockerRegistryHost, cluster.dockerRegistry.storage
- SMTP → cloud.smtp.*
Apps: Each app adds its configuration under apps..* based on its manifest

Manifest App Reference Resolution:

When you use {{ .apps.<ref>.* }} in templates:

System checks if <ref> matches any dependency's alias field
If no alias match, checks if <ref> matches any dependency's name field
Uses the installedAs value (automatically added when the app is added) to find actual app configuration in config.yaml

All manifest template variables must be defined in one of these locations.

Important: In the rest of the app templates, ALL configuration keys referenced in templates (via {{ .key }}) must be defined in defaultConfig. Only the app config is available to app templates.

Kustomization (`kustomization.yaml`)

The kustomization file defines how Kubernetes resources are built and applies Wild Cloud's standard labels.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: immich
labels:
  - includeSelectors: true
    pairs:
      app: immich
      managedBy: kustomize
      partOf: wild-cloud
resources:
  - deployment-server.yaml
  - deployment-machine-learning.yaml
  - deployment-microservices.yaml
  - ingress.yaml
  - namespace.yaml
  - pvc.yaml
  - service.yaml
  - db-init-job.yaml

Kustomization Requirements

Namespace: Must match the app name
Labels: Must include standard Wild Cloud labels with includeSelectors: true
Resources: List all Kubernetes manifest files

Labeling Strategy

Wild Cloud uses Kustomize's includeSelectors: true feature to automatically apply standard labels to all resources AND their selectors:

labels:
  - includeSelectors: true
    pairs:
      app: myapp              # App name (matches directory)
      managedBy: kustomize
      partOf: wild-cloud

This means individual resources can use simple, component-specific selectors like component: web, and Kustomize will automatically expand them to include all Wild Cloud labels.

Do NOT use Helm-style labels (app.kubernetes.io/name, app.kubernetes.io/instance). Use simple component labels (component: web, component: worker, etc.) instead.

Configuration Templates

Gomplate Templating

Resource files in this repository are templates that get compiled when users add apps via the web app, CLI, or API. Only variables defined in the manifest file's 'defaultConfig' section are available to the resource templates. Use gomplate syntax to reference configuration:

External DNS

Ingress resources should include external-dns annotations for automatic DNS management:

annotations:
  external-dns.alpha.kubernetes.io/target: {{ .domain }}
  external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"

Note: 'domain' must be defined in the app manifest's 'defaultConfig' section.

This creates a CNAME from the app subdomain to the cluster domain (e.g., myapp.cloud.example.com → cloud.example.com).

App Dependencies and Reference Mapping

How Dependency References Work

When an app depends on other apps, the reference system allows flexibility in naming while maintaining clear relationships:

Define dependencies in your manifest with optional aliases:

requires:
  - name: postgres      # Actual app to depend on
    alias: db          # Optional: how to reference it in templates
  - name: redis        # No alias means use 'redis' as reference

At installation time, the system:
- Prompts user to map dependencies to actual installed apps
- Sets installedAs field in the local app manifest to track the mapping
- Example: User might have postgres-primary installed, mapped to the db dependency

Example: Multiple Database Instances

If a user has multiple PostgreSQL instances:

# User's config.yaml
apps:
  postgres-primary:
    hostname: primary.postgres.svc.cluster.local
  postgres-analytics:
    hostname: analytics.postgres.svc.cluster.local

When adding an app that requires postgres, they can choose which instance to use, and the system tracks this in the manifest's installedAs field.

Database Patterns

Database Initialization Jobs

Apps requiring PostgreSQL or MySQL should include a database initialization job (db-init-job.yaml):

Purpose:

Creates the application database (if it doesn't exist)
Creates/updates the application user with proper credentials
Grants necessary permissions
Installs required database extensions (e.g., PostgreSQL's vector, cube, earthdistance)

Implementation requirements:

Use restartPolicy: OnFailure
Include in kustomization.yaml resources
Use appropriate security context (e.g., runAsUser: 999 for PostgreSQL)

Example apps: immich, gitea, openproject, discourse

Database URL Configuration

When apps need database URLs with embedded credentials, always use a dedicated dbUrl secret.

❌ Wrong - Kustomize cannot process runtime env var substitution:

- name: DB_URL
  value: "postgresql://user:$(DB_PASSWORD)@host/db"  # This won't work!

✅ Correct - Use a dedicated secret:

- name: DB_URL
  valueFrom:
    secretKeyRef:
      name: myapp-secrets
      key: apps.myapp.dbUrl

Add apps.myapp.dbUrl to your manifest's defaultSecrets, and the system will generate the complete URL with embedded credentials automatically when the app is added.

Security Requirements

Security Contexts

All pods must comply with Pod Security Standards. Include security contexts at both pod and container levels:

spec:
  template:
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 999        # Use appropriate non-root UID
        runAsGroup: 999       # Use appropriate GID
        seccompProfile:
          type: RuntimeDefault
      containers:
      - name: container-name
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop: [ALL]
          readOnlyRootFilesystem: false  # Set to true when possible

Common user IDs:

PostgreSQL: runAsUser: 999
Redis: runAsUser: 999
MySQL: Consult the container image documentation

Secrets Management

Secrets are managed through two mechanisms: default secrets for the app itself and required secrets from dependencies.

In manifest:

defaultSecrets:
  key: dbPassword      # This app's database password
  key: apiKey          # This app's API key
requiredSecrets:
  - db.password       # Password from postgres dependency (aliased as 'db')
  - redis.auth        # Auth from redis dependency

In resources:

env:
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: myapp-secrets
        key: dbPassword    # Points to the default secret
  - name: POSTGRES_PASSWORD
    valueFrom:
      secretKeyRef:
        name: myapp-secrets
        key: db.password    # Points to the required secret

Secret workflow:

Define app's own secrets in defaultSecrets (key, default mappings)
Reference dependency secrets in requiredSecrets (list)
When adding an app, the system:
- Generates random values for empty defaultSecrets
- Copies referenced secrets from dependencies
- Stores all in the instance's secrets.yaml
When deploying, creates a Kubernetes Secret named <app-name>-secrets containing:
- All defaultSecrets with key format: <key>
- All requiredSecrets with key format: <app-ref>.<key>

Key collision handling: If the same key exists in both defaultSecrets and requiredSecrets, the requiredSecrets value takes precedence. Authors should ensure their local secrets don't collide with their required secrets.

Important: Never commit secrets.yaml to Git. Templates should only reference secrets, never contain actual secret values.

Converting from Helm Charts

Wild Cloud prefers Kustomize over Helm for simplicity and Git-friendliness. When an official Helm chart exists, convert it rather than creating manifests from scratch.

Conversion Process

Extract and render the Helm chart:

helm fetch --untar --untardir charts repo/chart-name
helm template --output-dir base --namespace myapp --values values.yaml myapp charts/chart-name
cd base/chart-name

Add namespace manifest:

cat <<EOF > namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: myapp
EOF

Create kustomization:

kustomize create --autodetect

Convert to Wild Cloud format:
- Create manifest.yaml with app metadata
- Replace hardcoded values with gomplate variables (e.g., {{ .cloud.domain }})
- Update secrets to use dotted-path convention
- Replace Helm labels with Wild Cloud standard labels
- Add includeSelectors: true to kustomization
- Use simple component labels (component: web, not app.kubernetes.io/name)
- Add security contexts to all pods
- Add external-dns annotations to ingresses

Example Label Migration

❌ Helm style:

labels:
  app.kubernetes.io/name: myapp
  app.kubernetes.io/instance: release-name
  app.kubernetes.io/component: server

✅ Wild Cloud style:

# In kustomization.yaml (applied automatically)
labels:
  - includeSelectors: true
    pairs:
      app: myapp
      managedBy: kustomize
      partOf: wild-cloud

# In individual resources
labels:
  component: server  # Simple component label

Validation Checklist

Before submitting a new or modified app, verify:

Manifest
- name matches directory name
- All required fields present (name, description, version, defaultConfig)
- All template variables defined in defaultConfig
- defaultSecrets uses maps with 'key' and 'default' attributes
- requiredSecrets references use <app-ref>.<key> format
- Dependencies listed in requires with optional alias fields
- Manifest template references match dependency aliases or names
Kustomization
- Includes standard Wild Cloud labels with includeSelectors: true
- Namespace matches app name
- All resource files listed under resources:
Resources
- Security contexts on all pods (both pod-level and container-level)
- Simple component labels, no Helm-style labels
- Ingresses include external-dns annotations
- Database apps include init jobs (if applicable)
Testing
- Templates compile successfully with sample config
- App deploys without errors in test cluster
- All dependencies work correctly

Contributing

Contributions are welcome! To contribute:

Fork the repository
Create a new app directory following the structure above
Test your app thoroughly
Submit a pull request with:
- Description of the app and its purpose
- Any special configuration notes
- Dependencies required

Notice: Third-Party Software

The Kubernetes manifests and Kustomize files in this directory are designed to deploy third-party software.

Unless otherwise stated, the software deployed by these manifests is not authored or maintained by this project. All copyrights, licenses, and responsibilities for that software remain with the respective upstream authors.

These files are provided solely for convenience and automation. Users are responsible for reviewing and complying with the licenses of the software they deploy.

This project is licensed under the GNU AGPLv3 or later, but this license does not apply to the third-party software being deployed.

See individual deployment directories for upstream project links and container sources.

20 KiB Raw Blame History