From f68ff43879c1e84eeab193266a8e32f5ac8550aa Mon Sep 17 00:00:00 2001 From: Paul Payne Date: Sat, 18 Oct 2025 18:57:35 +0000 Subject: [PATCH] Updates documentation. --- ADDING-APPS.md | 329 ++++++++++++++++++++++++++++++------------------- CLAUDE.md | 157 +++++++++++++++++++++++ README.md | 283 +++++++----------------------------------- 3 files changed, 404 insertions(+), 365 deletions(-) create mode 100644 CLAUDE.md diff --git a/ADDING-APPS.md b/ADDING-APPS.md index a110db6..5298e5c 100644 --- a/ADDING-APPS.md +++ b/ADDING-APPS.md @@ -1,16 +1,22 @@ -# Adding custom apps +# Adding Wild Cloud Apps -Custom apps can be added to your Wild Cloud apps directory. +This guide is for contributors and maintainers who want to create or modify Wild Cloud apps. If you're looking to use existing apps, see [README.md](README.md). -Custom apps can be deployed using Wild Cloud scripts. Wild Cloud apps follow a specific structure and naming convention to ensure compatibility with the Wild Cloud ecosystem. +## Overview -## App Structure +Wild Cloud apps are Kubernetes applications packaged as Kustomize configurations with standardized conventions for configuration management, secrets handling, and deployment. -Each subdirectory in this directory represents a Wild Cloud app. Each app directory contains an "app manifest" (`manifest.yaml`), a "kustomization" (`kustomization.yaml`), and one or more "configurations" (yaml files containing definitions/configurations of Kubernetes objects/resources). +## Required Files -### App Manifest +Each app directory must contain: -The required `manifest.yaml` file contains metadata about the app. +1. **`manifest.yaml`** - App metadata and configuration schema +2. **`kustomization.yaml`** - Kustomize configuration with Wild Cloud labels +3. **Resource files** - Kubernetes manifests (deployments, services, ingresses, etc.) + +### App Manifest (`manifest.yaml`) + +The manifest defines the app's metadata, dependencies, configuration schema, and secret requirements. This is the contents of an example `manifest.yaml` file for an app named "immich": @@ -39,21 +45,23 @@ requiredSecrets: - apps.postgres.password ``` -Explanation of the fields: +#### Manifest Fields -- `name`: The name of the app, used for identification. -- `description`: A brief description of the app. -- `version`: The version of the app. This should generally follow the versioning scheme of the app itself. -- `icon`: A URL to an icon representing the app. -- `requires`: A list of other apps that this app depends on. Each entry should be the name of another app. -- `defaultConfig`: A set of default configuration values for the app. When an app is added using `wild-app-add`, these values will be added to the Wild Cloud `config.yaml` file. -- `requiredSecrets`: A list of secrets that must be set in the Wild Cloud `secrets.yaml` file for the app to function properly. These secrets are typically sensitive information like database passwords or API keys. Keys with random values will be generated automatically when the app is added. +| Field | Required | Description | +|-------|----------|-------------| +| `name` | Yes | App identifier (must match directory name) | +| `description` | Yes | Brief app description shown in listings | +| `version` | Yes | App version (follow upstream versioning) | +| `icon` | No | URL to app icon for UI display | +| `requires` | No | List of dependency apps (e.g., `postgres`, `redis`) | +| `defaultConfig` | Yes | Default configuration values merged into operator's `config.yaml` | +| `requiredSecrets` | No | List of secrets in dotted-path format (e.g., `apps.appname.dbPassword`) | -### Kustomization +**Important:** All configuration keys referenced in templates (via `{{ .apps.appname.key }}`) must be defined in `defaultConfig` or be standard Wild Cloud variables. -Each app directory should also contain a `kustomization.yaml` file. This file defines how the app's Kubernetes resources are built and deployed. It can include references to other Kustomize files, patches, and configurations. +### Kustomization (`kustomization.yaml`) -Here is an example `kustomization.yaml` file for the "immich" app: +The kustomization file defines how Kubernetes resources are built and applies Wild Cloud's standard labels. ```yaml apiVersion: kustomize.config.k8s.io/v1beta1 @@ -74,105 +82,108 @@ resources: - pvc.yaml - service.yaml - db-init-job.yaml - ``` +``` -Kustomization requirements: +#### Kustomization Requirements -- Every Wild Cloud kustomization should include the Wild Cloud labels in its `kustomization.yaml` file. This allows the Wild Cloud to identify and manage the app correctly. The labels should be defined under the `labels` key, as shown in the example above. -- The `app` label and `namespace` keys should the app's name/directory. +- **Namespace**: Must match the app name +- **Labels**: Must include standard Wild Cloud labels with `includeSelectors: true` +- **Resources**: List all Kubernetes manifest files -#### Standard Wild Cloud Labels +#### Labeling Strategy -Wild Cloud uses a consistent labeling strategy across all apps: +Wild Cloud uses Kustomize's `includeSelectors: true` feature to automatically apply standard labels to all resources AND their selectors: ```yaml labels: - includeSelectors: true pairs: - app: myapp # The app name (matches directory) - managedBy: kustomize # Managed by Kustomize - partOf: wild-cloud # Part of Wild Cloud ecosystem + app: myapp # App name (matches directory) + managedBy: kustomize + partOf: wild-cloud ``` -The `includeSelectors: true` setting automatically applies these labels to all resources AND their selectors, which means: +This means individual resources can use simple, component-specific selectors like `component: web`, and Kustomize will automatically expand them to include all Wild Cloud labels. -1. **Resource labels** - All resources get the standard Wild Cloud labels -2. **Selector labels** - All selectors automatically include these labels for robust selection +**Do NOT use Helm-style labels** (`app.kubernetes.io/name`, `app.kubernetes.io/instance`). Use simple component labels (`component: web`, `component: worker`, etc.) instead. -This allows individual resources to use simple, component-specific selectors: +## Configuration Templates + +### Gomplate Templating + +Resource files in this repository are **templates** that get compiled when operators run `wild-app-add`. Use gomplate syntax to reference configuration: ```yaml -selector: - matchLabels: - component: web +# Common template variables +domain: {{ .cloud.domain }} # Operator's domain +email: {{ .operator.email }} # Operator's email +image: {{ .apps.myapp.serverImage }} # App-specific config +dbHost: {{ .apps.myapp.dbHostname }} # App-specific config ``` -Which Kustomize automatically expands to: +**Template variable sources:** +1. Standard Wild Cloud variables (`{{ .cloud.* }}`, `{{ .operator.* }}`) +2. App-specific variables defined in your manifest's `defaultConfig` + +All template variables must be defined in one of these locations. The compiled files are placed in the operator's Wild Cloud home directory as standard Kubernetes manifests. + +### External DNS + +Ingress resources should include external-dns annotations for automatic DNS management: ```yaml -selector: - matchLabels: - app: myapp - component: web - managedBy: kustomize - partOf: wild-cloud +annotations: + external-dns.alpha.kubernetes.io/target: {{ .cloud.domain }} + external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" ``` -### Configuration Files +This creates a CNAME from the app subdomain to the cluster domain (e.g., `myapp.cloud.example.com` → `cloud.example.com`). -Wild Cloud apps use Kustomize as kustomize files are simple, transparent, and easier to manage in a Git repository. +## Database Patterns -#### Templates +### Database Initialization Jobs -For operators, Wild Cloud apps use standard configuration files. This makes modifying the app's configuration straightforward, as operators can customize their app files as needed. They can choose to manage modifications and updates directly on the configuration files using `git` tools, or they can use Kustomize patches or overlays. As a convenience for operators, when adding an app (using `wild-app-add`), the app's configurations will be compiled with the operator's Wild Cloud configuration and secrets. This results in standard Kustomize files being placed in the Wild Cloud home directory, which can then be modified as needed. This means the configuration files in this repository are actually templates, but they will be compiled into standard Kustomize files when the app is added to an operator's Wild Cloud home directory. +Apps requiring PostgreSQL or MySQL should include a database initialization job (`db-init-job.yaml`): -To reference operator configuration in the configuration files, use gomplate variables, such as `{{ .cloud.domain }}` for the domain name. All configuration variables you use need to exist in the operator's `config.yaml`, so they should be either standard Wild Cloud operator variables, or be defined in the app's `manifest.yaml` under `defaultConfig`. +**Purpose:** +- Creates the application database (if it doesn't exist) +- Creates/updates the application user with proper credentials +- Grants necessary permissions +- Installs required database extensions (e.g., PostgreSQL's `vector`, `cube`, `earthdistance`) -When `wild-app-add` is run, the app's Kustomize files will be compiled with the operator's Wild Cloud configuration and secrets resulting in standard Kustomize files being placed in the Wild Cloud home directory. +**Implementation requirements:** +- Use `restartPolicy: OnFailure` +- Include in `kustomization.yaml` resources +- Use appropriate security context (e.g., `runAsUser: 999` for PostgreSQL) -#### External DNS Configuration +**Example apps:** `immich`, `gitea`, `openproject`, `discourse` -Wild Cloud apps use external-dns annotations in their ingress resources to automatically manage DNS records: +### Database URL Configuration -- `external-dns.alpha.kubernetes.io/target: {{ .cloud.domain }}` - Creates a CNAME record pointing the app subdomain to the main cluster domain (e.g., `ghost.cloud.payne.io` → `cloud.payne.io`) -- `external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"` - Disables Cloudflare proxy for direct DNS resolution +When apps need database URLs with embedded credentials, **always use a dedicated `dbUrl` secret**. -#### Database Initialization Jobs - -Apps that rely on PostgreSQL or MySQL databases typically need a database initialization job to create the required database and user before the main application starts. These jobs: - -- Run as Kubernetes Jobs that execute once and complete -- Create the application database if it doesn't exist -- Create the application user with appropriate permissions -- Should be included in the app's `kustomization.yaml` resources list -- Use the same database connection settings as the main application - -Examples of apps with db-init jobs: `gitea`, `codimd`, `immich`, `openproject` - -##### Database URL Configuration - -**Important:** When apps require database URLs with embedded credentials, always use a separate `dbUrl` secret instead of trying to construct the URL with environment variable substitution in Kustomize templates. - -❌ **Wrong** (Kustomize cannot process runtime env var substitution): +❌ **Wrong** - Kustomize cannot process runtime env var substitution: ```yaml - name: DB_URL - value: "postgresql://user:$(DB_PASSWORD)@host/db" + value: "postgresql://user:$(DB_PASSWORD)@host/db" # This won't work! ``` -✅ **Correct** (Use a dedicated secret): +✅ **Correct** - Use a dedicated secret: ```yaml - name: DB_URL valueFrom: secretKeyRef: - name: app-secrets - key: apps.appname.dbUrl + name: myapp-secrets + key: apps.myapp.dbUrl ``` -Add `apps.appname.dbUrl` to the manifest's `requiredSecrets` and the `wild-app-add` script will generate the complete URL with embedded credentials. +Add `apps.myapp.dbUrl` to your manifest's `requiredSecrets`, and `wild-app-add` will generate the complete URL with embedded credentials automatically. -##### Security Context Requirements +## Security Requirements -Pods must comply with Pod Security Standards. All pods should include proper security contexts to avoid deployment warnings: +### Security Contexts + +**All pods must comply with Pod Security Standards.** Include security contexts at both pod and container levels: ```yaml spec: @@ -180,8 +191,8 @@ spec: spec: securityContext: runAsNonRoot: true - runAsUser: 999 # Use appropriate non-root user ID - runAsGroup: 999 # Use appropriate group ID + runAsUser: 999 # Use appropriate non-root UID + runAsGroup: 999 # Use appropriate GID seccompProfile: type: RuntimeDefault containers: @@ -189,77 +200,147 @@ spec: securityContext: allowPrivilegeEscalation: false capabilities: - drop: - - ALL + drop: [ALL] readOnlyRootFilesystem: false # Set to true when possible ``` -For PostgreSQL init jobs, use `runAsUser: 999` (postgres user). For other database types, use the appropriate non-root user ID for that database container. +**Common user IDs:** +- PostgreSQL: `runAsUser: 999` +- Redis: `runAsUser: 999` +- MySQL: Consult the container image documentation -#### Secrets +### Secrets Management -Secrets are managed in the `secrets.yaml` file in the Wild Cloud home directory. The app's `manifest.yaml` should list any required secrets under `requiredSecrets`. When the app is added, default secret values will be generated and stored in the `secrets.yaml` file. Secrets are always stored and referenced in the `apps..` yaml path. When `wild-app-deploy` is run, a Secret resource will be created in the Kubernetes cluster with the name `-secrets`, containing all secrets defined in the manifest's `requiredSecrets` key. These secrets can then be referenced in the app's Kustomize files using a `secretKeyRef`. - -**Important:** Always use the full dotted path from the manifest as the secret key, not just the last segment. For example, to mount a secret in an environment variable, you would use: +Secrets use a **full dotted-path naming convention** to prevent naming conflicts: +**In manifest:** ```yaml -env: - - name: DB_PASSWORD - valueFrom: - secretKeyRef: - name: immich-secrets - key: apps.immich.dbPassword # Use full dotted path, not just "dbPassword" +requiredSecrets: + - apps.myapp.dbPassword + - apps.postgres.password ``` -This approach prevents naming conflicts between apps and makes secret keys more descriptive and consistent with the `secrets.yaml` structure. +**In resources:** +```yaml +env: + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: myapp-secrets + key: apps.myapp.dbPassword # Full dotted path, not just "dbPassword" +``` -`secrets.yaml` files should not be checked in to a git repository and are ignored by default in Wild Cloud home directories. Checked in kustomize files should only reference secrets, not compile them. +**Secret workflow:** +1. List secrets in manifest's `requiredSecrets` +2. `wild-app-add` generates random values in operator's `secrets.yaml` +3. `wild-app-deploy` creates a Kubernetes Secret named `-secrets` +4. Resources reference secrets using full dotted paths -## App Lifecycle +**Important:** Never commit `secrets.yaml` to Git. Templates should only reference secrets, never contain actual secret values. -Apps in Wild Cloud are managed by operators using a set of commands run from their Wild Cloud home directory. +## Converting from Helm Charts -- `wild-apps-list`: Lists all available apps. -- `wild-app-add `: Reads the app from the Wild Cloud repository, adds the app manifest to your Wild Cloud home `apps` directory, updates missing values in `config.yaml` and `secrets.yaml` with the app's default configurations, and compiles the app's Kustomize files. -- `wild-app-deploy `: Deploys the app to your Wild Cloud. +Wild Cloud prefers Kustomize over Helm for simplicity and Git-friendliness. When an official Helm chart exists, convert it rather than creating manifests from scratch. -## Contributing - -If you would like to contribute an app to the Wild Cloud, issue a pull request with the app's directory containing the `manifest.yaml` file and any necessary Kustomize files. Ensure that your app follows the structure outlined above. - -## Tips for App Packagers - -### Converting from Helm Charts - -Wild Cloud apps use Kustomize as kustomize files are simpler, more transparent, and easier to manage in a Git repository than Helm charts. - -IMPORTANT! If an official Helm chart is available for an app, it is recommended to convert that chart to a Wild Cloud app rather than creating a new app from scratch. - -If you have a Helm chart that you want to convert to a Wild Cloud app, the following example steps can simplify the process for you: +### Conversion Process +1. **Extract and render the Helm chart:** ```bash -helm fetch --untar --untardir charts nginx-stable/nginx-ingress -helm template --output-dir base --namespace ingress --values values.yaml ingress-controller charts/nginx-ingress -cat < base/nginx-ingress/namespace.yaml +helm fetch --untar --untardir charts repo/chart-name +helm template --output-dir base --namespace myapp --values values.yaml myapp charts/chart-name +cd base/chart-name +``` + +2. **Add namespace manifest:** +```bash +cat < namespace.yaml apiVersion: v1 kind: Namespace metadata: - name: ingress + name: myapp EOF -cd base/nginx-ingress +``` + +3. **Create kustomization:** +```bash kustomize create --autodetect ``` -After running these commands against your own Helm chart, you will have a Kustomize directory structure that can be used as a Wild Cloud app. All you need to do then, usually, is: +4. **Convert to Wild Cloud format:** + - Create `manifest.yaml` with app metadata + - Replace hardcoded values with gomplate variables (e.g., `{{ .cloud.domain }}`) + - Update secrets to use dotted-path convention + - Replace Helm labels with Wild Cloud standard labels + - Add `includeSelectors: true` to kustomization + - Use simple component labels (`component: web`, not `app.kubernetes.io/name`) + - Add security contexts to all pods + - Add external-dns annotations to ingresses -- add an app manifest (a `manifest.yaml` file). -- replace any hardcoded operator values with Wild Cloud operator variables, such as `{{ .cloud.domain }}` for the domain name. -- modify how secrets are referenced in the Kustomize files (see above) -- update labels and selectors to use the Wild Cloud standard: - - Replace complex Helm labels (like `app.kubernetes.io/name`, `app.kubernetes.io/instance`) with simple component labels - - Use `component: web`, `component: worker`, etc. in selectors and pod template labels - - Let Kustomize handle the common labels (`app`, `managedBy`, `partOf`) automatically -- remove any Helm-specific labels from the Kustomize files, as Wild Cloud apps do not use Helm labels. +### Example Label Migration + +❌ **Helm style:** +```yaml +labels: + app.kubernetes.io/name: myapp + app.kubernetes.io/instance: release-name + app.kubernetes.io/component: server +``` + +✅ **Wild Cloud style:** +```yaml +# In kustomization.yaml (applied automatically) +labels: + - includeSelectors: true + pairs: + app: myapp + managedBy: kustomize + partOf: wild-cloud + +# In individual resources +labels: + component: server # Simple component label +``` + +## Validation Checklist + +Before submitting a new or modified app, verify: + +- [ ] **Manifest** + - [ ] `name` matches directory name + - [ ] All required fields present (`name`, `description`, `version`, `defaultConfig`) + - [ ] All template variables defined in `defaultConfig` or are standard Wild Cloud variables + - [ ] Secrets use dotted-path format (e.g., `apps.appname.secretname`) + - [ ] Dependencies listed in `requires` (if any) + +- [ ] **Kustomization** + - [ ] Includes standard Wild Cloud labels with `includeSelectors: true` + - [ ] Namespace matches app name + - [ ] All resource files listed under `resources:` + +- [ ] **Resources** + - [ ] All hardcoded values replaced with gomplate variables + - [ ] Secrets reference full dotted paths + - [ ] Security contexts on all pods (both pod-level and container-level) + - [ ] Simple component labels, no Helm-style labels + - [ ] Ingresses include external-dns annotations + - [ ] Database apps include init jobs (if applicable) + +- [ ] **Testing** + - [ ] Templates compile successfully with sample config + - [ ] App deploys without errors in test cluster + - [ ] All dependencies work correctly + +## Contributing + +Contributions are welcome! To contribute: + +1. Fork the repository +2. Create a new app directory following the structure above +3. Test your app thoroughly +4. Submit a pull request with: + - Description of the app and its purpose + - Any special configuration notes + - Dependencies required ## Notice: Third-Party Software diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..004e967 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,157 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Overview + +This repository contains the Wild Cloud apps directory - a collection of Kubernetes applications packaged as Kustomize configurations. Each app is a self-contained directory with standardized manifests that can be deployed to Wild Cloud clusters using Wild Cloud CLI tools. + +## Repository Architecture + +### App Structure + +Each app follows a strict structure: +- **`manifest.yaml`** - App metadata, dependencies, default configuration, and secret requirements +- **`kustomization.yaml`** - Kustomize configuration with standard Wild Cloud labels +- **Resource files** - Kubernetes objects (deployments, services, ingresses, PVCs, jobs, etc.) + +### Templating System + +Configuration files use **gomplate templating** to reference operator configuration: +- Use `{{ .cloud.domain }}` for the operator's domain +- Use `{{ .apps.appname.configKey }}` for app-specific configuration +- Use `{{ .operator.email }}` for operator email +- All template variables must be defined in either the app's `manifest.yaml` under `defaultConfig` or be standard Wild Cloud operator variables + +Templates are compiled by `wild-app-add` when operators add apps to their Wild Cloud home directory. + +### Label Strategy + +Wild Cloud uses a consistent labeling approach powered by Kustomize's `includeSelectors: true` feature: + +```yaml +labels: + - includeSelectors: true + pairs: + app: appname # App name (matches directory) + managedBy: kustomize + partOf: wild-cloud +``` + +This automatically applies labels to all resources AND their selectors. Individual resources can use simple component-specific selectors like `component: web` or `component: worker`, and Kustomize will expand them to include the standard Wild Cloud labels. + +**Important:** Do NOT use Helm-style labels (`app.kubernetes.io/name`, `app.kubernetes.io/instance`). Use simple component labels instead. + +## Working with Apps + +### Creating/Modifying Apps + +1. **Manifest fields:** + - `name` - Must match directory name + - `description` - Brief app description + - `version` - App version (follow upstream versioning) + - `icon` - URL to app icon + - `requires` - List of dependency apps (e.g., `postgres`, `redis`, `memcached`) + - `defaultConfig` - Default configuration values (will be added to operator's `config.yaml`) + - `requiredSecrets` - List of secrets in dotted path format (e.g., `apps.appname.dbPassword`) + +2. **Kustomization requirements:** + - Must include standard Wild Cloud labels with `includeSelectors: true` + - Namespace must match app name + - List all resource files under `resources:` + +3. **Security contexts:** + All pods must comply with Pod Security Standards: + ```yaml + securityContext: + runAsNonRoot: true + runAsUser: 999 # Use appropriate non-root UID + runAsGroup: 999 + seccompProfile: + type: RuntimeDefault + containers: + - securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false # Set to true when possible + ``` + +### Secrets Management + +Secrets use **full dotted paths** as keys: +```yaml +env: + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: appname-secrets + key: apps.appname.dbPassword # Full path, not just "dbPassword" +``` + +**Database URL secrets:** When apps need database URLs with embedded credentials, always use a dedicated `dbUrl` secret in `requiredSecrets`. Do NOT try to construct URLs with env var substitution in templates - Kustomize cannot process runtime environment variables. + +### Database Initialization Jobs + +Apps requiring PostgreSQL/MySQL databases should include a `db-init-job.yaml`: +- Creates the app database if it doesn't exist +- Creates/updates the app user with proper password +- Grants appropriate permissions +- For PostgreSQL jobs: use `runAsUser: 999` (postgres user) +- Include any required database extensions (e.g., Immich requires `vector`, `cube`, `earthdistance`) + +Examples: [immich/db-init-job.yaml](immich/db-init-job.yaml), [gitea/db-init-job.yaml](gitea/db-init-job.yaml) + +### External DNS Configuration + +Ingress resources should include external-dns annotations: +```yaml +annotations: + external-dns.alpha.kubernetes.io/target: {{ .cloud.domain }} + external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" +``` + +This creates CNAME records pointing app subdomains to the main cluster domain. + +### Converting Helm Charts + +Wild Cloud prefers Kustomize over Helm for transparency and Git-friendliness. To convert a Helm chart: + +```bash +helm fetch --untar --untardir charts repo/chart-name +helm template --output-dir base --namespace namespace --values values.yaml release-name charts/chart-name +cd base/chart-name +kustomize create --autodetect +``` + +Then: +1. Add `manifest.yaml` +2. Replace hardcoded values with gomplate variables +3. Update secrets to use Wild Cloud's dotted-path approach +4. Replace Helm labels with simple component labels +5. Add standard Wild Cloud labels to `kustomization.yaml` +6. Add security contexts to all pods + +## Common Wild Cloud Commands + +These commands are run by operators from their Wild Cloud home directory (not this repository): + +- `wild-apps-list` - List all available apps +- `wild-app-add ` - Add an app (compiles templates, updates config/secrets) +- `wild-app-deploy ` - Deploy an app to the cluster + +## Validation + +Before submitting changes: +1. Ensure `manifest.yaml` has all required fields +2. Verify `kustomization.yaml` includes standard Wild Cloud labels +3. Check all templates use valid configuration paths defined in `defaultConfig` +4. Confirm secrets use full dotted-path keys +5. Verify all pods have proper security contexts +6. Test template compilation works with sample operator config + +## Examples of Well-Structured Apps + +- **immich** - Multi-deployment app with database init, multiple services, PostgreSQL extensions +- **openproject** - App with multiple dependencies (postgres, memcached), configmap-based configuration +- **gitea** - Standard app with database init, PVC for storage diff --git a/README.md b/README.md index 606eeb2..d329712 100644 --- a/README.md +++ b/README.md @@ -1,265 +1,66 @@ -# Wild Cloud-maintained apps +# Wild Cloud Apps Directory -This is the Wild Cloud apps repository. +This is the official Wild Cloud apps repository containing a curated collection of self-hosted applications that can be deployed to your Wild Cloud cluster. -This repository contains a collection of apps that can be deployed using Wild Cloud scripts. Wild Cloud apps follow a specific structure and naming convention to ensure compatibility with the Wild Cloud ecosystem. +## What are Wild Cloud Apps? -## App Structure +Wild Cloud apps are pre-packaged Kubernetes applications using Kustomize that follow standardized conventions for configuration, secrets management, and deployment. Each app includes: -Each subdirectory in this directory represents a Wild Cloud app. Each app directory contains an "app manifest" (`manifest.yaml`), a "kustomization" (`kustomization.yaml`), and one or more "configurations" (yaml files containing definitions/configurations of Kubernetes objects/resources). +- **App manifest** (`manifest.yaml`) - Metadata, dependencies, and default configuration +- **Kustomization** (`kustomization.yaml`) - Kubernetes resource definitions +- **Configuration templates** - Deployments, services, ingresses, and other resources -### App Manifest +Apps use gomplate templates that compile with your Wild Cloud configuration when added, making them easy to customize while maintaining a consistent deployment experience. -The required `manifest.yaml` file contains metadata about the app. +## Using Wild Cloud Apps -This is the contents of an example `manifest.yaml` file for an app named "immich": +### Commands -```yaml -name: immich -description: Immich is a self-hosted photo and video backup solution that allows you to store, manage, and share your media files securely. -version: 1.0.0 -icon: https://immich.app/assets/images/logo.png -requires: - - name: redis - - name: postgres -defaultConfig: - serverImage: ghcr.io/immich-app/immich-server:release - mlImage: ghcr.io/immich-app/immich-machine-learning:release - timezone: UTC - serverPort: 2283 - mlPort: 3003 - storage: 250Gi - cacheStorage: 10Gi - redisHostname: redis.redis.svc.cluster.local - dbHostname: postgres.postgres.svc.cluster.local - dbUsername: immich - domain: immich.{{ .cloud.domain }} -requiredSecrets: - - apps.immich.dbPassword - - apps.postgres.password -``` +Run these commands from your Wild Cloud home directory: -Explanation of the fields: +- **`wild-apps-list`** - Browse all available apps in the repository +- **`wild-app-add `** - Add an app to your cluster: + - Copies the app manifest to your `apps` directory + - Adds default configuration to your `config.yaml` + - Generates required secrets in your `secrets.yaml` + - Compiles templates with your configuration +- **`wild-app-deploy `** - Deploy or update the app in your cluster -- `name`: The name of the app, used for identification. -- `description`: A brief description of the app. -- `version`: The version of the app. This should generally follow the versioning scheme of the app itself. -- `icon`: A URL to an icon representing the app. -- `requires`: A list of other apps that this app depends on. Each entry should be the name of another app. -- `defaultConfig`: A set of default configuration values for the app. When an app is added using `wild-app-add`, these values will be added to the Wild Cloud `config.yaml` file. -- `requiredSecrets`: A list of secrets that must be set in the Wild Cloud `secrets.yaml` file for the app to function properly. These secrets are typically sensitive information like database passwords or API keys. Keys with random values will be generated automatically when the app is added. +### How It Works -### Kustomization +1. **Add an app**: `wild-app-add` compiles the app's templates using your Wild Cloud configuration (domain, email, etc.) and creates standard Kustomize files in your Wild Cloud home directory +2. **Customize** (optional): Modify the compiled files in your home directory, or use Kustomize patches/overlays +3. **Deploy**: `wild-app-deploy` applies the Kustomize configuration to your cluster +4. **Manage**: Track changes with Git, update configurations as needed -Each app directory should also contain a `kustomization.yaml` file. This file defines how the app's Kubernetes resources are built and deployed. It can include references to other Kustomize files, patches, and configurations. +### Dependencies -Here is an example `kustomization.yaml` file for the "immich" app: +Some apps require other apps to function. For example: +- **Immich** requires PostgreSQL and Redis +- **OpenProject** requires PostgreSQL and Memcached +- **Gitea** requires PostgreSQL -```yaml -apiVersion: kustomize.config.k8s.io/v1beta1 -kind: Kustomization -namespace: immich -labels: - - includeSelectors: true - pairs: - app: immich - managedBy: kustomize - partOf: wild-cloud -resources: - - deployment-server.yaml - - deployment-machine-learning.yaml - - deployment-microservices.yaml - - ingress.yaml - - namespace.yaml - - pvc.yaml - - service.yaml - - db-init-job.yaml - ``` +When you add an app, check its `requires` field in the manifest and ensure dependencies are added first. -Kustomization requirements: +## Available Apps -- Every Wild Cloud kustomization should include the Wild Cloud labels in its `kustomization.yaml` file. This allows the Wild Cloud to identify and manage the app correctly. The labels should be defined under the `labels` key, as shown in the example above. -- The `app` label and `namespace` keys should the app's name/directory. +This repository includes apps for: +- Content management (Ghost, Discourse) +- Project management (OpenProject) +- Photo management (Immich) +- Code hosting (Gitea) +- Email marketing (Listmonk, Keila) +- AI interfaces (Open WebUI, vLLM) +- Databases (PostgreSQL, MySQL, Redis, Memcached) +- And more... -#### Standard Wild Cloud Labels - -Wild Cloud uses a consistent labeling strategy across all apps: - -```yaml -labels: - - includeSelectors: true - pairs: - app: myapp # The app name (matches directory) - managedBy: kustomize # Managed by Kustomize - partOf: wild-cloud # Part of Wild Cloud ecosystem -``` - -The `includeSelectors: true` setting automatically applies these labels to all resources AND their selectors, which means: - -1. **Resource labels** - All resources get the standard Wild Cloud labels -2. **Selector labels** - All selectors automatically include these labels for robust selection - -This allows individual resources to use simple, component-specific selectors: - -```yaml -selector: - matchLabels: - component: web -``` - -Which Kustomize automatically expands to: - -```yaml -selector: - matchLabels: - app: myapp - component: web - managedBy: kustomize - partOf: wild-cloud -``` - -### Configuration Files - -Wild Cloud apps use Kustomize as kustomize files are simple, transparent, and easier to manage in a Git repository. - -#### Templates - -For operators, Wild Cloud apps use standard configuration files. This makes modifying the app's configuration straightforward, as operators can customize their app files as needed. They can choose to manage modifications and updates directly on the configuration files using `git` tools, or they can use Kustomize patches or overlays. As a convenience for operators, when adding an app (using `wild-app-add`), the app's configurations will be compiled with the operator's Wild Cloud configuration and secrets. This results in standard Kustomize files being placed in the Wild Cloud home directory, which can then be modified as needed. This means the configuration files in this repository are actually templates, but they will be compiled into standard Kustomize files when the app is added to an operator's Wild Cloud home directory. - -To reference operator configuration in the configuration files, use gomplate variables, such as `{{ .cloud.domain }}` for the domain name. All configuration variables you use need to exist in the operator's `config.yaml`, so they should be either standard Wild Cloud operator variables, or be defined in the app's `manifest.yaml` under `defaultConfig`. - -When `wild-app-add` is run, the app's Kustomize files will be compiled with the operator's Wild Cloud configuration and secrets resulting in standard Kustomize files being placed in the Wild Cloud home directory. - -#### External DNS Configuration - -Wild Cloud apps use external-dns annotations in their ingress resources to automatically manage DNS records: - -- `external-dns.alpha.kubernetes.io/target: {{ .cloud.domain }}` - Creates a CNAME record pointing the app subdomain to the main cluster domain (e.g., `ghost.cloud.payne.io` → `cloud.payne.io`) -- `external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"` - Disables Cloudflare proxy for direct DNS resolution - -#### Database Initialization Jobs - -Apps that rely on PostgreSQL or MySQL databases typically need a database initialization job to create the required database and user before the main application starts. These jobs: - -- Run as Kubernetes Jobs that execute once and complete -- Create the application database if it doesn't exist -- Create the application user with appropriate permissions -- Should be included in the app's `kustomization.yaml` resources list -- Use the same database connection settings as the main application - -Examples of apps with db-init jobs: `gitea`, `codimd`, `immich`, `openproject` - -##### Database URL Configuration - -**Important:** When apps require database URLs with embedded credentials, always use a separate `dbUrl` secret instead of trying to construct the URL with environment variable substitution in Kustomize templates. - -❌ **Wrong** (Kustomize cannot process runtime env var substitution): -```yaml -- name: DB_URL - value: "postgresql://user:$(DB_PASSWORD)@host/db" -``` - -✅ **Correct** (Use a dedicated secret): -```yaml -- name: DB_URL - valueFrom: - secretKeyRef: - name: app-secrets - key: apps.appname.dbUrl -``` - -Add `apps.appname.dbUrl` to the manifest's `requiredSecrets` and the `wild-app-add` script will generate the complete URL with embedded credentials. - -##### Security Context Requirements - -Pods must comply with Pod Security Standards. All pods should include proper security contexts to avoid deployment warnings: - -```yaml -spec: - template: - spec: - securityContext: - runAsNonRoot: true - runAsUser: 999 # Use appropriate non-root user ID - runAsGroup: 999 # Use appropriate group ID - seccompProfile: - type: RuntimeDefault - containers: - - name: container-name - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - readOnlyRootFilesystem: false # Set to true when possible -``` - -For PostgreSQL init jobs, use `runAsUser: 999` (postgres user). For other database types, use the appropriate non-root user ID for that database container. - -#### Secrets - -Secrets are managed in the `secrets.yaml` file in the Wild Cloud home directory. The app's `manifest.yaml` should list any required secrets under `requiredSecrets`. When the app is added, default secret values will be generated and stored in the `secrets.yaml` file. Secrets are always stored and referenced in the `apps..` yaml path. When `wild-app-deploy` is run, a Secret resource will be created in the Kubernetes cluster with the name `-secrets`, containing all secrets defined in the manifest's `requiredSecrets` key. These secrets can then be referenced in the app's Kustomize files using a `secretKeyRef`. - -**Important:** Always use the full dotted path from the manifest as the secret key, not just the last segment. For example, to mount a secret in an environment variable, you would use: - -```yaml -env: - - name: DB_PASSWORD - valueFrom: - secretKeyRef: - name: immich-secrets - key: apps.immich.dbPassword # Use full dotted path, not just "dbPassword" -``` - -This approach prevents naming conflicts between apps and makes secret keys more descriptive and consistent with the `secrets.yaml` structure. - -`secrets.yaml` files should not be checked in to a git repository and are ignored by default in Wild Cloud home directories. Checked in kustomize files should only reference secrets, not compile them. - -## App Lifecycle - -Apps in Wild Cloud are managed by operators using a set of commands run from their Wild Cloud home directory. - -- `wild-apps-list`: Lists all available apps. -- `wild-app-add `: Reads the app from the Wild Cloud repository, adds the app manifest to your Wild Cloud home `apps` directory, updates missing values in `config.yaml` and `secrets.yaml` with the app's default configurations, and compiles the app's Kustomize files. -- `wild-app-deploy `: Deploys the app to your Wild Cloud. +Run `wild-apps-list` to see the full catalog with descriptions. ## Contributing -If you would like to contribute an app to the Wild Cloud, issue a pull request with the app's directory containing the `manifest.yaml` file and any necessary Kustomize files. Ensure that your app follows the structure outlined above. +Want to add a new app or improve an existing one? See [ADDING-APPS.md](ADDING-APPS.md) for detailed guidance on creating Wild Cloud apps. -## Tips for App Packagers - -### Converting from Helm Charts - -Wild Cloud apps use Kustomize as kustomize files are simpler, more transparent, and easier to manage in a Git repository than Helm charts. - -IMPORTANT! If an official Helm chart is available for an app, it is recommended to convert that chart to a Wild Cloud app rather than creating a new app from scratch. - -If you have a Helm chart that you want to convert to a Wild Cloud app, the following example steps can simplify the process for you: - -```bash -helm fetch --untar --untardir charts nginx-stable/nginx-ingress -helm template --output-dir base --namespace ingress --values values.yaml ingress-controller charts/nginx-ingress -cat < base/nginx-ingress/namespace.yaml -apiVersion: v1 -kind: Namespace -metadata: - name: ingress -EOF -cd base/nginx-ingress -kustomize create --autodetect -``` - -After running these commands against your own Helm chart, you will have a Kustomize directory structure that can be used as a Wild Cloud app. All you need to do then, usually, is: - -- add an app manifest (a `manifest.yaml` file). -- replace any hardcoded operator values with Wild Cloud operator variables, such as `{{ .cloud.domain }}` for the domain name. -- modify how secrets are referenced in the Kustomize files (see above) -- update labels and selectors to use the Wild Cloud standard: - - Replace complex Helm labels (like `app.kubernetes.io/name`, `app.kubernetes.io/instance`) with simple component labels - - Use `component: web`, `component: worker`, etc. in selectors and pod template labels - - Let Kustomize handle the common labels (`app`, `managedBy`, `partOf`) automatically -- remove any Helm-specific labels from the Kustomize files, as Wild Cloud apps do not use Helm labels. +Contributions are welcome via pull requests. Ensure your app follows the Wild Cloud conventions and includes all required files. ## Notice: Third-Party Software