diff --git a/ADDING-APPS-NOTES.md b/ADDING-APPS-NOTES.md deleted file mode 100644 index 1479cc7..0000000 --- a/ADDING-APPS-NOTES.md +++ /dev/null @@ -1,362 +0,0 @@ -# Adding Apps - Notes & Documentation Improvements - -## Documentation Improvement Candidates - -Issues and gaps discovered while adding apps. These should be folded back into ADDING-APPS.md. - -### 1. Node.js app memory requirements -Node.js apps (NocoDB, Outline, etc.) need more than 512Mi memory. NocoDB crashed with OOM at 512Mi. -- Recommended: 1Gi limit, 512Mi request for Node.js apps -- Set `NODE_OPTIONS=--max-old-space-size=768` to keep heap within the limit -- **Doc suggestion**: Add a section in ADDING-APPS.md about runtime-specific resource defaults: - - Node.js apps: 1Gi limit, 512Mi request; add NODE_OPTIONS env var - - JVM apps (Java/Kotlin): 1Gi+ limit - - Python/Ruby apps: 512Mi typically sufficient - - Go/Rust apps: 256-512Mi typically sufficient - -### 2. db-init-job is repeated boilerplate -The db-init-job.yaml for PostgreSQL apps is nearly identical across all apps. Only the app name, namespace, and secret name change. -- **Doc suggestion**: Provide a copy-paste template for the standard PostgreSQL db-init-job with placeholders marked clearly. - -### 4. Apps requiring hex-encoded secrets -Some apps (Outline) require secrets in specific formats (e.g., exactly 64 hex chars). The default Wild Cloud `GenerateSecret` produces alphanumeric (not hex) strings. -- **Solution**: Use `crypto.SHA256` gomplate function in the manifest `default` field to generate a valid 64-char hex string from another secret: - ```yaml - defaultSecrets: - - key: utilsSecret - - key: secretKey - default: '{{ crypto.SHA256 .secrets.utilsSecret }}' - ``` -- **Doc suggestion**: Add this pattern to ADDING-APPS.md with examples of apps that need it. Note that the order of `defaultSecrets` matters — referenced secrets must come first. - -### 6. Always use `?sslmode=disable` in PostgreSQL connection strings -The Wild Cloud internal PostgreSQL does not have SSL enabled. Apps that construct PostgreSQL connection strings must include `?sslmode=disable`, and deployments should set `PGSSLMODE=disable`. Without it, apps crash with "The server does not support SSL connections." -- Already done correctly in: listmonk, gitea, discourse -- Missing from: outline (fixed), any new app -- **Doc suggestion**: Add to the ADDING-APPS.md dbUrl template examples: always append `?sslmode=disable` to postgres connection strings. - -### 5. Re-running `wild app add` is required after template changes -When you modify files in wild-directory (e.g., fix a template), you must re-run `wild app add ` before deploying. The compiled templates in the instance `apps/` directory are NOT automatically updated when wild-directory changes. Only then will `wild app deploy` use the updated templates. -- **Doc suggestion**: Add to ADDING-APPS.md: "After modifying templates in wild-directory, always re-run `wild app add ` to recompile before deploying." - -### 7. WriteFreely requires root user (jrasanen/writefreely image) -The `jrasanen/writefreely` image startup script creates `config.ini` in `/writefreely/` which is a root-owned directory. The image's default user is non-root, causing "Permission denied" errors at startup. -- **Fix**: Set `runAsUser: 0` and `runAsNonRoot: false` in the pod securityContext -- Still safe with `allowPrivilegeEscalation: false` and `capabilities.drop: ALL` -- **Doc suggestion**: Note that some community images require root. When an image fails with permission denied on startup, check if it needs root and add the security context override with capabilities dropped. - -### 8. Redis URL must include password for authenticated Redis -Wild Cloud's Redis requires authentication. Apps that take a Redis URL must include the password in the URL: `redis://:password@host:6379`. -- **Pattern**: Add `redis.password` to `requiredSecrets`, then use K8s env var expansion: - ```yaml - - name: REDIS_PASSWORD - valueFrom: - secretKeyRef: - name: myapp-secrets - key: redis.password - - name: REDIS_URL - value: redis://:$(REDIS_PASSWORD)@{{ .redis.host }}:6379 - ``` -- The `$(VAR_NAME)` syntax is supported by Kubernetes in container env values (evaluated at pod start) -- **Doc suggestion**: Add this pattern to ADDING-APPS.md under "Redis Authentication" section. - -### 3. Consistent health check paths across apps -Different apps use different health check paths (`/health`, `/healthz`, `/_health`, `/api/v1/health`). This should be documented per app type or noted that you must verify the actual path from the app's Docker docs. -- For apps where no valid HTTP health path exists (e.g., Typebot viewer which only serves chatbot URLs), use `tcpSocket` probe instead of `httpGet`. - -### 9. Linuxserver.io and s6-overlay images require root + capabilities -Images using s6-overlay init system (most linuxserver.io images) must run as root and need full capabilities to switch to their internal "abc" user. -- Set `runAsUser: 0, runAsNonRoot: false` in pod securityContext -- Do NOT set `allowPrivilegeEscalation: false` or `capabilities.drop: ALL` in container securityContext -- Only keep `readOnlyRootFilesystem: false` -- Same applies to Nextcloud (uses rsync/chown during setup) -- **Affects**: BookStack (linuxserver.io), any linuxserver.io image - -### 10. CryptPad needs emptyDir + initContainer for config directory -The CryptPad image's `/cryptpad/config/` directory is in the image overlay filesystem and not writable by the container process even when running as root. -- **Fix**: Mount an emptyDir at `/cryptpad/config`, and use an initContainer to pre-seed `config.example.js` from the image into the emptyDir. -- The initContainer mounts the emptyDir at a different path (e.g., `/config-dest`) and copies the file. -- **CPAD_CONF** env var must be set to `/cryptpad/config/config.js` (startup script copies config.example.js to config.js if config.js doesn't exist). - -### 11. Pixelfed image requires authentication (ghcr.io access restriction) -~~`ghcr.io/pixelfed/pixelfed` returns 403 Forbidden for anonymous pulls.~~ -- **Resolved**: Use `ghcr.io/mattlqx/docker-pixelfed:v0.12.7-nginx` (community image, publicly accessible) -- Note: the mattlqx image uses port 80 (nginx), not 8080. Update `containerPort` and service `targetPort` accordingly. - -### 12. Bitnami images no longer on Docker Hub -Bitnami moved their images from Docker Hub (`bitnami/appname`) to their own OCI registry. Apps that were packaged using Bitnami images (e.g., Ghost) need to be updated to use the official upstream image or Bitnami's new registry. -- **Ghost fix**: Switched from `bitnami/ghost:5.x` to official `ghost:5.130.6-alpine` -- The official ghost image uses different env vars (`database__connection__host` instead of `GHOST_DATABASE_HOST`) and a different mount path (`/var/lib/ghost/content` instead of `/bitnami/ghost`) -- Check image availability before packaging: `docker manifest inspect docker.io/bitnami/appname:tag` - -### 13. Required secrets belong in app-secrets, not dep-secrets -Apps that use `requiredSecrets` (e.g., `mysql.rootPassword`) receive those secrets copied into their own `-secrets` K8s Secret, under the key `.` (e.g., `mysql.rootPassword`). db-init jobs and deployments must reference `-secrets`, NOT `mysql-secrets`. - -```yaml -# WRONG - mysql-secrets doesn't exist in the app namespace -secretKeyRef: - name: mysql-secrets - key: rootPassword - -# CORRECT - the required secret is copied into ghost-secrets -secretKeyRef: - name: ghost-secrets - key: mysql.rootPassword -``` - -### 15. mattlqx/docker-pixelfed requires APP_PORT env var -The `ghcr.io/mattlqx/docker-pixelfed` community image generates nginx config from env vars using `sed`. The `listen` directive uses `APP_PORT`, which must be set explicitly or nginx crashes with "invalid number of arguments in 'listen' directive". -- Add `APP_PORT: "80"` to the deployment env vars -- The image listens on port 80 (nginx), not 8080 as the original pixelfed image did -- Both the web and worker deployments must mount the shared storage PVC — use ReadWriteMany access mode, not ReadWriteOnce, since both pods need it simultaneously - -### 14. Your Priorities and Polis have no public Docker images -- `ghcr.io/citizensfoundation/your-priorities-app` — 403 Forbidden, images require auth -- `compdemocracy/polis-server` — private, no public Docker Hub or ghcr.io images -- Both apps require building custom images from source before they can be packaged -- Attempting to add these to wild-directory results in ImagePullBackOff - -### 16. PHP/Laravel and other heavy-framework apps need extended probe delays -Laravel's startup process runs package discovery, autoload optimization, and encryption key generation before the HTTP server is ready. This commonly takes 2–3 minutes on cluster restart or first boot. -- The default `initialDelaySeconds: 60` is too short — the liveness probe fires before the app is ready, kills the container, and a restart loop begins -- Set `initialDelaySeconds: 120` and `failureThreshold: 6` for liveness probes on Laravel/Symfony apps -- Set `initialDelaySeconds: 60` for readiness probes -- The symptom is: pod shows `Running` for ~2 minutes then gets `Killing` due to liveness probe failure, followed by a new pod starting the same cycle -- The same applies to other frameworks with heavy startup initialization: Rails (`bundle exec`, asset precompilation), Spring Boot (JVM + Spring context loading), Django with migrations -- **Affects**: Ushahidi API (Laravel) — fixed with 120s initial delay + failureThreshold: 6 - -### 17. MySQL db-init `CREATE USER IF NOT EXISTS` doesn't update passwords -The common db-init pattern `CREATE USER IF NOT EXISTS 'user'@'%' IDENTIFIED BY '${PASSWORD}'` only sets the password at creation time. If the MySQL user already exists from a previous deployment (e.g., after `wild app delete` + `wild app add` without dropping the database), the password is silently left unchanged and the new deployment fails with "Access denied for user". -- **Symptom**: App starts, connects to MySQL, gets "Access denied" even though the db-init job completed successfully -- **Quick fix**: Exec into the MySQL pod and run: `ALTER USER 'user'@'%' IDENTIFIED BY 'current_password'; FLUSH PRIVILEGES;` -- **Better pattern** — make db-init jobs truly idempotent for both user creation and password: - ```sql - CREATE USER IF NOT EXISTS '${DB_USERNAME}'@'%' IDENTIFIED BY '${DB_PASSWORD}'; - ALTER USER '${DB_USERNAME}'@'%' IDENTIFIED BY '${DB_PASSWORD}'; - GRANT ALL PRIVILEGES ON ${DB_DATABASE_NAME}.* TO '${DB_USERNAME}'@'%'; - FLUSH PRIVILEGES; - ``` -- This `CREATE ... IF NOT EXISTS` + `ALTER USER` pattern is safe to run repeatedly regardless of whether the user existed before -- **Affects**: Ghost (MySQL backend) — hit this when redeploying after a previous partial test run - -### 18. Gancio "Non empty db" crash on re-deploy -Gancio stores its setup state in `config.json` on the data PVC. If the PVC is lost or the app is deleted and redeployed with an existing database, gancio finds a non-empty DB but no `config.json` and refuses to start with "Non empty db! Please move your current db elsewhere than retry." -- **Fix**: Drop the database schema and let gancio recreate it: - ```bash - kubectl exec -n postgres -- psql -U postgres gancio \ - -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public; GRANT ALL ON SCHEMA public TO gancio; GRANT ALL ON SCHEMA public TO public;" - kubectl scale deployment -n gancio gancio --replicas=0 - # (wait for pod to terminate) - kubectl scale deployment -n gancio gancio --replicas=1 - ``` -- Scale down first to release the DB connection before dropping the schema -- This only affects re-deployments; fresh installs work fine - -### 19. Odoo requires explicit database name and `-i base` on first run -The official Odoo Docker image does not auto-initialize the database. Without specifying the database name and installing the base module, Odoo starts in multi-database manager mode and health checks fail with "Database not initialized". -- Add `args: ["-d", "DATABASE_NAME", "-i", "base"]` to the container spec -- The `-i base` flag installs Odoo's base module and creates all database tables on first run -- Subsequent runs with `-i base` are safe (idempotent) — it updates rather than reinstalls -- Use a `startupProbe` with `failureThreshold: 40, periodSeconds: 30` (20 minutes) since first-run initialization takes 10–15 minutes -- Once startup probe passes, the regular liveness/readiness probes with `initialDelaySeconds: 0` take over - -### 20. Headscale v0.29+ CLI: inconsistent user flag types -In headscale v0.29, different commands accept different user identifiers: -- `preauthkeys create --user ` — requires numeric ID -- `auth register --user ` — accepts username string -- `users destroy --identifier ` — requires numeric ID, not a positional arg - -```bash -# Always list users first to get the ID -headscale users list - -headscale preauthkeys create --user 1 # numeric ID -headscale auth register --auth-id --user payne # username string OK -headscale users destroy --identifier 1 # numeric ID -``` - -Node registration flow: when a Tailscale client connects via browser (not pre-auth key), headscale -shows a command like `headscale auth register --auth-id hskey-authreq-XXXX --user USERNAME`. -Run it via `kubectl exec -n headscale deploy/headscale -- headscale auth register --auth-id --user `. - -### 21. "No available server" from Traefik — kustomize label selector mismatch -If a deployment was originally created without kustomize labels in its selector (or with stale templates), re-deploying via `wild app deploy` cannot fix it because `spec.selector` on a Deployment is immutable once set. The Service selector will update (it's mutable) but pods won't match it, leaving the Service with no endpoints. -- **Symptom**: Traefik returns "No available server" even though pods are Running -- **Diagnosis**: `kubectl get endpoints -n ` shows ``; pod labels don't include `app: ` / `managedBy: kustomize` / `partOf: wild-cloud` -- **Fix**: Delete the affected deployments and re-apply kustomize — `kubectl delete deployment -n ` then `kubectl apply -k /apps//` -- **Root cause**: The Deployment selector is set correctly on first deploy, but if the resource already existed with only `component: web` (e.g., from a pre-kustomize deployment), the selector is stuck - -### 22. Apps using same-origin iframes need SAMEORIGIN frame policy -The global Traefik `security-headers` middleware applies `X-Frame-Options: SAMEORIGIN` to all responses (changed from `DENY`). Apps that use iframes internally (like Etherpad's pad editor, which loads `../static/empty.html` in an iframe) were broken by the stricter `DENY` policy. -- **Root cause**: `DENY` prevents even same-origin iframes. `SAMEORIGIN` still blocks cross-origin embedding while allowing apps to embed their own sub-pages. -- **Symptoms**: JavaScript error `Blocked a frame with origin "https://..." from accessing a cross-origin frame` — occurs when an app creates a same-origin iframe and then the `X-Frame-Options: DENY` header causes the browser to block access to it. -- **Important**: Route-level Traefik middlewares run BEFORE entrypoint middlewares on the response path. A per-app middleware cannot override a global entrypoint middleware — the entrypoint middleware always runs last. -- **Fix applied globally**: Changed `frameDeny: true` to `X-Frame-Options: SAMEORIGIN` in `customResponseHeaders` in the crowdsec `security-headers` middleware. -- **Affects**: Etherpad (pad editor iframe), any app that loads sub-pages in iframes on the same domain. - -### 24. nginx proxy containers: Docker DNS resolver (`127.0.0.11`) doesn't work in Kubernetes - -Apps that use nginx as a frontend proxy (e.g., karrot-frontend) often ship with an nginx config template that uses Docker's embedded DNS resolver: - -```nginx -resolver 127.0.0.11 valid=3s; -set $backend app-backend:8000; -proxy_pass http://$backend$request_uri; -``` - -This fails in Kubernetes with `send() failed (111: Connection refused) while resolving, resolver: 127.0.0.11`. There's a second trap: **any nginx variable anywhere in the `proxy_pass` URL** (even `$request_uri` in the path) forces runtime DNS resolution and requires a `resolver` directive. Removing the resolver line but keeping `proxy_pass http://backend:8000$request_uri;` still fails with `no resolver defined to resolve backend`. - -**Fix**: override the nginx config template via a ConfigMap mounted with `subPath`, and remove the variable from `proxy_pass` entirely: - -```nginx -# Instead of: -resolver 127.0.0.11 valid=3s; -set $backend ${BACKEND}; -proxy_pass http://$backend$request_uri; - -# Use: -proxy_pass http://${BACKEND}; -``` - -With no nginx variable in `proxy_pass`, nginx resolves the hostname at startup using the pod's `/etc/resolv.conf` (which points to CoreDNS). The full request URI is still forwarded automatically in a regex `location ~` block. Mount the ConfigMap with `subPath` to override just the template file without replacing the whole directory. - -### 25. ConfigMap changes don't restart pods — and `subPath` mounts never auto-update - -When you update a ConfigMap and run `wild app deploy` (`kubectl apply`), the ConfigMap object is updated but **pods are not restarted**. You must explicitly restart them: - -```bash -kubectl rollout restart deployment/ -n -``` - -There's a further subtlety: ConfigMaps mounted with `subPath` are **never** live-updated in the pod filesystem, even if you wait. The pod must be restarted to pick up changes regardless of the `subPath` mount update behavior. - -This means: after any ConfigMap change, always follow up with a rollout restart for affected deployments. - -### 26. The `includeSelectors: true` mismatch affects every deployment in the app — check them all - -When diagnosing "no available server" from Traefik (note #21), it's easy to focus only on the deployment that serves the HTTP route and miss other deployments in the same namespace. The `includeSelectors: true` label propagation affects every resource in the kustomization — if any deployment pre-existed with only `component: X` labels, its Service will have `` endpoints and dependent services (backends, workers, Redis) will silently fail. - -**Always run this after any selector-related redeploy**: -```bash -kubectl get endpoints --all-namespaces | grep "" -``` - -Check every deployment in the affected namespace, not just the one Traefik is routing to. A Redis pod with `` endpoints will cause the backend to crash with `ConnectionError` even though the Redis pod itself shows `1/1 Running`. - -**Checklist when fixing a selector mismatch in an app**: -1. `kubectl get endpoints -n ` — identify all services with `` -2. `kubectl get pods -n --show-labels` — confirm which pods have old labels -3. Delete ALL affected deployments (not just the web one), then redeploy -4. Verify all endpoints are populated before declaring the app fixed - -### 23. Post-deploy admin scripts for apps that require manual first-run steps - -Some apps can't complete setup through the web UI until an initial admin account is created via a CLI command. Package these as shell scripts in `scripts/` alongside the kustomize files so users have a repeatable, documented way to run them. - -**When to add a script**: If the app's README or upstream docs require running a management command after deploy (creating a superuser, registering a node, inviting the first user, etc.) — wrap it in a script. - -**Always register scripts in `manifest.yaml`** — the web UI automatically shows a button with a parameter form for every script listed there. Without this, users have to drop to the CLI even though the UI fully supports it. Format: -```yaml -scripts: - - name: create-superuser - path: scripts/create-superuser.sh - description: "Short description shown in the UI." - params: - - name: EMAIL - description: Description shown in the param form - required: true - - name: PASSWORD - description: Leave blank to generate a random one -``` - -**Script conventions** (follow `synapse/versions/v1/scripts/create-user.sh` as the reference): -- Require `KUBECONFIG`, `WILD_INSTANCE`, and `WILD_API_DATA_DIR` env vars; exit with a clear error if missing -- Read `namespace` from `config.yaml` via `yq` so the script doesn't hardcode values -- Auto-generate passwords with `openssl rand` if `PASSWORD` is not supplied -- Find the running pod by label (don't hardcode a pod name); fail clearly if none found -- Print credentials at the end with a "save this — it won't be shown again" warning -- Reference the script in the app README's "First-Time Setup" section in place of a raw `kubectl exec` command - -**Non-interactive Django `createsuperuser`**: -```bash -kubectl exec -n -c -- \ - env DJANGO_SUPERUSER_PASSWORD="${PASSWORD}" \ - python manage.py createsuperuser --email "${EMAIL}" --noinput -``` -The `--noinput` flag reads the password from `DJANGO_SUPERUSER_PASSWORD`. - -**Affected apps so far**: Eventyay (`scripts/create-superuser.sh`), Synapse (`scripts/create-user.sh`), Headscale (`scripts/register-node.sh`). - -## App Status Tracking - -| App | Status | Notes | -|-----|--------|-------| -| Firefly III | ✅ working | | -| Wiki.js | ✅ working | | -| Outline | ✅ working | Redis URL needs password via K8s env var expansion | -| WriteFreely | ✅ working | Needs runAsUser: 0 (community image quirk) | -| NocoDB | ✅ working | Node.js needs 2Gi memory | -| Mattermost | ✅ working | | -| Etherpad | ✅ working | | -| CryptPad | ✅ working | Needs emptyDir + initContainer for config seeding | -| MediaWiki | ✅ working | Post-deploy web installer required for LocalSettings.php | -| Nextcloud | ✅ working | | -| PeerTube | ✅ working | | -| BookStack | ✅ working | linuxserver.io needs root + no capabilities.drop | -| Typebot | ✅ working | Builder: /signin health path; Viewer: tcpSocket probe | -| Pixelfed | ✅ working | Use ghcr.io/mattlqx/docker-pixelfed:v0.12.7-nginx (community), port 80 not 8080 | -| Wekan | pending | needs MongoDB | -| Mautic | ✅ working | | -| Taiga | ✅ working | | -| Pol.is | ❌ blocked | Private AWS ECR images, no public alternative | -| Zulip | ✅ working | Bundles own postgres/rabbitmq/memcached; first-run migrations take ~10min | -| Rocket.Chat | pending | needs MongoDB | -| LimeSurvey | ✅ working | | -| Gancio | ✅ working | Re-deploy: must drop DB schema first if prior data exists (see note 18) | -| Mobilizon | ✅ working | | -| Formbricks | ✅ working | | -| Jitsi | ✅ working | | -| Mastodon | already done | | -| Chamilo | ✅ working | | -| Moodle | ✅ working | | -| Open edX | pending | complex | -| Pretix | ✅ working | | -| Indico | ✅ working | | -| Eventyay | ✅ working | No versioned Docker tags upstream — main tag is intentional | -| Leihs | pending | | -| LibreBooking | ✅ working | | -| uMap | ✅ working | Host header needed in health probes (Django ALLOWED_HOSTS) | -| MapComplete | pending | | -| Terrastories | pending | | -| Karrot | ✅ working | nginx frontend: override template via ConfigMap to remove Docker DNS resolver (see note 24) | -| LiquidFeedback | pending | | -| Your Priorities | ❌ blocked | No public Docker images (ghcr.io requires auth) | -| Helios Voting | pending | | -| Belenios | pending | | -| Decidim | already done | | -| Alaveteli | pending | | -| FixMyStreet | pending | | -| Ghost | ✅ working | Bitnami image gone; use official ghost:alpine; different env vars + mount path (see note 12) | -| Ushahidi | ✅ working | Client: runAsUser:0 + CHOWN/SETUID/SETGID; API: liveness initialDelay 120s (Laravel slow start) | -| CiviCRM | pending | | -| Open Collective | pending | | -| Open Food Network | pending | | -| Resonate | pending | | -| BookWyrm | ✅ working | | -| Lemmy | already done | | -| Akaunting | ✅ working | | -| GnuCash | skip | desktop app, not containerizable | -| OhMyForm | ✅ working | | -| Odoo | ✅ working | Needs `-d DATABASE -i base` args; uses startupProbe (20min window) for first-run DB init | -| Headscale | ✅ working | SQLite, ConfigMap-based config, no secrets; v0.29 CLI uses numeric user IDs (see note 20) | -| Baserow | ✅ working | | -| Keila | already done | | -| Listmonk | already done | | -| Discourse | already done | | -| Gitea | already done | | -| Immich | already done | | -| Loomio | already done | | -| Synapse | already done | App renamed from matrix to synapse | -| Open WebUI | already done | | -| OpenProject | already done | | -| vLLM | already done | | diff --git a/ADDING-APPS.md b/ADDING-APPS.md index ed44a14..a003a08 100644 --- a/ADDING-APPS.md +++ b/ADDING-APPS.md @@ -1,18 +1,15 @@ # Adding Wild Cloud Apps -This guide is for contributors and maintainers who want to create or modify Wild Cloud apps. If you're looking to use existing apps, see [README.md](README.md). - -## Overview - -Wild Cloud apps are Kubernetes applications packaged as Kustomize configurations with standardized conventions for configuration management, secrets handling, and deployment. +For contributors and maintainers creating or modifying Wild Cloud app packages. For using existing apps, see [README.md](README.md). ## Directory Structure -Each app has a two-level structure: an `app.yaml` meta file at the root, and version-specific files inside `versions/`. Version directories are named by **slot** (typically the major version), not by the full version string. The actual version lives in `manifest.yaml` inside the slot. +Each app has a two-level structure: an `app.yaml` meta file at the root, and version-specific files inside `versions/`. Name version directories by **slot** (typically the major version), not the full version string. The actual version lives in `manifest.yaml` inside the slot. ``` myapp/ ├── app.yaml # App identity, latest slot pointer, upgrade routing +├── notes.md # App-specific operational notes (optional but encouraged) └── versions/ ├── 2/ # Current latest slot (manifest.yaml has version: 2.3.1) │ ├── manifest.yaml # Version-specific config (requires, defaultConfig, etc.) @@ -24,21 +21,19 @@ myapp/ └── *.yaml ``` -Most apps have **one** version directory. A second appears only when a waypoint is needed for upgrade routing. +Most apps have **one** version directory. Add a second only when a waypoint is needed for upgrade routing. ## Required Files -Each app directory must contain: +Every app directory must contain: -1. **`app.yaml`** - App identity, latest slot pointer, and upgrade routing rules -2. **`versions/{slot}/manifest.yaml`** - Version-specific configuration schema -3. **`versions/{slot}/kustomization.yaml`** - Kustomize configuration with Wild Cloud labels -4. **`versions/{slot}/*.yaml`** - Kubernetes resource templates +1. **`app.yaml`** — App identity, latest slot pointer, and upgrade routing rules +2. **`versions/{slot}/manifest.yaml`** — Version-specific configuration schema +3. **`versions/{slot}/kustomization.yaml`** — Kustomize configuration with Wild Cloud labels +4. **`versions/{slot}/*.yaml`** — Kubernetes resource templates ## App Meta (`app.yaml`) -The `app.yaml` file at the app root defines identity, display info, and upgrade routing. These fields are version-independent. - ```yaml name: immich is: immich @@ -47,41 +42,37 @@ icon: https://immich.app/assets/images/logo.png latest: "1" ``` -### App Meta Fields - | Field | Required | Description | |-------|----------|-------------| -| `name` | Yes | App identifier (must match directory name) | -| `is` | Yes | Unique id for this app. Used for `requires` mapping | -| `description` | Yes | Brief app description shown in listings | -| `icon` | No | URL to app icon for UI display | -| `category` | Yes | Category for filtering in the app directory — use an established one or introduce a new one (see below) | -| `latest` | Yes | Slot name -- directory name under `versions/` (not a version string) | +| `name` | Yes | App identifier — must match directory name | +| `is` | Yes | Unique id used for `requires` mapping | +| `description` | Yes | Brief description shown in listings | +| `icon` | Yes | URL to app icon for UI display | +| `category` | Yes | Category for filtering in the App Directory | +| `latest` | Yes | Slot name — directory name under `versions/` (not a version string) | | `upgrade` | No | Upgrade routing rules (see Upgrade Metadata below) | ### Categories -The UI uses `category` to populate the filter buttons in the App Directory. Use an established category when one fits — new categories are fine when an app clearly doesn't belong anywhere existing. Current categories: +Use an established category when one fits. Introduce a new one only when an app clearly doesn't belong anywhere existing. | Category | Use for | |----------|---------| -| `services` | Wild Cloud infrastructure services (traefik, metallb, cert-manager, etc.) | +| `services` | Wild Cloud infrastructure (traefik, metallb, cert-manager, etc.) | | `database` | Database engines (postgres, mysql, redis, memcached) | -| `productivity` | Wikis, project management, file sharing, collaboration (nextcloud, outline, taiga, etc.) | -| `business` | ERP, CRM, accounting, forms, marketing automation (odoo, akaunting, mautic, etc.) | -| `communication` | Chat, messaging, video conferencing, email (mattermost, jitsi, listmonk, etc.) | -| `social` | Social networks and fediverse apps (mastodon, pixelfed, peertube, ghost, etc.) | -| `community` | Community organizing, civic tech, decision-making (loomio, decidim, discourse, etc.) | -| `education` | Learning management systems (moodle, chamilo) | -| `developer` | Dev tools, code hosting, package management (gitea, aptly, etc.) | -| `security` | VPN, password management (vaultwarden, headscale) | -| `ai` | AI/ML inference and interfaces (vllm, open-webui) | -| `media` | Photo and video management (immich) | +| `productivity` | Wikis, project management, file sharing, collaboration | +| `business` | ERP, CRM, accounting, forms, marketing automation | +| `communication` | Chat, messaging, video conferencing, email | +| `social` | Social networks and fediverse apps | +| `community` | Community organizing, civic tech, decision-making | +| `education` | Learning management systems | +| `developer` | Dev tools, code hosting, package management | +| `security` | VPN, password management | +| `ai` | AI/ML inference and interfaces | +| `media` | Photo and video management | ## Version Manifest (`versions/{slot}/manifest.yaml`) -Each version slot contains a `manifest.yaml` with version-specific installation details: dependencies, configuration schema, and secret requirements. - ```yaml version: 1.135.3-1 requires: @@ -95,92 +86,76 @@ defaultConfig: cacheStorage: 10Gi domain: immich.{{ .cloud.domain }} tlsSecretName: wildcard-wild-cloud-tls - db: # Configuration can be nested - host: "{{ .apps.pg.host }}" # Can reference 'requires' app configurations + db: + host: "{{ .apps.pg.host }}" name: immich user: immich redis: host: "{{ .apps.redis.host }}" defaultSecrets: - - key: password # Random value will be generated if empty + - key: password - key: dbUrl default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?pool=30" requiredSecrets: - - db.password # References postgres app via 'db' alias - - redis.auth # References redis app via 'redis' name (no alias) + - db.password + - redis.auth ``` -### Version Manifest Fields - | Field | Required | Description | |-------|----------|-------------| | `version` | Yes | App version (see Versioning Convention below) | -| `requires` | No | List of dependency apps with optional aliases | -| `defaultConfig` | Yes | Default configuration values merged into operator's `config.yaml` | -| `defaultSecrets` | No | This app's secrets (no 'default' = auto-generated) | -| `requiredSecrets` | No | List of secrets from dependency apps (format: `.`) | +| `requires` | No | Dependency apps with optional aliases | +| `defaultConfig` | Yes | Default configuration merged into the operator's `config.yaml` | +| `defaultSecrets` | No | This app's secrets — no `default` value means auto-generated | +| `requiredSecrets` | No | Secrets from dependency apps — format: `.` `[WC-DOT]` | ### Versioning Convention -Wild Cloud uses a two-part version scheme inspired by Debian packaging: `-`. +Use the two-part scheme `-`: -- **Upstream version** tracks the third-party software version (e.g., `v4.0.18`, `1.120.2`) -- **Packaging revision** tracks Wild Cloud packaging changes (template fixes, manifest cleanup, config restructuring) that don't change the upstream software version +- **Upstream version** tracks the third-party software (e.g., `1.120.2`, `v4.0.18`) +- **Packaging revision** tracks Wild Cloud packaging changes with no upstream change **Examples:** -- `v4.0.18` — initial packaging of upstream v4.0.18 -- `v4.0.18-1` — first packaging fix (no upstream change) -- `v4.0.18-2` — second packaging fix +- `v4.0.18` — initial packaging +- `v4.0.18-1` — first packaging fix, no upstream change - `v4.0.19` — upstream version bump, revision resets -**When to bump the packaging revision:** Any change to the app package that doesn't correspond to an upstream software update — manifest field changes, template improvements, kustomize restructuring, security context fixes, label corrections, etc. +**Bump the packaging revision** for: manifest field changes, template improvements, security context fixes, label corrections, etc. -**When to bump the upstream version:** When updating the container image tag or deploying a new version of the third-party software. - -The web UI uses version comparison to detect available updates. If the deployed version differs from the wild-directory version, operators see an update indicator and can apply it from the app detail panel. +**Bump the upstream version** when updating the container image tag. ### Slot Naming Convention -Version directory names are **slot names**, not version strings. The slot is a stable label; the actual version lives in `manifest.yaml` inside the slot. - -**Rules:** -- Use the **major version** as the slot name (e.g., `1`, `2`, `5`, `v3`) -- Preserve the `v` prefix if the upstream project uses it (e.g., `v1` for cert-manager) +- Use the **major version** as the slot name (`1`, `2`, `v3`) +- Keep the `v` prefix if upstream uses it (`v1` for cert-manager) - **Never** put packaging revisions (`-1`, `-2`) in directory names -- **Never** put minor/patch versions in directory names unless creating a waypoint that needs to be distinct from another slot at the same major version - -**Examples:** +- **Never** put minor/patch versions in directory names unless you need a distinct waypoint | App | Slot name | Version in manifest | |-----|-----------|-------------------| | Ghost 5.118.1-2 | `5` | `5.118.1-2` | | cert-manager v1.17.2 | `v1` | `v1.17.2` | | Immich 1.135.3-1 | `1` | `1.135.3-1` | -| Traefik v3.4 | `v3` | `v3.4` | -When bumping versions (upstream or packaging), update files inside the existing slot. Only create a new directory when you need a new waypoint. +When bumping versions, update files inside the existing slot. Only create a new directory when you need a waypoint. ### Upgrade Metadata -Most apps can upgrade from any version to any other version directly — no special metadata is needed. The `upgrade` field is **optional** and only required when an app has breaking changes that need controlled upgrade paths. +The `upgrade` field is **optional** — omit it for apps where any version can upgrade directly to any other. That covers 90% of apps. -**When you don't need `upgrade:`** Simple apps (Ghost, Redis, most stateless apps) where any version can safely replace any other version. This is the 90% case — just bump the version and the system handles it as a single-step update. - -**When you need `upgrade:`** Apps with breaking database schema changes, incompatible config formats, or upstream requirements for sequential version upgrades (e.g., Discourse requires stepping through major versions). +Add `upgrade:` only when an app has breaking changes that need a controlled path (e.g., Discourse requires stepping through major versions). #### The `upgrade` block in `app.yaml` -Upgrade routing rules live in `app.yaml`, centralized for all versions. The system iteratively re-evaluates these rules after each waypoint step. - ```yaml -# app.yaml name: myapp latest: "3" upgrade: from: - - version: ">=3.5.0" # Can upgrade directly from 3.5.x + - version: ">=3.5.0" - version: ">=3.4.0" - via: "2" # Must pass through slot "2" first (a waypoint) + via: "2" # Must pass through slot "2" first - version: "<3.4.0" blocked: true notes: "Requires sequential major upgrades. See upstream docs." @@ -188,9 +163,17 @@ upgrade: backup: required # "none", "recommended", or "required" ``` -Note: `latest` and `via` are **slot names** (directory names), not version strings. The system reads the actual version from the manifest inside each slot. +`latest` and `via` are **slot names** (directory names), not version strings. -Version-specific upgrade behavior (migrations, configMigrations) lives in the version's `manifest.yaml`: +| Field | Description | +|-------|-------------| +| `from[].version` | Version constraint: `>=`, `>`, `<=`, `<`, `=`, or `>0` | +| `from[].via` | Waypoint slot to pass through first | +| `from[].blocked` | Block upgrade with an error message | +| `from[].notes` | Message shown when blocked or as context | +| `preUpgrade.backup` | `"required"` blocks until backup is done; `"recommended"` warns | + +Version-specific migration behavior goes in the version's `manifest.yaml`: ```yaml # versions/3/manifest.yaml @@ -198,67 +181,38 @@ version: 3.6.0 upgrade: migrations: pre: - - migrations/pre-deploy.yaml # K8s Job YAML paths relative to version dir + - migrations/pre-deploy.yaml post: - migrations/post-deploy.yaml configMigrations: - oldKeyName: newKeyName # Renames config keys automatically + oldKeyName: newKeyName ``` -**`app.yaml` upgrade fields:** - | Field | Description | |-------|-------------| -| `from` | List of version constraint rules, evaluated in order (first match wins) | -| `from[].version` | Version constraint: `>=`, `>`, `<=`, `<`, `=`, or `>0` (matches any) | -| `from[].via` | Waypoint slot name in `versions/` — upgrade must pass through this slot first | -| `from[].blocked` | If true, upgrade is blocked with an error message | -| `from[].notes` | Human-readable message shown when blocked or as context | -| `preUpgrade.backup` | Backup requirement: `"required"` blocks upgrade until backup is done, `"recommended"` shows a warning | - -**Version `manifest.yaml` upgrade fields:** - -| Field | Description | -|-------|-------------| -| `migrations.pre` | K8s Job YAMLs to run before deploying this version step | -| `migrations.post` | K8s Job YAMLs to run after deploying this version step | +| `migrations.pre` | K8s Job YAMLs to run before deploying this version | +| `migrations.post` | K8s Job YAMLs to run after deploying | | `configMigrations` | Map of old config key → new config key for automatic renaming | #### Waypoint versions -When an upgrade requires passing through an intermediate version, add that version's files as a new slot in the `versions/` directory alongside the latest: +When an upgrade requires an intermediate step, add the waypoint as a new slot alongside `latest`: ``` myapp/ -├── app.yaml # Routing rules + latest pointer +├── app.yaml # latest: "3", routing rules └── versions/ - ├── 3/ # Latest slot (version: 3.6.0) - │ ├── manifest.yaml - │ ├── kustomization.yaml - │ └── *.yaml - └── 2/ # Waypoint slot (version: 2.8.0) - ├── manifest.yaml - ├── kustomization.yaml - └── *.yaml + ├── 3/ # Latest (version: 3.6.0) + └── 2/ # Waypoint (version: 2.8.0) ``` -Each waypoint is a complete app package. The system computes a chain automatically — for example, upgrading from 2.3.0 to 3.6.0 might produce: `2.3.0 → 2.8.0 (slot "2") → 3.6.0 (slot "3")`. - -**Creating a waypoint:** The current latest slot becomes the waypoint (leave it in place), then create a new slot for the new major version: - -```bash -# Current slot "2" (with version 2.8.0) stays as a waypoint -# Create the new slot for the next major version -mkdir -p wild-directory/myapp/versions/3 -# ... add manifest.yaml, kustomization.yaml, *.yaml for 3.0.0 ... -# Update app.yaml: set latest to "3", add upgrade routing rules with via: "2" -``` +To create a waypoint: leave the current latest slot in place and create a new directory for the new major version. Update `app.yaml` to point `latest` at the new slot and add routing rules. #### Migration jobs -Migration jobs are K8s Job manifests that run database migrations or other one-time tasks during an upgrade step. They must be **idempotent** (safe to re-run) since a failed upgrade might be retried. +Migration jobs are K8s Job manifests. Write them **idempotent** — safe to re-run if an upgrade is retried. -Place migration job files in the version slot directory and reference them from that version's `manifest.yaml`: +Place migration files in the version slot and reference them from `manifest.yaml`: ```yaml # versions/3/migrations/db-migrate.yaml @@ -276,173 +230,12 @@ spec: command: ["bundle", "exec", "rake", "db:migrate"] ``` -Each migration step belongs to the version that introduces the breaking change. If version 3.6.0 requires a schema migration, the migration lives in the slot `3/` directory. +- **`pre`** — schema changes the new code needs before it starts +- **`post`** — backfills or cleanup after the new version is running -#### Example: simple app with waypoint +#### Config key renames -```yaml -# myapp/app.yaml -name: myapp -latest: "2" -upgrade: - from: - - version: ">=1.0.0" - via: "1" - - version: "<1.0.0" - blocked: true - notes: "Versions before 1.0.0 are not supported" - preUpgrade: - backup: recommended -``` - -This creates a 2-step upgrade path: `1.x → slot "1" (e.g., version 1.0.0-1) → slot "2" (e.g., version 2.0.0)`. The waypoint at `versions/1/` is a complete app package used as an intermediate step. - -### Adding a New Version - -When an upstream app releases a new version, you update the Wild Directory package to track it. The process depends on whether the new version has breaking changes. - -#### Simple version bump (no breaking changes) - -Most version updates are simple — update the container image tag, adjust any changed config, and update the version in `manifest.yaml`. No directory rename or `app.yaml` change needed. - -```bash -# 1. Update files inside the existing slot -# - Bump version in manifest.yaml (e.g., 1.2.0 → 1.3.0) -# - Update container image tags in deployment YAMLs -# - Adjust defaultConfig if the new version adds/changes config -vi wild-directory/myapp/versions/1/manifest.yaml -vi wild-directory/myapp/versions/1/deployment.yaml - -# 2. app.yaml doesn't change — latest still points to slot "1" - -# 3. Test -wild app add myapp && wild app deploy myapp -``` - -The directory structure stays the same: -``` -myapp/ -├── app.yaml # latest: "1" (unchanged) -└── versions/ - └── 1/ - ├── manifest.yaml # version: 1.3.0 (bumped) - └── *.yaml -``` - -#### Version bump with breaking changes (waypoint required) - -When the new version can't safely upgrade from all previous versions — e.g., a database schema change requires stepping through an intermediate version — create a new slot for the new major version, keep the old slot as a waypoint, and add routing rules. - -```bash -# 1. The current slot (2/) becomes a waypoint — leave it in place -# 2. Create a new slot for the new major version -mkdir -p wild-directory/myapp/versions/3 -# ... add new version files (manifest.yaml, kustomization.yaml, *.yaml) ... - -# 3. Update app.yaml: point latest to new slot, add upgrade routing rules -``` - -```yaml -# app.yaml -name: myapp -latest: "3" -upgrade: - from: - - version: ">=2.5.0" # 2.5.x can upgrade directly - - version: ">=2.0.0" - via: "2" # Older 2.x must pass through slot 2 first - - version: "<2.0.0" - blocked: true - notes: "Upgrade to 2.x first. See upstream migration guide." - preUpgrade: - backup: recommended -``` - -The resulting directory: -``` -myapp/ -├── app.yaml # latest: "3", upgrade routing rules -└── versions/ - ├── 3/ # New latest (manifest.yaml has version: 3.0.0) - │ ├── manifest.yaml - │ └── *.yaml - └── 2/ # Waypoint (manifest.yaml has version: 2.5.0) - ├── manifest.yaml - └── *.yaml -``` - -#### Version bump with database migrations - -When the new version requires a schema migration (e.g., `ALTER TABLE`, new indexes, data transformations), add migration job files to the slot directory and reference them from the version's `manifest.yaml`. Since this is a minor/patch update within the same major version, update files in-place in the existing slot. - -```bash -# 1. Update files inside the existing slot -# - Bump version in manifest.yaml (e.g., 2.0.0 → 2.1.0) -# - Update container image tags in deployment YAMLs -vi wild-directory/myapp/versions/2/manifest.yaml -vi wild-directory/myapp/versions/2/deployment.yaml - -# 2. Add migration job files -mkdir -p wild-directory/myapp/versions/2/migrations -``` - -Create the migration job: -```yaml -# versions/2/migrations/pre-deploy.yaml -apiVersion: batch/v1 -kind: Job -metadata: - name: myapp-migrate-2-1-0 - namespace: myapp -spec: - backoffLimit: 3 - template: - spec: - restartPolicy: OnFailure - securityContext: - runAsNonRoot: true - runAsUser: 999 - seccompProfile: - type: RuntimeDefault - containers: - - name: migrate - image: myapp:2.1.0 - command: ["bundle", "exec", "rake", "db:migrate"] - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: [ALL] - env: - - name: DATABASE_URL - valueFrom: - secretKeyRef: - name: myapp-secrets - key: dbUrl -``` - -Reference the migration in the version manifest: -```yaml -# versions/2/manifest.yaml -version: 2.1.0 -upgrade: - migrations: - pre: - - migrations/pre-deploy.yaml -defaultConfig: - # ... -``` - -`app.yaml` doesn't change — `latest` still points to slot `"2"`. - -Migration jobs must be **idempotent** — safe to re-run if an upgrade is retried after a partial failure. Use `CREATE IF NOT EXISTS`, `ALTER TABLE IF NOT EXISTS`, etc. - -**Pre vs post migrations:** -- `pre` — runs before deploying the new version's manifests (schema changes that the new code needs) -- `post` — runs after deploying (data backfills, cleanup that the old code didn't need) - -#### Version bump with config key renames - -When a version renames config keys (e.g., `dbHost` → `db.host`), use `configMigrations` to automatically rename them during upgrade: +Use `configMigrations` to automatically rename config keys during upgrade: ```yaml # versions/2/manifest.yaml @@ -451,169 +244,103 @@ upgrade: configMigrations: dbHost: db.host dbPort: db.port - dbName: db.name -defaultConfig: - db: - host: "{{ .apps.pg.host }}" - port: "5432" - name: myapp ``` -The system renames the keys in the instance's `config.yaml` before recompiling templates with the new version. +### Adding a New Version -### Dependency Configuration +#### Simple version bump (no breaking changes) -- Each dependency in `requires` can have: - - `name`: The app name to depend on (any app with a matching `is` field can satisfy this requirement) - - `alias`: Optional reference name for templates (defaults to `name`) +Update files inside the existing slot. Do not rename directories or change `app.yaml`. -### Manifest Template Variables (configuration and secrets) +```bash +vi wild-directory/myapp/versions/1/manifest.yaml # bump version +vi wild-directory/myapp/versions/1/deployment.yaml # update image tag +wild app add myapp && wild app deploy myapp +``` -#### Manifest Template Variable Sources +#### Breaking changes (waypoint required) -1. Standard Wild Cloud variables: `{{ .cloud.* }}`, `{{ .cluster.* }}`, `{{ .operator.* }}` -2. App-specific variables: `{{ .app.* }}` - resolved from current app's config -3. Dependency variables: `{{ .apps..* }}` - resolved using app reference mapping -4. App-specific secrets (in 'defaultSecrets' ONLY): `{{ secrets.* }}` +```bash +mkdir -p wild-directory/myapp/versions/3 +# add manifest.yaml, kustomization.yaml, *.yaml for the new version +# update app.yaml: set latest: "3", add upgrade routing rules with via: "2" +``` -#### Available Configuration Variiables +#### Schema migrations -Here's a comprehensive rundown of all config variables that get set during cluster and service setup in config.yaml: +Add migration job files to the slot directory and reference them from `manifest.yaml`: -##### operator (Set during initial setup) +```yaml +# versions/2/manifest.yaml +version: 2.1.0 +upgrade: + migrations: + pre: + - migrations/pre-deploy.yaml +``` - - operator.email - Email for cluster operator/admin +Migration jobs must be idempotent. Use `CREATE IF NOT EXISTS`, `ALTER TABLE IF NOT EXISTS`, etc. -##### cloud (Infrastructure-level settings) +### Dependencies -###### DNS Configuration: -- cloud.dnsmasq.ip - IP address of the DNS server (Wild Central) -- cluster.internalDns.externalResolver - External DNS resolver (e.g., 1.1.1.1, 8.8.8.8) +```yaml +requires: + - name: postgres + alias: db + - name: redis +``` -###### Network Configuration: +- `name` — the app to depend on (any installed app with a matching `is` field satisfies it) +- `alias` — reference name in templates (defaults to `name`) -- cloud.router.ip - Router gateway IP -- cloud.router.dynamicDns - Dynamic DNS hostname (optional) -- cloud.dhcpRange - DHCP range for the network (e.g., "192.168.8.34,192.168.8.79") -- cloud.dnsmasq.interface - Network interface for dnsmasq +At installation time, the operator maps each dependency to an actual installed app instance. The system tracks this mapping via `installedAs` in the local manifest. -###### Domain Configuration: +### Template Variables -- cloud.baseDomain - Base domain for the cloud (e.g., "payne.io") -- cloud.domain - Full cloud domain (e.g., "cloud2.payne.io") -- cloud.internalDomain - Internal cluster domain (e.g., "internal.cloud2.payne.io") +All template variables referenced in resource files must be defined in `defaultConfig`. -###### Storage Configuration (NFS Service): +#### Variable namespaces -- cloud.nfs.host - NFS server hostname/IP -- cloud.nfs.mediaPath - NFS export path for media storage -- cloud.nfs.storageCapacity - NFS storage capacity (e.g., "50Gi", "1Ti") +1. `{{ .cloud.* }}`, `{{ .cluster.* }}`, `{{ .operator.* }}` — standard Wild Cloud variables +2. `{{ .app.* }}` — this app's config +3. `{{ .apps..* }}` — dependency app config, resolved via alias or name +4. `{{ .secrets.* }}` — this app's secrets (in `defaultSecrets` only) -###### Registry Configuration (Docker Registry Service): +#### Available configuration variables -- cloud.dockerRegistryHost - Docker registry hostname (e.g., "registry.internal.cloud2.payne.io") +##### operator +- `operator.email` -###### Backup Configuration: +##### cloud +- `cloud.domain`, `cloud.baseDomain`, `cloud.internalDomain` +- `cloud.dnsmasq.ip`, `cloud.dnsmasq.interface` +- `cloud.router.ip`, `cloud.router.dynamicDns`, `cloud.dhcpRange` +- `cloud.nfs.host`, `cloud.nfs.mediaPath`, `cloud.nfs.storageCapacity` +- `cloud.dockerRegistryHost` +- `cloud.backup.root` -- cloud.backup.root - Root path for backups +##### cluster +- `cluster.name`, `cluster.hostnamePrefix` +- `cluster.nodes.talos.version`, `cluster.nodes.talos.schematicId` +- `cluster.nodes.control.vip` +- `cluster.ipAddressPool`, `cluster.loadBalancerIp` +- `cluster.certManager.cloudflare.domain`, `cluster.certManager.cloudflare.zoneID` +- `cluster.externalDns.ownerId` +- `cluster.dockerRegistry.storage` -##### cluster (Kubernetes cluster settings) +##### apps +Each deployed app gets a section under `apps..*`. Common fields: +- `apps..namespace`, `domain`, `externalDnsDomain`, `tlsSecretName` +- `apps..db.host`, `db.port`, `db.name`, `db.user` +- `apps..redis.host` +- `apps..smtp.host`, `smtp.port`, `smtp.user`, `smtp.from`, `smtp.tls`, `smtp.startTls` -###### Basic Cluster Info: +#### Dependency reference resolution -- cluster.name - Cluster name identifier -- cluster.hostnamePrefix - Prefix for node hostnames - -###### Node Configuration: - -- cluster.nodes.talos.version - Talos Linux version (e.g., "v1.11.5") -- cluster.nodes.talos.schematicId - Talos Image Factory schematic ID -- cluster.nodes.control.vip - Virtual IP for control plane -- cluster.nodes.active.* - Individual node configurations with: - - role - "controlplane" or "worker" - - interface - Network interface name - - disk - Disk device path - - currentIp - Current IP address - - targetIp - Target IP address - - configured - Configuration status - - applied - Applied status - - maintenance - Maintenance mode - - schematicId - Node-specific schematic ID - - version - Node-specific Talos version - -###### MetalLB Service: - -- cluster.ipAddressPool - IP range for MetalLB (e.g., "192.168.8.80-192.168.8.89") -- cluster.loadBalancerIp - Primary load balancer IP (e.g., "192.168.8.80") - -###### Cert-Manager Service: - -- cluster.certManager.cloudflare.domain - Cloudflare domain for DNS-01 challenge -- cluster.certManager.cloudflare.zoneID - Cloudflare zone ID - -###### ExternalDNS Service: - -- cluster.externalDns.ownerId - Unique identifier for this cluster's DNS records - -###### Docker Registry Service: - -- cluster.dockerRegistry.storage - Storage size for registry (e.g., "10Gi") - -##### apps (Application configurations) - -Each app added to the cluster gets its own section under apps. with app-specific configuration from the app's manifest. Common patterns include: - -Standard app fields: -- apps..namespace - Kubernetes namespace -- apps..domain - App domain (e.g., "ghost.cloud2.payne.io") -- apps..externalDnsDomain - Domain for external DNS -- apps..tlsSecretName - TLS certificate secret name -- apps..image - Container image -- apps..port - Service port -- apps..storage - Persistent volume size -- apps..timezone - Timezone setting - -Database-dependent apps: -- apps..dbHost / dbHostname - Database hostname -- apps..dbPort - Database port -- apps..dbName - Database name -- apps..dbUser / dbUsername - Database user - -SMTP-enabled apps: -- apps..smtp.host - SMTP server -- apps..smtp.port - SMTP port -- apps..smtp.user - SMTP username -- apps..smtp.from - From address -- apps..smtp.tls - TLS enabled -- apps..smtp.startTls - STARTTLS enabled - -Configuration Flow - -1. Initial Setup: operator.email, basic cloud.* settings -2. Cluster Bootstrap: cluster.name, cluster.nodes.* settings -3. Infrastructure Services: Each service prompts for its serviceConfig from its manifest - - MetalLB → cluster.ipAddressPool, cluster.loadBalancerIp - - Cert-Manager → cluster.certManager.* - - ExternalDNS → cluster.externalDns.ownerId - - NFS → cloud.nfs.* - - Docker Registry → cloud.dockerRegistryHost, cluster.dockerRegistry.storage -4. Apps: Each app adds its configuration under apps..* based on its manifest (including SMTP as an infrastructure app at apps.smtp.*) - -#### Manifest App Reference Resolution: - -When you use `{{ .apps..* }}` in templates: -1. System checks if `` matches any dependency's `alias` field -2. If no alias match, checks if `` matches any dependency's `name` field -3. Uses the `installedAs` value (automatically added when the app is added) to find actual app configuration in `config.yaml` - -All manifest template variables must be defined in one of these locations. - -**Important:** In the rest of the app templates, ALL configuration keys referenced in templates (via `{{ .key }}`) must be defined in `defaultConfig`. Only the app config is available to app templates. +`{{ .apps..* }}` resolves by checking `alias` first, then `name`. The system uses the `installedAs` value to find the actual app config. ### Kustomization (`kustomization.yaml`) -The kustomization file defines how Kubernetes resources are built and applies Wild Cloud's standard labels. - ```yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization @@ -626,8 +353,6 @@ labels: partOf: wild-cloud resources: - deployment-server.yaml - - deployment-machine-learning.yaml - - deployment-microservices.yaml - ingress.yaml - namespace.yaml - pvc.yaml @@ -635,38 +360,21 @@ resources: - db-init-job.yaml ``` -#### Kustomization Requirements +- Set `namespace` to the app name `[WC-NS]` +- Include standard Wild Cloud labels with `includeSelectors: true` `[WC-SEL]` +- List every resource file under `resources:` `[WC-RES]` -- **Namespace**: Must match the app name -- **Labels**: Must include standard Wild Cloud labels with `includeSelectors: true` -- **Resources**: List all Kubernetes manifest files +`includeSelectors: true` automatically injects the standard labels into resource selectors. Individual resources can use simple component labels (`component: web`) — Kustomize expands them. -#### Labeling Strategy - -Wild Cloud uses Kustomize's `includeSelectors: true` feature to automatically apply standard labels to all resources AND their selectors: - -```yaml -labels: - - includeSelectors: true - pairs: - app: myapp # App name (matches directory) - managedBy: kustomize - partOf: wild-cloud -``` - -This means individual resources can use simple, component-specific selectors like `component: web`, and Kustomize will automatically expand them to include all Wild Cloud labels. - -**Do NOT use Helm-style labels** (`app.kubernetes.io/name`, `app.kubernetes.io/instance`). Use simple component labels (`component: web`, `component: worker`, etc.) instead. +**Do not use Helm-style labels** (`app.kubernetes.io/name`, etc.). Use `component: web`, `component: worker`, etc. `[WC-HELM]` ## Configuration Templates -### Gomplate Templating - -Resource files in this repository are **templates** that get compiled when users add apps via the web app, CLI, or API. Only variables defined in the manifest file's 'defaultConfig' section are available to the resource templates. Use gomplate syntax to reference configuration: +Resource files are gomplate templates compiled when operators add apps. Only variables defined in `defaultConfig` are available to resource templates. ### Ingress -Use this as the standard ingress template for all apps: +Use this as the standard ingress template: ```yaml apiVersion: networking.k8s.io/v1 @@ -696,133 +404,80 @@ spec: secretName: {{ .tlsSecretName }} ``` -**Rules:** -- Always use `spec.ingressClassName: traefik` — do NOT use the `kubernetes.io/ingress.class` annotation (deprecated) -- Do NOT add `cert-manager.io/cluster-issuer` — the wildcard TLS cert is pre-distributed to app namespaces and managed centrally; adding this annotation causes cert-manager to issue a wrong per-app cert that overwrites the wildcard secret -- Do NOT add Traefik redirect annotations — HTTPS redirect is handled globally by Traefik -- `externalDnsDomain`, `domain`, and `tlsSecretName` must be defined in `defaultConfig` +- Always use `spec.ingressClassName: traefik` — do not use the deprecated `kubernetes.io/ingress.class` annotation `[WC-ING-ANN]` `[WC-ING-CLS]` +- Do not add `cert-manager.io/cluster-issuer` — the wildcard TLS cert is pre-distributed to namespaces; this annotation overwrites it with a wrong per-app cert `[WC-CERT]` +- Do not add Traefik redirect annotations — HTTPS redirect is handled globally +- Define `externalDnsDomain`, `domain`, and `tlsSecretName` in `defaultConfig` -### External DNS +The `external-dns.alpha.kubernetes.io/target` annotation creates a CNAME from the app subdomain to the cluster domain. Always set `cloudflare-proxied: "false"` since Wild Cloud manages its own TLS. `[WC-DNS]` -The `external-dns.alpha.kubernetes.io/target` annotation creates a CNAME from the app subdomain to the cluster domain (e.g., `myapp.cloud.example.com` → `cloud.example.com`). Always set `cloudflare-proxied: "false"` since Wild Cloud manages its own TLS. - -## App Dependencies and Reference Mapping - -### How Dependency References Work - -When an app depends on other apps, the reference system allows flexibility in naming while maintaining clear relationships: - -1. **Define dependencies** in your manifest with optional aliases: -```yaml -requires: - - name: postgres # Actual app to depend on - alias: db # Optional: how to reference it in templates - - name: redis # No alias means use 'redis' as reference -``` - -2. **At installation time**, the system: - - Prompts user to map dependencies to actual installed apps - - Sets `installedAs` field in the local app manifest to track the mapping - - Example: User might have `postgres-primary` installed, mapped to the `db` dependency - -### Example: Multiple Database Instances - -If a user has multiple PostgreSQL instances: -```yaml -# User's config.yaml -apps: - postgres-primary: - hostname: primary.postgres.svc.cluster.local - postgres-analytics: - hostname: analytics.postgres.svc.cluster.local -``` - -When adding an app that requires postgres, they can choose which instance to use, and the system tracks this in the manifest's `installedAs` field. - -## Database Patterns +## Database Setup ### Database Initialization Jobs -Apps requiring PostgreSQL or MySQL should include a database initialization job (`db-init-job.yaml`): +Include a `db-init-job.yaml` for every app that uses a database. The job must: -**Purpose:** -- Creates the application database (if it doesn't exist) -- Creates/updates the application user with proper credentials -- Grants necessary permissions -- Installs required database extensions (e.g., PostgreSQL's `vector`, `cube`, `earthdistance`) - -**Implementation requirements:** +- Create the application database if it doesn't exist +- Create/update the application user with the correct credentials +- Grant permissions +- Install any required database extensions (`vector`, `bloom`, `pg_trgm`, etc.) - Use `restartPolicy: OnFailure` -- Include in `kustomization.yaml` resources -- Use appropriate security context (e.g., `runAsUser: 999` for PostgreSQL) -- **Write idempotent SQL**: For MySQL, use `CREATE USER IF NOT EXISTS ... ; ALTER USER ...` so re-running the job after a redeploy doesn't leave stale passwords. See "MySQL db-init User Password Idempotency" in Common Gotchas. +- Be listed in `kustomization.yaml` resources +- Use the appropriate security context (`runAsUser: 999` for PostgreSQL) +- Be **idempotent** — safe to re-run after a redeploy -**Example apps:** `immich`, `gitea`, `openproject`, `discourse` +See `immich`, `gitea`, or `openproject` for reference implementations. -### Database URL Configuration +### Database URLs -When apps need database URLs with embedded credentials, **always use a dedicated `dbUrl` secret**. +When apps need a database URL with embedded credentials, use a dedicated `dbUrl` secret — do not construct URLs in-place: -❌ **Wrong** - Kustomize cannot process runtime env var substitution: ```yaml +# Wrong: Kustomize cannot do runtime env var substitution - name: DB_URL - value: "postgresql://user:$(DB_PASSWORD)@host/db" # This won't work! -``` + value: "postgresql://user:$(DB_PASSWORD)@host/db" -✅ **Correct** - Use a dedicated secret: -```yaml +# Correct: use a secret with the full URL - name: DB_URL valueFrom: secretKeyRef: name: myapp-secrets - key: apps.myapp.dbUrl + key: dbUrl ``` -Add `apps.myapp.dbUrl` to your manifest's `defaultSecrets`, and the system will generate the complete URL with embedded credentials automatically when the app is added. +Add `dbUrl` to `defaultSecrets` with the URL template as the `default` value. -### Backup/Restore Database Name Conventions +### Database Env Var Naming -Wild Cloud's backup/restore system uses blue-green deployments. During restore, a standby copy of the app is created with a colored database name (e.g., `myapp_green`). The system automatically patches env vars in your Kubernetes resources to point to the standby database. +Name database-related env vars so the backup system can identify them: -**How it works:** The restore system compiles your kustomize resources, finds env vars whose values match the original database name, and generates kustomize JSON patches to replace them with the standby database name. It uses env var naming conventions to distinguish database name fields from username fields (since both often have the same value). +- **Database name**: include `DATABASE`, `DB_NAME`, `DBNAME`, or `__DATABASE` in the env var name +- **Database URLs**: the value must contain `://` +- **Usernames**: include `USER` in the name — these are not patched even if the value matches the DB name -**Env var naming guidelines for database-related fields:** - -- **Database name env vars** should contain one of: `DATABASE`, `DB_NAME`, `DBNAME`, or `__DATABASE` in the env var name (e.g., `LISTMONK_db__database`, `DB_NAME`, `POSTGRES_DB`) -- **Database URL env vars** are detected by containing `://` in the value (e.g., `postgresql://user:pass@host/dbname`) -- **Username env vars** should contain `USER` in the name (e.g., `DB_USER`, `LISTMONK_db__user`) — these will NOT be patched even if the value matches the database name -- Avoid env var names that are ambiguous about whether they hold a database name or username - -**Example — correct naming:** ```yaml -env: - - name: DB_NAME # Will be patched (contains "DB_NAME") - value: myapp - - name: DB_USER # Will NOT be patched (contains "USER") - value: myapp - - name: DATABASE_URL # Will be patched (contains "://") - value: "postgresql://myapp:secret@postgres/myapp" +- name: DB_NAME # patched on restore + value: myapp +- name: DB_USER # not patched (contains USER) + value: myapp +- name: DATABASE_URL # patched (value contains ://) + value: "postgresql://myapp:secret@postgres/myapp" ``` -## Deployment Strategy +## Deployment Strategy `[WC-RWO]` -Apps using `ReadWriteOnce` (RWO) persistent volumes **must** set `strategy: type: Recreate` on their Deployment. RWO volumes can only be attached to one pod at a time, so the default `RollingUpdate` strategy will cause Multi-Attach errors during updates (the new pod can't mount the volume while the old pod still holds it). +Set `strategy: type: Recreate` on any Deployment that uses a `ReadWriteOnce` PVC. RWO volumes attach to one pod at a time — `RollingUpdate` causes Multi-Attach errors when the new pod tries to mount while the old one still holds the volume. ```yaml spec: replicas: 1 strategy: type: Recreate - selector: - matchLabels: - component: web ``` -## Security Requirements +## Security Contexts `[WC-SC-POD]` `[WC-SC-CTR]` -### Security Contexts - -**All pods must comply with Pod Security Standards.** Include security contexts at both pod and container levels: +Set security contexts at both pod and container levels on all workloads: ```yaml spec: @@ -830,119 +485,81 @@ spec: spec: securityContext: runAsNonRoot: true - runAsUser: 999 # Use appropriate non-root UID - runAsGroup: 999 # Use appropriate GID + runAsUser: 999 + runAsGroup: 999 seccompProfile: type: RuntimeDefault containers: - - name: container-name + - name: app securityContext: allowPrivilegeEscalation: false capabilities: drop: [ALL] - readOnlyRootFilesystem: false # Set to true when possible + readOnlyRootFilesystem: false ``` -**Common user IDs:** -- PostgreSQL: `runAsUser: 999` -- Redis: `runAsUser: 999` -- MySQL: Consult the container image documentation +Common UIDs: PostgreSQL `999`, Redis `999`. Consult the image docs for others. -### Secrets Management +For images that must run as root (s6-overlay, linuxserver.io), omit `runAsNonRoot` from the pod context and suppress `[WC-SC-POD]` / `[WC-SC-CTR]` with `ignoreRules` if the upstream manifest doesn't support it. -Secrets are managed through two mechanisms: default secrets for the app itself and required secrets from dependencies. +## Secrets + +Define app secrets in `defaultSecrets`. Reference secrets from dependencies in `requiredSecrets`. -**In manifest:** ```yaml defaultSecrets: - key: dbPassword # This app's database password - key: apiKey # This app's API key + - key: dbPassword + - key: apiKey requiredSecrets: - - db.password # Password from postgres dependency (aliased as 'db') - - redis.auth # Auth from redis dependency + - db.password # from postgres (aliased as 'db') + - redis.auth ``` -**In resources:** +Reference them in resources as: + ```yaml -env: - - name: DB_PASSWORD - valueFrom: - secretKeyRef: - name: myapp-secrets - key: dbPassword # Points to the default secret - - name: POSTGRES_PASSWORD - valueFrom: - secretKeyRef: - name: myapp-secrets - key: db.password # Points to the required secret +- name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: myapp-secrets + key: dbPassword # own secret +- name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: myapp-secrets + key: db.password # required secret ``` -**Secret workflow:** -1. Define app's own secrets in `defaultSecrets` (key, default mappings) -2. Reference dependency secrets in `requiredSecrets` (list) -3. When adding an app, the system: - - Generates random values for empty `defaultSecrets` - - Copies referenced secrets from dependencies - - Stores all in the instance's `secrets.yaml` -4. When deploying, creates a Kubernetes Secret named `-secrets` containing: - - All `defaultSecrets` with key format: `` - - All `requiredSecrets` with key format: `.` +Never hardcode secret values in templates. Never commit `secrets.yaml`. -**Key collision handling:** If the same key exists in both `defaultSecrets` and `requiredSecrets`, the `requiredSecrets` value takes precedence. Authors should ensure their local secrets don't collide with their required secrets. +## Converting Helm Charts -**Important:** Never commit `secrets.yaml` to Git. Templates should only reference secrets, never contain actual secret values. +When an official Helm chart exists, convert it rather than writing manifests from scratch. -## Converting from Helm Charts - -Wild Cloud prefers Kustomize over Helm for simplicity and Git-friendliness. When an official Helm chart exists, convert it rather than creating manifests from scratch. - -### Conversion Process - -1. **Extract and render the Helm chart:** ```bash helm fetch --untar --untardir charts repo/chart-name helm template --output-dir base --namespace myapp --values values.yaml myapp charts/chart-name -cd base/chart-name ``` -2. **Add namespace manifest:** -```bash -cat < namespace.yaml -apiVersion: v1 -kind: Namespace -metadata: - name: myapp -EOF -``` +Then convert to Wild Cloud format: -3. **Create kustomization:** -```bash -kustomize create --autodetect -``` +- Replace hardcoded values with gomplate variables (`{{ .cloud.domain }}`, etc.) +- Replace Helm labels with Wild Cloud standard labels +- Add `includeSelectors: true` to kustomization +- Use simple component labels (`component: web`, not `app.kubernetes.io/name`) +- Add security contexts +- Add external-dns annotations to ingresses +- Update secrets to use dotted-path convention -4. **Convert to Wild Cloud format:** - - Create `manifest.yaml` with app metadata - - Replace hardcoded values with gomplate variables (e.g., `{{ .cloud.domain }}`) - - Update secrets to use dotted-path convention - - Replace Helm labels with Wild Cloud standard labels - - Add `includeSelectors: true` to kustomization - - Use simple component labels (`component: web`, not `app.kubernetes.io/name`) - - Add security contexts to all pods - - Add external-dns annotations to ingresses +**Helm style → Wild Cloud style:** -### Example Label Migration - -❌ **Helm style:** ```yaml +# Remove: labels: app.kubernetes.io/name: myapp app.kubernetes.io/instance: release-name - app.kubernetes.io/component: server -``` -✅ **Wild Cloud style:** -```yaml -# In kustomization.yaml (applied automatically) +# Add to kustomization.yaml: labels: - includeSelectors: true pairs: @@ -950,305 +567,127 @@ labels: managedBy: kustomize partOf: wild-cloud -# In individual resources +# In individual resources: labels: - component: server # Simple component label + component: server ``` +## Post-Deploy Scripts + +Some apps require a management command after first deploy to create an admin account, register a node, or invite the first user. Package these as shell scripts in `scripts/` alongside the kustomize files. + +Register every script in `manifest.yaml` — the web UI shows a parameter form for each one: + +```yaml +scripts: + - name: create-superuser + path: scripts/create-superuser.sh + description: "Create the initial admin account." + params: + - name: EMAIL + required: true + - name: PASSWORD + description: Leave blank to generate a random one +``` + +Follow `synapse/versions/v1/scripts/create-user.sh` as the reference implementation. See [docs/scripts.md](docs/scripts.md) for the full conventions and common commands. + ## Common Gotchas -Lessons learned from deploying apps to Wild Cloud. These are the non-obvious issues that cause failures. +Keep app-specific operational notes in a `notes.md` file inside the app directory. -### Always Verify Docker Image Availability +### Image availability +Before packaging, confirm the image pulls without authentication (`docker pull `). Bitnami images moved off Docker Hub; ghcr.io images often require auth even for public repos. If no public image exists, add `ignoreRules: [WC-IMG]` with a comment. -Before packaging an app, confirm the image is **publicly accessible without authentication**: +### After template changes +After editing any file in wild-directory, re-run `wild app add ` before deploying — compiled templates in the instance directory are not auto-updated. -```bash -docker pull registry.example.com/org/image:tag -# or check via curl for ghcr.io: -curl -s "https://ghcr.io/token?scope=repository:org/image:pull&service=ghcr.io" | python3 -c "import json,sys; print(json.load(sys.stdin).get('token','no token')[:20])" -``` +### PostgreSQL +If using PostgreSQL, see [docs/database.md](docs/database.md) — `sslmode=disable`, db-init job pattern, URL construction, env var naming. -Common pitfalls: -- **Bitnami images** moved away from Docker Hub — their `bitnami/appname` images may no longer be available; check `oci.bitnami.com` or use the official upstream image instead -- **ghcr.io images** frequently require authentication even when the repo is "public" on GitHub — test anonymous pull before committing to them -- **compdemocracy/polis** and some other community apps use private AWS ECR or require GitHub tokens — these cannot be packaged for general use without finding a public mirror +### MySQL +If using MySQL, see [docs/database.md](docs/database.md) — db-init idempotency (`CREATE USER` + `ALTER USER`), required secret references. -If no public image exists, note it clearly in the wild-directory `app.yaml` description rather than packaging a broken app. +### Redis +If using Redis, see [docs/redis.md](docs/redis.md) — authenticated URL pattern, apps that don't support passwords. -### Re-run `wild app add` After Template Changes - -`wild app deploy ` applies whatever is already compiled in the instance's `apps/` directory. It does **not** recompile templates from wild-directory. After modifying any template file in wild-directory, always re-run: - -```bash -wild app add # recompiles templates into the instance dir -wild app deploy # applies the recompiled files -``` - -### PostgreSQL Connection Strings: Always Include `?sslmode=disable` - -Wild Cloud's internal PostgreSQL does not have SSL enabled. Any app that constructs a PostgreSQL connection string must include `?sslmode=disable`, or the connection will fail with "The server does not support SSL connections." +### Runtime memory, startup delays, health probes +If the app uses Node.js, Python, PHP, Ruby, JVM, nginx, or linuxserver.io images, see [docs/](docs/). +### Hex-format secrets +If an app requires a hex secret (e.g. exactly 64 chars for Outline), derive one with `crypto.SHA256`: ```yaml defaultSecrets: - - key: dbUrl - default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?sslmode=disable" -``` - -Also add `PGSSLMODE=disable` as an env var for apps that use libpq directly. - -### Redis URLs Must Include the Password - -Wild Cloud's Redis requires authentication. Apps that accept a Redis URL must include the password: - -```yaml -# In manifest.yaml requiredSecrets: -requiredSecrets: - - redis.password - -# In deployment.yaml env — use K8s env var expansion for the URL: -- name: REDIS_PASSWORD - valueFrom: - secretKeyRef: - name: myapp-secrets - key: redis.password -- name: REDIS_URL - value: "redis://:$(REDIS_PASSWORD)@{{ .redis.host }}:6379" -``` - -The `$(VAR_NAME)` syntax is evaluated by Kubernetes at pod start from other env vars in the same container spec. - -### Runtime-Specific Resource Defaults - -Default resource requests vary by runtime. The standard 256Mi/50m is too low for heavier runtimes: - -| Runtime | Suggested request | Notes | -|---------|------------------|-------| -| Go/Rust | 64–128Mi | Very efficient | -| Python/Ruby/PHP | 128–256Mi | Typical range | -| Node.js | 256–512Mi | Add `NODE_OPTIONS=--max-old-space-size=` to cap heap | -| JVM (Java/Kotlin/Scala) | 512Mi–1Gi | Set `-Xmx` to keep within limit | -| Celery/Sidekiq workers | 256–512Mi | Full worker process per instance | - -For Node.js apps, cap memory use explicitly: -```yaml -- name: NODE_OPTIONS - value: "--max-old-space-size=768" # ~75% of 1Gi limit -``` - -### Health Check Probes for Django/FastAPI (ALLOWED_HOSTS) - -Django and similar frameworks reject health probe requests because Kubernetes sends them directly to the pod IP, which is not in `ALLOWED_HOSTS`. Fix by adding a `Host` header to the probe: - -```yaml -livenessProbe: - httpGet: - path: / - port: 8000 - httpHeaders: - - name: Host - value: "{{ .domain }}" -``` - -### Images That Need Root + Special Capabilities - -Some images run an init system (nginx, s6-overlay, custom entrypoint scripts) that requires elevated privileges to set up directories and drop to a lower-privileged user. These fail when `runAsNonRoot: true` or `capabilities.drop: [ALL]` is set. - -**Pattern for nginx-based images** (e.g., community app frontends): -```yaml -spec: - securityContext: - runAsNonRoot: false - runAsUser: 0 - seccompProfile: - type: RuntimeDefault - containers: - - name: app - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: [ALL] - add: [CHOWN, SETUID, SETGID] # needed for nginx to drop to worker user - readOnlyRootFilesystem: false -``` - -**Pattern for linuxserver.io / s6-overlay images** (e.g., BookStack, most linuxserver images): -```yaml -spec: - securityContext: - runAsNonRoot: false - runAsUser: 0 - seccompProfile: - type: RuntimeDefault - containers: - - name: app - securityContext: - readOnlyRootFilesystem: false # Only this — do NOT drop capabilities -``` - -Signs you need this pattern: -- `chown(...) failed (1: Operation not permitted)` — add `CHOWN` capability -- `Permission denied` on a directory during startup — try running as root -- Container exits immediately with code 1 despite image being "official" - -### Hex-Format Secrets - -Some apps require secrets in specific formats (e.g., exactly 64 hex chars for Outline). The default Wild Cloud secret generator produces random alphanumeric strings. Use `crypto.SHA256` in the manifest `default` field to derive a hex string from another secret: - -```yaml -defaultSecrets: - - key: utilsSecret # generated first (alphanumeric) + - key: utilsSecret - key: secretKey - default: '{{ crypto.SHA256 .secrets.utilsSecret }}' # hex derivative + default: '{{ crypto.SHA256 .secrets.utilsSecret }}' ``` +Referenced secrets must appear earlier in the list. -The order of `defaultSecrets` matters — a referenced secret must appear before the entry that references it. +### nginx proxy DNS `[WC-DKRDNS]` +If the app uses an nginx proxy config from Docker, see [docs/nginx.md](docs/nginx.md) — `127.0.0.11` doesn't exist in Kubernetes. -### Odoo (and Other Apps) Requiring Explicit Database Initialization +### Traefik "No available server" `[WC-SEL]` +If Traefik returns "No available server" after a redeploy, see [docs/traefik.md](docs/traefik.md) — immutable selector fix. -Some apps don't auto-initialize their database — they need to be told which database to use AND to install their schema. Odoo is the most common case: without explicit `-d DATABASE -i base` args, it starts in multi-database manager mode and all health checks fail. +### Same-origin iframes +If the app embeds its own sub-pages in iframes and they break, see [docs/traefik.md](docs/traefik.md) — `X-Frame-Options` interaction. -```yaml -# In the container spec: -args: ["-d", "{{ .db.name }}", "-i", "base"] -``` -The `-i base` flag is **idempotent**: it installs on first run and updates on subsequent runs. The downside is slightly slower startup on subsequent restarts. For very large apps, use an initContainer to run initialization separately from the main container. +## Validation -Combine with a `startupProbe` to give the initialization time to complete: +Run the validation script before submitting any new or modified app: -```yaml -startupProbe: - httpGet: - path: /web/health - port: 8069 - failureThreshold: 40 # 40 × 30s = 20 minutes - periodSeconds: 30 - timeoutSeconds: 10 -livenessProbe: - httpGet: - path: /web/health - port: 8069 - initialDelaySeconds: 0 # startupProbe takes over startup detection - periodSeconds: 30 - failureThreshold: 6 -``` - -The `startupProbe` runs until it succeeds (or exhausts `failureThreshold`), at which point the liveness and readiness probes kick in with `initialDelaySeconds: 0`. This is cleaner than inflating `initialDelaySeconds` on the main probes. - -### Gancio Re-deploy: "Non empty db" Crash - -Gancio stores setup state in `config.json` on its data PVC. If the PVC is deleted or the app is re-added without wiping the database, gancio finds tables in PostgreSQL but no `config.json` and refuses to start. - -**Fix**: Drop the schema before restarting: ```bash -kubectl exec -n postgres -- psql -U postgres gancio \ - -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public; GRANT ALL ON SCHEMA public TO gancio; GRANT ALL ON SCHEMA public TO public;" +# From the wild-directory root: +admin/scripts/validate-app.sh + +# Or from inside your app directory: +../../admin/scripts/validate-app.sh ``` -Scale the deployment to 0 before dropping the schema to avoid active connection errors. +**Errors must be fixed before submitting.** Warnings are strong suggestions — fix them unless you have a specific documented reason not to. -### Heavy-Framework Startup Times and Probe Delays +For the full list of rule IDs, severities, and fix instructions: -Frameworks with heavy startup initialization (Laravel, Rails, Spring Boot, Symfony) run tasks like package discovery, autoload generation, database migrations, and asset compilation before accepting HTTP requests. This takes 2–3 minutes — much longer than the default 60-second initial delay. +```bash +admin/scripts/validate-apps.py --list-rules +``` -The failure mode is a restart loop: pod runs for ~2 minutes, liveness probe fires and fails, container is killed, new pod starts. The app never gets far enough to become Ready. +**Suppressing rules:** -For these runtimes, use extended probe delays: +If your app has a legitimate reason to deviate (e.g. packaging upstream Helm-rendered manifests), add `ignoreRules` to `app.yaml` with a comment: ```yaml -livenessProbe: - tcpSocket: - port: 8080 - initialDelaySeconds: 120 # Give the framework time to fully initialize - periodSeconds: 30 - failureThreshold: 6 # Tolerate transient failures during startup -readinessProbe: - tcpSocket: - port: 8080 - initialDelaySeconds: 60 - periodSeconds: 15 - failureThreshold: 3 +ignoreRules: + - WC-HELM # upstream Helm-rendered manifests + - WC-SEL # upstream manifests; selectors are set by the Helm chart ``` -**Runtimes that typically need this:** -| Runtime | Suggested liveness initialDelaySeconds | -|---------|----------------------------------------| -| PHP/Laravel/Symfony | 120s | -| Ruby on Rails | 90–120s | -| Spring Boot (JVM) | 90–120s | -| Django (with migrations) | 60–90s | -| Node.js | 60s (usually sufficient) | +Only suppress rules when genuinely necessary. -### MySQL db-init User Password Idempotency +## Submission Checklist -The common pattern `CREATE USER IF NOT EXISTS 'user'@'%' IDENTIFIED BY '${PASSWORD}'` only sets the password when the user is first created. If the MySQL user already exists from a previous deployment (e.g., after deleting and re-adding an app without dropping the database), the password is silently unchanged and the new deployment fails with "Access denied". +Run the validator first — it catches most issues automatically: -Use the `CREATE ... IF NOT EXISTS` + `ALTER USER` pattern to make db-init jobs idempotent regardless of prior state: - -```sql -CREATE DATABASE IF NOT EXISTS ${DB_DATABASE_NAME}; -CREATE USER IF NOT EXISTS '${DB_USERNAME}'@'%' IDENTIFIED BY '${DB_PASSWORD}'; -ALTER USER '${DB_USERNAME}'@'%' IDENTIFIED BY '${DB_PASSWORD}'; -GRANT ALL PRIVILEGES ON ${DB_DATABASE_NAME}.* TO '${DB_USERNAME}'@'%'; -FLUSH PRIVILEGES; +```bash +admin/scripts/validate-app.sh ``` -The `ALTER USER` line is safe even when the user was just created — it's a no-op in that case. This applies to MySQL/MariaDB only; PostgreSQL's `CREATE USER IF NOT EXISTS` does not exist but `DO $$ BEGIN IF NOT EXISTS ... END $$` patterns serve the same purpose. +**Errors must be fixed. Warnings must be fixed or suppressed with a comment in `ignoreRules`.** -## Validation Checklist +Then verify the things the validator can't check: -Before submitting a new or modified app, verify: - -- [ ] **App Meta (`app.yaml`)** - - [ ] `name` matches directory name - - [ ] `latest` points to a valid version in `versions/` - - [ ] `description` present - - [ ] `upgrade` rules correct (if applicable) - -- [ ] **Version Manifest (`versions/{slot}/manifest.yaml`)** - - [ ] `version` field present with full version string (e.g., `1.135.3-1`) - - [ ] Slot directory follows naming convention (major version, e.g., `1`, `v1`) - - [ ] All required fields present (`version`, `defaultConfig`) - - [ ] All template variables defined in `defaultConfig` - - [ ] `defaultSecrets` uses maps with 'key' and 'default' attributes - - [ ] `requiredSecrets` references use `.` format - - [ ] Dependencies listed in `requires` with optional `alias` fields - - [ ] Manifest template references match dependency aliases or names - -- [ ] **Kustomization** - - [ ] Includes standard Wild Cloud labels with `includeSelectors: true` - - [ ] Namespace matches app name - - [ ] All resource files listed under `resources:` - -- [ ] **Resources** - - [ ] Security contexts on all pods (both pod-level and container-level) - - [ ] `strategy: type: Recreate` on deployments with ReadWriteOnce PVCs - - [ ] Simple component labels, no Helm-style labels - - [ ] Ingresses include external-dns annotations - - [ ] Database apps include init jobs (if applicable) - -- [ ] **Images** - - [ ] Docker images are publicly accessible (test anonymous pull) - - [ ] Image tags are specific versions, not `latest` or branch names - - [ ] Architecture constraints (amd64-only) noted in manifest if applicable - -- [ ] **Testing** - - [ ] Templates compile successfully with sample config - - [ ] App deploys without errors in test cluster - - [ ] All dependencies work correctly +- [ ] Docker images pull without authentication (`docker pull `) +- [ ] App deploys and runs correctly in the `test-cloud` instance +- [ ] Dependencies (database, redis, etc.) work correctly end-to-end ## Contributing -Contributions are welcome! To contribute: - 1. Fork the repository 2. Create a new app directory following the structure above -3. Test your app thoroughly -4. Submit a pull request with: - - Description of the app and its purpose - - Any special configuration notes - - Dependencies required +3. Test your app in `test-cloud` +4. Submit a pull request with a description of the app and any special notes ## Notice: Third-Party Software @@ -1259,5 +698,3 @@ Unless otherwise stated, the software deployed by these manifests **is not autho These files are provided solely for convenience and automation. Users are responsible for reviewing and complying with the licenses of the software they deploy. This project is licensed under the GNU AGPLv3 or later, but this license does **not apply** to the third-party software being deployed. - -See individual deployment directories for upstream project links and container sources. diff --git a/CLAUDE.md b/CLAUDE.md index 541d54e..b8f1852 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,6 +1,7 @@ - @README.md - ADDING-APPS.md is the expert advice you need when adding new apps to the Wild Cloud Directory. +- docs/ contains deep-dive guides — runtime-specific (nodejs, python, php, ruby, jvm, nginx, linuxserver) and topic-specific (database, redis, traefik, scripts). ## Finding good sources of documentation for adding a new app to the Wild Cloud Directory diff --git a/baserow/versions/2/deployment.yaml b/baserow/versions/2/deployment.yaml index 02d970c..55983d9 100644 --- a/baserow/versions/2/deployment.yaml +++ b/baserow/versions/2/deployment.yaml @@ -69,6 +69,15 @@ spec: volumeMounts: - name: baserow-data mountPath: /baserow/data + securityContext: + runAsNonRoot: false + runAsUser: 0 + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false + seccompProfile: + type: RuntimeDefault livenessProbe: httpGet: path: /api/_health/ diff --git a/baserow/versions/2/manifest.yaml b/baserow/versions/2/manifest.yaml index e54cbe0..5a80aa6 100644 --- a/baserow/versions/2/manifest.yaml +++ b/baserow/versions/2/manifest.yaml @@ -1,4 +1,4 @@ -version: 2.2.2-1 +version: 2.2.2-2 requires: - name: postgres - name: redis diff --git a/bookwyrm/notes.md b/bookwyrm/notes.md new file mode 100644 index 0000000..c00d518 --- /dev/null +++ b/bookwyrm/notes.md @@ -0,0 +1,38 @@ +# BookWyrm — Notes + +## SCSS themes must be compiled before collectstatic + +BookWyrm ships SCSS source files instead of pre-compiled CSS. The static root is an `emptyDir` +(ephemeral), so it is empty on every pod restart. Running `collectstatic` alone fails: + +``` +ValueError: Missing staticfiles manifest entry for 'css/themes/...' +``` + +**Fix**: compile themes before collecting static files: + +```yaml +command: ["/bin/sh", "-c"] +args: + - | + python manage.py compile_themes \ + && python manage.py collectstatic --noinput \ + && python manage.py migrate \ + && exec gunicorn bookwyrm.wsgi:application ... +``` + +Check the image's `Dockerfile` or `docker_start.sh` to confirm the current required build steps. + +## PostgreSQL extensions require superuser + +BookWyrm uses extensions (`bloom`, `pg_trgm`, etc.) that require superuser to install. Migrations +fail with `permission denied to create extension "bloom"` if the app DB user is not a superuser. + +**Fix**: create the extensions in the `db-init-job`, which connects as the `postgres` superuser: + +```bash +psql -d "$APP_DB_NAME" -c "CREATE EXTENSION IF NOT EXISTS bloom;" +psql -d "$APP_DB_NAME" -c "CREATE EXTENSION IF NOT EXISTS pg_trgm;" +``` + +This must run before migrations (i.e. in `db-init-job`, not in the app startup command). diff --git a/bookwyrm/versions/0/db-init-job.yaml b/bookwyrm/versions/0/db-init-job.yaml index 236d03e..63dbb30 100644 --- a/bookwyrm/versions/0/db-init-job.yaml +++ b/bookwyrm/versions/0/db-init-job.yaml @@ -80,4 +80,7 @@ spec: GRANT USAGE ON SCHEMA public TO \"$BOOKWYRM_DB_USER\"; " + # Create bloom extension (required by bookwyrm migrations, needs superuser) + psql -d "$BOOKWYRM_DB_NAME" -c "CREATE EXTENSION IF NOT EXISTS bloom;" + echo "Database initialization completed." diff --git a/bookwyrm/versions/0/deployment.yaml b/bookwyrm/versions/0/deployment.yaml index 480af40..957623d 100644 --- a/bookwyrm/versions/0/deployment.yaml +++ b/bookwyrm/versions/0/deployment.yaml @@ -23,7 +23,7 @@ spec: containers: - name: bookwyrm image: ghcr.io/bookwyrm-social/bookwyrm:v0.8.6 - command: ["gunicorn", "bookwyrm.wsgi:application"] + command: ["/bin/sh", "-c", "python manage.py compile_themes && python manage.py collectstatic --noinput && python manage.py migrate && exec gunicorn bookwyrm.wsgi:application"] ports: - name: http containerPort: 8000 diff --git a/cert-manager/app.yaml b/cert-manager/app.yaml index 78a2d46..e6a7e79 100644 --- a/cert-manager/app.yaml +++ b/cert-manager/app.yaml @@ -2,4 +2,5 @@ name: cert-manager is: cert-manager description: X.509 certificate management for Kubernetes category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/cert-manager.svg latest: "v1" diff --git a/community-search/app.yaml b/community-search/app.yaml index 1a47ac0..374d9d9 100644 --- a/community-search/app.yaml +++ b/community-search/app.yaml @@ -3,3 +3,5 @@ is: community-search description: Community Search is a federated, self-hosted search engine built on community-curated indexes rather than global web crawling. category: community latest: "0" +ignoreRules: + - WC-ICON # no established icon available diff --git a/coredns/app.yaml b/coredns/app.yaml index 79b5b4c..cbc1aaf 100644 --- a/coredns/app.yaml +++ b/coredns/app.yaml @@ -2,4 +2,5 @@ name: coredns is: coredns description: DNS server for internal cluster DNS resolution category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/coredns.svg latest: "v1" diff --git a/crowdsec/app.yaml b/crowdsec/app.yaml index 3f3bf5e..c05b4ab 100644 --- a/crowdsec/app.yaml +++ b/crowdsec/app.yaml @@ -2,4 +2,5 @@ name: crowdsec is: crowdsec description: CrowdSec security engine with Traefik bouncer for threat detection and rate limiting category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/crowdsec.svg latest: "v1" diff --git a/cryptpad/notes.md b/cryptpad/notes.md new file mode 100644 index 0000000..8b5ff18 --- /dev/null +++ b/cryptpad/notes.md @@ -0,0 +1,25 @@ +# CryptPad — Notes + +## Config directory requires emptyDir + initContainer + +CryptPad's `/cryptpad/config/` directory is in the image overlay filesystem and is not writable +by the container process even when running as root. + +**Fix**: mount an `emptyDir` at `/cryptpad/config` and use an initContainer to pre-seed +`config.example.js` from the image into the emptyDir: + +```yaml +initContainers: + - name: seed-config + image: cryptpad/cryptpad:version-X.Y.Z + command: [sh, -c, "cp /cryptpad/config/config.example.js /config-dest/config.example.js"] + volumeMounts: + - name: cryptpad-config + mountPath: /config-dest +volumes: + - name: cryptpad-config + emptyDir: {} +``` + +Set `CPAD_CONF=/cryptpad/config/config.js` — the startup script copies `config.example.js` to +`config.js` on first run if `config.js` doesn't exist. diff --git a/cryptpad/versions/2024/deployment.yaml b/cryptpad/versions/2024/deployment.yaml index 4044c27..b829eb3 100644 --- a/cryptpad/versions/2024/deployment.yaml +++ b/cryptpad/versions/2024/deployment.yaml @@ -22,7 +22,7 @@ spec: type: RuntimeDefault initContainers: - name: seed-config - image: cryptpad/cryptpad:latest + image: cryptpad/cryptpad:version-2026.5.1 command: - sh - -c @@ -33,9 +33,18 @@ spec: volumeMounts: - name: cryptpad-config mountPath: /config-dest + securityContext: + runAsNonRoot: false + runAsUser: 0 + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false + seccompProfile: + type: RuntimeDefault containers: - name: cryptpad - image: cryptpad/cryptpad:latest + image: cryptpad/cryptpad:version-2026.5.1 ports: - name: http containerPort: 3000 @@ -80,7 +89,14 @@ spec: periodSeconds: 10 failureThreshold: 3 securityContext: + runAsNonRoot: false + runAsUser: 0 + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] readOnlyRootFilesystem: false + seccompProfile: + type: RuntimeDefault volumes: - name: cryptpad-data persistentVolumeClaim: diff --git a/cryptpad/versions/2024/manifest.yaml b/cryptpad/versions/2024/manifest.yaml index 9860de2..70e0b08 100644 --- a/cryptpad/versions/2024/manifest.yaml +++ b/cryptpad/versions/2024/manifest.yaml @@ -1,4 +1,4 @@ -version: 2024.x-1 +version: 2026.5.1-2 defaultConfig: namespace: cryptpad externalDnsDomain: '{{ .cloud.domain }}' diff --git a/decidim/versions/0/manifest.yaml b/decidim/versions/0/manifest.yaml index c8b2f7a..114d9e7 100644 --- a/decidim/versions/0/manifest.yaml +++ b/decidim/versions/0/manifest.yaml @@ -1,4 +1,4 @@ -version: 0.31.0-2 +version: 0.31.0-3 requires: - name: postgres - name: redis @@ -32,7 +32,7 @@ defaultSecrets: default: "{{ random.AlphaNum 128 }}" - key: dbPassword - key: dbUrl - default: "postgres://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}" + default: "postgres://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?sslmode=disable" requiredSecrets: - postgres.password - redis.password diff --git a/discourse/versions/3/ingress.yaml b/discourse/versions/3/ingress.yaml index 2277b03..c7cc993 100644 --- a/discourse/versions/3/ingress.yaml +++ b/discourse/versions/3/ingress.yaml @@ -9,6 +9,7 @@ metadata: external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" external-dns.alpha.kubernetes.io/target: "{{ .externalDnsDomain }}" spec: + ingressClassName: traefik rules: - host: "{{ .domain }}" http: diff --git a/discourse/versions/3/manifest.yaml b/discourse/versions/3/manifest.yaml index dca3d78..d51badc 100644 --- a/discourse/versions/3/manifest.yaml +++ b/discourse/versions/3/manifest.yaml @@ -1,4 +1,4 @@ -version: 3.5.3-3 +version: 3.5.3-4 requires: - name: postgres - name: redis diff --git a/docker-registry/app.yaml b/docker-registry/app.yaml index a81f562..98e9802 100644 --- a/docker-registry/app.yaml +++ b/docker-registry/app.yaml @@ -2,4 +2,8 @@ name: docker-registry is: docker-registry description: Private Docker image registry for cluster category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/docker.svg latest: "3" + +ignoreRules: + - WC-DNS # internal registry; external DNS not needed diff --git a/docker-registry/versions/3/deployment.yaml b/docker-registry/versions/3/deployment.yaml index 7f375a4..93f59ce 100644 --- a/docker-registry/versions/3/deployment.yaml +++ b/docker-registry/versions/3/deployment.yaml @@ -11,10 +11,7 @@ spec: matchLabels: app: docker-registry strategy: - rollingUpdate: - maxSurge: 0 - maxUnavailable: 1 - type: RollingUpdate + type: Recreate template: metadata: labels: diff --git a/docker-registry/versions/3/ingress.yaml b/docker-registry/versions/3/ingress.yaml index 478f2a0..af3abd9 100644 --- a/docker-registry/versions/3/ingress.yaml +++ b/docker-registry/versions/3/ingress.yaml @@ -3,6 +3,7 @@ kind: Ingress metadata: name: docker-registry spec: + ingressClassName: traefik rules: - host: {{ .host }} http: diff --git a/docker-registry/versions/3/kustomization.yaml b/docker-registry/versions/3/kustomization.yaml index 848522c..b7a5a26 100644 --- a/docker-registry/versions/3/kustomization.yaml +++ b/docker-registry/versions/3/kustomization.yaml @@ -5,7 +5,8 @@ labels: - includeSelectors: true pairs: app: docker-registry - managedBy: wild-cloud + managedBy: kustomize + partOf: wild-cloud resources: - deployment.yaml - ingress.yaml diff --git a/docker-registry/versions/3/manifest.yaml b/docker-registry/versions/3/manifest.yaml index c66248b..0f64b5a 100644 --- a/docker-registry/versions/3/manifest.yaml +++ b/docker-registry/versions/3/manifest.yaml @@ -1,4 +1,4 @@ -version: "3.0.0" +version: "3.0.0-1" requires: - name: traefik - name: cert-manager diff --git a/docs/database.md b/docs/database.md new file mode 100644 index 0000000..c4a9aee --- /dev/null +++ b/docs/database.md @@ -0,0 +1,78 @@ +# Database Patterns + +## PostgreSQL + +### Always use `?sslmode=disable` `[WC-SSL]` + +Wild Cloud's internal PostgreSQL has no SSL configured. Without `?sslmode=disable`, connections fail with "The server does not support SSL connections." + +```yaml +defaultSecrets: + - key: dbUrl + default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?sslmode=disable" +``` + +Also set `PGSSLMODE=disable` for apps that use libpq directly. + +### db-init-job + +Include a `db-init-job.yaml` for every app that uses PostgreSQL. See `immich`, `gitea`, or `openproject` for reference implementations. The job must: + +- Create the database if it doesn't exist +- Create/update the user with correct credentials +- Grant permissions +- Install required extensions (`vector`, `pg_trgm`, etc.) +- Use `restartPolicy: OnFailure` and `runAsUser: 999` +- Be idempotent — safe to re-run after redeploy + +### Database URL secrets + +When an app needs a connection URL with embedded credentials, use a `dbUrl` secret — do not construct URLs inline: + +```yaml +# Wrong: Kustomize cannot do runtime env var substitution +- name: DB_URL + value: "postgresql://user:$(DB_PASSWORD)@host/db" + +# Correct: use a secret with the full URL +- name: DB_URL + valueFrom: + secretKeyRef: + name: myapp-secrets + key: dbUrl +``` + +## MySQL + +### db-init user password idempotency `[WC-DBIN]` + +`CREATE USER IF NOT EXISTS` only sets the password on first creation. On redeploy against an existing database the password stays stale, causing "Access denied". + +Always follow `CREATE USER` with `ALTER USER`: + +```sql +CREATE USER IF NOT EXISTS '${DB_USERNAME}'@'%' IDENTIFIED BY '${DB_PASSWORD}'; +ALTER USER '${DB_USERNAME}'@'%' IDENTIFIED BY '${DB_PASSWORD}'; +GRANT ALL PRIVILEGES ON ${DB_DATABASE_NAME}.* TO '${DB_USERNAME}'@'%'; +FLUSH PRIVILEGES; +``` + +`ALTER USER` is a no-op when the user was just created — it is safe to always include it. + +### Required secrets reference + +MySQL secrets are copied into `-secrets`, not `mysql-secrets`. Reference them as: + +```yaml +secretKeyRef: + name: myapp-secrets + key: mysql.rootPassword # not mysql-secrets / rootPassword +``` + +## Database env var naming + +Name database-related env vars so the backup system can identify them: + +- **Database name**: include `DATABASE`, `DB_NAME`, `DBNAME`, or `__DATABASE` +- **Database URLs**: value must contain `://` +- **Usernames**: include `USER` — these are not patched on restore diff --git a/docs/jvm.md b/docs/jvm.md new file mode 100644 index 0000000..19d11d2 --- /dev/null +++ b/docs/jvm.md @@ -0,0 +1,44 @@ +# JVM (Spring Boot / Scala / Kotlin) + +## Startup delay + +Spring Boot loads the full application context before accepting HTTP. Cold starts typically take 60–120 seconds depending on the number of beans and auto-configuration classes. + +```yaml +livenessProbe: + tcpSocket: + port: 8080 + initialDelaySeconds: 120 + periodSeconds: 30 + failureThreshold: 6 +readinessProbe: + httpGet: + path: /actuator/health + port: 8080 + initialDelaySeconds: 90 + periodSeconds: 15 + failureThreshold: 3 +``` + +If the app exposes a Spring Actuator health endpoint, prefer it for readiness over the root path. + +## Heap limit + +Set `-Xmx` to keep the heap within the container memory limit. The JVM does not automatically respect cgroup limits on older JDK versions. + +```yaml +env: + - name: JAVA_OPTS + value: "-Xmx512m -Xms256m" +resources: + limits: + memory: 768Mi + requests: + memory: 512Mi +``` + +Keep `-Xmx` 25–33% below the container limit to leave room for off-heap memory (metaspace, direct buffers, GC overhead). + +## Resources + +512Mi–1Gi request is typical. JVM startup can spike higher — use `requests` to reserve and `limits` to cap. diff --git a/docs/linuxserver.md b/docs/linuxserver.md new file mode 100644 index 0000000..8149e08 --- /dev/null +++ b/docs/linuxserver.md @@ -0,0 +1,44 @@ +# linuxserver.io / s6-overlay images + +## Security context + +Images using the s6-overlay init system (all linuxserver.io images) must run as root and need full capabilities to switch to their internal `abc` user. **Do not drop capabilities** — s6-overlay's user-switching will fail. + +```yaml +spec: + template: + spec: + securityContext: # pod level + runAsNonRoot: false + runAsUser: 0 + seccompProfile: + type: RuntimeDefault + containers: + - name: app + securityContext: # container level — only this field + readOnlyRootFilesystem: false +``` + +Do NOT set `allowPrivilegeEscalation: false` or `capabilities.drop: ALL` for these images. + +Suppress `WC-SC-POD` and `WC-SC-CTR` in `app.yaml`: + +```yaml +ignoreRules: + - WC-SC-POD # s6-overlay requires root + - WC-SC-CTR # s6-overlay requires full capabilities +``` + +## PUID / PGID + +linuxserver.io images accept `PUID` and `PGID` env vars to set the internal `abc` user's UID/GID. Set them explicitly for consistent file ownership on PVCs: + +```yaml +env: + - name: PUID + value: "1000" + - name: PGID + value: "1000" +``` + +**Affected apps**: BookStack, and any app using a linuxserver.io image. diff --git a/docs/nginx.md b/docs/nginx.md new file mode 100644 index 0000000..bb23182 --- /dev/null +++ b/docs/nginx.md @@ -0,0 +1,54 @@ +# nginx-based images + +## Security context + +nginx binds to port 80 (or 443) and needs to `chown` files before dropping to `www-data`. It must run as root but can drop most capabilities after startup. + +```yaml +spec: + template: + spec: + securityContext: # pod level + runAsNonRoot: false + runAsUser: 0 + seccompProfile: + type: RuntimeDefault + containers: + - name: app + securityContext: # container level + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + add: [CHOWN, SETUID, SETGID, NET_BIND_SERVICE] + readOnlyRootFilesystem: false +``` + +Add `NET_BIND_SERVICE` only if the container binds to a port below 1024. Drop it if the app uses a high port (8080, 3000, etc.). + +## Docker DNS resolver (`WC-DKRDNS`) + +nginx proxy configs shipped for Docker often contain: + +```nginx +resolver 127.0.0.11 valid=3s; +set $backend app-backend:8000; +proxy_pass http://$backend$request_uri; +``` + +`127.0.0.11` is Docker's embedded DNS resolver — it does not exist in Kubernetes. This fails with `send() failed (111: Connection refused) while resolving`. + +There's a second trap: **any nginx variable anywhere in the `proxy_pass` URL** (even `$request_uri` in the path) forces runtime DNS resolution and requires a `resolver` directive. Removing the resolver line but keeping `proxy_pass http://backend:8000$request_uri;` still fails with `no resolver defined to resolve backend`. + +**Fix**: override the nginx config template via a ConfigMap mounted with `subPath`, and remove the variable from `proxy_pass` entirely: + +```nginx +# Instead of: +resolver 127.0.0.11 valid=3s; +set $backend ${BACKEND}; +proxy_pass http://$backend$request_uri; + +# Use: +proxy_pass http://${BACKEND}; +``` + +With no nginx variable in `proxy_pass`, nginx resolves the hostname at startup using the pod's `/etc/resolv.conf` (which points to CoreDNS). The full request URI is still forwarded automatically in a regex `location ~` block. Mount the ConfigMap with `subPath` to override just the template file without replacing the whole directory. diff --git a/docs/nodejs.md b/docs/nodejs.md new file mode 100644 index 0000000..cce6090 --- /dev/null +++ b/docs/nodejs.md @@ -0,0 +1,20 @@ +# Node.js + +## Memory limits + +Node.js apps frequently OOM-crash at the default 512Mi limit. Set a higher limit and cap the V8 heap so it stays under it: + +```yaml +resources: + limits: + memory: 1Gi + requests: + memory: 512Mi +env: + - name: NODE_OPTIONS + value: "--max-old-space-size=768" +``` + +`--max-old-space-size` is in MiB. Keep it 25–33% below the container limit so the process has headroom for non-heap memory (buffers, native modules, etc.). + +**Applies to**: NocoDB, Outline, Gitea (web), and any Electron/Express/Next.js app. diff --git a/docs/php.md b/docs/php.md new file mode 100644 index 0000000..ce90cb9 --- /dev/null +++ b/docs/php.md @@ -0,0 +1,31 @@ +# PHP (Laravel / Symfony) + +## Startup delay + +Laravel's startup sequence (autoload optimization, key generation, migrations, package discovery) commonly takes 2–3 minutes on cluster restart or first boot. The default `initialDelaySeconds: 60` is too short — the liveness probe fires before the app is ready, kills the container, and a restart loop begins. + +**Symptom**: pod shows `Running` for ~2 minutes, then gets `Killing` due to liveness probe failure, then a new pod starts the same cycle. + +Use a `tcpSocket` liveness probe (avoids the Host header issue) with an extended initial delay: + +```yaml +livenessProbe: + tcpSocket: + port: 8080 + initialDelaySeconds: 120 + periodSeconds: 30 + failureThreshold: 6 +readinessProbe: + httpGet: + path: / + port: 8080 + initialDelaySeconds: 60 + periodSeconds: 15 + failureThreshold: 3 +``` + +Symfony has the same behavior. Adjust `initialDelaySeconds` up if migrations are slow. + +## Resources + +256–512Mi request is typical for a PHP-FPM or Artisan-served app. Workers (queue:work, horizon) are separate processes — give them their own Deployment and 256–512Mi each. diff --git a/docs/python.md b/docs/python.md new file mode 100644 index 0000000..460802a --- /dev/null +++ b/docs/python.md @@ -0,0 +1,65 @@ +# Python (Django / FastAPI) + +## Health probe host header + +Kubernetes sends probe requests directly to the pod IP. Django's `ALLOWED_HOSTS` rejects requests without a matching `Host` header, causing every probe to return 400 and the pod to restart. Add a `Host` header to all HTTP probes: + +```yaml +livenessProbe: + httpGet: + path: / + port: 8000 + httpHeaders: + - name: Host + value: "{{ .domain }}" +readinessProbe: + httpGet: + path: / + port: 8000 + httpHeaders: + - name: Host + value: "{{ .domain }}" +``` + +FastAPI and Flask apps with explicit host validation have the same issue. + +## Startup delay (apps that run migrations) + +Django apps that run `migrate` on startup need extra time before the liveness probe fires. The default 60-second delay is too short for large migration sets. + +```yaml +livenessProbe: + httpGet: + path: / + port: 8000 + httpHeaders: + - name: Host + value: "{{ .domain }}" + initialDelaySeconds: 90 + periodSeconds: 30 + failureThreshold: 6 +readinessProbe: + httpGet: + path: / + port: 8000 + httpHeaders: + - name: Host + value: "{{ .domain }}" + initialDelaySeconds: 60 + periodSeconds: 15 + failureThreshold: 3 +``` + +## Celery workers + +Celery workers are full Python processes. Size them at 256–512Mi and deploy them as a separate Deployment from the web process. They do not need an HTTP probe — use a `tcpSocket` or `exec` check if a liveness probe is needed at all. + +## Non-interactive superuser creation + +```bash +kubectl exec -n -- \ + env DJANGO_SUPERUSER_PASSWORD="${PASSWORD}" \ + python manage.py createsuperuser --email "${EMAIL}" --noinput +``` + +`--noinput` reads the password from `DJANGO_SUPERUSER_PASSWORD`. diff --git a/docs/redis.md b/docs/redis.md new file mode 100644 index 0000000..aa13653 --- /dev/null +++ b/docs/redis.md @@ -0,0 +1,33 @@ +# Redis + +## Authenticated Redis URL + +Wild Cloud's Redis requires a password. Apps that take a Redis URL must embed the password. Use Kubernetes env var expansion so the password isn't hardcoded: + +```yaml +requiredSecrets: + - redis.password + +# In deployment env: +- name: REDIS_PASSWORD + valueFrom: + secretKeyRef: + name: myapp-secrets + key: redis.password +- name: REDIS_URL + value: "redis://:$(REDIS_PASSWORD)@{{ .redis.host }}:6379" +``` + +`$(REDIS_PASSWORD)` is evaluated by Kubernetes at pod start from other env vars in the same container spec. + +## Apps that don't support Redis passwords + +Some apps only accept `REDIS_HOST` and `REDIS_PORT` with no password option (e.g. Ushahidi). Deploy a dedicated unauthenticated Redis sidecar for those apps and remove `redis` from `requires` in `manifest.yaml`: + +```yaml +- name: redis + image: redis:7-alpine + args: ["--save", ""] # disable persistence +``` + +Point the app's `REDIS_HOST` at the sidecar's service name. diff --git a/docs/ruby.md b/docs/ruby.md new file mode 100644 index 0000000..bb93be7 --- /dev/null +++ b/docs/ruby.md @@ -0,0 +1,32 @@ +# Ruby (Rails) + +## Startup delay + +Rails startup (bundle exec, asset precompilation, initializer chain) typically takes 30–90 seconds. Extend the liveness probe delay to avoid restart loops: + +```yaml +livenessProbe: + tcpSocket: + port: 3000 + initialDelaySeconds: 90 + periodSeconds: 30 + failureThreshold: 6 +readinessProbe: + httpGet: + path: / + port: 3000 + initialDelaySeconds: 60 + periodSeconds: 15 + failureThreshold: 3 +``` + +## Sidekiq workers + +Sidekiq is a full Ruby process, not a thread pool. Deploy it as a separate Deployment from the web process with 256–512Mi memory. It does not need an HTTP liveness probe — omit it or use a simple `exec` check. + +## Non-interactive rails console / rake tasks + +```bash +kubectl exec -n -- bundle exec rake db:migrate +kubectl exec -n -- bundle exec rails runner "User.create!(...)" +``` diff --git a/docs/scripts.md b/docs/scripts.md new file mode 100644 index 0000000..e594ccb --- /dev/null +++ b/docs/scripts.md @@ -0,0 +1,45 @@ +# Post-Deploy Scripts + +Some apps require a management command after first deploy to create an admin account, register a node, or invite the first user. Package these as shell scripts in `scripts/` alongside the kustomize files. + +## Registering scripts + +Register every script in `manifest.yaml` — the web UI shows a button with a parameter form for each one: + +```yaml +scripts: + - name: create-superuser + path: scripts/create-superuser.sh + description: "Create the initial admin account." + params: + - name: EMAIL + required: true + - name: PASSWORD + description: Leave blank to generate a random one +``` + +## Script conventions + +Follow `synapse/versions/v1/scripts/create-user.sh` as the reference implementation: + +- Require `KUBECONFIG`, `WILD_INSTANCE`, and `WILD_API_DATA_DIR`; exit with a clear error if missing +- Read `namespace` from `config.yaml` via `yq` — never hardcode it +- Auto-generate passwords with `openssl rand` if `PASSWORD` is not supplied +- Find the running pod by label — never hardcode a pod name +- Print credentials at the end with a "save this — it won't be shown again" warning + +## Common commands + +**Django non-interactive superuser**: +```bash +kubectl exec -n -- \ + env DJANGO_SUPERUSER_PASSWORD="${PASSWORD}" \ + python manage.py createsuperuser --email "${EMAIL}" --noinput +``` + +**Rails**: +```bash +kubectl exec -n -- bundle exec rake db:migrate +``` + +**Affected apps**: Eventyay, Synapse, Headscale. diff --git a/docs/traefik.md b/docs/traefik.md new file mode 100644 index 0000000..cd409a7 --- /dev/null +++ b/docs/traefik.md @@ -0,0 +1,47 @@ +# Traefik + +## "No available server" after redeploy `[WC-SEL]` + +If a Deployment was originally created without kustomize labels in its selector, re-deploying cannot fix it — `spec.selector` is immutable once set. The Service selector updates (it's mutable) but pods won't match it, leaving the Service with no endpoints. + +**Symptom**: Traefik returns "No available server" even though pods are Running. + +**Diagnosis**: +```bash +kubectl get endpoints -n +# shows — pods aren't matching the service +kubectl get pods -n --show-labels +# pod labels don't include app: / managedBy: kustomize / partOf: wild-cloud +``` + +**Fix**: delete the Deployment and re-deploy: +```bash +kubectl delete deployment -n +wild app deploy +``` + +## Same-origin iframes + +The global Traefik `security-headers` middleware sets `X-Frame-Options: SAMEORIGIN`. Apps that embed their own sub-pages in iframes (e.g. Etherpad's pad editor) work correctly with this setting. + +`DENY` was the previous setting and broke same-origin iframes. If you see: +``` +Blocked a frame with origin "https://..." from accessing a cross-origin frame +``` +the app is likely creating a same-origin iframe that a stale `DENY` header is blocking. Verify the cluster's Traefik middleware is using `SAMEORIGIN`. + +**Note**: route-level Traefik middlewares run before entrypoint middlewares on the response path. A per-app middleware cannot override the global one. + +## TLS-terminating reverse proxy (DISABLE_HTTPS) + +Some apps (e.g. Zulip) redirect port 80 → HTTPS internally. When Traefik terminates TLS and forwards plain HTTP, this causes an infinite redirect loop. + +Set these env vars to disable the internal redirect and trust the forwarded proto: +```yaml +- name: DISABLE_HTTPS + value: "true" +- name: LOADBALANCER_IPS + value: "10.244.0.0/16" # Kubernetes pod CIDR (Flannel default) +``` + +Also update liveness/readiness probes from HTTPS port 443 to HTTP port 80. diff --git a/e2e-test-app/app.yaml b/e2e-test-app/app.yaml index 366c742..8480a08 100644 --- a/e2e-test-app/app.yaml +++ b/e2e-test-app/app.yaml @@ -3,6 +3,8 @@ is: e2e-test-app description: End-to-end test application for automated integration testing. Includes PVC and PostgreSQL dependency to exercise all backup strategies. category: services latest: "2" +ignoreRules: + - WC-ICON # test/QA app; not user-facing upgrade: from: - version: ">=1.0.0" diff --git a/eventyay/app.yaml b/eventyay/app.yaml index 547d9c7..8e4cd7b 100644 --- a/eventyay/app.yaml +++ b/eventyay/app.yaml @@ -4,3 +4,5 @@ description: Eventyay is an open-source event management platform covering ticke category: community icon: https://raw.githubusercontent.com/fossasia/eventyay/main/app/eventyay/static/common/img/logo.svg latest: "1" +ignoreRules: + - WC-IMG # no versioned Docker tags upstream; main tag is intentional diff --git a/example-admin/app.yaml b/example-admin/app.yaml index ed2b315..918de62 100644 --- a/example-admin/app.yaml +++ b/example-admin/app.yaml @@ -3,3 +3,6 @@ is: example description: An example application that is deployed with internal-only access. category: services latest: "1" +ignoreRules: + - WC-ICON # example/demo app + - WC-IMG # example/demo app intentionally uses latest for simplicity diff --git a/example-admin/versions/1/deployment.yaml b/example-admin/versions/1/deployment.yaml index a54d149..7da2ab5 100644 --- a/example-admin/versions/1/deployment.yaml +++ b/example-admin/versions/1/deployment.yaml @@ -15,6 +15,9 @@ spec: labels: app: example-admin spec: + securityContext: + seccompProfile: + type: RuntimeDefault containers: - name: example-admin image: nginx:latest @@ -28,6 +31,15 @@ spec: requests: cpu: 50m memory: 10Mi + securityContext: + runAsNonRoot: false + runAsUser: 0 + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false + seccompProfile: + type: RuntimeDefault livenessProbe: httpGet: path: / diff --git a/example-admin/versions/1/ingress.yaml b/example-admin/versions/1/ingress.yaml index d395421..1539e6b 100644 --- a/example-admin/versions/1/ingress.yaml +++ b/example-admin/versions/1/ingress.yaml @@ -3,7 +3,10 @@ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example-admin + annotations: + external-dns.alpha.kubernetes.io/target: '{{ .externalDnsDomain }}' spec: + ingressClassName: traefik rules: - host: "{{ .host }}" http: diff --git a/example-admin/versions/1/manifest.yaml b/example-admin/versions/1/manifest.yaml index c086a50..b0f8639 100644 --- a/example-admin/versions/1/manifest.yaml +++ b/example-admin/versions/1/manifest.yaml @@ -1,4 +1,4 @@ -version: 1.0.0-1 +version: 1.0.0-2 defaultConfig: namespace: example-admin externalDnsDomain: '{{ .cloud.domain }}' diff --git a/example-app/app.yaml b/example-app/app.yaml index 53bdc19..f5eb47d 100644 --- a/example-app/app.yaml +++ b/example-app/app.yaml @@ -3,3 +3,5 @@ is: example description: An example application that is deployed with public access. category: services latest: "1" +ignoreRules: + - WC-ICON # example/demo app diff --git a/example-app/versions/1/deployment.yaml b/example-app/versions/1/deployment.yaml index 9a64ff8..1b343e2 100644 --- a/example-app/versions/1/deployment.yaml +++ b/example-app/versions/1/deployment.yaml @@ -13,6 +13,9 @@ spec: labels: app: example-app spec: + securityContext: + seccompProfile: + type: RuntimeDefault containers: - name: example-app image: nginx:alpine @@ -26,6 +29,15 @@ spec: requests: cpu: 50m memory: 32Mi + securityContext: + runAsNonRoot: false + runAsUser: 0 + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false + seccompProfile: + type: RuntimeDefault livenessProbe: httpGet: path: / diff --git a/externaldns/app.yaml b/externaldns/app.yaml index b87eeb4..8fb48bc 100644 --- a/externaldns/app.yaml +++ b/externaldns/app.yaml @@ -3,3 +3,7 @@ is: externaldns description: Automatically configures DNS records for services category: services latest: "v0" +ignoreRules: + - WC-ICON # infrastructure service; not user-facing + - WC-SEL # infrastructure service; selectors are managed by the upstream manifest + - WC-NS # fixed namespace: externaldns diff --git a/externaldns/versions/v0/externaldns-cloudflare.yaml b/externaldns/versions/v0/externaldns-cloudflare.yaml index c3e3b0c..9bf1e0d 100644 --- a/externaldns/versions/v0/externaldns-cloudflare.yaml +++ b/externaldns/versions/v0/externaldns-cloudflare.yaml @@ -16,6 +16,10 @@ spec: app: external-dns spec: serviceAccountName: external-dns + securityContext: + runAsNonRoot: true + seccompProfile: + type: RuntimeDefault containers: - name: external-dns image: registry.k8s.io/external-dns/external-dns:v0.13.4 @@ -30,6 +34,11 @@ spec: - --publish-internal-services - --no-cloudflare-proxied - --log-level=debug + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: true env: - name: CF_API_TOKEN valueFrom: diff --git a/externaldns/versions/v0/manifest.yaml b/externaldns/versions/v0/manifest.yaml index 4cf035c..426a52b 100644 --- a/externaldns/versions/v0/manifest.yaml +++ b/externaldns/versions/v0/manifest.yaml @@ -1,4 +1,4 @@ -version: v0.13.4-1 +version: v0.13.4-2 deploymentName: external-dns requires: - name: cert-manager diff --git a/gancio/notes.md b/gancio/notes.md new file mode 100644 index 0000000..a88a00f --- /dev/null +++ b/gancio/notes.md @@ -0,0 +1,21 @@ +# Gancio — Notes + +## "Non empty db" crash on re-deploy + +Gancio stores its setup state in `config.json` on the data PVC. If the PVC is lost or the app is +deleted and redeployed against an existing database, Gancio finds a non-empty DB but no +`config.json` and refuses to start: `"Non empty db! Please move your current db elsewhere than retry."` + +**Fix**: drop and recreate the public schema, then restart the deployment: + +```bash +kubectl exec -n postgres -- psql -U postgres gancio \ + -c "DROP SCHEMA public CASCADE; CREATE SCHEMA public; GRANT ALL ON SCHEMA public TO gancio; GRANT ALL ON SCHEMA public TO public;" + +kubectl scale deployment -n gancio gancio --replicas=0 +# wait for pod to terminate +kubectl scale deployment -n gancio gancio --replicas=1 +``` + +Scale down before dropping the schema to release the database connection first. +This only affects re-deployments; fresh installs work fine. diff --git a/gancio/versions/1/README.md b/gancio/versions/1/README.md index a7f6b01..13d685d 100644 --- a/gancio/versions/1/README.md +++ b/gancio/versions/1/README.md @@ -28,6 +28,6 @@ Key settings in `config.yaml`: ## Notes -- **Re-deploy warning**: If the app is deleted and redeployed with an existing database, Gancio will refuse to start with a "Non empty db" error. Fix by dropping and recreating the public schema in the `gancio` database before restarting. See ADDING-APPS-NOTES.md note 18 for the exact commands. +- **Re-deploy warning**: If the app is deleted and redeployed with an existing database, Gancio will refuse to start with a "Non empty db" error. See `gancio/notes.md` for the fix. - Gancio stores its setup state in `config.json` on the data PVC — do not delete the PVC without also clearing the database - Events are federated to Mastodon and other ActivityPub platforms diff --git a/ghost/notes.md b/ghost/notes.md new file mode 100644 index 0000000..4e5da89 --- /dev/null +++ b/ghost/notes.md @@ -0,0 +1,15 @@ +# Ghost — Notes + +## Bitnami image migration + +Ghost was originally packaged using `bitnami/ghost`. Bitnami moved their images off Docker Hub +in 2023, so the Bitnami image no longer pulls without authentication. + +**Fix**: use the official `ghost:X.Y.Z-alpine` image. The official image has different conventions: + +| | Bitnami | Official | +|---|---|---| +| DB host env | `GHOST_DATABASE_HOST` | `database__connection__host` | +| Content path | `/bitnami/ghost` | `/var/lib/ghost/content` | + +Check `docker manifest inspect ghost:latest` for the current version tag before updating. diff --git a/ghost/versions/5/db-init-job.yaml b/ghost/versions/5/db-init-job.yaml index a15ef5d..5635da1 100644 --- a/ghost/versions/5/db-init-job.yaml +++ b/ghost/versions/5/db-init-job.yaml @@ -19,6 +19,7 @@ spec: mysql -h ${DB_HOSTNAME} -P ${DB_PORT} -u root -p${MYSQL_ROOT_PASSWORD} <` | numeric ID only | +| `auth register --user ` | username string | +| `users destroy --identifier ` | numeric ID only (not positional) | + +Always list users first to get the ID: + +```bash +headscale users list + +headscale preauthkeys create --user 1 # numeric ID +headscale auth register --auth-id --user payne # username OK +headscale users destroy --identifier 1 # numeric ID +``` + +## Node registration via browser + +When a Tailscale client connects via browser (not pre-auth key), headscale shows a command like: + +``` +headscale auth register --auth-id hskey-authreq-XXXX --user USERNAME +``` + +Run it via: + +```bash +kubectl exec -n headscale deploy/headscale -- \ + headscale auth register --auth-id --user +``` diff --git a/immich/versions/1/deployment-machine-learning.yaml b/immich/versions/1/deployment-machine-learning.yaml index fc861b3..969302b 100644 --- a/immich/versions/1/deployment-machine-learning.yaml +++ b/immich/versions/1/deployment-machine-learning.yaml @@ -16,6 +16,11 @@ spec: app: immich-machine-learning component: machine-learning spec: + securityContext: + runAsNonRoot: true + runAsUser: 1000 + seccompProfile: + type: RuntimeDefault containers: - image: "ghcr.io/immich-app/immich-machine-learning:v1.135.3" name: immich-machine-learning @@ -25,6 +30,11 @@ spec: env: - name: TZ value: "UTC" + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false volumeMounts: - mountPath: /cache name: immich-cache diff --git a/immich/versions/1/deployment-microservices.yaml b/immich/versions/1/deployment-microservices.yaml index e7f8699..a3a261b 100644 --- a/immich/versions/1/deployment-microservices.yaml +++ b/immich/versions/1/deployment-microservices.yaml @@ -9,16 +9,18 @@ spec: matchLabels: app: immich-microservices strategy: - rollingUpdate: - maxSurge: 0 - maxUnavailable: 1 - type: RollingUpdate + type: Recreate template: metadata: labels: app: immich-microservices component: microservices spec: + securityContext: + runAsNonRoot: true + runAsUser: 1000 + seccompProfile: + type: RuntimeDefault containers: - image: "ghcr.io/immich-app/immich-server:v1.135.3" name: immich-microservices @@ -43,6 +45,11 @@ spec: value: "UTC" - name: IMMICH_WORKERS_EXCLUDE value: api + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false volumeMounts: - mountPath: /usr/src/app/upload name: immich-storage diff --git a/immich/versions/1/deployment-server.yaml b/immich/versions/1/deployment-server.yaml index ac8f70e..48b5886 100644 --- a/immich/versions/1/deployment-server.yaml +++ b/immich/versions/1/deployment-server.yaml @@ -9,16 +9,18 @@ spec: matchLabels: app: immich-server strategy: - rollingUpdate: - maxSurge: 0 - maxUnavailable: 1 - type: RollingUpdate + type: Recreate template: metadata: labels: app: immich-server component: server spec: + securityContext: + runAsNonRoot: true + runAsUser: 1000 + seccompProfile: + type: RuntimeDefault containers: - image: "ghcr.io/immich-app/immich-server:v1.135.3" name: immich-server @@ -46,6 +48,11 @@ spec: value: "UTC" - name: IMMICH_WORKERS_EXCLUDE value: microservices + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false volumeMounts: - mountPath: /usr/src/app/upload name: immich-storage diff --git a/immich/versions/1/ingress.yaml b/immich/versions/1/ingress.yaml index c5826db..88cc0e8 100644 --- a/immich/versions/1/ingress.yaml +++ b/immich/versions/1/ingress.yaml @@ -7,6 +7,7 @@ metadata: external-dns.alpha.kubernetes.io/target: "{{ .externalDnsDomain }}" external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" spec: + ingressClassName: traefik rules: - host: "{{ .domain }}" http: diff --git a/immich/versions/1/manifest.yaml b/immich/versions/1/manifest.yaml index e0108ac..9502eb9 100644 --- a/immich/versions/1/manifest.yaml +++ b/immich/versions/1/manifest.yaml @@ -1,4 +1,4 @@ -version: 1.135.3-2 +version: 1.135.3-4 requires: - name: redis - name: postgres diff --git a/indico/versions/3/pvc.yaml b/indico/versions/3/pvc.yaml index b92a019..21e0ca4 100644 --- a/indico/versions/3/pvc.yaml +++ b/indico/versions/3/pvc.yaml @@ -6,7 +6,6 @@ metadata: spec: accessModes: - ReadWriteMany - storageClassName: nfs resources: requests: storage: {{ .storage }} diff --git a/karrot/versions/1/pvc.yaml b/karrot/versions/1/pvc.yaml index bb82973..11cffef 100644 --- a/karrot/versions/1/pvc.yaml +++ b/karrot/versions/1/pvc.yaml @@ -6,7 +6,6 @@ metadata: spec: accessModes: - ReadWriteMany - storageClassName: nfs resources: requests: storage: {{ .storage }} diff --git a/keila/versions/0/deployment.yaml b/keila/versions/0/deployment.yaml index 8e90009..808fb9a 100644 --- a/keila/versions/0/deployment.yaml +++ b/keila/versions/0/deployment.yaml @@ -14,6 +14,9 @@ spec: labels: component: web spec: + securityContext: + seccompProfile: + type: RuntimeDefault containers: - name: keila image: "pentacent/keila:0.17.1" @@ -69,6 +72,11 @@ spec: volumeMounts: - name: uploads mountPath: /var/lib/keila/uploads + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false livenessProbe: httpGet: path: / diff --git a/keila/versions/0/ingress.yaml b/keila/versions/0/ingress.yaml index 1f22453..4045a82 100644 --- a/keila/versions/0/ingress.yaml +++ b/keila/versions/0/ingress.yaml @@ -9,6 +9,7 @@ metadata: external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" traefik.ingress.kubernetes.io/router.middlewares: keila-cors@kubernetescrd spec: + ingressClassName: traefik rules: - host: {{ .domain }} http: diff --git a/keila/versions/0/manifest.yaml b/keila/versions/0/manifest.yaml index 5e838e3..59bd01f 100644 --- a/keila/versions/0/manifest.yaml +++ b/keila/versions/0/manifest.yaml @@ -1,4 +1,4 @@ -version: 0.17.1-2 +version: 0.17.1-4 requires: - name: postgres - name: smtp diff --git a/lemmy/versions/0/deployment-backend.yaml b/lemmy/versions/0/deployment-backend.yaml index b518f76..0b9a33a 100644 --- a/lemmy/versions/0/deployment-backend.yaml +++ b/lemmy/versions/0/deployment-backend.yaml @@ -22,7 +22,7 @@ spec: type: RuntimeDefault initContainers: - name: config-prep - image: busybox:stable + image: busybox:1.38.0 securityContext: allowPrivilegeEscalation: false capabilities: diff --git a/lemmy/versions/0/deployment-pictrs.yaml b/lemmy/versions/0/deployment-pictrs.yaml index 74d4f41..b206102 100644 --- a/lemmy/versions/0/deployment-pictrs.yaml +++ b/lemmy/versions/0/deployment-pictrs.yaml @@ -5,6 +5,8 @@ metadata: namespace: {{ .namespace }} spec: replicas: 1 + strategy: + type: Recreate selector: matchLabels: component: pictrs diff --git a/longhorn/app.yaml b/longhorn/app.yaml index 21ee7c1..3d5cdf3 100644 --- a/longhorn/app.yaml +++ b/longhorn/app.yaml @@ -2,4 +2,12 @@ name: longhorn is: longhorn description: Cloud-native distributed block storage for Kubernetes category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/longhorn.svg latest: "v1" +ignoreRules: + - WC-HELM + - WC-SEL + - WC-NS # fixed namespace: longhorn-system + - WC-DNS # internal-only ingress (internalDomain), no external DNS + - WC-SC-POD # upstream manifest: longhorn.yaml has its own pod security policy + - WC-SC-CTR # upstream manifest: longhorn.yaml has its own container security policy diff --git a/longhorn/versions/v1/ingress.yaml b/longhorn/versions/v1/ingress.yaml index 23ca68a..5c47ae7 100644 --- a/longhorn/versions/v1/ingress.yaml +++ b/longhorn/versions/v1/ingress.yaml @@ -4,6 +4,7 @@ metadata: name: longhorn-ingress namespace: longhorn-system spec: + ingressClassName: traefik rules: - host: "longhorn.{{ .internalDomain }}" http: diff --git a/longhorn/versions/v1/manifest.yaml b/longhorn/versions/v1/manifest.yaml index 4b8c429..08e279b 100644 --- a/longhorn/versions/v1/manifest.yaml +++ b/longhorn/versions/v1/manifest.yaml @@ -1,4 +1,4 @@ -version: v1.8.1-3 +version: v1.8.1-4 deploymentName: longhorn-ui requires: - name: traefik diff --git a/loomio/app.yaml b/loomio/app.yaml index e1a8fd5..69c25ec 100644 --- a/loomio/app.yaml +++ b/loomio/app.yaml @@ -4,3 +4,5 @@ description: Loomio is a collaborative decision-making tool that makes it easy f category: community icon: https://www.loomio.com/brand/logo_gold.svg latest: "3" +ignoreRules: + - WC-IMG # loomio_channel_server has no versioned tags; stable is the best available pin diff --git a/loomio/notes.md b/loomio/notes.md new file mode 100644 index 0000000..675b7b1 --- /dev/null +++ b/loomio/notes.md @@ -0,0 +1,24 @@ +# Loomio — Notes + +## Custom startup command must reproduce ENTRYPOINT setup steps + +Loomio's Docker image uses `docker_start.sh` as its ENTRYPOINT, which copies +`client3-build/` → `public/client3/` before starting Rails. Overriding the entrypoint with a +direct Rails command skips this step, leaving `client3/` unpopulated and causing 500 errors on +all routes serving the SPA. + +**Fix**: reproduce the copy step explicitly in the deployment `command:`: + +```yaml +command: ["/bin/bash", "-c"] +args: + - | + set -e + mkdir -p /loomio/public/client3 + cp -r /loomio/client3-build/* /loomio/public/client3/ + bundle exec rake db:migrate db:seed + bundle exec thrust puma -C config/puma.rb +``` + +**General rule**: whenever you override `command:`, inspect the image's `ENTRYPOINT`/`CMD` +(via `docker inspect` or the Dockerfile) and reproduce any required setup steps. diff --git a/loomio/versions/3/deployment-channels.yaml b/loomio/versions/3/deployment-channels.yaml index 1a7a832..d4b9fa5 100644 --- a/loomio/versions/3/deployment-channels.yaml +++ b/loomio/versions/3/deployment-channels.yaml @@ -12,9 +12,14 @@ spec: labels: component: channels spec: + securityContext: + runAsNonRoot: true + runAsUser: 1000 + seccompProfile: + type: RuntimeDefault containers: - name: channels - image: loomio/loomio_channel_server:latest + image: loomio/loomio_channel_server:stable ports: - containerPort: 3000 name: http diff --git a/loomio/versions/3/deployment-worker.yaml b/loomio/versions/3/deployment-worker.yaml index d43c3cc..d3d35fe 100644 --- a/loomio/versions/3/deployment-worker.yaml +++ b/loomio/versions/3/deployment-worker.yaml @@ -12,9 +12,12 @@ spec: labels: component: worker spec: + securityContext: + seccompProfile: + type: RuntimeDefault containers: - name: worker - image: loomio/loomio:latest + image: loomio/loomio:3.0.24 env: - name: TASK value: worker diff --git a/loomio/versions/3/deployment.yaml b/loomio/versions/3/deployment.yaml index 8d8dc94..c033c89 100644 --- a/loomio/versions/3/deployment.yaml +++ b/loomio/versions/3/deployment.yaml @@ -14,14 +14,19 @@ spec: labels: component: web spec: + securityContext: + seccompProfile: + type: RuntimeDefault containers: - name: loomio - image: loomio/loomio:latest + image: loomio/loomio:3.0.24 command: - /bin/bash - -c - | set -e + mkdir -p /loomio/public/client3 + cp -r /loomio/client3-build/* /loomio/public/client3/ bundle exec rake db:migrate db:seed bundle exec thrust puma -C config/puma.rb ports: diff --git a/loomio/versions/3/manifest.yaml b/loomio/versions/3/manifest.yaml index 2ea4116..e7a3c00 100644 --- a/loomio/versions/3/manifest.yaml +++ b/loomio/versions/3/manifest.yaml @@ -1,4 +1,4 @@ -version: 3.0.11-5 +version: 3.0.24-2 requires: - name: postgres installed_as: postgres @@ -34,7 +34,7 @@ defaultSecrets: - key: dbPassword default: "{{ random.AlphaNum 32 }}" - key: dbUrl - default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?pool=30" + default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?pool=30&sslmode=disable" - key: deviseSecret default: "{{ random.AlphaNum 32 }}" - key: secretCookieToken diff --git a/memcached/versions/1/deployment.yaml b/memcached/versions/1/deployment.yaml index dc1b025..9968364 100644 --- a/memcached/versions/1/deployment.yaml +++ b/memcached/versions/1/deployment.yaml @@ -12,6 +12,11 @@ spec: labels: component: cache spec: + securityContext: + runAsNonRoot: true + runAsUser: 11211 + seccompProfile: + type: RuntimeDefault containers: - name: memcached image: "memcached:1.6.32-alpine" diff --git a/metallb/app.yaml b/metallb/app.yaml index 2dd811b..7ed1adf 100644 --- a/metallb/app.yaml +++ b/metallb/app.yaml @@ -2,4 +2,5 @@ name: metallb is: metallb description: Bare metal load-balancer for Kubernetes category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/metallb.svg latest: "v0" diff --git a/moodle/app.yaml b/moodle/app.yaml index 1ab9c2c..d65fbf3 100644 --- a/moodle/app.yaml +++ b/moodle/app.yaml @@ -4,3 +4,5 @@ description: Moodle is a free, open-source learning management system (LMS) used category: education icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/moodle.svg latest: "4" +ignoreRules: + - WC-BITNAMI # no official Moodle image exists; bitnamilegacy is the standard diff --git a/moodle/notes.md b/moodle/notes.md new file mode 100644 index 0000000..2b1b95b --- /dev/null +++ b/moodle/notes.md @@ -0,0 +1,27 @@ +# Moodle — Notes + +## No official Docker image — Bitnami is the standard + +The Moodle project does not publish an official Docker image. `bitnamilegacy/moodle` is the +widely-used standard and is the basis for the official Bitnami Helm chart. The `WC-BITNAMI` +warning is suppressed in `app.yaml` because there is no viable alternative. + +Note: Bitnami images require Docker Hub authentication since 2023. Ensure the cluster has +Docker Hub credentials configured if pull rate limits or auth errors appear. + +## Bitnami filesystem layout + +Bitnami images store persistent data under `/bitnami`. The PVC is mounted at `/bitnami` to +cover both Moodle files and any Bitnami-managed config. Do not change the mount path without +also adjusting Bitnami's internal configuration. + +## MySQL-only + +Moodle's `MOODLE_DATABASE_TYPE=mysqli` requires MySQL/MariaDB. PostgreSQL is not supported. +The deployment uses Wild Cloud's MySQL dependency (`requires: [mysql]`). + +## Long first-run initialization + +Moodle runs database migrations and installs core plugins on first startup, which can take +5–15 minutes. The `startupProbe` is configured with a long timeout (`failureThreshold: 240`, +`periodSeconds: 15` = up to 60 minutes) to accommodate this. diff --git a/moodle/versions/4/deployment.yaml b/moodle/versions/4/deployment.yaml index f1afcd9..82c149b 100644 --- a/moodle/versions/4/deployment.yaml +++ b/moodle/versions/4/deployment.yaml @@ -18,6 +18,7 @@ spec: securityContext: runAsNonRoot: true runAsUser: 994 + fsGroup: 994 seccompProfile: type: RuntimeDefault containers: diff --git a/nfs/app.yaml b/nfs/app.yaml index 3cdd10a..c4de849 100644 --- a/nfs/app.yaml +++ b/nfs/app.yaml @@ -3,3 +3,7 @@ is: nfs description: NFS client provisioner for external NFS storage category: services latest: "v4" + +ignoreRules: + - WC-ICON # config-only NFS settings; not a deployable app + - WC-CFG-NS # config-only app, no Kubernetes namespace diff --git a/node-feature-discovery/app.yaml b/node-feature-discovery/app.yaml index 9dcdda4..45d9ed3 100644 --- a/node-feature-discovery/app.yaml +++ b/node-feature-discovery/app.yaml @@ -3,3 +3,6 @@ is: node-feature-discovery description: Detects hardware features available on each node category: services latest: "v0" +ignoreRules: + - WC-ICON # infrastructure/hardware plugin; not user-facing + - WC-SEL diff --git a/nvidia-device-plugin/app.yaml b/nvidia-device-plugin/app.yaml index 19019f0..b1d4b7f 100644 --- a/nvidia-device-plugin/app.yaml +++ b/nvidia-device-plugin/app.yaml @@ -3,3 +3,7 @@ is: nvidia-device-plugin description: NVIDIA device plugin for Kubernetes category: services latest: "v0" +ignoreRules: + - WC-ICON # hardware plugin; not user-facing + - WC-HELM + - WC-SEL diff --git a/odoo/notes.md b/odoo/notes.md new file mode 100644 index 0000000..b1ee894 --- /dev/null +++ b/odoo/notes.md @@ -0,0 +1,27 @@ +# Odoo — Notes + +## Explicit database name and `-i base` required on first run + +The official Odoo Docker image does not auto-initialize the database. Without specifying the +database name and installing the base module, Odoo starts in multi-database manager mode and +health checks fail with "Database not initialized". + +```yaml +args: ["-d", "DATABASE_NAME", "-i", "base"] +``` + +The `-i base` flag installs Odoo's base module and creates all database tables on first run. +Subsequent runs with `-i base` are safe — it updates rather than reinstalls. + +## First-run initialization takes 10–15 minutes + +Use a `startupProbe` with a long window to prevent premature restarts: + +```yaml +startupProbe: + failureThreshold: 40 + periodSeconds: 30 # up to 20 minutes total +``` + +Once the startup probe passes, regular liveness/readiness probes with `initialDelaySeconds: 0` +take over. diff --git a/open-webui/versions/0/deployment.yaml b/open-webui/versions/0/deployment.yaml index 39cdd30..5ee0677 100644 --- a/open-webui/versions/0/deployment.yaml +++ b/open-webui/versions/0/deployment.yaml @@ -38,7 +38,10 @@ spec: - name: OPENAI_API_BASE_URL value: "{{ .vllmApiUrl }}" - name: OPENAI_API_KEY - value: "sk-placeholder" # Required but not used with vLLM + valueFrom: + secretKeyRef: + name: open-webui-secrets + key: vllmApiKey - name: WEBUI_SECRET_KEY valueFrom: secretKeyRef: diff --git a/open-webui/versions/0/ingress.yaml b/open-webui/versions/0/ingress.yaml index 3deb83f..6de69a0 100644 --- a/open-webui/versions/0/ingress.yaml +++ b/open-webui/versions/0/ingress.yaml @@ -8,6 +8,7 @@ metadata: external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" traefik.ingress.kubernetes.io/router.middlewares: crowdsec-crowdsec-bouncer@kubernetescrd,crowdsec-rate-limit@kubernetescrd spec: + ingressClassName: traefik rules: - host: {{ .domain }} http: diff --git a/open-webui/versions/0/manifest.yaml b/open-webui/versions/0/manifest.yaml index e8d1c72..d4afa58 100644 --- a/open-webui/versions/0/manifest.yaml +++ b/open-webui/versions/0/manifest.yaml @@ -1,4 +1,4 @@ -version: 0.9.5-2 +version: 0.9.5-4 requires: [] defaultConfig: namespace: open-webui @@ -10,3 +10,4 @@ defaultConfig: defaultSecrets: - key: secretKey - key: adminPassword + - key: vllmApiKey diff --git a/openproject/versions/16/ingress.yaml b/openproject/versions/16/ingress.yaml index 6571153..1688819 100644 --- a/openproject/versions/16/ingress.yaml +++ b/openproject/versions/16/ingress.yaml @@ -8,6 +8,7 @@ metadata: external-dns.alpha.kubernetes.io/target: {{ .externalDnsDomain }} external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" spec: + ingressClassName: traefik tls: - hosts: - "{{ .domain }}" diff --git a/openproject/versions/16/manifest.yaml b/openproject/versions/16/manifest.yaml index 386c9ab..9b3ec41 100644 --- a/openproject/versions/16/manifest.yaml +++ b/openproject/versions/16/manifest.yaml @@ -1,4 +1,4 @@ -version: 16.1.1-3 +version: 16.1.1-4 requires: - name: postgres - name: memcached diff --git a/pixelfed/notes.md b/pixelfed/notes.md new file mode 100644 index 0000000..2c498cb --- /dev/null +++ b/pixelfed/notes.md @@ -0,0 +1,21 @@ +# Pixelfed — Notes + +## mattlqx/docker-pixelfed: APP_PORT required + +The `ghcr.io/mattlqx/docker-pixelfed` community image generates its nginx config from env vars +using `sed`. The `listen` directive uses `${APP_PORT}`, which must be set explicitly — without it, +nginx crashes with `invalid number of arguments in 'listen' directive`. + +```yaml +env: + - name: APP_PORT + value: "80" +``` + +The image listens on port 80 (nginx), not 8080 as the original pixelfed image did. + +## Shared storage PVC requires ReadWriteMany + +Both the web and worker deployments must mount the shared storage PVC simultaneously. Use +`ReadWriteMany` access mode — `ReadWriteOnce` with two pods causes multi-attach errors. +(`RollingUpdate` + RWO also triggers this; use `Recreate` strategy or RWX.) diff --git a/polis/app.yaml b/polis/app.yaml index 1b6ec2d..466e3ad 100644 --- a/polis/app.yaml +++ b/polis/app.yaml @@ -2,4 +2,7 @@ name: polis is: polis description: Pol.is is an open-source large-scale conversational survey tool that helps communities gather, understand, and act on the expressed opinions of large groups of people. category: community +icon: https://avatars.githubusercontent.com/u/5333592?s=200&v=4 latest: dev +ignoreRules: + - WC-IMG # private AWS ECR images; no public versioned alternative available diff --git a/polis/notes.md b/polis/notes.md new file mode 100644 index 0000000..b110d75 --- /dev/null +++ b/polis/notes.md @@ -0,0 +1,9 @@ +# Polis — Notes + +## No public Docker images + +`compdemocracy/polis-server` images are private — they are not publicly available on Docker Hub +or ghcr.io. Deploying without pre-built images results in `ImagePullBackOff`. + +The current package uses private AWS ECR images. This is tracked in `app.yaml` via +`ignoreRules: [WC-IMG]`. Building images from source is required for a fully self-hostable package. diff --git a/polis/versions/dev/manifest.yaml b/polis/versions/dev/manifest.yaml index 0272142..e80a01f 100644 --- a/polis/versions/dev/manifest.yaml +++ b/polis/versions/dev/manifest.yaml @@ -15,7 +15,7 @@ defaultConfig: defaultSecrets: - key: dbPassword - key: dbUrl - default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}" + default: "postgresql://{{ .app.db.user }}:{{ .secrets.dbPassword }}@{{ .app.db.host }}:{{ .app.db.port }}/{{ .app.db.name }}?sslmode=disable" - key: loginCodePepper - key: jwtPrivateKey - key: jwtPublicKey diff --git a/postgres/versions/1/deployment.yaml b/postgres/versions/1/deployment.yaml index 4b0abfe..f59a35a 100644 --- a/postgres/versions/1/deployment.yaml +++ b/postgres/versions/1/deployment.yaml @@ -15,6 +15,13 @@ spec: labels: app: postgres spec: + securityContext: + runAsNonRoot: true + runAsUser: 999 + runAsGroup: 999 + fsGroup: 999 + seccompProfile: + type: RuntimeDefault containers: - name: postgres image: "pgvector/pgvector:pg15" @@ -39,6 +46,10 @@ spec: -c idle_in_transaction_session_timeout=600000 ports: - containerPort: 5432 + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] env: - name: PGDATA value: /var/lib/postgresql/data/pgdata diff --git a/redis/versions/1/deployment.yaml b/redis/versions/1/deployment.yaml index 107494b..ac5e3c5 100644 --- a/redis/versions/1/deployment.yaml +++ b/redis/versions/1/deployment.yaml @@ -13,6 +13,12 @@ spec: labels: app: redis spec: + securityContext: + runAsNonRoot: true + runAsUser: 999 + runAsGroup: 999 + seccompProfile: + type: RuntimeDefault containers: - image: "redis:alpine" name: redis @@ -24,6 +30,10 @@ spec: secretKeyRef: name: redis-secrets key: password + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] command: - redis-server - --requirepass diff --git a/smtp/app.yaml b/smtp/app.yaml index ad2783a..ea3464f 100644 --- a/smtp/app.yaml +++ b/smtp/app.yaml @@ -3,3 +3,7 @@ is: smtp description: SMTP relay service for cluster applications category: services latest: "1" + +ignoreRules: + - WC-ICON # config-only SMTP settings; not a deployable app + - WC-CFG-NS # config-only app, no Kubernetes namespace diff --git a/snapshot-controller/app.yaml b/snapshot-controller/app.yaml index e9efe51..dd352ab 100644 --- a/snapshot-controller/app.yaml +++ b/snapshot-controller/app.yaml @@ -3,3 +3,11 @@ is: snapshot-controller description: Kubernetes CSI Snapshot Controller for managing VolumeSnapshots category: services latest: "v8" +ignoreRules: + - WC-ICON # K8s internal addon; not user-facing + - WC-SEL # upstream manifest + - WC-HELM # upstream manifest uses Helm-style labels + - WC-NS # fixed namespace: kube-system + - WC-NSFILE # deploys into kube-system (pre-existing) + - WC-SC-POD # upstream manifest + - WC-SC-CTR # upstream manifest diff --git a/supabase/app.yaml b/supabase/app.yaml new file mode 100644 index 0000000..e839e32 --- /dev/null +++ b/supabase/app.yaml @@ -0,0 +1,9 @@ +name: supabase +is: supabase +description: Supabase is an open-source Firebase alternative providing a Postgres database, authentication, instant REST and realtime APIs, edge functions, and a web dashboard. +category: developer +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/supabase.svg +latest: "1" +ignoreRules: + - WC-SC-POD # Most Supabase images run as root; only postgres enforces non-root + - WC-SC-CTR # See above diff --git a/supabase/notes.md b/supabase/notes.md new file mode 100644 index 0000000..6503d3f --- /dev/null +++ b/supabase/notes.md @@ -0,0 +1,126 @@ +# Supabase Notes + +## Overview + +Supabase is a multi-component application. All components run in the `supabase` namespace: + +| Component | Image | Role | +|-----------|-------|------| +| `db` | supabase/postgres | PostgreSQL with Supabase extensions | +| `kong` | kong/kong | API gateway, entry point for all traffic | +| `auth` | supabase/gotrue | JWT-based authentication | +| `rest` | postgrest/postgrest | Auto-generated REST API from Postgres | +| `realtime` | supabase/realtime | WebSocket subscriptions | +| `storage` + `imgproxy` | supabase/storage-api + darthsim/imgproxy | File storage (imgproxy as sidecar) | +| `meta` | supabase/postgres-meta | DB introspection for Studio | +| `studio` | supabase/studio | Web dashboard (protected by Kong basic-auth) | + +Traffic flows: browser → Traefik → Kong (port 8000) → individual services. + +## First-Deploy Checklist + +1. Add and deploy with: `wild app add supabase && wild app deploy supabase` +2. Wait for postgres to initialize (~2–3 min on first boot — it runs SQL init scripts) +3. Access the Studio at `https://supabase.` using the dashboard credentials +4. **Replace example API keys before going to production** (see below) + +## JWT Keys — MUST Replace for Production + +The default `jwtSecret`, `anonKey`, and `serviceRoleKey` in `secrets.yaml` are Supabase's well-known **example keys** (publicly known, expire Jan 2027). They are sufficient for testing but **must not be used in production**. + +Generate new keys using the built-in script: + +```bash +wild app run supabase generate-keys +``` + +Then copy the output values into `secrets.yaml` and redeploy: + +```bash +wild app deploy supabase +``` + +The postgres database stores the JWT secret in its settings (`app.settings.jwt_secret`). After replacing keys, the postgres pod must restart to pick up the new secret — a redeploy handles this. + +## Postgres — Internal Database + +Supabase uses its own dedicated postgres instance (`supabase/postgres` image) rather than the shared Wild Cloud postgres app. This image includes: +- ~390 extensions pre-compiled (pgjwt, wal2json, vector, pg_net, etc.) +- Supabase service roles (supabase_admin, authenticator, supabase_auth_admin, supabase_storage_admin, etc.) +- Initialization scripts for realtime, webhooks, JWT settings + +All Supabase service roles (auth, storage, realtime, etc.) use the same password as the postgres admin (`dbPassword` secret). This is the Supabase default behavior. + +### Bootstrap User: supabase_admin + +`POSTGRES_USER` is set to `supabase_admin` (not `postgres`). This is required because the image's `migrate.sh` script connects immediately as `supabase_admin` before running any init scripts. Using `POSTGRES_USER=postgres` would make postgres the bootstrap superuser, which prevents `migrate.sh` from demoting it (PostgreSQL prohibits removing SUPERUSER from the bootstrap user). + +With `POSTGRES_USER=supabase_admin`: +1. `initdb` creates `supabase_admin` as the bootstrap superuser +2. `migrate.sh` connects as `supabase_admin` and creates the `postgres` role +3. After init-scripts run, `migrate.sh` demotes `postgres` to non-superuser (this is Supabase's intended model) + +The `postgres` user remains available with database-level privileges. All Supabase service roles use specific functional accounts (supabase_auth_admin, authenticator, etc.), not the postgres superuser. + +### Volume Mount: /var/lib/postgresql (parent dir) + +The PVC is mounted at `/var/lib/postgresql` (the parent), NOT at `/var/lib/postgresql/data` (PGDATA). This is required because Longhorn's ext4 filesystem creates a `lost+found` directory, which blocks postgres `initdb` when the PVC is mounted directly at PGDATA. With the parent-dir mount, `initdb` creates `/var/lib/postgresql/data` as a subdirectory, leaving `lost+found` at the parent level where it doesn't interfere. + +## Realtime Tenant ID + +The Realtime service must identify itself with the tenant ID `realtime-dev`. The Deployment sets `spec.template.spec.hostname: realtime-dev` so the Elixir application seeds itself with this tenant ID. The Kubernetes service for Realtime is also named `realtime-dev` so Kong can route to it correctly. + +## Storage + +File storage uses the local filesystem backend by default. Objects are stored in the `supabase-storage` PVC. The `imgproxy` container is a sidecar in the same pod, sharing the storage PVC for image transformation. + +For S3-backed storage: edit `deployment-storage.yaml` to change `STORAGE_BACKEND` to `s3` and add S3 credentials as secrets. + +## SMTP + +SMTP is required for auth email flows (email confirmation, password reset). Configure the `smtp` app dependency and it will be wired automatically. If you want to disable email confirmation during initial testing, you can temporarily set `GOTRUE_MAILER_AUTOCONFIRM=true` in `deployment-auth.yaml`. + +## Studio Dashboard Credentials + +The Studio dashboard is protected by HTTP Basic Auth via Kong. Credentials: +- Username: configured via `dashboardUsername` config key (default: `supabase`) +- Password: auto-generated `dashboardPassword` secret + +To view the generated password: +```bash +kubectl -n supabase get secret supabase-secrets -o jsonpath='{.data.dashboardPassword}' | base64 -d +``` + +## Connecting Applications + +Applications connect to Supabase via the Kong API gateway: + +- **REST API**: `https://supabase./rest/v1/` +- **Auth**: `https://supabase./auth/v1/` +- **Realtime**: `wss://supabase./realtime/v1/` +- **Storage**: `https://supabase./storage/v1/` + +Use the `anonKey` for client-side (browser) access and `serviceRoleKey` for server-side (admin) access. + +## Supabase JavaScript Client + +```js +import { createClient } from '@supabase/supabase-js' + +const supabase = createClient( + 'https://supabase.', + '' // from secrets.yaml +) +``` + +## Troubleshooting + +**Postgres won't start**: Check that the supabase/postgres image can be pulled and that the PVC is bound. + +**Auth returns 500**: Postgres is not ready or the init scripts haven't completed. Check DB pod logs and wait for all init SQL to finish. + +**Kong returns 401 for all requests**: The `anonKey` in secrets doesn't match the `jwtSecret`. Regenerate keys with `wild app run supabase generate-keys`. + +**Realtime health check fails**: The pod hostname may not be `realtime-dev`. Check the Deployment `spec.template.spec.hostname` field. + +**Studio shows "Project not found"**: Studio needs Kong to be healthy. Check Kong pod logs for declarative config errors. diff --git a/supabase/versions/1/configmap-db-init.yaml b/supabase/versions/1/configmap-db-init.yaml new file mode 100644 index 0000000..aa28017 --- /dev/null +++ b/supabase/versions/1/configmap-db-init.yaml @@ -0,0 +1,159 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: supabase-db-init +data: + jwt.sql: | + \set jwt_secret `echo "$JWT_SECRET"` + \set jwt_exp `echo "$JWT_EXP"` + + ALTER DATABASE postgres SET "app.settings.jwt_secret" TO :'jwt_secret'; + ALTER DATABASE postgres SET "app.settings.jwt_exp" TO :'jwt_exp'; + + roles.sql: | + \set pgpass `echo "$POSTGRES_PASSWORD"` + + ALTER USER authenticator WITH PASSWORD :'pgpass'; + ALTER USER pgbouncer WITH PASSWORD :'pgpass'; + ALTER USER supabase_auth_admin WITH PASSWORD :'pgpass'; + ALTER USER supabase_functions_admin WITH PASSWORD :'pgpass'; + ALTER USER supabase_storage_admin WITH PASSWORD :'pgpass'; + + realtime.sql: | + \set pguser `echo "$POSTGRES_USER"` + + create schema if not exists _realtime; + alter schema _realtime owner to :pguser; + + webhooks.sql: | + BEGIN; + CREATE EXTENSION IF NOT EXISTS pg_net SCHEMA extensions; + CREATE SCHEMA supabase_functions AUTHORIZATION supabase_admin; + GRANT USAGE ON SCHEMA supabase_functions TO postgres, anon, authenticated, service_role; + ALTER DEFAULT PRIVILEGES IN SCHEMA supabase_functions GRANT ALL ON TABLES TO postgres, anon, authenticated, service_role; + ALTER DEFAULT PRIVILEGES IN SCHEMA supabase_functions GRANT ALL ON FUNCTIONS TO postgres, anon, authenticated, service_role; + ALTER DEFAULT PRIVILEGES IN SCHEMA supabase_functions GRANT ALL ON SEQUENCES TO postgres, anon, authenticated, service_role; + CREATE TABLE supabase_functions.migrations ( + version text PRIMARY KEY, + inserted_at timestamptz NOT NULL DEFAULT NOW() + ); + INSERT INTO supabase_functions.migrations (version) VALUES ('initial'); + CREATE TABLE supabase_functions.hooks ( + id bigserial PRIMARY KEY, + hook_table_id integer NOT NULL, + hook_name text NOT NULL, + created_at timestamptz NOT NULL DEFAULT NOW(), + request_id bigint + ); + CREATE INDEX supabase_functions_hooks_request_id_idx ON supabase_functions.hooks USING btree (request_id); + CREATE INDEX supabase_functions_hooks_h_table_id_h_name_idx ON supabase_functions.hooks USING btree (hook_table_id, hook_name); + COMMENT ON TABLE supabase_functions.hooks IS 'Supabase Functions Hooks: Audit trail for triggered hooks.'; + CREATE FUNCTION supabase_functions.http_request() + RETURNS trigger + LANGUAGE plpgsql + AS $function$ + DECLARE + request_id bigint; + payload jsonb; + url text := TG_ARGV[0]::text; + method text := TG_ARGV[1]::text; + headers jsonb DEFAULT '{}'::jsonb; + params jsonb DEFAULT '{}'::jsonb; + timeout_ms integer DEFAULT 1000; + BEGIN + IF url IS NULL OR url = 'null' THEN + RAISE EXCEPTION 'url argument is missing'; + END IF; + IF method IS NULL OR method = 'null' THEN + RAISE EXCEPTION 'method argument is missing'; + END IF; + IF TG_ARGV[2] IS NULL OR TG_ARGV[2] = 'null' THEN + headers = '{"Content-Type": "application/json"}'::jsonb; + ELSE + headers = TG_ARGV[2]::jsonb; + END IF; + IF TG_ARGV[3] IS NULL OR TG_ARGV[3] = 'null' THEN + params = '{}'::jsonb; + ELSE + params = TG_ARGV[3]::jsonb; + END IF; + IF TG_ARGV[4] IS NULL OR TG_ARGV[4] = 'null' THEN + timeout_ms = 1000; + ELSE + timeout_ms = TG_ARGV[4]::integer; + END IF; + CASE + WHEN method = 'GET' THEN + SELECT http_get INTO request_id FROM net.http_get(url, params, headers, timeout_ms); + WHEN method = 'POST' THEN + payload = jsonb_build_object('old_record', OLD, 'record', NEW, 'type', TG_OP, 'table', TG_TABLE_NAME, 'schema', TG_TABLE_SCHEMA); + SELECT http_post INTO request_id FROM net.http_post(url, payload, params, headers, timeout_ms); + ELSE + RAISE EXCEPTION 'method argument % is invalid', method; + END CASE; + INSERT INTO supabase_functions.hooks (hook_table_id, hook_name, request_id) VALUES (TG_RELID, TG_NAME, request_id); + RETURN NEW; + END + $function$; + DO $$ BEGIN + IF NOT EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'supabase_functions_admin') THEN + CREATE USER supabase_functions_admin NOINHERIT CREATEROLE LOGIN NOREPLICATION; + END IF; + END $$; + GRANT ALL PRIVILEGES ON SCHEMA supabase_functions TO supabase_functions_admin; + GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA supabase_functions TO supabase_functions_admin; + GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA supabase_functions TO supabase_functions_admin; + ALTER USER supabase_functions_admin SET search_path = "supabase_functions"; + ALTER table "supabase_functions".migrations OWNER TO supabase_functions_admin; + ALTER table "supabase_functions".hooks OWNER TO supabase_functions_admin; + ALTER function "supabase_functions".http_request() OWNER TO supabase_functions_admin; + GRANT supabase_functions_admin TO postgres; + DO $$ BEGIN + IF EXISTS (SELECT 1 FROM pg_roles WHERE rolname = 'supabase_pg_net_admin') THEN + REASSIGN OWNED BY supabase_pg_net_admin TO supabase_admin; + DROP OWNED BY supabase_pg_net_admin; + DROP ROLE supabase_pg_net_admin; + END IF; + END $$; + DO $$ BEGIN + IF EXISTS (SELECT 1 FROM pg_extension WHERE extname = 'pg_net') THEN + GRANT USAGE ON SCHEMA net TO supabase_functions_admin, postgres, anon, authenticated, service_role; + ALTER function net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) SECURITY DEFINER; + ALTER function net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) SECURITY DEFINER; + ALTER function net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) SET search_path = net; + ALTER function net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) SET search_path = net; + REVOKE ALL ON FUNCTION net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) FROM PUBLIC; + REVOKE ALL ON FUNCTION net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) FROM PUBLIC; + GRANT EXECUTE ON FUNCTION net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) TO supabase_functions_admin, postgres, anon, authenticated, service_role; + GRANT EXECUTE ON FUNCTION net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) TO supabase_functions_admin, postgres, anon, authenticated, service_role; + END IF; + END $$; + CREATE OR REPLACE FUNCTION extensions.grant_pg_net_access() + RETURNS event_trigger LANGUAGE plpgsql AS $$ + BEGIN + IF EXISTS (SELECT 1 FROM pg_event_trigger_ddl_commands() AS ev JOIN pg_extension AS ext ON ev.objid = ext.oid WHERE ext.extname = 'pg_net') THEN + GRANT USAGE ON SCHEMA net TO supabase_functions_admin, postgres, anon, authenticated, service_role; + ALTER function net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) SECURITY DEFINER; + ALTER function net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) SECURITY DEFINER; + ALTER function net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) SET search_path = net; + ALTER function net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) SET search_path = net; + REVOKE ALL ON FUNCTION net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) FROM PUBLIC; + REVOKE ALL ON FUNCTION net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) FROM PUBLIC; + GRANT EXECUTE ON FUNCTION net.http_get(url text, params jsonb, headers jsonb, timeout_milliseconds integer) TO supabase_functions_admin, postgres, anon, authenticated, service_role; + GRANT EXECUTE ON FUNCTION net.http_post(url text, body jsonb, params jsonb, headers jsonb, timeout_milliseconds integer) TO supabase_functions_admin, postgres, anon, authenticated, service_role; + END IF; + END; + $$; + COMMENT ON FUNCTION extensions.grant_pg_net_access IS 'Grants access to pg_net'; + DO $$ BEGIN + IF NOT EXISTS (SELECT 1 FROM pg_event_trigger WHERE evtname = 'issue_pg_net_access') THEN + CREATE EVENT TRIGGER issue_pg_net_access ON ddl_command_end WHEN TAG IN ('CREATE EXTENSION') + EXECUTE PROCEDURE extensions.grant_pg_net_access(); + END IF; + END $$; + INSERT INTO supabase_functions.migrations (version) VALUES ('20210809183423_update_grants'); + ALTER function supabase_functions.http_request() SECURITY DEFINER; + ALTER function supabase_functions.http_request() SET search_path = supabase_functions; + REVOKE ALL ON FUNCTION supabase_functions.http_request() FROM PUBLIC; + GRANT EXECUTE ON FUNCTION supabase_functions.http_request() TO postgres, anon, authenticated, service_role; + COMMIT; diff --git a/supabase/versions/1/configmap-kong.yaml b/supabase/versions/1/configmap-kong.yaml new file mode 100644 index 0000000..cd4dc21 --- /dev/null +++ b/supabase/versions/1/configmap-kong.yaml @@ -0,0 +1,291 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: supabase-kong-config +data: + kong-entrypoint.sh: | + #!/bin/sh + # Build Lua expressions for request-transformer (legacy HS256 key path only). + # $SUPABASE_ANON_KEY and $SUPABASE_SERVICE_KEY are the HS256 JWTs. + export LUA_AUTH_EXPR="\$((headers.authorization ~= nil and headers.authorization:sub(1, 10) ~= 'Bearer sb_' and headers.authorization) or headers.apikey)" + export LUA_RT_WS_EXPR="\$(query_params.apikey)" + + # Substitute environment variables in the Kong declarative config. + awk '{ + result = "" + rest = $0 + while (match(rest, /\$[A-Za-z_][A-Za-z_0-9]*/)) { + varname = substr(rest, RSTART + 1, RLENGTH - 1) + if (varname in ENVIRON) { + result = result substr(rest, 1, RSTART - 1) ENVIRON[varname] + } else { + result = result substr(rest, 1, RSTART + RLENGTH - 1) + } + rest = substr(rest, RSTART + RLENGTH) + } + print result rest + }' /home/kong/temp.yml > "$KONG_DECLARATIVE_CONFIG" + + # Remove empty key-auth credentials (unconfigured opaque keys) + sed -i '/^[[:space:]]*- key:[[:space:]]*$/d' "$KONG_DECLARATIVE_CONFIG" + + exec /entrypoint.sh kong docker-start + + kong.yml: | + _format_version: '2.1' + _transform: true + + consumers: + - username: DASHBOARD + - username: anon + keyauth_credentials: + - key: $SUPABASE_ANON_KEY + - username: service_role + keyauth_credentials: + - key: $SUPABASE_SERVICE_KEY + + acls: + - consumer: anon + group: anon + - consumer: service_role + group: admin + + basicauth_credentials: + - consumer: DASHBOARD + username: '$DASHBOARD_USERNAME' + password: '$DASHBOARD_PASSWORD' + + services: + - name: auth-v1-open + url: http://auth:9999/verify + routes: + - name: auth-v1-open + strip_path: true + paths: + - /auth/v1/verify + plugins: + - name: cors + + - name: auth-v1-open-callback + url: http://auth:9999/callback + routes: + - name: auth-v1-open-callback + strip_path: true + paths: + - /auth/v1/callback + plugins: + - name: cors + + - name: auth-v1-open-authorize + url: http://auth:9999/authorize + routes: + - name: auth-v1-open-authorize + strip_path: true + paths: + - /auth/v1/authorize + plugins: + - name: cors + + - name: auth-v1-open-jwks + url: http://auth:9999/.well-known/jwks.json + routes: + - name: auth-v1-open-jwks + strip_path: true + paths: + - /auth/v1/.well-known/jwks.json + plugins: + - name: cors + + - name: auth-v1 + url: http://auth:9999/ + routes: + - name: auth-v1-all + strip_path: true + paths: + - /auth/v1/ + plugins: + - name: cors + - name: key-auth + config: + hide_credentials: false + - name: request-transformer + config: + add: + headers: + - "Authorization: $LUA_AUTH_EXPR" + replace: + headers: + - "Authorization: $LUA_AUTH_EXPR" + - name: acl + config: + hide_groups_header: true + allow: + - admin + - anon + + - name: rest-v1 + url: http://rest:3000/ + routes: + - name: rest-v1-all + strip_path: true + paths: + - /rest/v1/ + plugins: + - name: cors + - name: key-auth + config: + hide_credentials: false + - name: request-transformer + config: + add: + headers: + - "Authorization: $LUA_AUTH_EXPR" + replace: + headers: + - "Authorization: $LUA_AUTH_EXPR" + - name: acl + config: + hide_groups_header: true + allow: + - admin + - anon + + - name: graphql-v1 + url: http://rest:3000/rpc/graphql + routes: + - name: graphql-v1-all + strip_path: true + paths: + - /graphql/v1 + plugins: + - name: cors + - name: key-auth + config: + hide_credentials: false + - name: request-transformer + config: + add: + headers: + - "Content-Profile: graphql_public" + - "Authorization: $LUA_AUTH_EXPR" + replace: + headers: + - "Authorization: $LUA_AUTH_EXPR" + - name: acl + config: + hide_groups_header: true + allow: + - admin + - anon + + - name: realtime-v1-ws + url: http://realtime-dev:4000/socket + protocol: ws + routes: + - name: realtime-v1-ws + strip_path: true + paths: + - /realtime/v1/ + plugins: + - name: cors + - name: key-auth + config: + hide_credentials: false + - name: request-transformer + config: + add: + headers: + - "x-api-key:$LUA_RT_WS_EXPR" + replace: + querystring: + - "apikey:$LUA_RT_WS_EXPR" + - name: acl + config: + hide_groups_header: true + allow: + - admin + - anon + + - name: realtime-v1-rest + url: http://realtime-dev:4000/api + protocol: http + routes: + - name: realtime-v1-rest + strip_path: true + paths: + - /realtime/v1/api + plugins: + - name: cors + - name: key-auth + config: + hide_credentials: false + - name: request-transformer + config: + add: + headers: + - "Authorization: $LUA_AUTH_EXPR" + replace: + headers: + - "Authorization: $LUA_AUTH_EXPR" + - name: acl + config: + hide_groups_header: true + allow: + - admin + - anon + + - name: storage-v1 + url: http://storage:5000/ + routes: + - name: storage-v1-all + strip_path: true + paths: + - /storage/v1/ + plugins: + - name: cors + - name: request-transformer + config: + add: + headers: + - "Authorization: $LUA_AUTH_EXPR" + replace: + headers: + - "Authorization: $LUA_AUTH_EXPR" + - name: post-function + config: + access: + - | + local auth = kong.request.get_header("authorization") + if auth == nil or auth == "" or auth:find("^%s*$") then + kong.service.request.clear_header("authorization") + end + + - name: meta + url: http://meta:8080/ + routes: + - name: meta-all + strip_path: true + paths: + - /pg/ + plugins: + - name: key-auth + config: + hide_credentials: false + - name: acl + config: + hide_groups_header: true + allow: + - admin + + - name: dashboard + url: http://studio:3000/ + routes: + - name: dashboard-all + strip_path: true + paths: + - / + plugins: + - name: cors + - name: basic-auth + config: + hide_credentials: true diff --git a/supabase/versions/1/deployment-auth.yaml b/supabase/versions/1/deployment-auth.yaml new file mode 100644 index 0000000..f255b91 --- /dev/null +++ b/supabase/versions/1/deployment-auth.yaml @@ -0,0 +1,105 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: supabase-auth +spec: + replicas: 1 + selector: + matchLabels: + component: auth + template: + metadata: + labels: + component: auth + spec: + containers: + - name: auth + image: supabase/gotrue:v2.189.0 + ports: + - containerPort: 9999 + env: + - name: GOTRUE_API_HOST + value: "0.0.0.0" + - name: GOTRUE_API_PORT + value: "9999" + - name: API_EXTERNAL_URL + value: "https://{{ .domain }}" + - name: GOTRUE_DB_DRIVER + value: postgres + - name: GOTRUE_DB_DATABASE_URL + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dbUrlAuth + - name: GOTRUE_SITE_URL + value: "{{ .siteUrl }}" + - name: GOTRUE_URI_ALLOW_LIST + value: "" + - name: GOTRUE_DISABLE_SIGNUP + value: "false" + - name: GOTRUE_JWT_ADMIN_ROLES + value: service_role + - name: GOTRUE_JWT_AUD + value: authenticated + - name: GOTRUE_JWT_DEFAULT_GROUP_NAME + value: authenticated + - name: GOTRUE_JWT_EXP + value: "{{ .jwtExpiry }}" + - name: GOTRUE_JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: GOTRUE_JWT_ISSUER + value: "https://{{ .domain }}/auth/v1" + - name: GOTRUE_EXTERNAL_EMAIL_ENABLED + value: "true" + - name: GOTRUE_EXTERNAL_ANONYMOUS_USERS_ENABLED + value: "false" + - name: GOTRUE_MAILER_AUTOCONFIRM + value: "false" + - name: GOTRUE_SMTP_ADMIN_EMAIL + value: "{{ .smtp.from }}" + - name: GOTRUE_SMTP_HOST + value: "{{ .smtp.host }}" + - name: GOTRUE_SMTP_PORT + value: "{{ .smtp.port }}" + - name: GOTRUE_SMTP_USER + value: "{{ .smtp.user }}" + - name: GOTRUE_SMTP_PASS + valueFrom: + secretKeyRef: + name: supabase-secrets + key: smtp.password + - name: GOTRUE_SMTP_SENDER_NAME + value: "{{ .smtp.from }}" + - name: GOTRUE_MAILER_URLPATHS_INVITE + value: "/auth/v1/verify" + - name: GOTRUE_MAILER_URLPATHS_CONFIRMATION + value: "/auth/v1/verify" + - name: GOTRUE_MAILER_URLPATHS_RECOVERY + value: "/auth/v1/verify" + - name: GOTRUE_MAILER_URLPATHS_EMAIL_CHANGE + value: "/auth/v1/verify" + - name: GOTRUE_EXTERNAL_PHONE_ENABLED + value: "false" + livenessProbe: + httpGet: + path: /health + port: 9999 + initialDelaySeconds: 15 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + readinessProbe: + httpGet: + path: /health + port: 9999 + initialDelaySeconds: 10 + periodSeconds: 5 + timeoutSeconds: 5 + failureThreshold: 3 + resources: + requests: + memory: 128Mi + cpu: 50m diff --git a/supabase/versions/1/deployment-kong.yaml b/supabase/versions/1/deployment-kong.yaml new file mode 100644 index 0000000..5816bcd --- /dev/null +++ b/supabase/versions/1/deployment-kong.yaml @@ -0,0 +1,90 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: supabase-kong +spec: + replicas: 1 + selector: + matchLabels: + component: kong + template: + metadata: + labels: + component: kong + spec: + containers: + - name: kong + image: kong/kong:3.4.2 + command: ["/bin/sh", "/home/kong/kong-entrypoint.sh"] + ports: + - containerPort: 8000 + - containerPort: 8443 + env: + - name: LD_LIBRARY_PATH + value: "/usr/local/kong/lib" + - name: KONG_DATABASE + value: "off" + - name: KONG_DECLARATIVE_CONFIG + value: /usr/local/kong/kong.yml + - name: KONG_DNS_ORDER + value: LAST,A,CNAME + - name: KONG_DNS_NOT_FOUND_TTL + value: "1" + - name: KONG_PLUGINS + value: request-transformer,cors,key-auth,acl,basic-auth,request-termination,ip-restriction,post-function + - name: KONG_NGINX_PROXY_PROXY_BUFFER_SIZE + value: 160k + - name: KONG_NGINX_PROXY_PROXY_BUFFERS + value: 64 160k + - name: KONG_PROXY_ACCESS_LOG + value: /dev/stdout combined + - name: SUPABASE_ANON_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: anonKey + - name: SUPABASE_SERVICE_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: serviceRoleKey + - name: DASHBOARD_USERNAME + value: "{{ .dashboardUsername }}" + - name: DASHBOARD_PASSWORD + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dashboardPassword + volumeMounts: + - name: kong-config + mountPath: /home/kong/temp.yml + subPath: kong.yml + - name: kong-config + mountPath: /home/kong/kong-entrypoint.sh + subPath: kong-entrypoint.sh + - name: kong-output + mountPath: /usr/local/kong + livenessProbe: + exec: + command: ["kong", "health"] + initialDelaySeconds: 15 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + readinessProbe: + exec: + command: ["kong", "health"] + initialDelaySeconds: 10 + periodSeconds: 5 + timeoutSeconds: 5 + failureThreshold: 3 + resources: + requests: + memory: 256Mi + cpu: 100m + volumes: + - name: kong-config + configMap: + name: supabase-kong-config + - name: kong-output + emptyDir: {} diff --git a/supabase/versions/1/deployment-meta.yaml b/supabase/versions/1/deployment-meta.yaml new file mode 100644 index 0000000..69fdb9d --- /dev/null +++ b/supabase/versions/1/deployment-meta.yaml @@ -0,0 +1,60 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: supabase-meta +spec: + replicas: 1 + selector: + matchLabels: + component: meta + template: + metadata: + labels: + component: meta + spec: + containers: + - name: meta + image: supabase/postgres-meta:v0.96.6 + ports: + - containerPort: 8080 + env: + - name: PG_META_PORT + value: "8080" + - name: PG_META_DB_HOST + value: "{{ .db.host }}" + - name: PG_META_DB_PORT + value: "5432" + - name: PG_META_DB_NAME + value: "{{ .db.name }}" + - name: PG_META_DB_USER + value: "{{ .db.user }}" + - name: PG_META_DB_PASSWORD + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dbPassword + - name: CRYPTO_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: pgMetaCryptoKey + livenessProbe: + httpGet: + path: /health + port: 8080 + initialDelaySeconds: 10 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + readinessProbe: + httpGet: + path: /health + port: 8080 + initialDelaySeconds: 5 + periodSeconds: 5 + timeoutSeconds: 5 + failureThreshold: 3 + resources: + requests: + memory: 128Mi + cpu: 50m diff --git a/supabase/versions/1/deployment-realtime.yaml b/supabase/versions/1/deployment-realtime.yaml new file mode 100644 index 0000000..75b8f05 --- /dev/null +++ b/supabase/versions/1/deployment-realtime.yaml @@ -0,0 +1,78 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: supabase-realtime +spec: + replicas: 1 + selector: + matchLabels: + component: realtime + template: + metadata: + labels: + component: realtime + spec: + # hostname must be realtime-dev so Realtime seeds tenant as "realtime-dev" + # (Realtime parses the subdomain part of its hostname for the tenant id) + hostname: realtime-dev + containers: + - name: realtime + image: supabase/realtime:v2.102.3 + ports: + - containerPort: 4000 + env: + - name: PORT + value: "4000" + - name: DB_HOST + value: "{{ .db.host }}" + - name: DB_PORT + value: "5432" + - name: DB_USER + value: supabase_admin + - name: DB_PASSWORD + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dbPassword + - name: DB_NAME + value: "{{ .db.name }}" + - name: DB_AFTER_CONNECT_QUERY + value: "SET search_path TO _realtime" + - name: DB_ENC_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: realtimeEncKey + - name: API_JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: SECRET_KEY_BASE + valueFrom: + secretKeyRef: + name: supabase-secrets + key: secretKeyBase + - name: METRICS_JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: ERL_AFLAGS + value: "-proto_dist inet_tcp" + - name: DNS_NODES + value: "''" + - name: RLIMIT_NOFILE + value: "10000" + - name: APP_NAME + value: realtime + - name: SEED_SELF_HOST + value: "true" + - name: RUN_JANITOR + value: "true" + - name: DISABLE_HEALTHCHECK_LOGGING + value: "true" + resources: + requests: + memory: 256Mi + cpu: 100m diff --git a/supabase/versions/1/deployment-rest.yaml b/supabase/versions/1/deployment-rest.yaml new file mode 100644 index 0000000..b3abd6e --- /dev/null +++ b/supabase/versions/1/deployment-rest.yaml @@ -0,0 +1,70 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: supabase-rest +spec: + replicas: 1 + selector: + matchLabels: + component: rest + template: + metadata: + labels: + component: rest + spec: + containers: + - name: rest + image: postgrest/postgrest:v14.12 + command: ["postgrest"] + ports: + - containerPort: 3000 + env: + - name: PGRST_DB_URI + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dbUrlRest + - name: PGRST_DB_SCHEMAS + value: "public,storage,graphql_public" + - name: PGRST_DB_MAX_ROWS + value: "1000" + - name: PGRST_DB_EXTRA_SEARCH_PATH + value: "public" + - name: PGRST_DB_ANON_ROLE + value: anon + - name: PGRST_ADMIN_SERVER_PORT + value: "3001" + - name: PGRST_ADMIN_SERVER_HOST + value: localhost + - name: PGRST_JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: PGRST_DB_USE_LEGACY_GUCS + value: "false" + - name: PGRST_APP_SETTINGS_JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: PGRST_APP_SETTINGS_JWT_EXP + value: "{{ .jwtExpiry }}" + livenessProbe: + exec: + command: ["postgrest", "--ready"] + initialDelaySeconds: 15 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + readinessProbe: + exec: + command: ["postgrest", "--ready"] + initialDelaySeconds: 10 + periodSeconds: 5 + timeoutSeconds: 5 + failureThreshold: 3 + resources: + requests: + memory: 128Mi + cpu: 50m diff --git a/supabase/versions/1/deployment-storage.yaml b/supabase/versions/1/deployment-storage.yaml new file mode 100644 index 0000000..43c9e83 --- /dev/null +++ b/supabase/versions/1/deployment-storage.yaml @@ -0,0 +1,122 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: supabase-storage +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + component: storage + template: + metadata: + labels: + component: storage + spec: + containers: + - name: storage + image: supabase/storage-api:v1.60.4 + ports: + - containerPort: 5000 + env: + - name: ANON_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: anonKey + - name: SERVICE_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: serviceRoleKey + - name: POSTGREST_URL + value: "http://rest:3000" + - name: AUTH_JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: DATABASE_URL + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dbUrlStorage + - name: STORAGE_PUBLIC_URL + value: "https://{{ .domain }}" + - name: REQUEST_ALLOW_X_FORWARDED_PATH + value: "true" + - name: FILE_SIZE_LIMIT + value: "52428800" + - name: STORAGE_BACKEND + value: file + - name: GLOBAL_S3_BUCKET + value: stub + - name: REGION + value: stub + - name: FILE_STORAGE_BACKEND_PATH + value: /var/lib/storage + - name: TENANT_ID + value: stub + - name: ENABLE_IMAGE_TRANSFORMATION + value: "true" + - name: IMGPROXY_URL + value: "http://localhost:5001" + livenessProbe: + httpGet: + path: /status + port: 5000 + initialDelaySeconds: 20 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + readinessProbe: + httpGet: + path: /status + port: 5000 + initialDelaySeconds: 10 + periodSeconds: 5 + timeoutSeconds: 5 + failureThreshold: 3 + volumeMounts: + - name: supabase-storage + mountPath: /var/lib/storage + resources: + requests: + memory: 256Mi + cpu: 100m + + - name: imgproxy + image: darthsim/imgproxy:v3.30.1 + ports: + - containerPort: 5001 + env: + - name: IMGPROXY_BIND + value: ":5001" + - name: IMGPROXY_LOCAL_FILESYSTEM_ROOT + value: / + - name: IMGPROXY_USE_ETAG + value: "true" + - name: IMGPROXY_AUTO_WEBP + value: "true" + - name: IMGPROXY_MAX_SRC_RESOLUTION + value: "16.8" + volumeMounts: + - name: supabase-storage + mountPath: /var/lib/storage + livenessProbe: + exec: + command: ["imgproxy", "health"] + initialDelaySeconds: 10 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 3 + resources: + requests: + memory: 128Mi + cpu: 50m + + volumes: + - name: supabase-storage + persistentVolumeClaim: + claimName: supabase-storage diff --git a/supabase/versions/1/deployment-studio.yaml b/supabase/versions/1/deployment-studio.yaml new file mode 100644 index 0000000..1496fdf --- /dev/null +++ b/supabase/versions/1/deployment-studio.yaml @@ -0,0 +1,95 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: supabase-studio +spec: + replicas: 1 + selector: + matchLabels: + component: studio + template: + metadata: + labels: + component: studio + spec: + containers: + - name: studio + image: supabase/studio:2026.06.03-sha-0bca601 + ports: + - containerPort: 3000 + env: + - name: HOSTNAME + value: "0.0.0.0" + - name: STUDIO_PG_META_URL + value: "http://meta:8080" + - name: POSTGRES_PORT + value: "5432" + - name: POSTGRES_HOST + value: "{{ .db.host }}" + - name: POSTGRES_DB + value: "{{ .db.name }}" + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dbPassword + - name: POSTGRES_USER_READ_WRITE + value: "{{ .db.user }}" + - name: PG_META_CRYPTO_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: pgMetaCryptoKey + - name: PGRST_DB_SCHEMAS + value: "public,storage,graphql_public" + - name: PGRST_DB_MAX_ROWS + value: "1000" + - name: PGRST_DB_EXTRA_SEARCH_PATH + value: "public" + - name: DEFAULT_ORGANIZATION_NAME + value: "{{ .defaultOrg }}" + - name: DEFAULT_PROJECT_NAME + value: "{{ .defaultProject }}" + - name: SUPABASE_URL + value: "http://kong:8000" + - name: SUPABASE_PUBLIC_URL + value: "https://{{ .domain }}" + - name: SUPABASE_ANON_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: anonKey + - name: SUPABASE_SERVICE_KEY + valueFrom: + secretKeyRef: + name: supabase-secrets + key: serviceRoleKey + - name: AUTH_JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: ENABLED_FEATURES_LOGS_ALL + value: "false" + livenessProbe: + exec: + command: + - node + - -e + - "fetch('http://localhost:3000/api/platform/profile').then((r) => {if (r.status !== 200) throw new Error(r.status)})" + initialDelaySeconds: 30 + periodSeconds: 15 + timeoutSeconds: 10 + failureThreshold: 3 + readinessProbe: + httpGet: + path: /api/platform/profile + port: 3000 + initialDelaySeconds: 20 + periodSeconds: 10 + timeoutSeconds: 10 + failureThreshold: 3 + resources: + requests: + memory: 512Mi + cpu: 200m diff --git a/supabase/versions/1/ingress.yaml b/supabase/versions/1/ingress.yaml new file mode 100644 index 0000000..63aafc2 --- /dev/null +++ b/supabase/versions/1/ingress.yaml @@ -0,0 +1,25 @@ +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: supabase + annotations: + external-dns.alpha.kubernetes.io/target: "{{ .externalDnsDomain }}" + external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" + external-dns.alpha.kubernetes.io/ttl: "60" +spec: + ingressClassName: traefik + rules: + - host: "{{ .domain }}" + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: kong + port: + number: 8000 + tls: + - hosts: + - "{{ .domain }}" + secretName: "{{ .tlsSecretName }}" diff --git a/supabase/versions/1/kustomization.yaml b/supabase/versions/1/kustomization.yaml new file mode 100644 index 0000000..eec5a4b --- /dev/null +++ b/supabase/versions/1/kustomization.yaml @@ -0,0 +1,24 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: supabase +labels: + - includeSelectors: true + pairs: + app: supabase + managedBy: kustomize + partOf: wild-cloud +resources: + - namespace.yaml + - pvc.yaml + - services.yaml + - ingress.yaml + - configmap-db-init.yaml + - configmap-kong.yaml + - statefulset-db.yaml + - deployment-kong.yaml + - deployment-auth.yaml + - deployment-rest.yaml + - deployment-realtime.yaml + - deployment-storage.yaml + - deployment-meta.yaml + - deployment-studio.yaml diff --git a/supabase/versions/1/manifest.yaml b/supabase/versions/1/manifest.yaml new file mode 100644 index 0000000..56a327f --- /dev/null +++ b/supabase/versions/1/manifest.yaml @@ -0,0 +1,56 @@ +version: 2026.07.02 +requires: + - name: smtp +defaultConfig: + namespace: supabase + domain: supabase.{{ .cloud.domain }} + externalDnsDomain: "{{ .cloud.domain }}" + tlsSecretName: wildcard-wild-cloud-tls + dbStorage: 10Gi + fileStorage: 5Gi + db: + host: db + name: postgres + user: supabase_admin + jwtExpiry: "3600" + siteUrl: "https://supabase.{{ .cloud.domain }}" + defaultOrg: "Default Organization" + defaultProject: "Default Project" + dashboardUsername: supabase + smtp: + host: "{{ .apps.smtp.host }}" + port: "{{ .apps.smtp.port }}" + user: "{{ .apps.smtp.user }}" + from: "{{ .apps.smtp.from }}" +defaultSecrets: + - key: dbPassword + - key: dashboardPassword + - key: jwtSecret + default: "your-super-secret-jwt-token-with-at-least-32-characters-long" + - key: anonKey + default: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyAgCiAgICAicm9sZSI6ICJhbm9uIiwKICAgICJpc3MiOiAic3VwYWJhc2UtZGVtbyIsCiAgICAiaWF0IjogMTY0MTc2OTIwMCwKICAgICJleHAiOiAxNzk5NTM1NjAwCn0.dc_X5iR_VP_qT0zsiyj_I_OZ2T9FtRU2BBNWN8Bu4GE" + - key: serviceRoleKey + default: "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyAgCiAgICAicm9sZSI6ICJzZXJ2aWNlX3JvbGUiLAogICAgImlzcyI6ICJzdXBhYmFzZS1kZW1vIiwKICAgICJpYXQiOiAxNjQxNzY5MjAwLAogICAgImV4cCI6IDE3OTk1MzU2MDAKfQ.DaYlNEoUrrEn2Ig7tqibS-PHK5vgusbcbo7X36XVt4Q" + - key: secretKeyBase + default: "UpNVntn3cDxHJpq99YMc1T1AQgQpc8kfYTuRgBiYa15BLrx8etQoXz3gZv1/u2oq" + - key: realtimeEncKey + default: "supabaserealtime" + - key: vaultEncKey + default: '{{ crypto.SHA256 .secrets.jwtSecret }}' + - key: pgMetaCryptoKey + default: '{{ crypto.SHA256 .secrets.dbPassword }}' + - key: dbUrlAuth + default: "postgres://supabase_auth_admin:{{ .secrets.dbPassword }}@{{ .app.db.host }}:5432/{{ .app.db.name }}?sslmode=disable" + - key: dbUrlStorage + default: "postgres://supabase_storage_admin:{{ .secrets.dbPassword }}@{{ .app.db.host }}:5432/{{ .app.db.name }}?sslmode=disable" + - key: dbUrlRest + default: "postgres://authenticator:{{ .secrets.dbPassword }}@{{ .app.db.host }}:5432/{{ .app.db.name }}?sslmode=disable" +requiredSecrets: + - smtp.password +scripts: + - name: generate-keys + path: scripts/generate-keys.sh + description: "Generate new JWT secret and API keys. Run this before going to production to replace the default example keys." + params: + - name: JWT_SECRET + description: "Optional: provide your own JWT secret (min 32 chars). Leave blank to auto-generate." diff --git a/supabase/versions/1/namespace.yaml b/supabase/versions/1/namespace.yaml new file mode 100644 index 0000000..b7816b8 --- /dev/null +++ b/supabase/versions/1/namespace.yaml @@ -0,0 +1,4 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: "{{ .namespace }}" diff --git a/supabase/versions/1/pvc.yaml b/supabase/versions/1/pvc.yaml new file mode 100644 index 0000000..4459933 --- /dev/null +++ b/supabase/versions/1/pvc.yaml @@ -0,0 +1,21 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: supabase-db +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: "{{ .dbStorage }}" +--- +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: supabase-storage +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: "{{ .fileStorage }}" diff --git a/supabase/versions/1/scripts/generate-keys.sh b/supabase/versions/1/scripts/generate-keys.sh new file mode 100644 index 0000000..7997b23 --- /dev/null +++ b/supabase/versions/1/scripts/generate-keys.sh @@ -0,0 +1,80 @@ +#!/bin/bash +# generate-keys.sh — Generate a new JWT secret and matching Supabase API keys. +# +# This script generates a cryptographically secure JWT_SECRET and derives +# ANON_KEY and SERVICE_ROLE_KEY JWTs from it. Run this script and update +# the supabase secrets in your Wild Cloud instance before going to production. +# +# Usage: +# wild app run supabase generate-keys [JWT_SECRET=] +# +# The script uses kubectl exec to run inside the cluster. Requires: +# - python3 with hmac, hashlib, base64, json modules (standard library) +# - The supabase namespace to exist +# +# After running, update secrets.yaml in your instance with the output values. + +set -e + +NAMESPACE="supabase" +INSTANCE="${WILD_INSTANCE:-$(wild instance current 2>/dev/null || echo 'test-cloud')}" + +# Generate or use provided JWT secret +if [ -n "$JWT_SECRET" ]; then + SECRET="$JWT_SECRET" +else + SECRET=$(python3 -c "import secrets; print(secrets.token_urlsafe(48))") +fi + +# Generate HS256 JWT tokens using python3 (no external deps needed) +GENERATE_JWT=$(cat <<'PYEOF' +import sys, hmac, hashlib, base64, json, time + +def b64url_encode(data): + if isinstance(data, str): + data = data.encode() + return base64.urlsafe_b64encode(data).rstrip(b'=').decode() + +def make_jwt(secret, role): + header = b64url_encode(json.dumps({"alg": "HS256", "typ": "JWT"}, separators=(',', ':'))) + # Tokens valid for 10 years from now + now = int(time.time()) + exp = now + (10 * 365 * 24 * 3600) + payload = b64url_encode(json.dumps({ + "role": role, + "iss": "supabase", + "iat": now, + "exp": exp + }, separators=(',', ':'))) + signing_input = f"{header}.{payload}" + sig = hmac.new(secret.encode(), signing_input.encode(), hashlib.sha256).digest() + signature = b64url_encode(sig) + return f"{signing_input}.{signature}" + +secret = sys.argv[1] +role = sys.argv[2] +print(make_jwt(secret, role)) +PYEOF +) + +ANON_KEY=$(python3 -c "$GENERATE_JWT" "$SECRET" "anon") +SERVICE_ROLE_KEY=$(python3 -c "$GENERATE_JWT" "$SECRET" "service_role") + +echo "" +echo "=== Supabase JWT Keys ===" +echo "" +echo "Add these to your secrets.yaml (or update via 'wild secrets set'):" +echo "" +echo " jwtSecret: |" +echo " $SECRET" +echo "" +echo " anonKey: |" +echo " $ANON_KEY" +echo "" +echo " serviceRoleKey: |" +echo " $SERVICE_ROLE_KEY" +echo "" +echo "IMPORTANT: After updating secrets.yaml, redeploy Supabase:" +echo " wild app deploy supabase" +echo "" +echo "The existing database will have its JWT secret updated on next pod restart." diff --git a/supabase/versions/1/services.yaml b/supabase/versions/1/services.yaml new file mode 100644 index 0000000..1ab0bd1 --- /dev/null +++ b/supabase/versions/1/services.yaml @@ -0,0 +1,87 @@ +apiVersion: v1 +kind: Service +metadata: + name: db +spec: + selector: + component: db + ports: + - port: 5432 + targetPort: 5432 +--- +apiVersion: v1 +kind: Service +metadata: + name: kong +spec: + selector: + component: kong + ports: + - port: 8000 + targetPort: 8000 +--- +apiVersion: v1 +kind: Service +metadata: + name: auth +spec: + selector: + component: auth + ports: + - port: 9999 + targetPort: 9999 +--- +apiVersion: v1 +kind: Service +metadata: + name: rest +spec: + selector: + component: rest + ports: + - port: 3000 + targetPort: 3000 +--- +apiVersion: v1 +kind: Service +metadata: + name: realtime-dev +spec: + selector: + component: realtime + ports: + - port: 4000 + targetPort: 4000 +--- +apiVersion: v1 +kind: Service +metadata: + name: storage +spec: + selector: + component: storage + ports: + - port: 5000 + targetPort: 5000 +--- +apiVersion: v1 +kind: Service +metadata: + name: meta +spec: + selector: + component: meta + ports: + - port: 8080 + targetPort: 8080 +--- +apiVersion: v1 +kind: Service +metadata: + name: studio +spec: + selector: + component: studio + ports: + - port: 3000 + targetPort: 3000 diff --git a/supabase/versions/1/statefulset-db.yaml b/supabase/versions/1/statefulset-db.yaml new file mode 100644 index 0000000..2791016 --- /dev/null +++ b/supabase/versions/1/statefulset-db.yaml @@ -0,0 +1,89 @@ +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: supabase-db +spec: + serviceName: db + replicas: 1 + selector: + matchLabels: + component: db + template: + metadata: + labels: + component: db + spec: + securityContext: + fsGroup: 999 + containers: + - name: db + image: supabase/postgres:17.6.1.136 + ports: + - containerPort: 5432 + env: + - name: POSTGRES_PASSWORD + valueFrom: + secretKeyRef: + name: supabase-secrets + key: dbPassword + - name: POSTGRES_DB + value: "{{ .db.name }}" + - name: POSTGRES_USER + value: "{{ .db.user }}" + - name: JWT_SECRET + valueFrom: + secretKeyRef: + name: supabase-secrets + key: jwtSecret + - name: JWT_EXP + value: "{{ .jwtExpiry }}" + volumeMounts: + - name: supabase-db + mountPath: /var/lib/postgresql + - name: db-init + mountPath: /docker-entrypoint-initdb.d/init-scripts/99-jwt.sql + subPath: jwt.sql + - name: db-init + mountPath: /docker-entrypoint-initdb.d/init-scripts/99-roles.sql + subPath: roles.sql + - name: db-init + mountPath: /docker-entrypoint-initdb.d/migrations/99-realtime.sql + subPath: realtime.sql + - name: db-init + mountPath: /docker-entrypoint-initdb.d/init-scripts/98-webhooks.sql + subPath: webhooks.sql + livenessProbe: + exec: + command: + - pg_isready + - -U + - "{{ .db.user }}" + - -d + - "{{ .db.name }}" + initialDelaySeconds: 30 + periodSeconds: 10 + timeoutSeconds: 5 + failureThreshold: 6 + readinessProbe: + exec: + command: + - pg_isready + - -U + - "{{ .db.user }}" + - -d + - "{{ .db.name }}" + initialDelaySeconds: 10 + periodSeconds: 5 + timeoutSeconds: 5 + failureThreshold: 3 + resources: + requests: + memory: 256Mi + cpu: 100m + volumes: + - name: supabase-db + persistentVolumeClaim: + claimName: supabase-db + - name: db-init + configMap: + name: supabase-db-init diff --git a/synapse/versions/v1/ingress.yaml b/synapse/versions/v1/ingress.yaml index 21e24cf..53d3db9 100644 --- a/synapse/versions/v1/ingress.yaml +++ b/synapse/versions/v1/ingress.yaml @@ -9,6 +9,7 @@ metadata: external-dns.alpha.kubernetes.io/target: {{ .externalDnsDomain }} external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" spec: + ingressClassName: traefik tls: - hosts: - {{ .domain }} @@ -35,6 +36,7 @@ metadata: external-dns.alpha.kubernetes.io/target: {{ .externalDnsDomain }} external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" spec: + ingressClassName: traefik tls: - hosts: - {{ .serverName }} diff --git a/synapse/versions/v1/manifest.yaml b/synapse/versions/v1/manifest.yaml index 7941f84..4031375 100644 --- a/synapse/versions/v1/manifest.yaml +++ b/synapse/versions/v1/manifest.yaml @@ -1,4 +1,4 @@ -version: v1.144.0-9 +version: v1.144.0-10 requires: - name: postgres - name: redis diff --git a/syncthing-discovery/app.yaml b/syncthing-discovery/app.yaml new file mode 100644 index 0000000..5ecd9f3 --- /dev/null +++ b/syncthing-discovery/app.yaml @@ -0,0 +1,6 @@ +name: syncthing-discovery +is: syncthing-discovery +description: Syncthing Discovery Server (stdiscosrv) lets Syncthing clients find each other on the internet by announcing and querying device addresses. +category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/syncthing.svg +latest: "1" diff --git a/syncthing-discovery/versions/1/deployment.yaml b/syncthing-discovery/versions/1/deployment.yaml new file mode 100644 index 0000000..5bb576d --- /dev/null +++ b/syncthing-discovery/versions/1/deployment.yaml @@ -0,0 +1,62 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: syncthing-discovery + namespace: {{ .namespace }} +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + component: server + template: + metadata: + labels: + component: server + spec: + securityContext: + runAsNonRoot: false + runAsUser: 0 + seccompProfile: + type: RuntimeDefault + containers: + - name: discosrv + image: syncthing/discosrv:2.1.1 + args: + - --listen=:8080 + - --http + - --db-dir=/var/stdiscosrv + ports: + - name: http + containerPort: 8080 + protocol: TCP + resources: + limits: + cpu: 500m + memory: 128Mi + requests: + cpu: 25m + memory: 32Mi + volumeMounts: + - name: discosrv-data + mountPath: /var/stdiscosrv + livenessProbe: + tcpSocket: + port: 8080 + initialDelaySeconds: 10 + periodSeconds: 30 + failureThreshold: 3 + readinessProbe: + tcpSocket: + port: 8080 + initialDelaySeconds: 5 + periodSeconds: 10 + failureThreshold: 3 + securityContext: + readOnlyRootFilesystem: false + volumes: + - name: discosrv-data + persistentVolumeClaim: + claimName: syncthing-discovery-data + restartPolicy: Always diff --git a/syncthing-discovery/versions/1/ingress.yaml b/syncthing-discovery/versions/1/ingress.yaml new file mode 100644 index 0000000..5be965c --- /dev/null +++ b/syncthing-discovery/versions/1/ingress.yaml @@ -0,0 +1,26 @@ +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: syncthing-discovery + namespace: {{ .namespace }} + annotations: + external-dns.alpha.kubernetes.io/target: {{ .externalDnsDomain }} + external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" + external-dns.alpha.kubernetes.io/ttl: "60" +spec: + ingressClassName: traefik + rules: + - host: {{ .domain }} + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: syncthing-discovery + port: + number: 80 + tls: + - hosts: + - {{ .domain }} + secretName: {{ .tlsSecretName }} diff --git a/syncthing-discovery/versions/1/kustomization.yaml b/syncthing-discovery/versions/1/kustomization.yaml new file mode 100644 index 0000000..20b2951 --- /dev/null +++ b/syncthing-discovery/versions/1/kustomization.yaml @@ -0,0 +1,15 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: syncthing-discovery +labels: + - includeSelectors: true + pairs: + app: syncthing-discovery + managedBy: kustomize + partOf: wild-cloud +resources: + - namespace.yaml + - pvc.yaml + - deployment.yaml + - service.yaml + - ingress.yaml diff --git a/syncthing-discovery/versions/1/manifest.yaml b/syncthing-discovery/versions/1/manifest.yaml new file mode 100644 index 0000000..d8146ec --- /dev/null +++ b/syncthing-discovery/versions/1/manifest.yaml @@ -0,0 +1,7 @@ +version: 2.1.1 +defaultConfig: + namespace: syncthing-discovery + domain: discosrv.{{ .cloud.domain }} + externalDnsDomain: "{{ .cloud.domain }}" + tlsSecretName: wildcard-wild-cloud-tls + storage: 1Gi diff --git a/syncthing-discovery/versions/1/namespace.yaml b/syncthing-discovery/versions/1/namespace.yaml new file mode 100644 index 0000000..b7816b8 --- /dev/null +++ b/syncthing-discovery/versions/1/namespace.yaml @@ -0,0 +1,4 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: "{{ .namespace }}" diff --git a/syncthing-discovery/versions/1/pvc.yaml b/syncthing-discovery/versions/1/pvc.yaml new file mode 100644 index 0000000..fa86210 --- /dev/null +++ b/syncthing-discovery/versions/1/pvc.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: syncthing-discovery-data + namespace: {{ .namespace }} +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: {{ .storage }} diff --git a/syncthing-discovery/versions/1/service.yaml b/syncthing-discovery/versions/1/service.yaml new file mode 100644 index 0000000..7effb3e --- /dev/null +++ b/syncthing-discovery/versions/1/service.yaml @@ -0,0 +1,13 @@ +apiVersion: v1 +kind: Service +metadata: + name: syncthing-discovery + namespace: {{ .namespace }} +spec: + selector: + component: server + ports: + - name: http + port: 80 + targetPort: 8080 + protocol: TCP diff --git a/syncthing-relay/README.md b/syncthing-relay/README.md new file mode 100644 index 0000000..0629014 --- /dev/null +++ b/syncthing-relay/README.md @@ -0,0 +1,35 @@ +# Syncthing Relay Server + +strelaysrv relays encrypted Syncthing traffic between devices that cannot connect directly (e.g. both behind NAT). + +## Bandwidth warning + +Unlike the discovery server, the relay **proxies all sync traffic**. Every byte synced between clients who cannot connect directly flows through your Wild Central device and your internet connection. On a home or community internet connection, heavy use by multiple members syncing large files can saturate your upstream bandwidth. + +Deploy this only if you know your members need relay support and your connection can handle it. If members can connect directly, the relay is never used. + +## Network setup + +Port **22067** (TCP) must be forwarded on your router to the relay's LoadBalancer IP. Check the assigned IP with: + +```bash +kubectl get svc -n syncthing-relay syncthing-relay +``` + +Set `externalAddress` in your app config to `your-domain-or-ip:22067` so the relay advertises the correct public address. + +## Privacy + +This relay is configured as private (not registered with the Syncthing global relay pool). Only Syncthing clients that explicitly add your relay URL will use it. The relay operator can see which device IDs are connecting and data volumes, but not file contents — traffic is end-to-end encrypted between Syncthing clients. + +## Syncthing client configuration + +Add the relay URI in Syncthing settings under **Settings → Connections → Relays**: + +``` +relay://your-domain:22067 +``` + +## Status + +The relay status page is available at `https://{{ domain }}/status`. diff --git a/syncthing-relay/app.yaml b/syncthing-relay/app.yaml new file mode 100644 index 0000000..2fd9572 --- /dev/null +++ b/syncthing-relay/app.yaml @@ -0,0 +1,6 @@ +name: syncthing-relay +is: syncthing-relay +description: Syncthing Relay Server (strelaysrv) relays encrypted Syncthing traffic between devices that cannot connect directly due to NAT or firewall restrictions. +category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/syncthing.svg +latest: "1" diff --git a/syncthing-relay/versions/1/deployment.yaml b/syncthing-relay/versions/1/deployment.yaml new file mode 100644 index 0000000..e98b0b4 --- /dev/null +++ b/syncthing-relay/versions/1/deployment.yaml @@ -0,0 +1,69 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: syncthing-relay + namespace: {{ .namespace }} +spec: + replicas: 1 + strategy: + type: Recreate + selector: + matchLabels: + component: server + template: + metadata: + labels: + component: server + spec: + securityContext: + runAsNonRoot: false + runAsUser: 0 + seccompProfile: + type: RuntimeDefault + containers: + - name: strelaysrv + image: syncthing/relaysrv:2.1.1 + args: + - --listen=:22067 + - --status-srv=:22068 + - --keys=/var/strelaysrv + - --pools= + - --ext-address={{ .externalAddress }} + ports: + - name: relay + containerPort: 22067 + protocol: TCP + - name: status + containerPort: 22068 + protocol: TCP + resources: + limits: + cpu: 500m + memory: 128Mi + requests: + cpu: 25m + memory: 32Mi + volumeMounts: + - name: relay-data + mountPath: /var/strelaysrv + livenessProbe: + httpGet: + path: /status + port: 22068 + initialDelaySeconds: 10 + periodSeconds: 30 + failureThreshold: 3 + readinessProbe: + httpGet: + path: /status + port: 22068 + initialDelaySeconds: 5 + periodSeconds: 10 + failureThreshold: 3 + securityContext: + readOnlyRootFilesystem: false + volumes: + - name: relay-data + persistentVolumeClaim: + claimName: syncthing-relay-data + restartPolicy: Always diff --git a/syncthing-relay/versions/1/ingress.yaml b/syncthing-relay/versions/1/ingress.yaml new file mode 100644 index 0000000..8838796 --- /dev/null +++ b/syncthing-relay/versions/1/ingress.yaml @@ -0,0 +1,26 @@ +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: syncthing-relay + namespace: {{ .namespace }} + annotations: + external-dns.alpha.kubernetes.io/target: {{ .externalDnsDomain }} + external-dns.alpha.kubernetes.io/cloudflare-proxied: "false" + external-dns.alpha.kubernetes.io/ttl: "60" +spec: + ingressClassName: traefik + rules: + - host: {{ .domain }} + http: + paths: + - path: / + pathType: Prefix + backend: + service: + name: syncthing-relay-status + port: + number: 80 + tls: + - hosts: + - {{ .domain }} + secretName: {{ .tlsSecretName }} diff --git a/syncthing-relay/versions/1/kustomization.yaml b/syncthing-relay/versions/1/kustomization.yaml new file mode 100644 index 0000000..193e00e --- /dev/null +++ b/syncthing-relay/versions/1/kustomization.yaml @@ -0,0 +1,15 @@ +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization +namespace: {{ .namespace }} +labels: + - includeSelectors: true + pairs: + app: syncthing-relay + managedBy: kustomize + partOf: wild-cloud +resources: + - namespace.yaml + - pvc.yaml + - deployment.yaml + - service.yaml + - ingress.yaml diff --git a/syncthing-relay/versions/1/manifest.yaml b/syncthing-relay/versions/1/manifest.yaml new file mode 100644 index 0000000..f8c8419 --- /dev/null +++ b/syncthing-relay/versions/1/manifest.yaml @@ -0,0 +1,8 @@ +version: 2.1.1 +defaultConfig: + namespace: syncthing-relay + domain: relay.{{ .cloud.domain }} + externalDnsDomain: "{{ .cloud.domain }}" + tlsSecretName: wildcard-wild-cloud-tls + externalAddress: "{{ .cloud.domain }}:22067" + storage: 100Mi diff --git a/syncthing-relay/versions/1/namespace.yaml b/syncthing-relay/versions/1/namespace.yaml new file mode 100644 index 0000000..b7816b8 --- /dev/null +++ b/syncthing-relay/versions/1/namespace.yaml @@ -0,0 +1,4 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: "{{ .namespace }}" diff --git a/syncthing-relay/versions/1/pvc.yaml b/syncthing-relay/versions/1/pvc.yaml new file mode 100644 index 0000000..df2d824 --- /dev/null +++ b/syncthing-relay/versions/1/pvc.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: PersistentVolumeClaim +metadata: + name: syncthing-relay-data + namespace: {{ .namespace }} +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: {{ .storage }} diff --git a/syncthing-relay/versions/1/service.yaml b/syncthing-relay/versions/1/service.yaml new file mode 100644 index 0000000..b60a569 --- /dev/null +++ b/syncthing-relay/versions/1/service.yaml @@ -0,0 +1,29 @@ +apiVersion: v1 +kind: Service +metadata: + name: syncthing-relay + namespace: {{ .namespace }} +spec: + type: LoadBalancer + selector: + component: server + ports: + - name: relay + port: 22067 + targetPort: 22067 + protocol: TCP +--- +apiVersion: v1 +kind: Service +metadata: + name: syncthing-relay-status + namespace: {{ .namespace }} +spec: + type: ClusterIP + selector: + component: server + ports: + - name: status + port: 80 + targetPort: 22068 + protocol: TCP diff --git a/taiga/versions/6/deployment-protected.yaml b/taiga/versions/6/deployment-protected.yaml index 772d72d..4d98822 100644 --- a/taiga/versions/6/deployment-protected.yaml +++ b/taiga/versions/6/deployment-protected.yaml @@ -47,16 +47,14 @@ spec: - name: taiga-media mountPath: /taiga/media livenessProbe: - httpGet: - path: / + tcpSocket: port: 8003 initialDelaySeconds: 15 periodSeconds: 30 timeoutSeconds: 5 failureThreshold: 3 readinessProbe: - httpGet: - path: / + tcpSocket: port: 8003 initialDelaySeconds: 10 periodSeconds: 10 diff --git a/taiga/versions/6/pvc.yaml b/taiga/versions/6/pvc.yaml index b14b05f..ff03924 100644 --- a/taiga/versions/6/pvc.yaml +++ b/taiga/versions/6/pvc.yaml @@ -5,7 +5,7 @@ metadata: namespace: {{ .namespace }} spec: accessModes: - - ReadWriteOnce + - ReadWriteMany resources: requests: storage: {{ .mediaStorage }} @@ -17,7 +17,7 @@ metadata: namespace: {{ .namespace }} spec: accessModes: - - ReadWriteOnce + - ReadWriteMany resources: requests: storage: {{ .staticStorage }} diff --git a/traefik/app.yaml b/traefik/app.yaml index e6b0842..08c1fc8 100644 --- a/traefik/app.yaml +++ b/traefik/app.yaml @@ -2,4 +2,9 @@ name: traefik is: traefik description: Cloud-native reverse proxy and ingress controller category: services +icon: https://cdn.jsdelivr.net/gh/homarr-labs/dashboard-icons/svg/traefik.svg latest: "v3" +ignoreRules: + - WC-HELM + - WC-SEL + - WC-NS # fixed namespace: traefik diff --git a/ushahidi/notes.md b/ushahidi/notes.md new file mode 100644 index 0000000..0b7ca76 --- /dev/null +++ b/ushahidi/notes.md @@ -0,0 +1,28 @@ +# Ushahidi — Notes + +## Requires a dedicated local Redis (no password support) + +Ushahidi provides no way to configure a Redis password (`REDIS_HOST` and `REDIS_PORT` only). +Connecting to Wild Cloud's shared authenticated Redis fails with `ConnectionError`. + +**Fix**: deploy a dedicated Redis sidecar or separate Deployment for Ushahidi with no auth: + +```yaml +- name: redis + image: redis:7-alpine + args: ["--save", ""] # disable persistence +``` + +Remove `redis` from `requires` in `manifest.yaml` (Ushahidi won't use the shared instance) and +point `REDIS_HOST` to the app-local Service name. + +## Laravel startup probe + +Ushahidi's API is Laravel-based. The startup process (autoload optimization, key generation, +migrations) commonly takes 2–3 minutes. Set an extended initial delay on the liveness probe: + +```yaml +livenessProbe: + initialDelaySeconds: 120 + failureThreshold: 6 +``` diff --git a/ushahidi/versions/5/kustomization.yaml b/ushahidi/versions/5/kustomization.yaml index 1269cf1..9d6dc6f 100644 --- a/ushahidi/versions/5/kustomization.yaml +++ b/ushahidi/versions/5/kustomization.yaml @@ -10,6 +10,7 @@ labels: resources: - namespace.yaml - db-init-job.yaml + - redis.yaml - deployment-api.yaml - deployment-worker.yaml - deployment-client.yaml diff --git a/ushahidi/versions/5/manifest.yaml b/ushahidi/versions/5/manifest.yaml index 491a5d5..19e87a8 100644 --- a/ushahidi/versions/5/manifest.yaml +++ b/ushahidi/versions/5/manifest.yaml @@ -1,7 +1,6 @@ version: 5.1.0-2 requires: - name: mysql - - name: redis defaultConfig: namespace: ushahidi externalDnsDomain: '{{ .cloud.domain }}' @@ -15,8 +14,8 @@ defaultConfig: name: ushahidi user: ushahidi redis: - host: '{{ .apps.redis.host }}' - port: '{{ .apps.redis.port }}' + host: ushahidi-redis + port: '6379' defaultSecrets: - key: appKey - key: dbPassword diff --git a/ushahidi/versions/5/redis.yaml b/ushahidi/versions/5/redis.yaml new file mode 100644 index 0000000..fa83561 --- /dev/null +++ b/ushahidi/versions/5/redis.yaml @@ -0,0 +1,51 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: ushahidi-redis + namespace: {{ .namespace }} +spec: + replicas: 1 + selector: + matchLabels: + component: redis + template: + metadata: + labels: + component: redis + spec: + securityContext: + runAsNonRoot: true + runAsUser: 999 + runAsGroup: 999 + seccompProfile: + type: RuntimeDefault + containers: + - name: redis + image: redis:7-alpine + args: ["--save", ""] + ports: + - containerPort: 6379 + resources: + limits: + cpu: 200m + memory: 128Mi + requests: + cpu: 10m + memory: 32Mi + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: [ALL] + readOnlyRootFilesystem: false +--- +apiVersion: v1 +kind: Service +metadata: + name: ushahidi-redis + namespace: {{ .namespace }} +spec: + selector: + component: redis + ports: + - port: 6379 + targetPort: 6379 diff --git a/utils/app.yaml b/utils/app.yaml index 9327c03..5e0b7f9 100644 --- a/utils/app.yaml +++ b/utils/app.yaml @@ -3,3 +3,8 @@ is: utils description: Utility tools and scripts for cluster administration category: services latest: "v1" +ignoreRules: + - WC-ICON # admin utility; not user-facing + - WC-SEL # utility/debug pods; not long-running services requiring rolling updates + - WC-NS # fixed namespace: debug + - WC-SC-POD # netdebug is a privileged debug pod diff --git a/utils/versions/v1/netdebug.yaml b/utils/versions/v1/netdebug.yaml index 37b81f0..7973039 100644 --- a/utils/versions/v1/netdebug.yaml +++ b/utils/versions/v1/netdebug.yaml @@ -45,7 +45,7 @@ spec: serviceAccountName: netdebug containers: - name: netdebug - image: nicolaka/netshoot:latest + image: nicolaka/netshoot:v0.16 command: ["/bin/bash"] args: ["-c", "while true; do sleep 3600; done"] resources: diff --git a/vllm/versions/0/deployment.yaml b/vllm/versions/0/deployment.yaml index 940c6c9..3e20dc4 100644 --- a/vllm/versions/0/deployment.yaml +++ b/vllm/versions/0/deployment.yaml @@ -4,6 +4,8 @@ metadata: name: vllm spec: replicas: 1 + strategy: + type: Recreate selector: matchLabels: component: inference @@ -12,14 +14,13 @@ spec: labels: component: inference spec: + runtimeClassName: nvidia + enableServiceLinks: false securityContext: - runAsNonRoot: true - runAsUser: 1000 - runAsGroup: 1000 seccompProfile: type: RuntimeDefault nodeSelector: - nvidia.com/gpu.product: "{{ .gpuProduct }}" + feature.node.kubernetes.io/pci-0300_10de.present: "true" containers: - name: vllm image: vllm/vllm-openai:v0.5.4 @@ -29,18 +30,27 @@ spec: capabilities: drop: - ALL - readOnlyRootFilesystem: false args: - --model={{ .model }} - --max-model-len={{ .maxModelLen }} - --tensor-parallel-size=1 - --gpu-memory-utilization={{ .gpuMemoryUtilization }} - - --enforce-eager=True + - --enforce-eager + - --api-key=$(VLLM_API_KEY) env: + - name: VLLM_API_KEY + valueFrom: + secretKeyRef: + name: vllm-secrets + key: apiKey - name: VLLM_TORCH_DTYPE value: "auto" - name: VLLM_WORKER_CONCURRENCY value: "1" + - name: HF_HOME + value: "/root/.cache/huggingface" + - name: NUMBA_CACHE_DIR + value: "/tmp" ports: - name: http containerPort: 8000 @@ -55,19 +65,21 @@ spec: nvidia.com/gpu: {{ .gpuCount }} readinessProbe: httpGet: - path: /v1/models + path: /health port: http - initialDelaySeconds: 30 - periodSeconds: 10 - timeoutSeconds: 5 + initialDelaySeconds: 300 + periodSeconds: 30 + timeoutSeconds: 10 + failureThreshold: 5 livenessProbe: httpGet: path: /health port: http - initialDelaySeconds: 60 - periodSeconds: 15 - timeoutSeconds: 5 + initialDelaySeconds: 600 + periodSeconds: 60 + timeoutSeconds: 10 + failureThreshold: 3 tolerations: - key: "nvidia.com/gpu" operator: "Exists" - effect: "NoSchedule" \ No newline at end of file + effect: "NoSchedule" diff --git a/vllm/versions/0/ingress.yaml b/vllm/versions/0/ingress.yaml index 5713eb2..b5d5dbc 100644 --- a/vllm/versions/0/ingress.yaml +++ b/vllm/versions/0/ingress.yaml @@ -8,6 +8,7 @@ metadata: traefik.ingress.kubernetes.io/router.tls: "true" traefik.ingress.kubernetes.io/router.tls.certresolver: letsencrypt spec: + ingressClassName: traefik rules: - host: {{ .domain }} http: diff --git a/vllm/versions/0/manifest.yaml b/vllm/versions/0/manifest.yaml index 5869d32..4d7b4ed 100644 --- a/vllm/versions/0/manifest.yaml +++ b/vllm/versions/0/manifest.yaml @@ -1,15 +1,16 @@ -version: 0.5.4-2 +version: 0.5.4-6 requires: [] defaultConfig: namespace: llm model: Qwen/Qwen2.5-7B-Instruct maxModelLen: 8192 gpuMemoryUtilization: 0.9 - gpuProduct: RTX 4090 cpuRequest: '4' cpuLimit: '8' memoryRequest: 16Gi memoryLimit: 24Gi gpuCount: 1 domain: vllm.{{ .cloud.domain }} -defaultSecrets: [] + externalDnsDomain: "{{ .cloud.domain }}" +defaultSecrets: + - key: apiKey diff --git a/writefreely/versions/0/deployment.yaml b/writefreely/versions/0/deployment.yaml index 8cf83ee..8bc60ed 100644 --- a/writefreely/versions/0/deployment.yaml +++ b/writefreely/versions/0/deployment.yaml @@ -22,7 +22,7 @@ spec: type: RuntimeDefault containers: - name: writefreely - image: jrasanen/writefreely:latest + image: jrasanen/writefreely:v0.15.1 ports: - name: http containerPort: 8080 diff --git a/writefreely/versions/0/manifest.yaml b/writefreely/versions/0/manifest.yaml index f0b8102..441238f 100644 --- a/writefreely/versions/0/manifest.yaml +++ b/writefreely/versions/0/manifest.yaml @@ -1,4 +1,4 @@ -version: 0.16.0-1 +version: 0.15.1-2 requires: - name: mysql defaultConfig: diff --git a/zulip/notes.md b/zulip/notes.md new file mode 100644 index 0000000..04699a6 --- /dev/null +++ b/zulip/notes.md @@ -0,0 +1,28 @@ +# Zulip — Notes + +## TLS-terminating reverse proxy configuration + +Zulip's internal nginx redirects port 80 → HTTPS by default. When Traefik terminates TLS and +forwards plain HTTP internally, this causes an infinite redirect loop. + +Set these two env vars in the deployment: + +```yaml +- name: DISABLE_HTTPS + value: "true" +- name: LOADBALANCER_IPS + value: "10.244.0.0/16" # Kubernetes pod CIDR +``` + +- `DISABLE_HTTPS=true`: configures nginx in `http_only` mode, removing the 80 → HTTPS redirect. +- `LOADBALANCER_IPS`: trusts `X-Forwarded-Proto: https` from the pod CIDR so Zulip generates + `https://` links instead of `http://`. Set to the cluster's pod network CIDR (typically + `10.244.0.0/16` for Flannel). + +Also update liveness/readiness probes from port 443 HTTPS to port 80 HTTP when +`DISABLE_HTTPS=true` is set. + +## Root URL returns 404 — this is expected + +Zulip's root URL (`/`) returns 404 "No organization found" — this is correct behavior. +The actual login page is at `/accounts/home/`. diff --git a/zulip/versions/9/deployment.yaml b/zulip/versions/9/deployment.yaml index 74e0d4b..e513c85 100644 --- a/zulip/versions/9/deployment.yaml +++ b/zulip/versions/9/deployment.yaml @@ -97,6 +97,11 @@ spec: secretKeyRef: name: zulip-secrets key: smtp.password + # Reverse proxy configuration (Traefik terminates TLS) + - name: DISABLE_HTTPS + value: "true" + - name: LOADBALANCER_IPS + value: "10.244.0.0/16" resources: limits: cpu: "2" @@ -112,8 +117,7 @@ spec: livenessProbe: httpGet: path: /accounts/home/ - port: 443 - scheme: HTTPS + port: 80 httpHeaders: - name: Host value: "{{ .domain }}" @@ -124,8 +128,7 @@ spec: readinessProbe: httpGet: path: /accounts/home/ - port: 443 - scheme: HTTPS + port: 80 httpHeaders: - name: Host value: "{{ .domain }}"