CSTF/wild-cloud
1
1
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects 1 Releases 1 Wiki Activity

Rename wild-app-config to wild-app-add. Update dev docs for apps.

Browse Source
This commit is contained in:
Paul Payne
2025-07-14 18:41:54 -07:00
parent 269fb29123
commit 2401423976
4 changed files with 203 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files

125
apps/README.md
View File

@@ -1,24 +1,113 @@
# Wild Cloud-maintained apps
## Usage
This is the Wild Cloud apps repository.
`generate-config <app-name>`
`kubectl apply -k apps/<app-name>`
This repository contains a collection of apps that can be deployed using Wild Cloud scripts. Wild Cloud apps follow a specific structure and naming convention to ensure compatibility with the Wild Cloud ecosystem.
## Best Practices
## App Structure
- `*.yaml`, not `*.yml`.
- Keep the service and deployment names the same as the app for easy DNS lookup.
- A starting `kustomization.yaml` for every app:
Each subdirectory in this directory represents a Wild Cloud app. Each app directory contains a `manifest.yaml` file and other necessary Kustomize files.
### App Manifest
The required `manifest.yaml` file contains metadata about the app.
This is the contents of an example `manifest.yaml` file for an app named "immich":
```yaml
name: immich
description: Immich is a self-hosted photo and video backup solution that allows you to store, manage, and share your media files securely.
version: 1.0.0
icon: https://immich.app/assets/images/logo.png
requires:
- name: redis
- name: postgres
defaultConfig:
serverImage: ghcr.io/immich-app/immich-server:release
mlImage: ghcr.io/immich-app/immich-machine-learning:release
timezone: UTC
serverPort: 2283
mlPort: 3003
storage: 250Gi
cacheStorage: 10Gi
redisHostname: redis.redis.svc.cluster.local
dbHostname: postgres.postgres.svc.cluster.local
dbUsername: immich
domain: immich.{{ .cloud.domain }}
requiredSecrets:
- apps.immich.dbPassword
- apps.postgres.password
```
Explanation of the fields:
- `name`: The name of the app, used for identification.
- `description`: A brief description of the app.
- `version`: The version of the app. This should generally follow the versioning scheme of the app itself.
- `icon`: A URL to an icon representing the app.
- `requires`: A list of other apps that this app depends on. Each entry should be the name of another app.
- `defaultConfig`: A set of default configuration values for the app. When an app is added using `wild-app-add`, these values will be added to the Wild Cloud `config.yaml` file.
- `requiredSecrets`: A list of secrets that must be set in the Wild Cloud `secrets.yaml` file for the app to function properly. These secrets are typically sensitive information like database passwords or API keys. Keys with random values will be generated automatically when the app is added.
### Kustomize Files
Wild Cloud apps use Kustomize as kustomize files are simple, transparent, and easier to manage in a Git repository.
#### Configuration (templates)
The only non-standard feature of Wild Cloud apps is the use of Wild Cloud configuration variables in the Kustomize files, such as `{{ .cloud.domain }}` for the domain name. This allows for dynamic configuration based on the operator's Wild Cloud configuration and secrets. All configuration variables need to exist in the operator's `config.yaml`, so they should be defined in the app's `manifest.yaml` under `defaultConfig`.
When `wild-app-add` is run, the app's Kustomize files will be compiled with the operator's Wild Cloud configuration and secrets resulting in standard Kustomize files being placed in the Wild Cloud home directory. This makes modifying the app's configuration straightforward, as operators can customize their app files as needed. When changes are pulled from upstream, the operator can run `wild-app-add` again to update their local configuration and Kustomize files and then view the changes with `git status` to see what has changed.
#### Secrets
Secrets are managed in the `secrets.yaml` file in the Wild Cloud home directory. The app's `manifest.yaml` should list any required secrets under `requiredSecrets`. When the app is added, default secret values will be generated and stored in the `secrets.yaml` file. Secrets are always stored and referenced in the `apps.<app-name>.<secret-name>` yaml path. When `wild-app-deploy` is run, a Secret resource will be created in the Kubernetes cluster with the name `<app-name>-secrets`, containing all secrets defined in the manifest's `requiredSecrets` key. These secrets can then be referenced in the app's Kustomize files using a `secretKeyRef`. For example, to mount a secret in an environment variable, you would use:
```yaml
env:
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: immich-secrets
key: dbPassword
```
`secrets.yaml` files should not be checked in to a git repository and are ignored by default in Wild Cloud home directories. Checked in kustomize files should only reference secrets, not compile them.
## App Lifecycle
Apps in Wild Cloud are managed by operators using a set of commands run from their Wild Cloud home directory.
- `wild-apps-list`: Lists all available apps.
- `wild-app-fetch <app-name>`: Fetches the latest app files from the Wild Cloud repository and stores them in your Wild Cloud cache.
- `wild-app-add <app-name>`: Adds the app manifest to your Wild Cloud home `apps` directory, updates missing values in `config.yaml` and `secrets.yaml` with the app's default configurations, and compiles the app's Kustomize files.
- `wild-app-deploy <app-name>`: Deploys the app to your Wild Cloud.
## Contributing
If you would like to contribute an app to the Wild Cloud, issue a pull request with the app's directory containing the `manifest.yaml` file and any necessary Kustomize files. Ensure that your app follows the structure outlined above.
## Tips for App Packagers
### Converting from Helm Charts
# Converting Helm Charts to Wild Cloud Kustomize definitions
Wild Cloud apps use Kustomize as kustomize files are simpler, more transparent, and easier to manage in a Git repository than Helm charts. If you have a Helm chart that you want to convert to a Wild Cloud app, the following example steps can simplify the process for you:
```bash
helm fetch --untar --untardir charts nginx-stable/nginx-ingress
helm template --output-dir base --namespace ingress --values values.yaml ingress-controller charts/nginx-ingress
cat <<EOF > base/nginx-ingress/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ingress
EOF
cd base/nginx-ingress
kustomize create --autodetect
```
After running these commands against your own Helm chart, you will have a Kustomize directory structure that can be used as a Wild Cloud app. All you need to do then, usually, is add a `manifest.yaml` file and replace any hardcoded values with Wild Cloud variables, such as `{{ .cloud.domain }}` for the domain name.
```yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: postgres
labels:
- includeSelectors: true
pairs:
app: <app>
managedBy: kustomize
partOf: wild-cloud
```

175
bin/wild-app-config → bin/wild-app-add
View File

@@ -111,100 +111,111 @@ if [ -d "${DEST_APP_DIR}" ]; then
fi
mkdir -p "${DEST_APP_DIR}"
echo "Pulling app '${APP_NAME}' from cache to ${DEST_APP_DIR}"
echo "Adding app '${APP_NAME}' from cache to ${DEST_APP_DIR}"
# Merge defaultConfig from manifest.yaml into .wildcloud/config.yaml
# Step 1: Copy only manifest.yaml from cache first
MANIFEST_FILE="${CACHE_APP_DIR}/manifest.yaml"
if [ -f "${MANIFEST_FILE}" ]; then
echo "Copying manifest.yaml from cache"
cp "${MANIFEST_FILE}" "${DEST_APP_DIR}/manifest.yaml"
else
echo "Warning: manifest.yaml not found in cache for app '${APP_NAME}'"
exit 1
fi
# Step 2: Add missing config and secret values based on manifest
echo "Processing configuration and secrets from manifest.yaml"
# Check if the app section exists in config.yaml, if not create it
if ! yq eval ".apps.${APP_NAME}" "${CONFIG_FILE}" >/dev/null 2>&1; then
yq eval ".apps.${APP_NAME} = {}" -i "${CONFIG_FILE}"
fi
# Extract defaultConfig from manifest.yaml and merge into config.yaml
if yq eval '.defaultConfig' "${DEST_APP_DIR}/manifest.yaml" | grep -q -v '^null$'; then
echo "Merging defaultConfig from manifest.yaml into .wildcloud/config.yaml"
# Check if the app section exists in config.yaml, if not create it
if ! yq eval ".apps.${APP_NAME}" "${CONFIG_FILE}" >/dev/null 2>&1; then
yq eval ".apps.${APP_NAME} = {}" -i "${CONFIG_FILE}"
fi
# Extract defaultConfig from manifest.yaml and merge into config.yaml
if yq eval '.defaultConfig' "${MANIFEST_FILE}" | grep -q -v '^null$'; then
# Merge each key from defaultConfig into the app's config, only if not already set
yq eval '.defaultConfig | keys | .[]' "${MANIFEST_FILE}" | while read -r key; do
# Get the value from defaultConfig
value=$(yq eval ".defaultConfig.${key}" "${MANIFEST_FILE}")
# Check if key exists and is not null in app config
current_value=$(yq eval ".apps.${APP_NAME}.${key} // \"null\"" ${CONFIG_FILE})
if [ "${current_value}" = "null" ]; then
# Process value through gomplate if it contains template syntax
if [[ "${value}" == *"{{"* && "${value}" == *"}}"* ]]; then
# Build gomplate command with config context
gomplate_cmd="gomplate -c .=${CONFIG_FILE}"
# Add secrets context if secrets.yaml exists
if [ -f "${SECRETS_FILE}" ]; then
gomplate_cmd="${gomplate_cmd} -c secrets=${SECRETS_FILE}"
fi
# Process the value through gomplate
processed_value=$(echo "${value}" | ${gomplate_cmd})
value="${processed_value}"
# Merge each key from defaultConfig into the app's config, only if not already set
yq eval '.defaultConfig | keys | .[]' "${DEST_APP_DIR}/manifest.yaml" | while read -r key; do
# Get the value from defaultConfig
value=$(yq eval ".defaultConfig.${key}" "${DEST_APP_DIR}/manifest.yaml")
# Check if key exists and is not null in app config
current_value=$(yq eval ".apps.${APP_NAME}.${key} // \"null\"" ${CONFIG_FILE})
if [ "${current_value}" = "null" ]; then
# Process value through gomplate if it contains template syntax
if [[ "${value}" == *"{{"* && "${value}" == *"}}"* ]]; then
# Build gomplate command with config context
gomplate_cmd="gomplate -c .=${CONFIG_FILE}"
# Add secrets context if secrets.yaml exists
if [ -f "${SECRETS_FILE}" ]; then
gomplate_cmd="${gomplate_cmd} -c secrets=${SECRETS_FILE}"
fi
if [[ "${value}" =~ ^[0-9]+$ ]] || [[ "${value}" =~ ^[0-9]+\.[0-9]+$ ]] || [ "${value}" = "true" ] || [ "${value}" = "false" ]; then
# Numeric, boolean values don't need quotes
yq eval ".apps.${APP_NAME}.${key} = ${value}" -i "${CONFIG_FILE}"
else
# String values need quotes
yq eval ".apps.${APP_NAME}.${key} = \"${value}\"" -i "${CONFIG_FILE}"
fi
# Process the value through gomplate
processed_value=$(echo "${value}" | ${gomplate_cmd})
value="${processed_value}"
fi
done
echo "Merged defaultConfig for app '${APP_NAME}'"
if [[ "${value}" =~ ^[0-9]+$ ]] || [[ "${value}" =~ ^[0-9]+\.[0-9]+$ ]] || [ "${value}" = "true" ] || [ "${value}" = "false" ]; then
# Numeric, boolean values don't need quotes
yq eval ".apps.${APP_NAME}.${key} = ${value}" -i "${CONFIG_FILE}"
else
# String values need quotes
yq eval ".apps.${APP_NAME}.${key} = \"${value}\"" -i "${CONFIG_FILE}"
fi
fi
done
echo "Merged defaultConfig for app '${APP_NAME}'"
fi
# Scaffold required secrets into .wildcloud/secrets.yaml if they don't exist
if yq eval '.requiredSecrets' "${DEST_APP_DIR}/manifest.yaml" | grep -q -v '^null$'; then
echo "Scaffolding required secrets for app '${APP_NAME}'"
# Ensure .wildcloud/secrets.yaml exists
if [ ! -f "${SECRETS_FILE}" ]; then
echo "# Wild Cloud Secrets Configuration" > "${SECRETS_FILE}"
echo "# This file contains sensitive data and should NOT be committed to git" >> "${SECRETS_FILE}"
echo "# Add this file to your .gitignore" >> "${SECRETS_FILE}"
echo "" >> "${SECRETS_FILE}"
fi
# Scaffold required secrets into .wildcloud/secrets.yaml if they don't exist
if yq eval '.requiredSecrets' "${MANIFEST_FILE}" | grep -q -v '^null$'; then
echo "Scaffolding required secrets for app '${APP_NAME}'"
# Ensure .wildcloud/secrets.yaml exists
if [ ! -f "${SECRETS_FILE}" ]; then
echo "# Wild Cloud Secrets Configuration" > "${SECRETS_FILE}"
echo "# This file contains sensitive data and should NOT be committed to git" >> "${SECRETS_FILE}"
echo "# Add this file to your .gitignore" >> "${SECRETS_FILE}"
echo "" >> "${SECRETS_FILE}"
fi
# Check if apps section exists, if not create it
if ! yq eval ".apps" "${SECRETS_FILE}" >/dev/null 2>&1; then
yq eval ".apps = {}" -i "${SECRETS_FILE}"
fi
# Check if app section exists, if not create it
if ! yq eval ".apps.${APP_NAME}" "${SECRETS_FILE}" >/dev/null 2>&1; then
yq eval ".apps.${APP_NAME} = {}" -i "${SECRETS_FILE}"
fi
# Add dummy values for each required secret if not already present
yq eval '.requiredSecrets[]' "${MANIFEST_FILE}" | while read -r secret_path; do
current_value=$(yq eval ".${secret_path} // \"null\"" "${SECRETS_FILE}")
if [ "${current_value}" = "null" ]; then
echo "Adding dummy secret: ${secret_path}"
# Extract just the key name for the dummy value
secret_key=$(basename "${secret_path}")
yq eval ".${secret_path} = \"CHANGE_ME_${secret_key^^}\"" -i "${SECRETS_FILE}"
fi
done
echo "Required secrets scaffolded for app '${APP_NAME}'"
# Check if apps section exists, if not create it
if ! yq eval ".apps" "${SECRETS_FILE}" >/dev/null 2>&1; then
yq eval ".apps = {}" -i "${SECRETS_FILE}"
fi
# Check if app section exists, if not create it
if ! yq eval ".apps.${APP_NAME}" "${SECRETS_FILE}" >/dev/null 2>&1; then
yq eval ".apps.${APP_NAME} = {}" -i "${SECRETS_FILE}"
fi
# Add dummy values for each required secret if not already present
yq eval '.requiredSecrets[]' "${DEST_APP_DIR}/manifest.yaml" | while read -r secret_path; do
current_value=$(yq eval ".${secret_path} // \"null\"" "${SECRETS_FILE}")
if [ "${current_value}" = "null" ]; then
echo "Adding dummy secret: ${secret_path}"
# Extract just the key name for the dummy value
secret_key=$(basename "${secret_path}")
yq eval ".${secret_path} = \"CHANGE_ME_${secret_key^^}\"" -i "${SECRETS_FILE}"
fi
done
echo "Required secrets scaffolded for app '${APP_NAME}'"
fi
# Step 3: Copy and compile all other files from cache to app directory
echo "Copying and compiling remaining files from cache"
# Function to process a file with gomplate if it's a YAML file
process_file() {
local src_file="$1"
local dest_file="$2"
echo "Processing YAML file: ${dest_file}"
echo "Processing file: ${dest_file}"
# Build gomplate command with config context (enables .config shorthand)
gomplate_cmd="gomplate -c .=${CONFIG_FILE}"
@@ -218,7 +229,7 @@ process_file() {
${gomplate_cmd} -f "${src_file}" > "${dest_file}"
}
# Copy directory structure and process files
# Copy directory structure and process files (excluding manifest.yaml which was already copied)
find "${CACHE_APP_DIR}" -type d | while read -r src_dir; do
rel_path="${src_dir#${CACHE_APP_DIR}}"
rel_path="${rel_path#/}" # Remove leading slash if present
@@ -230,6 +241,12 @@ done
find "${CACHE_APP_DIR}" -type f | while read -r src_file; do
rel_path="${src_file#${CACHE_APP_DIR}}"
rel_path="${rel_path#/}" # Remove leading slash if present
# Skip manifest.yaml since it was already copied in step 1
if [ "${rel_path}" = "manifest.yaml" ]; then
continue
fi
dest_file="${DEST_APP_DIR}/${rel_path}"
# Ensure destination directory exists
@@ -239,4 +256,4 @@ find "${CACHE_APP_DIR}" -type f | while read -r src_file; do
process_file "${src_file}" "${dest_file}"
done
echo "Successfully pulled app '${APP_NAME}' with template processing"
echo "Successfully added app '${APP_NAME}' with template processing"

1
bin/wild-setup
View File

@@ -71,7 +71,6 @@ while [[ $# -gt 0 ]]; do
done
# Initialize Wild Cloud environment
if [ -z "${WC_ROOT}" ]; then
print "WC_ROOT is not set."
exit 1

17
docs/guides/converting-helm-charts.md
View File

@@ -1,17 +0,0 @@
# Converting Helm Charts to Wild Cloud Kustomize definitions
_(This guide is a work in progress)_
```bash
helm fetch --untar --untardir charts nginx-stable/nginx-ingress
helm template --output-dir base --namespace ingress --values values.yaml ingress-controller charts/nginx-ingress
cat <<EOF > base/nginx-ingress/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ingress
EOF
cd base/nginx-ingress
kustomize create --autodetect
kubectl apply -k base/nginx-ingress
```