wild-cloud/docs/guides/making-backups.md

Making Backups

This guide covers how to create backups of your Wild Cloud applications and cluster configuration.

Overview

Wild Cloud's backup system creates backups using native tools for each data type:

• PostgreSQL databases: pg_dump in custom compressed format
• MySQL databases: mysqldump with gzip compression
• Persistent volumes: Longhorn native backup API
• Configuration: tar.gz archives of manifests, config, and secrets

Backups are stored at a configured destination (S3, Azure Blob, NFS, or local filesystem) and tracked via recovery plans that coordinate the full backup-restore lifecycle.

Prerequisites

Before making backups, ensure you have:

1. A backup destination configured — S3 bucket, Azure container, NFS share, or local path
2. Longhorn backup target configured if backing up persistent volumes
3. kubectl access to your cluster

Configuring Backup Destination

Web UI

Navigate to Backups and click Settings to configure your backup destination and retention policy.

CLI

Backup configuration is stored in your instance's config.yaml under the backup: section. Credentials are stored in secrets.yaml.

Example configuration:

# config.yaml
backup:
  destination:
    type: "s3"  # "s3", "azure", "nfs", or "local"
    s3:
      bucket: "my-backups"
      region: "us-east-1"
      endpoint: "minio.example.com"  # Optional, for S3-compatible services
  retention:
    daily: 7
    weekly: 4
    monthly: 6
    yearly: 1

# secrets.yaml
backup:
  s3:
    accessKeyId: "..."
    secretAccessKey: "..."

Supported Destinations

Destination	Config Fields	Notes
local	path	Default: instances/{instance}/backups
s3	bucket, region, endpoint, accessKeyId, secretAccessKey	Supports S3-compatible services like MinIO
azure	container, storageAccount, accessKey	Azure Blob Storage
nfs	server, path, mountPoint, mountOptions	Auto-recovers stale mounts

Making Backups

Single App Backup

Web UI: Navigate to Backups > [app] and click Backup Now.

CLI:

# Backup a single app
wild backup start gitea

# Shorthand
wild backup gitea

All Apps Backup

CLI:

# Backup all deployed apps
wild backup all

Cluster Config Backup

Cluster config backups archive the credentials and secrets not tracked in git — kubeconfig, talosconfig, config.yaml, secrets.yaml, and Talos generated configs.

Web UI: Navigate to Backups and click Backup on the Cluster Config row.

CLI / API:

curl -X POST http://localhost:5055/api/v1/instances/{instance}/backup/cluster

What Gets Backed Up

Application Backups

The backup system auto-discovers what to back up based on each app's manifest:

Component	Tool	Format	Storage Key
PostgreSQL database	pg_dump	Custom binary (compression level 9) + globals SQL	postgres/{instance}/{app}/{timestamp}.dump
MySQL database	mysqldump	Gzip-compressed SQL	mysql/{instance}/{app}/{timestamp}.sql.gz
Persistent volumes	Longhorn native API	Longhorn backup format	Stored in Longhorn backup target
App config & manifests	tar + gzip	tar.gz archive	config/{instance}/{app}/{timestamp}.tar.gz

Cache volumes (names containing -cache or -tmp) and cache databases (Redis, Memcached) are automatically excluded.

Cluster Config Backups

File	Purpose
kubeconfig	Kubernetes API credentials
config.yaml	Full instance configuration
secrets.yaml	App secrets (database passwords, API keys)
talos/generated/talosconfig	Talos API credentials
talos/generated/controlplane.yaml	Control plane node config
talos/generated/worker.yaml	Worker node config
talos/generated/secrets.yaml	Talos bootstrap secrets (cluster identity)

Storage key: cluster-config/{instance}/{timestamp}.tar.gz

Discovering Backup Resources

Before backing up for the first time, you can discover what persistent data an app has:

wild backup discover gitea

This analyzes the app's manifest and kustomize resources to find databases and PVCs, showing what will be backed up and what will be skipped.

Scheduled Backups

Creating a Schedule

Web UI: Navigate to Backups > [app] and click Schedule.

CLI:

# Daily backup at 2 AM
wild backup schedule create gitea --frequency daily --time 02:00

# Weekly backup on Sunday at 3 AM
wild backup schedule create gitea --frequency weekly --time 03:00 --day-of-week 0

# Monthly backup on the 1st at midnight
wild backup schedule create gitea --frequency monthly --time 00:00 --day-of-month 1

Managing Schedules

# List all schedules
wild backup schedule list

# Enable/disable a schedule
wild backup schedule enable <schedule-id>
wild backup schedule disable <schedule-id>

# Manually trigger a schedule
wild backup schedule run <schedule-id>

# Delete a schedule
wild backup schedule delete <schedule-id>

Retention is enforced automatically after each scheduled backup completes.

Listing and Verifying Backups

# List backups for an app
wild backup list gitea

# Verify a backup can be restored
wild backup verify gitea

# Verify a specific backup
wild backup verify gitea 20250314T021530Z

Deleting Backups

# Delete a specific backup
wild backup delete gitea 20250314T021530Z

# Skip confirmation
wild backup delete gitea 20250314T021530Z --yes

Backup Health

Check the overall health of your backup system:

Web UI: The Backups page shows a health summary across all apps — backup count, last backup time, scheduled status, and total size.

API:

curl http://localhost:5055/api/v1/instances/{instance}/backup/health

Recovery Plans

Each backup creates a recovery plan (recovery-plan.yaml) that tracks the backup's contents and coordinates restore operations. The plan records what strategies were used, where data is stored, and the current lifecycle status.

Plan statuses progress through: backing_up -> backed_up -> (restore phases when used).

Troubleshooting

"No databases or PVCs found"

• The app has no database dependencies in its manifest.yaml
• No PVCs with matching labels exist in the app namespace
• Run wild backup discover <app> to see what's detected

Longhorn backup fails

• Verify Longhorn backup target is configured (kubectl get settings -n longhorn-system backup-target)
• Check Longhorn manager pods are running on all worker nodes
• Ensure sufficient storage at the backup target

Database dump fails

• Verify the database pod is running: kubectl get pods -n postgres
• Check that the database name in config.yaml matches the actual database

Scheduled backups not running

• Verify the schedule is enabled: wild backup schedule list
• Check the Wild Central API is running: wild daemon status

Security Notes

• Encryption: S3 and Azure destinations support server-side encryption. Configure bucket/container encryption policies at your cloud provider.
• Secrets: Database credentials and API keys are included in cluster config backups. Store these backups securely.
• Access control: Restrict access to your backup destination. Cluster config backups contain everything needed to access your cluster.

Next Steps

• Restoring Backups — Learn how to restore from backups using blue-green deployment
• Disaster Recovery — Full cluster rebuild procedures
• Set up scheduled backups for all critical apps
• Store cluster config backups in a second location (not on the cluster itself)