wild-cloud-dev/ai/wildcloud-v.PoC/overview.md

Wild Cloud Overview

Wild Cloud is a complete, production-ready Kubernetes infrastructure designed for personal use. It combines enterprise-grade technologies to create a self-hosted cloud platform with automated deployment, HTTPS certificates, and web management interfaces.

What is Wild Cloud?

Vision

In a world where digital lives are increasingly controlled by large corporations, Wild Cloud puts you back in control by providing:

• Privacy: Your data stays on your hardware, under your control
• Ownership: No subscription fees or sudden price increases
• Freedom: Run the apps you want, the way you want them
• Learning: Gain valuable skills in modern cloud technologies
• Resilience: Reduce reliance on third-party services that can disappear

Core Capabilities

Complete Infrastructure Stack:

• Kubernetes cluster with Talos Linux
• Automatic HTTPS certificates via Let's Encrypt
• Load balancing with MetalLB
• Ingress routing with Traefik
• Distributed storage with Longhorn
• DNS management with CoreDNS and ExternalDNS

Application Platform:

• One-command application deployment
• Pre-built apps for common self-hosted services
• Automatic database setup and configuration
• Integrated backup and restore system
• Web-based management interfaces

Enterprise Features:

• High availability and fault tolerance
• Automated certificate management
• Network policies and security contexts
• Monitoring and observability
• Infrastructure as code principles

Technology Stack

Core Infrastructure

• Talos Linux - Immutable OS designed for Kubernetes
• Kubernetes - Container orchestration platform
• MetalLB - Load balancer for bare metal deployments
• Traefik - HTTP reverse proxy and ingress controller
• Longhorn - Distributed block storage system
• cert-manager - Automatic TLS certificate management

Supporting Services

• CoreDNS - DNS server for service discovery
• ExternalDNS - Automatic DNS record management
• Kubernetes Dashboard - Web UI for cluster management
• restic - Backup solution with deduplication
• gomplate - Template processor for configurations

Development Tools

• Kustomize - Kubernetes configuration management
• kubectl - Kubernetes command line interface
• talosctl - Talos Linux management tool
• Bats - Testing framework for bash scripts

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                          Internet                               │
└─────────────────┬───────────────────────────────────────────────┘
                  │
┌─────────────────▼───────────────────────────────────────────────┐
│                      DNS Provider                               │
│              (Cloudflare, Route53, etc.)                        │
└─────────────────┬───────────────────────────────────────────────┘
                  │
┌─────────────────▼───────────────────────────────────────────────┐
│                    Your Network                                 │
│  ┌─────────────┐  ┌─────────────────────────────────────────┐   │
│  │   dnsmasq   │  │          Kubernetes Cluster             │   │
│  │   Server    │  │  ┌─────────────┐ ┌─────────────────┐    │   │
│  │             │  │  │  MetalLB    │ │    Traefik      │    │   │
│  │ DNS + DHCP  │  │  │ LoadBalancer│ │   Ingress       │    │   │
│  └─────────────┘  │  └─────────────┘ └─────────────────┘    │   │
│                   │  ┌───────────────────────────────────┐  │   │
│                   │  │           Applications            │  │   │
│                   │  │   Ghost, Immich, Gitea, vLLM...   │  │   │
│                   │  └───────────────────────────────────┘  │   │
│                   └─────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

Traffic Flow

1. External Request → DNS resolution via provider
2. DNS Response → Points to your cluster's external IP
3. Network Request → Hits MetalLB load balancer
4. Load Balancer → Routes to Traefik ingress controller
5. Ingress Controller → Terminates TLS and routes to application
6. Application → Serves content from Kubernetes pod

Getting Started

Prerequisites

Hardware Requirements:

• Minimum 3 nodes for high availability
• 8GB RAM per node (16GB+ recommended)
• 100GB+ storage per node
• Gigabit network connectivity
• x86_64 architecture

Network Requirements:

• All nodes on same network segment
• One dedicated machine for dnsmasq (can be lightweight)
• Static IP assignments or DHCP reservations
• Internet connectivity for downloads and certificates

Quick Start Guide

1. Install Dependencies

# Clone Wild Cloud repository
git clone https://github.com/your-org/wild-cloud
cd wild-cloud

# Install required tools
scripts/setup-utils.sh

2. Initialize Your Cloud

# Create and initialize new cloud directory
mkdir my-cloud && cd my-cloud
wild-init

# Follow interactive setup prompts for:
# - Domain name configuration
# - Email for certificates
# - Network settings

3. Deploy Infrastructure

# Complete automated setup
wild-setup

# Or step-by-step:
wild-setup-cluster     # Deploy Kubernetes cluster
wild-setup-services    # Install core services

4. Deploy Your First App

# List available applications
wild-apps-list

# Deploy a blog
wild-app-add ghost
wild-app-deploy ghost

# Access at https://ghost.yourdomain.com

5. Verify Deployment

# Check system health
wild-health

# Access Kubernetes dashboard
wild-dashboard-token
# Visit https://dashboard.internal.yourdomain.com

Key Concepts

Configuration Management

Wild Cloud uses a dual-file configuration system:

config.yaml - Non-sensitive settings:

cloud:
  domain: "example.com"
  email: "admin@example.com"
apps:
  ghost:
    domain: "blog.example.com"
    storage: "10Gi"

secrets.yaml - Sensitive data (auto-generated):

apps:
  ghost:
    dbPassword: "secure-random-password"
  postgresql:
    rootPassword: "another-secure-password"

Template System

All configurations are templates processed with gomplate:

Before Processing (in repository):

domain: {{ .apps.ghost.domain }}
storage: {{ .apps.ghost.storage | default "5Gi" }}

After Processing (in your cloud):

domain: blog.example.com
storage: 10Gi

Application Lifecycle

1. Discovery: wild-apps-list - Browse available apps
2. Configuration: wild-app-add app-name - Configure and prepare application
3. Deployment: wild-app-deploy app-name - Deploy to cluster
4. Operations: wild-app-doctor app-name - Monitor and troubleshoot

Available Applications

Content Management & Publishing

• Ghost - Modern publishing platform for blogs and websites
• Discourse - Community discussion platform with modern features

Media & File Management

• Immich - Self-hosted photo and video backup solution

Development Tools

• Gitea - Self-hosted Git service with web interface
• Docker Registry - Private container image registry

Communication & Marketing

• Keila - Newsletter and email marketing platform
• Listmonk - High-performance newsletter and mailing list manager

Databases & Caching

• PostgreSQL - Advanced open-source relational database
• MySQL - Popular relational database management system
• Redis - In-memory data structure store and cache
• Memcached - Distributed memory caching system

AI & Machine Learning

• vLLM - High-performance LLM inference server with OpenAI-compatible API

Core Commands Reference

Setup & Initialization

wild-init                    # Initialize new cloud directory
wild-setup                   # Complete infrastructure deployment
wild-setup-cluster          # Deploy Kubernetes cluster only
wild-setup-services         # Deploy cluster services only

Application Management

wild-apps-list              # List available applications
wild-app-add <app>          # Configure application
wild-app-deploy <app>       # Deploy to cluster
wild-app-delete <app>       # Remove application
wild-app-doctor <app>       # Run diagnostics

Configuration Management

wild-config <key>           # Read configuration value
wild-config-set <key> <val> # Set configuration value
wild-secret <key>           # Read secret value
wild-secret-set <key> <val> # Set secret value

Operations & Monitoring

wild-health                 # System health check
wild-dashboard-token        # Get dashboard access token
wild-backup                 # Backup system and apps
wild-app-backup <app>       # Backup specific application

Best Practices

Security

• Never commit secrets.yaml to version control
• Use strong, unique passwords for all services
• Regularly update system and application images
• Monitor certificate expiration and renewal
• Implement network policies for production workloads

Configuration Management

• Store config.yaml in version control with proper .gitignore
• Document configuration changes in commit messages
• Use branches for experimental configurations
• Backup configuration files before major changes
• Test configuration changes in development environment

Operations

• Monitor cluster health with wild-health
• Set up regular backup schedules with wild-backup
• Keep applications updated with latest security patches
• Monitor disk usage and expand storage as needed
• Document custom configurations and procedures

Development

• Follow Wild Cloud app structure conventions
• Use proper Kubernetes security contexts
• Include comprehensive health checks and probes
• Test applications thoroughly before deployment
• Document application-specific configuration requirements

Common Use Cases

Personal Blog/Website

# Deploy Ghost blog with custom domain
wild-config-set apps.ghost.domain "blog.yourdomain.com"
wild-app-add ghost
wild-app-deploy ghost

Photo Management

# Deploy Immich for photo backup and management
wild-app-add postgresql immich
wild-app-deploy postgresql immich

Development Environment

# Set up Git hosting and container registry
wild-app-add gitea docker-registry
wild-app-deploy gitea docker-registry

AI/ML Workloads

# Deploy vLLM for local AI inference
wild-config-set apps.vllm.model "Qwen/Qwen2.5-7B-Instruct"
wild-app-add vllm
wild-app-deploy vllm

Troubleshooting

Common Issues

Cluster Not Responding:

# Check node status
kubectl get nodes
talosctl health

# Verify network connectivity
ping <node-ip>

Applications Not Starting:

# Check pod status
kubectl get pods -n <app-namespace>

# View logs
kubectl logs deployment/<app-name> -n <app-namespace>

# Run diagnostics
wild-app-doctor <app-name>

Certificate Issues:

# Check certificate status
kubectl get certificates -A

# View cert-manager logs
kubectl logs -n cert-manager deployment/cert-manager

DNS Problems:

# Test DNS resolution
nslookup <app-domain>

# Check external-dns logs
kubectl logs -n external-dns deployment/external-dns

Getting Help

Documentation:

• Check docs/ directory for detailed guides
• Review application-specific README files
• Consult Kubernetes and Talos documentation

Community Support:

• Report issues on GitHub repository
• Join community forums and discussions
• Share configurations and troubleshooting tips

Professional Support:

• Consider professional services for production deployments
• Engage with cloud infrastructure consultants
• Participate in training and certification programs

Advanced Topics

Custom Applications

Create your own Wild Cloud applications:

1. Create App Directory: apps/myapp/
2. Define Manifest: Include metadata and configuration defaults
3. Create Templates: Kubernetes resources with gomplate variables
4. Test Deployment: Use standard Wild Cloud workflow
5. Share: Contribute back to the community

Multi-Environment Deployments

Manage multiple Wild Cloud instances:

• Development: Single-node cluster for testing
• Staging: Multi-node cluster mirroring production
• Production: Full HA cluster with monitoring and backups

Integration with External Services

Extend Wild Cloud capabilities:

• External DNS Providers: Cloudflare, Route53, Google DNS
• Backup Storage: S3, Google Cloud Storage, Azure Blob
• Monitoring: Prometheus, Grafana, AlertManager
• CI/CD: GitLab CI, GitHub Actions, Jenkins

Performance Optimization

Optimize for your workloads:

• Resource Allocation: CPU and memory limits/requests
• Storage Performance: NVMe SSDs, storage classes
• Network Optimization: Network policies, service mesh
• Scaling: Horizontal pod autoscaling, cluster autoscaling

Wild Cloud provides a solid foundation for personal cloud infrastructure while maintaining the flexibility to grow and adapt to changing needs. Whether you're running a simple blog or a complex multi-service application, Wild Cloud's enterprise-grade technologies ensure your infrastructure is reliable, secure, and maintainable.