diff --git a/README.md b/README.md index d0e5f47..3e04ac0 100644 --- a/README.md +++ b/README.md @@ -24,57 +24,9 @@ This project provides a complete, production-ready Kubernetes infrastructure des - Deploy new applications with a single command - Manage everything through a slick web dashboard -## What Can You Run? - -The possibilities are endless! Here are just a few ideas: - -- **Personal Websites & Blogs** (WordPress, Ghost, Hugo) -- **Photo Storage & Sharing** (PhotoPrism, Immich) -- **Document Management** (Paperless-ngx) -- **Media Servers** (Jellyfin, Plex) -- **Home Automation** (Home Assistant) -- **Password Managers** (Bitwarden, Vaultwarden) -- **Note Taking Apps** (Joplin, Trilium) -- **Productivity Tools** (Nextcloud, Gitea, Plausible Analytics) -- **Database Servers** (PostgreSQL, MariaDB, MongoDB) -- **And much more!** - -## Key Features - -- **One-Command Setup**: Get a complete Kubernetes infrastructure with a single script -- **Secure by Default**: Automatic HTTPS certificates for all services -- **Split-Horizon DNS**: Access services internally or externally with the same domain -- **Custom Domains**: Use your own domain name for all services -- **Service Templates**: Deploy new applications with a simple command -- **Dashboard**: Web UI for monitoring and managing your infrastructure -- **No Cloud Vendor Lock-in**: Run on your own hardware, from a Raspberry Pi to old laptops - ## Getting Started -For detailed instructions, check out our documentation: - -- [**Setup Guide**](./docs/SETUP.md) - Step-by-step instructions for setting up your infrastructure -- [**Applications Guide**](./apps/README.md) - How to deploy and manage applications on your cloud -- [**Maintenance Guide**](./docs/MAINTENANCE.md) - Troubleshooting, backups, updates, and security - -After setup, visit your dashboard at `https://dashboard.internal.yourdomain.com` to start exploring your new personal cloud infrastructure! - -## Project Structure - -``` -. -├── bin/ # Helper scripts -├── apps/ # Apps -├── docs/ # Documentation -│ ├── SETUP.md # Setup instructions -│ ├── APPS.md # Application deployment guide -│ ├── MAINTENANCE.md # Maintenance and troubleshooting -│ ├── OPS.md # Operations guide -│ └── INGRESS.md # Network configuration guide -├── infrastructure_setup/ # Infrastructure setup scripts -├── services/ # Custom service templates and deployed services -└── load-env.sh # Environment variable loader -``` +See [Setup](./docs/SETUP.md). ## Contributing diff --git a/cluster-node-setup/README.md b/cluster-node-setup/README.md new file mode 100644 index 0000000..6b8ab1d --- /dev/null +++ b/cluster-node-setup/README.md @@ -0,0 +1,90 @@ +# Cluster Node Setup + +Cluster node setup is WIP. Any kubernetes setup will do. Currently, we have a working cluster using each of these methods and are moving towards Talos. + +## k3s cluster node setup + +K3s provides a fully-compliant Kubernetes distribution in a small footprint. + +To set up control nodes: + +```bash +# Install K3s without the default load balancer (we'll use MetalLB) +curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode=644 --disable servicelb --disable metallb + +# Set up kubectl configuration +mkdir -p ~/.kube +sudo cat /etc/rancher/k3s/k3s.yaml > ~/.kube/config +chmod 600 ~/.kube/config +``` + +Set up the infrastructure services after these are running, then you can add more worker nodes with: + +```bash +# On your master node, get the node token +NODE_TOKEN=`sudo cat /var/lib/rancher/k3s/server/node-token` +MASTER_IP=192.168.8.222 +# On each new node, join the cluster + +curl -sfL https://get.k3s.io | K3S_URL=https://$MASTER_IP:6443 K3S_TOKEN=$NODE_TOKEN sh - +``` + +## Talos cluster node setup + +This is a new experimental method for setting up cluster nodes. We're currently working through the simplest bootstrapping experience. + +Currently, though, all these steps are manual. + +Copy this entire directory to your personal cloud folder and modify it as necessary as you install. We suggest putting it in `cluster/bootstrap`. + +```bash + +# Install kubectl +curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" +curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256" +echo "$(cat kubectl.sha256) kubectl" | sha256sum --check +sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl + +# Install talosctl +curl -sL https://talos.dev/install | sh + +# In your LAN Router (which is your DHCP server), + +CLUSTER_NAME=test-cluster +VIP=192.168.8.20 # Non-DHCP + +# Boot your nodes with the ISO and put their IP addresses here. Pin in DHCP. +# Nodes must all be on the same switch. +# TODO: How to set these static on boot? +CONTROL_NODE_1=192.168.8.21 +CONTROL_NODE_2=192.168.8.22 +CONTROL_NODE_3=192.168.8.23 + +# Generate cluster config files (including pki and tokens) +cd generated +talosctl gen secrets -o secrets.yaml +talosctl gen config --with-secrets secrets.yaml $CLUSTER_NAME https://$VIP:6443 +talosctl config merge ./talosconfig +cd .. + +# If the disk you want to install Talos on isn't /dev/sda, you should +# update to the disk you want in patch/controlplane.yml and patch/worker.yaml. If you have already attempted to install a node and received an error about not being able to find /dev/sda, you can see what disks are available on it with: +# +# talosctl -n $VIP get disks --insecure + +# See https://www.talos.dev/v1.10/talos-guides/configuration/patching/ +talosctl machineconfig patch generated/controlplane.yaml --patch @patch/controlplane.yaml -o final/controlplane.yaml +talosctl machineconfig patch generated/worker.yaml --patch @patch/worker.yaml -o final/worker.yaml +$ + +# Apply control plane config +talosctl apply-config --insecure -n $CONTROL_NODE_1,$CONTROL_NODE_2,$CONTROL_NODE_3 --file final/controlplane.yaml + +# Bootstrap cluster on control plan +talosctl bootstrap -n $VIP + +# Merge new cluster information into kubeconfig +talosctl kubeconfig + +# You are now ready to use both `talosctl` and `kubectl` against your new cluster. +``` diff --git a/cluster-node-setup/generated/.gitkeep b/cluster-node-setup/generated/.gitkeep new file mode 100644 index 0000000..e69de29 diff --git a/infrastructure_setup/setup_node.sh b/cluster-node-setup/old/setup_node.sh similarity index 100% rename from infrastructure_setup/setup_node.sh rename to cluster-node-setup/old/setup_node.sh diff --git a/cluster-node-setup/patch/controlplane.yaml b/cluster-node-setup/patch/controlplane.yaml new file mode 100644 index 0000000..409de0b --- /dev/null +++ b/cluster-node-setup/patch/controlplane.yaml @@ -0,0 +1,17 @@ +machine: + install: + disk: /dev/sdc + network: + interfaces: + - interface: eth0 + vip: + ip: 192.168.8.20 + - interface: eth1 + dhcp: true +cluster: + discovery: + enabled: true + registries: + service: + disabled: true + allowSchedulingOnControlPlanes: true diff --git a/cluster-node-setup/patch/worker.yaml b/cluster-node-setup/patch/worker.yaml new file mode 100644 index 0000000..a286018 --- /dev/null +++ b/cluster-node-setup/patch/worker.yaml @@ -0,0 +1,3 @@ +machine: + install: + disk: /dev/sdc diff --git a/docs/SETUP.md b/docs/SETUP.md index 4db675b..41ec677 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -1,27 +1,25 @@ # Setting Up Your Wild Cloud -## Set up your cloud bootstrapper +## Set up your personal cloud operations directory -See [Dnsmasq Setup](./guides/dnsmasq-setup.md). +- Create a directory somewhere. We recommend you use an Ubuntu machine. +- Inside it, run `wild-init`. This will scaffold your cloud directory. +- In your cloud directory, update `.wildcloud/config.yaml`. Use the same values in this dir in a `.env` + +## Set up your Cloud Central + +See [Central Setup](../central-setup/README.md). ## Set up Control Nodes ### 2. Install K3s (Lightweight Kubernetes) -K3s provides a fully-compliant Kubernetes distribution in a small footprint: - -```bash -# Install K3s without the default load balancer (we'll use MetalLB) -curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode=644 --disable servicelb --disable metallb - -# Set up kubectl configuration -mkdir -p ~/.kube -sudo cat /etc/rancher/k3s/k3s.yaml > ~/.kube/config -chmod 600 ~/.kube/config -``` +See [Cluster Node Setup](../cluster-node-setup/README.md). ## Install Infrastructure Components +> Currently, these are set up to run from this directory. This will be moved to (1) a `bin/wild-generate-infrastructure-setup` script to copy them all to your personal cloud dir, (2) `wild-cli` (to do the same), or (3) `wild-central`, once I get my mind made up. + One command sets up your entire cloud infrastructure: ```bash @@ -38,19 +36,6 @@ This installs and configures: - **ExternalDNS**: Updates DNS records automatically - **Kubernetes Dashboard**: Web UI for managing your cluster -## Set up worker nodes - -For larger workloads or high availability, you can add more nodes: - -```bash -# On your master node, get the node token -NODE_TOKEN=`sudo cat /var/lib/rancher/k3s/server/node-token` -MASTER_IP=192.168.8.222 -# On each new node, join the cluster - -curl -sfL https://get.k3s.io | K3S_URL=https://$MASTER_IP:6443 K3S_TOKEN=$NODE_TOKEN sh - -``` - ## Next Steps Now that your infrastructure is set up, you can: diff --git a/docs/guides/talos-setup.md b/docs/guides/talos-setup.md deleted file mode 100644 index 89caf2c..0000000 --- a/docs/guides/talos-setup.md +++ /dev/null @@ -1,32 +0,0 @@ -# Talos - -This is an alternate setup to using ks that uses talos and bare kubernetes. IN PROGRESS. - -From https://www.talos.dev/v1.10/introduction/getting-started/ - -```bash -# Install kubectl -curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" -curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256" -echo "$(cat kubectl.sha256) kubectl" | sha256sum --check -sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl - -# Install talosctl -curl -sL https://talos.dev/install | sh - -# Generate cluster config files (and pki and tokens) -talosctl gen config test-cluster https://192.168.8.238:6443 - -talosctl -n 192.168.8.238 get disks --insecure -# Update disk in controlplane.yml - -# Apply control plane config -talosctl apply-config --insecure --nodes 192.168.8.238 --file controlplane.yaml - -# Bootstrap cluster on control plan -talosctl bootstrap --nodes 192.168.8.238 --endpoints 192.168.8.238 --talosconfig=./talosconfig - -# Merge into kubeconfig -talosctl kubeconfig --nodes 192.168.8.238 --endpoints 192.168.8.238 --talosconfig=./talosconfig - -```