CSTF/wild-cloud
1
1
Fork 0
Code Issues 12 Pull Requests Actions Packages Projects 1 Releases 1 Wiki Activity

Remove README.md for cluster node setup

Browse Source
This commit is contained in:
Paul Payne
2025-06-28 09:42:27 -07:00
parent 66ff0e90e0
commit ddac8775b1
1 changed files with 0 additions and 235 deletions
Download Patch File Download Diff File Expand all files Collapse all files

235
setup/cluster-nodes/README.md
View File

@@ -1,235 +0,0 @@
# Cluster Node Setup
This directory contains automation for setting up Talos Kubernetes cluster nodes with static IP configuration.
## Hardware Detection and Setup (Recommended)
The automated setup discovers hardware configuration from nodes in maintenance mode and generates machine configurations with the correct interface names and disk paths.
### Prerequisites
1. `source .env`
2. Boot nodes with Talos ISO in maintenance mode
3. Nodes must be accessible on the network
### Hardware Discovery Workflow
```bash
# ONE-TIME CLUSTER INITIALIZATION (run once per cluster)
./init-cluster.sh
# FOR EACH CONTROL PLANE NODE:
# 1. Boot node with Talos ISO (it will get a DHCP IP in maintenance mode)
# 2. Detect hardware and update config.yaml
./detect-node-hardware.sh <maintenance-ip> <node-number>
# Example: Node boots at 192.168.8.168, register as node 1
./detect-node-hardware.sh 192.168.8.168 1
# 3. Generate machine config for registered nodes
./generate-machine-configs.sh
# 4. Apply configuration - node will reboot with static IP
talosctl apply-config --insecure -n 192.168.8.168 --file final/controlplane-node-1.yaml
# 5. Wait for reboot, node should come up at its target static IP (192.168.8.31)
# Repeat steps 1-5 for additional control plane nodes
```
The `detect-node-hardware.sh` script will:
- Connect to nodes in maintenance mode via talosctl
- Discover active ethernet interfaces (e.g., `enp4s0` instead of hardcoded `eth0`)
- Discover available installation disks (>10GB)
- Update `config.yaml` with per-node hardware configuration
- Provide next steps for machine config generation
The `init-cluster.sh` script will:
- Generate Talos cluster secrets and base configurations (once per cluster)
- Set up talosctl context with cluster certificates
- Configure VIP endpoint for cluster communication
The `generate-machine-configs.sh` script will:
- Check which nodes have been hardware-detected
- Compile network configuration templates with discovered hardware settings
- Create final machine configurations for registered nodes only
- Include system extensions for Longhorn (iscsi-tools, util-linux-tools)
- Update talosctl context with registered node IPs
### Cluster Bootstrap
After all control plane nodes are configured with static IPs:
```bash
# Bootstrap the cluster using any control node
talosctl bootstrap --nodes 192.168.8.31 --endpoint 192.168.8.31
# Get kubeconfig
talosctl kubeconfig
# Verify cluster is ready
kubectl get nodes
```
## Complete Example
Here's a complete example of setting up a 3-node control plane:
```bash
# CLUSTER INITIALIZATION (once per cluster)
./init-cluster.sh
# NODE 1
# Boot node with Talos ISO, it gets DHCP IP 192.168.8.168
./detect-node-hardware.sh 192.168.8.168 1
./generate-machine-configs.sh
talosctl apply-config --insecure -n 192.168.8.168 --file final/controlplane-node-1.yaml
# Node reboots and comes up at 192.168.8.31
# NODE 2
# Boot second node with Talos ISO, it gets DHCP IP 192.168.8.169
./detect-node-hardware.sh 192.168.8.169 2
./generate-machine-configs.sh
talosctl apply-config --insecure -n 192.168.8.169 --file final/controlplane-node-2.yaml
# Node reboots and comes up at 192.168.8.32
# NODE 3
# Boot third node with Talos ISO, it gets DHCP IP 192.168.8.170
./detect-node-hardware.sh 192.168.8.170 3
./generate-machine-configs.sh
talosctl apply-config --insecure -n 192.168.8.170 --file final/controlplane-node-3.yaml
# Node reboots and comes up at 192.168.8.33
# CLUSTER BOOTSTRAP
talosctl bootstrap -n 192.168.8.30
talosctl kubeconfig
kubectl get nodes
```
## Configuration Details
### Per-Node Configuration
Each control plane node has its own configuration block in `config.yaml`:
```yaml
cluster:
nodes:
control:
vip: 192.168.8.30
node1:
ip: 192.168.8.31
interface: enp4s0 # Discovered automatically
disk: /dev/sdb # Selected during hardware detection
node2:
ip: 192.168.8.32
# interface and disk added after hardware detection
node3:
ip: 192.168.8.33
# interface and disk added after hardware detection
```
Worker nodes use DHCP by default. You can use the same hardware detection process for worker nodes if static IPs are needed.
## Talosconfig Management
### Context Naming and Conflicts
When running `talosctl config merge ./generated/talosconfig`, if a context with the same name already exists, talosctl will create an enumerated version (e.g., `demo-cluster-2`).
**For a clean setup:**
- Delete existing contexts before merging: `talosctl config contexts` then `talosctl config context <name> --remove`
- Or use `--force` to overwrite: `talosctl config merge ./generated/talosconfig --force`
**Recommended approach for new clusters:**
```bash
# Remove old context if rebuilding cluster
talosctl config context demo-cluster --remove || true
# Merge new configuration
talosctl config merge ./generated/talosconfig
talosctl config endpoint 192.168.8.30
talosctl config node 192.168.8.31 # Add nodes as they are registered
```
### Context Configuration Timeline
1. **After first node hardware detection**: Merge talosconfig and set endpoint/first node
2. **After additional nodes**: Add them to the existing context with `talosctl config node <ip1> <ip2> <ip3>`
3. **Before cluster bootstrap**: Ensure all control plane nodes are in the node list
### System Extensions
All nodes include:
- `siderolabs/iscsi-tools`: Required for Longhorn storage
- `siderolabs/util-linux-tools`: Utility tools for storage operations
### Hardware Detection
The `detect-node-hardware.sh` script automatically discovers:
- **Network interfaces**: Finds active ethernet interfaces (no more hardcoded `eth0`)
- **Installation disks**: Lists available disks >10GB for interactive selection
- **Per-node settings**: Updates `config.yaml` with hardware-specific configuration
This eliminates the need to manually configure hardware settings and handles different hardware configurations across nodes.
### Template Structure
Configuration templates are stored in `patch.templates/` and use gomplate syntax:
- `controlplane-node-1.yaml`: Template for first control plane node
- `controlplane-node-2.yaml`: Template for second control plane node
- `controlplane-node-3.yaml`: Template for third control plane node
- `worker.yaml`: Template for worker nodes
Templates use per-node variables from `config.yaml`:
- `{{ .cluster.nodes.control.node1.ip }}`
- `{{ .cluster.nodes.control.node1.interface }}`
- `{{ .cluster.nodes.control.node1.disk }}`
- `{{ .cluster.nodes.control.vip }}`
The `wild-compile-template-dir` command processes all templates and outputs compiled configurations to the `patch/` directory.
## Troubleshooting
### Hardware Detection Issues
```bash
# Check if node is accessible in maintenance mode
talosctl -n <NODE_IP> version --insecure
# View available network interfaces
talosctl -n <NODE_IP> get links --insecure
# View available disks
talosctl -n <NODE_IP> get disks --insecure
```
### Manual Hardware Discovery
If the automatic detection fails, you can manually inspect hardware:
```bash
# Find active ethernet interfaces
talosctl -n <NODE_IP> get links --insecure -o json | jq -s '.[] | select(.spec.operationalState == "up" and .spec.type == "ether" and .metadata.id != "lo") | .metadata.id'
# Find suitable installation disks
talosctl -n <NODE_IP> get disks --insecure -o json | jq -s '.[] | select(.spec.size > 10000000000) | .metadata.id'
```
### Node Status
```bash
# View machine configuration (only works after config is applied)
talosctl -n <NODE_IP> get machineconfig
```