New OPS-centric setup. Integrated with wild-init and wild-setup.

This commit is contained in:
2025-06-21 14:22:22 -07:00
parent e55b9b2b8c
commit f90baac653
70 changed files with 128 additions and 197 deletions

17
setup/README.md Normal file
View File

@@ -0,0 +1,17 @@
# Setup instructions
Follow the instructions to [set up a dnsmasq machine](./dnsmasq/README.md).
Follow the instructions to [set up cluster nodes](./cluster-nodes/README.md).
Set up cluster services:
```bash
./setup/cluster/setup-all.sh
```
Now make sure everything works:
```bash
./setup/cluster/validate-setup.sh
```

View File

@@ -0,0 +1,90 @@
# Cluster Node Setup
Cluster node setup is WIP. Any kubernetes setup will do. Currently, we have a working cluster using each of these methods and are moving towards Talos.
## k3s cluster node setup
K3s provides a fully-compliant Kubernetes distribution in a small footprint.
To set up control nodes:
```bash
# Install K3s without the default load balancer (we'll use MetalLB)
curl -sfL https://get.k3s.io | sh -s - --write-kubeconfig-mode=644 --disable servicelb --disable metallb
# Set up kubectl configuration
mkdir -p ~/.kube
sudo cat /etc/rancher/k3s/k3s.yaml > ~/.kube/config
chmod 600 ~/.kube/config
```
Set up the infrastructure services after these are running, then you can add more worker nodes with:
```bash
# On your master node, get the node token
NODE_TOKEN=`sudo cat /var/lib/rancher/k3s/server/node-token`
MASTER_IP=192.168.8.222
# On each new node, join the cluster
curl -sfL https://get.k3s.io | K3S_URL=https://$MASTER_IP:6443 K3S_TOKEN=$NODE_TOKEN sh -
```
## Talos cluster node setup
This is a new experimental method for setting up cluster nodes. We're currently working through the simplest bootstrapping experience.
Currently, though, all these steps are manual.
Copy this entire directory to your personal cloud folder and modify it as necessary as you install. We suggest putting it in `cluster/bootstrap`.
```bash
# Install kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"
echo "$(cat kubectl.sha256) kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
# Install talosctl
curl -sL https://talos.dev/install | sh
# In your LAN Router (which is your DHCP server),
CLUSTER_NAME=test-cluster
VIP=192.168.8.20 # Non-DHCP
# Boot your nodes with the ISO and put their IP addresses here. Pin in DHCP.
# Nodes must all be on the same switch.
# TODO: How to set these static on boot?
CONTROL_NODE_1=192.168.8.21
CONTROL_NODE_2=192.168.8.22
CONTROL_NODE_3=192.168.8.23
# Generate cluster config files (including pki and tokens)
cd generated
talosctl gen secrets -o secrets.yaml
talosctl gen config --with-secrets secrets.yaml $CLUSTER_NAME https://$VIP:6443
talosctl config merge ./talosconfig
cd ..
# If the disk you want to install Talos on isn't /dev/sda, you should
# update to the disk you want in patch/controlplane.yml and patch/worker.yaml. If you have already attempted to install a node and received an error about not being able to find /dev/sda, you can see what disks are available on it with:
#
# talosctl -n $VIP get disks --insecure
# See https://www.talos.dev/v1.10/talos-guides/configuration/patching/
talosctl machineconfig patch generated/controlplane.yaml --patch @patch/controlplane.yaml -o final/controlplane.yaml
talosctl machineconfig patch generated/worker.yaml --patch @patch/worker.yaml -o final/worker.yaml
$
# Apply control plane config
talosctl apply-config --insecure -n $CONTROL_NODE_1,$CONTROL_NODE_2,$CONTROL_NODE_3 --file final/controlplane.yaml
# Bootstrap cluster on control plan
talosctl bootstrap -n $VIP
# Merge new cluster information into kubeconfig
talosctl kubeconfig
# You are now ready to use both `talosctl` and `kubectl` against your new cluster.
```

View File

View File

@@ -0,0 +1,21 @@
#!/bin/bash
set -e
apt-get update
# Longhorn requirements
# Install iscsi on all nodes.
# apt-get install open-iscsi
# modprobe iscsi_tcp
# systemctl restart open-iscsi
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.8.1/deploy/prerequisite/longhorn-iscsi-installation.yaml
# Install NFSv4 client on all nodes.
# apt-get install nfs-common
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.8.1/deploy/prerequisite/longhorn-nfs-installation.yaml
apt-get install cryptsetup
# To check longhorn requirements:
# curl -sSfL https://raw.githubusercontent.com/longhorn/longhorn/v1.8.1/scripts/environment_check.sh | bash

View File

@@ -0,0 +1,17 @@
machine:
install:
disk: /dev/sdc
network:
interfaces:
- interface: eth0
vip:
ip: 192.168.8.20
- interface: eth1
dhcp: true
cluster:
discovery:
enabled: true
registries:
service:
disabled: true
allowSchedulingOnControlPlanes: true

View File

@@ -0,0 +1,3 @@
machine:
install:
disk: /dev/sdc

101
setup/cluster/README.md Normal file
View File

@@ -0,0 +1,101 @@
# Infrastructure setup scripts
Creates a fully functional personal cloud infrastructure on a bare metal Kubernetes (k3s) cluster that provides:
1. **External access** to services via configured domain names (using ${DOMAIN})
2. **Internal-only access** to admin interfaces (via internal.${DOMAIN} subdomains)
3. **Secure traffic routing** with automatic TLS
4. **Reliable networking** with proper load balancing
## Architecture
```
Internet → External DNS → MetalLB LoadBalancer → Traefik → Kubernetes Services
Internal DNS
Internal Network
```
## Key Components
- **MetalLB** - Provides load balancing for bare metal clusters
- **Traefik** - Handles ingress traffic, TLS termination, and routing
- **cert-manager** - Manages TLS certificates
- **CoreDNS** - Provides DNS resolution for services
- **Longhorn** - Distributed storage system for persistent volumes
- **NFS** - Network file system for shared media storage (optional)
- **Kubernetes Dashboard** - Web UI for cluster management (accessible via https://dashboard.internal.${DOMAIN})
- **Docker Registry** - Private container registry for custom images
## Configuration Approach
All infrastructure components use a consistent configuration approach:
1. **Environment Variables** - All configuration settings are managed using environment variables loaded by running `source load-env.sh`
2. **Template Files** - Configuration files use templates with `${VARIABLE}` syntax
3. **Setup Scripts** - Each component has a dedicated script in `infrastructure_setup/` for installation and configuration
## Idempotent Design
All setup scripts are designed to be idempotent:
- Scripts can be run multiple times without causing harm
- Each script checks for existing resources before creating new ones
- Configuration updates are applied cleanly without duplication
- Failed or interrupted setups can be safely retried
- Changes to configuration will be properly applied on subsequent runs
This idempotent approach ensures consistent, reliable infrastructure setup and allows for incremental changes without requiring a complete teardown and rebuild.
## NFS Setup (Optional)
The infrastructure supports optional NFS (Network File System) for shared media storage across the cluster:
### Host Setup
First, set up the NFS server on your chosen host:
```bash
# Set required environment variables
export NFS_HOST=box-01 # Hostname or IP of NFS server
export NFS_MEDIA_PATH=/data/media # Path to media directory
export NFS_STORAGE_CAPACITY=1Ti # Optional: PV size (default: 250Gi)
# Run host setup script on the NFS server
./infrastructure_setup/setup-nfs-host.sh
```
### Cluster Integration
Then integrate NFS with your Kubernetes cluster:
```bash
# Run cluster setup (part of setup-all.sh or standalone)
./infrastructure_setup/setup-nfs.sh
```
### Features
- **Automatic IP detection** - Uses network IP even when hostname resolves to localhost
- **Cluster-wide access** - Any pod can mount the NFS share regardless of node placement
- **Configurable capacity** - Set PersistentVolume size via `NFS_STORAGE_CAPACITY`
- **ReadWriteMany** - Multiple pods can simultaneously access the same storage
### Usage
Applications can use NFS storage by setting `storageClassName: nfs` in their PVCs:
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: media-pvc
spec:
accessModes:
- ReadWriteMany
storageClassName: nfs
resources:
requests:
storage: 100Gi
```

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,19 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: wildcard-internal-wild-cloud
namespace: cert-manager
spec:
secretName: wildcard-internal-wild-cloud-tls
dnsNames:
- "*.internal.${DOMAIN}"
- "internal.${DOMAIN}"
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
duration: 2160h # 90 days
renewBefore: 360h # 15 days
privateKey:
algorithm: RSA
size: 2048

View File

@@ -0,0 +1,26 @@
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
email: ${EMAIL}
privateKeySecretRef:
name: letsencrypt-prod
server: https://acme-v02.api.letsencrypt.org/directory
solvers:
# DNS-01 solver for wildcard certificates
- dns01:
cloudflare:
email: ${EMAIL}
apiTokenSecretRef:
name: cloudflare-api-token
key: api-token
selector:
dnsZones:
- "${CLOUDFLARE_DOMAIN}"
# Keep the HTTP-01 solver for non-wildcard certificates
- http01:
ingress:
class: traefik

View File

@@ -0,0 +1,26 @@
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
email: ${EMAIL}
privateKeySecretRef:
name: letsencrypt-staging
server: https://acme-staging-v02.api.letsencrypt.org/directory
solvers:
# DNS-01 solver for wildcard certificates
- dns01:
cloudflare:
email: ${EMAIL}
apiTokenSecretRef:
name: cloudflare-api-token
key: api-token
selector:
dnsZones:
- "${CLOUDFLARE_DOMAIN}"
# Keep the HTTP-01 solver for non-wildcard certificates
- http01:
ingress:
class: traefik

View File

@@ -0,0 +1,19 @@
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: wildcard-wild-cloud
namespace: cert-manager
spec:
secretName: wildcard-wild-cloud-tls
dnsNames:
- "*.${DOMAIN}"
- "${DOMAIN}"
issuerRef:
name: letsencrypt-prod
kind: ClusterIssuer
duration: 2160h # 90 days
renewBefore: 360h # 15 days
privateKey:
algorithm: RSA
size: 2048

View File

@@ -0,0 +1,49 @@
# CoreDNS
- https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/
- https://github.com/kubernetes/dns/blob/master/docs/specification.md
- https://coredns.io/
CoreDNS has the `kubernetes` plugin, so it returns all k8s service endpoints in well-known format.
All services and pods are registered in CoreDNS.
- <service-name>.<namespace>.svc.cluster.local
- <service-name>.<namespace>
- <service-name> (if in the same namespace)
- <pod-ipv4-address>.<namespace>.pod.cluster.local
- <pod-ipv4-address>.<service-name>.<namespace>.svc.cluster.local
Any query for a resource in the `internal.$DOMAIN` domain will be given the IP of the Traefik proxy. We expose the CoreDNS server in the LAN via MetalLB just for this capability.
## Default CoreDNS Configuration
Found at: https://github.com/k3s-io/k3s/blob/master/manifests/coredns.yaml
This is k3s default CoreDNS configuration, for reference:
```txt
.:53 {
errors
health
ready
kubernetes %{CLUSTER_DOMAIN}% in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
}
hosts /etc/coredns/NodeHosts {
ttl 60
reload 15s
fallthrough
}
prometheus :9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
import /etc/coredns/custom/*.override
}
import /etc/coredns/custom/*.server
```

View File

@@ -0,0 +1,28 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns-custom
namespace: kube-system
data:
# Custom server block for internal domains. All internal domains should
# resolve to the cluster proxy.
internal.server: |
internal.cloud.payne.io {
errors
cache 30
reload
template IN A {
match (.*)\.internal\.cloud\.payne\.io\.
answer "{{ .Name }} 60 IN A 192.168.8.240"
}
template IN AAAA {
match (.*)\.internal\.cloud\.payne\.io\.
rcode NXDOMAIN
}
}
# Custom override to set external resolvers.
external.override: |
forward . 1.1.1.1 8.8.8.8 {
max_concurrent 1000
}

View File

@@ -0,0 +1,25 @@
---
apiVersion: v1
kind: Service
metadata:
name: coredns-lb
namespace: kube-system
annotations:
metallb.universe.tf/loadBalancerIPs: "192.168.8.241"
spec:
type: LoadBalancer
ports:
- name: dns
port: 53
protocol: UDP
targetPort: 53
- name: dns-tcp
port: 53
protocol: TCP
targetPort: 53
- name: metrics
port: 9153
protocol: TCP
targetPort: 9153
selector:
k8s-app: kube-dns

View File

@@ -0,0 +1,2 @@
DOCKER_REGISTRY_STORAGE=10Gi
DOCKER_REGISTRY_HOST=docker-registry.$INTERNAL_DOMAIN

View File

@@ -0,0 +1,36 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: docker-registry
labels:
app: docker-registry
spec:
replicas: 1
selector:
matchLabels:
app: docker-registry
strategy:
rollingUpdate:
maxSurge: 0
maxUnavailable: 1
type: RollingUpdate
template:
metadata:
labels:
app: docker-registry
spec:
containers:
- image: registry:3.0.0
name: docker-registry
ports:
- containerPort: 5000
protocol: TCP
volumeMounts:
- mountPath: /var/lib/registry
name: docker-registry-storage
readOnly: false
volumes:
- name: docker-registry-storage
persistentVolumeClaim:
claimName: docker-registry-pvc

View File

@@ -0,0 +1,20 @@
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: docker-registry
spec:
rules:
- host: docker-registry.internal.${DOMAIN}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: docker-registry
port:
number: 5000
tls:
- hosts:
- docker-registry.internal.${DOMAIN}
secretName: wildcard-internal-wild-cloud-tls

View File

@@ -0,0 +1,40 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: docker-registry
labels:
- includeSelectors: true
pairs:
app: docker-registry
managedBy: wild-cloud
resources:
- deployment.yaml
- ingress.yaml
- service.yaml
- namespace.yaml
- pvc.yaml
configMapGenerator:
- name: docker-registry-config
envs:
- config/config.env
replacements:
- source:
kind: ConfigMap
name: docker-registry-config
fieldPath: data.DOCKER_REGISTRY_STORAGE
targets:
- select:
kind: PersistentVolumeClaim
name: docker-registry-pvc
fieldPaths:
- spec.resources.requests.storage
- source:
kind: ConfigMap
name: docker-registry-config
fieldPath: data.DOCKER_REGISTRY_HOST
targets:
- select:
kind: Ingress
name: docker-registry
fieldPaths:
- spec.rules.0.host
- spec.tls.0.hosts.0

View File

@@ -0,0 +1,4 @@
apiVersion: v1
kind: Namespace
metadata:
name: docker-registry

View File

@@ -0,0 +1,12 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: docker-registry-pvc
spec:
storageClassName: longhorn
accessModes:
- ReadWriteOnce
volumeMode: Filesystem
resources:
requests:
storage: 10Gi

View File

@@ -0,0 +1,13 @@
---
apiVersion: v1
kind: Service
metadata:
name: docker-registry
labels:
app: docker-registry
spec:
ports:
- port: 5000
targetPort: 5000
selector:
app: docker-registry

View File

@@ -0,0 +1,14 @@
# External DNS
See: https://github.com/kubernetes-sigs/external-dns
ExternalDNS allows you to keep selected zones (via --domain-filter) synchronized with Ingresses and Services of type=LoadBalancer and nodes in various DNS providers.
Currently, we are only configured to use CloudFlare.
Docs: https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/cloudflare.md
Any Ingress that has metatdata.annotions with
external-dns.alpha.kubernetes.io/hostname: `<something>.${DOMAIN}`
will have Cloudflare records created by External DNS.

View File

@@ -0,0 +1,39 @@
---
# CloudFlare provider for ExternalDNS
apiVersion: apps/v1
kind: Deployment
metadata:
name: external-dns
namespace: externaldns
spec:
selector:
matchLabels:
app: external-dns
strategy:
type: Recreate
template:
metadata:
labels:
app: external-dns
spec:
serviceAccountName: external-dns
containers:
- name: external-dns
image: registry.k8s.io/external-dns/external-dns:v0.13.4
args:
- --source=service
- --source=ingress
- --txt-owner-id=${OWNER_ID}
- --provider=cloudflare
- --domain-filter=payne.io
#- --exclude-domains=internal.${DOMAIN}
- --cloudflare-dns-records-per-page=5000
- --publish-internal-services
- --no-cloudflare-proxied
- --log-level=debug
env:
- name: CF_API_TOKEN
valueFrom:
secretKeyRef:
name: cloudflare-api-token
key: api-token

View File

@@ -0,0 +1,35 @@
---
# Common RBAC resources for all ExternalDNS deployments
apiVersion: v1
kind: ServiceAccount
metadata:
name: external-dns
namespace: externaldns
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: external-dns
rules:
- apiGroups: [""]
resources: ["services", "endpoints", "pods"]
verbs: ["get", "watch", "list"]
- apiGroups: ["extensions", "networking.k8s.io"]
resources: ["ingresses"]
verbs: ["get", "watch", "list"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: external-dns-viewer
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: external-dns
subjects:
- kind: ServiceAccount
name: external-dns
namespace: externaldns

347
setup/cluster/get_helm.sh Executable file
View File

@@ -0,0 +1,347 @@
#!/usr/bin/env bash
# Copyright The Helm Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# The install script is based off of the MIT-licensed script from glide,
# the package manager for Go: https://github.com/Masterminds/glide.sh/blob/master/get
: ${BINARY_NAME:="helm"}
: ${USE_SUDO:="true"}
: ${DEBUG:="false"}
: ${VERIFY_CHECKSUM:="true"}
: ${VERIFY_SIGNATURES:="false"}
: ${HELM_INSTALL_DIR:="/usr/local/bin"}
: ${GPG_PUBRING:="pubring.kbx"}
HAS_CURL="$(type "curl" &> /dev/null && echo true || echo false)"
HAS_WGET="$(type "wget" &> /dev/null && echo true || echo false)"
HAS_OPENSSL="$(type "openssl" &> /dev/null && echo true || echo false)"
HAS_GPG="$(type "gpg" &> /dev/null && echo true || echo false)"
HAS_GIT="$(type "git" &> /dev/null && echo true || echo false)"
HAS_TAR="$(type "tar" &> /dev/null && echo true || echo false)"
# initArch discovers the architecture for this system.
initArch() {
ARCH=$(uname -m)
case $ARCH in
armv5*) ARCH="armv5";;
armv6*) ARCH="armv6";;
armv7*) ARCH="arm";;
aarch64) ARCH="arm64";;
x86) ARCH="386";;
x86_64) ARCH="amd64";;
i686) ARCH="386";;
i386) ARCH="386";;
esac
}
# initOS discovers the operating system for this system.
initOS() {
OS=$(echo `uname`|tr '[:upper:]' '[:lower:]')
case "$OS" in
# Minimalist GNU for Windows
mingw*|cygwin*) OS='windows';;
esac
}
# runs the given command as root (detects if we are root already)
runAsRoot() {
if [ $EUID -ne 0 -a "$USE_SUDO" = "true" ]; then
sudo "${@}"
else
"${@}"
fi
}
# verifySupported checks that the os/arch combination is supported for
# binary builds, as well whether or not necessary tools are present.
verifySupported() {
local supported="darwin-amd64\ndarwin-arm64\nlinux-386\nlinux-amd64\nlinux-arm\nlinux-arm64\nlinux-ppc64le\nlinux-s390x\nlinux-riscv64\nwindows-amd64\nwindows-arm64"
if ! echo "${supported}" | grep -q "${OS}-${ARCH}"; then
echo "No prebuilt binary for ${OS}-${ARCH}."
echo "To build from source, go to https://github.com/helm/helm"
exit 1
fi
if [ "${HAS_CURL}" != "true" ] && [ "${HAS_WGET}" != "true" ]; then
echo "Either curl or wget is required"
exit 1
fi
if [ "${VERIFY_CHECKSUM}" == "true" ] && [ "${HAS_OPENSSL}" != "true" ]; then
echo "In order to verify checksum, openssl must first be installed."
echo "Please install openssl or set VERIFY_CHECKSUM=false in your environment."
exit 1
fi
if [ "${VERIFY_SIGNATURES}" == "true" ]; then
if [ "${HAS_GPG}" != "true" ]; then
echo "In order to verify signatures, gpg must first be installed."
echo "Please install gpg or set VERIFY_SIGNATURES=false in your environment."
exit 1
fi
if [ "${OS}" != "linux" ]; then
echo "Signature verification is currently only supported on Linux."
echo "Please set VERIFY_SIGNATURES=false or verify the signatures manually."
exit 1
fi
fi
if [ "${HAS_GIT}" != "true" ]; then
echo "[WARNING] Could not find git. It is required for plugin installation."
fi
if [ "${HAS_TAR}" != "true" ]; then
echo "[ERROR] Could not find tar. It is required to extract the helm binary archive."
exit 1
fi
}
# checkDesiredVersion checks if the desired version is available.
checkDesiredVersion() {
if [ "x$DESIRED_VERSION" == "x" ]; then
# Get tag from release URL
local latest_release_url="https://get.helm.sh/helm-latest-version"
local latest_release_response=""
if [ "${HAS_CURL}" == "true" ]; then
latest_release_response=$( curl -L --silent --show-error --fail "$latest_release_url" 2>&1 || true )
elif [ "${HAS_WGET}" == "true" ]; then
latest_release_response=$( wget "$latest_release_url" -q -O - 2>&1 || true )
fi
TAG=$( echo "$latest_release_response" | grep '^v[0-9]' )
if [ "x$TAG" == "x" ]; then
printf "Could not retrieve the latest release tag information from %s: %s\n" "${latest_release_url}" "${latest_release_response}"
exit 1
fi
else
TAG=$DESIRED_VERSION
fi
}
# checkHelmInstalledVersion checks which version of helm is installed and
# if it needs to be changed.
checkHelmInstalledVersion() {
if [[ -f "${HELM_INSTALL_DIR}/${BINARY_NAME}" ]]; then
local version=$("${HELM_INSTALL_DIR}/${BINARY_NAME}" version --template="{{ .Version }}")
if [[ "$version" == "$TAG" ]]; then
echo "Helm ${version} is already ${DESIRED_VERSION:-latest}"
return 0
else
echo "Helm ${TAG} is available. Changing from version ${version}."
return 1
fi
else
return 1
fi
}
# downloadFile downloads the latest binary package and also the checksum
# for that binary.
downloadFile() {
HELM_DIST="helm-$TAG-$OS-$ARCH.tar.gz"
DOWNLOAD_URL="https://get.helm.sh/$HELM_DIST"
CHECKSUM_URL="$DOWNLOAD_URL.sha256"
HELM_TMP_ROOT="$(mktemp -dt helm-installer-XXXXXX)"
HELM_TMP_FILE="$HELM_TMP_ROOT/$HELM_DIST"
HELM_SUM_FILE="$HELM_TMP_ROOT/$HELM_DIST.sha256"
echo "Downloading $DOWNLOAD_URL"
if [ "${HAS_CURL}" == "true" ]; then
curl -SsL "$CHECKSUM_URL" -o "$HELM_SUM_FILE"
curl -SsL "$DOWNLOAD_URL" -o "$HELM_TMP_FILE"
elif [ "${HAS_WGET}" == "true" ]; then
wget -q -O "$HELM_SUM_FILE" "$CHECKSUM_URL"
wget -q -O "$HELM_TMP_FILE" "$DOWNLOAD_URL"
fi
}
# verifyFile verifies the SHA256 checksum of the binary package
# and the GPG signatures for both the package and checksum file
# (depending on settings in environment).
verifyFile() {
if [ "${VERIFY_CHECKSUM}" == "true" ]; then
verifyChecksum
fi
if [ "${VERIFY_SIGNATURES}" == "true" ]; then
verifySignatures
fi
}
# installFile installs the Helm binary.
installFile() {
HELM_TMP="$HELM_TMP_ROOT/$BINARY_NAME"
mkdir -p "$HELM_TMP"
tar xf "$HELM_TMP_FILE" -C "$HELM_TMP"
HELM_TMP_BIN="$HELM_TMP/$OS-$ARCH/helm"
echo "Preparing to install $BINARY_NAME into ${HELM_INSTALL_DIR}"
runAsRoot cp "$HELM_TMP_BIN" "$HELM_INSTALL_DIR/$BINARY_NAME"
echo "$BINARY_NAME installed into $HELM_INSTALL_DIR/$BINARY_NAME"
}
# verifyChecksum verifies the SHA256 checksum of the binary package.
verifyChecksum() {
printf "Verifying checksum... "
local sum=$(openssl sha1 -sha256 ${HELM_TMP_FILE} | awk '{print $2}')
local expected_sum=$(cat ${HELM_SUM_FILE})
if [ "$sum" != "$expected_sum" ]; then
echo "SHA sum of ${HELM_TMP_FILE} does not match. Aborting."
exit 1
fi
echo "Done."
}
# verifySignatures obtains the latest KEYS file from GitHub main branch
# as well as the signature .asc files from the specific GitHub release,
# then verifies that the release artifacts were signed by a maintainer's key.
verifySignatures() {
printf "Verifying signatures... "
local keys_filename="KEYS"
local github_keys_url="https://raw.githubusercontent.com/helm/helm/main/${keys_filename}"
if [ "${HAS_CURL}" == "true" ]; then
curl -SsL "${github_keys_url}" -o "${HELM_TMP_ROOT}/${keys_filename}"
elif [ "${HAS_WGET}" == "true" ]; then
wget -q -O "${HELM_TMP_ROOT}/${keys_filename}" "${github_keys_url}"
fi
local gpg_keyring="${HELM_TMP_ROOT}/keyring.gpg"
local gpg_homedir="${HELM_TMP_ROOT}/gnupg"
mkdir -p -m 0700 "${gpg_homedir}"
local gpg_stderr_device="/dev/null"
if [ "${DEBUG}" == "true" ]; then
gpg_stderr_device="/dev/stderr"
fi
gpg --batch --quiet --homedir="${gpg_homedir}" --import "${HELM_TMP_ROOT}/${keys_filename}" 2> "${gpg_stderr_device}"
gpg --batch --no-default-keyring --keyring "${gpg_homedir}/${GPG_PUBRING}" --export > "${gpg_keyring}"
local github_release_url="https://github.com/helm/helm/releases/download/${TAG}"
if [ "${HAS_CURL}" == "true" ]; then
curl -SsL "${github_release_url}/helm-${TAG}-${OS}-${ARCH}.tar.gz.sha256.asc" -o "${HELM_TMP_ROOT}/helm-${TAG}-${OS}-${ARCH}.tar.gz.sha256.asc"
curl -SsL "${github_release_url}/helm-${TAG}-${OS}-${ARCH}.tar.gz.asc" -o "${HELM_TMP_ROOT}/helm-${TAG}-${OS}-${ARCH}.tar.gz.asc"
elif [ "${HAS_WGET}" == "true" ]; then
wget -q -O "${HELM_TMP_ROOT}/helm-${TAG}-${OS}-${ARCH}.tar.gz.sha256.asc" "${github_release_url}/helm-${TAG}-${OS}-${ARCH}.tar.gz.sha256.asc"
wget -q -O "${HELM_TMP_ROOT}/helm-${TAG}-${OS}-${ARCH}.tar.gz.asc" "${github_release_url}/helm-${TAG}-${OS}-${ARCH}.tar.gz.asc"
fi
local error_text="If you think this might be a potential security issue,"
error_text="${error_text}\nplease see here: https://github.com/helm/community/blob/master/SECURITY.md"
local num_goodlines_sha=$(gpg --verify --keyring="${gpg_keyring}" --status-fd=1 "${HELM_TMP_ROOT}/helm-${TAG}-${OS}-${ARCH}.tar.gz.sha256.asc" 2> "${gpg_stderr_device}" | grep -c -E '^\[GNUPG:\] (GOODSIG|VALIDSIG)')
if [[ ${num_goodlines_sha} -lt 2 ]]; then
echo "Unable to verify the signature of helm-${TAG}-${OS}-${ARCH}.tar.gz.sha256!"
echo -e "${error_text}"
exit 1
fi
local num_goodlines_tar=$(gpg --verify --keyring="${gpg_keyring}" --status-fd=1 "${HELM_TMP_ROOT}/helm-${TAG}-${OS}-${ARCH}.tar.gz.asc" 2> "${gpg_stderr_device}" | grep -c -E '^\[GNUPG:\] (GOODSIG|VALIDSIG)')
if [[ ${num_goodlines_tar} -lt 2 ]]; then
echo "Unable to verify the signature of helm-${TAG}-${OS}-${ARCH}.tar.gz!"
echo -e "${error_text}"
exit 1
fi
echo "Done."
}
# fail_trap is executed if an error occurs.
fail_trap() {
result=$?
if [ "$result" != "0" ]; then
if [[ -n "$INPUT_ARGUMENTS" ]]; then
echo "Failed to install $BINARY_NAME with the arguments provided: $INPUT_ARGUMENTS"
help
else
echo "Failed to install $BINARY_NAME"
fi
echo -e "\tFor support, go to https://github.com/helm/helm."
fi
cleanup
exit $result
}
# testVersion tests the installed client to make sure it is working.
testVersion() {
set +e
HELM="$(command -v $BINARY_NAME)"
if [ "$?" = "1" ]; then
echo "$BINARY_NAME not found. Is $HELM_INSTALL_DIR on your "'$PATH?'
exit 1
fi
set -e
}
# help provides possible cli installation arguments
help () {
echo "Accepted cli arguments are:"
echo -e "\t[--help|-h ] ->> prints this help"
echo -e "\t[--version|-v <desired_version>] . When not defined it fetches the latest release tag from the Helm CDN"
echo -e "\te.g. --version v3.0.0 or -v canary"
echo -e "\t[--no-sudo] ->> install without sudo"
}
# cleanup temporary files to avoid https://github.com/helm/helm/issues/2977
cleanup() {
if [[ -d "${HELM_TMP_ROOT:-}" ]]; then
rm -rf "$HELM_TMP_ROOT"
fi
}
# Execution
#Stop execution on any error
trap "fail_trap" EXIT
set -e
# Set debug if desired
if [ "${DEBUG}" == "true" ]; then
set -x
fi
# Parsing input arguments (if any)
export INPUT_ARGUMENTS="${@}"
set -u
while [[ $# -gt 0 ]]; do
case $1 in
'--version'|-v)
shift
if [[ $# -ne 0 ]]; then
export DESIRED_VERSION="${1}"
if [[ "$1" != "v"* ]]; then
echo "Expected version arg ('${DESIRED_VERSION}') to begin with 'v', fixing..."
export DESIRED_VERSION="v${1}"
fi
else
echo -e "Please provide the desired version. e.g. --version v3.0.0 or -v canary"
exit 0
fi
;;
'--no-sudo')
USE_SUDO="false"
;;
'--help'|-h)
help
exit 0
;;
*) exit 1
;;
esac
shift
done
set +u
initArch
initOS
verifySupported
checkDesiredVersion
if ! checkHelmInstalledVersion; then
downloadFile
verifyFile
installFile
fi
testVersion
cleanup

View File

@@ -0,0 +1,32 @@
---
# Service Account and RBAC for Dashboard admin access
apiVersion: v1
kind: ServiceAccount
metadata:
name: dashboard-admin
namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: dashboard-admin
subjects:
- kind: ServiceAccount
name: dashboard-admin
namespace: kubernetes-dashboard
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
# Token for dashboard-admin
apiVersion: v1
kind: Secret
metadata:
name: dashboard-admin-token
namespace: kubernetes-dashboard
annotations:
kubernetes.io/service-account.name: dashboard-admin
type: kubernetes.io/service-account-token

View File

@@ -0,0 +1,84 @@
---
# Internal-only middleware
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: internal-only
namespace: kubernetes-dashboard
spec:
ipWhiteList:
# Restrict to local private network ranges
sourceRange:
- 127.0.0.1/32 # localhost
- 10.0.0.0/8 # Private network
- 172.16.0.0/12 # Private network
- 192.168.0.0/16 # Private network
---
# HTTPS redirect middleware
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: dashboard-redirect-scheme
namespace: kubernetes-dashboard
spec:
redirectScheme:
scheme: https
permanent: true
---
# IngressRoute for Dashboard
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: kubernetes-dashboard-https
namespace: kubernetes-dashboard
spec:
entryPoints:
- websecure
routes:
- match: Host(`dashboard.internal.${DOMAIN}`)
kind: Rule
middlewares:
- name: internal-only
namespace: kubernetes-dashboard
services:
- name: kubernetes-dashboard
port: 443
serversTransport: dashboard-transport
tls:
secretName: wildcard-internal-wild-cloud-tls
---
# HTTP to HTTPS redirect.
# FIXME: Is this needed?
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: kubernetes-dashboard-http
namespace: kubernetes-dashboard
spec:
entryPoints:
- web
routes:
- match: Host(`dashboard.internal.${DOMAIN}`)
kind: Rule
middlewares:
- name: dashboard-redirect-scheme
namespace: kubernetes-dashboard
services:
- name: kubernetes-dashboard
port: 443
serversTransport: dashboard-transport
---
# ServersTransport for HTTPS backend with skip verify.
# FIXME: Is this needed?
apiVersion: traefik.containo.us/v1alpha1
kind: ServersTransport
metadata:
name: dashboard-transport
namespace: kubernetes-dashboard
spec:
insecureSkipVerify: true
serverName: dashboard.internal.${DOMAIN}

View File

@@ -0,0 +1,20 @@
# Longhorn Storage
See: [Longhorn Docs v 1.8.1](https://longhorn.io/docs/1.8.1/deploy/install/install-with-kubectl/)
## Installation Notes
- Manifest copied from https://raw.githubusercontent.com/longhorn/longhorn/v1.8.1/deploy/longhorn.yaml
- Using kustomize to apply custom configuration (see `kustomization.yaml`)
## Important Settings
- **Number of Replicas**: Set to 1 (default is 3) to accommodate smaller clusters
- This avoids "degraded" volumes when fewer than 3 nodes are available
- For production with 3+ nodes, consider changing back to 3 for better availability
## Common Operations
- View volumes: `kubectl get volumes.longhorn.io -n longhorn-system`
- Check volume status: `kubectl describe volumes.longhorn.io <volume-name> -n longhorn-system`
- Access Longhorn UI: Set up port-forwarding with `kubectl -n longhorn-system port-forward service/longhorn-frontend 8080:80`

View File

@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- longhorn.yaml

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,18 @@
namespace: metallb-system
resources:
- pool.yaml
configMapGenerator:
- name: metallb-config
envs:
- config/config.env
replacements:
- source:
kind: ConfigMap
name: metallb-config
fieldPath: data.CLUSTER_LOAD_BALANCER_RANGE
targets:
- select:
kind: IPAddressPool
name: first-pool
fieldPaths:
- spec.addresses.0

View File

@@ -0,0 +1,19 @@
---
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
name: first-pool
namespace: metallb-system
spec:
addresses:
- PLACEHOLDER_CLUSTER_LOAD_BALANCER_RANGE
---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
name: l2-advertisement
namespace: metallb-system
spec:
ipAddressPools:
- first-pool

View File

@@ -0,0 +1,3 @@
namespace: metallb-system
resources:
- github.com/metallb/metallb/config/native?ref=v0.15.0

View File

@@ -0,0 +1,53 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- persistent-volume.yaml
- storage-class.yaml
replacements:
- source:
kind: ConfigMap
name: nfs-config
fieldPath: data.NFS_HOST_IP
targets:
- select:
kind: PersistentVolume
name: nfs-media-pv
fieldPaths:
- spec.nfs.server
- select:
kind: StorageClass
name: nfs
fieldPaths:
- parameters.server
- source:
kind: ConfigMap
name: nfs-config
fieldPath: data.NFS_MEDIA_PATH
targets:
- select:
kind: PersistentVolume
name: nfs-media-pv
fieldPaths:
- spec.nfs.path
- select:
kind: StorageClass
name: nfs
fieldPaths:
- parameters.path
- source:
kind: ConfigMap
name: nfs-config
fieldPath: data.NFS_STORAGE_CAPACITY
targets:
- select:
kind: PersistentVolume
name: nfs-media-pv
fieldPaths:
- spec.capacity.storage
configMapGenerator:
- name: nfs-config
envs:
- config/config.env

View File

@@ -0,0 +1,23 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-media-pv
labels:
storage: nfs-media
spec:
capacity:
storage: REPLACE_ME
accessModes:
- ReadWriteMany
persistentVolumeReclaimPolicy: Retain
storageClassName: nfs
nfs:
server: REPLACE_ME
path: REPLACE_ME
mountOptions:
- nfsvers=4.1
- rsize=1048576
- wsize=1048576
- hard
- intr
- timeo=600

View File

@@ -0,0 +1,10 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nfs
provisioner: nfs
parameters:
server: REPLACE_ME
path: REPLACE_ME
reclaimPolicy: Retain
allowVolumeExpansion: true

55
setup/cluster/setup-all.sh Executable file
View File

@@ -0,0 +1,55 @@
#!/bin/bash
set -e
# Navigate to script directory
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
echo "Setting up infrastructure components for k3s..."
# Make all script files executable
chmod +x *.sh
# Utils
./setup-utils.sh
# Setup MetalLB (must be first for IP allocation)
./setup-metallb.sh
# Setup Longhorn
./setup-longhorn.sh
# Setup Traefik
./setup-traefik.sh
# Setup CoreDNS
./setup-coredns.sh
# Setup cert-manager
./setup-cert-manager.sh
# Setup ExternalDNS
./setup-externaldns.sh
# Setup Kubernetes Dashboard
./setup-dashboard.sh
# Setup NFS Kubernetes integration (optional)
./setup-nfs.sh
# Setup Docker Registry
./setup-registry.sh
echo "Infrastructure setup complete!"
echo
echo "Next steps:"
echo "1. Install Helm charts for non-infrastructure components"
echo "2. Access the dashboard at: https://dashboard.internal.${DOMAIN}"
echo "3. Get the dashboard token with: ./bin/dashboard-token"
echo
echo "To verify components, run:"
echo "- kubectl get pods -n cert-manager"
echo "- kubectl get pods -n externaldns"
echo "- kubectl get pods -n kubernetes-dashboard"
echo "- kubectl get clusterissuers"

View File

@@ -0,0 +1,93 @@
#!/bin/bash
set -e
# Navigate to script directory
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
# Source environment variables
if [[ -f "../load-env.sh" ]]; then
source ../load-env.sh
fi
echo "Setting up cert-manager..."
# Create cert-manager namespace
kubectl create namespace cert-manager --dry-run=client -o yaml | kubectl apply -f -
# Install cert-manager using the official installation method
# This installs CRDs, controllers, and webhook components
echo "Installing cert-manager components..."
# Using stable URL for cert-manager installation
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.1/cert-manager.yaml || \
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.13.1/cert-manager.yaml
# Wait for cert-manager to be ready
echo "Waiting for cert-manager to be ready..."
kubectl wait --for=condition=Available deployment/cert-manager -n cert-manager --timeout=120s
kubectl wait --for=condition=Available deployment/cert-manager-cainjector -n cert-manager --timeout=120s
kubectl wait --for=condition=Available deployment/cert-manager-webhook -n cert-manager --timeout=120s
# Add delay to allow webhook to be fully ready
echo "Waiting additional time for cert-manager webhook to be fully operational..."
sleep 30
# Setup Cloudflare API token for DNS01 challenges
if [[ -n "${CLOUDFLARE_API_TOKEN}" ]]; then
echo "Creating Cloudflare API token secret in cert-manager namespace..."
kubectl create secret generic cloudflare-api-token \
--namespace cert-manager \
--from-literal=api-token="${CLOUDFLARE_API_TOKEN}" \
--dry-run=client -o yaml | kubectl apply -f -
else
echo "Warning: CLOUDFLARE_API_TOKEN not set. DNS01 challenges will not work."
fi
echo "Creating Let's Encrypt issuers..."
cat ${SCRIPT_DIR}/cert-manager/letsencrypt-staging-dns01.yaml | envsubst | kubectl apply -f -
cat ${SCRIPT_DIR}/cert-manager/letsencrypt-prod-dns01.yaml | envsubst | kubectl apply -f -
# Wait for issuers to be ready
echo "Waiting for Let's Encrypt issuers to be ready..."
sleep 10
# Configure cert-manager to use external DNS for challenge verification
echo "Configuring cert-manager to use external DNS servers..."
kubectl patch deployment cert-manager -n cert-manager --patch '
spec:
template:
spec:
dnsPolicy: None
dnsConfig:
nameservers:
- "1.1.1.1"
- "8.8.8.8"
searches:
- cert-manager.svc.cluster.local
- svc.cluster.local
- cluster.local
options:
- name: ndots
value: "5"'
# Wait for cert-manager to restart with new DNS config
echo "Waiting for cert-manager to restart with new DNS configuration..."
kubectl rollout status deployment/cert-manager -n cert-manager --timeout=120s
# Apply wildcard certificates
echo "Creating wildcard certificates..."
cat ${SCRIPT_DIR}/cert-manager/internal-wildcard-certificate.yaml | envsubst | kubectl apply -f -
cat ${SCRIPT_DIR}/cert-manager/wildcard-certificate.yaml | envsubst | kubectl apply -f -
echo "Wildcard certificate creation initiated. This may take some time to complete depending on DNS propagation."
# Wait for the certificates to be issued (with a timeout)
echo "Waiting for wildcard certificates to be ready (this may take several minutes)..."
kubectl wait --for=condition=Ready certificate wildcard-internal-wild-cloud -n cert-manager --timeout=300s || true
kubectl wait --for=condition=Ready certificate wildcard-wild-cloud -n cert-manager --timeout=300s || true
echo "cert-manager setup complete!"
echo ""
echo "To verify the installation:"
echo " kubectl get pods -n cert-manager"
echo " kubectl get clusterissuers"

30
setup/cluster/setup-coredns.sh Executable file
View File

@@ -0,0 +1,30 @@
#!/bin/bash
set -e
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
# Source environment variables
if [[ -f "../load-env.sh" ]]; then
source ../load-env.sh
fi
echo "Setting up CoreDNS for k3s..."
echo "Script directory: ${SCRIPT_DIR}"
echo "Current directory: $(pwd)"
# Apply the k3s-compatible custom DNS override (k3s will preserve this)
echo "Applying CoreDNS custom override configuration..."
cat "${SCRIPT_DIR}/coredns/coredns-custom-config.yaml" | envsubst | kubectl apply -f -
# Apply the LoadBalancer service for external access to CoreDNS
echo "Applying CoreDNS service configuration..."
cat "${SCRIPT_DIR}/coredns/coredns-lb-service.yaml" | envsubst | kubectl apply -f -
# Restart CoreDNS pods to apply the changes
echo "Restarting CoreDNS pods to apply changes..."
kubectl rollout restart deployment/coredns -n kube-system
kubectl rollout status deployment/coredns -n kube-system
echo "CoreDNS setup complete!"

View File

@@ -0,0 +1,46 @@
#!/bin/bash
set -e
# Store the script directory path for later use
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
# Source environment variables
if [[ -f "../load-env.sh" ]]; then
source ../load-env.sh
fi
echo "Setting up Kubernetes Dashboard..."
NAMESPACE="kubernetes-dashboard"
# Apply the official dashboard installation
echo "Installing Kubernetes Dashboard core components..."
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
# Copying cert-manager secrets to the dashboard namespace
copy-secret cert-manager:wildcard-internal-wild-cloud-tls $NAMESPACE
copy-secret cert-manager:wildcard-wild-cloud-tls $NAMESPACE
# Create admin service account and token
echo "Creating dashboard admin service account and token..."
cat "${SCRIPT_DIR}/kubernetes-dashboard/dashboard-admin-rbac.yaml" | kubectl apply -f -
# Apply the dashboard configuration
echo "Applying dashboard configuration..."
cat "${SCRIPT_DIR}/kubernetes-dashboard/dashboard-kube-system.yaml" | envsubst | kubectl apply -f -
# Restart CoreDNS to pick up the changes
kubectl delete pods -n kube-system -l k8s-app=kube-dns
echo "Restarted CoreDNS to pick up DNS changes"
# Wait for dashboard to be ready
echo "Waiting for Kubernetes Dashboard to be ready..."
kubectl rollout status deployment/kubernetes-dashboard -n $NAMESPACE --timeout=60s
echo "Kubernetes Dashboard setup complete!"
echo "Access the dashboard at: https://dashboard.internal.${DOMAIN}"
echo ""
echo "To get the authentication token, run:"
echo "./bin/dashboard-token"

View File

@@ -0,0 +1,51 @@
#!/bin/bash
set -e
# Navigate to script directory
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
# Source environment variables
if [[ -f "../load-env.sh" ]]; then
source ../load-env.sh
fi
echo "Setting up ExternalDNS..."
# Create externaldns namespace
kubectl create namespace externaldns --dry-run=client -o yaml | kubectl apply -f -
# Setup Cloudflare API token secret for ExternalDNS
if [[ -n "${CLOUDFLARE_API_TOKEN}" ]]; then
echo "Creating Cloudflare API token secret..."
kubectl create secret generic cloudflare-api-token \
--namespace externaldns \
--from-literal=api-token="${CLOUDFLARE_API_TOKEN}" \
--dry-run=client -o yaml | kubectl apply -f -
else
echo "Error: CLOUDFLARE_API_TOKEN not set. ExternalDNS will not work correctly."
exit 1
fi
# Apply common RBAC resources
echo "Deploying ExternalDNS RBAC resources..."
cat ${SCRIPT_DIR}/externaldns/externaldns-rbac.yaml | envsubst | kubectl apply -f -
# Apply ExternalDNS manifests with environment variables
echo "Deploying ExternalDNS for external DNS (Cloudflare)..."
cat ${SCRIPT_DIR}/externaldns/externaldns-cloudflare.yaml | envsubst | kubectl apply -f -
# Wait for ExternalDNS to be ready
echo "Waiting for Cloudflare ExternalDNS to be ready..."
kubectl rollout status deployment/external-dns -n externaldns --timeout=60s
# echo "Waiting for CoreDNS ExternalDNS to be ready..."
# kubectl rollout status deployment/external-dns-coredns -n externaldns --timeout=60s
echo "ExternalDNS setup complete!"
echo ""
echo "To verify the installation:"
echo " kubectl get pods -n externaldns"
echo " kubectl logs -n externaldns -l app=external-dns -f"
echo " kubectl logs -n externaldns -l app=external-dns-coredns -f"

16
setup/cluster/setup-longhorn.sh Executable file
View File

@@ -0,0 +1,16 @@
#!/bin/bash
set -e
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
if [[ -f "../load-env.sh" ]]; then
source ../load-env.sh
fi
echo "Setting up Longhorn..."
# Apply Longhorn with kustomize to apply our customizations
kubectl apply -k ${SCRIPT_DIR}/longhorn/
echo "Longhorn setup complete!"

30
setup/cluster/setup-metallb.sh Executable file
View File

@@ -0,0 +1,30 @@
#!/bin/bash
set -e
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
# Source environment variables
if [[ -f "../load-env.sh" ]]; then
source ../load-env.sh
fi
echo "Setting up MetalLB..."
echo "Deploying MetalLB..."
# cat ${SCRIPT_DIR}/metallb/metallb-helm-config.yaml | envsubst | kubectl apply -f -
kubectl apply -k metallb/installation
echo "Waiting for MetalLB to be deployed..."
kubectl wait --for=condition=Available deployment/controller -n metallb-system --timeout=60s
sleep 10 # Extra buffer for webhook initialization
echo "Customizing MetalLB..."
kubectl apply -k metallb/configuration
echo "✅ MetalLB installed and configured"
echo ""
echo "To verify the installation:"
echo " kubectl get pods -n metallb-system"
echo " kubectl get ipaddresspools.metallb.io -n metallb-system"

257
setup/cluster/setup-nfs-host.sh Executable file
View File

@@ -0,0 +1,257 @@
#!/bin/bash
set -e
set -o pipefail
# Navigate to script directory
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
# Source environment variables
source "${PROJECT_DIR}/load-env.sh"
echo "Setting up NFS server on this host..."
# Check if required NFS variables are configured
if [[ -z "${NFS_HOST}" ]]; then
echo "NFS_HOST not set. Please set NFS_HOST=<hostname> in your environment"
echo "Example: export NFS_HOST=box-01"
exit 1
fi
# Ensure NFS_MEDIA_PATH is explicitly set
if [[ -z "${NFS_MEDIA_PATH}" ]]; then
echo "Error: NFS_MEDIA_PATH not set. Please set it in your environment"
echo "Example: export NFS_MEDIA_PATH=/data/media"
exit 1
fi
# Set default for NFS_EXPORT_OPTIONS if not already set
if [[ -z "${NFS_EXPORT_OPTIONS}" ]]; then
export NFS_EXPORT_OPTIONS="*(rw,sync,no_subtree_check,no_root_squash)"
echo "Using default NFS_EXPORT_OPTIONS: ${NFS_EXPORT_OPTIONS}"
fi
echo "Target NFS host: ${NFS_HOST}"
echo "Media path: ${NFS_MEDIA_PATH}"
echo "Export options: ${NFS_EXPORT_OPTIONS}"
# Function to check if we're running on the correct host
check_host() {
local current_hostname=$(hostname)
if [[ "${current_hostname}" != "${NFS_HOST}" ]]; then
echo "Warning: Current host (${current_hostname}) differs from NFS_HOST (${NFS_HOST})"
echo "This script should be run on ${NFS_HOST}"
read -p "Continue anyway? (y/N): " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
exit 1
fi
fi
}
# Function to install NFS server and SMB/CIFS
install_nfs_server() {
echo "Installing NFS server and SMB/CIFS packages..."
# Detect package manager and install NFS server + Samba
if command -v apt-get >/dev/null 2>&1; then
# Debian/Ubuntu
sudo apt-get update
sudo apt-get install -y nfs-kernel-server nfs-common samba samba-common-bin
elif command -v yum >/dev/null 2>&1; then
# RHEL/CentOS
sudo yum install -y nfs-utils samba samba-client
elif command -v dnf >/dev/null 2>&1; then
# Fedora
sudo dnf install -y nfs-utils samba samba-client
else
echo "Error: Unable to detect package manager. Please install NFS server and Samba manually."
exit 1
fi
}
# Function to create media directory
create_media_directory() {
echo "Creating media directory: ${NFS_MEDIA_PATH}"
# Create directory if it doesn't exist
sudo mkdir -p "${NFS_MEDIA_PATH}"
# Set appropriate permissions
# Using 755 for directory, allowing read/execute for all, write for owner
sudo chmod 755 "${NFS_MEDIA_PATH}"
echo "Media directory created with appropriate permissions"
echo "Directory info:"
ls -la "${NFS_MEDIA_PATH}/"
}
# Function to configure NFS exports
configure_nfs_exports() {
echo "Configuring NFS exports..."
local export_line="${NFS_MEDIA_PATH} ${NFS_EXPORT_OPTIONS}"
local exports_file="/etc/exports"
# Backup existing exports file
sudo cp "${exports_file}" "${exports_file}.backup.$(date +%Y%m%d-%H%M%S)" 2>/dev/null || true
# Check if export already exists
if sudo grep -q "^${NFS_MEDIA_PATH}" "${exports_file}" 2>/dev/null; then
echo "Export for ${NFS_MEDIA_PATH} already exists, updating..."
sudo sed -i "s|^${NFS_MEDIA_PATH}.*|${export_line}|" "${exports_file}"
else
echo "Adding new export for ${NFS_MEDIA_PATH}..."
echo "${export_line}" | sudo tee -a "${exports_file}"
fi
# Export the filesystems
sudo exportfs -rav
echo "NFS exports configured:"
sudo exportfs -v
}
# Function to start and enable NFS services
start_nfs_services() {
echo "Starting NFS services..."
# Start and enable NFS server
sudo systemctl enable nfs-server
sudo systemctl start nfs-server
# Also enable related services
sudo systemctl enable rpcbind
sudo systemctl start rpcbind
echo "NFS services started and enabled"
# Show service status
sudo systemctl status nfs-server --no-pager --lines=5
}
# Function to configure SMB/CIFS sharing
configure_smb_sharing() {
echo "Configuring SMB/CIFS sharing..."
local smb_config="/etc/samba/smb.conf"
local share_name="media"
# Backup existing config
sudo cp "${smb_config}" "${smb_config}.backup.$(date +%Y%m%d-%H%M%S)" 2>/dev/null || true
# Check if share already exists
if sudo grep -q "^\[${share_name}\]" "${smb_config}" 2>/dev/null; then
echo "SMB share '${share_name}' already exists, updating..."
# Remove existing share section
sudo sed -i "/^\[${share_name}\]/,/^\[/{ /^\[${share_name}\]/d; /^\[/!d; }" "${smb_config}"
fi
# Add media share configuration
cat << EOF | sudo tee -a "${smb_config}"
[${share_name}]
comment = Media files for Jellyfin
path = ${NFS_MEDIA_PATH}
browseable = yes
read only = no
guest ok = yes
create mask = 0664
directory mask = 0775
force user = $(whoami)
force group = $(whoami)
EOF
echo "SMB share configuration added"
# Test configuration
if sudo testparm -s >/dev/null 2>&1; then
echo "✓ SMB configuration is valid"
else
echo "✗ SMB configuration has errors"
sudo testparm
exit 1
fi
}
# Function to start SMB services
start_smb_services() {
echo "Starting SMB services..."
# Enable and start Samba services
sudo systemctl enable smbd
sudo systemctl start smbd
sudo systemctl enable nmbd
sudo systemctl start nmbd
echo "SMB services started and enabled"
# Show service status
sudo systemctl status smbd --no-pager --lines=3
}
# Function to test NFS setup
test_nfs_setup() {
echo "Testing NFS setup..."
# Test if NFS is responding
if command -v showmount >/dev/null 2>&1; then
echo "Available NFS exports:"
showmount -e localhost || echo "Warning: showmount failed, but NFS may still be working"
fi
# Check if the export directory is accessible
if [[ -d "${NFS_MEDIA_PATH}" ]]; then
echo "✓ Media directory exists and is accessible"
else
echo "✗ Media directory not accessible"
exit 1
fi
}
# Function to show usage instructions
show_usage_instructions() {
echo
echo "=== NFS/SMB Host Setup Complete ==="
echo
echo "NFS and SMB servers are now running on this host with media directory: ${NFS_MEDIA_PATH}"
echo
echo "Access methods:"
echo "1. NFS (for Kubernetes): Use setup-nfs-k8s.sh to register with cluster"
echo "2. SMB/CIFS (for Windows): \\\\${NFS_HOST}\\media"
echo
echo "To add media files:"
echo "- Copy directly to: ${NFS_MEDIA_PATH}"
echo "- Or mount SMB share from Windows and copy there"
echo
echo "Windows SMB mount:"
echo "- Open File Explorer"
echo "- Map network drive to: \\\\${NFS_HOST}\\media"
echo "- Or use: \\\\$(hostname -I | awk '{print $1}')\\media"
echo
echo "To verify services:"
echo "- NFS: showmount -e ${NFS_HOST}"
echo "- SMB: smbclient -L ${NFS_HOST} -N"
echo "- Status: systemctl status nfs-server smbd"
echo
echo "Current NFS exports:"
sudo exportfs -v
echo
}
# Main execution
main() {
check_host
install_nfs_server
create_media_directory
configure_nfs_exports
start_nfs_services
configure_smb_sharing
start_smb_services
test_nfs_setup
show_usage_instructions
}
# Run main function
main "$@"

230
setup/cluster/setup-nfs.sh Executable file
View File

@@ -0,0 +1,230 @@
#!/bin/bash
set -e
set -o pipefail
# Navigate to script directory
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
# Source environment variables
source "${PROJECT_DIR}/load-env.sh"
echo "Registering NFS server with Kubernetes cluster..."
# Check if NFS_HOST is configured
if [[ -z "${NFS_HOST}" ]]; then
echo "NFS_HOST not set. Skipping NFS Kubernetes setup."
echo "To enable NFS media sharing:"
echo "1. Set NFS_HOST=<hostname> in your environment"
echo "2. Run setup-nfs-host.sh on the NFS host"
echo "3. Re-run this script"
exit 0
fi
# Set default for NFS_STORAGE_CAPACITY if not already set
if [[ -z "${NFS_STORAGE_CAPACITY}" ]]; then
export NFS_STORAGE_CAPACITY="250Gi"
echo "Using default NFS_STORAGE_CAPACITY: ${NFS_STORAGE_CAPACITY}"
fi
echo "NFS host: ${NFS_HOST}"
echo "Media path: ${NFS_MEDIA_PATH}"
echo "Storage capacity: ${NFS_STORAGE_CAPACITY}"
# Function to resolve NFS host to IP
resolve_nfs_host() {
if [[ "${NFS_HOST}" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
# NFS_HOST is already an IP address
NFS_HOST_IP="${NFS_HOST}"
else
# Resolve hostname to IP
NFS_HOST_IP=$(getent hosts "${NFS_HOST}" | awk '{print $1}' | head -n1)
if [[ -z "${NFS_HOST_IP}" ]]; then
echo "Error: Unable to resolve hostname ${NFS_HOST} to IP address"
echo "Make sure ${NFS_HOST} is resolvable from this cluster"
exit 1
fi
# Check if resolved IP is localhost - auto-detect network IP instead
if [[ "${NFS_HOST_IP}" =~ ^127\. ]]; then
echo "Warning: ${NFS_HOST} resolves to localhost (${NFS_HOST_IP})"
echo "Auto-detecting network IP for cluster access..."
# Try to find the primary network interface IP (exclude docker/k8s networks)
local network_ip=$(ip route get 8.8.8.8 | grep -oP 'src \K\S+' 2>/dev/null)
if [[ -n "${network_ip}" && ! "${network_ip}" =~ ^127\. ]]; then
echo "Using detected network IP: ${network_ip}"
NFS_HOST_IP="${network_ip}"
else
echo "Could not auto-detect network IP. Available IPs:"
ip addr show | grep "inet " | grep -v "127.0.0.1" | grep -v "10.42" | grep -v "172." | awk '{print " " $2}' | cut -d/ -f1
echo "Please set NFS_HOST to the correct IP address manually."
exit 1
fi
fi
fi
echo "NFS server IP: ${NFS_HOST_IP}"
export NFS_HOST_IP
}
# Function to test NFS accessibility
test_nfs_accessibility() {
echo "Testing NFS accessibility from cluster..."
# Check if showmount is available
if ! command -v showmount >/dev/null 2>&1; then
echo "Installing NFS client tools..."
if command -v apt-get >/dev/null 2>&1; then
sudo apt-get update && sudo apt-get install -y nfs-common
elif command -v yum >/dev/null 2>&1; then
sudo yum install -y nfs-utils
elif command -v dnf >/dev/null 2>&1; then
sudo dnf install -y nfs-utils
else
echo "Warning: Unable to install NFS client tools. Skipping accessibility test."
return 0
fi
fi
# Test if we can reach the NFS server
echo "Testing connection to NFS server..."
if timeout 10 showmount -e "${NFS_HOST_IP}" >/dev/null 2>&1; then
echo "✓ NFS server is accessible"
echo "Available exports:"
showmount -e "${NFS_HOST_IP}"
else
echo "✗ Cannot connect to NFS server at ${NFS_HOST_IP}"
echo "Make sure:"
echo "1. NFS server is running on ${NFS_HOST}"
echo "2. Network connectivity exists between cluster and NFS host"
echo "3. Firewall allows NFS traffic (port 2049)"
exit 1
fi
# Test specific export
if showmount -e "${NFS_HOST_IP}" | grep -q "${NFS_MEDIA_PATH}"; then
echo "✓ Media path ${NFS_MEDIA_PATH} is exported"
else
echo "✗ Media path ${NFS_MEDIA_PATH} is not found in exports"
echo "Available exports:"
showmount -e "${NFS_HOST_IP}"
echo
echo "Run setup-nfs-host.sh on ${NFS_HOST} to configure the export"
exit 1
fi
}
# Function to create test mount
test_nfs_mount() {
echo "Testing NFS mount functionality..."
local test_mount="/tmp/nfs-test-$$"
mkdir -p "${test_mount}"
# Try to mount the NFS export
if timeout 30 sudo mount -t nfs4 "${NFS_HOST_IP}:${NFS_MEDIA_PATH}" "${test_mount}"; then
echo "✓ NFS mount successful"
# Test read access
if ls "${test_mount}" >/dev/null 2>&1; then
echo "✓ NFS read access working"
else
echo "✗ NFS read access failed"
fi
# Unmount
sudo umount "${test_mount}" || echo "Warning: Failed to unmount test directory"
else
echo "✗ NFS mount failed"
echo "Check NFS server configuration and network connectivity"
exit 1
fi
# Clean up
rmdir "${test_mount}" 2>/dev/null || true
}
# Function to create Kubernetes resources
create_k8s_resources() {
echo "Creating Kubernetes NFS resources..."
# Generate config file with resolved variables
local nfs_dir="${SCRIPT_DIR}/nfs"
local env_file="${nfs_dir}/config/.env"
local config_file="${nfs_dir}/config/config.env"
echo "Generating NFS configuration..."
export NFS_HOST_IP
export NFS_MEDIA_PATH
export NFS_STORAGE_CAPACITY
envsubst < "${env_file}" > "${config_file}"
# Apply the NFS Kubernetes manifests using kustomize
echo "Applying NFS manifests from: ${nfs_dir}"
kubectl apply -k "${nfs_dir}"
echo "✓ NFS PersistentVolume and StorageClass created"
# Verify resources were created
echo "Verifying Kubernetes resources..."
if kubectl get storageclass nfs >/dev/null 2>&1; then
echo "✓ StorageClass 'nfs' created"
else
echo "✗ StorageClass 'nfs' not found"
exit 1
fi
if kubectl get pv nfs-media-pv >/dev/null 2>&1; then
echo "✓ PersistentVolume 'nfs-media-pv' created"
kubectl get pv nfs-media-pv
else
echo "✗ PersistentVolume 'nfs-media-pv' not found"
exit 1
fi
}
# Function to show usage instructions
show_usage_instructions() {
echo
echo "=== NFS Kubernetes Setup Complete ==="
echo
echo "NFS server ${NFS_HOST} (${NFS_HOST_IP}) has been registered with the cluster"
echo
echo "Kubernetes resources created:"
echo "- StorageClass: nfs"
echo "- PersistentVolume: nfs-media-pv (${NFS_STORAGE_CAPACITY}, ReadWriteMany)"
echo
echo "To use NFS storage in your applications:"
echo "1. Set storageClassName: nfs in your PVC"
echo "2. Use accessMode: ReadWriteMany for shared access"
echo
echo "Example PVC:"
echo "---"
echo "apiVersion: v1"
echo "kind: PersistentVolumeClaim"
echo "metadata:"
echo " name: my-nfs-pvc"
echo "spec:"
echo " accessModes:"
echo " - ReadWriteMany"
echo " storageClassName: nfs"
echo " resources:"
echo " requests:"
echo " storage: 10Gi"
echo
}
# Main execution
main() {
resolve_nfs_host
test_nfs_accessibility
test_nfs_mount
create_k8s_resources
show_usage_instructions
}
# Run main function
main "$@"

20
setup/cluster/setup-registry.sh Executable file
View File

@@ -0,0 +1,20 @@
#!/bin/bash
set -e
# Navigate to script directory
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
echo "Setting up Docker Registry..."
# Apply the docker registry manifests using kustomize
kubectl apply -k "${SCRIPT_DIR}/docker-registry"
echo "Waiting for Docker Registry to be ready..."
kubectl wait --for=condition=available --timeout=300s deployment/docker-registry -n docker-registry
echo "Docker Registry setup complete!"
# Show deployment status
kubectl get pods -n docker-registry
kubectl get services -n docker-registry

18
setup/cluster/setup-traefik.sh Executable file
View File

@@ -0,0 +1,18 @@
#!/bin/bash
set -e
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
# Source environment variables
if [[ -f "../load-env.sh" ]]; then
source ../load-env.sh
fi
echo "Setting up Traefik service and middleware for k3s..."
cat ${SCRIPT_DIR}/traefik/traefik-service.yaml | envsubst | kubectl apply -f -
cat ${SCRIPT_DIR}/traefik/internal-middleware.yaml | envsubst | kubectl apply -f -
echo "Traefik setup complete!"

37
setup/cluster/setup-utils.sh Executable file
View File

@@ -0,0 +1,37 @@
#!/bin/bash
set -e
SCRIPT_PATH="$(realpath "${BASH_SOURCE[0]}")"
SCRIPT_DIR="$(dirname "$SCRIPT_PATH")"
cd "$SCRIPT_DIR"
# Install gomplate
if command -v gomplate &> /dev/null; then
echo "gomplate is already installed."
else
curl -sSL https://github.com/hairyhenderson/gomplate/releases/latest/download/gomplate_linux-amd64 -o $HOME/.local/bin/gomplate
chmod +x $HOME/.local/bin/gomplate
echo "gomplate installed successfully."
fi
# Install kustomize
if command -v kustomize &> /dev/null; then
echo "kustomize is already installed."
else
curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash
mv kustomize $HOME/.local/bin/
echo "kustomize installed successfully."
fi
## Install yq
if command -v yq &> /dev/null; then
echo "yq is already installed."
else
VERSION=v4.45.4
BINARY=yq_linux_amd64
wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY}.tar.gz -O - | tar xz
mv ${BINARY} $HOME/.local/bin/yq
chmod +x $HOME/.local/bin/yq
rm yq.1
echo "yq installed successfully."
fi

View File

@@ -0,0 +1,7 @@
# Traefik
- https://doc.traefik.io/traefik/providers/kubernetes-ingress/
Ingress RDs can be create for any service. The routes specificed in the Ingress are added automatically to the Traefik proxy.
Traefik serves all incoming network traffic on ports 80 and 443 to their appropriate services based on the route.

View File

@@ -0,0 +1,13 @@
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: internal-only
namespace: kube-system
spec:
ipWhiteList:
# Restrict to local private network ranges - adjust these to match your network
sourceRange:
- 127.0.0.1/32 # localhost
- 10.0.0.0/8 # Private network
- 172.16.0.0/12 # Private network
- 192.168.0.0/16 # Private network

View File

@@ -0,0 +1,29 @@
---
# Traefik service configuration with static LoadBalancer IP
apiVersion: v1
kind: Service
metadata:
name: traefik
namespace: kube-system
annotations:
# Get a stable IP from MetalLB
metallb.universe.tf/address-pool: production
metallb.universe.tf/allow-shared-ip: traefik-lb
labels:
app.kubernetes.io/instance: traefik-kube-system
app.kubernetes.io/name: traefik
spec:
type: LoadBalancer
loadBalancerIP: 192.168.8.240
selector:
app.kubernetes.io/instance: traefik-kube-system
app.kubernetes.io/name: traefik
ports:
- name: web
port: 80
targetPort: web
- name: websecure
port: 443
targetPort: websecure
externalTrafficPolicy: Local

View File

@@ -0,0 +1,71 @@
---
apiVersion: v1
kind: Namespace
metadata:
name: debug
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: netdebug
namespace: debug
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: netdebug
subjects:
- kind: ServiceAccount
name: netdebug
namespace: debug
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: netdebug
namespace: debug
labels:
app: netdebug
spec:
replicas: 1
selector:
matchLabels:
app: netdebug
template:
metadata:
labels:
app: netdebug
spec:
serviceAccountName: netdebug
containers:
- name: netdebug
image: nicolaka/netshoot:latest
command: ["/bin/bash"]
args: ["-c", "while true; do sleep 3600; done"]
resources:
limits:
cpu: 200m
memory: 256Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
privileged: true
---
apiVersion: v1
kind: Service
metadata:
name: netdebug
namespace: debug
spec:
selector:
app: netdebug
ports:
- port: 22
targetPort: 22
name: ssh
type: ClusterIP

1072
setup/cluster/validate-setup.sh Executable file

File diff suppressed because it is too large Load Diff

1
setup/dnsmasq/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
setup-bundle/

View File

@@ -0,0 +1,32 @@
# Network
PRESET_NET_CHANGE_DEFAULTS="1"
PRESET_NET_ETHERNET_ENABLED="1"
PRESET_NET_WIFI_ENABLED="0"
PRESET_NET_USE_STATIC="1"
PRESET_NET_STATIC_IP="{{ (ds "config").cloud.dns.ip }}"
PRESET_NET_STATIC_MASK="255.255.255.0"
PRESET_NET_STATIC_GATEWAY="{{ (ds "config").cloud.router.ip }}"
PRESET_NET_STATIC_DNS="{{ (ds "config").cloud.dns.ip }}"
# For example:
# PRESET_NET_STATIC_IP="192.168.8.50"
# PRESET_NET_STATIC_MASK="255.255.255.0"
# PRESET_NET_STATIC_GATEWAY="192.168.8.1"
# PRESET_NET_STATIC_DNS="1.1.1.1"
# System
# SET_LANG_BASED_ON_LOCATION="y"
# PRESET_LOCALE="en_US.UTF-8"
# PRESET_TIMEZONE="Etc/UTC"
# Root
# PRESET_ROOT_PASSWORD="RootPassword"
# PRESET_ROOT_KEY=""
# User
# PRESET_USER_NAME="armbian"
# PRESET_USER_PASSWORD="UserPassword"
# PRESET_USER_KEY=""
# PRESET_DEFAULT_REALNAME="Armbian user"
PRESET_USER_SHELL="bash"

67
setup/dnsmasq/README.md Normal file
View File

@@ -0,0 +1,67 @@
# Central dnsmasq setup
## Overview
dnsmasq solves two problems for us. It provides:
- LAN DNS w/ forwarding of internal and external cloud domains to the cluster.
- PXE for setting up cluster nodes.
### PXE Bootloading
A "PXE client" is any machine that is booting using PXE. This is a great way to set up a new cluster node.
- PXE client broadcasts a request for help across the LAN.
- Dnsmasq's DHCP service responds with an IP address and TFTP server info.
- PXE client downloads PXE's iPEXE bootloader files:
- `ipxe.efi`
- `undionly.kpxe`
- `ipxe-arm64.efi`
(`pxelinux.0`) via TFTP.
- PXE client reads the bootloader config for the correct web address and fetches the boot files:
- The kernel, `vmlinuz`.
- The initial RAM disk, `initrd`.
- The Talos image,
## Setup
- Install a Linux machine on your LAN. Record it's IP address in your `config:cloud.dns.ip`.
- Ensure it is accessible with ssh.
- Run `setup/dnsmasq/bin/create-setup-bundle.sh`
- Run `setup/dnsmasq/bin/transfer-setup-bundle.sh`
Now ssh into your dnsmasq machine and do the following:
```bash
sudo -i
cd dnsmasq-setup
./setup.sh
```
## Future setup
To provide a better user experience, we have been creating a debian apt package that also does this while providing a UI.
Development repo: https://github.com/civil-society-dev/wild-central
The setup will look something like this:
```bash
# Download and install GPG key
curl -fsSL https://mywildcloud.org/apt/wild-cloud-central.gpg | sudo tee /usr/share/keyrings/wild-cloud-central-archive-keyring.gpg > /dev/null
# Add repository (modern .sources format)
sudo tee /etc/apt/sources.list.d/wild-cloud-central.sources << 'EOF'
Types: deb
URIs: https://mywildcloud.org/apt
Suites: stable
Components: main
Signed-By: /usr/share/keyrings/wild-cloud-central-archive-keyring.gpg
EOF
# Update and install
sudo apt update
sudo apt install wild-cloud-central
```
browse to `http://localhost:5050`!

View File

@@ -0,0 +1,7 @@
customization:
extraKernelArgs:
- net.ifnames=0
systemExtensions:
officialExtensions:
- siderolabs/gvisor
- siderolabs/intel-ucode

View File

@@ -0,0 +1,63 @@
#!/bin/bash
if [ ! -d ".wildcloud" ]; then
echo "Error: You must run this script from a wild-cloud directory"
exit 1
fi
WILDCLOUD_ROOT=$(wild-config wildcloud.root) || exit 1
# ---
DNSMASQ_SETUP_DIR="./setup/dnsmasq"
BUNDLE_DIR="${DNSMASQ_SETUP_DIR}/setup-bundle"
mkdir -p "${BUNDLE_DIR}"
# Copy iPXE bootloader to ipxe-web.
echo "Copying Talos kernel and initramfs for PXE boot..."
PXE_WEB_ROOT="${BUNDLE_DIR}/ipxe-web"
mkdir -p "${PXE_WEB_ROOT}/amd64"
cp "${DNSMASQ_SETUP_DIR}/boot.ipxe" "${PXE_WEB_ROOT}/boot.ipxe"
# Create Talos bare metal boot assets.
# This uses the Talos factory API to create boot assets for bare metal nodes.
# These assets include the kernel and initramfs needed for PXE booting Talos on bare metal.
echo "Creating Talos bare metal boot assets..."
TALOS_ID=$(curl -X POST --data-binary @${DNSMASQ_SETUP_DIR}/bare-metal.yaml https://factory.talos.dev/schematics | jq -r '.id')
if [ -z "${TALOS_ID}" ] || [ "${TALOS_ID}" = "null" ]; then
echo "Error: Failed to create Talos bare metal boot assets"
exit 1
fi
echo "Successfully created Talos bare metal boot assets with ID: ${TALOS_ID}"
# Download kernel to ipxe-web if it's not already there.
TALOS_VERSION=$(wild-config cluster.nodes.talos.version) || exit 1
if [ ! -f "${PXE_WEB_ROOT}/amd64/vmlinuz" ]; then
echo "Downloading Talos kernel..."
wget -O "${PXE_WEB_ROOT}/amd64/vmlinuz" "https://pxe.factory.talos.dev/image/${TALOS_ID}/${TALOS_VERSION}/kernel-amd64"
else
echo "Talos kernel already exists, skipping download"
fi
# Download initramfs to ipxe-web if it's not already there.
if [ ! -f "${PXE_WEB_ROOT}/amd64/initramfs.xz" ]; then
echo "Downloading Talos initramfs..."
wget -O "${PXE_WEB_ROOT}/amd64/initramfs.xz" "https://pxe.factory.talos.dev/image/${TALOS_ID}/${TALOS_VERSION}/initramfs-amd64.xz"
else
echo "Talos initramfs already exists, skipping download"
fi
# Update PXE's iPXE bootloader files.
# TODO: Put download to cache first.
echo "Updating iPXE ftpd bootloader files."
FTPD_DIR="${BUNDLE_DIR}/pxe-ftpd"
mkdir -p $FTPD_DIR
wget http://boot.ipxe.org/ipxe.efi -O ${FTPD_DIR}/ipxe.efi
wget http://boot.ipxe.org/undionly.kpxe -O ${FTPD_DIR}/undionly.kpxe
wget http://boot.ipxe.org/arm64-efi/ipxe.efi -O ${FTPD_DIR}/ipxe-arm64.efi
cp "${DNSMASQ_SETUP_DIR}/nginx.conf" "${BUNDLE_DIR}/nginx.conf"
cp "${DNSMASQ_SETUP_DIR}/dnsmasq.conf" "${BUNDLE_DIR}/dnsmasq.conf"
cp "${DNSMASQ_SETUP_DIR}/bin/setup.sh" "${BUNDLE_DIR}/setup.sh"

55
setup/dnsmasq/bin/setup.sh Executable file
View File

@@ -0,0 +1,55 @@
#!/bin/bash
# This file to be run on dnsmasq server (Central)
echo "Updating APT repositories."
sudo apt-get update
echo "Installing dnsmasq and nginx."
sudo apt install -y dnsmasq nginx
DNSMASQ_SETUP_DIR="."
PXE_FTPD_DIR="${DNSMASQ_SETUP_DIR}/pxe-ftpd"
PXE_WEB_ROOT="${DNSMASQ_SETUP_DIR}/pxe-web"
# Configure nginx.
echo "Configuring nginx."
sudo cp "${DNSMASQ_SETUP_DIR}/nginx.conf" /etc/nginx/sites-available/talos
sudo chown www-data:www-data /etc/nginx/sites-available/talos
sudo chmod -R 755 /etc/nginx/sites-available/talos
# Copy assets to nginx web root
echo "Copying Talos PXE boot assets to nginx web root."
TALOS_PXE_WEB_ROOT="/var/www/html/talos"
sudo mkdir -p "${TALOS_PXE_WEB_ROOT}"
sudo rm -rf ${TALOS_PXE_WEB_ROOT}/* # Clean the web root directory
sudo cp -r ${PXE_WEB_ROOT}/* "${TALOS_PXE_WEB_ROOT}"
sudo chown -R www-data:www-data "${TALOS_PXE_WEB_ROOT}"
sudo chmod -R 755 "${TALOS_PXE_WEB_ROOT}"
# Start nginx service to serve the iPXE script and images
echo "Starting nginx service."
sudo ln -s /etc/nginx/sites-available/talos /etc/nginx/sites-enabled/talos > /dev/null 2>&1 || true
sudo rm -f /etc/nginx/sites-enabled/default
sudo systemctl reload nginx
# Stop and disable systemd-resolved if it is running
if systemctl is-active --quiet systemd-resolved; then
echo "Stopping and disabling systemd-resolved..."
sudo systemctl disable systemd-resolved
sudo systemctl stop systemd-resolved
# sudo rm -f /etc/resolv.conf
echo "systemd-resolved stopped and disabled"
fi
# Update PXE's iPXE bootloader files.
echo "Updating iPXE ftpd bootloader files."
sudo mkdir -p /var/ftpd
sudo cp ${PXE_FTPD_DIR}/* /var/ftpd/
# Finally, install and configure DNSMasq.
echo "Configuring and starting DNSMasq."
sudo cp "${DNSMASQ_SETUP_DIR}/dnsmasq.conf" /etc/dnsmasq.conf
sudo systemctl restart dnsmasq
echo "DNSMasq installation and configuration completed successfully."

View File

@@ -0,0 +1,13 @@
#!/bin/bash
if [ ! -d ".wildcloud" ]; then
echo "Error: You must run this script from a wild-cloud directory"
exit 1
fi
SERVER_HOST=$(wild-config cloud.dns.ip2) || exit 1
SETUP_DIR="./setup/dnsmasq/setup-bundle"
DESTINATION_DIR="~/dnsmasq-setup"
echo "Copying DNSMasq setup files to ${SERVER_HOST}:${DESTINATION_DIR}..."
scp -r ${SETUP_DIR}/* root@${SERVER_HOST}:${DESTINATION_DIR}

5
setup/dnsmasq/boot.ipxe Normal file
View File

@@ -0,0 +1,5 @@
!ipxe
imgfree
kernel http://{{ (ds "config").cloud.dns.ip }}/amd64/vmlinuz talos.platform=metal console=tty0 init_on_alloc=1 slab_nomerge pti=on consoleblank=0 nvme_core.io_timeout=4294967295 printk.devkmsg=on ima_template=ima-ng ima_appraise=fix ima_hash=sha512 selinux=1 net.ifnames=0
initrd http://{{ (ds "config").cloud.dns.ip }}/amd64/initramfs.xz
boot

View File

@@ -0,0 +1,41 @@
# Configuration file for dnsmasq.
# Basic Settings
interface={{ (ds "config").cloud.dnsmasq.interface }}
listen-address={{ (ds "config").cloud.dns.ip }}
domain-needed
bogus-priv
no-resolv
# DNS Forwarding
# local=/{{ (ds "config").cloud.domain }}/
# address=/{{ (ds "config").cloud.domain }}/{{ (ds "config").cluster.endpointIp }}
# local=/{{ (ds "config").cloud.internalDomain }}/
# address=/{{ (ds "config").cloud.internalDomain }}/{{ (ds "config").cluster.endpointIp }}
server=/{{ (ds "config").cloud.domain }}/{{ (ds "config").cluster.endpointIp }}
server=/{{ (ds "config").cloud.internalDomain }}/{{ (ds "config").cluster.endpointIp }}
server=1.1.1.1
server=8.8.8.8
# --- DHCP Settings ---
dhcp-range={{ (ds "config").cloud.dhcpRange }},12h
dhcp-option=3,{{ (ds "config").cloud.router.ip }} # gateway to assign
dhcp-option=6,{{ (ds "config").cloud.dns.ip }} # dns to assign
# --- PXE Booting ---
enable-tftp
tftp-root=/var/ftpd
dhcp-match=set:efi-x86_64,option:client-arch,7
dhcp-boot=tag:efi-x86_64,ipxe.efi
dhcp-boot=tag:!efi-x86_64,undionly.kpxe
dhcp-match=set:efi-arm64,option:client-arch,11
dhcp-boot=tag:efi-arm64,ipxe-arm64.efi
dhcp-userclass=set:ipxe,iPXE
dhcp-boot=tag:ipxe,http://{{ (ds "config").cloud.dns.ip }}/boot.ipxe
log-queries
log-dhcp

View File

@@ -0,0 +1,706 @@
# Configuration file for dnsmasq.
#
# Format is one option per line, legal options are the same
# as the long options legal on the command line. See
# "/usr/sbin/dnsmasq --help" or "man 8 dnsmasq" for details.
# Listen on this specific port instead of the standard DNS port
# (53). Setting this to zero completely disables DNS function,
# leaving only DHCP and/or TFTP.
#port=5353
# The following two options make you a better netizen, since they
# tell dnsmasq to filter out queries which the public DNS cannot
# answer, and which load the servers (especially the root servers)
# unnecessarily. If you have a dial-on-demand link they also stop
# these requests from bringing up the link unnecessarily.
# Never forward plain names (without a dot or domain part)
domain-needed
# Never forward addresses in the non-routed address spaces.
bogus-priv
# Uncomment these to enable DNSSEC validation and caching:
# (Requires dnsmasq to be built with DNSSEC option.)
#conf-file=%%PREFIX%%/share/dnsmasq/trust-anchors.conf
#dnssec
# Replies which are not DNSSEC signed may be legitimate, because the domain
# is unsigned, or may be forgeries. Setting this option tells dnsmasq to
# check that an unsigned reply is OK, by finding a secure proof that a DS
# record somewhere between the root and the domain does not exist.
# The cost of setting this is that even queries in unsigned domains will need
# one or more extra DNS queries to verify.
#dnssec-check-unsigned
# Uncomment this to filter useless windows-originated DNS requests
# which can trigger dial-on-demand links needlessly.
# Note that (amongst other things) this blocks all SRV requests,
# so don't use it if you use eg Kerberos, SIP, XMMP or Google-talk.
# This option only affects forwarding, SRV records originating for
# dnsmasq (via srv-host= lines) are not suppressed by it.
filterwin2k
# Change this line if you want dns to get its upstream servers from
# somewhere other that /etc/resolv.conf
#resolv-file=
# By default, dnsmasq will send queries to any of the upstream
# servers it knows about and tries to favour servers to are known
# to be up. Uncommenting this forces dnsmasq to try each query
# with each server strictly in the order they appear in
# /etc/resolv.conf
#strict-order
# If you don't want dnsmasq to read /etc/resolv.conf or any other
# file, getting its servers from this file instead (see below), then
# uncomment this.
no-resolv
# If you don't want dnsmasq to poll /etc/resolv.conf or other resolv
# files for changes and re-read them then uncomment this.
#no-poll
# Add other name servers here, with domain specs if they are for
# non-public domains.
#server=/localnet/192.168.0.1
# Example of routing PTR queries to nameservers: this will send all
# address->name queries for 192.168.3/24 to nameserver 10.1.2.3
#server=/3.168.192.in-addr.arpa/10.1.2.3
# Add local-only domains here, queries in these domains are answered
# from /etc/hosts or DHCP only.
#local=/localnet/
local=/{{ (ds "config").cloud.domain }}/
local=/{{ (ds "config").cloud.internalDomain }}/
# Add domains which you want to force to an IP address here.
# The example below send any host in double-click.net to a local
# web-server.
#address=/double-click.net/127.0.0.1
address=/{{ (ds "config").cloud.domain }}/{{ (ds "config").cluster.endpointIp }}
address=/{{ (ds "config").cloud.internalDomain }}/{{ (ds "config").cluster.endpointIp }}
# --address (and --server) work with IPv6 addresses too.
#address=/www.thekelleys.org.uk/fe80::20d:60ff:fe36:f83
# Add the IPs of all queries to yahoo.com, google.com, and their
# subdomains to the vpn and search ipsets:
#ipset=/yahoo.com/google.com/vpn,search
# Add the IPs of all queries to yahoo.com, google.com, and their
# subdomains to netfilters sets, which is equivalent to
# 'nft add element ip test vpn { ... }; nft add element ip test search { ... }'
#nftset=/yahoo.com/google.com/ip#test#vpn,ip#test#search
# Use netfilters sets for both IPv4 and IPv6:
# This adds all addresses in *.yahoo.com to vpn4 and vpn6 for IPv4 and IPv6 addresses.
#nftset=/yahoo.com/4#ip#test#vpn4
#nftset=/yahoo.com/6#ip#test#vpn6
# You can control how dnsmasq talks to a server: this forces
# queries to 10.1.2.3 to be routed via eth1
# server=10.1.2.3@eth1
# and this sets the source (ie local) address used to talk to
# 10.1.2.3 to 192.168.1.1 port 55 (there must be an interface with that
# IP on the machine, obviously).
# server=10.1.2.3@192.168.1.1#55
# If you want dnsmasq to change uid and gid to something other
# than the default, edit the following lines.
#user=
#group=
# If you want dnsmasq to listen for DHCP and DNS requests only on
# specified interfaces (and the loopback) give the name of the
# interface (eg eth0) here.
# Repeat the line for more than one interface.
interface=eth0
# Or you can specify which interface _not_ to listen on
#except-interface=
# Or which to listen on by address (remember to include 127.0.0.1 if
# you use this.)
#listen-address=
# If you want dnsmasq to provide only DNS service on an interface,
# configure it as shown above, and then use the following line to
# disable DHCP and TFTP on it.
#no-dhcp-interface=
# On systems which support it, dnsmasq binds the wildcard address,
# even when it is listening on only some interfaces. It then discards
# requests that it shouldn't reply to. This has the advantage of
# working even when interfaces come and go and change address. If you
# want dnsmasq to really bind only the interfaces it is listening on,
# uncomment this option. About the only time you may need this is when
# running another nameserver on the same machine.
# bind-interfaces
# If you don't want dnsmasq to read /etc/hosts, uncomment the
# following line.
#no-hosts
# or if you want it to read another file, as well as /etc/hosts, use
# this.
#addn-hosts=/etc/banner_add_hosts
# Set this (and domain: see below) if you want to have a domain
# automatically added to simple names in a hosts-file.
#expand-hosts
# Set the domain for dnsmasq. this is optional, but if it is set, it
# does the following things.
# 1) Allows DHCP hosts to have fully qualified domain names, as long
# as the domain part matches this setting.
# 2) Sets the "domain" DHCP option thereby potentially setting the
# domain of all systems configured by DHCP
# 3) Provides the domain part for "expand-hosts"
#domain=thekelleys.org.uk
# Set a different domain for a particular subnet
#domain=wireless.thekelleys.org.uk,192.168.2.0/24
# Same idea, but range rather then subnet
#domain=reserved.thekelleys.org.uk,192.68.3.100,192.168.3.200
# Uncomment this to enable the integrated DHCP server, you need
# to supply the range of addresses available for lease and optionally
# a lease time. If you have more than one network, you will need to
# repeat this for each network on which you want to supply DHCP
# service.
dhcp-range={{ (ds "config").cloud.dhcpRange }},12h
# This is an example of a DHCP range where the netmask is given. This
# is needed for networks we reach the dnsmasq DHCP server via a relay
# agent. If you don't know what a DHCP relay agent is, you probably
# don't need to worry about this.
#dhcp-range=192.168.0.50,192.168.0.150,255.255.255.0,12h
# This is an example of a DHCP range which sets a tag, so that
# some DHCP options may be set only for this network.
#dhcp-range=set:red,192.168.0.50,192.168.0.150
# Use this DHCP range only when the tag "green" is set.
#dhcp-range=tag:green,192.168.0.50,192.168.0.150,12h
# Specify a subnet which can't be used for dynamic address allocation,
# is available for hosts with matching --dhcp-host lines. Note that
# dhcp-host declarations will be ignored unless there is a dhcp-range
# of some type for the subnet in question.
# In this case the netmask is implied (it comes from the network
# configuration on the machine running dnsmasq) it is possible to give
# an explicit netmask instead.
#dhcp-range=192.168.0.0,static
# Enable DHCPv6. Note that the prefix-length does not need to be specified
# and defaults to 64 if missing/
#dhcp-range=1234::2, 1234::500, 64, 12h
# Do Router Advertisements, BUT NOT DHCP for this subnet.
#dhcp-range=1234::, ra-only
# Do Router Advertisements, BUT NOT DHCP for this subnet, also try and
# add names to the DNS for the IPv6 address of SLAAC-configured dual-stack
# hosts. Use the DHCPv4 lease to derive the name, network segment and
# MAC address and assume that the host will also have an
# IPv6 address calculated using the SLAAC algorithm.
#dhcp-range=1234::, ra-names
# Do Router Advertisements, BUT NOT DHCP for this subnet.
# Set the lifetime to 46 hours. (Note: minimum lifetime is 2 hours.)
#dhcp-range=1234::, ra-only, 48h
# Do DHCP and Router Advertisements for this subnet. Set the A bit in the RA
# so that clients can use SLAAC addresses as well as DHCP ones.
#dhcp-range=1234::2, 1234::500, slaac
# Do Router Advertisements and stateless DHCP for this subnet. Clients will
# not get addresses from DHCP, but they will get other configuration information.
# They will use SLAAC for addresses.
#dhcp-range=1234::, ra-stateless
# Do stateless DHCP, SLAAC, and generate DNS names for SLAAC addresses
# from DHCPv4 leases.
#dhcp-range=1234::, ra-stateless, ra-names
# Do router advertisements for all subnets where we're doing DHCPv6
# Unless overridden by ra-stateless, ra-names, et al, the router
# advertisements will have the M and O bits set, so that the clients
# get addresses and configuration from DHCPv6, and the A bit reset, so the
# clients don't use SLAAC addresses.
#enable-ra
# Supply parameters for specified hosts using DHCP. There are lots
# of valid alternatives, so we will give examples of each. Note that
# IP addresses DO NOT have to be in the range given above, they just
# need to be on the same network. The order of the parameters in these
# do not matter, it's permissible to give name, address and MAC in any
# order.
# Always allocate the host with Ethernet address 11:22:33:44:55:66
# The IP address 192.168.0.60
#dhcp-host=11:22:33:44:55:66,192.168.0.60
# Always set the name of the host with hardware address
# 11:22:33:44:55:66 to be "fred"
#dhcp-host=11:22:33:44:55:66,fred
# Always give the host with Ethernet address 11:22:33:44:55:66
# the name fred and IP address 192.168.0.60 and lease time 45 minutes
#dhcp-host=11:22:33:44:55:66,fred,192.168.0.60,45m
# Give a host with Ethernet address 11:22:33:44:55:66 or
# 12:34:56:78:90:12 the IP address 192.168.0.60. Dnsmasq will assume
# that these two Ethernet interfaces will never be in use at the same
# time, and give the IP address to the second, even if it is already
# in use by the first. Useful for laptops with wired and wireless
# addresses.
#dhcp-host=11:22:33:44:55:66,12:34:56:78:90:12,192.168.0.60
# Give the machine which says its name is "bert" IP address
# 192.168.0.70 and an infinite lease
#dhcp-host=bert,192.168.0.70,infinite
# Always give the host with client identifier 01:02:02:04
# the IP address 192.168.0.60
#dhcp-host=id:01:02:02:04,192.168.0.60
# Always give the InfiniBand interface with hardware address
# 80:00:00:48:fe:80:00:00:00:00:00:00:f4:52:14:03:00:28:05:81 the
# ip address 192.168.0.61. The client id is derived from the prefix
# ff:00:00:00:00:00:02:00:00:02:c9:00 and the last 8 pairs of
# hex digits of the hardware address.
#dhcp-host=id:ff:00:00:00:00:00:02:00:00:02:c9:00:f4:52:14:03:00:28:05:81,192.168.0.61
# Always give the host with client identifier "marjorie"
# the IP address 192.168.0.60
#dhcp-host=id:marjorie,192.168.0.60
# Enable the address given for "judge" in /etc/hosts
# to be given to a machine presenting the name "judge" when
# it asks for a DHCP lease.
#dhcp-host=judge
# Never offer DHCP service to a machine whose Ethernet
# address is 11:22:33:44:55:66
#dhcp-host=11:22:33:44:55:66,ignore
# Ignore any client-id presented by the machine with Ethernet
# address 11:22:33:44:55:66. This is useful to prevent a machine
# being treated differently when running under different OS's or
# between PXE boot and OS boot.
#dhcp-host=11:22:33:44:55:66,id:*
# Send extra options which are tagged as "red" to
# the machine with Ethernet address 11:22:33:44:55:66
#dhcp-host=11:22:33:44:55:66,set:red
# Send extra options which are tagged as "red" to
# any machine with Ethernet address starting 11:22:33:
#dhcp-host=11:22:33:*:*:*,set:red
# Give a fixed IPv6 address and name to client with
# DUID 00:01:00:01:16:d2:83:fc:92:d4:19:e2:d8:b2
# Note the MAC addresses CANNOT be used to identify DHCPv6 clients.
# Note also that the [] around the IPv6 address are obligatory.
#dhcp-host=id:00:01:00:01:16:d2:83:fc:92:d4:19:e2:d8:b2, fred, [1234::5]
# Ignore any clients which are not specified in dhcp-host lines
# or /etc/ethers. Equivalent to ISC "deny unknown-clients".
# This relies on the special "known" tag which is set when
# a host is matched.
#dhcp-ignore=tag:!known
# Send extra options which are tagged as "red" to any machine whose
# DHCP vendorclass string includes the substring "Linux"
#dhcp-vendorclass=set:red,Linux
# Send extra options which are tagged as "red" to any machine one
# of whose DHCP userclass strings includes the substring "accounts"
#dhcp-userclass=set:red,accounts
# Send extra options which are tagged as "red" to any machine whose
# MAC address matches the pattern.
#dhcp-mac=set:red,00:60:8C:*:*:*
# If this line is uncommented, dnsmasq will read /etc/ethers and act
# on the ethernet-address/IP pairs found there just as if they had
# been given as --dhcp-host options. Useful if you keep
# MAC-address/host mappings there for other purposes.
#read-ethers
# Send options to hosts which ask for a DHCP lease.
# See RFC 2132 for details of available options.
# Common options can be given to dnsmasq by name:
# run "dnsmasq --help dhcp" to get a list.
# Note that all the common settings, such as netmask and
# broadcast address, DNS server and default route, are given
# sane defaults by dnsmasq. You very likely will not need
# any dhcp-options. If you use Windows clients and Samba, there
# are some options which are recommended, they are detailed at the
# end of this section.
# Override the default route supplied by dnsmasq, which assumes the
# router is the same machine as the one running dnsmasq.
#dhcp-option=3,1.2.3.4
dhcp-option=3,{{ (ds "config").cluster.router.ip }}
# Do the same thing, but using the option name
#dhcp-option=option:router,1.2.3.4
# Override the default route supplied by dnsmasq and send no default
# route at all. Note that this only works for the options sent by
# default (1, 3, 6, 12, 28) the same line will send a zero-length option
# for all other option numbers.
#dhcp-option=3
# Set the NTP time server addresses to 192.168.0.4 and 10.10.0.5
#dhcp-option=option:ntp-server,192.168.0.4,10.10.0.5
# Send DHCPv6 option. Note [] around IPv6 addresses.
#dhcp-option=option6:dns-server,[1234::77],[1234::88]
# dhcp-option=option6:dns-server,{{ (ds "config").cluster.dns.ip }}
# Send DHCPv6 option for namservers as the machine running
# dnsmasq and another.
#dhcp-option=option6:dns-server,[::],[1234::88]
# Ask client to poll for option changes every six hours. (RFC4242)
#dhcp-option=option6:information-refresh-time,6h
# Set option 58 client renewal time (T1). Defaults to half of the
# lease time if not specified. (RFC2132)
#dhcp-option=option:T1,1m
# Set option 59 rebinding time (T2). Defaults to 7/8 of the
# lease time if not specified. (RFC2132)
#dhcp-option=option:T2,2m
# Set the NTP time server address to be the same machine as
# is running dnsmasq
#dhcp-option=42,0.0.0.0
# Set the NIS domain name to "welly"
#dhcp-option=40,welly
# Set the default time-to-live to 50
#dhcp-option=23,50
# Set the "all subnets are local" flag
#dhcp-option=27,1
# Send the etherboot magic flag and then etherboot options (a string).
#dhcp-option=128,e4:45:74:68:00:00
#dhcp-option=129,NIC=eepro100
# Specify an option which will only be sent to the "red" network
# (see dhcp-range for the declaration of the "red" network)
# Note that the tag: part must precede the option: part.
#dhcp-option = tag:red, option:ntp-server, 192.168.1.1
# The following DHCP options set up dnsmasq in the same way as is specified
# for the ISC dhcpcd in
# https://web.archive.org/web/20040313070105/http://us1.samba.org/samba/ftp/docs/textdocs/DHCP-Server-Configuration.txt
# adapted for a typical dnsmasq installation where the host running
# dnsmasq is also the host running samba.
# you may want to uncomment some or all of them if you use
# Windows clients and Samba.
#dhcp-option=19,0 # option ip-forwarding off
#dhcp-option=44,0.0.0.0 # set netbios-over-TCP/IP nameserver(s) aka WINS server(s)
#dhcp-option=45,0.0.0.0 # netbios datagram distribution server
#dhcp-option=46,8 # netbios node type
# Send an empty WPAD option. This may be REQUIRED to get windows 7 to behave.
#dhcp-option=252,"\n"
# Send RFC-3397 DNS domain search DHCP option. WARNING: Your DHCP client
# probably doesn't support this......
#dhcp-option=option:domain-search,eng.apple.com,marketing.apple.com
# Send RFC-3442 classless static routes (note the netmask encoding)
#dhcp-option=121,192.168.1.0/24,1.2.3.4,10.0.0.0/8,5.6.7.8
# Send vendor-class specific options encapsulated in DHCP option 43.
# The meaning of the options is defined by the vendor-class so
# options are sent only when the client supplied vendor class
# matches the class given here. (A substring match is OK, so "MSFT"
# matches "MSFT" and "MSFT 5.0"). This example sets the
# mtftp address to 0.0.0.0 for PXEClients.
#dhcp-option=vendor:PXEClient,1,0.0.0.0
# Send microsoft-specific option to tell windows to release the DHCP lease
# when it shuts down. Note the "i" flag, to tell dnsmasq to send the
# value as a four-byte integer - that's what microsoft wants. See
# http://technet2.microsoft.com/WindowsServer/en/library/a70f1bb7-d2d4-49f0-96d6-4b7414ecfaae1033.mspx?mfr=true
#dhcp-option=vendor:MSFT,2,1i
# Send the Encapsulated-vendor-class ID needed by some configurations of
# Etherboot to allow is to recognise the DHCP server.
#dhcp-option=vendor:Etherboot,60,"Etherboot"
# Send options to PXELinux. Note that we need to send the options even
# though they don't appear in the parameter request list, so we need
# to use dhcp-option-force here.
# See http://syslinux.zytor.com/pxe.php#special for details.
# Magic number - needed before anything else is recognised
#dhcp-option-force=208,f1:00:74:7e
# Configuration file name
#dhcp-option-force=209,configs/common
# Path prefix
#dhcp-option-force=210,/tftpboot/pxelinux/files/
# Reboot time. (Note 'i' to send 32-bit value)
#dhcp-option-force=211,30i
# Set the boot filename for netboot/PXE. You will only need
# this if you want to boot machines over the network and you will need
# a TFTP server; either dnsmasq's built-in TFTP server or an
# external one. (See below for how to enable the TFTP server.)
#dhcp-boot=pxelinux.0
# The same as above, but use custom tftp-server instead machine running dnsmasq
#dhcp-boot=pxelinux,server.name,192.168.1.100
# Boot for iPXE. The idea is to send two different
# filenames, the first loads iPXE, and the second tells iPXE what to
# load. The dhcp-match sets the ipxe tag for requests from iPXE.
#dhcp-boot=undionly.kpxe
#dhcp-match=set:ipxe,175 # iPXE sends a 175 option.
#dhcp-boot=tag:ipxe,http://boot.ipxe.org/demo/boot.php
# For WildCloud cluster
dhcp-match=set:efi-x86_64,option:client-arch,7
dhcp-match=set:efi-arm64,option:client-arch,11
dhcp-boot=tag:!efi-x86_64,undionly.kpxe
dhcp-boot=tag:efi-x86_64,ipxe.efi
dhcp-boot=tag:efi-arm64,ipxe-arm64.efi
dhcp-userclass=set:ipxe,iPXE
dhcp-boot=tag:ipxe,http://{{ (ds="config").cloud.dns.ip }}/boot.ipxe
# Encapsulated options for iPXE. All the options are
# encapsulated within option 175
#dhcp-option=encap:175, 1, 5b # priority code
#dhcp-option=encap:175, 176, 1b # no-proxydhcp
#dhcp-option=encap:175, 177, string # bus-id
#dhcp-option=encap:175, 189, 1b # BIOS drive code
#dhcp-option=encap:175, 190, user # iSCSI username
#dhcp-option=encap:175, 191, pass # iSCSI password
# Test for the architecture of a netboot client. PXE clients are
# supposed to send their architecture as option 93. (See RFC 4578)
#dhcp-match=peecees, option:client-arch, 0 #x86-32
#dhcp-match=itanics, option:client-arch, 2 #IA64
#dhcp-match=hammers, option:client-arch, 6 #x86-64
#dhcp-match=mactels, option:client-arch, 7 #EFI x86-64
# Do real PXE, rather than just booting a single file, this is an
# alternative to dhcp-boot.
#pxe-prompt="What system shall I netboot?"
# or with timeout before first available action is taken:
#pxe-prompt="Press F8 for menu.", 60
# Available boot services. for PXE.
#pxe-service=x86PC, "Boot from local disk"
# Loads <tftp-root>/pxelinux.0 from dnsmasq TFTP server.
#pxe-service=x86PC, "Install Linux", pxelinux
# Loads <tftp-root>/pxelinux.0 from TFTP server at 1.2.3.4.
# Beware this fails on old PXE ROMS.
#pxe-service=x86PC, "Install Linux", pxelinux, 1.2.3.4
# Use bootserver on network, found my multicast or broadcast.
#pxe-service=x86PC, "Install windows from RIS server", 1
# Use bootserver at a known IP address.
#pxe-service=x86PC, "Install windows from RIS server", 1, 1.2.3.4
# If you have multicast-FTP available,
# information for that can be passed in a similar way using options 1
# to 5. See page 19 of
# http://download.intel.com/design/archives/wfm/downloads/pxespec.pdf
# Enable dnsmasq's built-in TFTP server
enable-tftp
# Set the root directory for files available via FTP.
tftp-root=/var/ftpd
# Do not abort if the tftp-root is unavailable
#tftp-no-fail
# Make the TFTP server more secure: with this set, only files owned by
# the user dnsmasq is running as will be send over the net.
#tftp-secure
# This option stops dnsmasq from negotiating a larger blocksize for TFTP
# transfers. It will slow things down, but may rescue some broken TFTP
# clients.
#tftp-no-blocksize
# Set the boot file name only when the "red" tag is set.
#dhcp-boot=tag:red,pxelinux.red-net
# An example of dhcp-boot with an external TFTP server: the name and IP
# address of the server are given after the filename.
# Can fail with old PXE ROMS. Overridden by --pxe-service.
#dhcp-boot=/var/ftpd/pxelinux.0,boothost,192.168.0.3
# If there are multiple external tftp servers having a same name
# (using /etc/hosts) then that name can be specified as the
# tftp_servername (the third option to dhcp-boot) and in that
# case dnsmasq resolves this name and returns the resultant IP
# addresses in round robin fashion. This facility can be used to
# load balance the tftp load among a set of servers.
#dhcp-boot=/var/ftpd/pxelinux.0,boothost,tftp_server_name
# Set the limit on DHCP leases, the default is 150
#dhcp-lease-max=150
# The DHCP server needs somewhere on disk to keep its lease database.
# This defaults to a sane location, but if you want to change it, use
# the line below.
#dhcp-leasefile=/var/lib/misc/dnsmasq.leases
# Set the DHCP server to authoritative mode. In this mode it will barge in
# and take over the lease for any client which broadcasts on the network,
# whether it has a record of the lease or not. This avoids long timeouts
# when a machine wakes up on a new network. DO NOT enable this if there's
# the slightest chance that you might end up accidentally configuring a DHCP
# server for your campus/company accidentally. The ISC server uses
# the same option, and this URL provides more information:
# http://www.isc.org/files/auth.html
#dhcp-authoritative
# Set the DHCP server to enable DHCPv4 Rapid Commit Option per RFC 4039.
# In this mode it will respond to a DHCPDISCOVER message including a Rapid Commit
# option with a DHCPACK including a Rapid Commit option and fully committed address
# and configuration information. This must only be enabled if either the server is
# the only server for the subnet, or multiple servers are present and they each
# commit a binding for all clients.
#dhcp-rapid-commit
# Run an executable when a DHCP lease is created or destroyed.
# The arguments sent to the script are "add" or "del",
# then the MAC address, the IP address and finally the hostname
# if there is one.
#dhcp-script=/bin/echo
# Set the cachesize here.
#cache-size=150
# If you want to disable negative caching, uncomment this.
#no-negcache
# Normally responses which come from /etc/hosts and the DHCP lease
# file have Time-To-Live set as zero, which conventionally means
# do not cache further. If you are happy to trade lower load on the
# server for potentially stale date, you can set a time-to-live (in
# seconds) here.
#local-ttl=
# If you want dnsmasq to detect attempts by Verisign to send queries
# to unregistered .com and .net hosts to its sitefinder service and
# have dnsmasq instead return the correct NXDOMAIN response, uncomment
# this line. You can add similar lines to do the same for other
# registries which have implemented wildcard A records.
#bogus-nxdomain=64.94.110.11
# If you want to fix up DNS results from upstream servers, use the
# alias option. This only works for IPv4.
# This alias makes a result of 1.2.3.4 appear as 5.6.7.8
#alias=1.2.3.4,5.6.7.8
# and this maps 1.2.3.x to 5.6.7.x
#alias=1.2.3.0,5.6.7.0,255.255.255.0
# and this maps 192.168.0.10->192.168.0.40 to 10.0.0.10->10.0.0.40
#alias=192.168.0.10-192.168.0.40,10.0.0.0,255.255.255.0
# Change these lines if you want dnsmasq to serve MX records.
# Return an MX record named "maildomain.com" with target
# servermachine.com and preference 50
#mx-host=maildomain.com,servermachine.com,50
# Set the default target for MX records created using the localmx option.
#mx-target=servermachine.com
# Return an MX record pointing to the mx-target for all local
# machines.
#localmx
# Return an MX record pointing to itself for all local machines.
#selfmx
# Change the following lines if you want dnsmasq to serve SRV
# records. These are useful if you want to serve ldap requests for
# Active Directory and other windows-originated DNS requests.
# See RFC 2782.
# You may add multiple srv-host lines.
# The fields are <name>,<target>,<port>,<priority>,<weight>
# If the domain part if missing from the name (so that is just has the
# service and protocol sections) then the domain given by the domain=
# config option is used. (Note that expand-hosts does not need to be
# set for this to work.)
# A SRV record sending LDAP for the example.com domain to
# ldapserver.example.com port 389
#srv-host=_ldap._tcp.example.com,ldapserver.example.com,389
# A SRV record sending LDAP for the example.com domain to
# ldapserver.example.com port 389 (using domain=)
#domain=example.com
#srv-host=_ldap._tcp,ldapserver.example.com,389
# Two SRV records for LDAP, each with different priorities
#srv-host=_ldap._tcp.example.com,ldapserver.example.com,389,1
#srv-host=_ldap._tcp.example.com,ldapserver.example.com,389,2
# A SRV record indicating that there is no LDAP server for the domain
# example.com
#srv-host=_ldap._tcp.example.com
# The following line shows how to make dnsmasq serve an arbitrary PTR
# record. This is useful for DNS-SD. (Note that the
# domain-name expansion done for SRV records _does_not
# occur for PTR records.)
#ptr-record=_http._tcp.dns-sd-services,"New Employee Page._http._tcp.dns-sd-services"
# Change the following lines to enable dnsmasq to serve TXT records.
# These are used for things like SPF and zeroconf. (Note that the
# domain-name expansion done for SRV records _does_not
# occur for TXT records.)
#Example SPF.
#txt-record=example.com,"v=spf1 a -all"
#Example zeroconf
#txt-record=_http._tcp.example.com,name=value,paper=A4
# Provide an alias for a "local" DNS name. Note that this _only_ works
# for targets which are names from DHCP or /etc/hosts. Give host
# "bert" another name, bertrand
#cname=bertrand,bert
# For debugging purposes, log each DNS query as it passes through
# dnsmasq.
log-queries
# Log lots of extra information about DHCP transactions.
log-dhcp
# Include another lot of configuration options.
#conf-file=/etc/dnsmasq.more.conf
#conf-dir=/etc/dnsmasq.d
# Include all the files in a directory except those ending in .bak
#conf-dir=/etc/dnsmasq.d,.bak
# Include all files in a directory which end in .conf
#conf-dir=/etc/dnsmasq.d/,*.conf
# If a DHCP client claims that its name is "wpad", ignore that.
# This fixes a security hole. see CERT Vulnerability VU#598349
#dhcp-name-match=set:wpad-ignore,wpad
#dhcp-ignore-names=tag:wpad-ignore

9
setup/dnsmasq/nginx.conf Normal file
View File

@@ -0,0 +1,9 @@
server {
listen 80;
server_name _;
location / {
autoindex on;
root /var/www/html/talos;
}
}