feat: Move cluster services to wild-directory as unified packages

Convert all 15 cluster services from embedded API format to wild-directory packages using the unified manifest format: - metallb, traefik, cert-manager, longhorn, snapshot-controller - nfs, smtp, coredns, node-feature-discovery, nvidia-device-plugin - externaldns, docker-registry, headlamp, crowdsec, utils Changes: - wild-manifest.yaml → manifest.yaml with is, defaultConfig, requires - Eliminated configReferences and serviceConfig fields - Flattened kustomize.template/ to package root - Template vars use flat defaultConfig keys - install.sh paths updated for apps/ layout - Updated 9 app manifests: cloud.smtp.* → apps.smtp.* with requires - Removed dead install: true field from 6 app manifests Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-17 02:26:46 +00:00
parent aaf74cc00c
commit 9687fad812
128 changed files with 10941 additions and 50 deletions
--- a/crowdsec/README.md
+++ b/crowdsec/README.md
@@ -0,0 +1,118 @@
+# CrowdSec Security Service
+
+CrowdSec is an open-source security engine that analyzes traffic patterns and blocks malicious actors. This service integrates CrowdSec with Traefik to provide automatic threat detection and rate limiting for all Wild Cloud ingresses.
+
+## Components
+
+- **CrowdSec Agent**: Analyzes traffic patterns, maintains decision lists, and connects to the CrowdSec threat intelligence network
+- **Traefik Bouncer**: Integrates with Traefik via ForwardAuth to enforce CrowdSec decisions
+- **Security Middlewares**: Traefik middleware for rate limiting and security headers
+
+## Default Protection
+
+After installation, **all ingresses are automatically protected** with:
+- Threat detection (blocks known malicious IPs and attack patterns)
+- Rate limiting (100 requests per minute per IP)
+- Security headers (HSTS, XSS protection, content-type sniffing prevention)
+
+## Configuration
+
+Configuration is stored in `config.yaml` under `apps.crowdsec`:
+
+```yaml
+apps:
+  crowdsec:
+    rateLimitAverage: "100"
+    rateLimitBurst: "100"
+```
+
+## Secrets
+
+Secrets are stored in `secrets.yaml` under `apps.crowdsec`:
+
+```yaml
+apps:
+  crowdsec:
+    agentPassword: <auto-generated>
+    bouncerApiKey: <auto-generated>
+```
+
+## Opting Out
+
+To disable CrowdSec protection for a specific ingress (e.g., webhooks, health checks):
+
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  annotations:
+    traefik.ingress.kubernetes.io/router.middlewares: ""
+```
+
+## Using Only Rate Limiting
+
+To use rate limiting without threat detection:
+
+```yaml
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+  annotations:
+    traefik.ingress.kubernetes.io/router.middlewares: crowdsec-rate-limit@kubernetescrd
+```
+
+## Monitoring
+
+View active decisions (blocked IPs):
+```bash
+kubectl exec -n crowdsec deploy/crowdsec -- cscli decisions list
+```
+
+View registered bouncers:
+```bash
+kubectl exec -n crowdsec deploy/crowdsec -- cscli bouncers list
+```
+
+View alerts:
+```bash
+kubectl exec -n crowdsec deploy/crowdsec -- cscli alerts list
+```
+
+View metrics (Prometheus format):
+```bash
+kubectl port-forward -n crowdsec svc/crowdsec-lapi 6060:6060
+curl http://localhost:6060/metrics
+```
+
+## Threat Intelligence
+
+CrowdSec includes these detection collections:
+- `crowdsecurity/traefik` - Traefik-specific detections
+- `crowdsecurity/http-cve` - Known HTTP CVE exploits
+- `crowdsecurity/whitelist-good-actors` - Whitelist for known good actors (search engines, etc.)
+
+Enabled scenarios:
+- HTTP probing and path traversal detection
+- Bad user agent detection
+- Sensitive file access attempts
+- HTTP crawling detection
+- SSH brute force (if exposed)
+
+## Troubleshooting
+
+**Bouncer not connecting to agent:**
+```bash
+kubectl logs -n crowdsec deploy/traefik-crowdsec-bouncer
+kubectl exec -n crowdsec deploy/crowdsec -- cscli bouncers list
+```
+
+**Check if middleware is applied:**
+```bash
+kubectl get middleware -n crowdsec
+kubectl describe ingressroute -n <app-namespace> <route-name>
+```
+
+**View CrowdSec logs:**
+```bash
+kubectl logs -n crowdsec deploy/crowdsec
+```
--- a/crowdsec/configmap.yaml
+++ b/crowdsec/configmap.yaml
@@ -0,0 +1,43 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: crowdsec-config
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+data:
+  acquis.yaml: |
+    filenames:
+      - /var/log/containers/traefik-*_traefik_*.log
+    force_inotify: true
+    poll_without_inotify: true
+    labels:
+      type: containerd
+      program: traefik
+  profiles.yaml: |
+    name: default_ip_remediation
+    debug: false
+    filters:
+    - Alert.Remediation == true && Alert.GetScope() == "Ip"
+    decisions:
+    - type: ban
+      duration: 4h
+    on_success: break
+    ---
+    name: default_range_remediation
+    debug: false
+    filters:
+    - Alert.Remediation == true && Alert.GetScope() == "Range"
+    decisions:
+    - type: ban
+      duration: 4h
+      scope: Range
+    on_success: break
+  postoverflows.yaml: |
+    # Post-overflow configuration for crowdsec
+    name: "rdns"
+    debug: false
+    filter: "evt.Enriched.IsoCode != ''"
+    # Add reverse DNS enrichment
--- a/crowdsec/crowdsec-deployment.yaml
+++ b/crowdsec/crowdsec-deployment.yaml
@@ -0,0 +1,126 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: crowdsec
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: crowdsec
+      managedBy: kustomize
+      partOf: wild-cloud
+  template:
+    metadata:
+      labels:
+        app: crowdsec
+        managedBy: kustomize
+        partOf: wild-cloud
+    spec:
+      serviceAccountName: crowdsec
+      affinity:
+        podAffinity:
+          preferredDuringSchedulingIgnoredDuringExecution:
+          - weight: 100
+            podAffinityTerm:
+              labelSelector:
+                matchLabels:
+                  app: traefik
+              topologyKey: kubernetes.io/hostname
+      securityContext:
+        runAsUser: 0
+        runAsNonRoot: false
+        fsGroup: 0
+        seccompProfile:
+          type: RuntimeDefault
+      containers:
+      - name: crowdsec
+        image: crowdsecurity/crowdsec:v1.7.8
+        env:
+        - name: COLLECTIONS
+          value: "crowdsecurity/traefik crowdsecurity/http-cve crowdsecurity/whitelist-good-actors crowdsecurity/iptables crowdsecurity/linux"
+        - name: PARSERS
+          value: "crowdsecurity/traefik-logs crowdsecurity/http-logs crowdsecurity/nginx-logs"
+        - name: SCENARIOS
+          value: "crowdsecurity/http-crawl-non_statics crowdsecurity/http-probing crowdsecurity/http-sensitive-files crowdsecurity/http-bad-user-agent crowdsecurity/http-path-traversal-probing crowdsecurity/ssh-bf crowdsecurity/ssh-slow-bf"
+        - name: POSTOVERFLOWS
+          value: "crowdsecurity/rdns crowdsecurity/cdn-whitelist"
+        - name: GID
+          value: "1000"
+        - name: LEVEL_TRACE
+          value: "false"
+        - name: LEVEL_DEBUG
+          value: "false"
+        - name: LEVEL_INFO
+          value: "true"
+        - name: AGENT_USERNAME
+          value: "kubernetes-cluster"
+        - name: AGENT_PASSWORD
+          valueFrom:
+            secretKeyRef:
+              name: crowdsec-agent-secret
+              key: password
+        ports:
+        - name: lapi
+          containerPort: 8080
+          protocol: TCP
+        - name: prometheus
+          containerPort: 6060
+          protocol: TCP
+        livenessProbe:
+          httpGet:
+            path: /health
+            port: 8080
+          initialDelaySeconds: 30
+          periodSeconds: 30
+        readinessProbe:
+          httpGet:
+            path: /health
+            port: 8080
+          initialDelaySeconds: 10
+          periodSeconds: 10
+        resources:
+          requests:
+            cpu: 100m
+            memory: 200Mi
+          limits:
+            cpu: 500m
+            memory: 512Mi
+        securityContext:
+          allowPrivilegeEscalation: false
+          capabilities:
+            drop:
+            - ALL
+          runAsNonRoot: false
+        volumeMounts:
+        - name: crowdsec-config
+          mountPath: /etc/crowdsec/acquis.yaml
+          subPath: acquis.yaml
+          readOnly: true
+        - name: crowdsec-config
+          mountPath: /etc/crowdsec/profiles.yaml
+          subPath: profiles.yaml
+          readOnly: true
+        - name: crowdsec-data
+          mountPath: /var/lib/crowdsec/data
+        - name: crowdsec-config-dir
+          mountPath: /etc/crowdsec/config
+        - name: varlog
+          mountPath: /var/log
+          readOnly: true
+      volumes:
+      - name: crowdsec-config
+        configMap:
+          name: crowdsec-config
+      - name: crowdsec-data
+        persistentVolumeClaim:
+          claimName: crowdsec-data
+      - name: crowdsec-config-dir
+        emptyDir: {}
+      - name: varlog
+        hostPath:
+          path: /var/log
--- a/crowdsec/crowdsec-service.yaml
+++ b/crowdsec/crowdsec-service.yaml
@@ -0,0 +1,24 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: crowdsec-lapi
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+spec:
+  type: ClusterIP
+  selector:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+  ports:
+  - name: lapi
+    port: 8080
+    targetPort: 8080
+    protocol: TCP
+  - name: prometheus
+    port: 6060
+    targetPort: 6060
+    protocol: TCP
--- a/crowdsec/install.sh
+++ b/crowdsec/install.sh
@@ -0,0 +1,118 @@
+#!/bin/bash
+set -e
+set -o pipefail
+
+if [ -z "${WILD_INSTANCE}" ]; then
+    echo "ERROR: WILD_INSTANCE is not set"
+    exit 1
+fi
+
+if [ -z "${WILD_API_DATA_DIR}" ]; then
+    echo "ERROR: WILD_API_DATA_DIR is not set"
+    exit 1
+fi
+
+if [ -z "${KUBECONFIG}" ]; then
+    echo "ERROR: KUBECONFIG is not set"
+    exit 1
+fi
+
+INSTANCE_DIR="${WILD_API_DATA_DIR}/instances/${WILD_INSTANCE}"
+CROWDSEC_DIR="${INSTANCE_DIR}/apps/crowdsec"
+SECRETS_FILE="${INSTANCE_DIR}/secrets.yaml"
+
+echo "=== Setting up CrowdSec Security Engine ==="
+echo ""
+
+echo "Verifying Traefik is ready (required for CrowdSec bouncer)..."
+kubectl wait --for=condition=Available deployment/traefik -n traefik --timeout=60s 2>/dev/null || {
+    echo "WARNING: Traefik not ready, but continuing with CrowdSec installation"
+    echo "Note: CrowdSec bouncer will not work until Traefik is available"
+}
+
+echo "Using pre-compiled CrowdSec templates..."
+if [ ! -f "${CROWDSEC_DIR}/kustomization.yaml" ]; then
+    echo "ERROR: Compiled templates not found at ${CROWDSEC_DIR}"
+    echo "Templates should be compiled before deployment."
+    exit 1
+fi
+
+echo "Deploying CrowdSec..."
+kubectl apply -k ${CROWDSEC_DIR}/
+
+echo "Creating CrowdSec agent secret..."
+AGENT_PASSWORD=$(yq '.apps.crowdsec.agentPassword' "$SECRETS_FILE" 2>/dev/null | tr -d '"')
+
+if [ -z "$AGENT_PASSWORD" ] || [ "$AGENT_PASSWORD" = "null" ]; then
+    echo "Generating new agent password..."
+    AGENT_PASSWORD=$(openssl rand -base64 32)
+    echo "WARNING: Agent password not found in secrets.yaml"
+    echo "Using generated password - you may want to persist this"
+fi
+
+kubectl create secret generic crowdsec-agent-secret \
+  --namespace crowdsec \
+  --from-literal=password="${AGENT_PASSWORD}" \
+  --dry-run=client -o yaml | kubectl apply -f -
+
+echo "Waiting for CrowdSec agent to be ready..."
+kubectl rollout status deployment/crowdsec -n crowdsec --timeout=120s
+
+echo "Registering bouncer with CrowdSec agent..."
+BOUNCER_API_KEY=$(yq '.apps.crowdsec.bouncerApiKey' "$SECRETS_FILE" 2>/dev/null | tr -d '"')
+
+if [ -z "$BOUNCER_API_KEY" ] || [ "$BOUNCER_API_KEY" = "null" ]; then
+    echo "Generating new bouncer API key from CrowdSec agent..."
+    kubectl exec -n crowdsec deploy/crowdsec -- cscli bouncers delete traefik-bouncer 2>/dev/null || true
+    BOUNCER_API_KEY=$(kubectl exec -n crowdsec deploy/crowdsec -- cscli bouncers add traefik-bouncer -o raw)
+    echo "Generated bouncer API key - you may want to persist this in secrets.yaml"
+fi
+
+kubectl create secret generic crowdsec-bouncer-secret \
+  --namespace crowdsec \
+  --from-literal=api-key="${BOUNCER_API_KEY}" \
+  --dry-run=client -o yaml | kubectl apply -f -
+
+echo "Copying bouncer secret to traefik namespace..."
+kubectl create secret generic crowdsec-bouncer-secret \
+  --namespace traefik \
+  --from-literal=api-key="${BOUNCER_API_KEY}" \
+  --dry-run=client -o yaml | kubectl apply -f -
+
+echo "Cleaning up old bouncer deployment..."
+kubectl delete deployment traefik-crowdsec-bouncer -n crowdsec --ignore-not-found
+kubectl delete service traefik-crowdsec-bouncer -n crowdsec --ignore-not-found
+
+echo "Restarting Traefik to load CrowdSec plugin..."
+kubectl rollout restart deployment/traefik -n traefik
+kubectl rollout status deployment/traefik -n traefik --timeout=120s
+
+echo "Configuring Traefik to use CrowdSec security chain by default..."
+kubectl patch deployment traefik -n traefik --type='json' -p='[
+  {
+    "op": "add",
+    "path": "/spec/template/spec/containers/0/args/-",
+    "value": "--entryPoints.websecure.http.middlewares=crowdsec-security-chain@kubernetescrd"
+  }
+]' 2>/dev/null || {
+    echo "Note: Traefik may already have middleware configured or patch failed"
+    echo "You can manually configure default middleware if needed"
+}
+
+echo ""
+echo "CrowdSec installed successfully (using Traefik plugin)"
+echo ""
+echo "All ingresses are now protected by default with:"
+echo "  - Threat detection (CrowdSec Traefik plugin, stream mode)"
+echo "  - Rate limiting (100 req/min)"
+echo "  - Security headers (HSTS, XSS protection, etc.)"
+echo ""
+echo "To verify the installation:"
+echo "  kubectl get pods -n crowdsec"
+echo "  kubectl get pods -n traefik"
+echo "  kubectl exec -n crowdsec deploy/crowdsec -- cscli bouncers list"
+echo "  kubectl exec -n crowdsec deploy/crowdsec -- cscli decisions list"
+echo ""
+echo "To opt-out a specific ingress from CrowdSec protection:"
+echo "  Add annotation: traefik.ingress.kubernetes.io/router.middlewares: \"\""
+echo ""
--- a/crowdsec/kustomization.yaml
+++ b/crowdsec/kustomization.yaml
@@ -0,0 +1,17 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+namespace: crowdsec
+labels:
+  - includeSelectors: true
+    pairs:
+      app: crowdsec
+      managedBy: kustomize
+      partOf: wild-cloud
+resources:
+- namespace.yaml
+- serviceaccount.yaml
+- configmap.yaml
+- pvc.yaml
+- crowdsec-deployment.yaml
+- crowdsec-service.yaml
+- middleware.yaml
--- a/crowdsec/manifest.yaml
+++ b/crowdsec/manifest.yaml
@@ -0,0 +1,15 @@
+name: crowdsec
+is: crowdsec
+description: CrowdSec security engine with Traefik bouncer for threat detection and rate limiting
+version: v1.7.8
+namespace: crowdsec
+category: infrastructure
+requires:
+  - name: longhorn
+  - name: traefik
+defaultConfig:
+  rateLimitAverage: "100"
+  rateLimitBurst: "100"
+defaultSecrets:
+  - key: agentPassword
+  - key: bouncerApiKey
--- a/crowdsec/middleware.yaml
+++ b/crowdsec/middleware.yaml
@@ -0,0 +1,89 @@
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: crowdsec-bouncer
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+spec:
+  plugin:
+    bouncer:
+      crowdsecLapiScheme: http
+      crowdsecLapiHost: crowdsec-lapi.crowdsec.svc.cluster.local:8080
+      crowdsecLapiKeyFile: /etc/traefik/crowdsec/api-key
+      crowdsecMode: stream
+      updateIntervalSeconds: 15
+      defaultDecisionSeconds: 60
+      crowdsecAppsecEnabled: false
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: rate-limit
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+spec:
+  rateLimit:
+    average: {{ .rateLimitAverage }}
+    burst: {{ .rateLimitBurst }}
+    period: 1m
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: security-headers
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+spec:
+  headers:
+    browserXssFilter: true
+    contentTypeNosniff: true
+    forceSTSHeader: true
+    frameDeny: true
+    sslRedirect: true
+    stsIncludeSubdomains: true
+    stsPreload: true
+    stsSeconds: 31536000
+    addVaryHeader: true
+    accessControlAllowMethods:
+    - GET
+    - POST
+    - PUT
+    - DELETE
+    - OPTIONS
+    accessControlAllowOriginList:
+    - "*"
+    accessControlMaxAge: 100
+    customRequestHeaders:
+      X-Forwarded-Proto: https
+    customResponseHeaders:
+      Server: ""
+      X-Robots-Tag: noindex,nofollow,nosnippet,noarchive,notranslate,noimageindex
+---
+apiVersion: traefik.io/v1alpha1
+kind: Middleware
+metadata:
+  name: security-chain
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+spec:
+  chain:
+    middlewares:
+    - name: security-headers
+      namespace: crowdsec
+    - name: rate-limit
+      namespace: crowdsec
+    - name: crowdsec-bouncer
+      namespace: crowdsec
--- a/crowdsec/namespace.yaml
+++ b/crowdsec/namespace.yaml
@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud
+    pod-security.kubernetes.io/enforce: privileged
--- a/crowdsec/pvc.yaml
+++ b/crowdsec/pvc.yaml
@@ -0,0 +1,12 @@
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: crowdsec-data
+spec:
+  storageClassName: longhorn
+  accessModes:
+    - ReadWriteOnce
+  volumeMode: Filesystem
+  resources:
+    requests:
+      storage: 512Mi
--- a/crowdsec/serviceaccount.yaml
+++ b/crowdsec/serviceaccount.yaml
@@ -0,0 +1,9 @@
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: crowdsec
+  namespace: crowdsec
+  labels:
+    app: crowdsec
+    managedBy: kustomize
+    partOf: wild-cloud