wild-cloud/wild-central
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Resilient HAProxy config: isolate broken services on validation failure

Browse Source 
When haproxy -c fails, parse ALERT line numbers, map to service domains
via # service: comments in generated config, exclude broken services,
and regenerate. One retry — prevents a single bad registration from
blocking all config updates.
This commit is contained in:
Paul Payne
2026-07-10 07:00:27 +00:00
parent 68d6fde80d
commit 6aa1e7d438
4 changed files with 221 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
TODO.md
View File

@@ -1,13 +1,18 @@
# TODO
## Header validation in service registration
## Response headers don't work with host ACL conditions
A bad header value from a client (e.g., `Access-Control-Allow-Headers: Content-Type, Authorization, ...` with commas/spaces) can produce invalid HAProxy config syntax. When HAProxy config validation fails, the reconciler skips the write — but the bad service persists, so the next reconcile cycle fails identically. This blocks ALL config updates for ALL services until the bad registration is fixed manually.
`http-response set-header ... if host_X` doesn't work because `hdr(host)` is a request-phase fetch that HAProxy can't evaluate in the response phase. The headers are generated but silently ignored (HAProxy logs a WARNING).
### Needed
Fix: capture the host in a transaction variable during the request phase, then use it in response rules:
```haproxy
http-request set-var(txn.host) hdr(host)
acl is_keila var(txn.host) -i keila.cloud.payne.io
http-response set-header Access-Control-Allow-Origin "https://cloud.payne.io" if is_keila
```
1. **Input validation at registration time** — reject or sanitize header values that would break HAProxy config. HAProxy header values with spaces/commas need quoting; values with special characters (newlines, NULs) should be rejected outright.
Affects: keila (CORS headers), plausible (Private-Network-Access header).
2. **Graceful degradation in reconciliation** — if generating config for one service produces an invalid HAProxy config, skip that service and log a warning rather than failing the entire reconciliation. Approach: generate config, validate, if invalid → bisect to find the offending service → exclude it and regenerate.
## Input validation for header values
3. **Current workaround** — header values are Go-`%q`-quoted in the HAProxy config output. This handles spaces/commas but may produce backslash escapes that HAProxy doesn't understand for edge cases.
Header values with special characters (newlines, NULs) should be rejected at registration time. Currently values are Go-`%q`-quoted in the HAProxy output which handles spaces/commas but may produce backslash escapes that HAProxy doesn't understand for edge cases.