Paul Payne
9fd046c378
New version slots (major version jumps): - gitea: 1.24.3 -> 28.2.0 (slot 1 -> 28) - ghost: 5.130.6 -> 6.72.0 (slot 5 -> 6) - formbricks: 3.6.0 -> 6.1.0 (slot 3 -> 6) - mattermost: 10.8.1 -> 12.0.1 (slot 10 -> 12) - zulip: 9.4 -> 12.3 (slot 9 -> 12) - immich: 1.135.3 -> 3.3.5 (slot 1 -> 3) - openproject: 16.1.1 -> 17.9.3 (slot 16 -> 17) - listmonk: 5.0.3 -> 6.1.0 (slot 5 -> 6) - discourse: 3.5.3 -> 2026.10.0 (slot 3 -> 2026, calver) Same-slot large bumps: - vllm: 0.9.2 -> 0.32.0 - longhorn: v1.8.1 -> v1.13.0 Infrastructure: - cert-manager: v1.17.2 -> v1.21.2 - coredns: v1.12.0 -> v1.14.7 - externaldns: v0.13.4 -> v0.23.0 - metallb: v0.15.0 -> v0.16.1 - mysql: 9.1.0 -> 9.7.4 - node-feature-discovery: v0.17.3 -> v0.20.1 - nvidia-device-plugin: v0.17.1 -> v0.20.1
vLLM
vLLM is a fast and easy-to-use library for LLM inference and serving with an OpenAI-compatible API. Use it to run large language models on your own hardware.
Dependencies
None, but requires a GPU node in your cluster.
Configuration
Key settings configured through your instance's config.yaml:
- model - Hugging Face model to serve (default:
Qwen/Qwen2.5-7B-Instruct) - maxModelLen - Maximum sequence length (default:
8192) - gpuProduct - Required GPU type (default:
RTX 4090) - gpuCount - Number of GPUs to use (default:
1) - gpuMemoryUtilization - Fraction of GPU memory to use (default:
0.9) - domain - Where the API will be accessible (default:
vllm.{your-cloud-domain})
Access
After deployment, the OpenAI-compatible API will be available at:
https://vllm.{your-cloud-domain}/v1
Other apps on the cluster (such as Open WebUI) can connect internally at http://vllm-service.llm.svc.cluster.local:8000/v1.
Hardware Requirements
This app requires a GPU node in your cluster. Adjust the gpuProduct, gpuCount, and memory settings to match your available hardware.