Home automation Kubernetes cluster on Proxmox with GPU passthrough, NFS storage, and automatic DNS.
- 3-node Kubernetes cluster on Proxmox VMs (1 control plane, 2 workers; plus an on-demand game node) provisioned via cloud-init
- GPU passthrough with NVIDIA container runtime support (optional)
- NFS persistent storage on the Proxmox host, provisioned dynamically via CSI driver
- Automatic DNS + HTTPS via external-dns, Cloudflare, and Let's Encrypt
- Home Assistant with NFS-backed config and Ingress routing
- NInfer LLM inference server tuned for the RTX 3090 (default), with Ollama as a fallback for other GPUs, CPU, or other models
- OpenClaw AI agent with chat support backed by the local LLM server
- Hermes AI agent gateway backed by the local LLM server
- Game servers (e.g. Satisfactory) on an on-demand node, exposed via a free playit.gg tunnel
Proxmox Host
├── NFS server (/mnt/nfs/k3s)
├── VM: control-1 ── control plane + Traefik ingress
├── VM: data-1 ── worker + GPU (optional)
├── VM: data-2 ── worker
└── VM: data-3 ── game node (powered on on-demand)
Kubernetes Cluster
├── NFS CSI driver ── dynamic persistent volumes
├── external-dns ── auto-syncs Ingress hostnames to Cloudflare DNS
├── NVIDIA device plugin ── exposes GPU to pods (optional)
├── Home Assistant ── ha.<domain>, config persisted on NFS
├── NInfer ── ninfer.<domain>, Qwen inference on the GPU (default)
├── Ollama ── ollama.<domain>, fallback LLM server (disabled by default)
├── OpenClaw ── openclaw.<domain>, AI agent gateway using NInfer/Ollama
├── Hermes ── hermes.<domain>, AI agent gateway using NInfer/Ollama
└── Game servers ── Satisfactory etc. on data-3, via a playit.gg tunnel
VM hostnames are registered automatically via DHCP DNS, so the cluster uses hostnames instead of hardcoded IPs. external-dns keeps Cloudflare DNS records in sync with the cluster's current IPs.
.
├── cluster.sh # Cluster lifecycle management (runs on Proxmox host)
├── config.env.example # Cluster config template
├── cloud-init/
│ ├── k3s-control-plane.yaml # Control plane provisioning
│ ├── k3s-data-plane.yaml # Worker node provisioning
│ └── k3s-data-plane-gpu.yaml # GPU worker node provisioning
├── helm/
│ ├── Chart.yaml # Helm chart with dependencies
│ ├── values.yaml.example # Config template (copy to values.yaml)
│ └── templates/
│ ├── cloudflare-secret.yaml
│ ├── game-servers/
│ │ ├── playit-secret.yaml
│ │ └── satisfactory.yaml
│ ├── hermes.yaml
│ ├── home-assistant.yaml
│ ├── nfs-storage-class.yaml
│ ├── ninfer.yaml
│ ├── nvidia-device-plugin.yaml
│ ├── ollama.yaml
│ ├── openclaw.yaml
│ └── traefik-config.yaml
- A Kubernetes cluster with kubectl access — if you don't have one, see Host Setup below
- An NFS server accessible from your cluster nodes
- Cloudflare account with a domain
- A router that registers VM hostnames via DHCP (e.g., UniFi)
- Helm 3
cp helm/values.yaml.example helm/values.yamlEdit helm/values.yaml:
| Key | Description | Example |
|---|---|---|
domain |
Base domain for your apps | home.example.com |
nfs.server |
NFS server hostname or IP | pve |
cloudflare.apiToken |
Cloudflare API token with Edit Zone DNS permission | |
external-dns.domainFilters |
Your Cloudflare zone | [example.com] |
homeAssistant.timezone |
Your timezone | America/Los_Angeles |
gpu.enabled |
Enable NVIDIA GPU support (required by NInfer) | true / false |
acme.email |
Email for Let's Encrypt certificate expiry notices | you@example.com |
ninfer.enabled / ollama.enabled |
LLM server; enable one (both need the whole GPU) | true / false |
ninfer.apiKey |
Require this key on the NInfer API (optional) | |
ollama.models |
Ollama models to auto-download on startup | [qwen3.5:27b] |
openclaw.gatewayToken |
OpenClaw gateway auth token | Any random string |
openclaw.discordToken |
Discord bot token | |
openclaw.discordAllowedUsers |
Discord user IDs allowed to DM the bot | ["123456789"] |
openclaw.model |
Ollama model OpenClaw uses (NInfer always serves ninfer.modelId) |
qwen3.5:27b |
hermes.model |
Ollama model Hermes uses (NInfer always serves ninfer.modelId) |
qwen3.5:27b |
hermes.contextLength |
Context window length for the model | 65536 |
hermes.telegramToken |
Telegram bot token from @BotFather (optional) | |
hermes.telegramAllowedUsers |
Numeric Telegram user IDs allowed to message the bot | ["123456789"] |
cd helm
helm dependency update
helm install home-prod .Services will be available at https://<app>.<your-domain> once DNS propagates (may take a few minutes).
After changing values or templates:
cd helm
helm upgrade home-prod .NInfer-3090 is the default LLM server: a Qwen-only inference engine tuned for the RTX 3090, about 2x faster than Ollama on the same model. It serves an OpenAI-compatible API at http://ninfer:8080/v1 (https://ninfer.<your-domain>), and Hermes and OpenClaw use it whenever ninfer.enabled is true. The model downloads to NFS on first start.
The image is an unofficial build published at ghcr.io/jiapingzeng/ninfer-3090 (source).
Use Ollama for a GPU other than an RTX 3090, CPU-only inference (ollama.gpu: false), or models NInfer doesn't support. Both servers need the whole GPU, so set ninfer.enabled: false and ollama.enabled: true; Hermes and OpenClaw switch to Ollama and use hermes.model / openclaw.model.
Ollama auto-downloads models listed in ollama.models on startup. Models are persisted on NFS so they survive restarts. To pull additional models:
kubectl exec deploy/ollama -- ollama pull <model-name>OpenClaw is an AI agent gateway that connects to the local LLM server (NInfer, or Ollama when NInfer is disabled). Access the Control UI at https://openclaw.<your-domain> using the gateway token from your config.
To connect a Discord bot, create one at discord.com/developers with the bot and applications.commands scopes, enable the Message Content Intent, and add the token to your config. Add your Discord user ID to discordAllowedUsers to skip the pairing step.
Hermes is an AI agent gateway that connects to the local LLM server (NInfer, or Ollama when NInfer is disabled). Access it at https://hermes.<your-domain>. Tune hermes.contextLength to the model; on Ollama, also set hermes.model to a pulled model.
Game servers live in helm/templates/game-servers/ (one file per game, toggled by gameServers.<game>.enabled). They share the data-3 node and one playit.gg agent (gameServers.playit), so swapping games is a config change. data-3 is powered on only to play (it trades off against the Windows VM):
qm shutdown 901 && qm start 203 # start session
qm shutdown 203 && qm start 901 # end sessionplayit.gg (free, one-time): generate a Docker secret key, set gameServers.playit.secretKey, helm upgrade. Create the tunnel in the dashboard, note the assigned port(s), set the game's ports to match.
Satisfactory: one TCP/UDP tunnel, port count 2 → two consecutive ports; set gamePort to the first, messagingPort to the second. Players join via Server Manager → Add Server at <tunnel>.playit.gg:<gamePort>. Set satisfactory.lan.enabled: true for direct LAN play.
TLS certificates are automatically provisioned via Let's Encrypt using Cloudflare DNS-01 validation. This works for internal services — no ports need to be exposed to the internet. Traefik (the built-in K3s ingress controller) handles certificate issuance, renewal, and TLS termination.
Enabled by default (gpu.enabled), since NInfer needs it. Without a GPU, set gpu.enabled: false and run Ollama on CPU instead (see Ollama). The NVIDIA device plugin only runs on nodes labeled nvidia.com/gpu=true. GPU nodes are also tainted nvidia.com/gpu=true:NoSchedule so only GPU workloads (NInfer/Ollama) schedule there. Both are set automatically by the cloud-init script.
Verify:
kubectl describe node <gpu-node> | grep nvidia.com/gpu
# Should show nvidia.com/gpu: 1 under AllocatableThis section covers setting up a K3s cluster on Proxmox from scratch. K3s is a lightweight Kubernetes distribution that's well-suited for home use — it bundles Traefik ingress, CoreDNS, and a local storage provider out of the box with minimal resource overhead. Skip this if you already have a Kubernetes cluster.
Create a dedicated volume for persistent storage that survives VM rebuilds.
1. Create and mount a volume
lvcreate -V 256G -T pve/data -n nfs # adjust size as needed
mkfs.ext4 /dev/pve/nfs
mkdir -p /mnt/nfs
mount /dev/pve/nfs /mnt/nfsAdd to /etc/fstab so it persists across reboots:
/dev/pve/nfs /mnt/nfs ext4 defaults 0 2
2. Install and configure NFS server
apt install -y nfs-kernel-server
mkdir -p /mnt/nfs/k3s
chown nobody:nogroup /mnt/nfs/k3sAdd to /etc/exports (adjust the subnet to match your network):
/mnt/nfs/k3s <your-subnet>(rw,sync,no_subtree_check,no_root_squash)
Apply:
exportfs -ra
systemctl restart nfs-kernel-serverSkip this section if you don't have a GPU.
1. Enable IOMMU
Edit /etc/default/grub and add IOMMU flags to GRUB_CMDLINE_LINUX_DEFAULT:
- AMD:
quiet amd_iommu=on iommu=pt - Intel:
quiet intel_iommu=on iommu=pt
Then run update-grub.
2. Load VFIO modules
Create /etc/modules-load.d/vfio.conf:
vfio
vfio_iommu_type1
vfio_pci
3. Blacklist GPU drivers on host
Create /etc/modprobe.d/blacklist.conf:
blacklist nouveau
blacklist nvidia
4. Bind GPU to VFIO
Find your GPU's vendor:device IDs with lspci -nn, then create /etc/modprobe.d/vfio.conf:
options vfio-pci ids=<gpu-id>,<audio-id>
5. Apply and reboot
update-initramfs -u -k all
rebootVerify after reboot: lspci -nnk -s <pci-slot> | grep -i driver should show vfio-pci.
6. Create GPU resource mapping
In the Proxmox UI: Datacenter > Resource Mappings > Add PCI Device. Use the mapping name as GPU_MAPPING in config.env.
Prepare SSH keys so you can access VMs from both the Proxmox host and your workstation.
# On the Proxmox host, combine keys into a single file
cat ~/.ssh/id_rsa.pub > /tmp/keys.pubAppend your workstation's public key to the same file:
echo "<your-workstation-public-key>" >> /tmp/keys.pubUse this path as SSH_KEYS in config.env.
In the Proxmox UI: Datacenter > Storage > local > Edit > Content > add Snippets. This allows cloud-init files to be stored on the host.
cp config.env.example config.envEdit config.env:
| Key | Description | Example |
|---|---|---|
K3S_TOKEN |
Shared secret for K3s nodes to join | Any random string |
K3S_VERSION |
Pinned K3s version (keeps rebuilds reproducible) | v1.34.6+k3s1 |
GPU_MAPPING |
Proxmox GPU resource mapping name | rtx3090 |
NFS_SERVER |
NFS server hostname | pve |
CONTROL_PLANE_HOST |
Control plane VM hostname | control-1 |
SSH_KEYS |
Path to SSH public keys file | /tmp/keys.pub |
# Create VM template and start cluster
./cluster.sh template-create
./cluster.sh start
# Verify (wait a few minutes for cloud-init to finish)
./cluster.sh statusNote: The GPU worker node reboots automatically after cloud-init finishes to load the NVIDIA kernel module. This is expected — it rejoins the cluster after ~1 minute.
From your workstation:
ssh ubuntu@<control-plane-host> "sudo cat /etc/rancher/k3s/k3s.yaml" \
| sed "s/127.0.0.1/<control-plane-host>/" > ~/.kube/config./cluster.sh start # Create and start all VMs
./cluster.sh clean # Stop and destroy all VMs
./cluster.sh rebuild # Destroy and recreate entire cluster (incl. template)
./cluster.sh rebuild 201 # Destroy and recreate a single node
./cluster.sh update # Apply cluster.sh VM settings (cores, memory, ...) in place
./cluster.sh update 201 # Same, for a single node
./cluster.sh status # Check node status
./cluster.sh template-create # Create VM template
./cluster.sh template-clean # Destroy VM template