Multi-Node Kubernetes (k3s), GitOps & Private AI Homelab Infrastructure
Self-hosted multi-node Proxmox and k3s production cluster running private LLM/Ollama inference, PostgreSQL + pgvector workloads, zero-trust networking, automated backups, and full observability.
Engineering Motivation & Architecture Goals
Advising European enterprises on sovereign on-premise AI, Kubernetes networking, and cost-predictable cloud alternatives requires operating the full stack firsthand — from bare-metal hypervisors and ZFS storage down to ingress controllers, TLS rotation, and local LLM GPU/CPU scheduling.
The platform hosts production web apps, .NET APIs, PostgreSQL + pgvector databases, Tolgee localization, and private AI inference under strict resource, security, and uptime discipline.
System Architecture & Technical Decisions
Segmented Bare-Metal Virtualization & Storage
Built on Proxmox VE nodes backed by TrueNAS ZFS pools with scheduled snapshots and off-node replication, isolated across VLANs behind an OPNsense firewall.
Declarative k3s Kubernetes & Traefik Edge Routing
Managed workloads via Kustomize overlays, Traefik IngressRoutes, strict security-header middlewares, liveness/readiness probes, and rolling zero-downtime deployments.
Private AI & Hybrid RAG Reference Stack
Runs local Ollama embedding and chat models alongside PostgreSQL + pgvector to benchmark latency, quantization quality, and air-gapped RAG architectures before client rollouts.
Full-Stack Telemetry & Self-Hosted CI Runners
Instrumented cluster nodes, pods, PostgreSQL, and ingress traffic with Prometheus, Grafana, structured logging, and self-hosted GitHub Actions build/deploy pipelines.
Hardening, Backup & Disaster Recovery
- Strict network policies, Cloudflare Tunnel / TLS termination, HSTS, and CSP header verification.
- Secrets isolation, least-privilege container security contexts, and non-root runtime images.
- Automated PostgreSQL & ZFS snapshot backups with tested restore runbooks.
Measured Engineering Value
- Hosts live multi-application production workloads (`razvanmares.ro`, `.NET 10` APIs, analytics, and private AI services) with sub-second edge response times.
- Serves as a battle-tested reference blueprint for GDPR-sovereign, on-premise Enterprise AI deployments.
- Eliminates cloud vendor lock-in while proving reproducible GitOps infrastructure discipline.