Homelab Kubernetes (K3s) Infrastructure Overview

This homelab is a production-grade Kubernetes (K3s) cluster designed to replicate real-world cloud and platform engineering patterns on affordable hardware. It focuses on high availability, security, observability, storage resilience, and network segmentation, while remaining fully self-hosted.

The cluster is used to run both publicly exposed services and private internal applications, protected by VPN and security tooling.

1. Compute Layer

Kubernetes Nodes

The cluster consists of multiple nodes with mixed architectures, providing realistic operational constraints:

• 4 × Raspberry Pi nodes

  • Used as Kubernetes control plane and worker nodes
  • Each node runs a CrowdSec firewall bouncer for node-level protection

• 1 × HP ProDesk 405 G4 (Mini PC)

  • Used as Kubernetes control plane
  • Higher-performance node for workloads requiring more CPU/RAM
  • Also runs a CrowdSec firewall bouncer

2. Kubernetes Distribution

• K3s is used as the Kubernetes distribution
• Cluster is deployed in HA mode (multiple control-plane nodes)

3. Networking & Traffic Management

Ingress & Load Balancing

• Traefik is used as the Kubernetes Ingress Controller

  • Handles HTTP/HTTPS routing
  • Separates public applications from internal-only services

• MetalLB provides bare-metal LoadBalancer functionality

  • Assigns stable IP addresses to services
  • Supports real client IP preservation

Network Segmentation

• Public apps

  • Accessible from the internet via Traefik

• Internal apps

  • Only accessible inside the cluster or via VPN

VPN Access

• WireGuard provides secure remote access to internal services
• Allows safe administration and private app access without exposing them publicly

Kube-VIP

• Kube-VIP provides a load balancer for the control plane.

4. Security Stack

Security is implemented at multiple layers:

CrowdSec

• CrowdSec agent runs on each node as a daemonset

  • Monitors logs and system metrics for suspicious activity
  • Detects brute-force attacks, port scans, and other threats

• Firewall bouncer installed on each node

  • Enforces bans at the OS firewall level
  • Protects both Kubernetes services and node-level access
• CrowdSec decisions are shared across the infrastructure
• Crowdsec WAF is used to protect public-facing services

Security & Upgrade

• Secrets are encrypted at rest using aescbc provider.
• Automatic cluster upgrades via Rancher System Upgrade Controller

5. Observability & Monitoring

A full monitoring stack is deployed inside Kubernetes:

• Prometheus Operator

  • Metrics collection

• Grafana

  • Dashboards and visualization

• Alertmanager

  • Alert routing and notifications

This provides visibility into:

• Node health
• Pod performance
• Resource usage
• Application metrics

6. Storage Architecture

Local & Network Storage

The infrastructure uses multiple storage backends, depending on workload needs:

• 2 SATA-SSD

  • Available on two different hosts
  • Used for Kubernetes persistent volumes
  • Provided via Longhorn

• 2 HDD

  • Used for Immich data
  • Managed with Longhorn for replication and snapshots

• External HDD

  • Used for media storage

• External HDD

  • Backup of the media storage

Backups

• Object storage (S3-compatible) hosted externally

  • Automated backups of databases

  • Enables disaster recovery and off-site redundancy

7. Data & Supporting Services

Inside the cluster, several shared services are deployed:

• Redis

  • Caching and background job support

• RabbitMQ

  • Asynchronous messaging

• Pi-hole

  • Internal DNS filtering

• CrowdSec API

  • Central decision engine for security enforcement

• PostgreSQL

  • Application databases
  • Installed directly on OS

8. Infrastructure management

• Hosts managed via Ansible for configuration consistency
• Kubernetes manifests stored in GitHub repository
• GitHub Actions used for CI/CD deployments
• Self-hosted GitHub runners deployed in the cluster for secure deployments
• External resources and record DNS managed with Terraform

History

The homelab started in 2020 by self-hosting Nextcloud on a single Raspberry Pi and then evolved through a multi-node Docker deployment before migrating to k3s in 2024.