Homelab Kubernetes (K3s) Infrastructure Overview

This homelab is a production-grade Kubernetes (K3s) cluster designed to replicate real-world cloud and platform engineering patterns on affordable hardware. It focuses on high availability, security, observability, storage resilience, and network segmentation, while remaining fully self-hosted.

The cluster is used to run both publicly exposed services and private internal applications, protected by VPN and security tooling.

1. Compute Layer

Kubernetes Nodes

The cluster consists of multiple nodes with mixed architectures, providing realistic operational constraints:

• 4 × Raspberry Pi nodes

  • Used as Kubernetes control plane and worker nodes
  • Each node runs a CrowdSec firewall bouncer for node-level protection

• 1 × Intel NUC (Mini PC)

  • Used as Kubernetes control plane
  • Higher-performance node for workloads requiring more CPU/RAM
  • Also runs a CrowdSec firewall bouncer

2. Kubernetes Distribution

• K3s is used as the Kubernetes distribution
• Cluster is deployed in HA mode (multiple control-plane nodes)

3. Networking & Traffic Management

Ingress & Load Balancing

• Traefik is used as the Kubernetes Ingress Controller

  • Handles HTTP/HTTPS routing
  • Separates public applications from internal-only services

• MetalLB provides bare-metal LoadBalancer functionality

  • Assigns stable IP addresses to services
  • Supports real client IP preservation

Network Segmentation

• Public apps

  • Accessible from the internet via Traefik

• Internal apps

  • Only accessible inside the cluster or via VPN

VPN Access

• WireGuard provides secure remote access to internal services
• Allows safe administration and private app access without exposing them publicly

4. Security Stack

Security is implemented at multiple layers:

CrowdSec

• CrowdSec agent runs on each node as a daemonset

  • Monitors logs and system metrics for suspicious activity
  • Detects brute-force attacks, port scans, and other threats

• Firewall bouncer installed on each node

  • Enforces bans at the OS firewall level
  • Protects both Kubernetes services and node-level access
• CrowdSec decisions are shared across the infrastructure
• Crowdsec WAF is used to protect public-facing services

Security & Upgrade

• Secrets are encrypted at rest using aescbc provider.
• Automatic cluster upgrades via Rancher System Upgrade Controller

5. Observability & Monitoring

A full monitoring stack is deployed inside Kubernetes:

• Prometheus Operator

  • Metrics collection

• Grafana

  • Dashboards and visualization

• Alertmanager

  • Alert routing and notifications

This provides visibility into:

• Node health
• Pod performance
• Resource usage
• Application metrics

6. Storage Architecture

Local & Network Storage

The infrastructure uses multiple storage backends, depending on workload needs:

• Btrfs-based raid-1 storage

  • Used for Kubernetes persistent volumes
  • Provides snapshots and data integrity
  • Mounted as NFS shares for cluster access

• External raid-1 HDD

  • Used for media storage

Backups

• Object storage (S3-compatible) hosted externally

  • Automated backups of databases

  • Enables disaster recovery and off-site redundancy

• Restic

  • Used for efficient, encrypted backups to an external disk

7. Data & Supporting Services

Inside the cluster, several shared services are deployed:

• Redis

  • Caching and background job support

• RabbitMQ

  • Asynchronous messaging

• Pi-hole

  • Internal DNS filtering

• CrowdSec API

  • Central decision engine for security enforcement

• PostgreSQL

  • Application databases
  • Installed directly on OS

8. Infrastructure management

• Hosts managed via Ansible for configuration consistency
• Kubernetes manifests stored in GitHub repository
• GitHub Actions used for CI/CD deployments
• Self-hosted GitHub runners deployed in the cluster for secure deployments
• External resources and record DNS managed with Terraform

History

The homelab was started in 2020 by just hosted service on a single Raspberry Pi and then evolved into a docker setup on multiple node before moving to k3s in 2024.