Building a small Kubernetes cluster from scratch - Can it really be that hard?
Why
I wanted to learn Kubernetes properly — not just read about it, but actually understand how it works by running it myself. So I set one up. This can’t be that hard, right? Right? Turns out I was wrong.
This post covers how I built the cluster, what went wrong, what I run on it now, and what I’m planning next. The happy-path guides already exist — I’ll focus on the parts they skip.
The Setup
The cluster is three nodes: one control plane and two workers. That means there’s a single point of failure — if the control plane dies, the whole cluster goes down. For a homelab that’s fine. If it breaks, I restart it. The goal wasn’t a production-grade cluster with guaranteed uptime; it was to learn how the thing actually works.
OS Choice
Most Kubernetes guides assume Ubuntu or Debian 12. Mine doesn’t follow one.
I went with Debian 13 (Trixie) — not for any technical reason, but because my entire homelab already runs on it and I wasn’t going to reinstall everything just to follow a guide more closely. Turns out it works fine, at least at this scale.
Networking
For the CNI plugin I chose Flannel over Calico or Cilium. Flannel does one thing: give pods a network. It’s simple,
well-documented, and one kubectl apply away from working. Calico and Cilium are powerful, but they bring complexity —
network policy enforcement, eBPF, a much bigger configuration surface. None of that buys me anything in a small homelab.
I’d rather understand the basics first.
Container Runtime
My first instinct was Docker — I’ve been using it for a while now, it runs (almost) everything else in my homelab, and it’s what you see being used everywhere. So I was a little bit surprised to find out Kubernetes dropped Docker support in 2022.
I learned that Docker uses containerd under the hood anyway. Kubernetes now talks to containerd directly, cutting out the Docker layer entirely. Once I understood that, the switch was obvious — there’s no reason to add that extra layer back.
Installation
Prerequisites
Before even touching Kubeadm, there are a few things Kubernetes requires on each node. With my Docker background, none of them were obvious. But Kubernetes works at a lower level than other tools I’ve used before, so this was new for me.
Disabling Swap
First, swap needs to be disabled. Kubernetes assumes that it has full control over memory allocation. This is because the scheduler decides where to place pods based on available memory; If a node reports 4GB free, Kubernetes trusts that. But if the OS is using swap, these numbers may become unreliable and pods might end up on nodes that don’t have the required resources available.
At first, I thought that I could just disable it and it would work. So I tried it with just the following:
swapoff -a
This seemed to work pretty well, until i did a reboot. After a bit of research, I found out that I needed to disable
swap in /etc/fstab. So that’s what I did. In the end, the following commands provided a working and consistent
solution:
swapoff -a
sed -i '/\bswap\b/d' /etc/fstab
Kernel Modules
On each node, there’s two kernel modules that need to be loaded. overlay is used by containerd
to manage the container filesystem. It allows containers to have their own filesystem
view without copying everything, by layering changes on top of a base image.
br_netfilter is needed so that iptables can see traffic crossing network bridges.
Kubernetes routes traffic between pods using iptables rules, and without this module,
bridged packets bypass those rules entirely, meaning pod-to-pod traffic would skip
all the routing logic Kubernetes sets up.
cat <<EOF | tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
modprobe overlay
modprobe br_netfilter
Sysctl settings
With br_netfilter loaded, iptables can see bridged network traffic, but only if the kernel is configured to actually
send it there. These three settings tell the kernel to pass bridged IPv4 and IPv6 packets through iptables, and to
forward traffic between interfaces. Without them, the loaded modules don’t fulfill their purpose.
cat <<EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF
sysctl --system
Installing Containerd
Containerd is the container runtime. It is the piece of software that pulls images, unpacks them, and runs containers. Kubernetes doesn’t run containers itself. Instead, it tells a runtime what to do via a standard interface called the CRI (Container Runtime Interface). Containerd implements that interface.
You might wonder why not just use Docker. Docker was originally used, but Kubernetes dropped direct Docker support in
The reason is that Docker was never designed to be a Kubernetes runtime. It’s a full developer tool with a daemon, a CLI, and a lot of machinery that Kubernetes doesn’t need. Docker actually uses Containerd under the hood, so using it directly just removes an unnecessary layer.
apt-get update && apt-get install -y containerd
After installing, you need to generate a proper config file. The default containerd install doesn’t create one, and without it containerd falls back to built-in defaults — which, depending on the version, can include disabling the CRI plugin (more on that in the “What went wrong” section):
mkdir -p /etc/containerd
containerd config default | tee /etc/containerd/config.toml
systemctl restart containerd
systemctl enable containerd
Installing kubeadm, kubelet and kubectl
Three separate tools, three separate jobs:
- kubelet runs on every node. It’s the agent that receives instructions from the control plane and manages the pods on that machine — starting containers, monitoring their health, and reporting status back.
- kubeadm is a one-time tool for bootstrapping the cluster. It generates certificates, configures etcd (the key-value store Kubernetes uses internally), and sets up the control plane components. You run it once per node and then mostly forget about it.
- kubectl is the CLI you use to interact with the cluster — deploying workloads, checking pod status, reading logs.
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.32/deb/Release.key \
| gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
https://pkgs.k8s.io/core:/stable:/v1.32/deb/ /' \
| tee /etc/apt/sources.list.d/kubernetes.list
apt update
apt install -y kubelet kubeadm kubectl
apt hold kubelet kubeadm kubectl
The apt hold is important. The three packages need to stay on the same
version — if kubelet and the control plane components drift out of sync, the cluster
breaks. Holding them prevents an unattended upgrade from silently causing that.
Initializing the control plane
With everything in place, kubeadm init does the heavy lifting. It generates the
cluster’s certificate authority, configures etcd, and starts the control plane
components as static pods.
kubeadm init --pod-network-cidr=10.244.0.0/16
The --pod-network-cidr flag defines the IP range that pods across the cluster will
use. This value needs to match what the CNI plugin expects — Flannel specifically
requires 10.244.0.0/16. Using a different range will result in pod networking
silently not working, which is a frustrating thing to debug after the fact.
Once it completes, kubeadm prints a kubeadm join command containing a bootstrap
token and a certificate hash. This is needed to join another node to the cluster.
Then configure kubectl to talk to the cluster:
mkdir -p $HOME/.kube
cp /etc/kubernetes/admin.conf $HOME/.kube/config
This copies the admin kubeconfig — the credentials and endpoint information kubectl needs — to the default location it looks for it.
Applying the CNI
At this point the control plane is up, but no pods can communicate yet. Without a
CNI plugin, pods get IP addresses but there’s no routing between them. Flannel fixes
that by creating a simple overlay network — each node gets a subnet from the
10.244.0.0/16 range, and Flannel handles routing traffic between them:
kubectl apply -f https://github.com/flannel-io/flannel/releases/latest/download/kube-flannel.yml
Joining the workers
Run the join command kubeadm generated on each worker node. It uses the bootstrap token to authenticate with the control plane and the certificate hash to verify it’s talking to the right cluster — preventing a worker from accidentally joining the wrong one:
kubeadm join <control-plane-ip>:6443 --token \
--discovery-token-ca-cert-hash sha256:
After a minute or two, all three nodes should appear as Ready:
watch kubectl get nodes
What went wrong
The Containerd CRI Bug
After joining the two workers, one of them kept crashing. The kubelet logs showed this:
failed to run Kubelet: validate service connection: validate CRI v1 runtime API for endpoint "unix:///var/run/containerd/containerd.sock": rpc error: code = Unimplemented desc = unknown service runtime.v1.RuntimeService
Containerd was running. The socket existed. The error message points at the socket, so naturally I started there — restarting containerd, checking permissions, verifying the socket path. None of it helps, because the socket isn’t the problem.
The actual culprit was a single line buried in /etc/containerd/config.toml:
disabled_plugins = ["cri"]
The CRI plugin — the exact interface kubelet uses to talk to containerd — was explicitly disabled. This isn’t something you’d configure intentionally. It can end up in the config depending on how containerd was installed, particularly if it was installed as a dependency of something else rather than directly. The default config generation step I mentioned earlier is supposed to prevent this, but if containerd was already installed and already had a config file, that step doesn’t overwrite it. The fix is simply deleting said line.
What makes this bug particularly unpleasant is that nothing in the error message, the kubelet logs, or the containerd logs mentions disabled plugins. You’re looking at a socket error, so you debug the socket. The config file doesn’t cross your mind until you’ve exhausted everything else. If you hit this error, check the config file first.
Docker without a systemd service
Worker 1 had Docker installed at some point before the cluster was set up. Somehow, this installation broke. I assume this happened because I did some basic setup on the control plane and then cloned the VM with its disks, probably while a process was running. Afterward, was no systemd unit file for it. Docker was technically installed, but it didn’t start on boot.
This didn’t matter for Kubernetes itself, which uses containerd. It mattered later when I set up Jenkins with autoscaling agents and the pipeline started building Docker images. Builds on worker-1 failed with:
Cannot connect to the Docker daemon at unix:///var/run/docker.sock.
Is the docker daemon running?
The daemon wasn’t running because nothing had started it. A reboot would have revealed this immediately, but the node hadn’t been rebooted since Docker was installed.
The fix was reinstalling Docker properly using the official script, which creates the systemd service:
curl -sSL https://get.docker.com | sudo sh
systemctl start docker
systemctl enable docker
The lesson here is that “it works right now” and “it will still work after a reboot” are two different things. If you install something manually, always verify that it works consistently across reboots.
The dashboard that looked broken but wasn’t
After fixing the containerd issue and restarting the cluster, the Kubernetes Dashboard appeared completely dead. The
port-forward service was running. The pod showed as running in kubectl get pods. But loading the UI in the browser
returned nothing.
I checked the port-forward logs, restarted it, checked the pod status again, looked for errors in the dashboard pod logs. Everything looked fine on paper. Then, about 15-20 seconds after the cluster finished coming back up, the dashboard loaded perfectly.
What happened is that the pod reports as Running as soon as the container starts, but the dashboard application inside
it takes a few extra seconds to actually start serving traffic. The port-forward connects immediately and the browser
tries to load before the app is ready, so you get nothing — which looks exactly like a broken setup.
It’s not a bug, and there’s nothing to fix. But after spending time debugging the containerd CRI issue and then the dashboard appearing dead, it’s easy to assume something is still wrong.
Conclusion
Setting up a Kubernetes cluster from scratch is genuinely worth doing if you want to understand how it works. Reading about the control plane, the kubelet, and the CNI is one thing — actually watching a pod fail to schedule because the CRI plugin is disabled is another.
The installation itself isn’t that complicated once you understand why each step exists. The prerequisites — swap, kernel modules, sysctl settings — exist for real reasons, and knowing those reasons makes the whole thing less intimidating. kubeadm handles the hard parts.
What actually takes time is the debugging. The containerd CRI bug cost me the most time, and the fix was removing a single line from a config file. That’s pretty typical of infrastructure problems: the fix is usually small, finding it is the hard part.
The cluster now runs Jenkins with autoscaling agents, which I’ll cover in a follow-up post. I may also deploy some other applications on there, just to see how it holds up. If you’re setting something similar up and hit an issue I didn’t cover, feel free to reach out.