Home Lab: Chapter 3 — Kubernetes Setup

Question

Howdy!

In this chapter, I'll walk through the setup of the Kubernetes cluster using Talos. The cluster will consist of three physical machines, each serving as both control-plane and worker to allow workloads scheduling on all nodes while maintaining control-plane functionality.

What is Talos?

Talos is a modern, minimalistic operating system designed specifically to run Kubernetes and nothing else. It is immutable, meaning the OS is read-only and cannot be modified, improving security by making it difficult for attackers to alter the system.

Talos is also managed entirely via Kubernetes, simplifying cluster operations.

Setting up Talos

Talos provides a command-line tool called talosctl, used to manage and interact with the cluster. To download it:

# Linux
curl -sL https://talos.dev/install | sh

# MacOS
brew install siderolabs/tap/talosctl

The process to install Talos OS involves:

1. Download the Talos image
2. Flash the image to a USB drive
3. Boot the node from the USB drive
4. Install Talos on the nodes
5. Reboot the nodes

Preparing Nodes

Before configuration, assign IP addresses to the nodes using DHCP with static leases by mapping each node's MAC address to a specific IP:

• Node 1: x.x.x.101
• Node 2: x.x.x.102
• Node 3: x.x.x.103

Restrict the DHCP pool (e.g., x.x.x.101 to x.x.x.104) to prevent assigning addresses to unexpected devices.

Setting up the Nodes

Download the Talos image from the Talos releases page and flash it to USB:

sudo dd if=talos.iso of=/dev/sdX bs=4M status=progress && sync

Replace /dev/sdX with the USB drive path. This command overwrites the USB drive data.

Boot the node from USB, enter BIOS/UEFI to configure boot priority, then install:

talosctl install --node x.x.x.x

Repeat for each node in the cluster.

Prepare Nodes Config

Generate initial cluster configuration:

talosctl gen config

This generates three files:

• talosconfig: Used by talosctl to connect to the cluster
• controlplane.yaml: Configuration for control plane nodes
• worker.yaml: Not needed since all nodes act as control plane nodes

Patching Nodes Config

Rather than manually editing files, use patches for a structured approach:

Key modifications include:

• Allow Control Plane Workloads: Enable workloads on control plane nodes
• Node Hostnames: Set unique hostnames (clustarino-k8s-1, -2, -3)
• Interface Names: Allow interface IDs to be identifiable
• DHCP: Enable DHCP on eth0
• Disable Kubeproxy and CNI: Install our own CNI
• DNS & NTP: Configure name resolution and time servers
• Disk: Specify installation disk
• Metrics Server: Enable cluster metrics
• VIP (Virtual IP): Configure virtual IP for cluster access

Note: This post has been shortened due to character limitations. For complete patch configurations and detailed explanations, refer to the full documentation.

Generating secrets

Generate secrets for secure cluster communication:

talosctl gen secrets --output-file outputs/secrets.yaml

Generating final Nodes Config

Generate final configuration with all patches:

talosctl gen config clustarino https://x.x.x.105:6443 \
    --with-secrets outputs/secrets.yaml \
    --config-patch @patches/allow-controlplane-workloads.yaml \
    --config-patch @patches/dhcp.yaml \
    --config-patch @patches/disable-kube-proxy-and-cni.yaml \
    --config-patch @patches/install-disk.yaml \
    --config-patch @patches/interface-names.yaml \
    --config-patch @patches/metrics-server.yaml \
    --config-patch @patches/ntp.yaml \
    --config-patch-control-plane @patches/vip.yaml \
    --output rendered/

Generate node-specific configurations:

talosctl machineconfig patch \
    --patch @patches/control-plane-node-1.yaml \
    rendered/controlplane.yaml | yq - > nodes/control-plane-node-1.yaml

talosctl machineconfig patch \
    --patch @patches/control-plane-node-2.yaml \
    rendered/controlplane.yaml | yq - > nodes/control-plane-node-2.yaml

talosctl machineconfig patch \
    --patch @patches/control-plane-node-3.yaml \
    rendered/controlplane.yaml | yq - > nodes/control-plane-node-3.yaml

Applying config to the Nodes

Apply configuration to each node:

talosctl apply -f nodes/control-plane-node-1.yaml --node x.x.x.101 --insecure
talosctl apply -f nodes/control-plane-node-2.yaml --node x.x.x.102 --insecure
talosctl apply -f nodes/control-plane-node-3.yaml --node x.x.x.103 --insecure

Bootstrapping the Cluster

Specify cluster endpoints:

talosctl config endpoint x.x.x.101 x.x.x.102 x.x.x.103

Bootstrap the cluster:

talosctl bootstrap --node x.x.x.x

Monitor progress:

talosctl dashboard --node x.x.x.x

Connecting to the Cluster

Set up talosconfig:

export TALOSCONFIG=./rendered/talosconfig

Generate kubeconfig:

talosctl kubeconfig --node x.x.x.x

Validate the connection:

kubectl get nodes

You should see nodes in Not Ready state until CNI is installed.

Adding CNI

Install Cilium for networking:

helm repo add cilium https://helm.cilium.io/
helm repo update

helm upgrade --install cilium cilium/cilium \
    --namespace kube-system \
    --set ipam.mode=kubernetes \
    --set hostFirewall.enabled=true \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=true \
    --set kubeProxyReplacement=true

Cilium provides transparent networking, security policies, and network observability through Hubble.

Note: This post has been shortened due to character limitations. For complete Cilium configuration options and network policies, refer to the full documentation.

Verify Cilium pods:

kubectl get pods -n kube-system

After installation, check node status:

kubectl get nodes

Nodes should now be in Ready state.

Conclusion

This concludes the Kubernetes cluster setup. We've successfully bootstrapped the cluster and installed Cilium as the CNI. The base setup is complete.

While the cluster is now up and running, additional components are needed to expose services outside the cluster. In the next chapter, we'll configure the Ingress Controller for external service access.