I was testing few features on Kubernetes for the same I was needed a new Kubernetes cluster to be setup in my lab environment for some POC purpose. Here I am installing a new Kubernetes cluster on ubuntu server. I am using k3d, also called Kubernetes on docker.
K3d is a easy and simple tool that lets you to run k3s (a slimmed-down version of Kubernetes from Rancher Labs) inside Docker containers. It streamlines the procedure of setting up both single-node and multi-node k3s clusters in Docker, making it perfect for local Kubernetes development. Although k3d is commonly used and maintained/supported by the community, it's significant to note that it is not an official product of Rancher (SUSE).
As I am installing K3d on Ubuntu Linux I will need to install docker and kubectl first as the prerequisites.
#Step 1: Docker Installation and Setup
Get and login to sudo su access. Update is to updates the package index to reflect the latest available packages and upgrade is upgrades all installed packages to their latest versions. Syntax -y is for without prompting for confirmation to install updates and upgrades.
sudo su -
apt-get update -y && apt-get upgrade -y
Install supporting packages using apt command.
sudo apt-get install ca-certificates curl -y
Next create the /etc/apt/keyrings directory with permissions set to 0755 (owner can read, write, and execute; group and others can read and execute), It is often used in scripts during the setup of repositories to securely store GPG keys for APT.
sudo install -m 0755 -d /etc/apt/keyrings
This command securely downloads the Docker GPG key and saves it to /etc/apt/keyrings/docker.asc, allowing you to verify the authenticity of Docker packages.
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
Following grants read permission to all users for the /etc/apt/keyrings/docker.asc file, allowing anyone to read the Docker GPG key. This is typically done to ensure that the package manager (APT) can access the key to verify the authenticity of Docker packages.
sudo chmod a+r /etc/apt/keyrings/docker.asc
By using below command adds the Docker repository to the system's package sources and updates the package index, allowing you to install Docker packages. Updates the package index to reflect the new repository.
echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update
Next I used apt to install the all docker related packages, they all are required..
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
The command sudo usermod -aG docker ${USER} is used to add a user to the docker group on a Linux system. This is commonly done to allow the user to run Docker commands without needing to use sudo (which requires administrative privileges).
sudo usermod -aG docker ${USER}
After complete docker setup is done, to verify docker is working and I am able to connect to the docker containerd engine, fire below command to show all the available containers. Since this just fresh installed hence no container is available to list.
docker ps
These commands are optional and create a wrapper script at /usr/bin/docker-compose that runs docker compose with compatibility mode enabled, allowing you to use the docker-compose command instead of docker compose.
sudo touch /usr/bin/docker-compose echo 'docker compose --compatibility "$@"' | sudo tee /usr/bin/docker-compose sudo chmod +x /usr/bin/docker-compose
#Step 2: K3d (Kubernetes on docker) Installation
Once docker is setup and configured, next command downloads and runs the installation script for k3d, a lightweight Kubernetes distribution.
wget -q -O - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
#Step 3: kubectl Installation and setup
Below command downloads the latest stable version of the Kubernetes command-line tool, kubectl, for Linux (amd64 architecture)
For more on different ways to install check official guide: https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#install-kubectl-binary-with-curl-on-linux
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
After installation of kubectl binary and below command sets its permissions, then verifies the installation by checking the version of the kubectl client.
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl kubectl version --client
#Step 4: K3d configuration
After installation of K3d, I am setting up first one node Kubernetes cluster with below command.
k3d cluster create mycluster
Once first Kubernetes cluster is set, use below command to verify it is listing a node in the Kubernetes cluster.
kubectl get node
Below is the output after running all above commands. Download this script Kubernetes on docker k3d installation setup and configuration on ubuntu linux.txt here or it also available on github.com.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 |
vjanvi@kubernetes01:~$ vjanvi@kubernetes01:~$ sudo su - [sudo] password for vjanvi: root@kubernetes01:~# root@kubernetes01:~# apt-get update -y && apt-get upgrade -y Hit:1 http://us.archive.ubuntu.com/ubuntu noble InRelease Hit:2 http://us.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:3 http://us.archive.ubuntu.com/ubuntu noble-backports InRelease Hit:4 http://security.ubuntu.com/ubuntu noble-security InRelease Reading package lists... Done Reading package lists... Done Building dependency tree... Done Reading state information... Done Calculating upgrade... Done The following upgrades have been deferred due to phasing: curl libcurl3t64-gnutls libcurl4t64 libicu74 liblz4-1 libpython3-stdlib python-apt-common python3 python3-apt python3-minimal 0 upgraded, 0 newly installed, 0 to remove and 10 not upgraded. root@kubernetes01:~# root@kubernetes01:~# sudo apt-get install ca-certificates curl Reading package lists... Done Building dependency tree... Done Reading state information... Done ca-certificates is already the newest version (20240203). ca-certificates set to manually installed. The following additional packages will be installed: libcurl3t64-gnutls libcurl4t64 The following packages will be upgraded: curl libcurl3t64-gnutls libcurl4t64 3 upgraded, 0 newly installed, 0 to remove and 7 not upgraded. Need to get 900 kB of archives. After this operation, 3,072 B of additional disk space will be used. Do you want to continue? [Y/n] Y Get:1 http://us.archive.ubuntu.com/ubuntu noble-updates/main amd64 curl amd64 8.5.0-2ubuntu10.3 [227 kB] Get:2 http://us.archive.ubuntu.com/ubuntu noble-updates/main amd64 libcurl4t64 amd64 8.5.0-2ubuntu10.3 [341 kB] Get:3 http://us.archive.ubuntu.com/ubuntu noble-updates/main amd64 libcurl3t64-gnutls amd64 8.5.0-2ubuntu10.3 [333 kB] Fetched 900 kB in 0s (2,115 kB/s) (Reading database ... 83391 files and directories currently installed.) Preparing to unpack .../curl_8.5.0-2ubuntu10.3_amd64.deb ... Unpacking curl (8.5.0-2ubuntu10.3) over (8.5.0-2ubuntu10.2) ... Preparing to unpack .../libcurl4t64_8.5.0-2ubuntu10.3_amd64.deb ... Unpacking libcurl4t64:amd64 (8.5.0-2ubuntu10.3) over (8.5.0-2ubuntu10.2) ... Preparing to unpack .../libcurl3t64-gnutls_8.5.0-2ubuntu10.3_amd64.deb ... Unpacking libcurl3t64-gnutls:amd64 (8.5.0-2ubuntu10.3) over (8.5.0-2ubuntu10.2) ... Setting up libcurl4t64:amd64 (8.5.0-2ubuntu10.3) ... Setting up libcurl3t64-gnutls:amd64 (8.5.0-2ubuntu10.3) ... Setting up curl (8.5.0-2ubuntu10.3) ... Processing triggers for man-db (2.12.0-4build2) ... Processing triggers for libc-bin (2.39-0ubuntu8.3) ... Scanning processes... Scanning candidates... Scanning linux images... Running kernel seems to be up-to-date. Restarting services... Service restarts being deferred: /etc/needrestart/restart.d/dbus.service systemctl restart systemd-logind.service systemctl restart unattended-upgrades.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. root@kubernetes01:~# root@kubernetes01:~# sudo install -m 0755 -d /etc/apt/keyrings root@kubernetes01:~# root@kubernetes01:~# sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc root@kubernetes01:~# root@kubernetes01:~# sudo chmod a+r /etc/apt/keyrings/docker.asc root@kubernetes01:~# root@kubernetes01:~# echo \ "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \ $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \ sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt-get update Hit:1 https://download.docker.com/linux/ubuntu noble InRelease Hit:2 http://us.archive.ubuntu.com/ubuntu noble InRelease Hit:3 http://us.archive.ubuntu.com/ubuntu noble-updates InRelease Hit:4 http://us.archive.ubuntu.com/ubuntu noble-backports InRelease Hit:5 http://security.ubuntu.com/ubuntu noble-security InRelease Reading package lists... Done root@kubernetes01:~# root@kubernetes01:~# root@kubernetes01:~# sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin Reading package lists... Done Building dependency tree... Done Reading state information... Done The following additional packages will be installed: docker-ce-rootless-extras libltdl7 libslirp0 pigz slirp4netns Suggested packages: aufs-tools cgroupfs-mount | cgroup-lite The following NEW packages will be installed: containerd.io docker-buildx-plugin docker-ce docker-ce-cli docker-ce-rootless-extras docker-compose-plugin libltdl7 libslirp0 pigz slirp4netns 0 upgraded, 10 newly installed, 0 to remove and 7 not upgraded. Need to get 122 MB of archives. After this operation, 437 MB of additional disk space will be used. Do you want to continue? [Y/n] Y Get:1 https://download.docker.com/linux/ubuntu noble/stable amd64 containerd.io amd64 1.7.20-1 [30.5 MB] Get:2 http://us.archive.ubuntu.com/ubuntu noble/universe amd64 pigz amd64 2.8-1 [65.6 kB] Get:3 http://us.archive.ubuntu.com/ubuntu noble/main amd64 libltdl7 amd64 2.4.7-7build1 [40.3 kB] Get:4 http://us.archive.ubuntu.com/ubuntu noble/main amd64 libslirp0 amd64 4.7.0-1ubuntu3 [63.8 kB] Get:5 http://us.archive.ubuntu.com/ubuntu noble/universe amd64 slirp4netns amd64 1.2.1-1build2 [34.9 kB] Get:6 https://download.docker.com/linux/ubuntu noble/stable amd64 docker-buildx-plugin amd64 0.16.2-1~ubuntu.24.04~noble [29.9 MB] Get:7 https://download.docker.com/linux/ubuntu noble/stable amd64 docker-ce-cli amd64 5:27.1.2-1~ubuntu.24.04~noble [14.6 MB] Get:8 https://download.docker.com/linux/ubuntu noble/stable amd64 docker-ce amd64 5:27.1.2-1~ubuntu.24.04~noble [25.2 MB] Get:9 https://download.docker.com/linux/ubuntu noble/stable amd64 docker-ce-rootless-extras amd64 5:27.1.2-1~ubuntu.24.04~noble [9,318 kB] Get:10 https://download.docker.com/linux/ubuntu noble/stable amd64 docker-compose-plugin amd64 2.29.1-1~ubuntu.24.04~noble [12.5 MB] Fetched 122 MB in 3s (39.3 MB/s) Selecting previously unselected package pigz. (Reading database ... 83391 files and directories currently installed.) Preparing to unpack .../0-pigz_2.8-1_amd64.deb ... Unpacking pigz (2.8-1) ... Selecting previously unselected package containerd.io. Preparing to unpack .../1-containerd.io_1.7.20-1_amd64.deb ... Unpacking containerd.io (1.7.20-1) ... Selecting previously unselected package docker-buildx-plugin. Preparing to unpack .../2-docker-buildx-plugin_0.16.2-1~ubuntu.24.04~noble_amd64.deb ... Unpacking docker-buildx-plugin (0.16.2-1~ubuntu.24.04~noble) ... Selecting previously unselected package docker-ce-cli. Preparing to unpack .../3-docker-ce-cli_5%3a27.1.2-1~ubuntu.24.04~noble_amd64.deb ... Unpacking docker-ce-cli (5:27.1.2-1~ubuntu.24.04~noble) ... Selecting previously unselected package docker-ce. Preparing to unpack .../4-docker-ce_5%3a27.1.2-1~ubuntu.24.04~noble_amd64.deb ... Unpacking docker-ce (5:27.1.2-1~ubuntu.24.04~noble) ... Selecting previously unselected package docker-ce-rootless-extras. Preparing to unpack .../5-docker-ce-rootless-extras_5%3a27.1.2-1~ubuntu.24.04~noble_amd64.deb ... Unpacking docker-ce-rootless-extras (5:27.1.2-1~ubuntu.24.04~noble) ... Selecting previously unselected package docker-compose-plugin. Preparing to unpack .../6-docker-compose-plugin_2.29.1-1~ubuntu.24.04~noble_amd64.deb ... Unpacking docker-compose-plugin (2.29.1-1~ubuntu.24.04~noble) ... Selecting previously unselected package libltdl7:amd64. Preparing to unpack .../7-libltdl7_2.4.7-7build1_amd64.deb ... Unpacking libltdl7:amd64 (2.4.7-7build1) ... Selecting previously unselected package libslirp0:amd64. Preparing to unpack .../8-libslirp0_4.7.0-1ubuntu3_amd64.deb ... Unpacking libslirp0:amd64 (4.7.0-1ubuntu3) ... Selecting previously unselected package slirp4netns. Preparing to unpack .../9-slirp4netns_1.2.1-1build2_amd64.deb ... Unpacking slirp4netns (1.2.1-1build2) ... Setting up docker-buildx-plugin (0.16.2-1~ubuntu.24.04~noble) ... Setting up containerd.io (1.7.20-1) ... Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /usr/lib/systemd/system/containerd.service. Setting up docker-compose-plugin (2.29.1-1~ubuntu.24.04~noble) ... Setting up libltdl7:amd64 (2.4.7-7build1) ... Setting up docker-ce-cli (5:27.1.2-1~ubuntu.24.04~noble) ... Setting up libslirp0:amd64 (4.7.0-1ubuntu3) ... Setting up pigz (2.8-1) ... Setting up docker-ce-rootless-extras (5:27.1.2-1~ubuntu.24.04~noble) ... Setting up slirp4netns (1.2.1-1build2) ... Setting up docker-ce (5:27.1.2-1~ubuntu.24.04~noble) ... Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /usr/lib/systemd/system/docker.service. Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /usr/lib/systemd/system/docker.socket. Processing triggers for man-db (2.12.0-4build2) ... Processing triggers for libc-bin (2.39-0ubuntu8.3) ... Scanning processes... Scanning candidates... Scanning linux images... Running kernel seems to be up-to-date. Restarting services... Service restarts being deferred: /etc/needrestart/restart.d/dbus.service systemctl restart systemd-logind.service systemctl restart unattended-upgrades.service No containers need to be restarted. No user sessions are running outdated binaries. No VM guests are running outdated hypervisor (qemu) binaries on this host. root@kubernetes01:~# root@kubernetes01:~# root@kubernetes01:~# sudo usermod -aG docker ${USER} root@kubernetes01:~# root@kubernetes01:~# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES root@kubernetes01:~# root@kubernetes01:~# root@kubernetes01:~# sudo touch /usr/bin/docker-compose echo 'docker compose --compatibility "$@"' | sudo tee /usr/bin/docker-compose sudo chmod +x /usr/bin/docker-compose docker compose --compatibility "$@" root@kubernetes01:~# root@kubernetes01:~# root@kubernetes01:~# root@kubernetes01:~# wget -q -O - https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash Preparing to install k3d into /usr/local/bin k3d installed into /usr/local/bin/k3d Run 'k3d --help' to see what you can do with it. root@kubernetes01:~# root@kubernetes01:~# root@kubernetes01:~# curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl" % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 138 100 138 0 0 1679 0 --:--:-- --:--:-- --:--:-- 1703 71 53.7M 71 38.6M 0 0 20.5M 0 0:00:02 0:00:01 0:00:01 22.1M 100 53.7M 100 53.7M 0 0 20.8M 0 0:00:02 0:00:02 --:--:-- 21.8M root@kubernetes01:~# root@kubernetes01:~# sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl root@kubernetes01:~# root@kubernetes01:~# kubectl version --client Client Version: v1.31.0 Kustomize Version: v5.4.2 root@kubernetes01:~# root@kubernetes01:~# k3d cluster create mycluster INFO[0000] Prep: Network INFO[0000] Created network 'k3d-mycluster' INFO[0000] Created image volume k3d-mycluster-images INFO[0000] Starting new tools node... INFO[0000] Pulling image 'ghcr.io/k3d-io/k3d-tools:5.7.3' INFO[0001] Creating node 'k3d-mycluster-server-0' INFO[0001] Pulling image 'docker.io/rancher/k3s:v1.30.3-k3s1' INFO[0006] Creating LoadBalancer 'k3d-mycluster-serverlb' INFO[0006] Starting node 'k3d-mycluster-tools' INFO[0007] Pulling image 'ghcr.io/k3d-io/k3d-proxy:5.7.3' INFO[0010] Using the k3d-tools node to gather environment information INFO[0010] HostIP: using network gateway 172.18.0.1 address INFO[0010] Starting cluster 'mycluster' INFO[0010] Starting servers... INFO[0010] Starting node 'k3d-mycluster-server-0' INFO[0015] All agents already running. INFO[0015] Starting helpers... INFO[0015] Starting node 'k3d-mycluster-serverlb' INFO[0022] Injecting records for hostAliases (incl. host.k3d.internal) and for 2 network members into CoreDNS configmap... INFO[0024] Cluster 'mycluster' created successfully! INFO[0024] You can now use it like this: kubectl cluster-info root@kubernetes01:~# root@kubernetes01:~# kubectl get node NAME STATUS ROLES AGE VERSION k3d-mycluster-server-0 Ready control-plane,master 5m44s v1.30.3+k3s1 root@kubernetes01:~# |
Useful Articles
How to install Ansible AWX on Ubuntu using Kubernetes K8S
Powershell Using vRealize Log Insight Rest API
Install kubectl.exe on windows using PowerShell
Using terraform to clone a virtual machine on VMware vSphere infrastructure
Terraform module clone VMware vSphere Linux and Windows virtual machine
Terraform VMware vSphere Virtual Machine customization clone failed on Windows
Terraform VMware vSphere Virtual Machine cloning Operating system not found
How to Install Minikube on Ubuntu - Step by Step
MINIKUBE Unable to start VM - This computer doesn't have VT-X AMD-v enabled
Install and Setup your own Kubernetes Cluster with K3s
Rancher k3s.yaml permission denied when using kubectl - Kubernetes
Configure Nginx Load Balancer for the Kubernetes API Server - Part 1
Install and configure Kubernetes cluster master nodes using kubeadm - Part 2
Install and configure Kubernetes cluster worker nodes using kubeadm - Part 3
Kubernetes kubeadm join couldn't validate the identity of the API server connection refused
Kubernetes kubeadm join could not find a jws signature in the cluster-info ConfigMap for token ID
Setup and deploy Ingress controller for Kubernetes on Bare Metal servers
Setup HAProxy for Ingress Controller Kubernetes Cluster
