Skip to content

Kubespray

Pre-requisite

We will need 1 control-plane(master) and 1 worker node to create a single control-plane kubernetes cluster using Kubespray. We are using following setting for this purpose:

  • 1 Linux machine for Ansible master, ubuntu-22.04-x86_64 or your choice of Ubuntu OS image, cpu-su.2 flavor with 2vCPU, 8GB RAM, 20GB storage.

  • 1 Linux machine for master, ubuntu-22.04-x86_64 or your choice of Ubuntu OS image, cpu-su.2 flavor with 2vCPU, 8GB RAM, 20GB storage - also assign Floating IP to the master node.

  • 1 Linux machines for worker, ubuntu-22.04-x86_64 or your choice of Ubuntu OS image, cpu-su.1 flavor with 1vCPU, 4GB RAM, 20GB storage.

  • ssh access to all machines: Read more here on how to set up SSH on your remote VMs.

  • To allow SSH from Ansible master to all other nodes: Read more here Generate SSH key for Ansible master node using:

    ssh-keygen -t rsa

Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa
Your public key has been saved in /root/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:OMsKP7EmhT400AJA/KN1smKt6eTaa3QFQUiepmj8dxroot@ansible-master
The key's randomart image is:
+---[RSA 3072]----+
|=o.oo.           |
|.o...            |
|..=  .           |
|=o.= ...         |
|o=+.=.o SE       |
|.+*o+. o. .      |
|.=== +o. .       |
|o+=o=..          |
|++o=o.           |
+----[SHA256]-----+

    Copy and append the contents of SSH public key i.e. ~/.ssh/id_rsa.pub to other nodes's ~/.ssh/authorized_keys file. Please make sure you are logged in as root user by doing sudo su before you copy this public key to the end of ~/.ssh/authorized_keys file of the other master and worker nodes. This will allow ssh <other_nodes_internal_ip> from the Ansible master node's terminal.

  • Create 2 security groups with appropriate ports and protocols:

    i. To be used by the master nodes: Control plane ports and protocols

    ii. To be used by the worker nodes: Worker node ports and protocols

  • setup Unique hostname to each machine using the following command:

    echo "<node_internal_IP> <host_name>" >> /etc/hosts
hostnamectl set-hostname <host_name>

    For example:

    echo "192.168.0.224 ansible_master" >> /etc/hosts
hostnamectl set-hostname ansible_master

In this step, you will update packages and disable swap on the all 3 nodes:

  • 1 Ansible Master Node - ansible_master

  • 1 Kubernetes Master Node - kubspray_master

  • 1 Kubernetes Worker Node - kubspray_worker1

The below steps will be performed on all the above mentioned nodes:

  • SSH into all the 3 machines

  • Switch as root: sudo su

  • Update the repositories and packages:

    apt-get update && apt-get upgrade -y

  • Turn off swap

    swapoff -a
sed -i '/ swap / s/^/#/' /etc/fstab

Configure Kubespray on ansible_master node using Ansible Playbook

Run the below command on the master node i.e. master that you want to setup as control plane.

  • SSH into ansible_master machine

  • Switch to root user: sudo su

  • Execute the below command to initialize the cluster:

  • Install Python3 and upgrade pip to pip3:

    apt install python3-pip -y
pip3 install --upgrade pip
python3 -V && pip3 -V
pip -V

  • Clone the Kubespray git repository:

    git clone https://github.com/kubernetes-sigs/kubespray.git
cd kubespray

  • Install dependencies from requirements.txt:

    pip install -r requirements.txt

  • Copy inventory/sample as inventory/mycluster

    cp -rfp inventory/sample inventory/mycluster

  • Update Ansible inventory file with inventory builder:

    This step is little trivial because we need to update hosts.yml with the nodes IP.

    Now we are going to declare a variable "IPS" for storing the IP address of other K8s nodes .i.e. kubspray_master(192.168.0.130), kubspray_worker1(192.168.0.32)

    declare -a IPS=(192.168.0.130 192.168.0.32)
CONFIG_FILE=inventory/mycluster/hosts.yml python3 \
    contrib/inventory_builder/inventory.py ${IPS[@]}

    This outputs:

    DEBUG: Adding group all
DEBUG: Adding group kube_control_plane
DEBUG: Adding group kube_node
DEBUG: Adding group etcd
DEBUG: Adding group k8s_cluster
DEBUG: Adding group calico_rr
DEBUG: adding host node1 to group all
DEBUG: adding host node2 to group all
DEBUG: adding host node1 to group etcd
DEBUG: adding host node1 to group kube_control_plane
DEBUG: adding host node2 to group kube_control_plane
DEBUG: adding host node1 to group kube_node
DEBUG: adding host node2 to group kube_node

  • After running the above commands do verify the hosts.yml and its content:

    cat inventory/mycluster/hosts.yml

    The contents of the hosts.yml file should looks like:

    all:
    hosts:
        node1:
            ansible_host: 192.168.0.130
            ip: 192.168.0.130
            access_ip: 192.168.0.130
        node2:
            ansible_host: 192.168.0.32
            ip: 192.168.0.32
            access_ip: 192.168.0.32
    children:
        kube_control_plane:
            hosts:
                node1:
                node2:
        kube_node:
            hosts:
                node1:
                node2:
        etcd:
            hosts:
                node1:
        k8s_cluster:
            children:
                kube_control_plane:
                kube_node:
        calico_rr:
            hosts: {}

  • Review and change parameters under inventory/mycluster/group_vars

    cat inventory/mycluster/group_vars/all/all.yml
cat inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml

  • It can be useful to set the following two variables to true in inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml: kubeconfig_localhost (to make a copy of kubeconfig on the host that runs Ansible in { inventory_dir }/artifacts) and kubectl_localhost (to download kubectl onto the host that runs Ansible in { bin_dir }).

    Very Important

    As Ubuntu 20 kvm kernel doesn't have dummy module we need to modify the following two variables in inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml: enable_nodelocaldns: false and kube_proxy_mode: iptables which will Disable nodelocal dns cache and Kube-proxy proxyMode to iptables respectively.

  • Deploy Kubespray with Ansible Playbook - run the playbook as root user. The option --become is required, as for example writing SSL keys in /etc/, installing packages and interacting with various systemd daemons. Without --become the playbook will fail to run!

    ansible-playbook -i inventory/mycluster/hosts.yml --become --become-user=root cluster.yml

    Note

    Running ansible playbook takes little time because it depends on the network bandwidth also.

Install kubectl on Kubernetes master node .i.e. kubspray_master

  • Install kubectl binary

    snap install kubectl --classic

    This outputs: kubectl 1.26.1 from Canonical✓ installed

  • Now verify the kubectl version:

    kubectl version -o yaml

Validate all cluster components and nodes are visible on all nodes

  • Verify the cluster

    kubectl get nodes

NAME    STATUS   ROLES                  AGE     VERSION
node1   Ready    control-plane,master   6m7s    v1.26.1
node2   Ready    control-plane,master   5m32s   v1.26.1

Deploy A Hello Minikube Application

  • Use the kubectl create command to create a Deployment that manages a Pod. The Pod runs a Container based on the provided Docker image.

    kubectl create deployment hello-minikube --image=k8s.gcr.io/echoserver:1.4

    kubectl expose deployment hello-minikube --type=LoadBalancer --port=8080

service/hello-minikube exposed

  • View the deployments information:

    kubectl get deployments

NAME             READY   UP-TO-DATE   AVAILABLE   AGE
hello-minikube   1/1     1            1           50s

  • View the port information:

    kubectl get svc hello-minikube

NAME             TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
hello-minikube   LoadBalancer   10.233.35.126   <pending>     8080:30723/TCP   40s

  • Expose the service locally

    kubectl port-forward svc/hello-minikube 30723:8080

Forwarding from [::1]:30723 -> 8080
Forwarding from 127.0.0.1:30723 -> 8080
Handling connection for 30723
Handling connection for 30723

Go to browser, visit http://<Master-Floating-IP>:8080 i.e. http://140.247.152.235:8080/ to check the hello minikube default page.

Clean up

Now you can clean up the app resources you created in your cluster:

kubectl delete service hello-minikube
kubectl delete deployment hello-minikube