CI/CD Automation on Jenkins Dynamic Cluster with Kubernetes

7 min readAug 29, 2020

let me first tell you about jenkins clusters and how to setup it.

What is a Jenkins Cluster ?

Basically a jenkins cluster is a master-slave system in which jenkins server runs in the master or the node(OS) in which jenkins server is running is known as master. By default whatever the jobs we build, jenkins build/run them in the master or the node in which it is running. But in real world we have large number of jenkins jobs , so it is impossible to run all the jobs in the master as master node has limited resources like RAM , CPU , Storage etc. So we need some more other nodes in which we can divide and run the jobs and these nodes are known as slave nodes. And jenkins master control these slave nodes and decides in which slave have to run which job. And this complete master-slave system is known as a Cluster. Jenkins master-slave architecture looks like this.

In jenkins there are 2 types of clusters.

Static cluster :- In this cluster , we can use any virtual machine , any base os like windows, mac or any docker conatiner as the slave nodes. But here we have to manually setup/connect all the VM’s or containers with the jenkins master. And in this these slave nodes runs continuously whether the job is building or not and this is a a heavy wastage of power in terms of RAM ,CPU.

Dynamics cluster :- Here we use slave nodes as docker containers and in this we have to connect the jenkins master with the docker server/engine for one time only. So when a job starts building then jenkins starts a docker container and runs the job in it and when the job builds completely then it automatically terminates the container.

So, in real technological world we mainly uses Jenkins Dynamic Cluster as it has many advantages over the static one and the main thing is jenkins automatically starts the slave nodes and terminates them when the job is finished and by this we can avoid resource wastage. So in this article , i will first show you that how to setup Dynamic Cluster.

Setup Jenkins Dynamic Cluster

In dynamic cluster , since we use docker containers as slave nodes so we need to have a Docker engine/server ready in our system. In my case, i have setup the docker in my RHEL 8 VM.

And since by default docker server is accessible within the VM only , so we have to make it to be accessible outside the VM or by anyone. So for this we have to go to this file /usr/lib/systemd/system/docker.service and then write the highlighted text as shown in the image below.

And after adding this line , we need to again restart our docker services for changes to take effects and for this we have to use these commands.

systemctl daemon-reload
systemctl restart docker

In this we are using slave nodes as docker containers and to create and run docker containers we need a docker image. So for this we can use some pre-created docker images or we can create our own. Here i am creating my own docker image by Dockerfile concept and the code that i write in my dockerfile is

In this docker image i have installed SSH server because jenkins need the shell of the slave node to run the job and so in the slave node we need to start the SSH services so that jenkins can use it. And i also installed kubectl command because in the jobs we used it. By these commands we build this Dockerfile to create docker image and push it to docker hub.

docker build -t atul0036/jenkube-slave:1.0 .
docker push atul0036/jenkube-slave:1.0

Now, we just have to configure the docker server in our jenkins so that jenkins can use it to create slave nodes. Here i am running my jenkins sever on top of kubernetes which i have already shown in my previous task and this is the interesting thing in this article as i running my jenkins server i.e., master in kubernetes and building the jobs in slave nodes i.e., docker containers running in some other VM.

First in jenkins we have to download a plugin Docker by which we can connect our docker engine with jenkins.

Now go to Jenkins > Manage Jenkins > Manage Nodes and Clouds > Configure Clouds and select Docker. And then fill the details as shown in the below images. In Docker HOST URL we have to write the ip address of the VM where docker engine is running.

Now, our Jenkins Dynamic Cluster is ready to use and now if we run a job in jenkins then it will first contact to the docker engine then docker will create a docker container as the slave node and then jenkins will run the job in that slave node.

Now we will create the jobs that we need for this project , So let’s get started..

Basically, in this project i will create a Build Pipeline of 3 Jobs which is a CI/CD automation so when a developer pushes the webpage to the github then this pipeline will deploy our application on production so that client can access our site and here i will deploy the app on kubernetes.

#JOB 1

This job will download the git repository pushed by the developer on GitHub and this repository contains webpage , YAML file to setup the webserver environment on kubernetes and a Dockerfile. This Dockerfile is very important as we create a docker image by this Dockerfile which contains the latest webpage and then we will use this docker image to update our webserver running on kubernetes. And this job will build the dockerfile and push the image to docker hub.

Here in this image you can see that i make this image to build in a slave node i.e., docker container and when the job builds completely it automatically terminates the container.

#JOB 2

This is the main job of the whole pipeline as this job will first analyse the type of webpage that developer pushes like html, php and then launches the respective type of webserver on kubernetes and before launching the webserver it checks whether the webserver environment is already setup or not and if not then setup it again and it uses the YAML file that is pushed by the developer and this file will creates a Deployment, a PVC to make the webpages persistent and a Service to expose the deployment on top of kubernetes for our webserver.

Then this job will rollout the updates for the website and for rolling out deployment in kubernetes will pull the docker image that we created in the Job 1 and then do rollouts. Since this job will also runs in new docker container, so it also needs the git repository that’s why we have to again pull the repository. This job will triggered by Job 1.

#JOB 3

This job will tests whether the webpage is working or not and if it not works then it will send the email to the developer and developer will do the changes in the webapage code and again push it to github. And it will also send the mail if due to some reasons the job failed. This job is triggered by Job 2.