How to deploy CI/CD Automation on top of Kubernetes

#DevOps Task 3

quick brief you about this project.

1. Create container image that’s has Jenkins installed using dockerfile.

2. When we launch this image, it should automatically start the Jenkins service in the container.

3. Create a job chain of job1, job2, job3 and job4 using build pipeline plugin in Jenkins

4. Job1: Pull the Github repo automatically when some developers push the repo to Github.

5. Job2 :

1. By looking at the code or program file, Jenkins should automatically start the respective language interpreter installed image container to deploy code on top of Kubernetes ( eg. If code is of PHP, then Jenkins should start the container that has PHP already installed )

2. Expose your pod so that testing team could perform the testing on the pod

3. Make the data to remain persistent ( If server collects some data like logs, other user information )

6. Job3: Test your app if it is working or not.

7. Job4: if an app is not working, then send email to the developer with error messages and redeploy the application after code is being edited by the developer.

Perform all the things at the top of Kubernetes with use of PVC, Deployment, Pods etc.

Now , first we have to install and setup the Kubernetes cluster in our system and in my case i have used Minikube to setup the kubernetes cluster and the installation of minikube is very easy as it setups the single-node cluster.

After this we have to setup the Jenkins server and previously for this we created a separate docker image using Dokckerfile concept. We can again use that same image but here by using jenkins we have to run our webserver on top of kubernetes and for this we need to install Kubernetes client program i.e., kubectl in the same base os in which our jenkins is running. So we need to modify the Dockerfile and again build it to create a new docker image. The code that i write in the Dockerfile is

Now , we just build this Dockerfile and then push it to docker hub by using these commands.

docker build -t atul0036/jenkins_kubectl:1.0 .
docker push atul0036/jenkins_kubectl:1.0

After this we will use this image to run jenkins on kubernetes as kubernetes by default uses docker behind the scenes to deploy the applications. Now in kubernetes we first create a PVC (Persistent Volume Claim) for our jenkins, so that whatever we do in jenkins like creating jobs, downloading plugins, credentials info. etc will not lost if by chance the pod in which jenkins is running gets failed or stopped. And for doing anything in kubernetes we can use command line or create some YAML file and here i am using the YAML file for creating PVC . The YAML file is

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-jenkins
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

Now we have to launch the jenkins server and for this i am creating a Deployment in kubernetes using the docker image that we created above and if due to some reason the pod in which our jenkins server is running gets failed then the deployment will again launch it and since we are using a PVC so that’s why we could not loose anything in our jenkins server. The YAML file for deployment is

apiVersion: apps/v1 
kind: Deployment
metadata:
  name: jen-deploy 
  labels:
    app: jenkins
spec:
  selector:
    matchLabels:
      app: jenkins
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
      - image: atul0036/jen:v1
        name: jenkins
        ports:
        - containerPort: 8080
          name: jenkins
        volumeMounts:
        - name: jenkins-pv
          mountPath: /root/.jenkins
      volumes:
      - name: jenkins-pv
        persistentVolumeClaim:
          claimName: pvc-jenkins

We use this command to create PVC and Deployment from the YAML files.

kubectl create -f (filename).yaml

We have to expose the deployment so that we can access the jenkins from the browser and for this we use this command

kubectl expose deployment jen-deploy --type=NodePort --port=8080

Now our jenkins is ready to use and after setting up the jenkins and installing required plugins we can create jobs and pipelines.

So , let’s create the jobs needed for this project ……

#JOB 1

This job will download the git repsitory pushed by the developer on the Github but this time along with the webpages this repository also contains a YAML file by which we can deploy/setup our Webserver on kubernetes using jenkins. And then copy everything in some local folder and in my case it is /task3. And in this job , i have used Poll SCM due to which when developer pushes something to github then it will automatically download it but you can also use any other Build Trigger.

#JOB 2

This job will first analyse the type of webpage that developer pushes like html, php and then launches the respective type of webserver on kubernetes. So this job basically deploys our webserver app on kubernetes .This job is triggered by Job 1 .

In this job the YAML file that i used for deploying webserver on kubernetes is

apiVersion: v1
kind: Service
metadata:
  name: webserver
  labels:
    app: webserver
spec:
  ports:
    - port: 80
      nodePort: 30001
  selector:
    type: html
  type: NodePort
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pvc-html
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: apps/v1 
kind: Deployment
metadata:
  name: httpd-deploy 
  labels:
    app: webserver
    type: html
spec:
  selector:
    matchLabels:
      type: html
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: webserver
        type: html
    spec:
      containers:
      - image: httpd
        name: webserver
        ports:
        - containerPort: 80
          name: webserver
        volumeMounts:
        - name: webserver-pv
          mountPath: /usr/local/apache2/htdocs/
      volumes:
      - name: webserver-pv
        persistentVolumeClaim:
          claimName: pvc-html

And this will also expose our webserver so we don’t need to use any other command to expose our webserver.

#JOB 3

This job will first copy the webpages to the pod in which our webserver is running and then tests whether the webpage is working or not and if it not works then it will send the email to the developer and developer will do the changes in the webapage code and again push it to github. And it will also send the mail if due to some reasons the job failed. This job is triggered by Job 2.

In this job for sending mail i am using Jenkins Default Email Notifier but you can also use any other way to send mails like by creating a python code for sending mails or any other way. My this job failed due to some reason , so the mail that jenkins send for failure is

So, the Build Pipeline output is….

And finally we can now access our webserver

In this project you can see , i have not created an extra job for monitoring our webserver like we created in the previous task. This is because previously we are running everything directly in docker and docker does not provide any service to manage the containers and if they fails then it also not launched them again. But in this we are running everything in kubernetes and since kubernetes is a Container Management Program so it will manage every pod/container for us and also we have used Deployment which will keep an eye on our pods and if anyone of them gets failed or stopped then it will launch it again.

#Github repository link — https://github.com/atuljha0036/devopstask3.git