Skip to main content

DevOps : Integration of Git, GitHub, Jenkins, Docker and Linux

  If you have knowledge of a subject, it is always great but if know how and where you can utilize it became your talent.
Likewise if you have knowledge of some technologies and also know how to use them. In other words I would like to say, you know how to integrate these technologies? Just by doing this you can create your own product, you can become a creator. I have lots of examples of it, but one of them is OpenStack (Collection of lots of services like Glance, shift, cinder, nova, neutron, Gnocchi etc).

  I am also writing this post to show you an interesting demo. In which I'll integrate multiple tools like Git, GitHub, Jenkins, Docker, Linux etc. and I will try to solve a use case. We have a web app developed by multiple developers will be first of all deploy in test environment, Then it will be checked by UAT team and if everything is fine and approved all the requirement from QAT team then automatically deployed in production environment.
So lets start with the activity -
Step 1 :- Create Git repository
$ cd /d/My/Msi\ Laptop/Training/DevOpsAL/Practice/
$ mkdir integration
$ cd integration
$ pwd
/d/My/Msi Laptop/Training/DevOpsAL/Practice/integration
$ touch index.html
-----I am not a web developer hence creating just a simple web page
$ cat >> index.html
This is my home page of my web aap
$ ls
index.html
$ git init
Initialized empty Git repository in D:/My/Msi Laptop/Training/DevOpsAL/Practice/integration/.git/
$ ls -la
total 5
drwxr-xr-x 1 Priyanshi 197121  0 May  7 15:58 ./
drwxr-xr-x 1 Priyanshi 197121  0 May  7 15:55 ../
drwxr-xr-x 1 Priyanshi 197121  0 May  7 15:58 .git/
-rw-r--r-- 1 Priyanshi 197121 28 May  7 15:57 index.html
$ git add .
$ cat >> .git/hooks/post-commit
#!/bin/bash
echo "---------------------------------------------------------------------"
echo "Welcome to hooks! Your latest changes are being pushed on GitHub"
echo "---------------------------------------------------------------------"
git push
$ ll  .git/hooks/post-commit
-rwxr-xr-x 1 Priyanshi 197121 253 May  7 16:01 .git/hooks/post-commit*
$ git commit . -m "First commit"
Step 2 :- Create GitHub repository

$ git remote add rk_web_app https://github.com/jay2tinku/git-github-jenkins-docker-integration.git
$ git push -u rk_web_app  master
cat >> index.html
Adding few more lines of code.
$ git commit . -m "2nd commit"
Till here there was just a developer was developing this web page, Now lets suppose more more developer joined the team. He will also start developing web app. So master developer created a new branch for new developer(Dev1). Below command will create a new branch dev1 and also checkout to it.
$ git checkout -b dev1
Switched to a new branch 'dev1'
$ git log --oneline
543176d (HEAD -> dev1, rk_web_app/master, master) 2nd commit
2d6e34d My first page commit
$ cat >> index.html
Dev1 is adding few lines of code, line3
$ git commit . -m "Dev1 2nd change in index.com"                                
---------------------------------------------------------------------
Welcome to hooks! Your latest changed are being pushed on GitHub
---------------------------------------------------------------------
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 315 bytes | 315.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To https://github.com/jay2tinku/git-github-jenkins-docker-integration.git
   b381ab8..2fc2c46  dev1 -> dev1
[dev1 2fc2c46] Dev1 2nd change in index.com
 1 file changed, 1 insertion(+)
$ git log --oneline
2fc2c46 (HEAD -> dev1, rk_web_app/dev1) Dev1 2nd change in index.com
b381ab8 Dev1 1st change in index.com
543176d (rk_web_app/master, master) 2nd commit
2d6e34d My first page commit
$ cat >> index.html
Dev1 3rd changes in index.html
$ git commit . -m "Dev1 3rdd change in index.com"
---------------------------------------------------------------------
Welcome to hooks! Your latest changed are being pushed on GitHub
---------------------------------------------------------------------
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 321 bytes | 321.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (1/1), completed with 1 local object.
To https://github.com/jay2tinku/git-github-jenkins-docker-integration.git
   2fc2c46..8a246e2  dev1 -> dev1
[dev1 8a246e2] Dev1 3rdd change in index.com
 1 file changed, 1 insertion(+)
$ git log --oneline
8a246e2 (HEAD -> dev1, rk_web_app/dev1) Dev1 3rdd change in index.com
2fc2c46 Dev1 2nd change in index.com
b381ab8 Dev1 1st change in index.com
543176d (rk_web_app/master, master) 2nd commit
2d6e34d My first page commit
$ git checkout master
Switched to branch 'master'
Your branch is up to date with 'rk_web_app/master'.
$ cat >> index.html
Few changes from master developer
$ git commit . -m "Main dev. 2nd change in index.com"
---------------------------------------------------------------------
Welcome to hooks! Your latest changes are being pushed on GitHub
---------------------------------------------------------------------
fatal: TaskCanceledException encountered.
   A task was canceled.
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 4 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 351 bytes | 175.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/jay2tinku/git-github-jenkins-docker-integration.git
   543176d..3030511  master -> master
[master 3030511] Main dev. 2nd change in index.com
 1 file changed, 1 insertion(+)
$ git checkout dev1
Switched to branch 'dev1'
Your branch is up to date with 'rk_web_app/dev1'.
Step 3 :- Create job1 in Jenkins. Jinkins will keep on checking GitHub repository and as soon as it find that Dev1 committed changes on GitHub. Jinkins will build(execute) job1 and job1 will deploy these changes on a httpd container. Interesting part is that, if container is already running it will just deploy else it will launch a container and then deploy. 
Create a job - Click on Jenkins >> New Item and configure as shown in below pictures.
Note :- We have many Build Triggers options but to make this demo easy to understand I have chosen(Poll SCM) one of simplest option. Which you can use but is is not recommended in production environment, because it will increase load on Jenkins server.




 Step 3 :- Create job2 in Jenkins. Same way as we did above, click on new New Item and put data as below-
General > Description -
************PRD_Deploy***********
Master Dev changes will be applied to PRD environment.
Source Code Management > Click on 'Git'
Repository URL - https://github.com/jay2tinku/git-github-jenkins-docker-integration.git
Branch Specifier (blank for 'any') - */master
Build Triggers > Select 'SCM'
Schedule - * * * * *
Build > Execute Shell -
sudo cp -rvf * /prd_config
if sudo docker ps |grep prd_server.examaple.com
then
echo "Already running!"
else
sudo docker run -dit -p 8085:80 --name prd_server.examaple.com -v /prd_config:/usr/local/apache2/htdocs/ httpd

fi
Step 4 :- Create a Job for QAT team, this department is also not known for me so creating a simple validation and approval process.
General > Description -
QAT Time confirmation
Source Code Management > Click on 'None'
Build Triggers > Select 'Build after other projects are built'
Projects to watch - job2,
Build > Execute Shell -
if sudo curl http://192.168.1.8:8084/
then
echo "This code is perfect and can be deployed on PRD server"
else
echo "Check config again!"

fi
Step 5 :- Create a job3 for final changes and commit.
General > Description -
************Production deployment after all changes***************
Source Code Management > Click on 'Git'
Repository URL - https://github.com/jay2tinku/git-github-jenkins-docker-integration.git
Branch Specifier (blank for 'any') - */dev1
Additional Behaviours
Name of repository - rk_web_app
Branch to merge to - master
Merge strategy - default
Fast-forward mode - --ff
Build Triggers > Select 'Build after other projects are built'
Projects to watch - QAT,

Result :-
Dev1 deployment on test_server status
[root@server109 ~]# curl http://192.168.1.4:8084/
My web app home page, line1
I am adding one more line, line2
Dev1 is adding few lines of code, line3
Dev1 2nd change in index.com
Dev1 3rd changes in index.html
Dev1 4th changes in index.html
Dev1 5th changes in index.html
Dev1 6th changes in index.html
Dev1 7th changes in index.html

 Final deployment on prd_server status
[root@server109 ~]# curl http://192.168.1.4:8085/
My web app home page, line1
I am adding one more line, line2
Main Developer changes in master branch
Main Developer changes in master branch_after tst env
Dev1 is adding few lines of code, line3
Dev1 2nd change in index.com
Dev1 3rd changes in index.html
Dev1 4th changes in index.html
Labels:

Comments

Post a Comment

Please share your experience.....

Popular posts from this blog

error: db5 error(11) from dbenv->open: Resource temporarily unavailable

If rpm command is not working in your system and it is giving an error message( error: db5 error(11) from dbenv->open: Resource temporarily unavailable ). What is the root cause of this issue? How to fix this issue?   just a single command- [root@localhost rpm]# rpm --rebuilddb Detailed error message- [root@localhost rpm]# rpm -q firefox ^Cerror: db5 error(11) from dbenv->open: Resource temporarily unavailable error: cannot open Packages index using db5 - Resource temporarily unavailable (11) error: cannot open Packages database in /var/lib/rpm ^Cerror: db5 error(11) from dbenv->open: Resource temporarily unavailable error: cannot open Packages database in /var/lib/rpm package firefox is not installed [root@localhost rpm]# RPM manage a database in which it store all information related to packages installed in our system. /var/lib/rpm, this is directory where this information is available. [root@localhost rpm]# cd /var/lib/rpm ...
2 comments
Read more

Failed to get D-Bus connection: Operation not permitted

" Failed to get D-Bus connection: Operation not permitted " - systemctl command is not working in Docker container. If systemctl command is not working in your container and giving subjected error message then simple solution of this error is, create container with -- privileged option and also provide init file full path  /usr/sbin/init [root@server109 ~]# docker container run -dit --privileged --name systemctl_not_working_centos1 centos:7 /usr/sbin/init For detailed explanation and understanding I am writing more about it, please have look below. If we have a daemon based program(httpd, sshd, jenkins, docker etc.) running inside a container and we would like to start/stop or check status of daemon inside docker then it becomes difficult for us to perform such operations , because by default systemctl and service  commands don't work inside docker. Normally we run below commands to check services status in Linux systems. [root@server109 ~]# systemctl status ...
4 comments
Read more

AWS cloud automation using Terraform

In this post I'll create multiple resources in AWS cloud using Terraform . Terraform is an infrastructure as code( IAC ) software which can do lots of things but it is superb in cloud automation. To use Terraform we have write code in a high-level configuration language known as Hashicorp Configuration Language , optionally we can write code in JSON as well. I'll create below service using Terraform- 1. Create the key-pair and security group which allow inbound traffic on port 80 and 22 2. Launch EC2 instance. 3. To create EC2 instance use same key and security group which created in step 1 4. Launch Volume(EBS) and mount this volume into /var/www/html directory 5. Upload index.php file and an image on GitHub repository 6. Clone GitHub repository into /var/www/html 7. Create S3 bucket, copy images from GitHub repo into it and set permission to public readable 8 Create a CloudFront use S3 bucket(which contains images) and use the CloudFront URL to update code in /var/w...
Post a Comment
Read more