Jenkins CICD Docker Pipeline Using Ephemeral Slaves and HTTPS

23. January 2017 Docker 0

Introduction

Note: This article only touches the surface of a Jenkins CICD pipeline. For much more information on setting up a private solution from scratch, see this article.

Having great tools to support a task only makes things easier. I’ve written about using GitLab CE with Docker in the past and have come to really embrace Docker for all of my development tasks. Keeping true to the adoption of best practices, I felt it was time to start using a CICD pipeline due to its many benefits. For example, my unit tests automatically run when code is deployed to my source control repository. In addition, my latest build is automatically deployed to my integration environment. These are only a few benefits and the list goes on for a mile.

There are a few build tools or task runners available on the market. One that always pops up is Jenkins. It is very powerful, fully featured, extendable, and best of all, it’s open source. It has a strong community backing and is in active development. It integrates well with tools such as GitHub and GitLab and as you’ll come to find out, plays nice with Docker.

Using Jenkins with Docker

The folks behind Jenkins have made it easy to get started using it with Docker. The image they provide is excellent and serves as a great starting point for most people. This will allow Jenkins to run in a Docker environment but isn’t configured to work with build slaves. Having dedicated machines to use in the build process is powerful. Multiple machines can take part in the build and automation process and be configured differently based on the needs of a project.

Something else that is quite important is setting everything up securely using HTTPS. After all, who wants to pass along important build information in plain text? Looking up how to setup Jenkins with HTTPS returned various results and setting it up to meet my needs was challenging at first. The most difficult part of the equation was setting up ephemeral build slaves.

Ephemeral Build Slaves

Maxfield Stewart wrote an article on the subject of ephemeral build slaves using Docker. It is extremely informative and inspired me to create my own solution. This is because he left out details on securing everything with HTTPS. Granted, I don’t blame him for leaving this out as it would have only added to the complexity of his article. The good news is I have filled in the gaps and identified a working solution using HTTPS along with the proper setup with NGINX as a reverse proxy. I also had to figure out how to get JNLP (the protocol used for the build slaves) traffic forwarded through NGINX as well. Therefore, all the heavy lifting has been done for you and I hope this solution helps to meet your needs and save you time.

In contrast to ephemeral build slaves, normal build slaves in Jenkins are static. In other words, they are allocated like a VM that is constantly running, taking up precious compute resources. However, with the magic of Docker these build slaves can be dynamic; they can be provisioned and removed when needed. Specific build slaves can be created for a particular purpose and using Yet Another Docker Plugin, configured to work for certain jobs.

Referencing the README.md

I spent some time creating the README.md which is included with the GitHub repository. Jump to that file for complete details and the latest updates.

Conclusion

Getting Jenkins working with HTTPS using a self-signed certificate and ephemeral build salves wasn’t a straight forward process. Certainly there are a few examples that exist in the wild but I could never find anything that worked perfectly for me. This is why I created my own and shared it with the world. My hope is you find this useful for your current needs. Feel free to ask any questions or submit pull requests to improve what I’ve shared and good luck with your future endeavors.

Daniel Eagle

Currently residing in the Austin area, Daniel Eagle is an avid gamer, writer, technology/science enthusiast, software developer, and educator. He takes great pride in spreading knowledge and helping others.

More Posts - Website

Follow Me:
TwitterLinkedInYouTube