Jenkins DevKit: Automating Jenkins

Part I: Automating the Jenkins Server

4 min readFeb 25, 2020

Jenkins is a popular build-task-scheduling tool used to develop continuous integration pipelines.

One thing Jenkins is not widely known for is automation and configuration of Jenkins itself. Instead, you must navigate through a maze of buttons and tabs to configure Jenkins.

This series can help remedy this problem and help you create disposable environments that are useful in testing Jenkins pipelines and other automation.

The goal of this series is to cover automation for the following for a local development:

Creating disposable Jenkins Dev environment
Configuration of Jenkins system
Installation of Plugins
Creating Jobs and Folder organization
Creating Pipelines
Creating Triggers for Job or Pipelines
Creating Credentials and Using Secrets

Required Tools

You need the following environments:

Docker: either Docker CE and Docker Compose for Linux; or Docker Desktop for macOS.
Bash shell (optional): instructions in this guide are oriented toward using Bash.

NOTE: This guide for local dev environment will use Unix sockets. As Unix sockets are not supported in Windows, this guide will not work for Windows operating system.

Creating Jenkins DevKit

In this setup, I show how to get started with crafting our own local Jenkins DevKit using the Docker image jenkins/jenkins.

We want to create a solution that can build and push docker images. This requires that the docker command line is available from the running container along with a docker daemon for this purpose.

Step 1: Create Project Structure

mkdir -p ~/projects/jenkins-devkit
cd ~/projects/jenkins-devkit
touch Dockerfile docker-compose.yaml jenkins.yaml

Step 2: Docker Compose script

Add the following content to docker-compose.yaml:

docker-compose.yaml

This script instructs Docker Compose to build from a local Dockerfile.

Step 2: Docker Build Script

Add the following content our initial Dockerfile:

Dockerfile

This initial Dockerfile will install docker-cli and required packages in a single layer as root user.

Step 3: Test Jenkins DevKit

We can bring up our environment by the following commands:

docker-compose up

In another terminal window or tab, we can test that docker does work from the container:

docker exec -ti jenkins-devkit bash
docker ps

If you can see the container in the list, when voilà, the solution is working, now we can add further automation.

Automating Jenkins DevKit

In this part, we’ll add some automation so any new Jenkins will be configured consistently.

Step 4: Automating Plugin Installation

The Jenkins docker image (jenkins/jenkins) comes with for automation installing plugins at start. All we need to do is supply a list of plugins (plugins.txt) to install-plugins.sh script.

First let’s copy down a list plug-ins for our environment:

curl -s https://gist.githubusercontent.com/darkn3rd/5da788f00466e2d5d1b3d619710fc647/raw/ > plugins.txt

Now that we have a list of plug-ins, we need to add support for installing the plugins. Append to our Dockerfile by typing this:

cat <<-DOCKERFILE >> Dockerfile
                      
####### INSTALL PLUGINS
COPY plugins.txt /usr/share/jenkins/ref/plugins.txt                       RUN /usr/local/bin/install-plugins.sh \
 < /usr/share/jenkins/ref/plugins.txtDOCKERFILE

Step 5: Automate Bypassing the Setup Wizard

Despite having plugins preinstalled with a new Jenkins server, we will be greeted by a Setup Wizard to install plugins and configure Jenkins. We can bypass this altogether.

Append to our Dockerfile by typing this:

cat <<-DOCKERFILE >> Dockerfile####### DISABLE SETUP WIZARD                       
RUN echo $JENKINS_VERSION > \
 /usr/share/jenkin/ref/jenkins.install.UpgradeWizard.state; \
 echo $JENKINS_VERSION > \
 /usr/share/jenkins/ref/jenkins.install.InstallUtil.lastExecVersionDOCKERFILE

Step 6: Automating Jenkins Configuration

We still need to configure Jenkins with some basic security, such as user accounts. This can be done through a plugin called Jenkins CaSC (or Configuration as Code). This provides a small DSL in YAML to describe the configuration Jenkins and installed Jenkins plug-ins.

Edit the jenkins.yaml and add the following:

jenkins.yaml

NOTE: We are purposefully not keeping the secrets secret for simplicity. This is fine for local development only, and on a real production Jenkins, these secrets would be referenced here, but stored secretly elsewhere.

With our configuration, we need add support to use this configuration. Append the following to the Dockerfile by typing this:

cat <<-DOCKERFILE >> Dockerfile####### CONFIGURE JENKINS AND INSTALLED PLUGINS
ENV CASC_JENKINS_CONFIG /var/jenkins_home/casc_configs
RUN mkdir -p /var/jenkins_home/casc_configs
COPY jenkins.yaml /var/jenkins_home/casc_configs/jenkins.yamlDOCKERFILE

The final Dockerfile should look something like this:

Dockerfile

Step 7: Testing the Final Solution

docker-compose stop
docker-compose build
docker-compose up -d

Now point your browser to http://localhost:8080.

Testing A Pipeline Manually

We can create a pipeline manually to test the solution. In a followup article, we’ll do this part automatically with automation. Below are two articles that I wrote on how to setup simple build→test pipeline.

In the articles, you can ignore the Running a Jenkins server Locally Section, as we’ll be using the current server we just created.

Flask (Python) Example

I wrote a previous article on how to create a small hello world web api service, using Flask web microframework:

Jenkins CI Pipeline with Python

Building a Jenkins CI Pipeline with Flask + unittest

medium.com

Sintra (Ruby) Example

I wrote a previous article that is Ruby oriented to create a small hello world web api service using Sinatra web micro-framework:

Jenkins CI Pipeline with Ruby

Building a Jenkins CI Pipeline with Sinatra + Test::Unit