This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

How to Write Jenkinsfiles

Comprehensive guide to writing Jenkins pipelines and Jenkinsfiles

This section provides comprehensive guides for writing Jenkinsfiles and working with Jenkins pipelines.

This is a complete guide covering Jenkins architecture, pipeline syntax, shared libraries, best practices, and annotated examples.

1. What You’ll Learn

  • How Jenkins works and its architecture
  • Declarative and scripted pipeline syntax
  • Creating and using Jenkins shared libraries
  • Jenkins best practices and configuration
  • Real-world Jenkinsfile examples with detailed annotations
  • Common recipes and troubleshooting tips

Articles in this section

TitleDescriptionUpdated
Annotated Jenkinsfiles - Part 1Detailed Jenkinsfile examples with annotations2026-02-22
How Jenkins WorksUnderstanding Jenkins architecture and concepts2026-02-17
Jenkins PipelinesDeclarative and scripted pipeline syntax2026-02-17
Jenkins LibraryCreating and using Jenkins shared libraries2026-02-17
Jenkins Best PracticesBest practices and patterns for Jenkins and Jenkinsfiles2026-02-17
Annotated Jenkinsfiles - Part 2More annotated Jenkinsfile examples2026-02-17
Annotated Jenkinsfiles - Part 3Additional Jenkinsfile pattern examples2026-02-17
Annotated Jenkinsfiles - Part 4Complex Jenkinsfile scenarios2026-02-17
Annotated Jenkinsfiles - Part 5Detailed Jenkinsfile examples with annotations2026-02-17
Jenkins Recipes and TipsUseful recipes and tips for Jenkins and Jenkinsfiles2026-02-17

1 - How Jenkins Works

Understanding Jenkins architecture and concepts

Source: https://www.jenkins.io/doc/book/managing/nodes/

Source glossary: https://www.jenkins.io/doc/book/glossary/

1. Jenkins Master Slave Architecture

Jenkins Master Slave Architecture

The Jenkins controller is the master node which is able to launch jobs on different nodes (machines) directed by an Agent. The Agent can the use one or several executors to execute the job(s) depending on configuration.

Jenkins is using Master/Slave architecture with the following components:

1.1. Jenkins controller/Jenkins master node

The central, coordinating process which stores configuration, loads plugins, and renders the various user interfaces for Jenkins.

The Jenkins controller is the Jenkins service itself and is where Jenkins is installed. It is a webserver that also acts as a “brain” for deciding how, when and where to run tasks. Management tasks (configuration, authorization, and authentication) are executed on the controller, which serves HTTP requests. Files written when a Pipeline executes are written to the filesystem on the controller unless they are off-loaded to an artifact repository such as Nexus or Artifactory.

1.2. Nodes

A machine which is part of the Jenkins environment and capable of executing Pipelines or jobs. Both the Controller and Agents are considered to be Nodes.

Nodes are the “machines” on which build agents run. Jenkins monitors each attached node for disk space, free temp space, free swap, clock time/sync and response time. A node is taken offline if any of these values go outside the configured threshold.

The Jenkins controller itself runs on a special built-in node. It is possible to run agents and executors on this built-in node although this can degrade performance, reduce scalability of the Jenkins instance, and create serious security problems and is strongly discouraged, especially for production environments.

1.3. Agents

An agent is typically a machine, or container, which connects to a Jenkins controller and executes tasks when directed by the controller.

Agents manage the task execution on behalf of the Jenkins controller by using executors. An agent is actually a small (170KB single jar) Java client process that connects to a Jenkins controller and is assumed to be unreliable. An agent can use any operating system that supports Java. Tools required for builds and tests are installed on the node where the agent runs; they can be installed directly or in a container (Docker or Kubernetes). Each agent is effectively a process with its own PID (Process Identifier) on the host machine.

In practice, nodes and agents are essentially the same but it is good to remember that they are conceptually distinct.

1.4. Executors

A slot for execution of work defined by a Pipeline or job on a Node. A Node may have zero or more Executors configured which corresponds to how many concurrent Jobs or Pipelines are able to execute on that Node.

An executor is a slot for execution of tasks; effectively, it is a thread in the agent. The number of executors on a node defines the number of concurrent tasks that can be executed on that node at one time. In other words, this determines the number of concurrent Pipeline stages that can execute on that node at one time.

The proper number of executors per build node must be determined based on the resources available on the node and the resources required for the workload. When determining how many executors to run on a node, consider CPU and memory requirements as well as the amount of I/O and network activity:

  • One executor per node is the safest configuration.
  • One executor per CPU core may work well if the tasks being run are small.
  • Monitor I/O performance, CPU load, memory usage, and I/O throughput carefully when running multiple executors on a node.

1.5. Jobs

A user-configured description of work which Jenkins should perform, such as building a piece of software, etc.

2. Jenkins dynamic node

Jenkins has static slave nodes and can trigger the generation of dynamic slave nodes

Jenkins Master/slave architecture

2 - Jenkins Pipelines

Declarative and scripted pipeline syntax

1. What is a pipeline ?

https://www.jenkins.io/doc/book/pipeline/

Jenkins Pipeline (or simply “Pipeline” with a capital “P”) is a suite of plugins which supports implementing and integrating continuous delivery pipelines into Jenkins.

A continuous delivery (CD) pipeline is an automated expression of your process for getting software from version control right through to your users and customers. Every change to your software (committed in source control) goes through a complex process on its way to being released. This process involves building the software in a reliable and repeatable manner, as well as progressing the built software (called a “build”) through multiple stages of testing and deployment.

Pipeline provides an extensible set of tools for modeling simple-to-complex delivery pipelines “as code” via the Pipeline domain-specific language (DSL) syntax. View footnote 1

The definition of a Jenkins Pipeline is written into a text file (called a Jenkinsfile) which in turn can be committed to a project’s source control repository. View footnote 2 This is the foundation of “Pipeline-as-code”; treating the CD pipeline a part of the application to be versioned and reviewed like any other code.

2. Pipeline creation via UI

it’s not recommended but it’s possible to create a pipeline via the UI.

There are several drawbacks:

  • no code revision
  • difficult to read, understand

3. Groovy

Scripted and declarative pipelines are using groovy language.

Checkout https://www.guru99.com/groovy-tutorial.html to have a quick overview of this derived language check Wikipedia

4. Difference between scripted pipeline (freestyle) and declarative pipeline syntax

What are the main differences ? Here are some of the most important things you should know:

  • Basically, declarative and scripted pipelines differ in terms of the programmatic approach. One uses a declarative programming model and the second uses an imperative programming mode.
  • Declarative pipelines break down stages into multiple steps, while in scripted pipelines there is no need for this. Example below

Declarative and Scripted Pipelines are constructed fundamentally differently. Declarative Pipeline is a more recent feature of Jenkins Pipeline which:

  • provides richer syntactical features over Scripted Pipeline syntax, and
  • is designed to make writing and reading Pipeline code easier.
  • By default automatically checkout stage

Many of the individual syntactical components (or “steps”) written into a Jenkinsfile, however, are common to both Declarative and Scripted Pipeline. Read more about how these two types of syntax differ in Pipeline concepts and Pipeline syntax overview.

5. Declarative pipeline example

Pipeline syntax documentation

 pipeline {
   agent {
     // executed on an executor with the label 'some-label'
     // or 'docker', the label normally specifies:
     // - the size of the machine to use
     //   (eg.: Docker-C5XLarge used for build that needs a powerful machine)
     // - the features you want in your machine
     //   (eg.: docker-base-ubuntu an image with docker command available)
     label "some-label"
   }

   stages {
     stage("foo") {
       steps {
         // variable assignment and Complex global
         // variables (with properties or methods)
         // can only be done in a script block
         script {
           foo = docker.image('ubuntu')
           env.bar = "${foo.imageName()}"
           echo "foo: ${foo.imageName()}"
         }
       }
     }
     stage("bar") {
       steps{
         echo "bar: ${env.bar}"
         echo "foo: ${foo.imageName()}"
       }
     }
   }
 }

6. Scripted pipeline example

Scripted pipelines permit a developer to inject code, while the declarative Jenkins pipeline doesn’t. should be avoided actually, try to use jenkins library instead

node {

  git url: 'https://github.com/jfrogdev/project-examples.git'

  // Get Artifactory server instance, defined in the Artifactory Plugin
  // administration page.
  def server = Artifactory.server "SERVER_ID"

  // Read the upload spec and upload files to Artifactory.
  def downloadSpec =
       '''{
       "files": [
         {
            "pattern": "libs-snapshot-local/*.zip",
            "target": "dependencies/",
            "props": "p1=v1;p2=v2"
         }
       ]
   }'''

  def buildInfo1 = server.download spec: downloadSpec

  // Read the upload spec which was downloaded from github.
  def uploadSpec =
     '''{
     "files": [
       {
          "pattern": "resources/Kermit.*",
          "target": "libs-snapshot-local",
          "props": "p1=v1;p2=v2"
       },
       {
          "pattern": "resources/Frogger.*",
          "target": "libs-snapshot-local"
       }
      ]
   }'''


  // Upload to Artifactory.
  def buildInfo2 = server.upload spec: uploadSpec

  // Merge the upload and download build-info objects.
  buildInfo1.append buildInfo2

  // Publish the build to Artifactory
  server.publishBuildInfo buildInfo1
}

7. Why Pipeline?

Jenkins is, fundamentally, an automation engine which supports a number of automation patterns. Pipeline adds a powerful set of automation tools onto Jenkins, supporting use cases that span from simple continuous integration to comprehensive CD pipelines. By modeling a series of related tasks, users can take advantage of the many features of Pipeline:

  • Code: Pipelines are implemented in code and typically checked into source control, giving teams the ability to edit, review, and iterate upon their delivery pipeline.
  • Durable: Pipelines can survive both planned and unplanned restarts of the Jenkins controller.
  • Pausable: Pipelines can optionally stop and wait for human input or approval before continuing the Pipeline run.
  • Versatile: Pipelines support complex real-world CD requirements, including the ability to fork/join, loop, and perform work in parallel.
  • Extensible: The Pipeline plugin supports custom extensions to its DSL see jenkins doc and multiple options for integration with other plugins.

While Jenkins has always allowed rudimentary forms of chaining Freestyle Jobs together to perform sequential tasks, see jenkins doc Pipeline makes this concept a first-class citizen in Jenkins.

More information on Official jenkins documentation - Pipeline

3 - Jenkins Library

Creating and using Jenkins shared libraries

1. What is a jenkins shared library ?

As Pipeline is adopted for more and more projects in an organization, common patterns are likely to emerge. Oftentimes it is useful to share parts of Pipelines between various projects to reduce redundancies and keep code “DRY”

for more information check pipeline shared libraries

2. Loading libraries dynamically

As of version 2.7 of the Pipeline: Shared Groovy Libraries plugin, there is a new option for loading (non-implicit) libraries in a script: a library step that loads a library dynamically, at any time during the build.

If you are only interested in using global variables/functions (from the vars/ directory), the syntax is quite simple:

library 'my-shared-library'

Thereafter, any global variables from that library will be accessible to the script.

3. jenkins library directory structure

The directory structure of a Shared Library repository is as follows:

(root)
+- src        # Groovy source files
|   +- org
|       +- foo
|           +- Bar.groovy  # for org.foo.Bar class
|
+- vars       # The vars directory hosts script
              # files that are exposed as a variable in Pipelines
|   +- foo.groovy          # for global 'foo' variable
|   +- foo.txt             # help for 'foo' variable
|
+- resources  # resource files (external libraries only)
|   +- org
|      +- foo
|         +- bar.json      # static helper data for org.foo.Bar

4. Jenkins library

remember that jenkins library code is executed on master node

if you want to execute code on the node, you need to use jenkinsExecutor

usage of jenkins executor

String credentialsId = 'babee6c1-14fe-4d90-9da0-ffa7068c69af'
def lib = library(
    identifier: 'jenkins_library@v1.0',
    retriever: modernSCM([
        $class: 'GitSCMSource',
        remote: 'git@github.com:fchastanet/jenkins-library.git',
        credentialsId: credentialsId
    ])
)
// this is the jenkinsExecutor instance
def docker = lib.fchastanet.Docker.new(this)

Then in the library, it is used like this:

def status = this.jenkinsExecutor.sh(
  script: "docker pull ${cacheTag}", returnStatus: true
)

5. Jenkins library structure

I remarked that a lot of code was duplicated between all my Jenkinsfiles so I created this library https://github.com/fchastanet/jenkins-library

(root)
+- doc    # markdown files automatically generated
          # from groovy files by generateDoc.sh
+- src    # Groovy source files
|   +- fchastanet
|       +- Cloudflare.groovy     # zonePurge
|       +- Docker.groovy         # getTagCompatibleFromBranch
                                 # pullBuildPushImage, ...
|       +- Git.groovy            # getRepoURL, getCommitSha,
                                 # getLastPusherEmail,
                                 # updateConditionalGithubCommitStatus
|       +- Kubernetes.groovy     # deployHelmChart, ...
|       +- Lint.groovy           # dockerLint,
                                 # transform lighthouse report
                                 # to Warnings NG issues format
|       +- Mail.groovy           # sendTeamsNotification,
                                 # sendConditionalEmail, ...
|       +- Utils.groovy          # deepMerge, isCollectionOrArray,
                                 # deleteDirAsRoot,
                                 # initAws (could be moved to Aws class)
+- vars   # The vars directory hosts script files that
          # are exposed as a variable in Pipelines
|   +- dockerPullBuildPush.groovy #
|   +- whenOrSkip.groovy          #

6. external resource usage

If you need you check out how I used this repository https://github.com/fchastanet/jenkins-library-resources in jenkins_library (Linter) that hosts some resources to parse result files.

4 - Jenkins Best Practices

Best practices and patterns for Jenkins and Jenkinsfiles

1. Pipeline best practices

Official Jenkins pipeline best practices

Summary:

  • Make sure to use Groovy code in Pipelines as glue
  • Externalize shell scripts from Jenkins Pipeline
    • for better jenkinsfile readability
    • in order to test the scripts isolated from jenkins
  • Avoid complex Groovy code in Pipelines
    • Groovy code always executes on controller which means using controller resources(memory and CPU)
      • it is not the case for shell scripts
    • eg1: prefer using jq inside shell script instead of groovy JsonSlurper
    • eg2: prefer calling curl instead of groovy http request
  • Reducing repetition of similar Pipeline steps (eg: one sh step instead of severals)
    • group similar steps together to avoid step creation/destruction overhead
  • Avoiding calls to Jenkins.getInstance

2. Shared library best practices

Official Jenkins shared libraries best practices

Summary:

  • Do not override built-in Pipeline steps
  • Avoiding large global variable declaration files
  • Avoiding very large shared libraries

And:

  • import jenkins library using a tag
    • like in docker build, npm package with package-lock.json or python pip lock, it’s advised to target a given version of the library
      • because some changes could break
  • The missing part: we miss on this library unit tests
    • but each pipeline is a kind of integration test
  • Because a pipeline can be resumed, your library’s classes should implement Serializable class and the following attribute has to be provided:
private static final long serialVersionUID = 1L

5 - Annotated Jenkinsfiles - Part 1

Detailed Jenkinsfile examples with annotations

Pipeline example

1. Simple one

This build is used to generate docker images used to build production code and launch phpunit tests. This pipeline is parameterized in the Jenkins UI directly with the parameters:

  • branch (git branch to use)
  • environment(select with 3 options: build, phpunit or all)
    • it would have been better to use simply 2 checkboxes phpunit/build
  • project_branch

Here the source code with inline comments:

Annotated jenkinsfile Expand source

// This method allows to convert the branch name to a docker image tag.
// This method is generally used by most of my jenkins pipelines, it's why it has been added to https://github.com/fchastanet/jenkins-library/blob/master/src/fchastanet/Docker.groovy#L31
def getTagCompatibleFromBranch(String branchName) {
    def String tag = branchName.toLowerCase()
    tag = tag.replaceAll("^origin/", "")
    return tag.replaceAll('/', '_')
}

// we declare here some variables that will be used in next stages
def String deploymentBranchTagCompatible = ''

pipeline {
    agent {
        node {
            // the pipeline is executed on a machine with docker daemon
            // available
            label 'docker-ubuntu'
        }
    }

    stages {
        stage ('checkout') {
            steps {
                // this command is actually not necessary because checkout is
                // done automatically when using declarative pipeline
                sh 'echo "pulling ... ${GIT_BRANCH#origin/}"'
                checkout scm

                // this particular build needs to access to some private github
                // repositories, so here we are copying the ssh key
                // it would be better to use new way of injecting ssh key
                // inside docker using sshagent
                // check https://stackoverflow.com/a/66897280
                withCredentials([
                    sshUserPrivateKey(
                      credentialsId: '855aad9f-1b1b-494c-aa7f-4de881c7f659',
                      keyFileVariable: 'sshKeyFile'
                   )
                ]) {
                    // best practice similar steps should be merged into one
                    sh 'rm -f ./phpunit/id_rsa'
                    sh 'rm -f ./build/id_rsa'
                    // here we are escaping '$' so the variable will be
                    // interpolated on the jenkins slave and not the jenkins
                    // master node instead of escaping, we could have used
                    // single quotes
                    sh "cp \$sshKeyFile ./phpunit/id_rsa"
                    sh "cp \$sshKeyFile ./build/id_rsa"
                }
                script {
                    // as actually scm is already done before executing the
                    // first step, this call could have been done during
                    // declaration of this variable
                    deploymentBranchTagCompatible = getTagCompatibleFromBranch(GIT_BRANCH)
                }
            }
        }
        stage("build Build env") {
            when {
                // the build can be launched with the parameter environment
                // defined in the configuration of the jenkins job, these
                // parameters could have been defined directly in the pipeline
                // see https://www.jenkins.io/doc/book/pipeline/syntax/#parameters
                expression { return params.environment != "phpunit"}
            }
            steps {
                // here we could have launched all this commands in the same sh
                // directive
                sh "docker build --build-arg BRANCH=${params.project_branch} -t build build"
                // use a constant for dockerRegistryId.dkr.ecr.eu-west-1.amazonaws.com
                sh "docker tag build dockerRegistryId.dkr.ecr.eu-west-1.amazonaws.com/build:${deploymentBranchTagCompatible}"
                sh "docker push dockerRegistryId.dkr.ecr.eu-west-1.amazonaws.com/build:${deploymentBranchTagCompatible}"
            }
        }
        stage("build PHPUnit env") {
            when {
                // it would have been cleaner to use
                // expression { return params.environment = "phpunit"}
                expression { return params.environment != "build"}
            }
            steps {
                sh "docker build --build-arg BRANCH=${params.project_branch} -t phpunit phpunit"
                sh "docker tag phpunit dockerRegistryId.dkr.ecr.eu-west-1.amazonaws.com/phpunit:${deploymentBranchTagCompatible}"
                sh "docker push dockerRegistryId.dkr.ecr.eu-west-1.amazonaws.com/phpunit:${deploymentBranchTagCompatible}"
            }
        }
    }
}

without seeing the Dockerfile files, we can advise :

  • to build these images in the same pipeline where build and phpunit are run
    • the images are built at the same time so we are sure that we are using the right version
  • apparently the docker build depend on the branch of the project, this should be avoided
  • ssh key is used in docker image, that could lead to a security issue as ssh key is still in the history of images layers even if it has been removed in subsequent layers, check https://stackoverflow.com/a/66897280 for information on how to use ssh-agent instead
  • we could use a single Dockerfile with 2 stages:
    • one stage to generate production image
    • one stage that inherits production stage, used to execute phpunit
    • it has the following advantages :
      • reduce the total image size because of the reuse different docker image layers
      • only one Dockerfile to maintain

2. More advanced and annotated Jenkinsfiles

6 - Annotated Jenkinsfiles - Part 2

More annotated Jenkinsfile examples

1. Introduction

This example is missing the use of parameters, jenkins library in order to reuse common code

This example uses :

  • post conditions https://www.jenkins.io/doc/book/pipeline/syntax/#post
  • github plugin to set commit status indicating the result of the build
  • usage of several jenkins plugins, you can check here to get the full list installed on your server and even generate code snippets by adding pipeline-syntax/ to your jenkins server url

But it misses:

check Pipeline syntax documentation

2. Annotated Jenkinsfile

// Define variables for QA environment
def String registry_id = 'awsAccountId'
def String registry_url = registry_id + '.dkr.ecr.us-east-1.amazonaws.com'
def String image_name = 'project'
def String image_fqdn_master = registry_url + '/' + image_name + ':master'
def String image_fqdn_current_branch = image_fqdn_master

// this method is used by several of my pipelines and has been added
// to jenkins_library <https://github.com/fchastanet/jenkins-library/blob/master/src/fchastanet/Git.groovy#L156>
void publishStatusToGithub(String status) {
  step([
    $class: "GitHubCommitStatusSetter",
    reposSource: [$class: "ManuallyEnteredRepositorySource", url: "https://github.com/fchastanet/project"],
    errorHandlers: [[$class: 'ShallowAnyErrorHandler']],
    statusResultSource: [
      $class: 'ConditionalStatusResultSource',
      results: [
        [$class: 'AnyBuildResult', state: status]
      ]
    ]
  ]);
}

pipeline {
  agent {
    node {
      // bad practice: try to indicate in your node labels, which feature it
      // includes for example, here we need docker, label could have been
      // 'eks-nonprod-docker'
      label 'eks-nonprod'
    }
  }
  stages {
    stage ('Checkout') {
      steps {
        // checkout is not necessary as it is automatically done
        checkout scm

        script {
          // 'wrap' allows to inject some useful variables like BUILD_USER,
          // BUILD_USER_FIRST_NAME
          // see https://www.jenkins.io/doc/pipeline/steps/build-user-vars-plugin/
          wrap([$class: 'BuildUser']) {
            def String displayName = "#${currentBuild.number}_${BRANCH}_${BUILD_USER}_${DEPLOYMENT}"

            // params could have been defined inside the pipeline directly
            // instead of defining them in jenkins build configuration
            if (params.DEPLOYMENT == 'staging') {
              displayName = "${displayName}_${INSTANCE}"
            }
            // next line allows to change the build name, check addHtmlBadge
            // plugin function for more advanced usage of this feature, you
            // check this jenkinsfile 05-02-Annotated-Jenkinsfiles.md
            currentBuild.displayName = displayName
          }
        }
      }
    }
    stage ('Run tests') {
      steps {
        // all these sh directives could have been merged into one
        // it is best to use a separated sh file that could take some parameters
        // as it is simpler to read and to eventually test separately
        sh 'docker build -t project-test "$PWD"/docker/test'
        sh 'cp "$PWD"/app/config/parameters.yml.dist "$PWD"/app/config/parameters.yml'
        // for better readability and if separated script is not possible, use
        // continuation line for better readability
        sh 'docker run -i --rm -v "$PWD":/var/www/html/ -w /var/www/html/ project-test  /bin/bash -c "composer install -a && ./bin/phpunit -c /var/www/html/app/phpunit.xml --coverage-html /var/www/html/var/logs/coverage/ --log-junit /var/www/html/var/logs/phpunit.xml  --coverage-clover /var/www/html/var/logs/clover_coverage.xml"'
      }
      // Run the steps in the post section regardless of the completion status
      // of the Pipeline’s or stage’s run.
      // see https://www.jenkins.io/doc/book/pipeline/syntax/#post
      post {
        always {
          // report unit test reports (unit test should generate result using
          // using junit format)
          junit 'var/logs/phpunit.xml'
          // generate coverage page from test results
          step([
            $class: 'CloverPublisher',
            cloverReportDir: 'var/logs/',
            cloverReportFileName: 'clover_coverage.xml'
          ])
          // publish html page with the result of the coverage
          publishHTML(
            target: [
              allowMissing: false,
              alwaysLinkToLastBuild: false,
              keepAll: true,
              reportDir: 'var/logs/coverage/',
              reportFiles: 'index.html',
              reportName: "Coverage Report"
            ]
          )
        }
      }
    }
    // this stage will be executed only if previous stage is successful
    stage('Build image') {
      when {
        // this stage is executed only if these conditions returns true
        expression {
          return
            params.DEPLOYMENT == "staging"
            || (
              params.DEPLOYMENT == "prod"
              && env.GIT_BRANCH == 'origin/master'
            )
        }
      }
      steps {
        script {
          // this code is used in most of the pipeline and has been centralized
          // in https://github.com/fchastanet/jenkins-library/blob/master/src/fchastanet/Git.groovy#L39
          env.IMAGE_TAG = env.GIT_COMMIT.substring(0, 7)
          // Update variable for production environment
          if ( params.DEPLOYMENT == 'prod' ) {
              registry_id = 'awsDockerRegistryId'
              registry_url = registry_id + '.dkr.ecr.eu-central-1.amazonaws.com'
              image_fqdn_master = registry_url + '/' + image_name + ':master'
          }

          image_fqdn_current_branch = registry_url + '/' + image_name + ':' + env.IMAGE_TAG
        }

        // As jenkins slave machine can be constructed on demand,
        // it doesn't always contains all docker image cache
        // here to avoid building docker image from scratch, we are trying to
        // pull an existing version of the docker image on docker registry
        // and then build using this image as cache, so all layers not updated
        // in Dockerfile will not be built again (gain of time)
        // It is again a recurrent usage in most of the pipelines
        // so the next 8 lines could be replaced by the call to this method
        // Docker
        // pullBuildPushImage https://github.com/fchastanet/jenkins-library/blob/master/src/fchastanet/Docker.groovy#L46

        // Pull the master from repository (|| true avoids errors if the image
        // hasn't been pushed before)
        sh "docker pull ${image_fqdn_master} || true"

        // Build the image using pulled image as cache
        // instead of using concatenation, it is more readable to use variable interpolation
        // Eg: "docker build --cache-from ${image_fqdn_master} -t ..."
        sh 'docker build \
            --cache-from ' + image_fqdn_master + ' \
            -t ' + image_name + ' \
            -f "$PWD/docker/prod/Dockerfile" \
            .'
      }
    }
    stage('Deploy image (Staging)') {
      when {
          expression { return params.DEPLOYMENT == "staging" }
      }

      steps {
        script {
          // Actually we should always push the image in order to be able to
          // feed the docker cache for next builds
          // Again the method Docker pullBuildPushImage https://github.com/fchastanet/jenkins-library/blob/master/src/fchastanet/Docker.groovy#L46
          // solves this issue and could be used instead of the next 6 lines
          // and "Push image (Prod)" stage

          // If building master, we should push the image with the tag master
          // to benefit from docker cache
          if ( env.GIT_BRANCH == 'origin/master' ) {
              sh label:"Tag the image as master",
                 script:"docker tag ${image_name} ${image_fqdn_master}"
              sh label:"Push the image as master",
                 script:"docker push ${image_fqdn_master}"
          }
        }

        sh label:"Tag the image", script:"docker tag ${image_name} ${image_fqdn_current_branch}"
        sh label:"Push the image", script:"docker push ${image_fqdn_current_branch}"
        // use variable interpolation instead of concatenation
        sh label:"Deploy on cluster", script:" \
          helm3 upgrade project-" + params.INSTANCE + " -i \
            --namespace project-" + params.INSTANCE + " \
            --create-namespace \
            --cleanup-on-fail \
            --atomic \
            -f helm/values_files/values-" + params.INSTANCE + ".yaml \
            --set deployment.php_container.image.pullPolicy=Always \
            --set image.tag=" + env.IMAGE_TAG + " \
            ./helm"
      }
    }
    stage('Push image (Prod)') {
      when {
        expression { return params.DEPLOYMENT == "prod" && env.GIT_BRANCH == 'origin/master'}
      }
      // The method Docker pullBuildPushImage https://github.com/fchastanet/jenkins-library/blob/master/src/fchastanet/Docker.groovy#L46
      // provides a generic way of managing the pull, build, push of the docker
      // images, by managing also a common way of tagging docker images
      steps {
        sh label:"Tag the image as master", script:"docker tag ${image_name} ${image_fqdn_current_branch}"
        sh label:"Push the image as master", script:"docker push ${image_fqdn_current_branch}"
      }
    }
  }
  post {
    always {
      // mark github commit as built
      publishStatusToGithub("${currentBuild.currentResult}")
    }
  }
}

This directive is really difficult to read and eventually debug it

sh 'docker run -i --rm -v "$PWD":/var/www/html/ -w /var/www/html/ project-test  /bin/bash -c "composer install -a && ./bin/phpunit -c /var/www/html/app/phpunit.xml --coverage-html /var/www/html/var/logs/coverage/ --log-junit /var/www/html/var/logs/phpunit.xml  --coverage-clover /var/www/html/var/logs/clover_coverage.xml"'

Another way to write previous directive is to:

  • use continuation line
  • avoid ‘&&’ as it can mask errors, use ‘;’ instead
  • use ‘set -o errexit’ to fail on first error
  • use ‘set -o pipefail’ to fail if eventual piped command is failing
  • ‘set -x’ allows to trace every command executed for better debugging

Here a possible refactoring:

sh ''''
  docker run -i --rm \
    -v "$PWD":/var/www/html/ \
    -w /var/www/html/ \
    project-test \
    /bin/bash -c "\
      set -x ;\
      set -o errexit ;\
      set -o pipefail ;\
      composer install -a ;\
      ./bin/phpunit \
        -c /var/www/html/app/phpunit.xml \
        --coverage-html /var/www/html/var/logs/coverage/ \
        --log-junit /var/www/html/var/logs/phpunit.xml  \
        --coverage-clover /var/www/html/var/logs/clover_coverage.xml
    "
'''

Note however it is best to use a separated sh file(s) that could take some parameters as it is simpler to read and to eventually test separately. Here a refactoring using a separated sh file:

runTests.sh

#!/bin/bash
set -x -o errexit -o pipefail

composer install -a

./bin/phpunit \
  -c /var/www/html/app/phpunit.xml \
  --coverage-html /var/www/html/var/logs/coverage/ \
  --log-junit /var/www/html/var/logs/phpunit.xml \
  --coverage-clover /var/www/html/var/logs/clover_coverage.xml

jenkinsRunTests.sh

#!/bin/bash
set -x -o errexit -o pipefail

docker build -t project-test "${PWD}/docker/test"

docker run -i --rm \
  -v "${PWD}:/var/www/html/" \
  -w /var/www/html/ \
  project-test \
  runTests.sh

Then the sh directive becomes simply

sh 'jenkinsRunTests.sh'

7 - Annotated Jenkinsfiles - Part 3

Additional Jenkinsfile pattern examples

1. Introduction

This build will:

  • pull/build/push docker image used to generate project files
  • lint
  • run Unit tests with coverage
  • build the SPA
  • run accessibility tests
  • build story book and deploy it
  • deploy spa on s3 bucket and refresh cloudflare cache

It allows to build for production and qa stages allowing different instances. Every build contains:

  • a summary of the build
    • git branch
    • git revision
    • target environment
  • all the available Urls:
    • spa url
    • storybook url

2. Annotated Jenkinsfile

// anonymized parameters
String credentialsId = 'jenkinsCredentialId'
def lib = library(
  identifier: 'jenkins_library@v1.0',
  retriever: modernSCM([
    $class: 'GitSCMSource',
    remote: 'git@github.com:fchastanet/jenkins-library.git',
    credentialsId: credentialsId
  ])
)
def docker = lib.fchastanet.Docker.new(this)
def git = lib.fchastanet.Git.new(this)
def mail = lib.fchastanet.Mail.new(this)
def utils = lib.fchastanet.Utils.new(this)
def cloudflare = lib.fchastanet.Cloudflare.new(this)

// anonymized parameters
String CLOUDFLARE_ZONE_ID = 'cloudflareZoneId'
String CLOUDFLARE_ZONE_ID_PROD = 'cloudflareZoneIdProd'
String REGISTRY_ID_QA  = 'dockerRegistryId'
String REACT_APP_PENDO_API_KEY = 'pendoApiKey'

String REGISTRY_QA  = REGISTRY_ID_QA + '.dkr.ecr.us-east-1.amazonaws.com'
String IMAGE_NAME_SPA = 'project-ui'
String STAGING_API_URL = 'https://api.host'
String INSTANCE_URL = "https://${params.instanceName}.host"
String REACT_APP_API_BASE_URL_PROD = 'https://ui.host'
String REACT_APP_PENDO_SOURCE_DOMAIN = 'https://cdn.eu.pendo.io'

String buildBucketPrefix
String S3_PUBLIC_URL = 'qa-spa.s3.amazonaws.com/project'
String S3_PROD_PUBLIC_URL = 'spa.s3.amazonaws.com/project'

List<String> instanceChoices = (1..20).collect { 'project' + it }

Map buildInfo = [
  apiUrl: '',
  storyBookAvailable: false,
  storyBookUrl: '',
  storyBookDocsUrl: '',
  spaAvailable: false,
  spaUrl: '',
  instanceName: '',
]

// add information on summary page
def addBuildInfo(buildInfo) {
  String deployInfo = ''
  if (buildInfo.spaAvailable) {
    String formatInstanceName = buildInfo.instanceName ?
      " (${buildInfo.instanceName})" : '';
    deployInfo += "<a href='${buildInfo.spaUrl}'>SPA${formatInstanceName}</a>"
  }
  if (buildInfo.storyBookAvailable) {
    deployInfo += " / <a href='${buildInfo.storyBookUrl}'>Storybook</a>"
    deployInfo += " / <a href='${buildInfo.storyBookDocsUrl}'>Storybook docs</a>"
  }
  String summaryHtml = """
    <b>branch : </b>${GIT_BRANCH}<br/>
    <b>revision : </b>${GIT_COMMIT}<br/>
    <b>target env : </b>${params.targetEnv}<br/>
    ${deployInfo}
  """
  removeHtmlBadges id: "htmlBadge${currentBuild.number}"
  addHtmlBadge html: summaryHtml, id: "htmlBadge${currentBuild.number}"
}

pipeline {
  agent {
    node {
      // this image has the features docker and lighthouse
      label 'docker-base-ubuntu-lighthouse'
    }
  }

  parameters {
    gitParameter(
      branchFilter: 'origin/(.*)',
      defaultValue: 'main',
      quickFilterEnabled: true,
      sortMode: 'ASCENDING_SMART',
      name: 'BRANCH',
      type: 'PT_BRANCH'
    )
    choice(
      name: 'targetEnv',
      choices: ['none', 'testing', 'production'],
      description: 'Where it should be deployed to? (Default: none - No deploy)'
    )
    booleanParam(
      name: 'buildStorybook',
      defaultValue: false,
      description: 'Build Storybook (will only apply if selected targetEnv is testing)'
    )
    choice(
      name: 'instanceName',
      choices: instanceChoices,
      description: 'Instance name to deploy the revision'
    )
  }

  stages {
    stage('Build SPA image') {
      steps {
        script {
          // set build status to pending on github commit
          step([$class: 'GitHubSetCommitStatusBuilder'])
          wrap([$class: 'BuildUser']) {
            currentBuild.displayName = "#${currentBuild.number}_${BRANCH}_${BUILD_USER}_${targetEnv}"
          }

          branchName = docker.getTagCompatibleFromBranch(env.GIT_BRANCH)
          shortSha = git.getShortCommitSha(env.GIT_BRANCH)

          if (params.targetEnv == 'production') {
            buildBucketPrefix = GIT_COMMIT
            buildInfo.apiUrl = REACT_APP_API_BASE_URL_PROD
            s3BaseUrl = 's3://project-spa/project'
          } else {
            buildBucketPrefix = params.instanceName
            buildInfo.instanceName = params.instanceName
            buildInfo.spaUrl = "${INSTANCE_URL}/index.html"
            buildInfo.apiUrl = STAGING_API_URL
            s3BaseUrl = 's3://project-qa-spa/project'
            buildInfo.storyBookUrl = "${INSTANCE_URL}/storybook/index.html"
            buildInfo.storyBookDocsUrl = "${INSTANCE_URL}/storybook-docs/index.html"
          }
          addBuildInfo(buildInfo)

          // Setup .env
          sh """
            set -x
            echo "REACT_APP_API_BASE_URL = '${buildInfo.apiUrl}'" > ./.env
            echo "REACT_APP_PENDO_SOURCE_DOMAIN = '${REACT_APP_PENDO_SOURCE_DOMAIN}'" >> ./.env
            echo "REACT_APP_PENDO_API_KEY = '${REACT_APP_PENDO_API_KEY}'" >> ./.env
          """

          withCredentials([
            sshUserPrivateKey(
              credentialsId: 'sshCredentialsId',
              keyFileVariable: 'sshKeyFile')
          ]) {
            docker.pullBuildPushImage(
              buildDirectory:   pwd(),
              // use safer way to inject ssh key during docker build
              buildArgs: "--ssh default=\$sshKeyFile --build-arg USER_ID=\$(id -u)",
              registryImageUrl: "${REGISTRY_QA}/${IMAGE_NAME_SPA}",
              tagPrefix:        "${IMAGE_NAME_SPA}:",
              localTagName:     "latest",
              tags: [
                shortSha,
                branchName
              ],
              pullTags: ['main']
            )
          }
        }
      }
    }

    stage('Linting') {
      steps {
        sh """
          docker run --rm \
            -v ${env.WORKSPACE}:/app \
            -v /app/node_modules \
            ${IMAGE_NAME_SPA} \
            npm run lint
        """
      }
    }

    stage('UT') {
      steps {
        script {
          sh """docker run --rm  \
            -v ${env.WORKSPACE}:/app \
            -v /app/node_modules \
            ${IMAGE_NAME_SPA} \
            npm run test:coverage -- --ci
          """

          junit 'output/junit.xml'

          // https://plugins.jenkins.io/clover/
          step([
            $class: 'CloverPublisher',
            cloverReportDir: 'output/coverage',
            cloverReportFileName: 'clover.xml',
            healthyTarget: [
              methodCoverage: 70,
              conditionalCoverage: 70,
              statementCoverage: 70
            ],
            // build will not fail but be set as unhealthy if coverage goes
            // below 60%
            unhealthyTarget: [
              methodCoverage: 60,
              conditionalCoverage: 60,
              statementCoverage: 60
            ],
            // build will fail if coverage goes below 50%
            failingTarget: [
              methodCoverage: 50,
              conditionalCoverage: 50,
              statementCoverage: 50
            ]
          ])
        }
      }
    }

    stage('Build SPA') {
      steps {
        script {
          sh """
            docker run --rm \
              -v ${env.WORKSPACE}:/app \
              -v /app/node_modules \
              ${IMAGE_NAME_SPA}
          """
        }
      }
    }

    stage('Accessibility tests') {
      steps {
        script {
          // the pa11y-ci could have been made available in the node image
          // to avoid installation each time, the build is launched
          sh '''
            sudo npm install -g serve pa11y-ci
            serve -s build > /dev/null 2>&1 &
            pa11y-ci --threshold 5 http://127.0.0.1:3000
          '''
        }
      }
    }

    stage('Build Storybook') {
      steps {
        whenOrSkip(
          params.targetEnv == 'testing'
          && params.buildStorybook == true
        ) {
          script {
            sh """
              docker run --rm \
                -v ${env.WORKSPACE}:/app \
                -v /app/node_modules \
                ${IMAGE_NAME_SPA} \
                sh -c 'npm run storybook:build -- --output-dir build/storybook \
                  && npm run storybook:build-docs -- --output-dir build/storybook-docs'
            """
            buildInfo.storyBookAvailable = true
          }
        }
      }
    }

    stage('Artifacts to S3') {
      steps {
        whenOrSkip(params.targetEnv != 'none') {
          script {
            if (params.targetEnv == 'production') {
              utils.initAws('arn:aws:iam::awsIamId:role/JenkinsSlave')
            }

            sh "aws s3 cp ${env.WORKSPACE}/build ${s3BaseUrl}/${buildBucketPrefix} --recursive --no-progress"
            sh "aws s3 cp ${env.WORKSPACE}/build ${s3BaseUrl}/project1 --recursive --no-progress"

            if (params.targetEnv == 'production') {
              echo 'project SPA packages have been pushed to production bucket.'
              echo '''You can refresh the production indexes with the CD
              production pipeline.'''
              cloudflare.zonePurge(CLOUDFLARE_ZONE_ID_PROD, [prefixes:[
                "${S3_PROD_PUBLIC_URL}/project1/"
              ]])
            } else {
              cloudflare.zonePurge(CLOUDFLARE_ZONE_ID, [prefixes:[
                "${S3_PUBLIC_URL}/${buildBucketPrefix}/"
              ]])

              buildInfo.spaAvailable = true
              publishChecks detailsURL: buildInfo.spaUrl,
                name: 'projectSpaUrl',
                title: 'project SPA url'
            }
            addBuildInfo(buildInfo)
          }
        }
      }
    }
  }

  post {
    always {
      script {
        git.updateConditionalGithubCommitStatus()
        mail.sendConditionalEmail()
      }
    }
  }
}

8 - Annotated Jenkinsfiles - Part 4

Complex Jenkinsfile scenarios

1. introduction

The project aim is to create a browser extension available on chrome and firefox

This build allows to:

  • lint the project using megalinter and phpstorm inspection
  • build necessary docker images
  • build firefox and chrome extensions
  • deploy firefox extension on s3 bucket
  • deploy chrome extension on google play store

2. Annotated Jenkinsfile

def credentialsId = 'jenkinsSshCredentialsId'
def lib = library(
    identifier: 'jenkins_library',
    retriever: modernSCM([
        $class: 'GitSCMSource',
        remote: 'git@github.com:fchastanet/jenkins-library.git',
        credentialsId: credentialsId
    ])
)
def docker = lib.fchastanet.Docker.new(this)
def git = lib.fchastanet.Git.new(this)
def mail = lib.fchastanet.Mail.new(this)

def String deploymentBranchTagCompatible = ''
def String gitShortSha = ''
def String REGISTRY_URL = 'dockerRegistryId.dkr.ecr.eu-west-1.amazonaws.com'
def String ECR_BROWSER_EXTENSION_BUILD = 'browser_extension_lint'
def String BUILD_TAG = 'build'
def String PHPSTORM_TAG = 'phpstorm-inspections'
def String REFERENCE_JOB_NAME = 'Browser_extension_deploy'
def String FIREFOX_S3_BUCKET = 'browser-extensions'

// it would have been easier to use checkboxes to avoid 'both'/'none'
// complexity
def DEPLOY_CHROME = (params.targetStore == 'both' || params.targetStore == 'chrome')
def DEPLOY_FIREFOX = (params.targetStore == 'both' || params.targetStore == 'firefox')

pipeline {
  agent {
    node {
      label 'docker-base-ubuntu'
    }
  }
  parameters {
    gitParameter branchFilter: 'origin/(.*)',
      defaultValue: 'master',
      quickFilterEnabled: true,
      sortMode: 'ASCENDING_SMART',
      name: 'BRANCH',
      type: 'PT_BRANCH'

    choice (
      name: 'targetStore',
      choices: ['none', 'both', 'chrome', 'firefox'],
      description: 'Where it should be deployed to? (Default: none, has effect only on master branch)'
    )
  }
  environment {
    GOOGLE_CREDS = credentials('GoogleApiChromeExtension')
    GOOGLE_TOKEN = credentials('GoogleApiChromeExtensionCode')
    GOOGLE_APP_ID = 'googleAppId'
    // provided by https://addons.mozilla.org/en-US/developers/addon/api/key/
    FIREFOX_CREDS = credentials('MozillaApiFirefoxExtension')
    FIREFOX_APP_ID='{d4ce8a6f-675a-4f74-b2ea-7df130157ff4}'
  }

  stages {

    stage("Init") {
      steps {
        script {
          deploymentBranchTagCompatible = docker.getTagCompatibleFromBranch(env.GIT_BRANCH)
          gitShortSha = git.getShortCommitSha(env.GIT_BRANCH)
          echo "Branch ${env.GIT_BRANCH}"
          echo "Docker tag = ${deploymentBranchTagCompatible}"
          echo "git short sha = ${gitShortSha}"
        }
        sh 'echo StrictHostKeyChecking=no >> ~/.ssh/config'
      }
    }

    stage("Lint") {
      agent {
        docker {
          image 'megalinter/megalinter-javascript:v5'
          args "-u root -v ${WORKSPACE}:/tmp/lint --entrypoint=''"
          reuseNode true
        }
      }
      steps {
        sh 'npm install stylelint-config-rational-order'
        sh '/entrypoint.sh'
      }
    }

    stage("Build docker images") {
      steps {
        // whenOrSkip directive is defined in https://github.com/fchastanet/jenkins-library/blob/master/vars/whenOrSkip.groovy
        whenOrSkip(currentBuild.currentResult == "SUCCESS") {
          script {
            docker.pullBuildPushImage(
              buildDirectory:   'build',
              registryImageUrl: "${REGISTRY_URL}/${ECR_BROWSER_EXTENSION_BUILD}",
              tagPrefix:        "${ECR_BROWSER_EXTENSION_BUILD}:",
              tags: [
                "${BUILD_TAG}_${gitShortSha}",
                "${BUILD_TAG}_${deploymentBranchTagCompatible}",
              ],
              pullTags: ["${BUILD_TAG}_master"]
            )
          }
        }
      }
    }

    stage("Build firefox/chrome extensions") {
      steps {
        whenOrSkip(currentBuild.currentResult == "SUCCESS") {
          script {
              sh """
                docker run \
                  -v \$(pwd):/deploy \
                  --rm '${ECR_BROWSER_EXTENSION_BUILD}' \
                  /deploy/build/build-extensions.sh
              """
              // multiple git statuses can be set on a given commit
              // you can configure github to authorize pull request merge
              // based on the presence of one or more github statuses
              git.updateGithubCommitStatus("BUILD_OK")
          }
        }
      }
    }

    stage("Deploy extensions") {
      // deploy both extensions in parallel
      parallel {
        stage("Deploy chrome") {
          steps {
            whenOrSkip(currentBuild.currentResult == "SUCCESS" && DEPLOY_CHROME) {
              // do not fail the entire build if this stage fail
              // so firefox stage can be executed
              catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
                script {
                  // best practice: complex sh files have been created outside
                  // of this jenkinsfile deploy-chrome-extension.sh
                  sh """
                  docker run \
                      -v \$(pwd):/deploy \
                      -e APP_CREDS_USR='${GOOGLE_CREDS_USR}' \
                      -e APP_CREDS_PSW='${GOOGLE_CREDS_PSW}' \
                      -e APP_TOKEN='${GOOGLE_APP_TOKEN}' \
                      -e APP_ID='${GOOGLE_APP_ID}' \
                      --rm '${ECR_BROWSER_EXTENSION_BUILD}' \
                      /deploy/build/deploy-chrome-extension.sh
                  """
                  git.updateGithubCommitStatus("CHROME_DEPLOYED")
                }
              }
            }
          }
        }
        stage("Deploy firefox") {
          steps {
            whenOrSkip(currentBuild.currentResult == "SUCCESS" && DEPLOY_FIREFOX) {
              catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
                script {
                  // best practice: complex sh files have been created outside
                  // of this jenkinsfile deploy-firefox-extension.sh
                  sh """
                    docker run \
                      -v \$(pwd):/deploy \
                      -e FIREFOX_JWT_ISSUER='${FIREFOX_CREDS_USR}' \
                      -e FIREFOX_JWT_SECRET='${FIREFOX_CREDS_PSW}' \
                      -e FIREFOX_APP_ID='${FIREFOX_APP_ID}' \
                      --rm '${ECR_BROWSER_EXTENSION_BUILD}' \
                      /deploy/build/deploy-firefox-extension.sh
                  """
                  sh """
                    set -x
                    set -o errexit
                    extensionVersion="\$(jq -r .version < package.json)"
                    extensionFilename="tools-\${extensionVersion}-an+fx.xpi"

                    echo "Upload new extension \${extensionFilename} to s3 bucket ${FIREFOX_S3_BUCKET}"
                    aws s3 cp "\$(pwd)/packages/\${extensionFilename}" "s3://${FIREFOX_S3_BUCKET}"
                    aws s3api put-object-acl --bucket "${FIREFOX_S3_BUCKET}" --key "\${extensionFilename}" --acl public-read
                    # url is https://tools.s3.eu-west-1.amazonaws.com/tools-2.5.6-an%2Bfx.xpi

                    echo "Upload new version as current version"
                    aws s3 cp "\$(pwd)/packages/\${extensionFilename}" "s3://${FIREFOX_S3_BUCKET}/tools-an+fx.xpi"
                    aws s3api put-object-acl --bucket "${FIREFOX_S3_BUCKET}" --key "tools-an+fx.xpi" --acl public-read
                    # url is https://tools.s3.eu-west-1.amazonaws.com/tools-an%2Bfx.xpi

                    echo "Upload updates.json file"
                    aws s3 cp "\$(pwd)/packages/updates.json" "s3://${FIREFOX_S3_BUCKET}"
                    aws s3api put-object-acl --bucket "${FIREFOX_S3_BUCKET}" --key "updates.json" --acl public-read
                    # url is https://tools.s3.eu-west-1.amazonaws.com/updates.json
                  """
                  git.updateGithubCommitStatus("FIREFOX_DEPLOYED")
                }
              }
            }
          }
        }
      }
    }
  }
  post {
    always {
      script {
        archiveArtifacts artifacts: 'report/mega-linter.log'
        archiveArtifacts artifacts: 'report/linters_logs/*'
        archiveArtifacts artifacts: 'packages/*', fingerprint: true, allowEmptyArchive: true
        // send email to the builder and culprits of the current commit
        // culprits are the committers since the last commit successfully built
        mail.sendConditionalEmail()
        git.updateConditionalGithubCommitStatus()
      }
    }
    success {
      script {
        if (params.targetStore != 'none' && env.GIT_BRANCH == 'origin/master') {
          // send an email to a teams channel so every collaborators knows
          // when a production ready extension has been deployed
          mail.sendSuccessfulEmail('teamsChannelId.onmicrosoft.com@amer.teams.ms')
        }
      }
    }
  }
}

9 - Annotated Jenkinsfiles - Part 5

Detailed Jenkinsfile examples with annotations

1. introduction

In jenkins library you can create your own directive that allows to generate jenkinsfile code. Here we will use this feature to generate a complete Jenkinsfile.

2. Annotated Jenkinsfile

library identifier: 'jenkins_library@v1.0',
  retriever: modernSCM([
      $class: 'GitSCMSource',
      remote: 'git@github.com:fchastanet/jenkins-library.git',
      credentialsId: 'jenkinsCredentialsId'
  ])

djangoApiPipeline repoUrl: 'git@github.com:fchastanet/django_api_project.git',
                  imageName: 'django_api'

3. Annotated library custom directive

In the jenkins library just add a file named vars/djangoApiPipeline.groovy with the following content

#!/usr/bin/env groovy

def call(Map args) {
  // content of your pipeline
}

4. Annotated library custom directive djangoApiPipeline.groovy

#!/usr/bin/env groovy

def call(Map args) {

  def gitUtil = new Git(this)
  def mailUtil = new Mail(this)
  def dockerUtil = new Docker(this)
  def kubernetesUtil = new Kubernetes(this)
  def testUtil = new Tests(this)

  String workerLabelNonProd = args?.workerLabelNonProd ?: 'eks-nonprod'
  String workerLabelProd = args?.workerLabelProd ?: 'docker-ubuntu-prod-eks'
  String awsRegionNonProd = workerLabelNonProd == 'eks-nonprod' ? 'us-east-1' : 'eu-west-1'
  String awsRegionProd = 'eu-central-1'
  String regionName = params.targetEnv == 'prod' ? awsRegionProd : awsRegionNonProd
  String teamsEmail = args?.teamsEmail ?: 'teamsChannel.onmicrosoft.com@amer.teams.ms'
  String helmDirectory = args?.helmDirectory ?: './helm'
  Boolean sendCortexMetrics = args?.sendCortexMetrics ?: false
  Boolean skipTests = args?.skipTests ?: false
  List environments = args?.environments ?: ['none', 'qa', 'prod']
  Short skipBuild = 0

  pipeline {
    agent {
      node {
        label params.targetEnv == 'prod' ? workerLabelProd : workerLabelNonProd
      }
    }

    parameters {
      gitParameter branchFilter: 'origin/(.*)',
                    defaultValue: 'main',
                    quickFilterEnabled: true,
                    sortMode: 'ASCENDING_SMART',
                    name: 'BRANCH',
                    type: 'PT_BRANCH'

      choice (
        name: 'targetEnv',
        choices: environments,
        description: 'Where it should be deployed to? (Default: none - No deploy)'
      )

      string (
        name: 'instance',
        defaultValue: '1',
        description: '''The instance ID to define which QA instance it should
        be deployed to (Will only apply if targetEnv is qa). Default is 1 for
        CK and 01 for Darwin'''
      )

      booleanParam(
        name: 'suspendCron',
        defaultValue: true,
        description: 'Suspend cron jobs scheduling'
      )

      choice (
        name: 'upStreamImage',
        choices: ['latest', 'beta'],
        description: '''Select beta to check if your build works with the
        future version of the upstream image'''
      )
    }

    stages {
      stage('Checkout from SCM') {
        steps {
          script {
            echo "Checking out from origin/${BRANCH} branch"
            gitUtil.branchCheckout(
              '',
              'babee6c1-14fe-4d90-9da0-ffa7068c69af',
              args.repoUrl,
              '${BRANCH}'
            )
            wrap([$class: 'BuildUser']) {
              def String displayName = "#${currentBuild.number}_${BRANCH}_${BUILD_USER}_${targetEnv}"

              if (params.targetEnv == 'qa' || params.targetEnv == 'qe') {
                displayName = "${displayName}_${instance}"
              }

              currentBuild.displayName = displayName
            }

            env.imageName = env.BUILD_TAG.toLowerCase()
            env.buildDirectory = args?.buildDirectory ?
              args.buildDirectory + "/" : ""
            env.runCoverage = args?.runCoverage
            env.shortSha = gitUtil.getShortCommitSha(env.GIT_BRANCH)
            skipBuild = dockerUtil.checkImage(args.imageName, shortSha)
          }
        }
      }

      stage('Build') {
        when {
          expression { return skipBuild != 0 }
        }
        steps {
          script {
            String registryUrl = 'dockerRegistryId.dkr.ecr.' +
              awsRegionNonProd + '.amazonaws.com'
            String buildDirectory = args?.buildDirectory ?: pwd()

            if (params.targetEnv == "prod") {
              registryUrl = 'dockerRegistryId.dkr.ecr.' + awsRegionProd + '.amazonaws.com'
            }

            dockerUtil.pullBuildImage(
              registryImageUrl: "${registryUrl}/${args.imageName}",
              pullTags: [
                "${params.targetEnv}"
              ],
              buildDirectory: "${buildDirectory}",
              buildArgs: "--build-arg UPSTREAM_VERSION=${params.upStreamImage}",
              tagPrefix: "${env.imageName}:",
              tags: [
                "${env.shortSha}"
              ]
            )
          }
        }
      }

      stage('Test') {
        when {
          expression { return skipBuild != 0 && skipTests == false }
        }
        steps {
          script {
            testUtil.execTests(args.imageName)
          }
        }
      }
      stage('Push') {
        when {
          expression { return params.targetEnv != 'none' }
        }
        steps {
          script {
            //pipeline execution starting time for CD part
            Map argsMap = [:]

            if (params.targetEnv == "prod") {
              registryUrl = 'registryIdProd.dkr.ecr.' +
                awsRegionProd + '.amazonaws.com'
            } else {
              registryUrl = 'registryIdNonProd.dkr.ecr.' +
                awsRegionNonProd + '.amazonaws.com'
            }

            argsMap = [
              registryImageUrl: "${registryUrl}/${args.imageName}",
              pullTags: [
                "${env.shortSha}",
              ],
              tagPrefix: "${registryUrl}/${args.imageName}:",
              localTagName: "${env.shortSha}",
              tags: [
                "${params.targetEnv}"
              ]
            ]

            if (skipBuild == 0) {
              dockerUtil.promoteTag(argsMap)
            } else {
              argsMap.remove("pullTags")
              argsMap.put("tagPrefix", "${env.imageName}:")
              argsMap.put("tags", ["${env.shortSha}","${params.targetEnv}"])
              dockerUtil.tagPushImage(argsMap)
            }
          }
        }
      }
      stage("Deploy to Kubernetes") {
        when {
          expression { return params.targetEnv != 'none' }
        }
        steps {
          script {
            if (params.targetEnv == 'prod') {
              // not sure it is a good practice as it forces the operator to
              // wait for build to reach this stage
              timeout(time: 300, unit: "SECONDS") {
                input(
                  message: """Do you want go ahead with ${env.shortSha}
                  image tag for prod helm deploy?""",
                  ok: 'Yes'
                )
              }
            }
            CHART_NAME = (args.imageName).contains("_") ?
              (args.imageName).replaceAll("_", "-") :
              (args.imageName)
            if (params.targetEnv == 'qa' || params.targetEnv == 'qe') {
              helmValueFilePath = "${helmDirectory}" +
                "/value_files/values-" + params.targetEnv +
                params.instance + ".yaml"
              NAMESPACE = "${CHART_NAME}-" + params.targetEnv + params.instance
            } else {
              helmValueFilePath = "${helmDirectory}" +
                "/value_files/values-" + params.targetEnv + ".yaml"
              NAMESPACE = "${CHART_NAME}-" + params.targetEnv
            }
            ingressUrl = kubernetesUtil.getIngressUrl(helmValueFilePath)
            echo "Deploying into k8s.."
            echo "Helm release: ${CHART_NAME}"
            echo "Target env: ${params.targetEnv}"
            echo "Url: ${ingressUrl}"
            echo "K8s namespace: ${NAMESPACE}"
            kubernetesUtil.deployHelmChart(
              chartName: CHART_NAME,
              nameSpace: NAMESPACE,
              imageTag: "${env.shortSha}",
              helmDirectory: "${helmDirectory}",
              helmValueFilePath: helmValueFilePath
            )
          }
        }
      }
    }
    post {
      always {
        script {
          gitUtil.updateGithubCommitStatus("${currentBuild.currentResult}", "${env.WORKSPACE}")
          mailUtil.sendConditionalEmail()
          if (params.targetEnv == 'prod') {
              mailUtil.sendTeamsNotification(teamsEmail)
          }
        }
      }
    }
  }
}

5. Final thoughts about this technique

This technique is really useful when you have a lot of similar projects reusing over and over the same pipeline. It allows:

  • code reuse
  • avoid duplicated code
  • easier maintenance

However it has the following drawbacks:

  • some projects using this generic pipeline could have specific needs
    • eg 1: not the same way to run unit tests, to overcome that issue the method testUtil.execTests is used allowing to run a specific sh file if it exists
    • eg 2: more complex way to launch docker environment
  • be careful, when you upgrade this jenkinsfile as all the projects using it will be upgraded at once
    • it could be seen as an advantage, but it is also a big risk as it could impact all the prod environment at once
    • to overcome that issue I suggest to use library versioning when using the jenkins library in your project pipeline Eg: check Annotated Jenkinsfile @v1.0 when cloning library project
  • I highly suggest to use a unit test framework of the library to avoid at most bad surprises

In conclusion, I’m still not sure it is a best practice to generate pipelines like this.

10 - Jenkins Recipes and Tips

Useful recipes and tips for Jenkins and Jenkinsfiles

1. Jenkins snippet generator

Use jenkins snippet generator by adding /pipeline-syntax/ to your jenkins pipeline. to allow you to generate jenkins pipeline code easily with inline doc. It also list the available variables.

jenkins snippet generator

2. Declarative pipeline allows you to restart a build from a given stage

restart from stage

3. Replay a pipeline

Replaying a pipeline allows you to update your jenkinsfile before replaying the pipeline, easier debugging !

replay a pipeline

4. VS code Jenkinsfile validation

Please follow this documentation enable jenkins pipeline linter in vscode

5. How to chain pipelines ?

Simply use the build directive followed by the name of the build to launch

build 'OtherBuild'

6. Viewing pipelines hierarchy

The downstream-buildview plugin allows to view the full chain of dependent builds.

Jenkins Downstream Build Pipeline Visualization