Internal Errigal Collaboration Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Jenkins 2 ======
===== Introduction =====

Jenkins 2 can be accessed at: [[https://jenkins.errigal.com:8443//|https://jenkins.errigal.com:8443/]] (password for admin ins in pwSafe).

It is encouraged to create new projects using the Pipeline plugin of Jenkins 2 and with Jenkinsfile being part of the project SCM.
It is suggested to get yourself familiar with the Pipeline plugin (e.g. using this [[https://jenkins.io/solutions/pipeline/|introduction]].
There are also some best practises to be followed as described [[https://www.cloudbees.com/blog/top-10-best-practices-jenkins-pipeline-plugin|here]].

The main benefits of using Jenkins Pipelines:
  * The complete definition of Jenkins build is part of SCM (as Jenkinsfile)
  * Jenkinsfile is versioned and reflects particular branches
  * It allows splitting the build into several stages, which allows pipelining (once a stage is finished, later build can perform that stage)

===== Integration to BitBucket =====

The installed Jenkins has prepared credentials for accessing the BitBucket repositories.

The credentials have 'bitbucket' id and can be administered at [[http://10.91.100.106:8080/credentials/store/system/domain/_/|Global credentials (unrestricted)]].

To create password for Jenkins (or other application) go to https://bitbucket.org/account/user/**<user-name>**/app-passwords and create a new App Password (with desired Authorization). 

{{:toolsandtechnologies:bitbucket-robot-app-password.jpg?400|}}

It is crucial to add at least read/write access to repositories in order to make cloning and tagging work.

**errigal_robot** account should be used for all builds, and NOT personal accounts due to the issue of account closure in Bitbucket directly affecting the build process

Example below shows how to use these credentials in order to clone a repo and tag it with a build number.

===== Sample Pipeline =====
Following code represents very early and basic Pipeline of the Fiber project (http://10.91.100.112:8080/job/fiber_component/):. Another sample of our Jenkins pipeline can be found in the Content Distributor project (https://bitbucket.org/errigal/content-distributor). The pipeline code can be more easily tested by using the Replay functionality (it is accessible as a button in each particular build) - this is better than pushing changes to Jenkinsfile to resolve/debug it.
<code groovy>
#!groovy
// Define versioning
versionNumber = "0.1.0." + currentBuild.number
currentBuild.displayName = versionNumber
// ID of the Jenkins credentials for interaction with BitBucket
bitbucketCredentialsId = 'bitbucket'
// The hostname/path part of the git URL
repoLocation = 'bitbucket.org/errigal/fiber_component.git'

node() {
    stage "Checkout"
    // Checkout the GIT using provided URL and credentials
    git credentialsId: bitbucketCredentialsId, url: "https://$repoLocation"

    stage "Build/Analyse/Test"
    // Run the Gradle wrapper to clean, build and test the code
    sh "./gradlew clean build"
    // Archive test result so they are browsable from Jenkins
    step([$class: 'JUnitResultArchiver', testResults: '**/build/test-results/test/TEST-*.xml'])

    stage "Tag build in Git"
    // Use credentials to create versionNumber tag on current commit and push it back to BitBucket
    withCredentials([[$class: 'UsernamePasswordMultiBinding', credentialsId: bitbucketCredentialsId, usernameVariable: 'GIT_USERNAME', passwordVariable: 'GIT_PASSWORD']]) {
        sh("git tag -a $versionNumber -m 'Jenkins'")
        sh("git push https://${env.GIT_USERNAME}:${env.GIT_PASSWORD}@$repoLocation --tags")
    }
}
</code>

The resulting Jenkins job looks following:

{{:toolsandtechnologies:screen_shot_2017-03-27_at_12.16.37.png?1024|}}

===== How to Install on Ubuntu =====  
  * Ensure java 8 or later is the defaulf java of the system, if not install it  - note it will run with java 7, not java 6 but SonarQube needs Java 8
  * java -version
  * install Jenkins
  * wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
  * sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
  * sudo apt-get update
  * sudo apt-get install jenkins

===== Enabling Email Notification =====  
  * Go to Jenkins, Manager Jenins, Configure System
  * Go to System Admin e-mail address and make it - SERVER/BUILDERNAME <errigaldev@gmail.com>
  * Go to E-mail Notification, click advanced
  * Make it look like :

{{ :toolsandtechnologies:screenshot_2017-05-12_09.51.46.png |}}


===== Extra Plugins =====  
On top of the default Jenkins load, the following plugins need to be installed
  * Ansible plugin
  * AntExec
  * Bitbucket OAuth Plugin
  * Bitbucket Plugin
  * Hudson SCP publisher plugin
  * JaCoCo plugin
  * SonarQube Scanner for Jenkins
  * JIRA Plugin
  * Discard Old Build
  * Build Keeper Plugin
  * Database Plugin
  * Database MySQL plugin
  * Pipeline Utility Plugin
  * Copy Artifact Plugin
  * Run Selector Plugin
  * Slack Notification Plugin
  * ThinBackup plugin
  * Parameterized Scheduler
  * Gatling Plugin


===== Setting up SonarQube ===== 

Go to Manage Jenkins, Configure System and make SonarQube servers look like :
{{ :toolsandtechnologies:screenshot_2017-05-12_12.32.23.png |}}

===== Automatic build artefacts discarding and keeping important builds ===== 
Following snippet (using Declarative Jenkins Pipeline) shows how to setup automatic build discarding (this one is setup to keep only last 20 builds) - Discard Old Build plugin is needed for this. In case the build is important and you want to keep it, there is a button in each build (blue, on the right side), which allows keeping that build regardless of the discarding plugin - this needs the Build Keeper plugin to be installed.

<code groovy>
pipeline {
  agent any
  options {
    // Skip the default SCM Checkout Stage - we are doing our own checkout
    skipDefaultCheckout(true)
    // Keep build artifacts and logs only for last 20 builds
    buildDiscarder(logRotator(numToKeepStr:'20'))
  }
</code>

===== Artifactory storage ===== 

Artifactory configuration is covered [[development:artifactory]]

Each pipeline has a step to push the build suite (war/jar, pom, XML etc) to the artifactory store in  [[http://artifactory2.err:4041/#browse/browse:maven-candidates|Maven Candidate Storage]]. Each application has its own folder, and inside each folder there is a branch folder. These folders are either IDMS, AUTO, Support or Rel- based.

Ideally (as of June 6th, 2023 not working) we should keep

   *- 15 days of IDMS & AUTO 
   *- 30 days of SUPPORT
   *- Last 4 REL-

Deployment is covered via
<code>
 stage ('Deploying to candidates repo') {
            steps {
                nexusPublisher(
                        nexusInstanceId: 'nexusArtifactory2',
                        nexusRepositoryId: 'maven-candidates',
                        packages: [
                                [$class: 'MavenPackage',
                                 mavenAssetList: [
                                         [classifier: '', extension: 'jar', filePath: 'build/libs/elastic-replicator.jar']
                                 ],
                                 mavenCoordinate: [artifactId: 'elastic-replicator', groupId: 'com.errigal', packaging: 'jar', version: versionNumber]
                                ]
                        ]
                )
            }
        }
</code>

The deployment playbooks should then source the artefact from the repo and not from our dev jenkins with a public IP interface (security risk).





===== Generating API token (e.g. for Ansible) =====
The tokens used in API (e.g. in ansible role artefact-deployment) are generated per Jenkins user so you have to navigate to the currently logged in user (usually top right corner of the Jenkins page). Or alternatively you can go to URL like http://calliope.err:8080/user/admin/configure (this is on calliope for user admin).
There is a section **API Token**, which allows you to view/create tokens. This token then needs to be used in the API (check the role artefact-deployment in deployment-playbooks for an example, the token goes to default/main.yml for that particular role).

===== General Troubleshooting =====
==== Immediately stopping a stuck build ====
"Manage Jenkins" => "Script Console"

Script (adjust values accordingly):
<code groovy>
Jenkins.instance.getView('All').getBuilds().findAll() { it.getResult().equals(null) }.each {
  if (it.toString() == 'SnmpManager/snmpmanager-feature-branches/dev #9113') {
   it.doKill()
  }
}
</code>
Sidebar

Internal Errigal Collaboration Wiki

User Tools

Site Tools

Page Tools
