jenkins

My First Groovy DSL

I decided to have a go at automating the management of our Jenkins jobs. Basically most of our projects have similar builds, and I want to keep the job configs standard – they tend to diverge when we manage them via the UI. I’ve previously played with the Jenkins API using HttpBuilder to post XML config files. My DSL doesn’t do anything clever to generate the XML, but it makes the Jenkins config very readable:

view source
print?
1jenkins {  
2    url = "http://jenkins.url/"
3    username = "jenkins"
4    apiKey = "asdfadsgdsfshgdsfg"
5 
6    buildGroup {
7        name = "Maven Builds"
8        xml {
9            feature = "release/xml/maven-test-config.xml"
10            develop = "release/xml/maven-install-config.xml"
11        }
12        repos {
13            project1 = "ssh://git.repo/project1.git"
14            project2 = "ssh://git.repo/project2.git"
15        }
16    }
17 
18    buildGroup {
19        name = "Grails Builds"
20        xml {
21            feature = "release/xml/grails-test-config.xml"
22            develop = "release/xml/grails-install-config.xml"
23        }
24        repos {
25            project4 = "ssh://git.repo/project4.git"
26            project5 = "ssh://git.repo/project5.git"
27        }
28    }
29 
30}

I think this is so easy to read.

I’m not really sure if I’ve gone the right way about writing this, but here goes:

First, I have a main method, which parses the file. It binds the “jenkins” method to an instance of my JenkinsDsl class.

view source
print?
1Binding binding = new Binding();
2JenkinsDsl jenkinsDsl = new JenkinsDsl();
3binding.setVariable("jenkins",jenkinsDsl);
4GroovyShell shell = new GroovyShell(binding);
5Script script = shell.parse(file);
6script.run();

The JenkinsDsl class overrides the `call` method, and uses `methodMissing` to define how we handle the buildGroups. The really cool bit is I didn’t have to write anything special to get the url, username and password from the jenkins config file – groovy is automatically calling a ‘setProperty’ method on the delegate.

view source
print?
1class JenkinsDsl {
2 
3    def call(Closure cl) {
4        log.info "Processing Main Configuration"
5 
6        cl.setDelegate(this);
7        cl.setResolveStrategy(Closure.DELEGATE_ONLY)
8        cl.call();
9    }
10 
11    def url
12    def apiKey
13    def username
14 
15    def jenkinsHttp
16 
17    def methodMissing(String methodName, args) {
18 
19        if (methodName.equals("buildGroup")) {
20 
21            Closure closure = args[0]
22            closure.setDelegate(new JenkinsBuildGroup())
23            closure.setResolveStrategy(Closure.DELEGATE_ONLY)
24            closure()
25            closure.delegate.updateJenkins(getJenkinsHttp())
26        } else {
27            log.error "Unsupported option: " + methodName
28        }
29    }
30 
31    def getJenkinsHttp() {
32        if (!jenkinsHttp) {
33            jenkinsHttp = new JenkinsHttp(url, username, apiKey)
34        }
35        jenkinsHttp
36    }  
37}

The JenkinsBuildGroup class is very similar – to save time I’ve used the ‘Expando’ class to collect the xml and repository details.

view source
print?
1class JenkinsBuildGroup {
2 
3    def name
4    def xml
5    def repoList
6 
7    /**
8     * Use methodMissing to load the xml and repos closures
9     * @param methodName
10     * @param args
11     */
12    def methodMissing(String methodName, args) {
13 
14        if (methodName.equals("xml")) {
15            xml = new Expando()
16            Closure closure = args[0]
17            closure.setDelegate(xml)
18            closure.setResolveStrategy(Closure.DELEGATE_ONLY)
19            closure()
20        } else if (methodName.equals("repos")) {
21            repoList = new Expando()
22            Closure closure = args[0]
23            closure.setDelegate(repoList)
24            closure.setResolveStrategy(Closure.DELEGATE_ONLY)
25            closure()
26        } else {
27            log.error "Unsupported option: " + methodName
28        }
29    }
30 
31    def updateJenkins(def jenkinsHttp) {
32        xml.getProperties().each { String jobType, String filePath ->
33 
34            def configXml = new File(filePath).text
35 
36            repoList.getProperties().each { String jobName, String repo ->
37                String jobXml = configXml.replaceAll("REPOSITORY_URL", repo)
38                jenkinsHttp.createOrUpdate(jobName + "-" + jobType, jobXml)
39            }
40        }
41    }
42}

To actually update jenkins, I substitute the correct repository in the template XML file and post it to the Jenkins API. The JenkinsHttp class is just a wrapper for HTTP Builder. It checks if a job exists so that it can correctly create or update.

I’m not sure if this is the right way to go about writing a groovy DSL, but it works! And once I’ve finalised the template XML files, Jenkins is going to be a vision of consistency.

67

Continuous Build Woes

I’ve not been happy with our Jenkins continuous build setup for a while. It was fine back in the day when: we had about 3 projects; we were still using Subversion(!); and we didn’t work on branches. We now have lots of projects, we use git, we follow a git flow based branching strategy, and we use Atlassian Stash to review pull requests before merging. I’ve tweaked the Jenkins config a bit, and we wrote a Stash plugin to prompt builds on commit, but it’s become harder and harder to maintain to the point that we’re not really using it, so deploying is never as easy as it should be, and test failures get where they shouldn’t.

Here are my key requirements; I am sure they aren’t that unusual.

1. Job Templates. We have various types of project (Maven, Gradle, Grails) but for each type, the build process is pretty much the same, with a different repository URL. What I want to do is create a template for a certain type of project, and then create jobs using that template. If I want to change the build process, I just change the template and all the jobs should change.

2. Stash Integration. I want to prevent merging to develop or master if there are failing tests.

3. Branch specific configuration. At the moment we have one job for building develop and feature branches, and a second for releasing from master. I’d like all builds associated with a project to be configured in the same place, with some conditional behaviour based on the branch. Feature branches just get compiled and tested. Develop branch also gets deployed to a SNAPSHOT release (maven), or a staging environment (grails). Master branch gets released.

Bamboo Evaluation

We use Atlassian Jira, Confluence and Stash (all of which I think are brilliant!) so my first thought was to move to Bamboo. I spent yesterday setting up an evaluation and trying out a few builds.

First off, it looks great – nice familiar UI, well laid out etc. However, it wasn’t as quick to get started as I’d hoped. Here’s my initial thoughts:

1. Too complicated! Each plan (which equates to a source code repository) can have multiple sequential Stages, containing multiple Jobs (which can run in parallel), each containing sequential Tasks. This might be great if you have a big and complex build process, but all our projects are pretty straightforward, so we’d always be using just the “Default Stage”, leaving lots of navigation with not much in it.

2. No instant Stash Integration. I sort of expected to be provided with some sort of “Add builds from Stash” function, which would automatically set up build triggers from Stash, and successful build notifications back again. But it seems I have to install a plugin on Stash, and for each repository enable it and paste in the notification URL for the correct plan.

3. No templating or hierarchical configuration. Configuring and maintaining lots of identical jobs for different repositories is going to be just as time consuming as in Jenkins.

4. No conditional Branch configuration. Bamboo offers “Branch Plans” so your plan can run on multiple branches (although I couldn’t seem to get it to pick up branches automatically as described). However, there doesn’t seem to be a way to conditionally run stages (or jobs/tasks) based on the branch. So it looks like I’d still have to have separate jobs for building develop (including deploying to staging) and feature branches (test only).

Back to Jenkins

So, I’m a bit disappointed that Bamboo doesn’t seem to solve the main problems I’m facing with Jenkins. It looks like I might as well spend some time working out how to manage Jenkins a little better (as I’m a lot more familiar with it, and we already have it).

I’ve already solved one of the problems, as I’ve found the Jenkins Stash Notifier plugin. And I’m working on some scripts to use the Jenkins rest api to manage jobs. I’ll post an update if I get to a solution I’m happy with!

36

Don’t commit your target folder

or an insidious error which resulted in jenkins builds where the sources jar did not match the binary jar

I still can’t work out exactly how this works. I recently made some changes to one of our Maven archetypes, and released the new version using Jenkins. I was confused to discover that projects created with the new version of the archetype did not contain the changes. I checked the git repository tag. I checked the source jar in Artifactory. Both contained the changes. I checked the binary jar and it did not (the changes were to resource files).

I checked out locally, ran maven clean and then maven package. The resulting binary jar was fine.

I wiped out the Jenkins workspace and repeated the build. The binary jar was created incorrectly! Cue lots of head scratching, until a quick thinking colleague asked if I’d checked in the target directory at any point. And I HAD! So although I’d wiped the jenkins workspace, the fresh checkout had got back the out of date target contents. AND the jenkins job did not begin with a maven clean.

A lesson in better house-keeping.

34

Grails unit tests failing on jenkins

I spent a while confused today, when some fairly straightforward unit tests were running fine locally, but failing on our continuous integration server, jenkins.

The error was occurring in the setUp method, annotated @Before, where we were setting a flash variable prior to running tests.

view source
print?
1java.lang.NullPointerException: Cannot invoke method getFlashScope() on null object
2at grails.test.mixin.web.ControllerUnitTestMixin.getFlash(ControllerUnitTestMixin.groovy:159)

After much head scratching I came across this link
http://www.mopri.de/2013/grails-unit-testing-and-a-little-fun-with-before/
which explains a similar problem.

It appears that because the Controller uses a Mixin, it effectively extends ControllerUnitTestMixin which is not abstract and also has a method annotated with @Before. Junit does not guarantee the order in which the two methods annotated with @Before will be run (See http://junit.sourceforge.net/doc/faq/faq.htm#tests_2)

If the Mixin method does not run before my setUp method, the flash scope has not been mocked, hence the failure.

The solution suggested in the post is to call super.bindGrailsWebRequest() in the setUp method. In our case it was simple enough to set the flash variable within the individual tests, instead of the setUp method.

31