ACPI PCC probe failed

I had one problem getting Ubuntu 14.04 to install on a Dell Inspiron 5000 series. Booting from the USB was no problem – hit F12 at startup to go to boot menu, and chose to boot from the USB drive. I didn’t have to disable Secure Boot or UEFI mode.

However, after selecting “Try Ubuntu”, all went blank. When I tried a second time in Legacy boot mode, I got a more helpful error message:

ACPI PCC probe failed

A quick google suggested this was a problem introduced Ubuntu 14.04.3. Some scary sounding fixes were suggested, but I took the easy option of downloading the 14.04.1 iso instead, which installed fine.

27

What I’m reading

The more time I spend working in Software Development, the more I realise how little I knew before. I remember going on a Skills Matter Spring course about 10 years ago, and realising that until that point I wasn’t really getting OO. (To excuse myself, I was self taught and relatively new to Java, having previously hacked around in ASP amongst other things).

I am a fan of software books and recently I have been re-reading a couple of old favourites – hence the new Recommended Reading page.

I’ve also realised there are plenty of classics I haven’t read yet, so it’s time to work my way through a few of these. This Recommended Reading for Developers looks like a fairly comprehensive list, so I’m starting at number one. See you in 960 pages…

14

Reading the timestamp data from the hibernate envers revinfo table

A short blog post in case this helps anyone!

We’re using Hibernate Envers to keep an audit trail of data changes. I thought I’d have a look into the (Postgres) database to get some stats on usage – how many records of a certain type we’re creating by month for example.

The “data_aud” tables have a “rev” field which joins to the “revinfo” table, which looks like this:

             Table "public.revinfo"
  Column   |          Type          | Modifiers 
-----------+------------------------+-----------
 id        | integer                | not null
 version   | bigint                 | not null
 timestamp | bigint                 | not null
 user_name | character varying(255) | not null

I was surprised to see that the timestamp was of type bigint – the DDL was generated by Envers, not us!

Attempting to get realistic dates out of the field using Postgres functions such as 

to_timestamp(revinfo.timestamp)

failed dismally.

I found this post which gives a working answer for Oracle: SQL query to get a human readable display of the *number* REVINFO.timestamp ?

With a bit of tweaking, here is a useful Postgres view which I created to make future querying a bit easier: 

select id, version, user_name, timestamp,
  to_char(
    TO_DATE('01/01/1970 00:00:00','DD/MM/YYYY HH24:MI:SS')
    + (r.timestamp /1000/60/60/24 ||' day')::interval, 'YYYY/MM/DD'
  ) as rev_date, 
  to_char(
    TO_DATE('01/01/1970 00:00:00','DD/MM/YYYY HH24:MI:SS') 
    + (r.timestamp /1000/60/60/24 ||' day')::interval, 'YYYY/MM'
  ) as rev_month 
from revinfo r;
59

Gradle and the “Parent Pom”

Coincidentally, something came up I wanted to blog about, and it turns out it’s a follow up to the last blog post I wrote 9 months ago.

I’ve been increasingly unhappy with using “apply from” to apply a script plugin from a hard coded location. It’s no use if you’re offline, or not on the right VPN to access the file, or whatever. What I didn’t realise is that I could do apply all the same configuration defaults from a binary plugin. Then you get proper dependency management like you would with a Maven parent pom.

Here’s the code for the Plugin, which is just a standard gradle plugin

import org.gradle.api.Project
import org.gradle.api.Plugin
import org.apache.log4j.Logger
import org.gradle.api.GradleException

public final class CommonConfigPlugin implements Plugin<Project>{

    @Override
    void apply(Project project) {
        addDependencies(project)
        applyPlugins(project)
        configureReleasePlugin(project)
    }
    
    private void configureReleasePlugin(Project project) {

        if (!project.parent) {
            project.createReleaseTag.dependsOn { project.allprojects.uploadArchives }
        }

        //do this for all projects
        project.uploadArchives {
            ... config here ...
        }               
    }

    private void addDependencies(Project project) {
        project.afterEvaluate {            
            project.dependencies.add("testCompile", "junit:junit:4.11")
            project.dependencies.add("compile", "log4j:log4j:1.2.14")
        }
    }

    private void applyPlugins(Project project) {
        project.configure(project){
            apply plugin: 'java'
            apply plugin: 'maven'               
        }

        //apply these to parent project only
        if (!project.parent) {
            project.configure(project){
                apply plugin: 'sonar-runner'
                apply plugin: 'release'
            }
        }
    }
}

Then in the main build.gradle file for any project, I add the jar as a dependency and apply the plugin.

allprojects {
    repositories {
        mavenLocal()
        maven { url "http://my.repo"}
    }
    apply plugin: "my-common-config"

}
buildscript {

    repositories {
        mavenLocal()
        maven { url "http://my.repo"}
    }
    dependencies {
        classpath 'uk.co.anorakgirl.gradle:my-common-config-plugin:1.0.0'
    }
}

Any dependencies which the plugin has end up as dependencies in the buildscript of the project, meaning I don’t have to repeat the dependency on the release plugin, because my common config plugin already depends on it.

I’m going to put more in this common-config-plugin.

53

Gradle release ramblings

Over the last couple of weeks I had to get up to speed on gradle. We got inspired to try out Gradle at work following an great talk on Groovy for SysAdmins by Dan Woods at the Groovy & Grails eXchange 2013, during which Groovy and Gradle were used as a mechanism for building and deploying VMs.

First off, I seem to love Gradle! The build.gradle files are so slimline and readable. The challenges for the last couple of weeks were:

  • How to manage releases in a similar way to maven
  • How to include common elements in our builds (parent pom functionality

Gradle Release plugin

We decided to use the townsfolk gradle-release plugin as it seems to do pretty much what Maven release has been doing for us: check for snapshot dependencies, check version control is up to date, set version, build, tag, set next snapshot and commit.

To use the plugin, the dependency has to be made available to the build.gradle file itself. Standard dependencies are only available to the project being built. This is where the buildscript block comes in – dependencies declared here can be used in the groovy script itself. 

buildscript{
    repositories {
        mavenCentral()
        maven { url "https://oss.sonatype.org/content/groups/public"}
    }
    dependencies {
        classpath 'com.github.townsfolk:gradle-release:1.2'
    }
}

The one thing that the plugin does not do is upload artifacts to a maven repository, but this is easy as Gradle allows you to declare dependencies between tasks. We add a dependency on the uploadArchives task, which comes from the maven plugin:

createReleaseTag.dependsOn uploadArchives

A couple of gotchas here. When using this with a multi-module project, you can only apply the plugin to the parent project, and all modules have to have the same version. We found initially that only the parent project was getting deployed to artifactory. The task dependency must be modified as follows for multi-module projects:

createReleaseTag.dependsOn allprojects.uploadArchives

Extracting common gradle elements to use across projects

The parent pom we use for maven projects has common plugin configurations, standard dependencies and so on. We decided to try and create a similar “parent” gradle file, to use across all our projects. The apply from syntax can be used to configure the project with an external script:

apply from: 'https://artifactory.url/../gradle-common-1.0.gradle'

Notice that the apply from is coming from a maven repository. We used the release plugin described above to release a common gradle file as a maven artifact, so this can managed following all our standard release practices too!

The common gradle file can contain all of the gradle build language, including applying other scripts. One thing it cannot contain is a buildscript block. To include common configuration in a buildscript block, you must create a second common file, containing a project.buildscript block:

project.buildscript {
    repositories {
        mavenCentral()
        maven { url "https://oss.sonatype.org/content/groups/public"}
    }
    dependencies {
        classpath 'com.github.townsfolk:gradle-release:1.2'
    }
}

Within the project, apply this script within the project buildscript block:

buildscript {
	apply from: 'https://artifactory.url/../gradle-common-buildscript-1.0.gradle'
}

This seems like doubling up to me, but is how it appears to work. It means we have two common gradle files to apply separately.

One final gotcha I found when moving the release plugin configuration into our common gradle files – when I tried to configure the task dependency in the external script, I got this error when building multi-module projects:

Could not find property 'uploadArchives' on project ':submodule'.

This seems to be because the parent is trying to configure the dependency on the subproject uploadArchives task, but the subproject does not yet have that task. It must be to do with the order in which gradle applies external scripts? Anyway, the solution was to delay evaluation of the dependency (those clever gradle folk think of everything!). Putting the dependency in a closure does this (Found this at #3 here 12-new-things-i-learned-from-a-three-day-gradle-training. So in the main common gradle file I have these lines, which apply the release plugin to the parent project only and create a lazy dependency on the uploadArchives task for all modules.

if (!project.parent) {
	apply plugin: 'release'
	project.createReleaseTag.dependsOn { allprojects.uploadArchives }
}

And that’s it, infrastructure in place!

(Credit for setting a lot of this up goes to JCDubs but I was so excited when I started working with it I had to write up what we learned on the way. I’m looking forward to lots more XML free building…)

50

My First Groovy DSL

I decided to have a go at automating the management of our Jenkins jobs. Basically most of our projects have similar builds, and I want to keep the job configs standard – they tend to diverge when we manage them via the UI. I’ve previously played with the Jenkins API using HttpBuilder to post XML config files. My DSL doesn’t do anything clever to generate the XML, but it makes the Jenkins config very readable:

jenkins {   
    url = "http://jenkins.url/"
    username = "jenkins"
    apiKey = "asdfadsgdsfshgdsfg"

    buildGroup {
        name = "Maven Builds"
        xml {
            feature = "release/xml/maven-test-config.xml"
            develop = "release/xml/maven-install-config.xml"
        }
        repos {
            project1 = "ssh://git.repo/project1.git"
            project2 = "ssh://git.repo/project2.git"
        }
    }

    buildGroup {
        name = "Grails Builds"
        xml {
            feature = "release/xml/grails-test-config.xml"
            develop = "release/xml/grails-install-config.xml"
        }
        repos {
            project4 = "ssh://git.repo/project4.git"
            project5 = "ssh://git.repo/project5.git"
        }
    }

}

I think this is so easy to read.

I’m not really sure if I’ve gone the right way about writing this, but here goes:

First, I have a main method, which parses the file. It binds the “jenkins” method to an instance of my JenkinsDsl class.

    Binding binding = new Binding();
    JenkinsDsl jenkinsDsl = new JenkinsDsl();
    binding.setVariable("jenkins",jenkinsDsl);
    GroovyShell shell = new GroovyShell(binding);
    Script script = shell.parse(file);
    script.run();

The JenkinsDsl class overrides the `call` method, and uses `methodMissing` to define how we handle the buildGroups. The really cool bit is I didn’t have to write anything special to get the url, username and password from the jenkins config file – groovy is automatically calling a ‘setProperty’ method on the delegate.

class JenkinsDsl {

    def call(Closure cl) {
        log.info "Processing Main Configuration"

        cl.setDelegate(this);
        cl.setResolveStrategy(Closure.DELEGATE_ONLY)
        cl.call();
    }

    def url
    def apiKey
    def username

    def jenkinsHttp

    def methodMissing(String methodName, args) {

        if (methodName.equals("buildGroup")) {

            Closure closure = args[0]
            closure.setDelegate(new JenkinsBuildGroup())
            closure.setResolveStrategy(Closure.DELEGATE_ONLY)
            closure()
            closure.delegate.updateJenkins(getJenkinsHttp())
        } else {
            log.error "Unsupported option: " + methodName
        }
    }

    def getJenkinsHttp() {
        if (!jenkinsHttp) {
            jenkinsHttp = new JenkinsHttp(url, username, apiKey)
        }
        jenkinsHttp
    }   
}

The JenkinsBuildGroup class is very similar – to save time I’ve used the ‘Expando’ class to collect the xml and repository details. 

class JenkinsBuildGroup {

    def name
    def xml
    def repoList

    /**
     * Use methodMissing to load the xml and repos closures
     * @param methodName
     * @param args
     */
    def methodMissing(String methodName, args) {

        if (methodName.equals("xml")) {
            xml = new Expando()
            Closure closure = args[0]
            closure.setDelegate(xml)
            closure.setResolveStrategy(Closure.DELEGATE_ONLY)
            closure()
        } else if (methodName.equals("repos")) {
            repoList = new Expando()
            Closure closure = args[0]
            closure.setDelegate(repoList)
            closure.setResolveStrategy(Closure.DELEGATE_ONLY)
            closure()
        } else {
            log.error "Unsupported option: " + methodName
        }
    }

    def updateJenkins(def jenkinsHttp) {
        xml.getProperties().each { String jobType, String filePath ->

            def configXml = new File(filePath).text

            repoList.getProperties().each { String jobName, String repo ->
                String jobXml = configXml.replaceAll("REPOSITORY_URL", repo)
                jenkinsHttp.createOrUpdate(jobName + "-" + jobType, jobXml)
            }
        }
    }
}

To actually update jenkins, I substitute the correct repository in the template XML file and post it to the Jenkins API. The JenkinsHttp class is just a wrapper for HTTP Builder. It checks if a job exists so that it can correctly create or update.

I’m not sure if this is the right way to go about writing a groovy DSL, but it works! And once I’ve finalised the template XML files, Jenkins is going to be a vision of consistency.

67

Continuous Build Woes

I’ve not been happy with our Jenkins continuous build setup for a while. It was fine back in the day when: we had about 3 projects; we were still using Subversion(!); and we didn’t work on branches. We now have lots of projects, we use git, we follow a git flow based branching strategy, and we use Atlassian Stash to review pull requests before merging. I’ve tweaked the Jenkins config a bit, and we wrote a Stash plugin to prompt builds on commit, but it’s become harder and harder to maintain to the point that we’re not really using it, so deploying is never as easy as it should be, and test failures get where they shouldn’t.

Here are my key requirements; I am sure they aren’t that unusual.

1. Job Templates. We have various types of project (Maven, Gradle, Grails) but for each type, the build process is pretty much the same, with a different repository URL. What I want to do is create a template for a certain type of project, and then create jobs using that template. If I want to change the build process, I just change the template and all the jobs should change.

2. Stash Integration. I want to prevent merging to develop or master if there are failing tests.

3. Branch specific configuration. At the moment we have one job for building develop and feature branches, and a second for releasing from master. I’d like all builds associated with a project to be configured in the same place, with some conditional behaviour based on the branch. Feature branches just get compiled and tested. Develop branch also gets deployed to a SNAPSHOT release (maven), or a staging environment (grails). Master branch gets released.

Bamboo Evaluation

We use Atlassian Jira, Confluence and Stash (all of which I think are brilliant!) so my first thought was to move to Bamboo. I spent yesterday setting up an evaluation and trying out a few builds.

First off, it looks great – nice familiar UI, well laid out etc. However, it wasn’t as quick to get started as I’d hoped. Here’s my initial thoughts:

1. Too complicated! Each plan (which equates to a source code repository) can have multiple sequential Stages, containing multiple Jobs (which can run in parallel), each containing sequential Tasks. This might be great if you have a big and complex build process, but all our projects are pretty straightforward, so we’d always be using just the “Default Stage”, leaving lots of navigation with not much in it.

2. No instant Stash Integration. I sort of expected to be provided with some sort of “Add builds from Stash” function, which would automatically set up build triggers from Stash, and successful build notifications back again. But it seems I have to install a plugin on Stash, and for each repository enable it and paste in the notification URL for the correct plan.

3. No templating or hierarchical configuration. Configuring and maintaining lots of identical jobs for different repositories is going to be just as time consuming as in Jenkins.

4. No conditional Branch configuration. Bamboo offers “Branch Plans” so your plan can run on multiple branches (although I couldn’t seem to get it to pick up branches automatically as described). However, there doesn’t seem to be a way to conditionally run stages (or jobs/tasks) based on the branch. So it looks like I’d still have to have separate jobs for building develop (including deploying to staging) and feature branches (test only).

Back to Jenkins

So, I’m a bit disappointed that Bamboo doesn’t seem to solve the main problems I’m facing with Jenkins. It looks like I might as well spend some time working out how to manage Jenkins a little better (as I’m a lot more familiar with it, and we already have it).

I’ve already solved one of the problems, as I’ve found the Jenkins Stash Notifier plugin. And I’m working on some scripts to use the Jenkins rest api to manage jobs. I’ll post an update if I get to a solution I’m happy with!

36

Groovy Grails Exchange 2013

This week I attended the 2013 Groovy Grails Exchange, organised by Skills Matter (with whom I’ve also been lucky enough to attend a couple of courses over the years).

Finally got time to jot down a few thoughts – I came back with an almost full notebook, but these are the highlights:

1. What a great conference! Skills Matter did an amazing job of hosting. The speakers were varied and interesting. It was fantastically motivating to be amongst all these people who are actually pushing the technology forwards. As I work in a very small dev team (out in the sticks) I found it really interesting to see what goes on in the rest of the world.

2. Must. Learn. Spock.

3. Must also get back on board with Geb. I dabbled a little a while ago, and now that we’re about to start a new Grails project, functional testing with Spock & Geb has to be central to it. Lots of food for thought in the DevQA talk by Alvaro Sanchez-Mariscal, applicable even though we don’t have a dedicated QA team for our internal dev.

4. Forces on code – what makes code “good” depends on the context. Common sense really but well illustrated in the talk by by David the Coder

5. Do I need to learn a JavaScript framework? I enjoyed the very persuasive talk on Developing SPI Applications by Alvaro Sanchez-Mariscal. Separating the front end and back end into independent apps makes a lot of sense especially if you have dedicated UI developers. I’m not sure we have the resources to move away from GSPs just yet. But I’m certainly going to have a browse around ToDoMVC to get an idea of the options.

6. Jeff Brown’s live coding demo of Metaprogramming With The Groovy Runtime was a great refresher – nothing particularly that hadn’t been covered on the Groovy course I attended last year, but a reminder that I am pretty guilty of just writing Java like code inside .groovy files and not taking full advantage of groovy’s awesomeness.

7. The “Open Source and You” session by Peter Ledbrook made me think a bit more deeply about Open Source software – what to expect from it, the costs involved, how to manage a successful open source project. I’m definitely motivated to get more involved.

8. Gradle for deployment? @danveloper’s talk on Groovy for Sysadmins gave me lots to think about. I doubt I’ll ever end up hacking the kernel, but I like the software centric approach to deploying and maintaining servers.

9. Must pay more attention to release notes and road maps. New and upcoming versions of Groovy and Grails have some great new features that I’m looking forward to using. Changes notes for Groovy 2.2 and Grails 2.3 definitely worth a look. Also looking forward to plugin and build system changes in Grails 3.0 some time in 2014.

10. And finally, how come all these Open Source aficionados use Macs?! I’ve not got an iAnything yet and don’t plan to. Linux rocks :)

Anyway, in summary, well worth going for anyone working with Groovy or Grails. Podcasts of all the talks are available on the Skills Matter website, but I think attending in real is a fantastic opportunity to absorb knowledge from an enthusiastic and knowledgeable crowd, and worth every penny. Better get my early bird ticket for next year…

35

Updated JIRA sub-task sorting plugin

Following some user feedback, I’ve made some changes to the JIRA plugin I wrote for dragging sub-tasks, first mentioned in this post Jira web-resource plugin to drag and drop subtasks.

If you’ve used it, you may have found it annoying that it was actually too easy to drag subtasks, causing page refreshes when they weren’t wanted! Copying and pasting from the subtask list was impossible. You could also drag issues in the Issue Navigator, although it didn’t achieve anything.

Just a few teething problems then.

First I put a delay in the jQuery sortable options. I also stopped the page from submitting if the list order had not been changed. This was a definite improvement, but users still wanted to copy text from the list, which wasn’t possible. So now I’ve limited the “drag handle” to the cell in the table where the up and down arrows appear for sorting. You can only drag if you click in that area, anywhere else won’t work. It’s still not ideal, as it’s not immediately obvious how to drag. Better ideas welcomed!

You can get the updated source from github: https://github.com/anorakgirl/subtask-dragger/releases/tag/v0.6.
Or if you don’t feel like packaging it yourself, there’s a jar for download here: subtask-dragger-0.6.jar. USE AT YOUR OWN RISK!

45

Issue Security in Jira sub-tasks and the Script Runner plugin

I’ve been trying to do some advanced JIRA configuration recently, with mixed success.

The requirement was to have issues in a project which could be viewed only by a specific user (not necessarily the assignee).

I discovered that “Issue Security Schemes” can have levels which allow access based on a “User Custom Field Value”. Fantastic. I added a custom field, let’s call it “Employee”, and added it to an Issue Security Level. Applied to the project and bingo, users can only see issues where they have been entered into the “Employee” field.

Too easy. I added a subtask to an issue where I was the employee. No error message, but no subtask appeared either.

My understanding was that you could not set issue security on subtasks, because they inherit the issue security of the parent. I assumed the intention of this was that if you could see an issue, you should be able to see its children. But no – they specifically inherit the issue security setting from the parent, rather than its visibility – as confirmed in Atlassian Answers. So because the sub-task does not have an “Employee” field, I can’t see it. Bother.

Of course, I could use a workflow post-function so that when subtasks are created, the employee field is set to that of the parent. But what if the value is changed in the parent? It’s a nuisance to have to change all the sub-tasks too.

Time to investigate the Script Runner plugin.

Here’s the code for my Scripted field, which gets the value of the Employee field in the parent issue.

import com.atlassian.jira.ComponentManager
import com.atlassian.jira.issue.CustomFieldManager
import com.atlassian.jira.issue.Issue
import com.atlassian.jira.issue.fields.CustomField

def componentManager = ComponentManager.getInstance()
def parentIssue = issue.getParentObject();
CustomFieldManager cfManager = ComponentManager.getInstance().getCustomFieldManager()


CustomField customField = cfManager.getCustomFieldObject(99999)
parentIssue.getCustomFieldValue(customField)

It is almost too easy. I think it may be the answer to some of the other requests I’ve had for custom fields in JIRA, but unfortunately it doesn’t solve this problem. The calculated value appears perfectly on the page when viewing a sub-task. But, when I go to the “Issue Security Level” and click in the “User Custom Field Value” drop down, it doesn’t appear! It’s configured with the “User Picker” Searcher and template, so I don’t think there’s much more I can do.

Back to the drawing board…

52