December 17, 2016December 17, 2016

Docker and Maven for integration testing

I recently implemented LDAP authentication in our Dropwizard application. How to integration test this?

Here’s what I used:

* This docker image: nickstenning/slapd, containing a basic configuration of the OpenLDAP server.
* The fabric8io docker-maven-plugin
* Some maven foo.

The docker image

I required LDAP configured with some known groups and users to use for testing. I put the following Dockerfile in src/test/docker/ldap, along with users.ldif which defined the users I wanted to load.

FROM nickstenning/slapd

#Our envt variables
ENV LDAP_ROOTPASS topsecret
ENV LDAP_ORGANISATION My Org
ENV LDAP_DOMAIN domain.com

# Users and groups to be loaded
COPY users.ldif        /users.ldif

# Run slapd to load ldif files
RUN apt-get update && apt-get install -y ldap-utils && rm -rf /var/lib/apt/lists/*
RUN (/etc/service/slapd/run &) && sleep 2 && \
  ldapadd -h localhost -p 389 -c -x -D cn=admin,dc=domain,dc=com -w topsecret -v -f /users.ldif && killall slapd

The parent config

In my parent pom, I put the shared configuration for the fabric8io plugin into pluginManagement. This includes any images I want to run for multiple sub modules (I’m using postgres for other integration tests, in combination with the liquibase for setting up a test database), and the executions which apply to all submodules.

 <pluginManagement>
    <plugins> 
        <plugin>
            <groupId>io.fabric8</groupId>
            <artifactId>docker-maven-plugin</artifactId>
            <version>0.18.1</version>
            <configuration>                    
                <images>                       
                    <image>
                        <name>postgres:9.5.1</name>
                        <alias>pg</alias>                           
                        <run>                                    
                            <env>
                                <POSTGRES_PASSWORD>topsecret</POSTGRES_PASSWORD>
                                <POSTGRES_USER>myuser</POSTGRES_USER>
                            </env>
                            <ports>
                                <port>5432:5432</port>
                            </ports> 
                            <wait>
                                <tcp>
                                    <ports>
                                        <port>5432</port>
                                    </ports>
                                </tcp>
                            </wait>  
                        </run>
                    </image>
                </images>
            </configuration>
            <executions>
                <execution>
                    <id>build</id>
                    <phase>pre-integration-test</phase>
                    <goals>
                        <goal>build-nofork</goal>
                    </goals>
                </execution>
                <execution>
                    <id>run</id>
                    <phase>pre-integration-test</phase>
                    <goals>
                        <goal>start</goal>
                    </goals>
                </execution>
                <execution>
                    <id>stop</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>stop</goal>
                    </goals>
                </execution>
            </executions>                    
        </plugin>
    </plugins>
</pluginManagement>

Remember that putting a plugin in pluginManagement in the parent does not mean it will be active in the child modules. It simply defines the default config which will be used, should you add the plugin to the plugins section in the child pom. This approach means we can avoid spinning up docker contained for sub modules which don’t need them.

Remember to use the maven-failsafe-plugin to run your integration tests. This ensures that the docker containers will be stopped even if your integration tests fail, so you don’t get containers hanging around disrupting your next test run.

The sub module config

A key optimisation I wanted to make was not to run the maven-docker-plugin if I was skipping tests – I’d assume I’m skipping tests for a fast build, so the last thing I want is to spin up unnecessary containers. To do this, I activated the docker maven plugin within a profile in the appropriate submodules, which is only active if you haven’t supplied the skipTests parameter.

<profiles>  
    <profile>
        <id>integ-test</id>
        <activation>
            <property>
                <name>!skipTests</name>
            </property>
        </activation>
        <build>
            <plugins>
                <plugin>
                    <groupId>io.fabric8</groupId>
                    <artifactId>docker-maven-plugin</artifactId>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

For submodules which only require the docker images defined in the parent pom, the above configuration is sufficient. However, in one submodule I also want to spin up the ldap image. To do that, I wanted to add to the images defined in the parent. This led me to a useful blog on how to Merging Plugin Configuration in Complex Maven Projects. The combine.children=”append”
attribute ensures that the merge behaviour is as required.

 <profile>
    <id>integ-test</id>
    <activation>
        <property>
            <name>!skipTests</name>
        </property>
    </activation>
    <build>
        <plugins>
            <plugin>
                <groupId>io.fabric8</groupId>
                <artifactId>docker-maven-plugin</artifactId>                
                <configuration>   
                    <!-- append docker images for this module -->                 
                    <images combine.children="append">
                        <image>
                            <name>test-ldap</name>
                            <build>
                                <skip>${env.skip.docker.run}</skip>
                                <dockerFileDir>${project.basedir}/src/test/docker/ldap</dockerFileDir>
                            </build>
                            <run>
                                <skip>${env.skip.docker.run}</skip>
                                <ports>
                                    <port>389:389</port>
                                </ports> 
                                <wait>
                                    <tcp>
                                        <ports>
                                            <port>389</port>
                                        </ports>
                                    </tcp>
                                </wait>  
                            </run>
                        </image>                                         
                    </images>                      
                </configuration>
            </plugin>
        </plugins>
    </build>
</profile>

Result: completely portable integration tests.

186

102

November 2, 2016

Use Docker Compose to run a PHP site without contaminating your system

This must be the quickest way to get the LAMP stack up and running locally.
Nothing installed locally -> you can run different versions of PHP/MySQL without conflicts.
Database gets created on first startup and persisted in a named volume.
PHP files are in a mapped volume so you can edit without rebuilding the container.

The docker-compose.yml file:

version: "2"
services:
  site:
    image: php:5.6.27-apache   
    volumes:
      - ./site:/var/www/html
    depends_on:
      - mysql
    networks:
      back-tier:        
  mysql:
    image: mysql:5.5
    environment:
      MYSQL_ROOT_PASSWORD: topsecret
      MYSQL_DATABASE: sitedbname
      MYSQL_USER: sitedbuser
      MYSQL_PASSWORD: sitedbpassword
    volumes:
      - site_db:/var/lib/mysql
    networks:
      back-tier:
networks:
  back-tier:
volumes:
  site_db:

Then just

docker-compose up

It takes a bit of the pain out of working on a legacy PHP site!

170

November 27, 2015December 31, 2015

Super quick Sonar/Postgres setup with docker

Maybe I am easily impressed but wow! Here is how I used docker to setup sonar on my (Ubuntu) laptop superquick.

Postgres

First, set up a postgres container. The command below creates and starts a container called sonar-postgres, using the official docker postgres image.

docker run --name sonar-postgres -e POSTGRES_USER=sonar -e POSTGRES_PASSWORD=secret -d postgres

The container is created containing a sonar database and user with the supplied password. There are various options to the run command, for example to restart the container automatically. See docker run for more details. -d means detach from the container and run it in the background.

The command above does not publish any ports to the host, so we can’t psql to localhost port 5432 to see the database. However, the postgres container does “expose” port 5432 to linked containers. To have a quick peek inside, the following command creates a temporary container which executes the psql command. After entering the password you chose earlier, you are logged into the sonar database. When you exit, the container is gone.

docker run -it --link sonar-postgres:postgres --rm postgres sh -c 'exec psql -h "$POSTGRES_PORT_5432_TCP_ADDR" -p "$POSTGRES_PORT_5432_TCP_PORT" -U sonar'

An alternative would have been to modify the original run command to include -p 5432:5432 which would publish the ports to the host and allow direct access via psql. This was handy for me when getting started but obviously not ideal in an environment where you might have several postgres containers.

Sonar

The default command to start sonar uses the built in h2 database:

docker run -d --name sonarqube -p 9000:9000 -p 9092:9092 sonarqube:5.1

It’s not good practice to use the embedded database for continued use. To set up a container which uses the postgres container created above, use the following command:

docker run -d --name sonarqube --link sonar-postgres:pgsonar -p 9000:9000 -e SONARQUBE_JDBC_USERNAME=sonar -e SONARQUBE_JDBC_PASSWORD=secret -e SONARQUBE_JDBC_URL=jdbc:postgresql://pgsonar:5432/sonar sonarqube:5.1

In the above command, the –link option links the postgres container with a hostname of “pgsonar”, which is used again in the SONARQUBE_JDBC_URL setting to tell sonar where to find the postgres database.

Tip! If the container won’t start, run the command without the -d switch, so it remains in the foreground and you can see the log output.

Once the container is started, navigate to http://localhost:9000/ and you should see the familiar sonarqube dashboard. This process felt quicker than the usual sonar install done properly, and I can definitely see myself using docker more to supercharge my dev environment!

100