Github actions caching with Cypress

This has been on my To Do list for a while. To make builds faster (and save github minutes) you can use the cache action to cache dependencies between builds (or jobs in the same workflow) rather than installing them each time.

To cache your node_modules folder, this is the step you would need:

- name: Cache Dependencies
  id: cache-deps
  uses: actions/cache@v2
  with:
     path: ~/node_modules
     key: cache-${{ hashFiles('yarn.lock') }}

In subsequent builds, if your yarn.lock file has not changed, the cache will be restored (in a few seconds) and you won’t need to wait for dependencies to install.

However, if you are using Cypress, configuring the cache as above results in a missing Cypress binary.

The cypress npm package is installed, but the Cypress binary is missing.

We expected the binary to be installed here: /home/runner/work/xxx/xxx/cypress/cache/9.2.0/Cypress/Cypress

Reasons it may be missing:

- You're caching 'node_modules' but are not caching this path: /home/runner/.cache/Cypress
- You ran 'npm install' at an earlier build step but did not persist: /home/runner/.cache/Cypress

It took me a while to get this right, but the steps below work to also cache the Cypress binary

- name: Cache Dependencies
  id: cache-deps
  uses: actions/cache@v2
  with:
    path: |
       ~/.cache/Cypress
       ~/node_modules
       **/node_modules
    key: cache-${{ hashFiles('yarn.lock') }}
- name: Install dependencies
  if: steps.cache-deps.outputs.cache-hit != 'true'
  run: yarn --frozen-lockfile

(Note: this project is a monorepo so I am also caching node_modules wherever it exists, not just in the root folder)

Setting input values in AngularJS 1.6 Unit tests

Having spent hours yesterday wondering why my unit test would not work, I can confirm that the correct way to enter data into an input field in an AngularJS 1.6 unit test is as follows:

var field = element.find('input');       
var wrappedField = angular.element(field);
wrappedField.val('Some text');
wrappedField.triggerHandler('change');

I can further confirm that this will not be enough if you have set a debounce value in the ngModelOptions for the field. The debounce value defines a delay between input changes and triggering a $digest cycle. This can improve performance, as it saves thrashing all the code linked to the two way binding every time a key is pressed. However for the purposes of unit testing it also means that the model will not be updated immediately after running the code above.

I found the answer in this stackoverflow post: How to test an AngularJS watch using debounce function with Jasmine.

Adding the following after setting the field value causes the model updates to be processed immediately:

$timeout.flush();

I’ve added a working example of this unit test to my angular-testing repository on github.

Unexpected side effects using Groovy MockFor in a Grails test

Recently, I was trying to write a Unit Test for a Grails method which instantiates an HTTPBuilder object within a method.

I couldn’t use the normal Grails ‘mockFor’ syntax, because the HTTBuilder was not injected or accessible from outside the class. No problem, because standard Groovy MockFor is also available.

With a bit of help from google (http://stackoverflow.com/questions/9101084/groovy-httpbuilder-mocking-the-response) I came up with the test below.

def testDoSomething() {
    def requestDelegate = [response: [:], uri: [:]]

    def mockHttpBuilder = new MockFor(groovyx.net.http.HTTPBuilder)
			
    mockHttpBuilder.demand.request { Method method, Closure body ->
        body.delegate = requestDelegate
        body.call()
        requestDelegate.response.success() // make it call success
    }

    mockHttpBuilder.use {
        // All calls to HTTPBuilder will be intercepted within this block of code.      
        MyClient myClient = new MyClient();
        assert myClient.doSomething("Some Param")
    }
}

This works really well in isolating the code under test from the HTTP request.

However after spending a bit of time working on this test, and getting it to pass, I ran test-app on the entire project, and was very confused to see that some totally unrelated tests had begin to fail. Grails was reporting errors like “No more calls to ‘get’ expected at this point.” in tests where I had used no mocking at all. I was bewildered.

Eventually I came across GRAILS-8535. Although unit tests are supposed to be just that, it seems that grails continues to use the proxy in subsequent tests. This appears to be fixed in Grails 2.2.3, but as of the time of writing, this wasn’t released.

Luckily, it is possible to manually reset the MetaClass in the Grails MetaClassRegistry at the end of the test.

At the start of the test which uses MockFor, I record what the class was originally:

MetaClassRegistry registry = GrailsMetaClassUtils.getRegistry()
def originalMetaClass = registry.getMetaClass(HTTPBuilder)

And at the end of the test, set it back:

MetaClassRegistryCleaner.addAlteredMetaClass(HTTPBuilder, originalMetaClass) 

Best method for functional testing in grails

or ‘why isn’t there more quality control on the internet…?’

This week I tried to add some automated functional tests to a grails project. (See my post on agile best practices; we agreed that our definition of ‘Done’ should definitely include functional tests)

There are a few plugins for grails which look like they could do the job.

My final selection criteria turned out to be: the only one which I could get working within less than half a day.

Canoo Webtest

http://grails.org/plugin/webtest

I came away from a recent grails training course thinking that this was the de-facto standard for functional testing. In fact, I have since then used the standalone Webtest tool to test a legacy PHP application, with good success.

However, trying to get going in grails was a different matter.

UNRESOLVED DEPENDENCIES ::
::::::::::::::::::::::::::::::::::::::::::::::
:: net.sourceforge.htmlunit#htmlunit;2.8-SNAPSHOT: not found
::::::::::::::::::::::::::::::::::::::::::::::

I found the following bug report, and following the advice to specify the htmlunit version did allow me to get started.

http://jira.grails.org/browse/GPWEBTEST-72

But I was already disappointed.  If I install a plugin using the grails install-plugin command, I think there should be some level of assurance that the plugin version to be installed will be a tested version, dependent on NON-snapshot versions of any required libraries.

Grails functional-test plugin

I then realised that other plugins were available, as mentioned in the grails functional testing user guide.
The documentation for this one defines the dependency as

compile ":functional-test:2.0.RC1"

Again, I do not want the install-plugin command to install a Release Candidate. I want the tested and released version. I tried using an older version as well, but gave up on this pretty quickly too.

selenium-rc

See: http://grails.org/plugin/selenium-rc
The grails plugin page states:

This plugin is no longer maintained. Consider looking at Geb.

So this is what I did.

Geb

After a bad day, I was pleasantly surprised to get going quickly with Geb. And I really like the separation between modelling the available functions on the page and calling them in tests. Although the plugin version is only 0.9.0 it feels like it is current and will be maintained.

I’m surprised at how much trouble I had getting going with functional testing. It feels like there is a lack of quality control on the publicly available Grails plugins, and little attempt to ensure versions are compatible with other standard plugins such as jQuery.

I’ll be spending some more time learning to use Geb in the near future, so I’ll try and report my progress…

Grails unit tests failing on jenkins

I spent a while confused today, when some fairly straightforward unit tests were running fine locally, but failing on our continuous integration server, jenkins.

The error was occurring in the setUp method, annotated @Before, where we were setting a flash variable prior to running tests.

java.lang.NullPointerException: Cannot invoke method getFlashScope() on null object
at grails.test.mixin.web.ControllerUnitTestMixin.getFlash(ControllerUnitTestMixin.groovy:159)

After much head scratching I came across this link
http://www.mopri.de/2013/grails-unit-testing-and-a-little-fun-with-before/
which explains a similar problem.

It appears that because the Controller uses a Mixin, it effectively extends ControllerUnitTestMixin which is not abstract and also has a method annotated with @Before. Junit does not guarantee the order in which the two methods annotated with @Before will be run (See http://junit.sourceforge.net/doc/faq/faq.htm#tests_2)

If the Mixin method does not run before my setUp method, the flash scope has not been mocked, hence the failure.

The solution suggested in the post is to call super.bindGrailsWebRequest() in the setUp method. In our case it was simple enough to set the flash variable within the individual tests, instead of the setUp method.

Sharing junit tests with Maven

We’ve now had two cases at work where we have created a set of junit tests, which we want to re-use in other maven modules. It is a two stage process, and there are a couple of options for how to run the tests in the project which imports them.

Creating re-usable tests

First, we create the tests in the ‘tester-module’. These are standard unit tests in src/test/java. Of course, they must pass, or your project will not build. So you need some sample implementations of the what you are testing within that project. For ease of use, we put all the tests which will be exported into a single package.

To create a test-jar, which can be a dependency for other projects, add to the pom:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jar-plugin</artifactId>
    <version>2.3.1</version>
    
    <executions>
        <execution>           
            <id>test-jar</id>
            <goals>
                <goal>test-jar</goal>
            </goals>
            <configuration>
                <includes>
                    <include>**/exported_tests/*</include>
                </includes>
            </configuration>
        </execution>       
    </executions>    
</plugin>

This will generate a test-jar containing only the tests in the package specified.

Importing the tests

To import the test to the module which will use them, the following dependency is added.

<dependency>
    <groupId>tester-group</groupId>
    <artifactId>tester-module</artifactId>
    <version>1.0</version>
    <type>test-jar</type>
    <scope>test</scope>
</dependency>

Running the tests

There are two ways of running the attached tests. They DO NOT get run automatically when you run mvn test.

Method 1. Extracting the dependency

Adding the following to the pom will extract the test classes into target/test-classes, so that they all get run when you run mvn test. This works well if you always want to run ALL the attached tests.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <executions>
        <execution>
            <id>unpack</id>
            <phase>process-test-classes</phase>
            <goals>
                <goal>unpack</goal>
            </goals>
            <configuration>
                <artifactItems>
                    <artifactItem>
                        <groupId>tester-group</groupId>
                        <artifactId>tester-module</artifactId>
                        <version>1.0</version>
                        <type>test-jar</type>
                        <outputDirectory>${project.build.directory}/test-classes</outputDirectory>
                    </artifactItem>
                </artifactItems>
            </configuration>
        </execution>
    </executions>
</plugin>

Method 2. Using test suites

If you want to run certain tests only, you can add a TestSuite to the module, and define which of the attached tests should be run as part of the test suite.

@RunWith(Suite.class)
@Suite.SuiteClasses({TestClass1.class, TestClass2.class})
public class TestSuite {
  //nothing
}

An afterthought: testing the tests

In the situations where we are re-using our tests, the tests themselves become a critical component of our systems. We wanted to be able to test the tests too. We got round this by not including logic in the tests themselves. Instead, the test-module contains a set of ‘validators’, in src/main. We can then write tests for these ‘validators’ as usual. The tests in ‘exported_tests’ can then simply delegate to an appropriate validator, which we know has been tested.

assertTrue(validator.validate(classUnderTest));

The only difference this makes is that you have to add a normal dependency to the tester-module, as well as the test-jar dependency.

We’ve found this approach very useful, as we’re using maven and junit as a framework for testing other file resources. However I think it is useful for java code too, if you have a set of interfaces, and different implementations in different modules.

References:
http://maven.apache.org/guides/mini/guide-attached-tests.html
http://softwaremavens.blogspot.co.uk/2009/09/running-tests-from-maven-test-jar-in.html