Friday, October 18, 2013

Running pure Jasmine unit tests through Maven

I have always really liked writing unit tests. For the simple reason that with those I know that I did all I could to ensure my algorithms worked as planned. Sure, even with high code coverage there is still a chance that you're missing a situation in your tests, but at least once you know this you can fill the gap by adding an additional test. And, of course, you want to run these tests automatically as part of a regular build. No manual testing please :)

So when I started looking at some projects that use JavaScript I wanted to use the same ideas. Write unit tests that are automatically run during a headless build.
I started using Jasmine, as it seems to be the most popular JavaScript testing framework today. Since the project I was working with was using Maven already I wanted to integrate my Jasmine testing as part of the ordinary Maven test cycle.
Additionally, I wanted the setup of my environment be trivial. I really don't want any developer to install additional software besides what they already have to run Maven. And, I don't want to depend on any platform specific software, if possible.

This got me looking around on the internet and I found a really good post by Code Cop that describes how you can do something like this for Apache Ant. What he did was test JavaScript logic using Jasmine, outside of the browser. So you don't have the browser JavaScript environment present, but you can test all your algorithms. This is precisely what I was looking for too. Another nice thing of his work is that the test results are stored in the same XML format as JUnit uses, so you can inspect these files with any tool that can work with ordinary JUnit output XML files (e.g. you can open them in Eclipse and view them in the JUnit view).

I started with the code by Code Cop, and reduced it to the bare minimum, only supporting Jasmine (Code Cop's work also supports other JS test frameworks). You can find this minimal ant-jasmine test integration at coderthoughts/jasmine-ant. The next step: get it working in Maven.

There were a couple of things that needed to be changed to be able to do this:
  1. I wanted to obtain the Java-level dependencies via Maven: the original Rhino scripting engine (can't use the one in the JRE, because JavaAdapter was removed, see here) and js-engine.jar that adds Rhino as the rhino-nonjdk scripting language.
  2. I want to have the source .js files in src/main/js and the tests in test/main/js, the usual locations in Maven.
  3. I needed to make the output directory configurable so that the results are written to target/surefire-reports, where Maven expects these files.
In the end I got things going. I'm still using Ant inside Maven to actually do the Jasmine test running, using a slightly modified version of Code Cop's Jasmine runner Ant task. But the whole end result fits nicely with the rest of the Maven setup.

<project>
  <modelVersion>4.0.0</modelVersion>

  <groupId>org.coderthoughts</groupId>
  <artifactId>jasmine-maven-example</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <packaging>war</packaging> <!-- your JavaScript will likely end up in a .war file -->

  <dependencies>
    <dependency>
      <!-- Bring in the original Rhino implementation that contains the JavaAdapter class -->
      <groupId>org.mozilla</groupId>
      <artifactId>rhino</artifactId>
      <version>1.7R3</version>
      <scope>test</scope>
    </dependency>

    <dependency>
      <!-- Adds the 'rhino-nonjdk' language to the supported scripting languages -->
      <!-- Obtained from the repository at http://dist.codehaus.org/mule/dependencies/maven2/ -->
      <groupId>javax.script</groupId>
      <artifactId>js-engine</artifactId>
      <version>1.0</version>
      <scope>test</scope>
    </dependency>
  </dependencies>

  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-antrun-plugin</artifactId>
        <version>1.7</version>
        <executions>
          <execution>
            <phase>test</phase>
            <configuration>
              <target>
                <property name="jasmine.dir" location="lib/jasmine-ant" />
                <property name="script.classpath" refid="maven.test.classpath" />

                <scriptdef name="jasmine" src="${jasmine.dir}/jasmineAnt.js"
                  language="rhino-nonjdk" classpath="${script.classpath}">
                  <!-- Jasmine (jasmine-rhino.js) needs pure Rhino because 
                       JDK-Rhino does not define JavaAdapter. -->
                  <attribute name="options" />
                  <attribute name="ignoredGlobalVars" />
                  <attribute name="haltOnFirstFailure" />
                  <attribute name="jasmineSpecRunnerPath" />
                  <attribute name="testOutputDir" />
                  <element name="fileset" type="fileset" />
                </scriptdef>

                <jasmine options="{verbose:true}"
                  testOutputDir="target/surefire-reports" haltOnFirstFailure="false"
                  jasmineSpecRunnerPath="${jasmine.dir}/AntSpecRunner.js">
                  <fileset dir="test" includes="**/*Spec.js" />
                </jasmine>
              </target>
            </configuration>
            <goals>
              <goal>run</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

      <!-- ... other plugins ... -->

    </plugins>
  </build>
  
  <repositories>
    <repository>
      <id>codehaus-mule</id>
      <url>http://dist.codehaus.org/mule/dependencies/maven2/</url>
    </repository>
  </repositories>
</project>

A couple of things to note here:
  • I couldn't find the js-engine.jar in Maven Central. Fortunately it was available in the Mule repo at codehaus.org.
  • I added the testOutputDir as a configuration attribute for where the test results go.
  • No setup whatsoever required, no platform specific binaries needed, if you can run Maven you can run these Jasmine tests.
When I run it, it looks like this:

$ mvn test
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building jasmine-maven-example 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
...
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
[INFO] ...
[INFO] --- maven-antrun-plugin:1.7:run (default) @ jasmine-maven-example ---
[INFO] Executing tasks

main:
  [jasmine] Spec: main/js/RomanNumeralsSpec.js
  [jasmine] Tests run: 7, Failures: 0, Errors: 0
  [jasmine]
[INFO] Executed tasks
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.551s


Of course, the build fails when a test fails and also the test reports can be processed using anything that can JUnit test reports, such as mvn surefire-report:report

I find it pretty handy. A minimal project that does this that you can try yourself is available here: coderthoughts/jasmine-maven

So it's a little different from the jasmine-maven-plugin in that this doesn't fork a browser and is hence a little bit faster. It should be possible to speed it up even further by writing a proper Maven plugin for it...
It's more of a pure unit testing environment, where running the jasmine-maven-plugin is closer to a system test setup...
And of course, thanks again to Code Cop for providing an excellent starting point for this stuff.

5 comments:

Peter Kofler said...

Well done. Just an idea - instead of using Ant under the hood you could try the script-maven-plugin, probably just need to tweek the jasmineAnt.js so it works inside the scripting environment. This should give a performance bonus as Ant is not started in the background.

Sandeep Dahiya said...

Unit Software Testing = The unit testing features of Visual Studio 2012 were shown to be an effective way to improve software quality by introducing various tests.

David Bosschaert said...

@Sandeep unit testing is not limited to Visual Studio. Unit testing applies to any piece of software in any programming language.

Jim Cook said...

Any chance of getting a parameter added to JasmineAnt that will let it watch source directories and auto-run tests when a change is detected?

Peter Kofler said...

Jim,
this is more an Ant issue than a JasmineAnt thing. If you add Ant code to the build file (and a loop running the build file from command line) then it might be what you are looking for. Have a look StackOverflow related question for ways to code Ant so you can trigger JasmineAnt only if files change.