This chapter explains the purpose of the test harness and describes its key features.

The JBoss Test Harness is a testing framework based on TestNG that provides a series of extensions that allow runtime packaging and deployment of Java EE artifacts (EAR or WAR) for in-container testing. It's important to note that the JBoss Test Harness has no relation with, or dependency on, the JBoss Application Server (JBoss AS).

The last thing Java developers want is yet another testing framework to make their life more complicated. That's why the JBoss Test Harness is built entirely upon TestNG. TestNG is one of two prominent test frameworks for Java (the other being JUnit). Furthermore, what developers want is a good integration with their Integrated Development Environment (IDE). These days, if a tool doesn't have an IDE plugin, then it won't get the attention it deserves. TestNG plugins are available for all major IDEs and build tools (Ant and Maven 2). Again, a motivating factor for extending TestNG.

Because it leverages the existing TestNG ecosystem, there is no need for a special test launcher for the JBoss Test Harness. You simply use the IDE or build tool of your choice (so long as it has TestNG support). You also get reporting and debugging for free (various reporting plugins are provided for TestNG).

You can read more about TestNG at testng.org.

The JBoss Test Harness supports the following features:

A test is designated by a method annotated with @org.testng.annotations.Test in a class which extends org.jboss.testharness.AbstractTest and is annotated with @org.jboss.testharness.impl.packaging.Artifact.

The @Test annotation is provided by TestNG, the @Artifact annotation is provided by the JBoss Test Harness and the AbstractTest is part of the JBoss Test Harness. There is a one-to-one mapping between a TestNG test class and an artifact. The packaging type is defined by the @org.jboss.testharness.impl.packaging.Packaging annotation on the test class, defaulting to a WAR if not specified.

Prior to executing the tests for a given class, the JBoss Test Harness packages the class as a deployable artifact (EAR or WAR), along with any extra resources specified, and deploys the artifact to the container. The harness provides test execution and result reporting via HTTP communication to a simple Servlet using a thin layer over the TestNG test launcher. The test harness can also catch and enforce expected deployment exceptions. This setup and tear down activity is provided by the super class org.jboss.testharness.AbstractTest, which all test classes must extend (directly or indirectly).

If the annotation @org.jboss.testharness.impl.packaging.IntegrationTest is not present on the test class, then it means the test class can be executed in standalone mode. In standalone mode, the deployable artifact is assembled on the local classpath and the tests execute in the same JVM as the launcher, just as though it were a regular TestNG test case. The standalone mode is provided for convenience and efficiency, allowing you the speed of mock-based testing and the confidence of an in-container test, using the same test objects and tests.

8.1. Negotiating the execution of an in-container test

The basic procedure of an in-container test is as follows. The JBoss Test Harness produces a deployable artifact from an @Artifact test class and any declared dependent classes, descriptors or other resources. Then it deploys the artifact to the container using the Containers SPI, negotiates with the container to execute the test and return the result and, finally, undeploys the artifact. TestNG collects the results of all the tests run in the typical way and produces a report.

The question is, how does the JBoss Test Harness negotiate with the container to execute the test when TestNG is being invoked locally? Technially the mechanism is pluggable, but JBoss Test Harness provides a default implementation that uses HTTP communication that you will likely use. Here's how the default implementation works.

The artifact generator bundles and registers (in the web.xml descriptor) an HttpServlet, org.jboss.testharness.impl.runner.servlet.ServletTestRunner, that responds to test execution GET requests. TestNG running on the client side delegates to a test launcher (more on that in a moment) which originates these text execution requests to transfer control to the container JVM. The name of the test method to be executed is specified in a request query parameter named methodName.

When the test execution request is received, the servlet delegates to an instance of org.jboss.testharness.impl.runner.TestRunner, passing it the name of the test method. TestRunner reads the name of the test class from the resource META-INF/jboss-test-harness.properties, which is bundled in the artifact by the artifact generator. It then combines the class name and the method name to produce a TestNG test suite and runs the suite (within the context of the container).

TestNG returns the results of the run as an ITestResult object. ServletTestRunner translates this object into a org.jboss.testharness.api.TestResult and passes it back to the test launcher on the client side by encoding the translated object into the response. The object gets encoded as either html or a serialized object, depending on the value of the outputMode request parameter that was passed to the servlet. Once the result has been transfered to the client-side TestNG, TestNG wraps up the run of the test as though it had been executed in the same JVM.

There's one piece missing. How does TestNG on the client side know to submit a request to the ServletTestRunner servlet to get TestNG to execute the test in the container JVM? That's the role of the test launcher.

The test launcher is the API that allows test suite to launch the test in a pluggable fashion. AbstractTest implements IHookable, a TestNG interface which allows the execution of the test method to be intercepted. Using that mechanism, AbstractTest delegates execution of the test method (a method annotated with @Test in an @Artifact class) to an implementation of org.jboss.testharness.api.TestLauncher if the tests are being executed in-container. As you might anticipate, the implementation is specified using a property with the same name as the interface in a META-INF/jboss-test-launcher.properties resource. The JBoss Test Harness provides a default implementation, org.jboss.testharness.impl.runner.servlet.ServletTestLauncher, that hooks into the HTTP communication infrastructure described above. It invokes the ServletTestRunner servlet for each method annotated with @Test in the @Artifact that is not otherwise disabled.

If you wish to implement the runner yourself, you must return a TestResult as a result of executing the method in the container. You must also ensure that any exception which occurs during deployment is wrapped as a org.jboss.testharness.api.DeploymentException, and that any communication problem is rethrown as an IOException. The deployment exception may be transformed by an implementation of the org.jboss.testharness.api.DeploymentExceptionTransformer interface, which is specified using the org.jboss.testharness.container.deploymentExceptionTransformer property. The default implementation passes on the original exception unchanged.

So in short, JBoss Test Harness takes care of all the interfaces you need to execute tests in-container except for the implementation of the Containers SPI. That is, unless you are deploying to one of the containers supported by the JBoss Test Harness.