Chapter 12. JBoss Test Suite

The JBoss Testsuite module is a collection of JUnit tests which require a running JBoss instance for in-container testing. Unit tests not requiring the container reside in the module they are testing.

The setup and initialization of the container is performed in the testsuite's build.xml file. The testsuite module also provides utility classes which support the deployment of test artifacts to the container.

12.1. How To Run the JBoss Testsuite

A source distribution of JBoss must be available to run the testsuite. This document applies only to JBoss 3.2.7 and above.

12.1.1. Build JBoss

Before building the testsuite, the rest of the project must be built:


cd build


cd build

12.1.2. Build and Run the Testsuite

To build and run the testsuite, type the following. Note that you no longer are required to seperately start a JBoss server instance before running the testsuite.


You must not have a JBoss instance running before you run the testsuite.


cd ../testsuite
./ tests


cd ../testsuite
build.bat tests

The build script will start and stop various configurations of JBoss, and then run tests against those configurations.

12.1.3. Running One Test at a Time

To run an individual test, you will need to start the appropriate configuration. For most tests, this will be the "all" configuration:

build/output/jboss-5.0.0alpha/bin/ -c all 

And then tell the testsuite which test you want to run:

cd testsuite
./ one-test -Dtest=org.jboss.test.package.SomeTestCase

12.1.4. Clustering Tests Configuration

Most of the tests are against a single server instance started on localhost. However, the clustering tests require two server instances. By default, the testsuite will bind one of these instances to localhost, and the other will be bound to hostname. You can override this in the testsuite/ file.


The nodes must be bound to different IP addresses, otherwise there will be port conflicts. Also, note these addresses must be local to the box you are running the testsuite on, the testsuite will need to start each server process before running the tests.

You can also use the udpGroup property to prevent your clustering tests from interfering with others on the same network using the udpGroup property. This can be passed at the command line or in the file. This will be passed to the servers under test using the -u option:

./ -DudpGroup= tests
[server:start] java org.jboss.Main -c minimal -b localhost -u 

12.1.5. Viewing the Results

A browsable HTML document containing the testsuite results is available under testsuite/output/reports/html, and a text report (useful for emailing) is available under testsuite/output/reports/text.

12.2. Testsuite Changes

The testsuite build.xml has been refactored to allow automated testing of multiple server configurations. The testsuite build scripts include facilities for customizing server configurations and starting and stopping these configurations. Most notably, this improvement allows clustering unit tests to be completely automated.

12.2.1. Targets

Tests are now grouped into targets according to which server configuration they require. Here is a summary of the targets called by the top-level tests target:

Table 12.1. Build Targets and Descriptions

jboss-minimal-testsTests requiring the minimal configuration.
jboss-all-config-testsRuns the all configuration. Most tests can go here.
tests-security-managerRuns the default configuration with a security manager.
tests-clusteringCreates two custom configurations based on the all configuration. Tests run in this target should extend JBossClusteredTestCase to access cluster information.
tomcat-ssl-testsCreates and runs a configuration with Tomcat SSL enabled.
tomcat-sso-testsCreates and runs a configuration with SSO enabled.
tomcat-sso-clustered-testsCreates and runs two nodes with SSO enabled.

12.2.2. Files

The testsuite build scripts have been reorganized. The code generation and jar targets have been extracted to their own files in testsuite/imports. These targets are imported for use by the main build.xml file. Also, it is important to note that module and library definitions are in different files.

Table 12.2. Summary of build files

Build FileDescription
testsuite/build.xmlContains test targets. This file imports the macros and targets from the files below.
testsuite/imports/server-config.xmlContains macros for creating and starting different server configurations.
tools/etc/buildmagic/modules.xmlSimilar to modules.ent, this file contains the Ant classpath definitions for each JBoss module.
tools/etc/buildmagic/thirdparty.xmlLike thirdparty.ent, this contains the Ant classpath definitions for each third party library.
testsuite/imports/code-generation.xmlXdoclet code generation. This file has the following targets: compile-bean-source, compile-mbean-sources, compile-xmbean-dds, compile-proxycompiler-bean-source.
testsuite/imports/test-jars.xmlAll jar tasks. The top-level jars target calls each module's _jar-* target (eg: _jar-aop).

12.3. Functional Tests

Functional tests need to be located in the module which they test. The testsuite needs to be able to include these in the "tests" target.

To contribute functional tests to the testsuite, each module should contain a tests directory with with a build.xml. The build.xml should contain at least one target, functional-tests, which executes JUnit tests. The functional-tests target should build the tests, but should assume that the module itself has been built. The tests/build.xml should use the Ant <import/> task to reuse targets and property definitions from the module's main build.xml.

Functional test source code belongs in the tests/src directory. The package structure of the tests should mirror the module's package structure, with an additional test package below org/jboss.

For example, classes under org.jboss.messaging.core should have tests under org.jboss.test.messaging.core.

12.3.1. Integration with Testsuite

The testsuite/build.xml will include a functional-tests target which uses the <subant> task to call the funtional-tests target on each module's tests/build.xml. The testsuite will only override properties relevant to the junit execution, and the module's tests/build.xml must use these properties as values for the corresponding attributes:

  1. junit.printsummary
  2. junit.haltonerror
  3. junit.haltonfailure
  4. junit.fork
  5. junit.timeout
  6. junit.jvm
  7. junit.jvm.options
  8. junit.formatter.usefile
  9. junit.batchtest.todir
  10. junit.batchtest.haltonerror
  11. junit.batchtest.haltonfailure
  12. junit.batchtest.fork

The following properties are not set by the testsuite:

  1. junit.sysproperty.log4j.configuration
  2. junit.sysproperty.*

Example 12.1. Example Build Script for Functional Tests

<?xml version="1.0" encoding="UTF-8"?>

<!-- ====================================================================== -->
<!--                                                                        -->
<!--  JBoss, the OpenSource J2EE webOS                                      -->
<!--                                                                        -->
<!--  Distributable under LGPL license.                                     -->
<!--  See terms of license at                           -->
<!--                                                                        -->
<!-- ====================================================================== -->

<!-- $Id: testsuite.xml,v 1.4 2006/02/22 20:56:55 rgenova Exp $ -->

<project default="tests" name="JBoss/Messaging">

   <!-- overridden to resolve thirdparty & module deps -->
   <dirname property="remote.root" file="${basedir}"/>
   <dirname property="project.root" file="${remote.root}"/>

   <import file="../../tools/etc/buildmagic/build-common.xml"/>
   <import file="../../tools/etc/buildmagic/libraries.xml"/>
   <import file="../../tools/etc/buildmagic/modules.xml"/>

   <!-- ================================================================== -->
   <!-- Configuration                                                      -->
   <!-- ================================================================== -->

   <!-- Module name(s) & version -->
   <property name="" value="jms"/>
   <property name="module.Name" value="JBoss Messaging"/>
   <property name="module.version" value="5.0.0"/>

   <!-- ========= -->
   <!-- Libraries -->
   <!-- ========= -->
   <!-- The combined library classpath -->
   <path id="library.classpath">
      <path refid="apache.log4j.classpath"/>
      <path refid="oswego.concurrent.classpath"/>
      <path refid="junit.junit.classpath"/>
      <path refid="jgroups.jgroups.classpath"/>
      <path refid="apache.commons.classpath"/>

   <!-- ======= -->
   <!-- Modules -->
   <!-- ======= -->
   <!-- The combined dependent module classpath -->
   <path id="dependentmodule.classpath">
      <path refid="jboss.common.classpath"/>
      <path refid="jboss.jms.classpath"/>

   <!-- ===== -->
   <!-- Tasks -->
   <!-- ===== -->

   <property name="" value="${module.source}"/>
   <property name="build.tests.classes" value="${module.output}/classes"/>
   <property name="build.tests.lib" value="${module.output}/lib"/>
   <property name="build.tests.output" value="${module.output}/reports"/>
   <property name="build.performance.tests.output" value="${module.output}/reports/performance"/>
   <property name="build.tests.archive" value="jboss-messaging-tests.jar"/>

   <path id="test.classpath">
      <path refid="library.classpath"/>
      <path refid="dependentmodule.classpath"/>

   <!-- Compile all test files -->
   <target name="compile-test-classes">
      <mkdir dir="${build.tests.classes}"/>
      <javac destdir="${build.tests.classes}"
         <src path="${}"/>
         <classpath refid="test.classpath"/>
         <include name="**/*.java"/>

   <target name="tests-jar"
           description="Creates the jar file with all the tests">

      <mkdir dir="${build.tests.lib}"/>

      <!-- Build the tests jar -->
      <jar jarfile="${build.tests.lib}/${build.tests.archive}">
         <fileset dir="${build.tests.classes}">
            <include name="org/jboss/test/messaging/**"/>

      The values from imported files or set by the calling ant tasks will take precedence over
      the values specified below.
   <property name="junit.printsummary" value="true"/>
   <property name="junit.haltonerror" value="true"/>
   <property name="junit.haltonfailure" value="true"/>
   <property name="junit.fork" value="true"/>
   <property name="junit.includeantruntime" value="true"/>
   <property name="junit.timeout" value=""/>
   <property name="junit.showoutput" value="true"/>
   <property name="junit.jvm" value=""/>
   <property name="junit.jvm.options" value=""/>
   <property name="junit.formatter.usefile" value="false"/>
   <property name="junit.batchtest.todir" value="${build.tests.output}"/>
   <property name="junit.batchtest.haltonerror" value="true"/>
   <property name="junit.batchtest.haltonfailure" value="true"/>
   <property name="junit.batchtest.fork" value="true"/>
   <property name="junit.test.haltonfailure" value="true"/>
   <property name="junit.test.haltonerror" value="true"/>

   <target name="prepare-testdirs"
           description="Prepares the directory structure required by a test run">
      <mkdir dir="${build.tests.output}"/>

   <target name="tests"
           depends="tests-jar, prepare-testdirs"
           description="Runs all available tests">

      <junit printsummary="${junit.printsummary}"
            <path refid="test.classpath"/>
            <pathelement location="${build.tests.lib}/${build.tests.archive}"/>
            <pathelement location="${module.root}/etc"/>
         <formatter type="plain" usefile="${junit.formatter.usefile}"/>
         <batchtest fork="${junit.batchtest.fork}"
            <formatter type="plain" usefile="${junit.formatter.usefile}"/>
            <fileset dir="${build.tests.classes}">
               <include name="**/messaging/**/*Test.class"/>
               <exclude name="**/messaging/**/performance/**"/>

   <target name="test"
           depends="tests-jar, prepare-testdirs"
           description="Runs a single test, specified by its FQ class name via 'test.classname'">

      <fail unless="test.classname"
            message="To run a single test, use: ./ test -Dtest.clasname=org.package.MyTest"/>

      <junit printsummary="${junit.printsummary}"
            <path refid="test.classpath"/>
            <pathelement location="${build.tests.lib}/${build.tests.archive}"/>
            <pathelement location="${module.root}/etc"/>
         <formatter type="plain" usefile="${junit.formatter.usefile}"/>
         <test name="${test.classname}"

   <target name="performance-tests"/>

   <target name="functional-tests" depends="tests"/>

   <!-- Clean up all build output -->
   <target name="clean"
      description="Cleans up most generated files.">
      <delete dir="${module.output}"/>

   <target name="clobber" depends="clean"/>


12.4. Adding a test requiring a custom JBoss Configuration

Custom JBoss configurations can be added using the create-config macro as demonstrated by this tomcat-sso-tests target. The create-config target has the following attributes/elements:

  1. baseconf : The existing jboss configuration that will be used as the base configuration to copy
  2. newconf : The name of the new configuration being created
  3. patternset : This is the equivalent of the standard patternset element which is used to restrict which content from the baseconf is to be copied into newconf.

In addition, if you need to override configuration settings or add new content, this can be done by creating a directory with the same name as the newconf attribute value under the testsuite/src/resource/tests-configs directory. In this case, there is a tomcat-sso directory which adds some security files to the conf directory, removes the jbossweb sar dependencies it does not need, and enables the sso value in the server.xml:

$ ls -R src/resources/test-configs/tomcat-sso
CVS/  conf/  deploy/

CVS/  login-config.xml***

CVS/  jbossweb-tomcat50.sar/

CVS/  META-INF/  server.xml*

CVS/  jboss-service.xml*

The full tomcat-sso-tests target is shown here.

   <target name="tomcat-sso-tests"
      description="Tomcat tests requiring SSO configured">
      <!-- Create the sso enabled tomcat config starting with the default config -->
      <create-config baseconf="default" newconf="tomcat-sso">
            <include name="conf/**" />
            <include name="deploy/jbossweb*.sar/**" />
            <include name="deploy/jmx-invoker-adaptor-server.sar/**" />
            <include name="lib/**" />
      <start-jboss conf="tomcat-sso" />
      <wait-on-host />
      <junit dir="${module.output}"

         <jvmarg value="${junit.jvm.options}"/>
         <sysproperty key="jbosstest.deploy.dir" file="${build.lib}"/>
         <sysproperty key="build.testlog" value="${build.testlog}"/>
         <sysproperty key="log4j.configuration" value="file:${build.resources}/log4j.xml"/>

            <pathelement location="${build.classes}"/>
            <pathelement location="${build.resources}"/>
            <path refid="tests.classpath"/>

         <formatter type="xml" usefile="${junit.formatter.usefile}"/>

         <batchtest todir="${build.reports}"

            <fileset dir="${build.classes}">
               <patternset refid="tc-sso.includes"/>
      <stop-jboss />

12.5. Tests requiring Deployment Artifacts

This section describes how to write tests that depend on a deployed artifact such as an EAR.

Deployment of any test deployments is done in the setup of the test. For example, the HibernateEjbInterceptorUnitTestCase would add a suite method to deploy/undeploy a har-test.ear:

public class HibernateEjbInterceptorUnitTestCase extends JBossTestCase {
   /** Setup the test suite.
   public static Test suite() throws Exception
      return getDeploySetup(HibernateEjbInterceptorUnitTestCase.class, "har-test.ear");

If you need to perform additional test setup/tearDown you can do that by extending the test setup class like this code from the SRPUnitTestCase:

   /** Setup the test suite.
   public static Test suite() throws Exception
      TestSuite suite = new TestSuite();
      suite.addTest(new TestSuite(SRPUnitTestCase.class));

      // Create an initializer for the test suite
      TestSetup wrapper = new JBossTestSetup(suite)
         protected void setUp() throws Exception
            // Establish the JAAS login config
            String authConfPath = super.getResourceURL("security-srp/auth.conf");
            System.setProperty("", authConfPath);
         protected void tearDown() throws Exception
      return wrapper;

12.6. JUnit for different test configurations

We use the ant-task <junit> to execute tests. That task uses the concept of formatters. The actual implementation uses the XML formater by specifying type="xml" in the formatter attribute.

If we need to execute the same test more than once, using this default formatter will always overwrite the results. For keeping these results alive, we have created another formatter. So, use these steps to keep JUnit results between different runs:

Define the sysproperty "jboss-junit-configuration" during the jUnit calls. Change the formatter and set a different extension for keeping the files between different executions:

Set the class by classname="org.jboss.ant.taskdefs.XMLJUnitMultipleResultFormatter

Here is a complete example of the changes:

      <junit dir="${module.output}"
	 <sysproperty key="jboss-junit-configuration" value="${jboss-junit-configuration}"/>
         <formatter classname="org.jboss.ant.taskdefs.XMLJUnitMultipleResultFormatter" usefile="${junit.formatter.usefile}" extension="-${jboss-junit-configuration}.xml" />

12.7. Excluding Bad Tests

If a test cannot be fixed for some reason, it should be added to the bad.test excludes. This is maintained near the top of the testsuite/build.xml. The patternset will be used to exclude tests all calls to JUnit within the testsuite.

  <!-- Tests that are currently broken -->
  <patternset id="badtest.excludes">
    <exclude name="org/jboss/test/aop/test/RemotingUnitTestCase"/>
    <!-- The media ejb is not active -->
    <exclude name="org/jboss/test/media/**"/>
    <!-- Needs apache ? -->
    <exclude name="org/jboss/test/cluster/httpsessionreplication/**"/>
    <exclude name="org/jboss/test/cache/test/local/ConcurrentTransactionalUnitTestCase.class"/>