Specifying which tests to run

A single test can be executed using the test property. The value is the short name (not the fully qualified package name) of the test. For example:

$ mvn -Dtest=FqnTest test

Alternatively, if there is more than one test with a given classname in your test suite, you could provide the path to the test.

$ mvn -Dtest=org/infinispan/api/MixedModeTest test

Alternatively, you can always pass your own Log4j configuration file via -Dlog4.configuration with your own logging settings.

Patterns are also supported:

$ mvn -Dtest=org/infinispan/api/* test

Skipping the test run

It is sometimes desirable to install the Infinispan package in your local repository without performing a full test run. To do this, simply use the maven.test.skip.exec property:

$ mvn -Dmaven.test.skip.exec=true install

Note that you should never use -Dmaven.test.skip=true since modules' test classes depend on other module test classes, and this will cause compilation errors.

Running tests using @Parameters

If you want to execute tests relying on TestNG's @Parameters from an IDE (such as Eclipse or IntelliJ IDEA), please check this blog entry.

Enabling TRACE in test logs

When you run tests, you can get TRACE logging via using the traceTests profile

mvn test -PtraceTests

Executing this will generate a GZIP file called trace-infinispan.log.gz. This file is not fully closed, so to extract the log file, execute:

gunzip -c trace-infinispan.log.gz > trace-infinsipan.log

Enabling code coverage generation

When you run tests, you can enable code coverage recorder for calculating and analysing the Infinispan code coverage. You can do this using coverage and jacocoReport profiles.
As a code coverage evaluation tool, the JaCoCo is used.

mvn test -Pcoverage -Dmaven.test.failure.ignore=true

Please note, that -Dmaven.test.failure.ignore=true is used for generating JaCoCo code coverage report, even if there are test failures.

Executing this will generate jacoco.exec file in each module's target/ directory. These are the JaCoCo execution data files, which contain full data about the specific module's coverage.

As soon as the coverage execution command is finished, you will need to generate the JaCoCo report, which will merge the generated jacoco.exec files as well as will create the code coverage report.

For having the report in place, run the following command from your Infinispan home directory:

mvn install -pl parent -PjacocoReport

The jacoco-html/ directory will be generated in Infinispan Home directory, which will contain the code coverage report.

Which group should I use?

Running permutations manually or in an IDE

Sometimes you want to run a test using settings other than the defaults (such as UDP for jgroups group tests or the DummyTransactionManager for transaction group tests). This can be achieved by referring to the Maven POM file to figure out which system properties are passed in to the test when doing something different. For example to run a jgroups group test in your IDE using TCP instead of the default UDP, set -Dinfinispan.test.jgroups.protocol=tcp. Or, to use JBoss JTA (Arjuna TM) instead of the DummyTransactionManager in a transaction group test, set -Dinfinispan.test.jta.tm=jbosstm Please refer to the POM file for more properties and permutations.

Tips for writing and debugging parallel tests

There are a number of constraints and best practices that need to be followed in order to ensure correctness and keep the execution time to a minimum. If you follow these guidelines you will find your tests are more reliable:

Each test class is run in a single thread
There is no need to synchronize unit test's fixture, as test methods will be run in sequence. However, multiple test classes are executed in parallel.

Each test class must have an unique test name
As a convention, the name of the test should be the FQN of the test class with the org.infinispan prefix removed. For example, given a test class org.infinispan.mypackage.MyTest the name of the test should be mypackage.MyTest. This convention guarantees a unique name.

package org.infinispan.mypackage;
@Test (testName = "mypackage.MyTest")
public class MyTest { ... }

Use TestCacheManagerFactory.createXyzCacheManager and _never_create managers using new DefaultCacheManager()
This ensures that there are no conflicts on resources e.g. a cluster created by one test won't interfere with a cluster created by another test.

Where possible, extend SingleCacheManagerTestorMultipleCacheManagersTest
Tests inheriting from these template method classes will only create a cache/cluster once for all the test methods, rather than before each method. This helps keep the execution time down.

Never rely onThread.sleep()
When running in heavily threaded environments this will most often not work. For example, if using ASYNC_REPL, don't use a sleep(someValue) and expect the data will be replicated to another cache instance after this delay has elpased. Instead, use a ReplicationListener (check the javadoc for more information). Generally speaking, if you expect something will happen and you don't have a guarantee when, a good approach is to try that expectation in a loop, several times, with an generous (5-10secs) timeout. For example:

while (Systet.currentTimeMillis() - startTime < timeout) {
   if (conditionMeet()) break;
   Thread.sleep(50);
}

Thread.sleep(10) may not work in certain OS/JRE combos (e.g. Windows XP/Sun JRE 1.5)
Use values grater than 10 for these statements, e.g. 50. Otherwise, a System.currentTimeMillis() might return same value when called before and after such a sleep statement.

JMX
For each cache that is create with TestCacheManagerFactory.createXyzCacheManager() the test harness will allocate a unique JMX domain name which can be obtained through CacheManager.getJmxDomain(). This ensures that no JMX collisions will takes place between any tests executed in parallel. If you want to enforce a JMX domain name, this can be done by using one of the TestCacheManagerFactory.createCacheManagerEnforceJmxDomain methods. These methods must be used with care, and you are responsible for ensuring no domain name collisions happen when the parallel suite is executed.

Use obscure words
Insert uncommon or obscure words into the cache that has been generated with a random word generator. In a multi-threaded environment like Infinispan's testsuite, grepping for these words can greatly help the debugging process. You may find this random word generator useful.

Use the test method name as the key
Grab the test method and use it as part of the cached key. You can dynamically grab the test method using code like this:

Thread.currentThread().getStackTrace()(1).getMethodName()

Even though we've tried to reduce them to a min, intermittent failures might still appear from time to time. If you see such failures in existing code please report them in the issue tracker.