The test suite is executed by the Maven 2 TestNG plugin during the test phase of the Maven 2 life cycle. The execution happens within a TCK runner project (as opposed to the TCK project itself). Hibernate Validator includes a TCK runner project (hibernate-validator-tck-runner) that executes the Bean Validation TCK on Hibernate Validator running inside JBoss AS 5.1. To execute the Bean Validation TCK on your own Bean Validation implementation, you could modify the TCK runner project included with Hibernate Validator to use your Bean Validation implementation as described in Chapter 4, Configuration.

To execute the TCK test suite against Bean Validation, first switch to the hibernate-validator-tck-runner directory in the checked out Hibernate Validator distribution:

cd ri/hibernate-validator-tck-runner

Then execute the Maven 2 life cycle through the test phase:

mvn test

Without any command-line flags, the test suite is run in standalone mode, which means that any test class with the @org.jboss.testharness.impl.packaging.IntegrationTest annotation is skipped. This mode uses the StandaloneContainers SPI to invoke the test artifact within a mock Java EE life cycle and capture the results of the test. However, passing the suite in this mode is not sufficient to pass the TCK as a whole. The suite must be passed while executing using the in-container mode.

To execute the test suite using in-container mode with the JBoss TCK runner, you first have to setup JBoss AS as described in the Running the TCK against the Bean Validation RI (Hibernate Validator) and JBoss AS 7 callout.

Then, execute the TCK runner with Maven 2 as follows:

mvn test -Dincontainer

The presence of the incontainer property activates a Maven 2 profile that assigns the org.jboss.testharness.standalone system property to false and the org.jboss.testharness.runIntegrationTests system property to true, hence activating the in-container test mode. This time, all the test artifacts in the test suite are executed.

The in-container profile will also start and stop the application server automatically, a feature which the profile activates by setting the org.jboss.testharness.container.forceRestart to true.

The in-container mode uses the Containers SPI to deploy the test artifact to the container and execute the test in a true Java EE life cycle. The JBoss TCK runner has a dependency on the library that provides an implementation of this interface for JBoss AS 5.1.

Since in-container tests are executed in a remote JVM, the results of the test must be communicated back to the runner over a container-supported protocol. The JBoss Test Harness provides servlet-based communication over HTTP as described in Section 8.1, “Negotiating the execution of an in-container test”.

As you have learned, when the test suite is executing using in-container mode, each test class is packaged as a deployable artifact and deployed to the container. The test is then executed within the context of the deployed application. This leaves room for errors in packaging. When investigating a test failure, you may find it helpful to inspect the artifact after it's generated. The TCK can accommodate this type of inspection by "dumping" the generated artifact to disk.

The feature just described is activated in the jboss-tck-runner project by appending the dumpArtifacts command line property to the end of the command that invokes the Maven 2 test phase.

mvn test-compile -DdumpArtifacts

The directory where the artifacts get written is configured using the org.jboss.testharness.outputDirectory property. The dumpArtifacts profile in the jboss-tck-runner project sets this value to the relative directory path target/jsr303-artifacts.

You can read more about this feature in Section 10.2, “Dumping the Test Artifacts to Disk”.