The TCK distribution comes with a directory setup-examples which contains two example projects for running the TCK. If you followed the instructions in Chapter 3, Installation you find the directory under jsr349/tck/setup-examples. Both setups are using Hibernate Validator as Bean Validation Provider and Glassfish 4 as EE constainer. However, one is using Maven as build tool to run the TCK, the other Ant. Depending which of the examples you want to use, you need to install the corresponding build tool.

Each example comes with a readme.md containing the prerequisites for using this setup, how to run the TCK against Hibernate Validator and Glassfish. The readme in setup-examples itself contains information about what needs to be changed to use a different Bean Validation provider and EE container.

The following chapters contain some more information about the general structure of the TCK which will give you a deeper understanding above the simple readme files.

The Bean Validation test harness is built atop TestNG, and it is TestNG that is responsible for selecting the tests to execute, the order of execution, and reporting the results. Detailed TestNG documentation can be found at testng.org.

The tck-tests.xml artifact provided in the TCK distribution must be run by TestNG (described by the TestNG documentation as "with a testng.xml file") unmodified for an implementation to pass the TCK. For testing purposes it is of course ok to modify the file (see also the TestNG documentation)

<suite name="JSR-349-TCK" verbose="1">
    <test name="JSR-349-TCK">

        <method-selectors>
            <method-selector>
                <selector-class name="org.hibernate.beanvalidation.tck.util.IntegrationTestsMethodSelector"/>
            </method-selector>
        </method-selectors>

        <packages>
            <package name="org.hibernate.beanvalidation.tck.tests"/>
        </packages>
    </test>
</suite>

TestNG provides extensive reporting information. Depending on the build tool or IDE you use, the reporting will take a different format. Please consult the TestNG documentation and the tool documentation for more information.

The most important configuration you have make in order to run the Bean Validation TCK is to specify your ValidationProvider you want to run your tests against. To do so you need to set the Java system property validation.provider to the fully specified class name of your ValidationProvider. In Maven this is done via the systemProperties configuration option of the maven-surefire-plugin, whereas sysproperty is used in an Ant testng task. This system property will be picked up by org.hibernate.beanvalidation.tck.util.TestUtil which will instantiate the Validator under test. This means the test harness does not rely on the service provider mechanism to instantiate the Bean Validation provider under test, partly because this selection mechanism is under test as well.

After setting the ValidationProvider you have to make a choice on the right DeployableContainer. Arquillian picks which container it is going to use to deploy the test archive and negotiate test execution using Java's service provider mechanism. Concretely Arquillian is looking for an implementation of the DeployableContainer SPI on the classpath. The setup examples use a remote Glassfish container adapter, which means that Arquillian tries to deploy the test artifacts onto a specified remote Glassfish instance, run the tests remotely and report the results back to the current JVM. The installation directory of the remote container is specified via the container.home property in the example build files.

The next piece in the configuration puzzle is arquillian.xml. This xml file needs to be in the root of the classpath and is used to pass additional options to the selected container. Let's look at an example:

<arquillian xmlns="http://jboss.org/schema/arquillian" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xsi:schemaLocation="http://jboss.org/schema/arquillian
        http://jboss.org/schema/arquillian/arquillian_1_0.xsd">
    <defaultProtocol type="Servlet 3.0"/>

    <engine>
        <property name="deploymentExportPath">target/artifacts</property>
    </engine>

    <container qualifier="incontainer" default="true">
        <configuration>
            <property name="glassFishHome">@CONTAINER.HOME@</property>
            <property name="adminHost">localhost</property>
            <property name="adminPort">4848</property>
            <property name="debug">true</property>
        </configuration>
    </container>

</arquillian>

The most important container configuration option is the protocol type which determines how Arquillian communicates with the selected container. The most popular types are Servlet 3.0 and Local. The former is used when connecting to a remote container whereas the latter is used for the in JVM mode.

Another interesting property is deploymentExportPath which is optional and instructs Arquillian to dump the test artifacts to the specified directory on disk. Inspection of the deployed artifacts can be very useful when debugging test failures.