The test suite is executed by the Maven 2 TestNG plugin during the test phase of the Maven 2 life cycle. The execution happens within a TCK runner project (as opposed to the TCK project itself). Weld includes a TCK runner project that executes the CDI TCK on Weld running inside JBoss AS 5.1. To execute the CDI TCK on your own CDI implementation, you could modify the TCK runner project included with Weld to use your CDI implementation as described in Chapter 4, Configuration.

To execute the TCK test suite against Weld, first switch to the jboss-tck-runner directory in the extracted Weld distribution:

cd weld/jboss-tck-runner

Then execute the Maven 2 life cycle through the test phase:

mvn test

Without any command-line flags, the test suite is run in standalone mode, which means that any test class with the @org.jboss.testharness.impl.packaging.IntegrationTest annotation is skipped. This mode uses the StandaloneContainers SPI to invoke the test artifact within a mock Java EE life cycle and capture the results of the test. However, passing the suite in this mode is not sufficient to pass the TCK as a whole. The suite must be passed while executing using the in-container mode.

To execute the test suite using in-container mode with the JBoss TCK runner, you first have to setup JBoss AS as described in the Running the TCK against the CDI RI () and JBoss AS callout.

Then, execute the TCK runner with Maven 2 as follows:

mvn test -Dincontainer

The presence of the incontainer property activates a Maven 2 profile that assigns the org.jboss.testharness.standalone system property to false and the org.jboss.testharness.runIntegrationTests system property to true, hence activating the in-container test mode. This time, all the test artifacts in the test suite are executed.

The in-container profile will also start and stop the application server automatically, a feature which the profile activates by setting the org.jboss.testharness.container.forceRestart to true.

The in-container mode uses the Containers SPI to deploy the test artifact to the container and execute the test in a true Java EE life cycle. The JBoss TCK runner has a dependency on the library that provides an implementation of this interface for JBoss AS 5.1.

Since in-container tests are executed in a remote JVM, the results of the test must be communicated back to the runner over a container-supported protocol. The JBoss Test Harness provides servlet-based communication over HTTP as described in ???.

As you have learned, when the test suite is executing using in-container mode, each test class is packaged as a deployable artifact and deployed to the container. The test is then executed within the context of the deployed application. This leaves room for errors in packaging. When investigating a test failure, you may find it helpful to inspect the artifact after it's generated. The TCK can accommodate this type of inspection by "dumping" the generated artifact to disk.

The feature just described is activated in the jboss-tck-runner project by appending the dumpArtifacts command line property to the end of the command that invokes the Maven 2 test phase.

mvn test-compile -DdumpArtifacts

The directory where the artifacts get written is configured using the org.jboss.testharness.outputDirectory property. The dumpArtifacts profile in the jboss-tck-runner project sets this value to the relative directory path target/jsr299-artifacts.

You can read more about this feature in Section 11.2, “Dumping the Test Artifacts to Disk”.

Using an existing test harness (TestNG) allows the tests to be executed and debugged in an Integrated Development Environment (IDE) using available plugins. Using an IDE is also the easiest way to execute a test class in isolation.

The TCK can be executed in any IDE for which there is a TestNG plugin available. Running a test from the CDI TCK test suite using the Eclipse TestNG plugin is almost as simple as running any other TestNG test. You can also use the plugin to debug a test, which is described in the next chapter.

Before running a test from the TCK test suite in Eclipse, you must have the Eclipse TestNG plugin and either the m2eclipse plugin or an Eclipse project generated use the Maven 2 Eclipse plugin (maven-eclipse-plugin). Refer to Section 3.3, “Eclipse Plugins” for more information on these plugins.

With the m2eclipse plugin installed, Eclipse should recognize the CDI TCK projects as valid Eclipse projects (or any Weld project for that matter). Import them into the Eclipse workspace at this time. You should also import the Weld projects if you want to debug into that code, which is covered later.

cd tck
mvn clean eclipse:clean eclipse:eclipse -DdownloadSources -DdownloadJavadocs
cd ../weld
mvn clean eclipse:clean eclipse:eclipse -DdownloadSources -DdownloadJavadocs

When setting up your Ecilpse workspace, we recommended creating three workings sets:

The dependencies between the projects will either be established automatically by the m2eclipse plugin, based on the dependency information in the pom.xml files, or as generated by the mvn eclipse:eclipse command.

Your workspace should appear as follows:

Weld
  cdi-api
  weld-api
  weld-core
  weld-core-test
  weld-logging
  weld-parent
  weld-spi
  weld-version-matrix
CDI TCK
  jsr299-tck-api
  jsr299-tck-impl
  jsr299-tck-parent
Weld JBoss TCK Runner
  weld-jboss-tck-runner
  weld-porting-package

The tests in the TCK test suite are located in the jsr299-tck-impl project. You'll be working within this project in Eclipse when you are developing tests. However, as you learned earlier, there are no references to a CDI implementation in the TCK. So how can you execute an individual test in Eclipse? The secret is that you need to establish a link in Eclipse (not in Maven) between the jsr299-tck-impl project and your TCK runner project, which in this case is weld-jboss-tck-runner (the project in the jboss-tck-runner directory).

Here are the steps to establish the link:

Of course, the weld-jboss-tck-runner also depends on the jsr299-tck-impl at runtime (so it can actually find the tests to execute). But Eclipse doesn't distinguish between build-time and runtime dependencies. As a result, we've created a circular dependency between the projects. In all likelihood, Eclipse will struggle (if not fail) to compile one or more projects. How can we break this cycle?

As it turns out, the TCK runner doesn't need to access the tests to build. It only needs its classes, configurations and other dependenices at runtime (when the TestNG plugin executes). Therefore, we can remove the TCK runner's dependence on the jsr299-tck-impl project by following these steps:

You are now ready to execute an individual test class (or artifact). Let's start with a test artifact capable of running in standalone mode.

A standalone test artifact is a class which extends AbstractJSR299Test, is annotated with @Artifact, but is not annotated with @IntegrationTest. Select a standalone test artifact and open it in the Eclipse editor. Now right click in the editor view and select Run As > TestNG Test. The TestNG view should pop out and you should see all the tests in that artifact pass (if all goes well).

So far you have executed a test in standalone mode. That's not sufficient to pass the TCK. The test must be executed using in-container mode. Using in-container mode is also the only way to execute a test annotated with @IntegrationTest (that's what the annotation means) as it requires container resources.

Let's see what has to be done to execute an integration test. This will result in the artifact being deployed to the container, which is JBoss AS 5.1 if you are using the JBoss TCK runner.

As you have learned, the JBoss test harness determines how to behave based on the values of numerous system properties or properties defined in META-INF/jboss-test-harness.properties classpath resources. If the property named org.jboss.testharness.standalone is not defined, the harness assumes that the test is to be run in standalone mode. In order to run the tests in the container, you need to add a properties file to the classpath that sets the standalone property to false and provides values for other properties required to run an in-container test.

The JBoss TCK runner project conveniently provides the properties file src/test/debug-resources/META-INF/jboss-test-harness.properties that contains all of the necessary properties for in-container testing in Eclipse. Assuming you followed the directory structure recommended in Section 3.2, “The TCK Environment”, you are good to go. Otherwise, you may have to tune the org.jboss.testharness.container.extraConfigurationDir and org.jboss.testharness.libraryDirectory properties to point to the relative location of the related projects. The properties should be defined as follows:

The other properties in that file are defined as follows:

org.jboss.testharness.standalone=false
orjboss.testharness.container.forceRestart=false
orjboss.testharness.runIntegrationTests=true

You're now ready to execute an integration test. Select an integration test (a class that extends AbstractJSR299Test and is annotated with both @Artifact and @IntegrationTest) and open it in your Eclipse editor. Follow these steps to execute the class with the TestNG plugin:

When you run the test this time, it should pass. If you get a failure that the container (e.g., JBoss AS 5.1) must be run with assertions enabled, you need to stop the container and start it with the -ea flag (or just leave it stopped and the test will start it appropriately).

You can simply right click and select Run As > TestNG Test for all subsequent runs for the reason cited earlier, the run configuration for a class is retained indefinitely.

Alternatively, you can configure TestNG to execute all tests in-container by default by adding the properties file in the debug-resources folder to the project's classpath as follows:

Now you don't have to do any special configuration per test class.

You can stop the individual tests from running in-container by reversing the steps above to remove the debug-resources folder from the Eclipse classpath.

You have now mastered running the CDI TCK against Weld using both Maven 2 and within Eclipse. Now you're likely interested in how to debug a test so that you can efficiently investigate test failures.

There is almost no difference in how you debug a standalone test from how you run it. With the test class open in the Eclipse editor, simply right click in the editor view and select Debug As > TestNG Test. Eclipse will stop at any breakpoints you set just like it would with any other local debug process.

If you plan to step into a class in the Weld implementation (or any other dependent library), you must ensure that the source is properly associated with the library. Below are the steps to follow to associate the source of Weld with the TestNG debug configuration:

You'll have to complete those steps for any test class you are debugging, though you only have to do it once (the debug configuration hangs around indefinitely).

Again, running a test in standalone isn't enough to pass the TCK and cannot be used to run or debug an integration test. Let's look at how to debug a test running in the context of the container.

In order to debug an integration test, or any test run using in-container mode, the test must be configured to run in-container, as described in Section 7.4, “Running integration tests”, and you must attach the IDE debugger to the container. That puts the debugger on both sides of the fence, so to speak.

Since setting up a test to run in-container has already been covered, we'll look at how to attach the IDE debugger to the container, and then move on launching the test in debug mode.