Quilt

Quilt is a Java software development tool that measures coverage , the extent to which unit testing exercises the software under test. It is optimized for use with the JUnit unit test package, the Ant Java build facility, and the Maven project management toolkit.

Quilt is currently undergoing a major transformation. The first release of the new version, 0.6a, became available on 31 August 2003. This is now forking off as a separate project while development of the 0.4/0.5 branch will continue at Codehaus. More on the changes in progress below.

This is the Quilt 0.6a Web site. The earlier Quilt 0.4 Web site is available here.

Coverage

The primary aim of the Quilt Project is the development of tools for measuring coverage in the testing of Java programs. Coverage is a measure of the extent to which tests exercise the code under test.

The simplest measure of coverage is statement coverage. For example, if there are 100 lines in a program and only 75 are actually used when tests are being run, then the coverage may be said to be 75%.

This is a simple but very significant check on the testing being done. If 75% of the code is covered, then 25% isn't: you don't know whether 25% of your application works .

Quilt's coverage tools can't actually help you write better tests or better code. But they can tell you whether or not your code is being tested.

How It Works

Quilt intercepts code as it is being loaded and alters it. It doesn't work at the source code level. It manipulates compiled classes and their bytecode , the machine code of the Java Virtual Machine, the JVM.

To measure coverage, Quilt inserts instrumentation , typically bytecode that increments counters whenever a block of code or branch is executed. This is done transparently. The programmer simply sets a switch at test time. Then when program modules are loaded for testing, Quilt instrumentation is automatically added and coverage reports generated.

Using Quilt with Other Software Development Tools

The current version of Quilt is used with the JUnit unit test program. Typically you will run unit tests under they succeed. Then, when the debugging is largely done, you run Quilt coverage checks as well as JUnit unit tests. If the Quilt run shows that you need more testing, if coverage is too low, you add more tests or alter the ones that you have until you are satisfied with the level of coverage.

One of the objectives of the Quilt project is full JUnit interoperability , in that sense that the interface to Quilt should be a superset of the JUnit interface. If you are used to working with JUnit, you should be able to work with Quilt in exactly the same way.

Quilt and JUnit can also be used in conjunction with Ant, Java's equivalent of the make command-line build facility. Quilt's Ant task, the module interfacing Ant with Quilt and JUnit is complete and available either via CVS or in tarball form.

As you can see from inspecting the Quilt distribution, the Maven project management software package has also been used in developing Quilt. Maven is used primarily for Web site and project documentation generation. Quilt is being integrated back into Maven as part of the development effort; that is, a Quilt goal for Maven is in development.

In the somewhat longer term, we expect to integrate Quilt into other software development packages, in particular into the Eclipse IDE. However, work on this has not yet begun.

Other Applications

Quilt does its magic using a variety of tools. These include

• transforming class loaders, which load a compiled Java class and change it in some way, typically while leaving its semantics unchanged
• instrumenting class loaders, a special case of transforming loaders which insert counters and other blocks of bytecode for monitoring program behavior
• a class synthesizer, which builds a class at run time, from a description, rather than loading it from disk
• tools for generating and modifying method control flow graphs
• report generation facilities

These and other Quilt components can be used to build other tools for monitoring and modifying Java code at run time.

Project Status

At the time of writing, there are three distinct versions of Quilt. Quilt 0.4 has been available for download since mid-June 2002. quilt on the Sourceforge CVS site is Quilt 0.5a as of August/September 2002. And quilt-dev in CVS is Quilt 0.6a, which is mostly new code, written along the same lines and inspired by the earlier version.

0.6a5 is the final version of this branch of the code. Quilt 0.4/0.5 development will continue at Codehaus , while the 0.6 branch will live on, at least in spirit, at The JXCL Project on Sourceforge.

0.6a1 tarballs became available on 31 August 2003. This release contained a fully-functional QuiltTask, allowing JUnit and Quilt to be run from Ant. The QuiltTask and TestRunner modules have been thoroughly tested and have no known bugs when run without coverage tests.

0.6a4 remains primarily a developer release. The Quilt classloader and its class transformation framework are connected to the QuiltTask front end. Simple statement coverage reports are generated. However, there are at least eight significant bugs. Work on the upcoming 0.6a5 release will concentrate on bug fixes rather than code enhancements.

Some users may wish to experiment with Quilt's Ant task, test runner, and report formatters, which we believe will become a significant improvement on the existing Ant JUnit task.

After a reasonable period of time, when the 0.6 release has been reviewed and proved satisfactory, the various other versions currently found in CVS will be withdrawn to reduce confusion. This is unlikely to occur before 1 January 2004.