A coverage tool is a tool used when testing to determine exactly how much of the application you have tested. A good tool will also give you specific examples of what hasn't been tested. Coverage of a program is reported on the basis of metrics.
You can think of coverage in terms of a graph with vertices and edges. The vertices represent statements that can be executed, and the edges represent flow control within the program. There are three classes of coverage that can be divined from this model:
There are other types of coverage which can be measured. These are primarily focused around where these metrics cannot easily reach. The two which come to mind first are:
Coverage tools provide assurance that Unit tests and Integration tests are indeed exercising the class adequately. This information is vital to determine if enough testing is being done.
Coverage also provides valuable insight into the risk of software failure. The more a piece of code is executed under test, the less likely that a software failure will show up at the last minute, or even be found by a customer.
Quilt is a coverage analysis tool which operates on Java bytecode. It intercepts the bytecode before it is loaded into the JVM and instruments the class. The instruments in the class are then queried after the tests are run, and coverage is calculated. In the future, Quilt should also be able to provide insight into what tests can be added to improve coverage.
There are three ways of building a coverage tool, each with their own ups and downs. The three ways you can do it are:
There are a couple of reasons why Quilt is Open Source. First, its because Coverage tool providers charge an arm and a leg for their tools. I wanted to experiment with Code Coverage, and at the time I couldn't buy a tool for less that $500.
And of course, the idea of intercepting bytecode before it hits the JVM, and adding code to it is kind of fun. Why wouldn't anyone want to do it?