The second VTK Hackathon
It has been a busy start to 2014, the twentieth year of VTK’s existance. To begin with, we’ve just released VTK 6.1.0. Besides bug fixes this release brings cool new capabilities including visualization on the web, fine grained paralellism in multithreaded and GPU accelerated contexts, a zero copy interface to populate vtk arrays with, and the return of easy-to-get-started-with binary distributions. This is the first release since the big restructuring that went on with 6.0. For those who aren’t ready to jump on board with VTK 6.x just yet, we’ve begun to merge patches into the release-5.10 branch in the repository that extend the lifetime, if not the functionality of, the old code base a little while longer while we all get comfortable with 6.x.
VTK has thrived for as long as it has, in part because of the developers’ shared attention to the often mundane software engineering chores that go on behind the scenes in a large software project. Take regression test coverage for example. Every day tens of volunteer machines run up to two thousand regression tests on VTK. 2000 some odd tests sounds like a lot, but when you consider that VTK has roughly 2.1 million lines of code in it, those tests could mean very little. One good heuristic for examining how good the regression suite is is to measure the number of lines executed and skipped during the test suite.
We currently measure that on two dashboard machines that compile VTK with “-fprofile-arcs -ftest-coverage” flags. These flags cause the regression tests to make a note of the specific lines in the library that are executed as they run. Afterward we run gcov to analyze the reports and then submit summaries to CDash. A Windows machine with Bullseye coverage will join them soon. From these reports we can tell that our tests exercise a bit better than 60% of VTK, which is quite good considering the size of the toolkit.
Still, we want to know that VTK never acts poorly. We can get closer to knowing that if we hit the missing 40% in our tests. So on January 28’th, we gathered as many VTK developers together as we could with an offer of free coffee and donughts and attacked exactly this problem. Guided by some reports assembled by Bill Lorensen we each picked different poorly tested parts of VTK and hacked away for the day. In some cases we removed or deprecated underused code, in others we wrote new tests to prove that well used parts of the library work today and will continue to work as VTK evolves.
Over the day we were able to increase the coverage from 67.94 to 70.47%. That isn’t bad, and coincidentally was just high enough to pass the arbitrarily chosen threshold configured into CDash to give us the first green coverage report in my not so recent memory. We’ve been doing well since the Hackathon too, the metric is up almost half a point since then, and we hope to keep the momentum going. In particular we will be more strict about enforcing rule #31 from in the VTK software process guide that says all new code must include regression tests.
