Happy Birthday VTK: 30 Years of Innovation

According to git history, January marks the 30th anniversary of the first commit into the VTK’s source code repository. For those of us who have been involved from the early days this is truly mind boggling. We often thought that VTK might last a few years; and in our wildest dreams, maybe ten years. But what we didn’t count on was the power of an open community, the incredible talent that emerged over the three decades, and the vision of developers, customers, and research partners who pushed the system forward to support new applications and technologies.

Over the course of the coming year we are going to celebrate VTK’s birthday with a series of blog posts. We’ll start in this post with a brief history, followed by addressing topics such as what made VTK succeed; the role of community; progeny and derivative works of VTK; business models, future directions, and more. We hope to convey something of the lessons learned, large scale software development, community practices, the ever popular and controversial topics of backward compatibility versus innovation, and dive into related topics such as software sustainability.

So let’s get started with a brief history.

A Brief History of VTK

There were two major influences motivating the creation of VTK. Bill Lorensen and his co-workers at GE Research developed a precursor system called LYMB [1]; and the book “Object-Oriented Modeling and Design” (Bill was one of five authors) had been published to resounding success [2]. LYMB was a 3D graphics, Smalltalk-like object-oriented system written in the C programming language, with a simple yet powerful interpreted scripting language, with expressive object-oriented designs (such as the lights, cameras, actors model found in VTK to this day). The book established the Object Modeling Technique (OMT) [3], resulted in the printing of hundreds of thousands of copies, and brought positive and deserved acclaim and recognition to GE Research.

Actually, upon reflection, there was a third major influence: The desire to collaborate with other researchers and customers. As visualization and graphics researchers at GE Research, the three principal authors of the original book–Will Schroeder, Ken Martin, and Bill Lorensen (see figure)–were constantly reaching out to others for data, and embedding 3D graphics and visualization methods into various applications (got golf? [4]). Working in a commercially-oriented, proprietary environment made collaboration complicated. So the seeds of “community” were born, although if you asked us at the time we weren’t fully aware of what that meant, or how to go about building a community. But it did mean that we adopted a permissive open source license, a huge decision that was central to VTK’s success. (This was in the early 1990’s when open source was an up and coming movement – we were indeed fortunate to have been part of the emergence of OS software approaches.)

VTK emerged from this fertile ground soon after. LYMB, while wonderful, was performance limited, and the rise of C++ was appealing to us as technologists. GE management was quite positive about books after the success of the OMT book. Our desire to collaborate was strong, and with the success of algorithms like Marching Cubes [5] and Decimation [6] we knew we had something good going. So after asking for and receiving formal permission from GE management to write (on our own time) a book and include some open source “demonstration” software, we began to write  the VTK Textbook “The Visualization Toolkit: An Object-Oriented Approach to 3D Graphics” with Prentice-Hall as the original publisher [7].

What happened next was surprising in its speed and diversity of action: the coalescence of talented people from around the world into a community. While we will say more about this is a later blog post, here are some highlights. Users from a variety of application areas began experimenting and providing feedback. External developers fixed, improved, and added new classes and implementations. Customers, and this includes GE businesses, paid for and created new VTK classes and capabilities (and assigned copyright). Professors used the book to teach visualization. And we were able to get the word out through courses at conferences, and technical publications. (In fact, the 1996 VTK publication in the IEEE Visualization Conference proceedings [8] just recently received the prestigious IEEE “Test-of-Time” award [9]). And we added new co-authors to following editions of the book (Lisa and Rick Avila brought volume rendering; Charles Law the beginnings of VTK’s imaging pipeline.) Eventually the demand for support, technology development, and software integration condensed into a business opportunity which was realized when Kitware was founded in 1998 – more about this in later blog posts.

An under-appreciated influence on VTK’s success is that it was expertly engineered from its inception. To this day, much of the VTK community has exceptional software and system integration skills. In some ways this engineering proficiency was surprising since VTK emerged from a research environment, and it is not uncommon for research software to focus on algorithmic implementation, not software engineering. Meaning that research software is often buggy, non-portable, hard to use, and arcane. However from its humble beginnings, software process was essential to VTK, with the notion of running test suites and regression testing built-in to the system. Later on, folks like Bill Hoffman would take these early concepts and develop tools like CMake [10], CTest, and CDash, central to many software processes in the world today, and significantly contribute to VTK’s success. Besides the resulting portable, cross-platform build system and testing process, other important engineering features relate to usability: such as VTK’s wrapping interface to languages such as Python and Java. Early on Ken built a rudimentary wrapping system, which has since been greatly expanded and improved by people like David Gobbi. Such interfaces to VTK mean that users can rapidly build applications without the complexity and long development cycle times of C++. The result is a larger community, with broader application to a range of technical computing fields.

In the early days of developing VTK there was also a palpable vision of future possibilities. This was personified in Bill Lorensen who had the extraordinary gifts of combining technical abilities with extremely high emotional intelligence. He was the sort of person that was the center of attention in meetings: not only could he hold court on a wide range of technical discussions, but made both newcomers and old timers feel welcome, all the while having a wonderful time doing it. As far as vision goes, one story in particular captures Bill’s spirit. At a large technical conference, an unnamed visualization company at the time asked for a meeting with the three VTK authors. During the meeting with two of the company’s marketing and technical executives, they asked us about future plans for VTK (obviously worried about the impact of VTK on their products). Without missing a beat, Bill bluntly declared “World Domination”. What brought down the house (for us anyway) in the silence that followed was that one of the executive’s faces engaged in a series of pronounced facial twitches–it took everything we had to retain some semblance of composure. That spirit of ambitious vision and fun, and the joy of collaborating with others, is something that lives on today in VTK. With World Domination still a major goal and promise of even better things to come, we expect VTK to continue its impactful influence across many more future birthdays.

Later years saw VTK grow in many directions. The early implementations of VTK algorithms were often slow. A good example is the initial volume rendering, using the high-level dataset abstractions, took many minutes to produce an image. (Of course today this can be done in tens of frames per second.) Fundamental capabilities such as interaction widgets and selection capabilities were completely missing. Several of the US National Labs, with a vision towards creating the research platform ParaView [11], worked to add parallel computing support, and support for composite data sets such as AMR. The 3D Slicer medical computing system [12] and its associated community pushed the system to better support medical applications. And throughout the decades, dogged and talented engineers worked to reimplement data structures and algorithms, create new ones when technological demands warranted it, and establish an amazing build system and software process based on the CMake family of tools. Documentation too was greatly improved- while the original textbook was followed by a user’s guide, folks like Andrew Maclean built on the repository of  VTK examples to create a comprehensive learning resource, and led an effort to convert the original VTK textbook into an online, open source LaTeX book.

Current Status

Currently in 2024, VTK has grown to over 5 million lines of code [13] depending on how you count. It is downloaded over 2.5 million times a year from Kitware’s servers (this does not count git clone and other accesses to the software). It serves as a foundational toolkit to dozens of major applications, with thousands of users using it for teaching, research, and application development. VTK is a vital engine for innovation, and is embedded in many commercial systems and used by start-ups to quickly create quality data analysis, graphics and visualization applications. VTK, through systems such as ParaView, runs on the largest supercomputers in the world; through web tools like trame [14], it runs on mobile devices. It is used to improve human health, provide security, build better products, investigate the frontiers of science, and provide an example of what a good large-scale software process looks like. It’s been quite a 30-year ride, but it’s clear that the VTK community has made amazing impacts far beyond the author’s original desire to dabble in C++, build cool software, and collaborate with fellow technologists. For these and many other reasons, VTK’s 30th birthday is worthy of a celebration. And if you’d like to celebrate with us, I’d recommend following Bill Lorensen’s [15] lead by taking a beer in hand, celebrate the community, and ponder the next steps towards World Domination.

References

[1] LYMB Object Oriented Graphics System.
[2] J. Rumbaugh, M. Blaha, W. Premerlani, F. Eddy, and W. Lorensen. Object-Oriented Modeling and Design. Prentice-Hall. 1991.
[3] Object Modeling Technique.
[4] Golf Green Visualization.
[5] W.E. Lorensen and H. Cline. Marching Cubes: A High Resolution 3D Surface Construction Algorithm. SIGGRAPH ’87. Association for Computing Machinery. 1987.
[6] W.J. Schroeder, J.A. Zarge, and W.E. Lorensen. Decimation of Triangle Meshes. SIGGRAPH ’92. Association for Computing Machinery. 1992.
[7] W.J. Schroeder, K.M. Martin, and W.E Lorensen. The Visualization Toolkit: An Object-Oriented Approach to 3D Graphics, First Edition, Prentice-Hall, 1996.
[8] W.J. Schroeder, K.M. Martin and W.E. Lorensen, The design and implementation of an object-oriented toolkit for 3D graphics and visualization. Proceedings of Seventh Annual IEEE Visualization ’96, San Francisco, CA, USA, 1996.
[9] IEEE VTK Test of Time Award.
[10] CMake.
[11] ParaView.
[12] 3D Slicer.
[13] VTK Statistics.
[14] trame: An Open Source Web Framework for Visual Analytics.
[15] Bill Lorensen: A Life Well-Lived.