The Scientific Computing and Imaging (SCI) Institute and Kitware: Putting Software Sustainability into Practice

Authors:

SCI Institute, University of Utah: Jess Tate, Daniel White, Alexandra Warner, Ross Whitaker, Christopher Johnson, Rob MacLeod

Kitware: Will Schroeder, Jean-Christophe Fillion-Robin, Dženan Zukić, Matt McCormick, and Lee Newberg

The Scientific Computing and Imaging (SCI) Institute at the University of Utah, and Kitware, Inc., are working together to transition leading edge biomedical computing software to sustainable open source software communities. The origin of these software systems was in large part funded by the National Institutes of Health (NIH) P41 Center for Integrative Biomedical Computing (CIBC) [1] led by SCI along with collaborators at Northeastern University. The CIBC, established in 1998, has reached the NIH limit on P41 centers, and we have been awarded an NIH grant supporting the transition of these research tools into community supported resources to ensure that they continue to remain impactful well into the future.

Software sustainability is an increasing concern to the scientific research community and may also affect the long-term viability of commercial systems [2]. Since science is built on a foundation of reproducibility, the computational methods and data used to produce results must be available to current and future practitioners. Similarly, much of today’s computing infrastructure is built on open source systems; without ongoing support, detrimental impacts to business activity have already occurred and will likely occur again. Recognizing these concerns, many organizations including the NIH are actively supporting the transformation of software into more sustainable forms. The goal is to ensure that once a funding cycle completes, the appropriate resources, communities, and outreach exists to ensure that software continues to make positive impacts well into the future.

To increase the sustainability of the many biomedical computing tools developed through the CIBC, Kitware and SCI created a formal evaluation process based on a software sustainability matrix [2]. This matrix measures metrics in the four general categories of: Impact, Risk, Community, and Technology. The metrics are assigned values and combined to produce a final sustainability score. While admittedly some of the metrics are subjective in nature (e.g., how do you assess the perceived value of software?) they are helpful in that they reveal areas in which sustainability improvements can be made. This scoring is enabling us to work together to focus our efforts to efficiently improve the software and position it for a sustainable future.

SCI has a history of coupling world-class research with exemplary software processes that are underutilized in academic work. For example, SCI relies on open source licensing, good documentation practices, software testing dashboards, CMake build tools, and modern git-based repositories for all of its systems. This has provided a leg up on sustainability; however there are technical challenges such as the emergence of web-based applications that may drive the team to repackage systems into client-server applications to ensure they are widely used in the future. Such attention to software practices is unusual in many academic labs, mostly because such institutions are measured on research output such as publications. However, as software becomes larger and more complex, it is becoming increasingly untenable for fresh students and postdocs to create entirely new systems – out of necessity they must build their research on top of proven software foundations. Again, this argues for sustainable, robust platforms within which users can develop and contribute new algorithms and methods.

In the first year of our three year project, SCI and Kitware are focusing on two major CIBC software products: Seg3D [3] and Cleaver [4]. Both systems have high perceived value and are already used very broadly by the biomedical computing community. In subsequent years we will focus on additional CIBC systems. One of our goals is to engage with the research community and integrate these packages with existing, popular open source software systems such as VTK [5], ITK [6], 3D Slicer [7], and ParaView [8], and to ensure they are usable from scientific computing tools such as Python. Such integration will expose the CIBC software to more users; our intention is to increase community size thereby increasing its sustainability.

As part of these efforts, we welcome contributions and involvement from others. Please feel free to contact Rob MacLeod (SCI), Jess Tate (SCI),  Jean-Christophe Fillion-Robin (Kitware), or Will Schroeder (Kitware) if you would like more information or would like to contribute.

Acknowledgements

This work is largely funded by a grant from the National Institute of General Medical Sciences of the National Institutes of Health: R24 GM136986.

References

[1] Center for Integrative Biomedical Computing (CIBC). https://www.sci.utah.edu/cibc.html.
[2] Will Schroeder, Michael Grauer, Matt McCormick, Marcus D. Hanwell and Jean-Christophe Fillion-Robin. How Sustainable is Your Software? https://blog.kitware.com/how-sustainable-is-your-software.
[3] Seg3D: Volumetric Image Segmentation and Visualization. http://www.seg3d.org.
[4] Cleaver: A MultiMaterial Tetrahedral Meshing Library and Application.  https://www.sci.utah.edu/cibc-software/cleaver.html.
[5] The Visualization Toolkit VTK. vtk.org.
[6] The Insight Segmentation and Registration Toolkit ITK. itk.org.
[7] 3D Slicer for medical image informatics, image processing, and visualization. https://www.slicer.org.
[8] ParaView multi-platform data analysis and visualization application. paraview.org.