3D Slicer Adds Volumetric Mesh Support

February 1, 2017

Alexis Girault, Jean-Christophe Fillion-Robin, Andras Lasso and Sandy McKenzie

Earlier last week, support for volumetric mesh was integrated in 3D Slicer. The work was initiated, discussed and mostly completed during the National Alliance for Medical Image Computing (NA-MIC 2017) Winter Project Week [1] at the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT) in Cambridge, MA. Highlights and major contributions of the project week are summarized in the post “Deep learning and web technologies are the focus of 3D Slicer 2017 Winter Project Week.”

Alexis Girault (Kitware, Inc) led the effort to add volumetric mesh support as a 3D Slicer core feature. The project week allowed him to meet with some of the 3D Slicer primary investigators. Andras Lasso (PerkLab, Queen’s University), Jean-Christophe Fillion-Robin (Kitware, Inc), Curtis Lisle (KnowledgeVis, LLC), Csaba Pinter (PerkLab, Queen’s University) and Steve Pieper (Isomics, Inc), among others, gave useful advice and helped design the infrastructure described below. Details of the collaborative reviewing effort can be found on the associated Github pull request.

Volumetric Mesh

As outlined on the corresponding Wikipedia article, volumetric meshes are a polygonal representation of an object structure. Unlike surface meshes, which only define the outside layer of an object, volumetric meshes also discretize the interior structure of the object. One application of volumetric meshes is in finite element modeling. Figure 1.a and Figure 1.b (below) showcase a clipped surface mesh and its associated volumetric mesh. The figures use the latest improvements integrated into 3D Slicer nightly, which are described further down in this post.

Figure 1.a: Clipped surface representation showing outside layer of segmented mouse head

Figure 1.b: Clipped volumetric representation showing internal cells of segmented mouse head

History of Mesh Support in Slicer

Volumetric meshes are an important feature. They were introduced in 3D Slicer version 3.4 through the NA-MIC National Consortium of Breast Centers collaboration “Automated FE Mesh Development.” This collaboration resulted in the creation of IA_FEMesh, a finite element meshing module from the University of Iowa, which encapsulates tools to create a volumetric mesh, to assign material properties and boundary conditions (such as external forces) to its elements and to export it in the Abaqus format in order to perform finite element analysis. The resulting mesh was, however, not part of the Medical Reality Markup Language (MRML) architecture. Therefore, it was not easily accessible to other 3D Slicer modules.

Fig. 2: Snapshot of the IA_FEMesh module in 3D Slicer 3.4

Due to resource allocation and priorities, support for volumetric meshes was not ported to the 4.0 release of 3D Slicer; only surface meshes made the cut as part of the Model MRML node class. Nevertheless, some work was done in more recent versions of 3D Slicer to offer mesh-creation capabilities, like the below illustrated Body Centric Cubic Mesh (BCCM) and CleaverExtension.

Fig. 4: Resulting mesh of CleaverExtension

Need for Volumetric Mesh in Optical Imaging

Fig. 5: NIRFAST MATLAB displaying a finite element model that is based on a label map and fiducial inputs created in NIRFAST-Slicer

NIRFAST is an interactive open-source software application package based on MATLAB that is widely used in the optical imaging research community. NIRFAST provides functionalities for modeling and reconstructing near-infrared light transport in tissue from optical data, which is acquired by tissue spectroscopy, to recover internal functional parameters such as blood oxygen saturation, water content, and lipid concentration [2]. These measurements can provide valuable information for detecting breast cancer and analyzing imaging brain functions, among other applications.

Fig. 6: NIRFAST-Slicer showcasing reconstructed optical parameters from NIRFAST, resampled and overlaid above a structural image of a cancerous breast

The optical modeling and image reconstruction in NIRFAST uses finite element modeling, which requires structural information that can be provided through multi-model methods such as magnetic resonance imaging (MRI) or computed tomography (CT) guided optical spectroscopy [3]. The NIRFAST primary investigators at Thayer School of Engineering at Dartmouth College reached out to Kitware to help them put together segmentation tools for creating suitable volumetric meshes of complex tissue geometries. The current solution is a 3D Slicer based custom application named NIRFAST-Slicer, which is described in more detail in the blog post “Kitware customer highlight: Optics in Medicine Laboratory.”

One of the requirements of NIRFAST is to visualize the optical data overlaid above the coupled structural image used during the guided optical spectroscopy. Due to the lack of volumetric mesh support in 3D Slicer 4.6, it was necessary to resample the output finite element mesh from NIRFAST (Fig. 5) within its coupled structural image structure (e.g., offset, spacing and dimensions) to be able to visualize the optical data in NIRFAST-Slicer (Fig. 6). This process was slow for a high number of elements, introduced interpolation errors and forced the user to go through additional steps. These limitations exposed the need to directly display the optical data in its raw format, as a finite element structure. This need motivated the improvements for volumetric mesh support that are available today in the 3D Slicer Nightly build and will be offered soon in the 4.7 release.

Improvements to 3D Slicer

Surface meshes were represented in 3D Slicer 4.6 by the MRML model node class, which served as a thin wrapper over the Visualization Toolkit (VTK) polygon mesh class (vtkPolyData). Since surface and volumetric meshes share a lot of properties, the development team decided to support volumetric meshes by generalizing MRML model nodes to be able to store both surface and volumetric meshes. Model nodes now contain a vtkPointSet pointer, which can either refer to a vtkPolyData (for surface mesh) or to a vtkUnstructuredGrid (for volumetric mesh). This generalization was carried out with minimal changes in the interface of the model node.

Fig. 7: The Models module Information widget shows the mesh type for a volumetric mesh.

The storage node for Models was extended to support reading and writing of both .vtk (legacy) and .vtu (XML) files as well as to make to identify a .vtk surface mesh file (Fig. 8). Leveraging the existing infrastructure, those changes also allow the passing of Models containing volumetric meshes between 3D Slicer and command-line interface (CLI) modules.

Fig. 8: 3D Slicer data-saving widget showing appropriate export file formats based on the inner data structure of the MRML model node

The display node for Models was refactored to avoid inefficiencies, simplify the code and bring visualization consistency within the 3D view and the 2D views. Clipping functionalities were maintained using the vtkExtractGeometry filter (instead of vtkPolyDataClipper for surface mesh), allowing users to see the internal structure of the volumetric mesh based on the slice positions. The projection representation was also maintained. It allows users to visualize interpolated values in 2D slice views by making use of the vtkCutter filter (Fig. 9).

Fig. 9: Volumetric imaging of fluorescent tracers in a mouse head, acquired through MRI coupled fluorescence molecular tomography

Improvements to the MRML model display node were also made to consolidate the selection of the scalar range mode and to dynamically adapt its precision in the module widget for better usability (Fig. 10).

Fig. 10: Models module display widget with improved scalar range mode selection and dynamic range precision

Finally, a new addition was brought to the MRML model display node to threshold the volumetric mesh based on its active scalar values (Fig. 11). This feature helps to hide cells, which hold data that can be ignored for visualization purposes.

Fig. 11: Thresholding volumetric imaging of fluorescent tracers in a mouse head improve overlaid visualization with the structural image underneath

Future Work

These recent contributions offer room for potential future improvements, such as the following:

adding an opacity option for the 2D slice views [Mantis];
adding a “smooth” clipping option to show a flat surface when clipping the volumetric mesh in the 3D view, instead of removing cells [Mantis];
cleaning up and consolidating the Models module interface for usability [Mantis]; and
implementing modules to convert the volumetric mesh model node to other MRML nodes such as surface mesh model, segmentation, and label volume. Converting those MRML nodes to a volumetric mesh model can be done by adapting the previous mesh-creation tools referenced above (IA_FEMesh, the BCCM module, and CleaverExtension) to rely on these new and streamlined core functionalities [Mantis].

Summary

With the collaboration of multiple 3D Slicer community members reunited during the NA-MIC 2017 Winter Project Week, volumetric mesh visualization support was brought back to the 3D Slicer 4.x series.

The basic capabilities gravitate around an enhanced MRML model node class, which now wraps a vtkPointSet data structure, a base class of vtkPolyData (surface mesh) and vtkUnstructuredGrid (volumetric mesh), as well as improved display functionalities for clipping, thresholding, 3D and 2D visualization and scalar range selection. These capabilities offer better consistency and usability.

This work will be used right away within NIRFAST-Slicer, a 3DSlicer based custom application that is part of the NIRFAST package. Based on tissue spectroscopy, NIRFAST-Slicer helps to recover internal functional parameters that can provide valuable information for diverse medical applications.

These features, which are already available in the Slicer Nightly build, will be a part of the 4.7 release of 3D Slicer and will facilitate upcoming work to bring more volumetric mesh capabilities (such as mesh creation and conversion to other data structures) to the platform. To download the software, please visit http://download.slicer.org/.

To learn more about how Kitware can tailor solutions to meet your research needs, please contact kitware@kitware.com.

Acknowledgments

This work was supported by the National Cancer Institute under award number R01CA184354 as part of NIRFAST-Slicer improvements for NIRFAST. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

References

[1] Kapur T, Pieper S, Fedorov A, Fillion-Robin JC, Halle M, O’donnell L, Lasso A, Ungi T, Pinter C, Finet J, others. Increasing the Impact of Medical Image Computing Using Community-Based Open-Access Hackathons: the NA-MIC and 3D Slicer Experience. Medical Image Analysis 2016.

[2] H. Dehghani, M.E. Eames, P.K. Yalavarthy, S.C. Davis, S. Srinivasan, C.M. Carpenter, B.W. Pogue, and K.D. Paulsen, “Near infrared optical tomography using NIRFAST: Algorithm for numerical model and image reconstruction,” Communications in Numerical Methods in Engineering, vol. 25, 711-732 (2009).

[3] M. Jermyn, H. Ghadyani, M.A. Mastanduno, W. Turner, S.C. Davis, H. Dehghani, and B.W. Pogue, “Fast segmentation and high-quality three-dimensional volume mesh creation from medical images for diffuse optical tomography,” J. Biomed. Opt. 18 (8), 086007 (August 12, 2013), doi: 10.1117/1.JBO.18.8.086007.

Tags:

3D Slicer Medical Imaging NIRFAST VTK