Rendering VTK into Unity

Kitware is pleased to announce the integration of VTK into Unity with the release of two assets on the Unity Asset Store: VTKUnity-MedicalViewer and VTKUnity-Activiz.

Introduction

There has been growing interest in integrating VTK within Unity over the past few years. Unity developers are interested in bringing the algorithms, data representation, scalability, and visualization techniques offered by VTK into the high quality and convenient environment that Unity provides. Unity also offers a quick and easy way to immerse 3D scenes in virtual environments.

Kitware has developed a plugin based on the low-level native plugin interface offered by Unity [1] which makes it possible to render a VTK scene into the Unity rendering pipeline. The VTK plugin takes advantage of VTK’s external rendering capabilities [2] to create, populate and update the VTK scene within Unity. This work was originally funded by P. Drzazga, Paul Sabatier University – Toulouse III, and adapted from the approach developed in the 3D Heart project [3, 4].

(Note that Kitware has previously developed another Unity asset for the Pulse Physiology Engine. Pulse is an open-source library for accurate, consistent, dynamic simulation of the whole-body response to injury, disease, and treatment.)

VTKUnity – Medical Viewer

The free VTKUnity-MedicalViewer asset demonstrates VTK’s rendering and processing capabilities inside a Unity application. This asset provides a minimal infrastructure required to create a medical image viewer in Unity.

This asset exposes a subset of VTK functionalities that have been wrapped into the VTK native plugin. This limited version allows users to:

• Visualize a volume with volume rendering
• Display anatomical planes
• Extract isosurfaces
• Show an image histogram

Try out this plugin with your own data using the sample scene (SampleScene.unity) to quickly visualize a volume. Supported file formats include:

• VTK XML Image (.vti)
• MetaImage (.mha/.mhd)
• DICOM folder (Path to a folder containing DICOM images)

A bug tracker is available to report any issues with this asset.

VTKUnity – ActiViz

The VTKUnity-Activiz asset provides full access to VTK functionalities. It uses to instantiate and manage VTK classes in a Unity C# script. With this asset, users can perform all the processing and visualization techniques VTK has to offer from within the Unity environment.

The VTK-Unity native plugin hosts and exposes a VTK renderer instance that can be accessed from Unity’s scripts and populated with actors using ActiViz. It also provides scripts to synchronize lights and cameras, as well as examples that demonstrate best practices to set up a VTK scene. Of note is the support for virtual reality (VR) – the Unity VR framework drives the VTK rendering pipeline to allow rendering in the immersive environments supported by Unity.

Contact

Please let us know how to improve the VTK Unity assets at kitware@kitware.fr, or use the bug tracker to report issues.

References

1. https://docs.unity3d.com/Manual/NativePluginInterface.html
2. https://blog.kitware.com/new-in-vtk-rendering-in-external-immersive-environments/
3. Gavin Wheeler, Shujie Deng, Nicolas Toussaint, Kuberan Pushparajah, Julia A. Schnabel, John M. Simpson, and Alberto Gomez. “Virtual interaction and visualisation of 3D medical imaging data with VTK and Unity.” Healthcare technology letters 5, no. 5 (2018): 148-153.
4. https://gitlab.com/3dheart_public/vtktounity