Kitware announces the release of LookingGlassVTKModule version 1.0.  This new release supports new Looking Glass Factory displays, cinematic volume rendering, and Python!

Background

Looking Glass Factory, the global leader of holographic displays, has introduced the world to 3D holograms that do not require headsets or glasses. Their proprietary light field technology engages parallax to create a realistic 3D experience, thereby allowing multiple users to simultaneously see the same data in 3D from different perspectives. Looking Glass now offers a wide range of holographic display formats, from professional 65” monitors to the personal Looking Glass Portrait. 

Kitware’s visualization toolkit (VTK) streamlines the visualization of scientific data on a multitude of platforms from mobile devices to AR/VR systems, in support of cutting-edge research and end-user product development. For several years, Kitware and Looking Glass have been working together to provide support for holographic displays in VTK via the LookingGlassVTKModule. Our collaboration is now proud to announce the release of version 1.0 of the LookingGlassVTKModule . This release offers many enhancements for developers and users, including support for Looking Glass’ newly released displays and display drivers as well as VTK 9.2.2’s newly released cinematic volume rendering capabilities and easy-to-install and easy-to-use Python packages.  These enhancements are summarized below.

Enhancements

Python Packages: Simplified Installation and Use

Although VTK has supported holographic rendering since version 9.1, it was not included in VTK’s compiled packages. So until now, a user would either have to access the Looking Glass features through the ParaView plugin (starting in ParaView 5.9) or by building VTK locally with the relevant options enabled.

Recent updates to the VTK build system, however, have allowed the Looking Glass features to be packaged as a Python module. As such, VTK’s Looking Glass features are now easily accessible via PyPI, and may be installed with pip:

pip install vtk-lookingglass

This command also automatically installs the specific version of VTK that is required by the module. The current vtk-lookingglass module requires the recently released VTK 9.2.2.

After the vtk-lookingglass package is installed, a Looking Glass render window may be created in Python using only a few lines of code:

from vtkmodules import vtkRenderingLookingGlass as vtklg
# Create a looking glass render windowrenWin = vtklg.vtkLookingGlassInterface.CreateLookingGlassRenderWindow()

This render window may be used in the place of a regular vtkRenderWindow. In this way, many of the VTK examples can be easily modified to use a vtkLookingGlassRenderWindow in order to display them on a Looking Glass device.

Simplified Interface

Several functions have been added to the vtkLookingGlassRenderWindow interface. Many of these features were available before, but were hidden inside the vtkLookingGlassInterface class. Exposing them on the render window class simplifies the process of accessing them. Some examples can be seen below.

# Setting clipping planes
renWin.SetNearClippingPlane(0.8)
renWin.SetFarClippingPlane(1.2)

# Saving a quilt image
# The suffix encodes the tile pattern, such as "_qs5x9"
# This allows software to automatically determine the tile pattern
suffix = renWin.QuiltFileSuffix()
renWin.SaveQuilt(f"quilt{suffix}.png")

# Creating a quilt movie
suffix = renWin.QuiltFileSuffix()
ext = renWin.MovieFileExtension()
renWin.StartRecordingQuilt(f"quilt{suffix}.{ext}")
num_frames = 30
for i in range(num_frames):
    # Change something about the data or the camera
    …
    renWin.Render()  # a movie frame is saved for each render of the render window
renWin.StopRecordingQuilt()

Note that movie files may need to be converted to a format compatible with HoloPlay Studio.

Device Selection

Device selection enables users to create quilts for devices not connected to their computer. This is especially helpful for generating a quilt to send to someone who has a different type of device. For example, if you do not have a portrait device, but want to create and send a quilt for a portrait device to someone else, the following can be done:

# Change devices
renWin.SetDeviceType("portrait")

# Must render before writing out the quilt
renWin.Render()

# Write out the quilt
suffix = renWin.QuiltFileSuffix()
renWin.SaveQuilt(f"quilt{suffix}.png")

Quilts for each type of device may also be generated and distributed so that the hologram can be displayed on any type of device.

# Loop through each device type
for device_type in renWin.GetDeviceTypes():

    # Switch to the device
    renWin.SetDeviceType(device_type)

    # Must render before writing out the quilt
    renWin.Render()

    # Write out the quilt
    suffix = renWin.QuiltFileSuffix()
    renWin.SaveQuilt(f"quilt_{device_type}{suffix}.png")

Examples

The LookingGlassVTKModule can be used in a variety of ways, including in Jupyter notebooks as well as regular Python scripts. Several examples are currently available in the GitHub repository, which include a Jupyter notebook example that renders a simple interactive cone, Python scripts that demonstrate creating quilt images and movies, and an example of changing device types.

Cinematic Rendering

Recent advances in light scattering models in VTK have led to photo-realistic (or “cinematic”) rendering of volumetric data. Cinematic rendering of volumes is of particular value in medical computing, where volumetric data is abundant and realistic renderings are highly beneficial.

We have created several examples of holographic renderings of medical volumes, which may be found here. One example renders a CT scan of a patient with a tumor at the base of their neck. 

This work was funded, in part, by the NIH via NIBIB and NIGMS R01EB021396, NIBIB R01EB014955, NCI R01CA220681, and NINDS R42NS086295.