Volume Rendering on iOS and Android

Over the past two years, we have been working on updating VTK’s rendering engine to leverage the capabilities of more recent OpenGL implementations. An exciting part of that effort involved turning our attention to running VTK on iOS and Android. To get there we have added a lot of new capabilities to VTK including:

• We added support in the RenderWindowInteractor for multitouch events. We currently have built-in support for receiving and handling multitouch events on iOS, Android, and Windows.

• We developed a new multitouch camera joystick interactor style. This interactor style includes support for using two fingers to translate, rotate, and zoom the camera while also supporting the usual single finger motions for rotation.​​
• We completely rewrote VTK’s OpenGL rendering engine to support OpenGL version 3.2 and OpenGL ES versions 2.0 and 3.0. The new engine is much faster than the old and includes some neat new capabilities. Please read this source article for some more information on the rendering engine update.  The example in this blog post uses the new GPU-based volume mapper developed as part of this updating effort. You can find more information on our volume rendering improvements in this source article and this blog post. ​
• We added support for compiling VTK on iOS and Android. While still in a beta stage we have added in support for compiling VTK for Android and iOS platforms.

So how do we see it all in action? Well, we have created an example called VolumeRender which is tested on both Android and iOS. It can be found as part of VTK in

• VTK/Examples/iOS/VolumeRender for iOS
• VTK/Examples/Android/VolumeRender for Android

The example has a #if in the code that switches between using a synthetic dataset or reading from a file. The video was rendered using the CT-chest.nrrd medical dataset from Slicer converted to 8 bit and resized to 256x256x130.

To build it you will need a recent version of VTK (from November 2015 or later) and, as of this writing, it is best done on an OSX or Linux system. Volume rendering requires OpenGL ES 3.0 so make sure to select that during the CMake configure process and have a device to test on that support OpenGL ES 3.0. Most recent phones and tablets support ES 3.0. For Android, you will need both the Android SDK and NDK. For iOS, you will need XCode for the example.​ You should follow the latest step-by-step instructions on how to build for Android or iOS, however do note that the whole process is still a bit new so there will likely be a hiccup or two.

If you want to read from a file, you have to get the file onto your device. For iOS that means telling XCode that your app supports iTunes File Sharing (found in the Custom Target Properties area under the info section of Xcode for your app.) Then use iTunes to add a file to your app. For Android, you can use the adb command with the push option. Finally, here is a short video showing the application in action on an iPhone 6.

VolumeRender from Kitware on Vimeo.

We would like to recognize the National Institutes of Health for sponsoring this work under the grant NIH R01EB014955 “Accelerating Community-Driven Medical Innovation with VTK.”