Introducing glTF 2.0 support in VTK and ParaView

The glTF file format

glTF is an API-neutral royalty-free specification for the efficient transmission and loading of 3D scenes and models by applications. It is specified by Khronos Group (the consortium behind standards such as OpenGL, Vulkan and more) and is intended as a replacement for the Collada format. The glTF 2.0 format aims to be an efficient, interoperable and extensible asset delivery format that compresses the size of 3D scenes and minimizes runtime processing by applications using WebGL and other APIs. glTF 2.0 also defines a common publishing format for 3D content tools and services.

The glTF 2.0 specification supports modern features such as animations, skinned skeletons and morph targets. It also includes a specification for PBR (physically-based rendering) materials.

This format has gained a lot of momentum, and is now adopted by major actors, which is why VTK should be able to read and write files using it.

The new reader and importer in VTK

We have recently added glTF 2.0 support in VTK, in the form of a reader and an importer. Remember that in VTK, a reader is a filter that produces a data object (like a mesh or a multi-block dataset for instance), while an importer is a class that reads data and configures its representation in a renderer. As the glTF file format can be improved with file format extensions, we used an extensible architecture that should allow us to support them in the future.

The importer, vtkGLTFImporter, outputs fully textured scenes, and cameras. It also implements the KHR_lights_punctual extension. This extension allows glTF 2.0 models to contain punctual light sources. The light sources are converted to VTK lights, then added to the renderer.
It does not, however, animate models, as VTK importers do not support this functionality.

The reader, vtkGLTFReader, outputs a composite dataset that contains skinned, morphed and animated geometry, including all the attributes that could be defined in the glTF file. VTK readers only output geometry, but the glTF reader exposes functionality to indirectly access and apply textures to its geometry output.

Skinned and animated glTF 2.0 model, loaded with the VTK glTF reader (Original model from Cesium with modified textures, CC 4.0).

Morphed and animated glTF 2.0 model, loaded with the VTK glTF reader.

A file that uses the glTF 2.0 format can contain multiple animations of different lengths with no information on how these animations should be used at runtime. This is why the reader does not play any animations by default. Users can, however, choose to enable any of the available animations using the reader’s interface. They can also specify the rate at which those animations will be sampled. When an animation is enabled, the reader will start acting as a temporal source, and provide time steps for the different animation frames, using the user-specified sample rate, effectively enabling animation playback.

Note that deformations such as skinning and morphing are applied directly to the output geometry, using CPU calculations. Those deformations can be turned off, in which case the information necessary to compute those deformations is included in the output dataset as field data. We are currently working on a custom mapper that uses this field data to compute deformations on the GPU, and automatically apply textures.

Unrelated to this work, it should be mentioned that a glTF 2.0 exporter is also available in VTK, which can export geometry from a scene.

Support in ParaView

This new VTK functionality also lets us create a glTF 2.0 reader plugin for ParaView, which will be available in Paraview 5.7. The new plugin exposes the glTF reader, allowing users to load geometry from glTF 2.0 files, choose which of the glTF model’s scenes to display (in glTF, scenes define which visual objects should be rendered), toggle deformations, select and play animations.

Conclusion & future work

We hope that these features introduced with the support of the glTF file format will prove useful.
This work is actually a first step in glTF support within VTK and ParaView. We are currently working on the implementation of the aforementioned mapper and its integration into ParaView. This mapper should enable GPU deformations, and physically-based rendering features for glTF 2.0 models.

Acknowledgements

This work is part of the innovative effort of the SciViz team from Kitware SAS (France).