VTK WebGPU on the Desktop
Introduction
When it comes to rendering data, computer graphics libraries and high-level APIs have long been a moving target. Hardware advances and improvements to programming models have resulted in periodic changes to the way in which data is rendered. Graphics libraries such as PHIGS, Starbase, GL, Direct3D, OpenGL (various flavors and versions), and many others have come and gone. This is why VTK was designed with an abstract graphics API (actors, lights, cameras) with the ability to link against one or more underlying rendering engines–it was expected that the evolution of graphics systems would proceed unabated. Our goal in designing VTK’s graphics subsystem was to ensure that users, developers, and applications could continue using VTK without major disruptions as graphics libraries evolved and changed over time. Given that VTK is 30 years old, and still under active development and widely used, this approach has, thankfully, shown to be successful.
The VTK community is now facing a barrage of change, and this time with a twist. Not only are major vendors adopting new graphics libraries, but in recent years computing platforms have undergone massive changes with the introduction of 3D graphics on the web. Consequently we are again at a point of significant change, with emerging technology and computing models necessitating an overhaul of VTK’s rendering subsystem. In this post, we provide some background to these pending developments, summarize our early progress, and lay out the path forward.
Current Status
VTK currently uses OpenGL standard as its cross-platform graphics API for rasterizing complex mathematical and analytical data models to screen. However, OpenGL is no longer being actively developed. The latest OpenGL specification, 4.6, was released in 2017, and was limited to a small subset of extension additions to the core profile. The OpenGL steering consortium has been working on Vulkan as the new cross-platform API meant to replace OpenGL. Apple deprecated and dropped OpenGL support in 2017 in favor of a new API called Metal. Microsoft uses it’s own Direct3D 12 (DX3D12) as the low-level API for graphics on Windows machines. Android devices and VR headsets do not allow complete access to graphics capabilities through the more limited OpenGL ES (Embedded Systems) API.
What does this mean for the future of VTK? It means that VTK would have to use Vulkan on Linux/Android, DirectX12 on Windows, and Metal on Apple devices by either implementing all of those backends or relying upon an external abstract rendering hardware interface (RHI) library.
All of these low-level APIs aim to provide a graphics paradigm that is closer in form to the hardware and allow better asynchronous computation on the GPUs they interact with. Programmable shaders, uniform buffers, texture objects, storage buffers, compute shaders, hardware ray traversal, etc. are some of the features that these new APIs provide and can be used by VTK for its visualization needs. Providing a new graphics backend relying on these low-level APIs would open up VTK to these advanced utilities. However, developing and maintaining multiple different backends (for each operating system and device) can incur heavy costs in terms of the skills necessary to implement the core graphics features on each backend, as well as significantly increasing the burden of maintaining them.
It would be ideal for VTK to rely on a single graphics abstraction layer that would help delegate platform specific details to the low-level APIs but provides a clean interface for VTK developers and users. This is where WebGPU comes in.
WebGPU is not just about the web
Introduced as WebGL-next in 2016, one of the main goals of WebGPU is to provide a lowest common denominator across different platforms. The API is designed to support the lowest end mobile devices as well as next-generation compute capable graphics cards. Browser implementers are developing their own RHI (Rendering Hardware Interface) according to W3C WebGPU specification. Those RHIs enable JavaScript WebGPU applications in Google Chrome, Microsoft Edge, Firefox and Safari. What’s interesting is that while the initial target of WebGPU was JavaScript, there are implementations available in native languages like C++ and Rust. From the very beginning, Google has been working on both native and web implementations of the API which is now called dawn. wgpu is Mozilla’s implementation written primarily in Rust with wrapped bindings available for other languages like Python, C, C++, etc. With growing interest, efforts are underway to unify the API implementation target to a common WebGPU C specification available as a single webgpu.h header. VTK aims to target the common header and allow users/developers to switch between supported implementations.
VTK everywhere – on desktop, mobile, VR and in the browser
VTK – Desktop, mobile and browser
With standalone WebGPU libraries in place, the rendering modules in VTK can include and link to any WebGPU implementation. All components of VTK rendering would be rewritten using WebGPU C API. These include vtkRenderWindow, vtkRenderer, vtkActor, vtkPolyDataMapper and various other mappers. A major benefit of doing so is that the same code can be reused on the web by using the Emscripten toolchain. This toolchain provides a one-to-one mapping from WebGPU C++ to WebGPU JavaScript APIs. Emscripten marshals the WebGPU function arguments from C++ to JavaScript objects while the WebAssembly program is linked using emscripten/src/library_webgpu.js.
VTK – Native rendering in VR headsets
Another benefit of using WebGPU is that VTK will be able to render natively into VR headsets. The current VTK implementation requires streaming OpenGL-rendered results into the headset’s native Vulkan/GLES/DirectX graphics context. With WebGPU, VTK will be able to avoid this workaround and leverage the full performance and feature set of DirectX/Vulkan without any compromise.
An experiment with VTK and WebGPU
As a start, we’ve implemented a polydata mapper using dawn. We’ve looked at a smaller subset of the WebGPU API limited to rendering pipelines. Here is a list of features currently available from VTK using WebGPU.
- Render surface meshes in the form of vtkPolyData.
- Map point and cell scalars to color values.
- Draw actors with the representations: VTK_POINTS, VTK_WIREFRAME, VTK_SURFACE and VTK_SURFACE with edge visibility.
- Light actors using VTK headlights in combination with point/cell normal vectors on vtkPolyData.
- Adjust point size. Points are rendered using two adjacent triangles.
- Adjust line width. Lines are rendered in fragment shader using techniques similar to those used by VTK with OpenGL.
- A single VTK render window for desktop and browser – vtkSDL2WebGPURenderWindow is a concrete implementation of vtkWebGPURenderWindow that works on WebAssembly and desktop. Platform native render windows using Xlib, Cocoa and Win32 are in the works!
- Depth testing.
Check out our guide to building VTK with WebGPU.
Future work
We’ve not yet exploited WebGPU compute pipelines to speed up rendering. Apart from polydata mapper, the composite polydata, glyph, volume, and surface LIC mappers are yet to be developed with WebGPU. In the coming year, we will begin to flesh these out. If you are interested in helping, the VTK community would welcome your contributions.
Conclusion
With the use of a rendering hardware interface library like WebGPU, VTK can be forward-compatible with new-generation graphics hardware and no longer be limited by the platform-specific quirks of OpenGL.
In an upcoming blog post, we’ll show an exciting performance comparison of VTK OpenGL against VTK WebGPU on Apple, Linux and Windows machines through Google Chrome developer tools!