VTK-m Accelerated Filters in VTK and ParaView

Introduction

The first exascale capable computing machine is here¹ and more are on the way^2,3. Instead of the traditional CPUs, the primary source of computing power for these machines are going to be the next generation GPGPUs from Intel, AMD and NVIDIA. Therefore, legacy software, which does not have general-purpose GPU code, is not going to make efficient use of these machines. These devices have smaller processing cores but contain thousands of such cores. Algorithms that are best suited for this kind of architecture are ones that utilize fine-grained parallelism. To this end, we have been working on our next generation C++ visualization library, called VTK-m⁴, that uses fine grained parallelism to implement high performance visualization algorithms. The open-source VTK-m library is freely available here.

VTK and ParaView are two of the most popular visualization software. They are mature tools with an extensive set of features that are beyond the scope of the VTK-m library. The goal of the VTK-m library is to provide highly parallel implementations of visualization algorithms that can run on different types of parallel hardware. VTK/ParaView will use these implementations to accelerate their filters. Other features such as the visualization pipeline, rendering, IO, etc, will continue to be provided by VTK and ParaView. We have been gradually integrating VTK-m filters into VTK and several accelerated filters are available in VTK and ParaView right now.

In the following sections we will describe how the user can access these accelerated filters in VTK and ParaView. We will also provide a brief overview of how one can develop accelerated filters in VTK-m and enable them to be used in VTK and ParaView.

Using Accelerated filters in VTK

Direct instantiation of the wrapper filters

VTK-m accelerated filters in VTK are available in the VTK::AcceleratorsVTKmFilters module. This module consists of VTK filters that are thin wrappers over the corresponding VTK-m filters and can be used in regular VTK pipelines. Commonly used filters like Contour, Slice, Clip, Gradient and more are already available in this module. The following example pipeline demonstrates the use of the accelerated contour filter (vtkmContour):

vtkNew<vtkRenderer> ren;
vtkNew<vtkRenderWindow> renWin;
vtkNew<vtkRenderWindowInteractor> iren;

renWin->AddRenderer(ren);
iren->SetRenderWindow(renWin);

static constexpr int EXTENT = 30;
vtkNew<vtkRTAnalyticSource> imageSource;
imageSource->SetWholeExtent(-EXTENT, EXTENT, -EXTENT, EXTENT, -EXTENT, EXTENT);

vtkNew<vtkElevationFilter> ev;
ev->SetInputConnection(imageSource->GetOutputPort());
ev->SetLowPoint(-EXTENT, -EXTENT, -EXTENT);
ev->SetHighPoint(EXTENT, EXTENT, EXTENT);

vtkNew<vtkmContour> contour;
contour->SetInputConnection(ev->GetOutputPort());
contour->SetInputArrayToProcess(0, 0, 0, vtkDataObject::FIELD_ASSOCIATION_POINTS, "RTData");
contour->SetValue(0, 200.0);
contour->SetValue(1, 220.0);
contour->ComputeScalarsOn();
contour->ComputeNormalsOn();

vtkNew<vtkPolyDataMapper> mapper;
mapper->SetInputConnection(contour->GetOutputPort());
mapper->ScalarVisibilityOn();
mapper->SetScalarModeToUsePointFieldData();
mapper->SelectColorArray("Elevation");
mapper->SetScalarRange(0.0, 1.0);

vtkNew<vtkActor> actor;
actor->SetMapper(mapper);

ren->AddActor(actor);
ren->ResetCamera();
renWin->Render();
iren->Start();

Using VTK Factory to override existing filters

A recently implemented method to access the accelerated filters, enables using the VTK factory mechanism to override the existing VTK filters with their accelerated counterparts. This is possible because many of the filters in the VTK::AcceleratorsVTKmFilters module are sub-classes of the corresponding filters in VTK. For example, vtkmContour is a subclass of vtkContourFilter. That means that anywhere that the vtkContourFilter is being used, vtkmContour can be used instead and speed up the processing. This also allows us to automatically switch existing pipelines that contain such filters to use the accelerated filters without needing to modify the pipeline. Consider the pipeline in the example above. If the line: vtkNew<vtkmContour> contour; was instead: vtkNew<vtkContourFilter> contour; you will still benefit from accelerated processing because the VTK factory mechanism will actually instantiate the subclass vtkmContour.

The factory override mechanism for VTK-m based filters needs to be enabled at build time by turning on the CMake option VTK_ENABLE_VTKM_OVERRIDES. Also, the module that provides the override has to be linked to the module containing the overridden class, for the factory mechanism to work as expected. The overrides can also be turned on or off (on by default) at run-time using a global flag accessed through the class vtkmFilterOverrides. This allows accelerated processing to be toggled just by updating the vtkmFilterOverrides flag, without accessing the pipeline itself.

One thing to note is that not all of the features supported by a VTK filter are supported by the accelerated version of that filter. In such cases, the accelerated filter would fall-back to the default implementation to maintain compatibility. Moreover, due to the different algorithms used, the results of the accelerated filters will not 100% match the default implementation. Some minor differences should be expected.

Using Accelerated filters in ParaView

VTKmFilters Plugin

The filters in the VTK::AcceleratorsVTKmFilters module are also available in ParaView through the VTKmFilters plugin. The plugin may need to be explicitly loaded by accessing the plugin manager under the menu: Tools|Manage Plugins…. In the Plugin Manager, select VTKmFilters in the list and click “Load Selected” to load the plugin. Now the Filters menu should have a VTKm sub menu with the accelerated filters.

Using Factory Overrides

The factory override mechanism, described for VTK above, is also available in ParaView. When ParaView is built with the CMake option VTK_ENABLE_VTKM_OVERRIDES, some of the default ParaView filters will automatically use the accelerated filters. Please refer to the release notes to see which filters currently have overrides available for them. Similar to VTK, the override mechanism can be turned on or off at runtime using the ParaView setting under Settings|General|Use Accelerated Filters. This setting is shown irrespective of whether ParaView was built with the override support. To check if the overrides are actually available, look for “Accelerated filters overrides available” in “About ParaView” under “Client Information” (look under “Connection Information” for server support) . “About ParaView” can be accessed through menu Help|About…)

Using your own VTK-m accelerated filters in VTK and ParaView

If you have a filter that can benefit from fine grained parallelism, consider implementing that filter in VTK-m. VTK-m features a write once and compile anywhere functionality, where a filter is implemented in C++ using VTK-m’s parallel algorithms and scheduler, and it can be compiled to target different hardware backends, like the NVIDIA, Intel and AMD GPUs. Some good resources to get started developing in VTK-m are:

1. The VTK-m user guide⁵
2. The VTK-m tutorial⁶
3. The examples in the VTK-m repository

If you have any issues, feel free to discuss them on the VTK-m mailing list at: vtkm@public.kitware.com

Once you have a VTK-m filter, a VTK wrapper filter should be implemented. If there already exists an implementation of the filter in VTK, please consider inheriting that filter so that your filter can also be available through the factory override mechanism. 

The input VTK datasets need to be converted to VTK-m data structures before VTK-m can do any processing. The output of the VTK-m filter also needs to be converted back to VTK dataset formats, before returning the results. Conversion routines are available in the VTK::AcceleratorsVTKmCore and VTK::AcceleratorsVTKmDataModel modules. To avoid copies, in most cases, these conversion routines only move ownership of the underlying arrays from VTK to VTK-m and back. Where unified memory space between host and the device is available and enabled, even host-to-device and device-to-memory copies are minimized.

The wrapper filters aren’t very complicated. Already existing filters can be used as a guide to implement your own wrappers. The following steps describe the processing that should be performed inside a wrapper filter:

1. Check the filter parameters and only proceed with VTK-m processing for configurations supported by the VTK–m filter implementation.
2. Convert the input VTK datasets to VTK-m datasets.
3. Execute the VTK-m filter on the data.
4. Convert the output of the VTK-m filter back to VTK datasets
5. If at any point during the above steps, there is an error, fall-back to the default VTK implementation. Errors in VTK-m are typically signaled via C++ exceptions.

If factory override support is desired, add the override information in the VTK::AcceleratorsVTKmFilters module’s CMake file using the CMake function:

_vtkm_add_override("vtkBaseFilter" "vtkmAcceleratedFilter")

Making the filter available under ParaView follows the same process as exposing any other VTK filter in ParaView⁷. Nothing special needs to be done. If the filter overrides an existing filter in VTK and it is already exposed in ParaView, the existing override mechanism will automatically handle the turning on and off of the override based on the ParaView setting mentioned above.

Conclusion

With the integration of VTK-m, VTK and ParaView are becoming exascale ready. 

Acknowledgements

This research was supported by the Exascale Computing Project (17-SC-20-SC), a joint project of the U.S. Department of Energy’s Office of Science and National Nuclear Security Administration, responsible for delivering a capable exascale ecosystem, including software, applications, and hardware technology, to support the nation’s exascale computing imperative.

Thanks to Ryan Krattiger and Corey Wetterer-Nelson for building and running ParaView with the overridden filters on Crusher and generating the image above.

References