Reducing the temporal impact of loading plugins in ParaView

Using delayed load plugins

Loading a ParaView plugin can take some time, up to a few seconds for the biggest of plugins.

If you use autoload for many of your plugins, especially big ones, you pay the loading time everytime you open ParaView, delaying the opening of ParaView by crucial seconds.

What if there was a way to load the plugins you need only when you need it ?

In ParaView 5.13.0, We are introducing delayed load plugins.

The idea is to load the plugin C++ symbols only when a filter, reader or source that is provided by this plugin is actually needed by the user by only loading the plugin proxies first.

To add a delayed load plugin, make sure to not load it directly but to add its plugin configuration file instead.

When you create a filter provided by this plugin, the actual loading of C++ symbols will take place, hence delaying the load when it is actually needed.

Please note that this mechanism should be enabled by the plugin developer and cannot be enabled for any random plugin available in ParaView.

Of course, it has some limitations. For starters, it is specific to C++ plugins, as python plugins have a different architecture. Secondly, it also means the client-side additions like dock panels, toolbars and other UI elements will not be visible until the plugin symbols are loaded, so this feature should not be used with such plugins.

Create a delayed load plugin

There are three key elements to support delayed load:

• Specify XML files at top level

Make sure that all your XML files are specified in the SERVER_MANAGER_XML argument of the paraview_add_plugin cmake macro and not specified using paraview_add_server_manager_xmls.

from: Examples/Plugins/ElevationFilter/Plugin/CMakeLists.txt

paraview_add_plugin(ElevationFilter  REQUIRED_ON_CLIENT
  REQUIRED_ON_SERVER
  VERSION "1.0"
  DOCUMENTATION_DIR "${CMAKE_CURRENT_SOURCE_DIR}/Documentation"
  DOCUMENTATION_ADD_PATTERNS
    "*.txt"
    "img/*.jpg"
    "js/*.js"
    "style/style.css"
  DOCUMENTATION_DEPENDENCIES DummyDocumentationCustomTarget
  DOCUMENTATION_TOC "${table_of_contents}"
  MODULES ElevationFilters
  MODULE_FILES "${CMAKE_CURRENT_SOURCE_DIR}/ElevationFilters/vtk.module")

• Add DELAYED_LOAD in the top level CMakeList

In the top level CMakeLists.txt of your plugin, list all delayed load plugin in the DELAYED_LOAD argument of the paraview_plugin_build macro, and make sure a PLUGINS_FILE_NAME is specified.

From: Examples/Plugins/ElevationFilter/CMakeLists.txt

paraview_plugin_build(
  RUNTIME_DESTINATION "${CMAKE_INSTALL_BINDIR}"
  LIBRARY_DESTINATION "${CMAKE_INSTALL_LIBDIR}"
  LIBRARY_SUBDIRECTORY "${PARAVIEW_PLUGIN_SUBDIR}"
  PLUGINS_FILE_NAME "elev.plugins.xml"
  DELAYED_LOAD ElevationFilter
  PLUGINS ${plugins})

• Ship all required files

Once the plugin has been built and installed, make sure to provide all the needed files, which are contained in install/lib/paraview-5.12/plugins

• the plugin configuration XML file
• the ServerManager XML files
• the actual plugin library file

With the expected folder architecture:
./elev.plugins.xml
./ElevationFilter
./ElevationFilter/MyElevationFilter.xml
./ElevationFilter/ElevationFilter.so
./ElevationFilter/libElevationFilters.a

Then the user should be able to add the elev.plugins.xml as a plugin configuration file as described above.

The complete example used above can be found here: https://gitlab.kitware.com/paraview/paraview/-/tree/master/Examples/Plugins/ElevationFilter

What about the ParaView-Superbuild?

The ParaView-Superbuild has a very specific way to handle the creation of the plugin configuration file that is shipped with ParaView binary release.

Delayed load plugin are not currently supported in the context of the ParaView Superbuild.

Benchmark

We have run some quick benchmarks with big, medium and small plugins and their impacts on the startup of ParaView when loading them at startup, with or without the delayed load mechanism.

Benchmark run on Arch Linux x86_64 with Intel i9-9900K (16) @ 5.000GHz using ext4 filesystem.
Big plugin: TopologyToolkit, 60 Mo of libraries, many proxies.
Medium plugin: SurfaceTrackerCut, 7.8 Mo of libraries, few proxies.
Small plugin: VortexFinder, 777 Ko of libraries, many proxies

As one can see, when loading big plugins at startup like the TopologiToolKit, there is a potential time gain to use the delayed load plugin mechanism. However, if the plugin is smaller or if there is a lot of XML proxies to parse from the plugin, then the gain can be negligible or even negative.

This mechanism should only be enabled for plugins that clearly benefit from it.

That being said, loading time of libraries can vary wildly depending on OS and filesystem and we could expect different results on other systems.

Please note that in the ParaView 5.13.0 binary release, no plugin is yet using the delayed load mechanism.

Acknowledgement

This work was supported by EDF in the context of Salome development