zSpace Support Now Available in VTK and ParaView Binary Release

We are happy to announce that the zSpace device support is available in VTK 9.3.0 and in the ParaView 5.12.0 binary release on Windows.

Context

A zSpace device is composed of a stereo display, a stylus and tracking system. All zSpace devices come along with glasses to perform head tracking, except on the latest model (the zSpace Inspire), . 

A first version of the zSpace device support was previously developed for ParaView as a plugin. Here is a presentation of the plugin on an HP Zvr, taken from the previous blog:

Until now, it was necessary to build the plugin manually. But thanks to the release of the new zSpace Core Compatibility API, it is not needed anymore. Furthermore, we refactored the plugin in order to integrate all VTK-specific classes in a dedicated VTK module. Now, supported zSpace devices are also usable in your VTK applications.

zSpace ParaView Plugin Update

zSpace recently developed a new version of the zSpace API: the zSpace Core Compatibility API. It is a C API, similar to the original native zSpace Core API, but that is compatible with all currently supported zSpace hardware models, including the latest one, the Inspire. The original native zSpace Core API, on the other hand, is only compatible with pre-Inspire zSpace hardware models.

A benefit of the new API is that underlying libraries can be loaded at runtime only: we do not need the zSpace libraries at compile time anymore. We took advantage of this feature to enable the zSpace plugin in the ParaView binaries directly. It is now available in the nightly ParaView releases, and will also be available from the 5.12 version.

To launch the plugin, simply ensure that the zSpace SDK is installed on your device. You may want to point to the SDK in your PATH environment variable, if it’s not already configured on your zSpace device. For example : set PATH=%PATH%;C:\zSpace\zSpaceSdks\CoreCompatibility_1.0.0.12\bin

To use ParaView with zSpace, it is the same as before: launch ParaView, then select the zSpace plugin (“Tools → Manage Plugins” then select “zSpace”). Once done, you just need to open a new Layout and select the “zSpace” view (please refer to the previous blog post for more information about the plugin and possible interactions).

The zSpace VTK Module

You can now enable zSpace rendering and interactions in your VTK application with few changes. The new vtkRenderingZSpace module contains all classes dedicated to rendering and interacting with the zSpace devices. From user’s perspective, it is pretty straightforward: the structure of the VTK rendering pipeline does not change for the most part. You just have to replace the “standard” rendering classes with their zSpace counterparts.

Here is an overview of the new zSpace-specific classes:

vtkZSpaceCamera
Returns transformation matrices that are designed to map world coordinates into view coordinates, taking into account the new physical space, similarly to VR applications.

vtkZSpaceRayActor
Simply represents a ray shooting from the zSpace stylus, used for pointing or picking.

vtkZSpaceRenderer
vtkRenderer subclass that redefines the ResetCamera function, which uses the zSpace SDK to retrieve the “comfort zone” of the stereo frustum and fits the bounding box of the displayed actors in it.

vtkZSpaceRenderWindowInteractor
Handles the zSpace specific interactions, done with the stylus. It will internally update and retrieve the state of the zSpace devices (through the zSpace manager instance, in the ProcessEvents method) and emit events accordingly.

vtkZSpaceInteractorStyle
Inherits from vtkInteractorStyle3D (common to all 3D interactions, i.e. VR / AR). This class retrieves events emitted by the interactor and translates them to concrete actions, like picking and moving actors.

vtkZSpaceSDKManager
This class encapsulates all the calls to the zSpace SDK. It is the only part of the rendering pipeline that is specific to the zSpace devices. Its roles are to initialize the zSpace SDK and to synchronize the zSpace SDK with the VTK classes.

Bellow is a example code showing how the VTK rendering pipeline should be adapted to make it work on zSpace hardware. Here we simply create 2 simple geometric objects (a sphere and a cone) and we build all the pipeline to display the actors and interact with them.

  // Create a sphere source
  vtkNew<vtkSphereSource> sphereSource;
  sphereSource->SetThetaResolution(16);
  sphereSource->SetPhiResolution(16);
  sphereSource->Update();

  // Create a cone source
  vtkNew<vtkConeSource> coneSource;
  coneSource->SetResolution(16);
  coneSource->Update();

  // Connect each source to respective mappers
  vtkNew<vtkPolyDataMapper> mapper;
  mapper->SetInputConnection(sphereSource->GetOutputPort());

  vtkNew<vtkPolyDataMapper> mapper2;
  mapper2->SetInputConnection(coneSource->GetOutputPort());

  // Create an actor for each mapper
  vtkNew<vtkActor> actor;
  actor->SetMapper(mapper);
  actor->GetProperty()->SetRepresentation(VTK_SURFACE);
  actor->GetProperty()->SetEdgeVisibility(true);

  vtkNew<vtkActor> actor2;
  actor2->SetMapper(mapper2);
  actor2->GetProperty()->SetRepresentation(VTK_SURFACE);
  actor2->GetProperty()->SetEdgeVisibility(true);

  // [ZSPACE] Create the zSpace ray actor
  vtkNew<vtkZSpaceRayActor> rayActor;

  // [ZSPACE] Create the zSpace camera
  vtkNew<vtkZSpaceCamera> camera;

  // [ZSPACE] Create and setup the zSpace renderer
  // And add all the actors to the renderer
  vtkNew<vtkZSpaceRenderer> renderer;
  renderer->AddActor(actor);
  renderer->AddActor(actor2);
  renderer->AddActor(rayActor);
  renderer->SetActiveCamera(camera);
  renderer->SetBackground(0.2, 0.3, 0.4);

  // Setup the stereo render window
  // For zSpace (and quad buffer stereo in general),
  // the stereo type to set is "Crystal Eyes"
  vtkNew<vtkRenderWindow> renderWindow;
  renderWindow->AddRenderer(renderer);
  renderWindow->SetStereoCapableWindow(true);
  renderWindow->SetStereoRender(true);
  renderWindow->SetStereoTypeToCrystalEyes();
  renderWindow->SetSize(1600, 900); // Set resolution here

  // [ZSPACE] Setup the zSpace render window interactor
  vtkNew<vtkZSpaceRenderWindowInteractor> renderWindowInteractor;
  renderWindowInteractor->SetRenderWindow(renderWindow);

  // [ZSPACE] Setup the zSpace interactor style 
  // (used to handle the stylus actions)
  vtkZSpaceInteractorStyle* interactorStyle = 
   vtkZSpaceInteractorStyle::SafeDownCast(renderWindowInteractor->GetInteractorStyle());
  interactorStyle->SetZSpaceRayActor(rayActor);
  interactorStyle->SetCurrentRenderer(renderer);
  interactorStyle->SetPickingFieldAssociation(vtkDataObject::FIELD_ASSOCIATION_CELLS);

  // [ZSPACE] Setup the ZSpace Manager
  // This class communicates with the zSpace API and initialize
  // all needed variables. 
  vtkZSpaceSDKManager* manager = vtkZSpaceSDKManager::GetInstance();
  manager->SetRenderWindow(renderWindow);
  manager->Update();

  // Initialize the camera postion to fit the actors to the
  // comfort zone of the stereo frustum
  renderer->ResetCamera(actor->GetBounds());

  // Start the rendering and event loop
  renderWindow->Render();
  renderWindowInteractor->Start();

From a quick overview, it appears like any “standard” VTK rendering pipeline, using zSpace-specific classes as drop-in replacement. The only real new thing is the instantiation of the vtkZSpaceSDKManager, handling the bulk of the communication with the zSpace SDK.

This example code is available here.

Going Further

Now that VTK and ParaView support the new zSpace Core Compatibility API with pre-Inspire hardware, the next step is to support the Inspire Itself. The main benefit of this new hardware, besides being portable, is that it does not require glasses for head tracking and for stereo displaying. Despite VTK and ParaView already support interactions with the stylus on this device, VTK and ParaView still needs to support this new stereo rendering technique.

Kitware Europe is currently working to support the zSpace Inspire, and will not fail to keep you informed about it.
Stay tuned !

Acknowledgements

The zSpace technology is developed by the zSpace Company

Developments were done by Kitware Europe, France