Trame in JupyterLab: A Unified Approach for Web Apps in Scientific Computing

Overview

Trame is a relatively new Kitware platform that enables scientists and engineers to develop interactive feature rich web applications in Python without needing a deep knowledge of frontend/backend web development.

To enhance Trame’s capabilities further, Kitware and 3rd party developers have been actively working to develop new plugins, offering new features and integration with existing tools. One such example is the trame-vtk package that enables Trame applications to leverage server side VTK rendering with just a few lines of code.

In the broader context of scientific and engineering tools, JupyterLab has been among the most popular tools for researchers, data scientists, and engineers. It has enabled rapid prototyping, interactive visualization of  experiment results, and exploration of  large datasets, along with numerous other applications.

Integration of Trame with JupyterLab was critically needed to ensure that users of this widely used platform can fully leverage Trame’s extensive visualization features in order to facilitate more effective data analysis and communication in scientific and engineering workflows. Realizing this need, Trame developers worked towards achieving this integration, which proved beneficial to the community. However, certain shortcomings were realized in this initial integration. Specifically, the original version of Trame uses an internal web server to connect to the browser client, which causes an additional network port to be open in order to serve the application static files, as well as to establish the WebSocket for real time communication between the client and server.

Having an additional port open is fine for using Trame as a standalone desktop application, but isn’t a feasible solution in some scenarios, for example if the Jupyter server is behind a firewall, or if opening additional ports raises security concerns.

One alternative approach is to utilize the Jupyter Server Proxy integration provided by Trame, which enables handling of static files and WebSocket communication by the existing Jupyter web server. While this solution should work well in theory, in practice it has been proven quite unstable. Issues such as random closures of the connection between the client and server, or the messages exchanged between the two proving too large when using VTK local rendering, have been observed.

For this reason we set off to find a different strategy to improve the Trame integration with JupyterLab. Thus we have created a separate JupyterLab extension – trame-jupyter-extension – that leverages a new approach where we use the custom communication channels that are provided by the JupyterLab API.

In a nutshell, JupyterLab already instantiates a WebSocket connection for exchanging messages between the server and the client, and it allows extension developers to create custom communication channels that will be transported through this existing connection.  By creating a WebSocket adapter, we “trick” Trame into believing that it is communicating with the client via its own WebSocket, while in fact all the Trame messages are going through the already existing connection put in place by JupyterLab.

This new extension solves the same problem that the Jupyter Server Proxy was supposed to solve, but in a much more stable and predictable way.

Usage

In order to use this new extension, all you need to do is install it in the Python environment you are using when running JupyterLab (4.x)

pip install trame-jupyter-extension

If you are using JupyterLab 3.x, make sure to install version 1.x of the extension:

pip install "trame_jupyter_extension<2"

If you are using Conda, you can install the extension from the conda-forge channel:

conda install trame-jupyter-extension --channel conda-forge

Once the extension is installed, when running Trame in JupyterLab the communication will be handled by the extension automatically.
We have also added support in PyVista for using this new extension if it’s installed. You can find additional information here.

Try it Out

Binder is a popular platform to create and share notebooks. Naturally, Binder doesn’t allow users to just open random ports, thus making it a good candidate for showcasing the usefulness of the extension.

In this notebook you can run a basic Trame application deployed on Binder, where all the Trame communication is happening through the JupyterLab communication channel. You can find the source code of this example notebook here.

Acknowledgements

This work is funded by the US Department of Energy’s Office of Biological and Environmental Research (BER) to develop Pan3D, an open source toolkit supporting scalable and reproducible scientific workflows for 3D data analytics using Trame, VTK Python and open source tools.

Example of the pan3d viewer inside Jupyter loading a pangeo dataset from our sample catalogue.