This is part of a series of articles about Resonant, a platform for building end-to-end solutions to data analytics and visualization problems. 

Posts in this series:
Understanding the Resonant Platform
Girder Applications: An Inside Look
Girder Worker and the COVALIC Grand Challenge
Geospatial Visualization with GeoJS
Resonant Visualization with Candela
Recapping Resonant

In the last article, we learned about Girder, a framework for web-enabled data management. Now, we will take a closer look at an example web application, Resonant Lab, built around the services and capabilities provided by Girder.

Overview

Put simply, Resonant Lab is an application for constructing data visualizations for particular datasets without doing any programming.

Of course, providing this capability involves a lot of moving parts. Resonant Lab has facilities for uploading and managing datasets, user interfaces to mediate the creation and editing of visualizations, management of user accounts to provide privacy settings and ownership of visualization projects, and access to a library of built-in visualization types such as bar charts, scatter plots, and even some more advanced, cutting-edge visualization modes.

Resonant Lab’s basic mode of operation is to select a dataset, select a visualization, and then interactively connect data attributes to visual properties of the visualization. The application transparently saves your activity on the server, and includes other facilities for changing the privacy/sharing setting, making copies, exporting images, and sharing links to your projects to others.

A future article will provide an in-depth look at and discussion of Resonant Lab’s UI design features and visualization capabilities; in this article we’ll look at the structure of Resonant Lab’s design as a Girder application, which parts of Girder provide the various chunks of functionality, and how they come together to create a single, high-powered application.

Anatomy of a Girder Application

The previous article discusses how Girder’s core functionality can be extended, both on the server and client side, by plugins. A special class of plugins extends the client side by replacing the stock Girder user interface at the web root with custom content; such plugins can be thought of as Girder applications. Resonant Lab is structured this way, but it also builds upon several core features of Girder.

Backend

The following backend features of Girder provide several services used by Resonant Lab.

Data storage. Girder’s main feature is, of course, data storage and management. Resonant Lab makes heavy use of the capability, storing datasets and projects and offering familiar UI for saving and retrieving them.

Datasets are stored more or less as-is. Resonant Lab currently supports uploading of CSV and JSON data, which is stored in a Girder item for later loading into a project. Item metadata is used to record useful and necessary information about a dataset, such as its type (e.g., “Excel-flavored CSV”) and the distribution of its columns and values (for use in rendering histograms that can then be used to filter the dataset).

Projects are also stored in Girder items. The constituent attributes of a project – which dataset is being used, which visualization type, and what mappings between data attributes and visual attributes – are stored in the item metadata.

Custom server endpoints. Resonant Lab needs some custom logic to function. Some of this custom logic must operate on entire datasets, which may in turn be too large to process directly in the browser, so it is implemented as server side code accessible via custom REST endpoints.

Some routines are meant to executed one time on a new dataset, essentially preparing it for use in the application. For example, Resonant Lab tries to infer the data types of the different columns in a new dataset. If for instance, every value in some column of a CSV file can be converted to a floating point value, Resonant Lab will infer that the appropriate data type is Number. This is useful because certain visualizations – for example, a scatter plot’s x- and y-axes – demand numeric data, while others can accept different types.

Resonant Lab also needs to know the distribution of values in the various columns in order to render a histogram, which helps the user understand the shape of the data, and also enables filtering of the data based on value ranges.

Both of these computations only need to run once on a fresh dataset; the results are stored in the metadata of the Girder item representing that dataset.

Resonant Lab also includes a server endpoint for retrieving pages of data. The ordinary notion of “paging” on the web would be to ask the server for a byte-range of data from a given file. However, that notion of paging doesn’t work in Resonant Lab, which prefers to receive pages of data; to that end, some special logic is necessary for parsing out chunks of data from a CSV or JSON file, such that a page worth of values (regardless of how many bytes that may be) comes back from the server for use in a visualization.

User accounts. Girder includes user account management as a core feature; Resonant Lab takes advantage of it to keep track of your work and allow you to enforce privacy restrictions on various datasets and projects. For example, projects that are open to public viewing can generate a URL that can be shared to collaborators.

Resonant Lab also leverages Girder’s user management to provide a convenience for new users: without logging in or even creating an account, anyone can use Resonant Lab to get familiar with how it works and what it offers. This is accomplished by using a special account to handle all anonymous users who engage the application. The browser keeps track of which projects an anonymous user has created, and if and when they create a private account, the application knows to “adopt” all of that user’s anonymous projects into their new account.

Database assetstore plugin. Resonant Lab can also take advantage of other Girder plugins to supercharge any of the above usage. Currently, it uses the database assetstore plugin to allow for data stored in, say, a Mongo database instance, to become a Girder item, which in turn can be used in Resonant Lab. The database assetstore plugin bolsters Resonant Lab’s abilities to handle “big data”. Databases are optimized for fast access to large datasets, and the performance boost directly transfers to Resonant Lab.

This is an example of how Girder’s plugin architecture can represent a windfall for client applications. By providing a seamless way to embrace databases into Girder’s data storage model, Resonant Lab gains an immediate and decisive boost to its own capabilities.

Frontend

The bedrock features of Girder described above are tied together and presented to users through a carefully designed, user-centric frontend. The user interface is easy to use, even for new users, and provides ways to upload and manage data, select visualization options from a library of options, and easily build visualizations by filtering and paging the dataset, and setting visual properties of the visualization to correspond to different columns of data. The interface also allows for exporting visualizations to PNG or SVG images, or to standalone web applications, bundled up conveniently in a single HTML file.

Overall, the frontend is meant to provide the seamless experience of using a traditional desktop graphical application, abstracting all the messy details of doing so in the server components of the application.

Although we opted to create Resonant Lab’s frontend interface from scratch, so as to have total control over its user-centric design aspects, Girder offers its own tools to help build a frontend. These include widgets for browsing the file hierarchy, dialog boxes for user login/logout interactions, and many others as well. Despite forgoing these components, Resonant Lab does use a Girder client plugin that serves its custom frontend in place of the stock Girder application frontend, thus becoming a Girder application in its own right.

Future Development

As Resonant Lab develops, so does Girder, exposing more and more opportunities for improvement. Here are a few ideas we are thinking about for lines of future development.

Girder Worker. As we will discuss in the next post, Girder Worker is an analytics component of the Resonant platform that provides computational heavy lifting. In addition to the basic filtering and paging capabilities that Resonant Lab provides in its frontend, we can imagine a Girder Worker component that could provide basic data transforms on a larger dataset, such as computing five-number summaries for statistical visualization, or filtering a whole dataset based on criteria other than simple value ranges.

Project provenance. Using Girder’s provenance plugin, we would be able to track changes made to a project, enabling an undo-stack or simply exposing the evolution of a complex project. These features would allow others to study your projects and learn what you did to create a visualization, or simply allow you to go back in time to revive good ideas that went into a particular project for use elsewhere.

Anonymous user plugin. The feature mentioned earlier that allows for people who are not logged in to still use Resonant Laboratory is built around a special-purpose user account that transparently acts as the surrogate for all such anonymous activity. Such a capability is generally useful and could be abstracted out into its own Girder plugin for use in other applications.

These ideas demonstrate some of the ways in which applications may both draw from and contribute to the Resonant platform’s capabilities.

Conclusion

Resonant Lab has a fairly typical structure for a Girder application plugin. The flexibility and power of the application comes from imbuing custom server endpoints and item metadata with special meaning specific to Resonant Lab, and from drawing on existing Girder plugins. These custom services are then unified behind a well-designed user interface frontend.

The same basic pattern for constructing applications for Girder recurs in other projects: OSUMO, COVALIC, ISIC, and many others have created a Girder application to meet their goals, each taking advantage of Girder in its own way.

If you’d like to learn more about Girder and Girder application development, please visit:

• Girder documentation: https://girder.readthedocs.io/en/latest/
• Girder source code: https://github.com/girder/girder
• Resonant Lab source code: https://github.com/Kitware/candela/tree/master/app/resonantlab

Or, if you prefer, contact us directly at kitware@kitware.com and we can help you figure out how best to solve your data and analytics problems.

Stay tuned for more articles about Resonant.