Understanding the Resonant Platform

This is the first in a series of blog posts about Resonant, a data analytics and visualization platform for the web. 

Posts in this series:
Understanding the Resonant Platform
Girder Applications: An Inside Look
Girder Worker and the COVALIC Grand Challenge
Geospatial Visualization with GeoJS
Resonant Visualization with Candela
Recapping Resonant

Resonant is a general platform for data and analytics, based around the Girder data management application.  The tools comprising the Resonant suite are built as general, reusable components that are able to solve the problems of specific consulting projects.  Many of our solutions build upon Girder, along with other Resonant components to carry out high performance computing and visualization.

This series will cover many of the pieces that make up the Resonant Platform, demonstrating what they can do and how they work together, beginning with Girder.

Girder

Girder is the core piece of the Resonant platform, and Girder’s primary function is to manage data.  Girder provides data organization and management services through a Python server with a RESTful (HTTP) API.  Girder also offers a web client and a Python client, which can be used to build other clients, or completely separate clients can be written to consume the RESTful API.

Girder provides a number of models to help manage data including permission related models of users and groups, which are arbitrary sets of users who have the same permissions on a resource.  There are also models representing the managed resources: files, items (a set of zero or more files along with some metadata), folders (containers for folders and items), and collections (a special top level folder).  Item metadata takes the form of arbitrary key-value pairs, and can include nested hierarchical documents.  Girder uses specific implementations of assetstore models to store the bytes from its files, where an assetstore is an abstraction over file storage.  See the appendix below for more details on Girder’s assetstores.

Girder is designed to be extensible through plugins, which can add or override existing functionality on the server, the client, or both.  

As an example of expanded server functionality, Girder can manage users and authentication itself, and provides the ability to include arbitrary external authentication schemes.  There is a Girder plugin that supplies specific support for OAuth and can be extended to include new providers, with current support for OAuth providers of Google, GitHub, LinkedIn, Globus and Bitbucket.

The Girder web client can be extended through plugins to allow for specialized display and interaction with datasets through standard Girder web client pages.  The UNC-Wisconsin Neurodevelopment Rhesus Data is a longitudinal collection of primate brain images housed at https://data.kitware.com.  It is displayed in the Girder client with a specific plugin that takes the standard hierarchy of the dataset, and uses the dataset metadata such as subject date of birth and subject id to construct a visualization of the data on any of the collection pages that can also be used to navigate through the dataset.

The set of subjects

Subject metadata used to drive the longitudinal chart

Visualization of the longitudinal study dataset

Girder also provides reusable components that can form the basis of tailored applications.  In this screenshot we can see https://challenge.kitware.com, which is a Girder application whose purpose is to administer scientific grand challenges.

Girder 2.0 has recently been released, including a major overhaul of the client-side build system to use Webpack, which enables greater reuse of Girder’s client side libraries in web applications.

Due to Girder’s RESTful interface and the decoupling between the Girder web client and server, additional clients can be written to use the Girder server.  For example, the Girder Python client offers a command line interface to a Girder server, and can be used to build Python applications that communicate with Girder.  Another example is the HPCCloud application, which extends Girder on the server side via the Cumulus plugin, and creates a web client written in an entirely different web framework (ReactJs) from the Girder web client, but still is able to communicate with a Girder server.  Similarly, Resonant Lab, written as a Girder plugin, is an application that allows for easily constructing data visualizations without doing any programming.  Resonant Lab will be discussed in much greater detail in a future blog post.

Girder is the basis of the Resonant platform, providing a strong foundation of rich data management capabilities and extensibility that have been built with a commitment to software quality.  Subsequent articles in this blog post series will explore many of the other tools in the Resonant platform as well as how they have been deployed in projects alongside Girder. To learn more about using Girder or other Resonant tools for your own project, comment below or email kitware(at)kitware.com.

Appendix: Girder’s assetstores and file management

Girder is designed to address data in a unified way, providing an abstraction over any specific file storage mechanisms.  Girder’s assetstore model allows specific implementations to define how individual files are stored and provides a common interface to opaque blobs of bytes, despite the differences in their underlying storage technology.  Currently Girder supports managed assetstores–meaning that Girder itself will create/update/delete the files on the specific storage media–for files stored on S3, HDFS, GridFS (in Mongo), or a local or FUSE mounted distributed filesystem.  Girder also has unmanaged (read-only) assetstore types for files in S3 and over SFTP, meaning that Girder can read the files and create a snapshot proxy for the file hierarchy, but will not manage the lifecycle of the files and will not be updated about changes in the underlying files.  For both managed and unmanaged assetstores, Girder stores its representation of the hierarchy along with the metadata in MongoDB. A Girder instance may contain one or more assetstores, and can mix assetstores of various types, meaning that files might be stored in a combination of S3, HDFS and a local filesystem, but they can be read and processing through a uniform interface presented by Girder.  The Girder plugin system allows for creating new assetstore types, an example of which is the database assetstore plugin, which provides the ability to present PostgreSQL and MongoDB data as Girder Items.