Insight Toolkit Plug-Ins: VolView and V3D
VolView and V3D are applications for visualization and analysis of three-dimensional images. They both have tools which allow users to filter and analyze image data. The two applications serve two different niches: VolView was created with radiologists in mind, while V3D caters primarily to microscopists. However, a powerful part of both biomedical imaging tools is support for user defined extensions via custom plug-ins. This support allows users to extend how input data is filtered. This quick guide will help you get started with your own VolView and/or V3D plug-in.
Software applications such as Slicer, SNAP, Analyze, VolView, SCIRun, GoFigure and V3D use ITK filters as plug-ins to add desirable additional capability to their image analysis application of choice, thus removing the need to rewrite existing algorithms for each new piece of software while stamping out the hassles of requesting usage-permission.
ITK’s qualifications for use in scientific research makes it important for developers to make the most of ITK’s imaging tools and offer tailored combinations to those who desire them. The following table compares some of the main features of VolView [1] and V3D [2].
Structure of Plug-ins
Typically, a plug-in for V3D and VolView consists of source code compiled and packaged as a shared library. The plug-in name determines the name of the shared library used for deployment as well as the name of the plug-in initialization function. The shared library is copied into a specific directory of the VolView or V3D binary installation. No libraries from VolView or V3D are required in the process. Plug-in developers only need a set of source code headers defining the plug-in API offered by the application. This is essentially the set of data structures and function calls by which the plug-in communicates with the application.
A V3D PLUG-IN
The development of ITK plug-ins for V3D serves two purposes: 1) exposing ITK functionalities to researches who analyze microscopy data and 2) uncovering the areas in which ITK requires improvements in order to better serve the microscopy community. ITK filter plug-ins were added to V3D via a collaborative effort between Kitware and Janelia Farm (HHMI).
The simplest way to implement a plug-in is to copy and paste an existing V3D plug-in and modify two methods. More advanced plug-ins, typically those requiring more than one filter, may need to be modified further. For our V3D plug-in example, we will use the existing itkSigmoidImageFilter, which can be found under the Intensity Transformation directory to create another plug-in, itkLogImageFilter. For V3D, the V3DPluginCallback is used to get data structures and callbacks.
Create a V3D Plug-in
Copy and paste existing plug-in header and source files to the binary directory where plug-ins are set up in your system. An example path is: src/ITK-V3D-Plugins/Source/IntensityTransformations. Change the file names to correspond to the goal image filter.
Then find instances in files where filter references and names ought to be replaced. In the SetupParameters section, adjust your filter’s parameters, or if the section does not exist, refer to the SetUpParameters() below. If you’re uncertain about a default value use your best judgment or communicate with an appropriate end-user for a general value. The value chosen should reflect some noticeable changes in an image upon testing.
| void Execute (const QString &menu_name, Qwidget 50 *parent) { this->Compute(); } virtual void ComputeOneRegion() this->m_Filter->SetInput if( !this->ShouldGenerateNewWindow() ) this->m_Filter->Update(); virtual void SetupParameters() |
A VOLVIEW PLUG-IN
For this example, a plug-in named vvITKGradientMagnitude will be deployed in a shared library: libvvITKGradientMagnitude.so in Unix/OsX and vITKGradientMagnitude.dll on MS Windows. The initialization function is vvITKGradientMagnitudeInit(). The result of the example will be an implementation of a simple ITK based filter with only one GUI parameter. The example may be adapted to most other toolkits or C/C++ implementation.
Given the similar structure of V3D, the directions in this VolView example should be generic enough to be applied to a V3D plug-in with the respective style/naming differences.
Communication between the plug-in and the application is facilitated by a public header file that defines the data and GUI structures. The plug-in developer simply implements the methods that are defined within the header file.
Initialization function
A plug-in’s initialization function must conform to a particular API. For our particular example, this would be:
| extern “C” { void VV_PLUGIN_EXPORT vvITKGradientMagnitudeInit(vtkVVPluginInfo *info) { } } |
where the symbol VV_PLUGIN_EXPORT and the structure vtkVVPluginInfo are defined in the public header file, vtkVVPluginAPI.h. This initialization function will be invoked by VolView at start-up — after the shared library has been dynamically loaded.
Content
Below is the typical content of the public header file, vtkVVPluginAPI.h.
Call macro vvPluginVersionCheck() to verify the plug-in/API conforms to current version of VolView’s binary distribution. A plug-in cannot be executed if the versions do not match, and VolView displays an error message at run-time to indicate this when necessary.
| vvPluginVersionCheck(); |
Information Structure is initialized. Setup Information does not change.
| // Setup Information |
ProcessData is set to the pointer of the function that will perform the computation on the input data. This allows for freedom in the implementation of the function. This is further covered in the next section.
| info->ProcessData = ProcessData; |
Similarly, UpdateGUI is also set to a function pointer.
| info->UpdateGUI = UpdateGUI; |
SetProperty() is used to define general properties of the plug-in – some of these properties are simply informative text that is displayed on the GUI (i.e. textual name of the plug-in, terse and extended documentation). Properties are identified by tags to further enforce the decoupling between the internal representation of information in VolView and the structure of code in the plug-in. Other non-GUI properties are also set with this method.
| //Setup Information – SetProperty() |
The tag VVP_NAME specifies that the string being passed as third argument of the SetProperty() method should be used for the text label of the plug-in in the GUI. VVP_GROUP specifies the grouping of the filter within the plug-in menu, and VVP_TERSE_DOCUMENTATION provides a short description of the plug-in.
| info->SetProperty( info, VVP_NAME, “Gradient Magnitude IIR (ITK)”);
info->SetProperty( info, VVP_GROUP, “Utility”);
info->SetProperty( info, VVP_TERSE_DOCUMENTATION, |
The tag VVP_FULL_DOCUMENTATION specifies the complete description string.
| info->SetProperty( info, VVP_FULL_DOCUMENTATION, “This filter applies IIR filters to compute the equivalent of convolving the input image with the erivatives of a Gaussian kernel and then computing the magnitude of the resulting gradient.”); |
Other tags are used to specify:
Whether this filter is can perform in-place processing;
| info->SetProperty( info, VVP_SUPPORTS_IN_PLACE_PROCESSING, “0”); |
Whether this filter supports data streaming (processing in chunks);
| info->SetProperty( info, VVP_SUPPORTS_PROCESSING_PIECES, “0”); |
And other information about the filter implementation.
| info->SetProperty( info, VVP_NUMBER_OF_GUI_ITEMS, “1”);
info->SetProperty( info, VVP_REQUIRED_Z_OVERLAP, |
Memory consumption is an important consideration for processing. By providing an estimated number of bytes to be used per voxel of the input dataset:
| info-> SetProperty( info, VVP_PER_VOXEL_MEMORY_REQUIRED, “8”); |
Memory consumption can be estimated. VolView will use this factor to ensure that the system has enough memory for completing the plug-in processing and for determining if the undo information can be kept. Note that this estimate is not based on the size of the final dataset produced as output, but on the total amount of memory required for intermediate processing. In other words, it should provide the peak of memory consumption during the plug-in execution.
The ProcessData() Function
The ProcessData() function performs the filter computation on the data. The function signature of ProcessData() is:
| static int ProcessData( void *inf, vtkVVProcessDataStruct *pds) |
where the first argument is a pointer to a vtkVVPluginInfo structure which can be downcast to a vtkVVPluginInfo pointer using:
| vtkVVPluginInfo *info = (vtkVVPluginInfo *) inf; |
In this assignment, the right hand side is a structure, vtkVVProcessDataStruct, that carries information on the data set to be processed. This information includes: the actual buffer of voxel data, the number of voxels along each dimension in space, the voxel spacing and the voxel type.
The vtkVVProcessDataStruct also contains the members inData and outData, which are pointers to input and output data sets, respectively. ProcessData() extracts the data from the inData pointer, processes it, and stores the final results in the outData buffer.
ProcessData() Starting Code
The typical ProcessData() starting code of this function extracts meta information about the data set from the vtkVVProcessDataStruct and vtkVVPluginInfo structures. For example, the following code shows how to extract the dimensions and spacing of the data.
First, set up a data structure.
| SizeType size; IndexType start; double origin[3]; double spacing[3];
size[0] = info->InputVolumeDimensions[0];
for( unsigned int i=0; i<3; i++ ) |
Image data can be imported into an ITK image using the itkImportImageFilter.
| RegionType region; region.SetIndex( start ); region.SetSize( size ); m_ImportFilter->SetSpacing( spacing ); m_ImportFilter->SetOrigin( origin ); m_ImportFilter->SetRegion( region ); m_ImportFilter->SetImportPointer( pds->inData, totalNumberOfPixels, false ); |
The output of the import filter is then connected as the input of the ITK data pipeline and the pipeline execution is triggered by calling Update() on the last filter.
| m_FilterA->SetInput( m_ImportFilter->GetOutput() ); m_FilterB->SetInput( m_FilterA->GetOutput() ); m_FilterC->SetInput( m_FilterB->GetOutput() ); m_FilterD->SetInput( m_FilterC->GetOutput() ); m_FilterD->Update(); |
Finally the output data can be copied into the pointer provided by VolView. This is typically done using an ITK image iterator that will visit all the voxels.
| outputImage = m_Filter->GetOutput(); typedef itk::ImageRegionConstIterator < OutputImageType > OutputIteratorType; OutputIteratorType ot( outputImage, outputImage->GetBufferedRegion() ); OutputPixelType * outData = static_cast< OutputPixelType * >( pds->outData ); ot.GoToBegin(); while( !ot.IsAtEnd() ) { *outData = ot.Get(); ++ot; ++outData; } |
When memory consumption is critical, it is more convenient to actually connect the output memory buffer provided by VolView to the output image of the last filter in the ITK pipeline. This can be done by invoking the following lines of code before executing the pipeline.
| m_FilterD->GetOutput()->SetRegions(region); m_FilterD->GetOutput()->GetPixelContainer() ->SetImportPointer( static_cast< OutputPixelType * >( pds->outData ), totalNumberOfPixels, false); m_Filter->GetOutput()->Allocate( ); |
The current distribution of ITK provides support for not rewriting this same code for each new plug-in. A templated class containing this code is available in the current distribution of ITK in the InsightApplications module. New plug-ins only need to define their own ITK pipelines and invoke the methods of the base class in the appropriate order.
Refreshing the GUI
After the source code has been packaged into a shared library, it should be deposited into the plug-ins directory: VolView 3.2/bin/Plugins. In order for the plug-in to load, the GUI needs to be refreshed by re-scanning all plug-ins. A circular arrow next to the filters selection menu will refresh the filters list.
Image processing algorithms can take considerable time to execute on 3D data sets. It is important to provide user feedback as to how the processing is progressing and to allow the user to cancel an operation if the total execution time is excessively long. Calling the UpdateProgress() function of the vtkVVPluginInfo structure from within the ProcessData() function accomplishes this:
| float progress = 0.5; // 50% progress info->UpdateProgress( info, progress, “half data set processed”); |
This function provides feedback to the VolView GUI allowing VolView to update the progress bar and set the status bar message. The frequency with which the UpdateFunction should be called should be well balanced. If it is invoked too often, it will negatively impact the performance of the plug-in — a considerable amount of time will be spent in GUI refreshing. If it is not called often enough, it may produce the impression that the processing is failing and that the application is no longer responding to user commands.
A detailed skeleton plug-in and a more contextual version of this guide can be found starting on page 45 in the VolView Users Manual, available at kitware.com/volview.
REFERENCES
[1] Download VolView from: kitware.com/volview
[2] Download V3D from: http://penglab.janelia.org/proj/v3d
Sophie Chen recently completed her second summer as a Kitware intern where she worked under Luis Ibáñez, Wes Turner and Harvey Cline on programming algorithms, ITK and VTK. Sophie is a senior at RPI where she is working toward an IT degree in Managing Information Systems.