Ultrasound and ITKv4
The most common imaging mode in diagnostic medical ultrasound imaging is the B-Mode (Brightness-Mode). Recently, a manuscript [1] along with corresponding source code was submitted to the Insight Journal that describes classes for the Insight Toolkit (ITK) [2] to create B-Mode images.
This article summarizes the Insight Journal paper and also discusses challenges and successes that were encountered in the development of the software. We will examine how the ongoing activities for ITKv4 address some of these issues.
Calculation of the brightness from a raw received ultrasound signal amounts to approximating the envelope of the signal. The envelope is often calculated with the analytic signal,
where fH (x) is the Hilbert Transform of f (x). Given the analytic signal, the envelope is simply
The analytic signal is often calculated with the Fourier Transform,
A number of classes were implemented to perform the Fast Fourier Transform (FFT) along a single direction of an image, the direction of the ultrasound beam’s propagation.
We also presented code to perform scan conversion. This includes classes to resample data from a polar coordinate system, which is commonly encountered with curvilinear, phased array, or rotated single-element transducers.
Example high frequency ultrasound B-Mode images of the object shown in Figure 1 are rendered in Figures 2 and 3. The scan conversion and streaming capabilities described in the article were required to create this image.
Figure 1: Gummy bear photo
Figure 2: Gummy bear slices taken with the VisualSonics RMV710B 25MHz transducer.
a) axial-lateral, scan converted slice, b) axial-elevational slice where lines in the elevation direction are
created by a stepper motor, c) lateral-elevational, “C-Plane”, slice, and d) tri-plane view.
Images rendered with QGoImageCompare [3].
Figure 3: Volume rendering of the gummy bear B-Mode. Image created with Paraview [4].
GPGPU’s and the FFT
Since the FFT is the most computationally expensive part of the algorithm, we tested three different FFT implementations: VNL, FFTW, and an OpenCL implementation. We found that the FFTW was faster than VNL, but the OpenCL was slower in spite of the hype surrounding general purpose GPU (GPGPU) computing for scientific analysis.
The speed deficits were likely related to our implementation: the image buffer needed to be transferred to and from the GPU when performing the forward and inverse transform. For GPGPU computing to be successful, additional infrastructure is required in ITK to prevent this memory transfer.
There is a proposal to add GPGPU capabilities to ITK in ITKv4 [5]. This proposal addresses the problem of keeping the image buffer on the GPU as long as possible along with abstracting out some of the difficulties experienced when writing GPGPU code. Writing OpenCL code requires extra build steps and additional memory management, and it precludes the native use of C++ templates. When developing our code, we used the C++ OpenCL class wrappers that have since been incorporated into the OpenCL 1.1 standard. However, these classes do not provide a very object-oriented interface; they just provide destructors that ease memory management. In our opinion, writing OpenCL code, a C language with extensions, is considerably more work than vanilla C++.
Implementing GPGPU support in ITK will take substantial work, but supporting this class of hardware means ITK can stay relevant in the parallel future.
File Formats
Some manufacturers of clinical ultrasound systems provide methods to access the radiofrequency (RF) data collected on their systems. A considerable problem is that each system uses its own file format, and the format is proprietary. An astute examiner of the CMake configuration included with the project will notice that there is reference to optional support of a few of these formats.
Unfortunately, we could not include the code for the ImageIO class and supporting libraries that were developed. While it took considerable work to create readers for the complex formats and we would not like to see other researchers repeat this process, we lack the resources [6] to release this code. In order to create the readers, the companies provided documentation they considered proprietary.
To distribute our data, we converted the images to some of the standard supported ITK formats. We found it difficult to find a format that would support our additional metadata. We tried the MINC format, but we did not have success and ended up using the NRRD format. The NRRD format was not ideal, however, because it only supports string fields.
There is a proposal to add HDF5 support [7], which is a flexible format that has gained considerable traction in general scientific computing applications. Support for adding HDF5 fields to an image’s MetaDataDictionary would be beneficial for situations like this.
Streaming
Large 3D ultrasound images require data streaming to prevent the filling of system memory. In general, ITK’s excellent streaming infrastructure makes this an easy task. However, scan conversion requires the use of itk::ResampleImageFilter. In its current state, this filter does not allow streaming. We introduced a modification to this filter that would allow for streaming as long as the transform associated with it is linear. This feature may be helpful to ITK in general, and it could be incorporated into ITKv4. However, to retain its behavior in this example, the itk::Transform classes should change their IsLinear() method to return an array of booleans per dimension as opposed to a single boolean. This is required because the scan conversion transform is non-linear the r-θ plane.
Version Control
The submitted code of the repository was also a Git repository with its “origin” pointing to Gitorious [8]. This allows reviewers to easily fix or extend the work. We have been impressed by the power of Git and are happy to see it being adopted as the version control system for ITKv4.
The article was our first experimentation with Git submodules. This proved to be an effective method to manage logical sections of code, sometimes from disparate sources. It seems that ITKv4 will also soon take advantage of Git submodule capabilities with the modularization project [9]. There are rumors that these capabilities will be used to improve the accessibility of work published in the Insight Journal. This increased organization of such a complex project will allow users to find what they are looking for and contributers to make improvements with less effort.
Future Work
We hope to release additional tools for basic ultrasound research with ITK including the release of a block-matching framework for deformable image registration.
Acknowledgements
The authors would like to thank the NIH for their funding with grants T90DK070079 and R90DK071515. We would also like to thank the many developers of the Insight Toolkit.
References
[1] McCormick, M. “An Open Source, Fast Ultrasound B-Mode Software Implementation for Commodity Hardware.”
Insight Journal. May 2010. http://hdl.handle.net/10380/1338.
[2] Insight Segmentation and Registration Toolkit. itk.org/.
[3] Perrot-Audet A., Mosaliganti K., Gelas A., Rannou N., Souhait L., Megason S. “A simple Qt based comparison
program for ITK and VTK images.” Insight Journal. August 2010. http://hdl.handle.net/10380/3196
[4] ParaView is an open-source, multi-platform data analysis and visualization application. http://paraview.org/.
[5] Won-Ki. GPU Acceleration. Fall ITKv4 2010 Meeting. www.itk.org/Wiki/images/3/3a/GPU-ITKv4.pdf
[6] Zevon, W. “Lawyers, Guns, and Money.” Excitable Boy. 1978.
[7] HDF5 ImageIO. www.itk.org/Wiki/Proposals:HDF5_ImageIO
[8] Gitorious provides free open source infrastructure for open source projects that use Git.
http://gitorious.org/ultrasound-b-mode/b-mode
[9] Hoffman, B. Liu, Xiaoxiao. “Modularization.” Fall ITKv4
2010 Meeting. http://www.itk.org/Wiki/images/f/ff/ITK-Iowa-Nov-2010-Modularization.pptx
Matthew McCormick is a PhD. candidate at the University of Wisconsin-Madison working on his thesis in medical ultrasound characterization of carotid plaques.