Memory compression in ITK-SNAP
ITK-SNAP is a software application used to segment structures in 3D medical images. It is the product of a decade-long collaboration between Paul Yushkevich, Ph.D., of the Penn Image Computing and Science Laboratory (PICSL) at the University of Pennsylvania, and Guido Gerig, Ph.D., of the Scientific Computing and Imaging Institute (SCI) at the University of Utah, whose vision was to create a tool that would be dedicated to a specific function, segmentation, and would be easy to use and learn. ITK-SNAP is free, open-source, and multi-platform.
Internally, ITK-SNAP keeps both image grey-values and segmentation labels as 16-bit integers. It also keeps an undo image, so normally 6 bytes are allocated for each image voxel. The segmentation layer usually contains small number of labels, and they are grouped together into large chunks, which makes it a perfect target for run-length encoding (RLE) to save working memory (RAM). The savings are significant for large images.
| ITK-SNAP Version | Image (Size MVox) | Number of labels | Private Working Set (MB) | Commit Size (MB) | PWS difference | Commit difference |
| 3.x nightly | None (0) | 0 | 75 | 154 | 100% | 100% |
| 3.x nightly | vb-seg (116) | 9 | 626 | 731 | 100% | 100% |
| 3.x nightly | wb-seg (390) | 58672 | 2912 | 3058 | 100% | 100% |
| dev_3.6 | None (0) | 0 | 80 | 165 | +6% | +7% |
| dev_3.6 | vb-seg (116) | 9 | 403 | 508 | -36% | -31% |
| dev_3.6 | wb-seg (390) | 58672 | 2032 | 2267 | -30% | -26% |
The above table depicts memory savings of RLE implementation. 6% higher memory consumption without any image loaded probably comes from differences in compiler used (3.x nightly was downloaded, dev_3.6 locally compiled). The private working set is physical memory used, commit size is total allocated memory of the process. The decrease is slightly less than a third. Considering that about a third of the memory is dedicated to segmentation image, it means that RLE is very effective for the two typical test images used. In fact, the RLE compression ratio is 44x and 20x for the two test images used!
A run-length encoded image type (RLEImage) was implemented, designed to fit into Insight ToolKit (ITK), along with accompanying iterators and a region of interest image filter. Region of interest image filter has three scpecialized variants, so it can be used both to extract a subregion from an RLEImage, as well as convert between itk::Image and RLEImage (optionally with a subregion operation).
As image slicing is executed whenever mouse is scrolled or clicked, special care has been taken to keep it fast. To this end, slicing filter has a multithreaded implementation. For slicing planes which are parallel to RLE axis it is even faster than slicing without memory compression!
| Durations in milliseconds | Dimensions | itk::LabelMap | naiveRLE | SNAP+RLE | SNAP-noRLE | pureITK | |||||||||||||
| Image | Labels | X | Y | Z | X | Y | Z | X | Y | Z | X | Y | Z | X | Y | Z | X | Y | Z |
| MRIcrop-seg.gipl.gz | 9 | 78 | 110 | 64 | 5.1 | 3.9 | 4.1 | 3.0 | 2.7 | 2.7 | 1.3 | 0.2 | 0.3 | 0.5 | 0.4 | 0.5 | 2.0 | 0.3 | 0.3 |
| brainParc.mha | 114 | 256 | 256 | 256 | 51.0 | 39.2 | 39.0 | 33.3 | 28.3 | 28.2 | 11.7 | 1.1 | 1.0 | 5.2 | 3.6 | 3.6 | 10.3 | 0.5 | 0.4 |
| vb-seg.mha | 9 | 640 | 633 | 299 | 105.9 | 71.5 | 94.8 | 68.0 | 61.1 | 63.1 | 30.0 | 2.9 | 5.6 | 22.5 | 10.4 | 21.8 | 26.3 | 0.6 | 0.7 |
| wb-seg.nrrd | 58672 | 512 | 512 | 1559 | 993.6 | 912.1 | 874.7 | 472.1 | 392.6 | 386.6 | 154.2 | 15.4 | 8.1 | 73.9 | 43.4 | 14.1 | 79.1 | 1.4 | 0.6 |
The above table shows average slicing duration (in milliseconds) of several different slicing implementations. ITK-SNAP's pre-existing slicing code without memory compression is somewhat slower than pure ITK implementation because it does image flipping on the fly, and it is single-threaded too. The case of slicing perpendicular to X (RLE axis) is slower than non-compressed version, but only about twice slower. Slicing along other axes is faster, in same cases more than twice faster, so dual-core multi-threading alone cannot explain it. My conclusion is that taking advantage of RLE structure is reducing number of memory reads, which is the expensive operation in today's computers.
The code is currently commited to dev_3.6 git branch of ITK-SNAP. Once its stability has been proven, it could be integrated it into Insight ToolKit (ITK). This will enable other applications to take advantage of RLE compressed images.