The collection and preparation of healthcare data is a prerequisite to any development of AI applications. Optimizing annotation pipelines is critical because the evolution of AI model architectures and the imperative to make them generalize well for production involves more and more data being collected. As such, the annotation stage often becomes the bottleneck. For annotation workflows to keep up with the pace of data collection, it is important to deploy end-to-end optimized pipelines. In addition, in the medical domain, it is important to ensure privacy, security, and energy efficiency of the hardware and process for it to be adopted in healthcare settings. This is where edge and embedded compute platforms such as NVIDIA IGX  Developer Kits can shine.

Among other annotation problems, whole-body CT segmentation task is notoriously challenging both for annotators and reviewers, specifically:

1. How to handle centralized secure data storage and synchronization 
2. How to enable fast interaction with the data 
3. How to accelerate annotation speed and improve accuracy

In collaboration with the NVIDIA healthcare team, we have developed a workflow based on MONAI Label, using 3D Slicer as a client to tackle these challenges. We profiled the compute resource usage with a focus on energy consumption and computational efficiency.

Setup MONAI Label workflow on NVIDIA Clara AGX DevKit

MONAI is an open source platform designed to address the challenges of integrating deep learning into healthcare solutions. Inside the MONAI ecosystem lives MONAI Label, which focuses on the task of scalable and efficient medical image labeling. For more information on MONAI workflows, we invite the reader to check this post.

Here, we focused on spleen segmentation on the well-known medical decathlon dataset. The test dataset was annotated with a client-server paradigm, where the MONAI Label server runs on the NVIDIA Clara AGX Developer Kit, and the user interacts with the 3D Slicer client.

Platform used

NVIDIA Holoscan, optimized on the Clara AGX Developer Kit, is a computing architecture for next-generation AI medical devices. It is optimized for the Holoscan framework and system software, a multimodal real-time AI sensor streaming platform allowing devices to evolve from fixed function to software-defined.

The embedded system-on-chip (SoC) is a Jetson AGX Xavier module embedding an 8-core Carmel ARM v8.2 CPU with an integrated Volta GPU (512 CUDA and 64 Tensor cores), and also an RTX 6000 discrete GPU. The ARM-based CPU, widely used in mobile devices, allows for energy-efficient embedded applications. More information on the Developer Kit can be found in the technical specs.

Software requirements

With HoloPack, the Clara AGX Developer Kit is quite easy to set up. It embeds all APIs like CUDA and CUDNN needed to develop AI applications. Here, we updated the firmware to 5.10.65-tegra on Ubuntu 20.04.4 LTS.

MONAI Label server and client setup

For the server, we used monailabel 0.8.3, monai 1.3.2 and pytorch 2.2 with upgraded cuda-12.2. On the client side, we used 3D Slicer 5.6.2 with the MONAI Label extension.
Check the following documentation for setup instructions.

Methods

We annotated all the test datasets from MSD on the CLARA AGX Developer Kit with the RTX 6000 discrete GPU and compared it with a desktop, including a 12th Gen Intel i9-12900K and a similar GPU (NVIDIA RTX A6000) on Ubuntu 22.04 LTS.

For the annotation workflow, we first made sure that all the data was cached by the Slicer client in ~/.cache/slicer.org/Slicer/SlicerIO. After running the radiology app server on the selected device, we simulated an annotator workflow by loading each CT volume with the random strategy and sending the auto-segmented label to the data storage server.

While running the application, we profiled the auto-segmentation time along with the GPU and CPU consumption with the following custom commands:

• GPU NVIDIA (desktop & AGX)
  sudo nvidia-smi -lms 100 --query-gpu=timestamp,gpu_name,driver_version,utilization.gpu,memory.used,memory.total,utilization.memory,temperature.gpu,power.draw.instant,power.max_limit --format=csv
• CPU Intel (desktop)
  sudo turbostat -i 0.1 -s CoreTmp,Busy%,PkgWatt -S
• CPU Tegra (AGX)
  sudo tegrastats --interval 100

For an overview of the application running, please look at the following video:

Results

A summary table of the results is shown in Table 1.

Although the prediction time is twice as long as that of a desktop workstation (due to the higher-end RTX GPU on the workstation), the CPU consumption is impressive due to the reduced instruction set from the ARM CPU. When its AI compute capacity is sufficient on the iGPU, we could limit its use to the iGPU instead of the discrete RTX GPU (dGPU) to reduce consumption further to below 30W. The temperatures are also lower, which is more secure for regulatory considerations overall.

Looking Forward to an Exciting Future

Using a Clara AGX Developer Kit in the context of medical AI can improve the overall workflow and provide a powerful all-in-one platform for federated annotations. We expect more developments to be done on the platform, especially on the state-of-the-art NVIDIA Orin IGX platform, in the near future, so stay tuned! In the meantime, don’t hesitate to contact us if you’re interested in learning more about these capabilities and how you can leverage them.

Acknowledgments

We want to thank NVIDIA for providing us with a Clara AGX Developer Kit.