THE GEOINT SYMPOSIUM

Hosted by the United States Geospatial Intelligence Foundation

May 21-24, 2023 at America’s Center Convention Complex in St. Louis, Missouri 

The GEOINT Symposium is the largest gathering of geospatial intelligence professionals in the U.S., with attendees from government, industry, and academia. Kitware is a regular supporter of GEOINT, and we are proud to be a Silver-level sponsor and exhibitor in 2023. As highly involved members of the geospatial intelligence community, our AI, machine learning, and computer vision experts work closely with the intelligence community and government defense agencies. We develop and customize our open source software to meet the requirements of geospatial intelligence R&D projects. 

We look forward to meeting with attendees and exhibitors to discuss our computer vision expertise and open source tools. You can visit us at booth 408 or schedule a meeting by contacting our team.

Kitware’s Activities and Involvement

In addition to being a sponsor and exhibitor this year, Kitware is pleased to present two training sessions, two in-person lightning talks, and two virtual lightning talks. You can find the details about these presentations listed below.

Training Neural Networks for Broad Area Search using WATCH

Training Session | Monday, May 22 from 7:30-8:30 AM CT

Speaker: Matt Leotta, Ph.D.

The availability of satellite imagery has grown enormously in recent years, and the data volume is so large it is difficult to locate objects or activities of interest on a large scale. Deep learning can be used to solve this problem by training deep neural networks to analyze the imagery and perform broad area searches for specific spatial, temporal, and spectral signatures. Such networks are typically trained from a collection of annotated examples. However, the logistics of properly training such a network can be tricky, especially in the context of geospatial imagery that comes in a mix of spatial resolutions, spectral bands, revisit rates, etc. To make it easier to train and deploy geospatial search models, Kitware’s team (click here to see our subs) developed an open source software framework called WATCH (Wide Area Terrestrial Change Hypercube). Developed as part of the ongoing IARPA SMART program, WATCH aims to detect and characterize heavy construction sites across time using government and commercial imagery (LandSat, Sentinel-2, Maxar, and Planet). However, this software can be applied to other areas.  In this training session, we will walk participants through an example of leveraging WATCH to train a broad area search model. We will also explain some of the theories and best practices for training such models and summarize some of the latest research advances in the field.

Predicted construction sites in our open source web visualization tool.

Self-Supervised Neural Network Training Techniques for Remote Sensing Applications

Training Session | Tuesday, May 23 from 7:30-8:30 AM CT

Speaker: Scott McCloskey, Ph.D.

Remote sensing methods often use neural networks pre-trained on large ground-level imagery datasets (e.g. Imagenet). Unfortunately, features trained from ground-level imagery show reduced discrimination on remote sensing images. It is also prohibitively costly to manually label huge datasets for remote sensing applications. In this training session, we will examine self-supervised techniques for remote sensing. Self-supervision allows automated remote sensing approaches to leverage large amounts of unlabeled data and improve accuracy across many tasks. Self-supervised techniques do not use labels and can pre-train or fine-tune models to create image embeddings that perform similarly to fully-labeled ImageNet embeddings. The techniques work particularly well for problems with few to no labels. Our team has developed and used these methods on DARPA Learning with Less Labeling (LwLL), DARPA Science of Artificial Intelligence and Learning for Open-world Novelty (SAIL-ON), and AFRL’s VIGILANT and Bomb Damage Assessment programs. Examples of the state-of-the-art techniques we will cover include contrastive learning techniques (SimCLR and SelfSim), reconstructing a masked image with Mask AutoEncoders (MAE), using GSD for superresolution masked reconstruction (ScaleMAE), and object detection-focused techniques (ReSIM and CutLER). Our focus will be effectiveness in image classification and object detection, and we will touch on other applications, such as perceptually lossless compression. This tutorial will introduce GEOINT attendees to the latest research on self-supervised learning in remote sensing (EBK T14. Neural Networks/Artificial Intelligence) so they can apply these techniques to improve automated image analysis results while reducing the need for expensive labeling in their projects.

Updated Kitware Results on the IARPA SMART Program

Lightning Talk

Presenter: Matt Leotta, Ph.D.

The IARPA SMART program aims to research new methods and build a system to search vast catalogs of satellite images from various sources to find and characterize relevant change events as they evolve over time. Kitware is leading one of the performer teams on this program to address the problems of broad area search to find man-made change, with an initial focus on construction sites. This is a “changing needle in a changing haystack” search effort. Our system called WATCH will categorize detected sites into stages of construction with defined geospatial and temporal bounds and predict end dates for activities that are considered in progress. WATCH has been deployed on AWS infrastructure, and the software has been released as open source and is for free community use. This talk will summarize our progress during the first half of the eighteen-month Phase 2 program and share improvements since our early results presented at GEOINT 2022. Notably, our detection scores on validation regions have improved dramatically since last year. We will also highlight our research results, system integration/deployment progress, and open source tools that are now available for community use.

Ground truth construction sites vs. predicted sites overlaid on AI heatmap prediction

Satellite Imagery Forensics: A collaborative effort from DARPA MediFor

Lightning Talk

Presenter: Anthony Hoogs, Ph.D.

To gather meaningful information from geospatial imagery, GEOINT analysts need to trust its authenticity. This is especially important today as we face an elevated threat of disinformation, a rapidly growing number of commercial satellite providers, and quickly evolving AI techniques for image manipulation and generation. Modern forensics technology that is specialized in overhead imagery and can detect and localize manipulations can provide analysts with the tools they need to establish trust. This presentation will highlight Kitware’s collaborative effort on the DARPA MediFor and SemaFor programs. We provide several image forensic analytics from five academic and industry partners through VIGILANT, our satellite imagery exploitation tool (developed by Kitware with funding from AFRL and NGA) that is available with unlimited rights to the government. The great variability in the pedigree of overhead images (panchromatic, multispectral, SAR, metadata, etc.) calls for a versatile suite of forensic tools that leverage domain knowledge to the fullest extent. During this talk, we will also discuss the latest developments in image manipulation localization and characterization of image-text inconsistencies from the DARPA SemaFor program.

Three forensics analytics correctly identified this image as “Very Likely Manipulated where the clouds on the bottom right were added digitally. (Background image © Skybox Imaging, Inc. All Rights Reserved.)

LAVA: Fine-Grained Attribute-Based Object Classification in Overhead Imagery

Virtual Lightning Talk

Presenter: Brian Clipp, Ph.D.

Performing fine-grained classification of aircraft from satellite imagery is challenging due to subtle inter-class differences and long-tailed training sample distributions. Kitware developed Low-Shot Attribute Value Analysis (LAVA) to address these unique challenges. First, we show that automatic keypoint detection using convolutional neural networks (CNNs) can effectively find the nose, tail, and wingtips of aircraft in satellite imagery. Second, because CNNs are not rotation equivariant, they are trained with data augmentation to be rotation invariant. We hypothesize this invariance restricts the ability of CNNs to learn valuable features for fine-grained classification. Instead of learned invariance, we rotate aircraft into a canonical orientation during training and testing, enabling better feature learning and improving performance. We canonicalize aircraft by determining their heading from the detected key-points placed at the nose, tail, and each wingtip. We synthetically rotate aircraft so their nose is to the upper left corner of each training and test image chip. Third, we show that breaking fine-grained object classification into attribute-based sub-classifications, in the form of predicting wing, engine, fuselage, and tail (WEFT) features, outperforms direct prediction of aircraft type while offering better explanations via WEFT attributes (e.g. 2 engines, swept wings, rounded nose). Detected keypoint-based mensuration, including the handling of variable wing sweep aircraft, is used to filter aircraft types, as aircraft may have similar WEFT features but different measured dimensions. While our results are focused on aircraft, keypoint prediction, mensuration, canonicalization, and attribute-based classification generally apply to remote sensing for fine-grained object classification.

Characterizing AI-based Detection Performance for Sensor Designs

Virtual Lightning Talk

Presenter: Scott McCloskey, Ph.D.
This presentation summarizes work recently completed with NGA funding under the SAFFIRE program in which we characterized the performance of machine learning-based object detection models as a function of sensor design parameters, specifically for satellites and the resulting overhead imagery. As part of the study, Kitware extended its eXplainable Artificial Intelligence ToolKit (XAITK) to measure the performance of visual detectors over a range of sensor parameters. This extension leverages an image simulation tool called pyBSM to re-render imagery consistent with different sensor and/or platform configurations. Applying this set of tools to commercial satellite imagery, we characterized the performance of detectors for aircraft and several ground vehicle classes in the NIIRS 5-7 range. Experiments showed that deep network-based detectors have smooth performance degradations within the tested range of parameters. In a separate experiment, we attempted to validate a previously reported finding that the bit depth of overhead imagery has a significant impact on downstream detection; our experiments refuted this earlier work, as we found no significant effect from bit depth. Lastly, we used an end-to-end deep learning algorithm to select sparse multi-spectral bands from commercial satellite imagery that were co-optimized with downstream detection but found only minimal differences based on the specific bands that were selected. Kitware also performed experiments with classified imagery, using similar methodologies to those applied to unclassified data.