DARPA VIRAT, DARPA PerSEAS, & DARPA BLUE:

Detection of Events, Complex Activities, and Behaviors from Videos and Multi-INT (2006-2015)

Exploit existing detections and tracks from video to automatically detect behaviors associated with individuals and groups in surveillance and sports domains. Traditional probabilistic approaches with relational and contextual constraints are key to improved performance.

Publications: 2015 Thesis, 2014 CVPR, 2013 TPAMI, 2011 CVPR,  2009 WMVC, 2009 Learning

DARPA XAI & XAI-Toolkit:

Explainable Artificial Intelligence and Program Level Toolkit

(2020-2022)

Develop novel deep network models and GUIs to explain the reasoning behind a model’s decision. Focused on developing more efficient and accurate XAI based iterative query refinement tasks while leading the integration of the XAI artifacts from the entire program into a publicly releasable toolkit.

*This material is based upon work supported by the United States Air Force and DARPA under Cooperative Agreement numbers N66001-17-2-4028 and FA875021C0130. The views, opinions, and/or findings expressed are those of the author(s) and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government.

Product, Project, and Program Experience

Asset Protection Product: In-Store Shrink Assessment and Avoidance (2022-2023)

Reduce the $2.2B+ in front-end shrink in Walmart Stores along with the $4.4B+ in non-frontend shrink, at scale, through data driven shrink assessment, deterrence, and capture. 

Gulfstream/Multi-Signal Product: Next Generation Checkout (2022-2023)

Develop and deploy a frictionless customer shopping and checkout experience that optimizes labor and reduces shrink across the entire store using multiple signals and products. 

Supply Chain (Scorpion) Product: Distribution Center Automated Pallet Auditor (2022-2023)

Develop and deploy an automated pallet auditor, Scorpion, to alert the order fillers when the incorrect number or type of items are placed on the pallet in the distribution centers. 

DARPA ARAKNID: Anytime Reasoning and Analysis for Kill-Web Negotiation and Instantiation across Domains (2020-2022)

Automated battle management with optimal asset allocation for mission tasking among ground, air, and space vehicles. Sub to Raytheon BBN. 

DARPA SAIL-ON: Novelty Detection and Domain Adaptation (2019-2022)

Train models for image classification, activity video classification, and document transcription then evaluate on similar datasets that introduce novelties. Assess the system’s capability to detect the change in the world and then rapidly adapt to it. 

DARPA LEARN: Learning with Less Labels (LwLL) and Domain Adaptation (2019-2022)

Adapt pre-trained models for image classification, object detection, and activity video classification to a new domain with other source data and a limited number of labeled samples. Improve system performance with few labels. 

DARPA PANDA, & NGA SPOTTER: Normalcy Modeling, Anomaly Detection, Multi-modal Fusion, Long-term Patterns (2006-2008, 2018-2022)

Process overhead persistent satellite imagery and other traffic based data sources to learn spatial and temporal normalcy models and to detect anomalies relative to these models. Both traditional probabilistic and deep network based approaches are used to capture evidence and detect anomalies. 

Blackjack Pit Boss and Ball Yeti: Low SWaP Object Detection and Tracking for Satellites (2019-2021)

Develop multi-satellite detection fusion and track update software along with low-SWaP deep network object detection in EO overhead imagery for integration on low-earth-orbit satellites

DARPA SquadX: Small Object Detection and Threat Classification (2015-2019)

Develop deep network based small object detection and threat classification algorithms using ground and UAV imagery. Integrate software on moving platforms for real-time processing with low-SWaP devices.

DARPA Object Function: Functional Analysis and Anomaly Detection from Videos (2006-2015)

Pioneered Functional Analysis in surveillance video, where objects and scene elements are detected based on how other entities interact with them as opposed to their appearance. Behavior profiles are extracted for normalcy modeling and anomalous detection. 

Lockheed Martin MDA: Tracking and Maneuver Detection with Ballistic Missiles from Radar (2002-2006)

Developed a multi-object tracking algorithm that is capable of forming accurate tracks on boosting and ballistic missiles along with detecting burnout events and payload maneuvers all based on radar range and angle measurements.