Lav VarshneyDepartment of Electrical and Computer Engineering
varshney@illinois.edu
Research Topic
Information Lattices for Interpretable Knowledge Discovery and Nearest-Neighbor Classification
Research Summary
We plan to investigate a novel, human-interpretable machine learning technique for knowledge discovery and nearest-neighbor classification. This non-neural network technique is called information lattice learning and draws on mathematical techniques from information theory and group theory. Thus far we have shown it achieves state-of-the-art performance in image classification in the regime of limited training data per class. The proposed work will extend and apply the technique to geospatial data. Moreover, a hybrid human-interpretable machine learning architecture that combines deep learning with information lattice learning will be developed.
Melkior Ornik
Department of Aerospace Engineering
mornik@illinois.edu
Research Topic
LEONA: Logic-Based Context-Aware Activity Interpreter for Geospatial Intelligence
Research Summary
Successful geospatial intelligence critically relies on meaningful, correct, and quick interpretation of patterns gathered from data. While the quantity of available raw data is often overwhelming for a human analyst, statistical and machine learning methods for pattern identification often lack necessary context about the relevance of observed features. The objective of our project is to bridge the gap between context and computation. We will develop algorithms and demonstrations of automatic inference of important patterns and activities through time, aware of the environmental context, and easily communicable to a human analyst. The underlying framework will be built on a combination of temporal logic to describe inferred activities and communicate them to a human in an understandable manner, Markov decision processes to describe the underlying environmental context, and information theory to recognize relevant patterns from vast collected data. With a vision for a long-term research program, we will use the proposed summer period to develop a prototype of our algorithms and demonstrate it on geospatial data collected from openly available datasets of New York City.
Svetlana Lazebnik
Department of Computer Science
slazebni@illinois.edu
Research Topic
Lifelong Visual Learning for Multi-Modal Data
Research Summary
This project is focused on the creation of a lifelong learning system for large-scale, high-dimensional data with mixed modalities (such as images/video, language/audio, and various kinds of tags) and flexible supervision. This system will be able to learn from a stream of data that may change over time, and perform tasks that may also change or be defined by users on the fly with few or no training examples. This system will not be reliant on full supervision, instead of taking advantage of self-supervised learning by cross-modal prediction. This project will systematically investigate several open research problems, including automatic discovery of cross-modal supervision objectives, mechanisms for coping with a changing data distribution (including prevention of catastrophic forgetting and change detection), and acquisition of new skills with minimal user specification. The proposed system will also be used to derive compressed data representations for fast large-scale search and interactive querying. This work will build on the PI’s successful track record of research into cross-modal image representations, incremental learning, and large-scale image search.
Kevin Chang
Department of Computer Science
kcchang@illinois.edu
Research Topic
Automating Human Geospatial Intelligence
Research Summary
In the last few years, deep learning has, in conjunction with computer vision techniques, rapidly advanced to allow the classification of objects accurately and efficiently. Some of the most important objects to classify from satellite or unmanned aerial vehicle (UAV) images are motor vehicles. Counting and classification of motor vehicles are critical components of geospatial intelligence (GEOINT) and the National Geospatial-Intelligence Agency (NGA) has previously sponsored research related to identifying vehicles in images. The proposed study involves the development of an enhanced vehicle classification framework which can be used to identify specific vehicle types from aerial images. Aerial image data from labeling to validation will be parameterized and monitored to feed into neural networks. The new framework will lay special focus on Machine Learning (ML) data pipelines, to improve the quality of production ML on the operationalization of models for accurate vehicle classification and counting. Transfer learning will be used on different neural network architectures such as CenterNet, YOLOv4, and YOLOv5, which are all single stage object detection algorithms. High-performance Blue Waters GPUs can be used to train and validate the classification model.
Ramez Hajj
Department of Civil and Environmental Engineering
rhajj@illinois.edu
Research Topic
Computer Vision-Based Vehicle Classification Framework
Research Summary
In the last few years, deep learning has, in conjunction with computer vision techniques, rapidly advanced to allow the classification of objects accurately and efficiently. Some of the most important objects to classify from satellite or unmanned aerial vehicle (UAV) images are motor vehicles. Counting and classification of motor vehicles are critical components of geospatial intelligence (GEOINT) and the National Geospatial-Intelligence Agency (NGA) has previously sponsored research related to identifying vehicles in images. The proposed study involves the development of an enhanced vehicle classification framework that can be used to identify specific vehicle types from aerial images. Aerial image data from labeling to validation will be parameterized and monitored to feed into neural networks. The new framework will lay special focus on Machine Learning (ML) data pipelines, to improve the quality of production ML on the operationalization of models for accurate vehicle classification and counting. Transfer learning will be used on different neural network architectures such as CenterNet, YOLOv4, and YOLOv5, which are all single-stage object detection algorithms. High-performance Blue Waters GPUs can be used to train and validate the classification model.
Roman Makhnenko
Department of Civil and Environmental Engineering
Research Topic
Geospatial monitoring for safegeologic carbon storage
Research Summary
In alignment with the National Geospatial-Intelligence Agency goals, we propose to use geospatial monitoring to observe the surface deformation near the CO2 injection sites in Illinois. The advanced analytics and modeling of the reservoir behavior will allow for calibration of the relationships that predict the uplift of the surface due to the deep injection, as well as the potential movement of the CO2 plume and its leakages. The models are based on the laboratory and field geophysical data and previous experiences show that accurate use of geospatial monitoring allows for prediction of reservoir behavior and detection of the discontinuities. A realistic numerical model needs to solve billions of equations at every time step, so the use of supercomputers is required. The combination of accurate prediction for the storage site responses with the numerical modeling allows for safer operation of the technologies aimed at mitigating climate change. This perfectly fits into the NGA’s mission to address the global environment changes and meet XXIst-century challenges.