Focus 1: Foundational Research in Trustworthy Artificial Intelligence / Machine Learning


Broad Goals

1. Develop explainable AI (XAI) methods aligned with environmental science domain perspectives and priorities

  • Develop XAI methods for ES data (including regression-based predictions, data with high spatiotemporal autocorrelations, and fielded data) 
  • Develop XAI methods that integrate physics into the explanations 
  • Develop XAI methods to explain AI model failures 
  • Develop XAI methods that facilitate knowledge and hypothesis discovery 
  • Develop XAI approaches to effectively communicate (measured through RC research) estimated uncertainty to the end user and tailor these to the needs of the end-user  

real-time visualization identifying supercells and squall line segments for forecastersReal-time storm morphology prediction with self-supervised Convolutional Neural Networks (CNNs). Built real-time visualization identifying supercells and squall line segments for forecasters from CNN trained on proxy task.NCAR Realtime Forecasts

CNN analysis of severe hailstorms graphicApplication: Convolutional Neural Network analysis of severe hail storms. Saliency maps identify different types of storms that produce severe hail and enable generation of storm type distributions.Interpretable Deep Learning for Spatial Analysis of Severe Hailstorms (Gagne et al. 2019)

 

Progress

  • Developed benchmark to evaluate XAI methods for ES applications (paper under review, figure above).
  • Paper: Analyzing Severe Storms in a Future Climate with XAI
  • Created PyMint XAI Python package

Leaders: McGovern (OU), Gagne (NCAR), Ebert-Uphoff (CSU), Barnes (CSU)

Members: Anderson, Mamalakis, McGraw, Musgrave, Stock (CSU). Lagerquist (CSU/NOAA). Gantos, Molina (NCAR). Potvin, Stewart (NOAA). Fagg, Diochnos (OU).    

2. Develop physics-based AI techniques for environmental science domains

Physical constraints and physics-based AI/ML give us:

  • Robust feature creation
  • Physical, semantic, and redundancy constraints on generated features
  • Physics-constrained loss functions
  • Conditional hybrids of physical model and AI system predictions
  • Novel architectures for XAI

Overall Goal: Improved AI predictions and understanding of physically-based ES phenomena

  • Ensure AI methods fail gracefully, e.g. failure is always physically plausible 
  • Develop physics-guided approaches to autonomous feature discovery
  • Develop hybrid models that incorporate physics-based AI

Progress:


Leaders: McGovern (OU), Hickey (Google), Gagne (NCAR), Ebert-Uphoff (CSU)

Members: Kurbanovas (Albany). Anderson, Barnes, McGraw, Musgrave, Stock, Ver Hoef (CSU). Lagerquist (CSU/NOAA). Becker, Schreck (NCAR). Potvin, Stewart (NOAA). Chase (OU). Snook (OU).

3. Develop robust AI prediction techniques, and empirically and theoretically validate their performance with adversarial data (e.g., missing data or intentionally wrong data).

Some ES datasets are limited by data collection challenges and the rareness of extreme events. AI2ES will address robust AI (theoretically and empirically) with the following approaches:

  • Class imbalance: characterize theoretical sample size needed to meet important verification metrics
  • Transfer learning: train ML on common phenomena and transfer to rare events; train within simulations and transfer to observations
  • Self-supervised learning: identify proxy supervised learning tasks that most aid in prediction when reliable labels are expensive or unavailable
  • Adversarial Data/Classifier Robustness: provide theoretical guarantees for robustness of ES ML models to intentionally or accidentally adversarial data

Goals:

  • Develop methods to allow transfer learning to be used to train AI models for rare phenomena 
  • Develop robust semi-supervised and unsupervised learning algorithms for situations where reliable labels are not available
  • Develop theoretical and practical bounds on the robustness of the AI methods given class imbalance, a lack of reliable labels, and for adversarial situations (e.g. data may be missing or corrupted based on weather conditions)
Map of Oklahoma Mesonet highest wind gusts by county
Map of Oklahoma Mesonet highest wind gusts by county, May 24, 2011.
damaged MESONET wind station
El Reno Mesonet station after recording 151 mph wind gust on May 24, 2011.

Progress:

  • Developed and refined methods for neural networks to say “I Don’t Know” and learn the more confident samples better, termed controlled abstention networks (CAN)
  • Submitted two papers on CAN (Barnes and Barnes 2021a,b) to JAMES for earth science applications
  • Subdivision algorithm for computing a lower bound on the rate of the minority class
  • Reduction of a learning problem with high recall and high precision, to a learning problem with improved risk bound, within Probably Approximately Correct (PAC) learning
  • Paper in SIAM International Conference on Data Mining, SDM 2021

Leaders: McGovern (OU), Gagne (NCAR), Diochnos (OU)

Members: Kurbanovas (Albany). Anderson, Ebert-Uphoff, Musgrave, Stock (CSU). Lagerquist (CSU/NOAA). Williams (IBM). Becker, Molina (NCAR). Potvin, Stewart (NOAA). Flansburg (OU). Medrano, Vicens Miquel (TAMUCC).