Focus 2: Use-Inspired Research in Environmental Science

Broad Goals

  1. Use trustworthy AI to provide actionable ES information to diverse users
  2. Enhance scientific and physical understanding of basic ES processes through trustworthy AI.  

These goals focus on five specific use cases to achieve the broader science goals of the Center. The AI2ES Risk Communication (RC) team will interact with the users of selected ES AI models and the development team to investigate and quantify users’ perceptions of, trust in, and uses of the AI models.   

Use Case #1 – Convective Weather 

Motivation: Thunderstorms hazards, e.g., tornadoes and hail, produce billions of dollars in damage and kill hundreds of people each year. Tornado warning skill has not significantly improved in the past decade.

Goals: Develop physically based Explainable AI (XAI) models to improve understanding of severe weather phenomena (tornado and hail formation). Facilitate the development of improved tornado and hail guidance.

  • Develop physically based, robust AI methods that improve convective hazard prediction.
  • Develop and use XAI methods to improve scientific understanding of convective hazards.
  • Develop physically based, robust AI and XAI methods that are deemed trustworthy by forecasters for convective hazard prediction.

Interpretation of convolutional neural networks for tornado predictionInterpretation of convolutional neural networks for tornado prediction. From Lagerquist, McGovern, From Lagerquist, McGovern, Gagne MWR 2020 ICLR Workshop 2020


  • Convective weather is our prototype use-case for the synergistic cycle with AI & risk communication
  • Submitted a paper summarizing our work to data on developing and using AI for convective weather. 
  • Hosted two summer REU students (Lydia Spychalla and Jordan Robinson) who began the work of processing >100 TB of data to make the machine learning possible.
  • Abstract accepted for the NOAA AI conference in September

    Leaders: Snook (OU), Gagne (NCAR), McGovern (OU)

    Members: Becker, Molina (NCAR), Chase, Diochnos, Homeyer (OU), Foster, Griffin (Disaster Tech), Lagerquist (CSU/NOAA), Kumler, Potvin, Stewart (NOAA)

NY snowfall prediction graphicEstimated 24 hour snow accumulation derived from automated snow depth observations during the December 16-17th snowstorm. Image created by Nick Bassill using NYS Mesonet observations. Current data available here.

Use Case #2 – Winter weather

Motivation: Winter weather is a major hazard in the US, with societal impacts such as travel (road, air, rail) and utilities. An opportunity exists to increase resiliency through exploitation of weather data and AI.

Goal: Develop AI methodologies to exploit underutilized winter weather data (observations and forecasts) and non-meteorological sources (e.g., roadway data) to provide tailored guidance to emergency managers and decision makers.

  • Train graduate students and postdoc in interdisciplinary research and Explainable AI (XAI) methods for winter weather applications.
  • Improve our knowledge and understanding of the nature and predictability of winter weather from synoptic to local scale.
    • Use XAI to identify features most predictive for short-term forecasting of winter events 
  • Develop physics-based AI that improves winter precipitation forecasts including precipitation type and transitions.
    • Develop and use XAI methods using data from New York State Mesonet and Oklahoma Mesonet as well as other meteorological data and non-traditional data to create tailored analyses and predictions to support local and state-level transportation decision-making before and during winter weather events.
  • Quantify uncertainty of AI predictions for winter weather prediction, identify the relationship between uncertainty and trustworthiness, and communicate the uncertainty to And end-users


  • Data acquisition underway including redundant data copy/synchronization of NYSM on xCITE servers, GFS, NAM, and HRRR models.
  • Data pre-processing script development underway.
  • Development of training tutorials for ML, SLURM, Kubernetes for effectively running large modeling tasks.
  • Initial research on XAI model development started.
  • Establish methodology for image labeling using NYSM and ASOS data. Other labeling methods explored (e.g., canny edge map, transmission map, contrast)
  • Hosted two summer REU students
    • Joshua Pan – Visibility Classification:

      • Hand-crafted features: Contrast, edge detection and transmission; Densely connected network; Testing accuracy: 74.6%
      • Full approach: Raw images as input; Convolutional neural network; Testing accuracy: 60.1%
    • Kaelia Okamura – Precipitation Classification: Automatic ground-truth classification of 59,207 images; Classification of grayscale images higher accuracy than color images; Grayscale validation set classification accuracy: 83%
  • Begin ML development for image classification using CNN

    Leaders: Thorncroft (Albany), Sulia (Albany), Fagg (OU)

    Members: Bassill, Kurbanovas, Gaudet, Torn, Tyle (Albany), Becker, Gagne, Schreck (NCAR), (Albany), Foster, Griffin (Disaster Tech), Diochnos, McGovern (OU), Stewart (NOAA)

Use Case #3 – Tropical Cyclones

Motivation: Tropical cyclones (TCs) have tremendous societal impact in terms of damage and flooding. 

Goal: Improve understanding of TC temporal evolution and rapid intensification to improve understanding and forecasting of TCs.

  • Develop a physics-based AI algorithm that generates synthetic microwave imagery from satellite data
  • Develop physics-based AI algorithms to improve TC prediction including: rainfall and wind-field estimation as well as and intensity changes
  • Use Explainable AI (XAI) to discover relationships between TC structure evolution and rapid intensification onset

tropical cyclone prediction improvement with AI

 Adapted from C. J. Slocum and J. Knaff, Using Geostationary Imagery to Peer through the Clouds Revealing Hurricane Structure, AMS annual meeting, 19th Conference on Artificial Intelligence for Environmental Science, Wed, Jan 15, 2020



  • Hired a research scientist (Jan 2021) and a postdoc (July 2021).
  • Collected and pre-processed data. 
  • Developed simple, pixel-based, fully connected neural network algorithm.  Performed hyper-parameter optimization.
  • Developed CNN for image-to-image translation. Optimization in progress.
  • Currently evaluating algorithms, including comparison to existing random forest algorithm. 

Leaders: Ebert-Uphoff, Musgrave (CSU), Thorncroft (Albany), Gagne (NCAR)

Members: Foster, Griffin (Disaster Tech), Hall (NVIDIA), Haynes (CSU), Kumler, Stewart (NOAA)

Use Case #4 – Subseasonal to Seasonal (S2S) Prediction of Extreme Weather

Motivation: S2S prediction will improve resiliency as the climate changes.

Goal: Predict extreme weather two weeks to two months ahead (seeCongressional Weather Research and Forecasting Innovation Act of 2017).

  • Develop physics-based AI to improve prediction of extreme weather at S2S scales
  • Develop and use Explainable AI (XAI) methods to identify sources of predictability (and their physical mechanisms) on S2S timescales.
  • Develop and use XAI methods to leverage imperfect dynamical model simulations, along with observations, to make more accurate S2S predictions of the real world at the S2S timescales (i.e. through transfer learning)

Sub-seasonal to seasonal scale gapsAdapted from the Subseasonal Prediction Project, Earth Institute, Columbia University.


  • Started obtaining hindcast and climate model data for developing, testing and implementing transfer-learning framework.
  • Started research on applying transfer learning between climate models and ERA5 observations to improve S2S predictions of precipitation and temperature
  • Submitted two papers to JAMES (on arxiv as well) on “abstention networks” for skillful forecasts of opportunity
  • Wrote blog post sharing new abstention network research

Leaders: Barnes (CSU), Ebert-Uphoff (CSU), Tissot (TAMUCC)

Members: He (NCSU), Hickey (Google), Molina (NCAR), Stewart (NOAA), Williams (IBM)

Use Case #5 – Coastal Oceanography

Motivation: Coastal phenomena impact humanity on a regular basis.  Improving our prediction of coastal events will save lives and property.

Goal: Develop physics-based AI methodologies to provide more accurate predictions of Ocean eddy shedding, Harmful algal blooms, and Compound flooding.

  • Create shared multivariate high-resolution data set for the Gulf of Mexico and U.S. east coast shelf seas that will support the specific use-cases and AI
  • Design a common deep learning platform allowing for the spatio-temporal characterization of air-sea-land interactions while accommodating different spatial resolutions
  • Implement, test, and validate (with RC) the AI predictions with stakeholders
  • Develop physics-based AI to improve the prediction and understanding of:
    • Cold-stunning events of sea turtles in the Laguna Madre
    • Timing and intensity of marine fog
    • Coastal beach flooding (cross-cutting with TC) 
    • Compound flooding from storm-surge and rainfall (cross-cutting with TC)
    • Loop Current Eddy shedding in the Gulf of Mexico
    • General trend, timing, and locations of harmful algal blooms

Graph of green turtle count.

Estimated green turtle abundance over time, compared with water temperature. Courtesy of Dr. Amy McGovern, University of Oklahoma – Norman.


Loop Current

Loop Current pinched-off eddy in the Gulf of Mexico. over 4,000 oil and gas rigs in the northern Gulf waters are at risk of damage from these eddies. From the CNAPS model of the Ocean Observing and Modeling Group, North Carolina State University.



  • The paper describing FogNet (Kamangir et al.) was accepted “FogNet: A Multiscale 3D CNN with Double-Branch Dense Block and Attention Mechanism for Fog Prediction” in  Machine Learning with Applications 
  • The performance of FogNet was compared to operational models, HREF, SREF showing substantial performance improvement.
  • Two other journal submissions in review
  • Working with IBM on potential R2O VAE implementation of coastal fog predictions
  • Tested Channel-Wise PartitionSHAP on 13-channel EuroSAT dataset (FogNet)
  • Created experiment where adding an additional channel (from RGB to RGB & NIR) expected to allow CNN to learn strategy. Channel-Wise PartitionSHAP results strongly suggest that model exploited the additional channel (FogNet)
  • Development on visualization tool progressing (FogNet)
  • Used on analysis of FogNet paper currently under submission (FogNet)
  • Labeled over 3,000 coastal images (Coastal Inundation Predictions)
  • Processing of images from Florida and Texas (Coastal Inundation Predictions)
  • Developed methods to label orthomosaics with ESRI software (Coastal Inundation Predictions)
  • Undergraduate students developing water level predictions visualizations for stakeholders and scientists (Coastal Inundation Predictions)
  • Completed first drone flight to acquire beach imagery including wet/dry line. Ready to acquire more imagery in different conditions (Coastal Inundation Predictions)
  • Provided guidance including recommended start and stop of navigation interruptions in the Laguna Madre during the largest sea turtle cold stunning event in recorded US history (February 2021).
  • Developed visualisations of IBM ensemble predictions and NWS predictions to help with cold stunning prediction guidance (Sea Turtle Conservation Models)
  • Held meeting with state and federal agency stakeholders to review predictions. Invited presentation at industry navigation association annual meeting (Sea Turtle Conservation Models)
  • Long-term ocean observations including both remote sensing and in situ ocean data have been collected. A 26-years (1993-2018) data assimilative ocean circulation reanalysis is being produced. These observations and numerical model output will be used for AI model training and testing starting in fall 2021
  • Self-organizing map (SOM) method was applied, which can effectively cluster LC patterns (Meso-scale Ocean Eddy Prediction)
  • A preliminary  ML (ANN) model for the LC variations and eddy shedding process has been developed (Meso-scale Ocean Eddy Prediction)
  • Further validation and refinements of this LC ML model is ongoing (Meso-scale Ocean Eddy Prediction)
  • One NCSU PhD student (Laura McGee) has been working on this ML technique and remote sensing data analysis as a part of her Ph.D. dissertation (Cloud-free satellite marine data reconstructions)
  • Based on MODIS observations, we have successfully reconstructed a 18-year (2003-2020) cloud-free, daily time series for SST, CHL, POC, and PIC for the US Atlantic coast and the GoMEX waters (Cloud-free satellite marine data reconstructions)
  • The product is being used for feature detection and extreme event prediction (Cloud-free satellite marine data reconstructions)

Leaders: He (NCSU), Tissot (TAMUCC), Williams (IBM), McGovern (OU)

Members: Gray, Lowe, McGee, Warrillow, Zambon (NCSU), B. Colburn, K. Colburn, Dinh, Duff, Durham, Estrada, Flores, Huang, King, Kamangir,  Krell, Medrano, Nguyen, Starek, Vicens Miquel (TAMUCC), Collins (NOAA), Demuth, Gagne (NCAR), Bostrom (UW)