Knowledge Transfer

For the NSF AI Institute for Research on Trustworthy AI in Weather, Climate, and Coastal Oceanography (AI2ES)

Knowledge transfer and collaboration are critical components of AI2ES and will ensure that we successfully reach our goals.  Knowledge transfer activities involve the mutual exchange of scientific information and may be accomplished in a variety of ways.  Below we outline how we plan to maximize knowledge transfer both at the Lead Institution (OU) and among partners, including industry, government, colleges and universities, and the public. 

 

Software and Data Sharing Policy 

We have decided to share software products and data in open-source format to make them fully accessible to external entities, including colleges and universities, government, industry, and the public.  This applies to any substantial software, products and data.  We recognize that writing reusable code requires good coding practices, so we plan to develop institute-wide standards and offer required training to all members contributing code (for more information, see “Developing standards” in the Integration section).  Furthermore, making large data sets available requires significant resources, and will benefit from assistance from our industry and government partners.

 

Levels of Knowledge Transfer 

For knowledge transfer we first acknowledge that some types of knowledge are easier to transfer than others, and thus require different means of knowledge transfer.

Factors that determine transferability include:

  • Amount of specialized expertise required by the person utilizing the new knowledge. 
  • Amount of customization required to utilize knowledge for a new application.
  • “Approval factor”, i.e. the attitude of the relevant community toward acceptance and integration of the new knowledge.

Given the varying amount of effort required to transfer different types of knowledge we need to make strategic decisions as to 

  • Which knowledge we can realistically transfer to large communities within the 5 years of the institute,
    versus 
  • Which knowledge we might only be able to transfer to smaller, carefully selected groups representing key high impact applications. 

We roughly distinguish three different levels of transferability.

 

Level 1: Difficulty of Transferability – Easy

Characteristics:  Any knowledge that is easy for a relative novice to apply and that generalizes to many applications without requiring any customization.

Examples: Insights such as how-to guidelines or best practice recommendations.  Or a code snippet that can just replace or be added to an existing algorithm without requiring any other changes.
Suitable means of knowledge transfer (combination of any of the following): 

  • Research papers, 
  • Github tutorial providing key code snippets, and demonstrating use for a simple example.
  • Workshops/tutorials at conferences.
  • Guest lectures.
  • Video tutorials  

Target audience:  Lead Institution, partners, including industry, government, colleges and universities, entire research communities, and the public.

Level 2: Difficulty of Transferability – Medium

Characteristics: Knowledge that requires significant expertise by the person gaining the knowledge to apply to their own application, and might require help from an AI2ES expert but only for a short amount of time (no major research effort).  
Example: Some XAI methods that can be added to AI models without requiring much customization, e.g. permutation invariance or ablation studies. 
Two goals of knowledge transfer at this level of difficulty:

  1. Intensive training of a member of the target application team to apply
  2. Intensive training of a member of a key target community to be able to teach knowledge to others. Target community might be key stakeholders, and/or STEM minority researchers to give them an extra edge (e.g., Del Mar college). 

Suggested means of knowledge transfer: 

  • All means from Level 1: research papers, github tutorials, workshops/tutorials at conferences, guest lectures, and video tutorials.
  • Short term immersion visits (can be virtual).

Target audience:  Lead Institution, partners, including industry, government, college and university partners.  Selected stakeholders and community representatives who can disseminate the knowledge further in their respective communities.

Level 3: Difficulty of Transferability – Most Difficult / Most Time-Consuming

Characteristics: Knowledge that requires a significant amount of additional research effort to be utilized for a new application.
Examples: 
(1) A risk communication approach that requires one to understand end users’ decision contexts and perceptions, uses, and needs for AI information.  This requires deeply collaborative research that integrates social science expertise with computer and atmospheric science.
(2) A physics-guided machine learning approach that requires one to first identify physical constraints (e.g., conservation of energy, etc.) that are important in an application, then to integrate those constraints in the AI method using a variety of different options.  This requires an AI scientist with very specific knowledge as well as a domain expert with deep knowledge of the application to be part of the research effort.
Either process can take years for a new application!
Suggested means of knowledge transfer:  this requires long-term interactions

  • Cohort training in institute’s research labs by having folks from different disciplines work together in the same lab. This generates more experts who can do this – benefiting workforce development – and uses the fact that students are a very efficient way to transfer knowledge between different disciplines and labs.
  • External members from stakeholder communities serving on student committees from the start.  Knowledge transfer occurs by following student’s challenges and progress over the years.
  • Long term or regular immersion visits (e.g., 1-2 weeks per quarter) where a member (e.g., student) of one team agrees to spend time exclusively with another team.  These can be virtual.
  • Creation of working groups in stakeholder communities, such as the “LEAP-RT” working group at NOAA created and led by Imme Ebert-Uphoff for ML for radiative transfer. This could be expanded to other components of the LEAP project. Such as for Data Assimilation (LEAP-DA).
  • Development and sharing of a framework and methodologies for conducting collaborative computer-atmospheric-social science research that can guide and be tailored by other scientists.
  • Collaborative Testing in an O2R environment and undertaking of R2O transitions to transfer software, applications, systems resulting from the AI2ES activities, into the target institution (Industry or Government).

Target audience: 

Note that Level 3 requires a high-level of long-term support/involvement from an expert within the institute, which limits the number of applications we can tackle.  

Targets for Knowledge Transfer at Levels 1, 2, and 3:  

  • We see no limits to transferring knowledge of Level 1 and 2. 
  • For Level 3 we can only start to engage with roughly two new applications / stakeholder groups per year.  Once an application is underway, we can add two new applications in the following year.  
  • These limits are reflected in the milestones throughout this document.

Research to operations (R2O)

Research to operations refers to the transfer of knowledge into software systems or business practices of government or industry entities. Target entities include both industry and government partners of the Institute, those who may participate under the umbrella of the IAB, or others who may utilize either knowledge or software created by the Institute without explicit ties or even visibility to the Institute.

R2O can occur at multiple levels, similar to the levels of knowledge transfer outlined above. At a simple level, R2O can involve stakeholder adoption or adaptation of ideas, methodologies, best practice recommendations and software tools. An intermediate level would involve following templates or adopting code snippets. The most extensive level of R2O would involve stakeholder participation in the Institute-led open source project and direct adoption of software provided there. Performing this level of R2O will likely require training Institute researchers to create containerized code capable of running agnostically on any cloud platform. Training and guidance from Institute partners will be essential to establishing this capability and fully realizing the potential benefit of R2O.

Strategies to enhance R2O will include:

  • Working closely with our Industrial Advisory board.
  • Working closely with Govt partners
  • Developing code that works on the cloud.
  • Collaborate early with target system of the R2O effort (O2R environment)
  • Ensuring all code that is developed adheres to the AI2ES coding standards and practices.
  • Training AI2ES researchers how to create operational/containerized code, in collaboration with the Operational team.