Day 3 Discussion
On day 3, we learned about the importance of trustworthy data and workflows as well as how to present case studies. In the trust-a-thon session today, we want you to investigate the data and workflow of the Space problem and plot some different cases of space weather predictions.
Please have one of your team members reply in the comments to to each of these questions after discussing them with the team. If you have not commented on the posts from the previous days, please add your thoughts there as well.
Here are the Space Weather Jupyter notebooks:
- More beginner oriented: https://github.com/ai2es/tai4es-trustathon-2022/blob/main/space/magnet_lstm_tutorial.ipynb
- More advanced user oriented: https://github.com/ai2es/tai4es-trustathon-2022/blob/main/space/magnet_cnn_tutorial.ipynb
The TAI4ES Space GitHub Readme page is here.
Discussion prompts
- In the lecture series we presented a framework for thinking about the implications of the decisions we make throughout the AI development process. Consider the framework below and answer the following hypothetical questions about the data and workflow you’re using for the Trust-a-thon:
- Where in the data/data collection process do you think there was room for error and/or bias? What are the potential implications of this error and/or bias for the end user?
- Where in your workflow (as well as the workflow outlined in your assigned notebook) is there room for error and/or bias? What are the potential implications of this error and/or bias for the end user?
- How could you leverage social science and or user engagement to mitigate these issues?
- We also talked a lot about using case studies as a way to communicate about AI with end users. Outline how you would present a case study of one of your models to your specific end user. Be as specific as possible.
Notes team 2, Day 3 06/29/2022
1. In the lecture series we presented a framework for thinking about the implications of the decisions we make throughout the AI development process. Consider the framework below and answer the following hypothetical questions about the data and workflow you’re using for the Trust-a-thon: User’s perception of AI/ML trustworthiness diagram
1.1 Where in the data/data collection process do you think there was room for error and/or bias? What are the potential implications of this error and/or bias for the end user?
– Errors could arise due to performance issues with instruments or failing instruments which would have impacts on model performance depending on how important the feature is that the instrument is measuring
– There can be model bias errors depending on the data distribution process. E.g. median loss function can bias against high values while mean loss function can bias against lower values. So a weighted loss function can be useful to address the data distribution.
1.2 Where in your workflow (as well as the workflow outlined in your assigned notebook) is there room for error and/or bias? What are the potential implications of this error and/or bias for the end user?
– Selection of input features could lead to bias
– Selection of unsuitable case studies could lead to bias
– Generally, not working together with the end user and not having their needs in mind could lead to the development of a model that is not suited for the end users’ need
– Model hyperparameters (number of layers in the neural network, drop-out rate etc.) and loss functions can also lead to bias if they do not consider the end user needs and the data distribution. Slow optimization won’t help time sensitive decisions. Imperfect optimization will affect precise decision making in geomagnetic navigation.
– Highlighting feature importance with Shapley values can increase a system’s trustworthiness.
1.3 How could you leverage social science and or user engagement to mitigate these issues?
– Working together with the end user to identify what they need and care about is important
– We need to find out what cases are most relevant for them
2. We also talked a lot about using case studies as a way to communicate about AI with end users. Outline how you would present a case study of one of your models to your specific end user. Be as specific as possible.
– First, it will be important to collaborate with the end user to identify what cases are relevant for them and what kind of case study would be useful for them (a case study of a very strong geomagnetic storm, an average storm, …)
– It would probably be helpful to then present them case studies with the best hits and worst misses to highlight strengths and limitations of the model and to be very clear about when and when not to use the model, i.e., be clear about the failure modes of the model
– It would be nice to have fail-safe design case studies to identify stop-gap solutions in case of failure to precisely predict by these models