Day 3 Discussion
On day 3, we learned about the importance of trustworthy data and workflows as well as how to present case studies. In the trust-a-thon session today, we want you to investigate the data and workflow of the Severe problem and plot the predictions and observations for individual storms to identify potential biases and other issues in the model.
Please have one of your team members reply in the comments to to each of these questions after discussing them with the team. If you have not commented on the posts from the previous days, please add your thoughts there as well.
Here is the TAI4ES Severe notebook directory for reference. Revisit the training and XAI notebooks and use the modules to plot individual case studies and investigate the workflow in more detail.
- In the lecture series we presented a framework for thinking about the implications of the decisions we make throughout the AI development process. Consider the framework below and answer the following hypothetical questions about the data and workflow you’re using for the Trust-a-thon:
- Where in the data/data collection process do you think there was room for error and/or bias? What are the potential implications of this error and/or bias for the end user?
- Where in your workflow (as well as the workflow outlined in your assigned notebook) is there room for error and/or bias? What are the potential implications of this error and/or bias for the end user?
- How could you leverage social science and or user engagement to mitigate these issues?
- We also talked a lot about using case studies as a way to communicate about AI with end users. Outline how you would present a case study of one of your models to your specific end user. Be as specific as possible.