Please join us on November 18 for the next installment in the SCHARE Think-a-Thon series, where we will present Assessing Machine Learning Models’ Performance with Python Libraries: Beyond Development – Before Implementation
This webinar, which will take place 2:00–4:30 p.m. Eastern Time, builds on the machine learning models developed in the previous Think-a-Thon and focuses on using Python libraries to assess data, develop model features, and explain model predictions. Through a hands-on, research-focused example, participants will use scikit-learn to assess model performance, Fairlearn to examine performance across population groups, and SHAP to explore the factors driving model predictions. The session will emphasize going beyond a single accuracy score to determine whether a model is fit for purpose, generalizable to varying population groups, and risks for misuse, while also examining the opportunities and risks of using generative AI to assist with Python coding.
Attendees will learn how to use Python libraries and code to:
- Evaluate model predictions with scikit-learn using confusion matrices and metrics such as accuracy, precision, recall, specificity, F1 score, and ROC-AUC
- Assess whether a model is fit for purpose by examining performance, generalizability, target population, and intended application in population health research
- Use Fairlearn to compare model performance across population groups and identify potential disparities that may be hidden by overall performance metrics
- Move from built-in feature importance to SHAP to examine how individual features influence model predictions at both the population and individual level
- Use generative AI as a coding assistant while critically reviewing AI-generated Python code for errors, inappropriate methods, hidden assumptions, and potential sources of bias
Registration Link: Forthcoming