A Cross-Cultural Confusion Model for Detecting and Evaluating Students’ Confusion In a Large Classroom (2024 Fall)

Project Overview

This project focused on developing a cross-cultural Convolutional Neural Network (CNN) confusion model, aimed at detecting and alerting instructors of their students’ confusion.

Key Features

• Confusion Model: Developed a CNN confusion model within DeepFace, analyzing over 1 million facial images from 15 classroom videos.
• Validation with SHAP: Used SHapley Additive exPlanations (SHAP) to validate the model, particularly assessing racial differences in confusion detection as highlighted in literature.
• Multimodal Data Integration: Implemented audio-based Retrieval Augmented Generation (RAG) for enhanced insights.
• User Interface: Built an interactive user interface using Streamlit to visualize and manage confusion alerts during 15 classroom sessions.