Fudan University's 'Data Mining' course final exam replaced traditional tests with a 'Human vs AI' format where 51 students each created 10 calculation questions to challenge three AI models; the more the AI answered incorrectly, the higher the student's score. Only 4 students managed to make any AI model score zero, and the strongest model, Claude, was never completely defeated. This innovative assessment reflects a shift in pedagogy from memorization to AI literacy, emphasizing the ability to direct and critique AI. It signals a broader rethinking of education in the AI era, where teaching students how to leverage and evaluate AI becomes more important than traditional knowledge recall. The three AI models evaluated were not named except for Claude (by Anthropic), which proved the most robust. The class average score was 85.7 points, and 50 out of 51 students managed to stump at least one model at least once. Instructor Xiao Yanghua noted that traditional exams are obsolete in the AI era.
Background
AI literacy is an emerging skill set that goes beyond using AI tools to understanding their capabilities, limitations, and ethical implications. Adversarial testing, often used to probe AI robustness, involves deliberately crafting inputs to fool AI models. This exam is a practical application of such concepts in education, training students to think critically about AI behavior.