| Course Subject: | Economics (Statistics for Economists; Economics of Inequality) |
| Student Level: | Undergraduate |
| Number of Students: | Approximately 60 per semester (two sections of ~30) |
| Developed by: | Gustavo Vicentini, Faculty, Department of Economics, College of Social Sciences and Humanities (CSSH) |
What the Students Did
Students in an undergraduate Statistics for Economists course completed a take-home assignment that had been redesigned to incorporate direct engagement with generative AI. The assignment is centered on problems requiring algebraic calculations and careful reasoning, and that were known by the instructor to be solved incorrectly by generative AI tools. Students first solved the problem on their own without AI assistance. They then entered the same question prompt into Claude and into ChatGPT, reviewed each tool’s response, and identified incorrect answers in the AI-generated responses and what was incorrect about them.
Purpose
The assignment’s purpose was to develop students’ ability to critically evaluate AI-generated output in the context of quantitative economic reasoning. Key lessons were that AI tools can produce incorrect answers to quantitative problems, that the ability to independently verify AI output is an essential skill, and that different AI tools may generate different responses including differently wrong to the same prompt.
The assignment was created in response to a prior semester when approximately 85% of students submitted incorrect answers that closely resembled the plausible but wrong outputs produced by both Claude and ChatGPT. By requiring students to solve the problem themselves before consulting AI, the assignment reinforced the importance of foundational analytical skills as a prerequisite for meaningful AI use. The design also aimed to help students develop a more discerning and evidence-based approach to integrating AI into their academic and professional work.
Assessment
Students were evaluated on two components. First, they were assessed on whether they could solve the original quantitative problem correctly without AI assistance. Second, they were assessed on the quality of their written explanations identifying where and why each AI tool’s answer was conceptually incorrect. The assessment focused on students’ reasoning and their ability to articulate the specific errors in the AI-generated responses rather than simply noting that the answers were wrong.
Faculty Reflections
I have been sharing this approach with colleagues in the economics department, including our “AI and Teaching” ad hoc committee, and the strategy has generated a lot of interest. The redesign itself is simple to implement; any instructor who finds a question where AI produces an incorrect answer can apply this same structure. I have also been developing other AI-integrated assignment redesigns for different contexts such as using AI to assist with guided academic literature reviews, identifying data sources for empirical analysis, and validating AI-provided economic data across sources such as the U.S. Census Bureau, the OECD, and academic research centers.
My advice to colleagues considering something similar is to start by testing your existing assignment prompts in one or two AI tools. If you find a question where AI fails, you already have the raw material for a productive redesign. This approach does not require any new technology or major course restructuring—just a willingness to make AI’s limitations visible to students.
Step-by-Step Directions
| Step 1 | Solve the assigned quantitative problem on your own, without using any AI tools. Show your work, including all algebraic calculations and reasoning. |
| Step 2 | Copy the full question prompt and paste it into Claude. Review the response Claude provides. |
| Step 3 | Write a clear explanation of why Claude’s answer is incorrect. Identify the specific conceptual or computational error in its reasoning. |
| Step 4 | Copy the same question prompt and paste it into ChatGPT. Review the response ChatGPT provides. |
| Step 5 | Write a clear explanation of why ChatGPT’s answer is incorrect. Identify the specific conceptual or computational error in its reasoning, and note how it differs from Claude’s error. |
| Step 6 | Submit your complete assignment, including your own solution (Step 1), both AI-generated responses, and your written evaluations of each (Steps 3 and 5). |
Note for instructors: This assignment structure works when the question prompt reliably produces incorrect AI output. Before deploying the assignment, test your prompt in the AI tools you plan to use and verify that the outputs are incorrect. AI model behavior can change over time, so re-test before each semester.
Download the assignment document
Return to AI Gallery Home