Design It by Hand, Then Rebuild It with AI

TLDR: Designing something by hand first made students’ own instincts visible. Only against that baseline did AI’s limits –along with some  of its capabilities– become legible.

What she’s doing: In her Human-Machine Systems engineering course, Kris built a two-pass assignment around a “facilitator,” which is a one-page quick-reference guide students design to support a real human-asset task. In the first pass, students worked individually and without AI, applying the human-factors, user experience, and visual-design principles from the course to a topic of their own expertise. They could use a variety of non-AI tools including Canva, PowerPoint, MS Draw, and other electronic tools to create the original. One student used her unique insight to design a guide to the parts of a sewing machine and another, how to settle in and watch a film well, for the most fulfilling viewing experience. The focus for this part was students practicing design thinking. 

In the second pass, students rebuilt that same facilitator using a free text-to-image AI tool, without uploading the original. They iterated through a series of AI prompts until they reached a stopping point, recording every prompt along the way, and then analyzed the gap themselves (no AI permitted for the analysis). Limiting students to free tools served as a leveling device, so no result hinged on a paid subscription. The activity was offered as optional contributing points on each exam rather than a required deliverable.

What’s working and what isn’t: Across 26 submissions, students converged on a finding: AI is a capable drafter, but not a principal designer. The tools approximated a facilitator’s content and structure quickly and fairly accurately but fell short on design quality and usability: the mapping to human-factors principles, the intentional use of visual hierarchy and color, the flow. Tellingly, students had applied those principles instinctively in their non-AI version, then were surprised by what AI did and didn’t reproduce, because they had not thought to instruct it explicitly. The exercise made their own tacit expertise apparent to them.

A counterintuitive pattern also surfaced in the data: more iterations tracked with lower satisfaction with the final version, not higher. Students described hitting AI’s “stubbornness”; a point of diminishing returns from prompts. Prompt specificity at the beginning generated higher satisfaction, with students who engineered precise prompts faring better than those who nudged broadly. Nearly everyone indicated that the tool served as a time-saver that was “good, but not great.”

One early friction became a teaching opportunity. A student used AI on the no-AI midterm pass, assuming Kris had simply overlooked how easy it was to accomplish the task using AI – a “natural choice”. Shown the instructions again, the student then agreed to collaborate with Kris on refining the final assignment’s instructions and examples — turning a misstep into collaboration.

What’s next: The question she keeps returning to is less about the artifact than the judgment behind it: what does it mean to know when a design is good enough? The next time Kris runs the program, she will have students record why and at what point they stopped iterating, and how they defined “satisfied.” She is also weighing whether to keep the activity as optional or make it required, noting the trade-off: the voluntary, points-building format produced richer engagement, while requiring it might lead students to stop sooner. The work around this activity is headed toward a forthcoming paper.

Have you tried something new with AI? How did it go? Send an email to [email protected] to let us know!