🤖 Would you trust AI to give learners feedback?

"ChatGPT" by Focal Foto is licensed under CC BY-NC 2.0

The challenge of feedback

We know that for learning to be effective, learners need frequent, targeted feedback on their output.

The problem? This can be burdensome and time-consuming for a facilitator.

Can AI help?

In a conversation about advanced use cases of AI for instructional design, ChatGPT suggested to me that it could help “generate targeted feedback and learning activities to help them improve in areas where they need it most.” I wanted to test this theory.

ChatGPT suggests giving learners feedback as one of its advanced use cases for instructional designers

I asked ChatGPT to assess a work sample from my time as an educator: a poetry analysis (the most human assignment I could think of!) of William Blake’s poem “London,” using a four-point rubric that I had written.

How it went:

When given a detailed rubric, ChatGPT assessed the work sample as Proficient, citing depth of analysis and mechanical errors as areas for improvement. I agreed with its assessment!

ChatGPT accurately assesses the learner’s submission using the provided rubric

But even when pressed with follow-up prompts, ChatGPT was unable to meaningfully offer paths forward for the learner based on that feedback. For example, it suggested that the learner could improve their analysis of poetic devices by “us[ing] quotes from the poem to support your analysis of how the poetic device develops the theme.” The learner had already done this.

ChatGPT’s first attempt to give the learner feedback: the first point was already addressed in the response; the second point is incorrect

(By contrast, as an instructor, I’d offer my learners further questions about the poetic devices they had identified, like, “Based on the rest of the stanza, what kinds of things does Blake consider to make up the ‘mind-forg’d manacles,’ and why? What are the manacles chaining the people of London to?”)

My takeaways (using a medical metaphor):

• 🩺 At this moment, ChatGPT may be useful in providing learners with a “diagnosis” of their current performance and a general sense of the areas they need to improve.

• 📈 But without getting another set of human eyes on the chatbot’s feedback, I still wouldn’t feel comfortable letting it “update a patient chart,” or using its assessments as formal scores/grades -- we know it can make mistakes. So for scored submissions, I’m not sure how time-saving this tool would really be.

• 👩🏽‍⚕️ ChatGPT WASN’T able to use that diagnosis to generate a meaningful “treatment plan,” or specific feedback for improvement that learners could take and use. Facilitators and educators still have a valuable role to play in coaching learners up.

Further learning

To explore some other use cases of ChatGPT in L&D, check out Dr. Luke Hobson’s video “15 Ways to Use ChatGPT as an Instructional Designer, Instructor, and Teacher,” in which he explores strategies like using ChatGPT to write learning objectives, generate course outlines, write scripts, and more.