Kavli Affiliate: Xian Chen
| First 5 Authors: Jie Cao, Chloe Qianhui Zhao, Xian Chen, Shuman Wang, Christian Schunn
| Summary:
Producing large volumes of high-quality, timely feedback poses significant
challenges to instructors. To address this issue, automation
technologies-particularly Large Language Models (LLMs)-show great potential.
However, current LLM-based research still shows room for improvement in terms
of feedback quality. Our study proposed a multi-agent approach performing
"generation, evaluation, and regeneration" (G-E-RG) to further enhance feedback
quality. In the first-generation phase, six methods were adopted, combining
three feedback theoretical frameworks and two prompt methods: zero-shot and
retrieval-augmented generation with chain-of-thought (RAG_CoT). The results
indicated that, compared to first-round feedback, G-E-RG significantly improved
final feedback across six methods for most dimensions. Specifically:(1)
Evaluation accuracy for six methods increased by 3.36% to 12.98% (p<0.001); (2)
The proportion of feedback containing four effective components rose from an
average of 27.72% to an average of 98.49% among six methods, sub-dimensions of
providing critiques, highlighting strengths, encouraging agency, and
cultivating dialogue also showed great enhancement (p<0.001); (3) There was a
significant improvement in most of the feature values (p<0.001), although some
sub-dimensions (e.g., strengthening the teacher-student relationship) still
require further enhancement; (4) The simplicity of feedback was effectively
enhanced (p<0.001) for three methods.
| Search Query: ArXiv Query: search_query=au:”Xian Chen”&id_list=&start=0&max_results=3