Master of Science in Computer Science
Faculty / School
Faculty of Computer Sciences (FCS)
Department of Computer Science
Date of Submission
Dr. Sajjad Haider, Professor, Department of Computer Science, Institute of Business Administration (IBA), Karachi
MSCS Survey Report
Transformer based models have achieved state-of-the-art results in various fields of NLP by using pre-trained embeddings and then fine-tuning it with task-specific data. An automated short answer grading system is a task with great potential to add substantial value in the real world. Manually grading assignments and exams is a tedious and exhausting task. Automating the process of grading is a major step forward in building intelligent learning systems. It can help teachers effectively use their time to improve the quality of content, resulting in better education. This survey discusses recent works, including techniques and approaches to building an automatic grading system. Moreover, a Transformer based approach has also been suggested.
Since short answer grading primarily focuses on the accuracy of response rather than grammar and conventions, using architectures that give weighted importance to part of responses and cater to long-range dependencies along with pretrained word embeddings (vector representations of words) can help us improve the state-of-the-art.
In this work, we have used ASAP-SAS (Automated Student Assessment Prize Short Answer Scoring) dataset as it has questions and answers from various domains.
The development of automated scoring systems revolves around lexical features, finding similarities between RA and SA, and scoring the answers. It results in not taking the perspective of stakeholders of AGS into account. A study suggested that stakeholders including score-users, teachers, and business units should be considered in addition to the technicality of AGS and focusing on the accuracy of the system.
Hamza, Muhammad. "Transformers based neural network approach for short question answer grading." Unpublished graduate research project. Institute of Business Administration. 2019. https://ir.iba.edu.pk/research-projects-mscs/17
The full text of this document is only accessible to authorized users.