Master of Science in Computer Science

Faculty / School

Faculty of Computer Sciences (FCS)


Department of Computer Science

Date of Submission



Dr. Sajjad Haider, Professor, Department of Computer Science, Institute of Business Administration (IBA), Karachi

Project Type

MSCS Survey Report


Transformer based models have achieved state-of-the-art results in various fields of NLP by using pre-trained embeddings and then fine-tuning it with task-specific data. An automated short answer grading system is a task with great potential to add substantial value in the real world. Manually grading assignments and exams is a tedious and exhausting task. Automating the process of grading is a major step forward in building intelligent learning systems. It can help teachers effectively use their time to improve the quality of content, resulting in better education. This survey discusses recent works, including techniques and approaches to building an automatic grading system. Moreover, a Transformer based approach has also been suggested.

Since short answer grading primarily focuses on the accuracy of response rather than grammar and conventions, using architectures that give weighted importance to part of responses and cater to long-range dependencies along with pretrained word embeddings (vector representations of words) can help us improve the state-of-the-art.

In this work, we have used ASAP-SAS (Automated Student Assessment Prize Short Answer Scoring) dataset as it has questions and answers from various domains.


The development of automated scoring systems revolves around lexical features, finding similarities between RA and SA, and scoring the answers. It results in not taking the perspective of stakeholders of AGS into account. A study suggested that stakeholders including score-users, teachers, and business units should be considered in addition to the technicality of AGS and focusing on the accuracy of the system.

The full text of this document is only accessible to authorized users.