Transformers based neural network approach for short question answer grading

Author

Muhammad Hamza

Degree

Master of Science in Computer Science

Department

Department of Computer Science

Faculty / School

Faculty of Computer Sciences (FCS)

Date of Submission

2019-12-30

Supervisor

Dr. Sajjad Haider, Professor, Department of Computer Science

Document type

MSCS Survey Report

Keywords

Short Answer Grading, Dataset, Semantic Space, Semantic Similarity, Term Weighting, Corpus, Transformers , BERT , Bi-Directional LSTM

Abstract

Transformer based models have achieved state-of-the-art results in various fields of NLP by using pre-trained embeddings and then fine-tuning it with task-specific data. An automated short answer grading system is a task with great potential to add substantial value in the real world. Manually grading assignments and exams is a tedious and exhausting task. Automating the process of grading is a major step forward in building intelligent learning systems. It can help teachers effectively use their time to improve the quality of content, resulting in better education. This survey discusses recent works, including techniques and approaches to building an automatic grading system. Moreover, a Transformer based approach has also been suggested.

Since short answer grading primarily focuses on the accuracy of response rather than grammar and conventions, using architectures that give weighted importance to part of responses and cater to long-range dependencies along with pretrained word embeddings (vector representations of words) can help us improve the state-of-the-art.

In this work, we have used ASAP-SAS (Automated Student Assessment Prize Short Answer Scoring) dataset as it has questions and answers from various domains.

Recommended Citation

Hamza, M. (2019). Transformers based neural network approach for short question answer grading (Unpublished MSCS survey report). Institute of Business Administration, Pakistan. Retrieved from https://ir.iba.edu.pk/survey-reports-mscs/15