Evaluating machine translation post-editing training in undergraduate translation programs - an exploratory study in Saudi Arabia

Abstract

The modern translation industry is using machine translation post-editing (MTPE) widely, and the translation industry in the Arab World is following the global lead. However, while MTPE training is offered in many language pairs around the world, MTPE training in English-Arabic is still not officially offered in translation training programmes in the Arab World. The aim of this study is to evaluate the effectiveness of MTPE training in a female undergraduate translation programme in Saudi Arabia by examining students’ opinions about MTPE and comparing its productivity and quality with an established practice in the translation classroom, i.e., human translation (HT). To achieve its aim, this study used a mixed-method design of the ‘Kirkpatrick Model of Learning Evaluation’. Focus group discussions and retrospective pre-test surveys were used to examine students’ opinions as well as a pre-post experiment which involved two groups of students (29 in the control group and 31 in the experimental group) that was used to compare the productivity of students and the quality of translated texts when using MTPE as compared with HT. Students’ opinions that were revealed through the pre-intervention focus group discussions were generally mixed with a preference shown in favour of HT, except for translation speed as most of the students thought that MTPE was the faster method of translation. As for the survey, students’ pre-intervention responses supported those opinions revealed in the focus group discussions. However, post-intervention responses revealed a statistically significant shift towards more acceptance of MTPE training and use, indicating that the more students learned about the features of MT and MTPE skills and practiced them, the more positive their opinions became. Statistical results from comparing students’ productivity showed a medium effect size which indicates that MTPE cannot be ignored as a method to increase productivity in translation. The effectiveness of MTPE in translation quality was evaluated by measuring error count and error type. Error count analysis indicated that students who used MTPE have increased scores in a similar manner to those who used HT but not more. The analysis of error type showed that while MTPE helped students avoid deletion and technical errors, the number of errors relating to accuracy, comprehension and grammar were more frequent in Arabic MTPE translated texts.

More information

Identifiers

Catalogue record

Export record