The University of Southampton
University of Southampton Institutional Repository

Evaluation and automatic analysis of MOOC forum comments

Evaluation and automatic analysis of MOOC forum comments
Evaluation and automatic analysis of MOOC forum comments
Moderators of Massive Open Online Courses (MOOCs) undertake a dual role. Their work entails not just facilitating an effective learning environment, but also identifying excelling and struggling learners, and providing pedagogical encouragement and direction. Supporting learners is a critical part of moderators’ work, and identifying learners’ level of critical thinking is an important part of this process. As many thousands of learners may communicate 24 hours a day, 7 days a week using MOOC comment forums, providing support in this environment is a significant challenge for the small numbers of moderators typically engaged in this work. In order to address this challenge, I adopt established coding schemes used for pedagogical content analysis of online discussions to classifying comments, and report on several studies I have undertaken which seek to ascertain the reliability of these approaches, establishing associations with these methods and linguistic and other indicators of critical thinking. I develop a simple algorithmic method of classification based on automatically sorting comments according to their linguistic composition, and evaluate an interview-based case study, where this algorithm is applied to an on-going MOOC. The algorithm method achieved good reliability when applied to a prepared test data set, and when applied to unlabelled comments in a live MOOC and evaluated by MOOC moderators, it was considered to have provided useful, actionable feedback. This thesis provides contributions that help to understand the usefulness of automatic analysis of levels of critical thinking in MOOC comment forums, and as such has implications for future learning analytics research, and e-learning policy making.
University of Southampton
O'Riordan, Tim
d6ba191a-e432-41f8-b3da-176d28355579
O'Riordan, Tim
d6ba191a-e432-41f8-b3da-176d28355579
Millard, David
4f19bca5-80dc-4533-a101-89a5a0e3b372
Schultz, J.
7faf041d-9163-4e34-9dd7-29ce4398b8b0

O'Riordan, Tim (2018) Evaluation and automatic analysis of MOOC forum comments. University of Southampton, Doctoral Thesis, 177pp.

Record type: Thesis (Doctoral)

Abstract

Moderators of Massive Open Online Courses (MOOCs) undertake a dual role. Their work entails not just facilitating an effective learning environment, but also identifying excelling and struggling learners, and providing pedagogical encouragement and direction. Supporting learners is a critical part of moderators’ work, and identifying learners’ level of critical thinking is an important part of this process. As many thousands of learners may communicate 24 hours a day, 7 days a week using MOOC comment forums, providing support in this environment is a significant challenge for the small numbers of moderators typically engaged in this work. In order to address this challenge, I adopt established coding schemes used for pedagogical content analysis of online discussions to classifying comments, and report on several studies I have undertaken which seek to ascertain the reliability of these approaches, establishing associations with these methods and linguistic and other indicators of critical thinking. I develop a simple algorithmic method of classification based on automatically sorting comments according to their linguistic composition, and evaluate an interview-based case study, where this algorithm is applied to an on-going MOOC. The algorithm method achieved good reliability when applied to a prepared test data set, and when applied to unlabelled comments in a live MOOC and evaluated by MOOC moderators, it was considered to have provided useful, actionable feedback. This thesis provides contributions that help to understand the usefulness of automatic analysis of levels of critical thinking in MOOC comment forums, and as such has implications for future learning analytics research, and e-learning policy making.

Text
Final thesis - Version of Record
Available under License University of Southampton Thesis Licence.
Download (6MB)

More information

Published date: February 2018

Identifiers

Local EPrints ID: 424796
URI: https://eprints.soton.ac.uk/id/eprint/424796
PURE UUID: eb999a5a-f172-4686-bf82-ef74cfa6e1ed
ORCID for Tim O'Riordan: ORCID iD orcid.org/0000-0003-4905-7430
ORCID for David Millard: ORCID iD orcid.org/0000-0002-7512-2710

Catalogue record

Date deposited: 05 Oct 2018 11:46
Last modified: 21 Jun 2019 04:01

Export record

Contributors

Author: Tim O'Riordan ORCID iD
Thesis advisor: David Millard ORCID iD
Thesis advisor: J. Schultz

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of https://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×