The University of Southampton
University of Southampton Institutional Repository

Synthesis report on assessment and feedback with technology enhancement

Synthesis report on assessment and feedback with technology enhancement
Synthesis report on assessment and feedback with technology enhancement
This report has been commissioned by The Higher Education Academy (The Academy) in January 2010. The report addresses the main questions an academic audience is likely to have about assessment and feedback with technology enhancement by reference to evidence-based, widely available and peer-reviewed materials and papers available on technological support and enhancement for assessment and feedback. Of the 142 references which were recommended, the report reviews the 124 which were accessible, of which 67.7% were peer-reviewed, 28.2% provided quantitative data, and 18.5% provided experimental designs or statistical analyses. Most references focused on the reaction of students and teachers to the use of technology for assessment and feedback. Although it may be ideal to have high-quality evidence before implementing a new assessment approach, in the absence of this level of support, the insights and advice of reputable authors are valued by other practitioners and their messages have impact. The report provides a detailed introduction to and summary of references useful to practitioners on technology-enhanced assessment applicable to Higher (and Further) Education in a UK context. Some of the messages that are supported by evidence in the recommended literature include: Assessment for learning shows an effect size of between 0.34 and 0.46; Tutors can use technology-enhanced methods to implement effective learning designs that would not otherwise be possible because of factors such as time constraints, student numbers and geographical or temporal distribution; Effective regular, online testing can encourage student learning and improve their performance in tests; Student retention and inclusion can be increased by using technology-enhanced methods. Exam anxiety can also be reduced; Using technology-based methods does not disadvantage women or older students; Automated marking can be more reliable than human markers and there is no medium effect between paper and computerized exams; The success of assessment and feedback with technology enhancement lies with the pedagogy rather than the technology itself; technology is an enabler; Technology-enhanced assessment is not restricted to simple questions and clear-cut right and wrong answers, much more sophisticated questions are being used as well; Modern technology can be matched to the learning characteristics of the contemporary learner; The design of appropriate and constructive feedback plays a vital role in the success of assessment, especially assessment for learning. The literature offers detailed guidance on designing effective feedback such as conditions, research-backed principles and a typology, as well as specific advice for the design of audio feedback and peer assessment; What the characteristics of useful technologies to use for assessment and feedback are; Taking a team approach to the creation of technology-enhanced assessment and feedback is valuable because successful implementation requires skills in the application of technology and how to use to the technology itself as well as learning and the subject content; Staff development and support are vital when introducing and developing assessment and feedback with technology enhancement; Testing the assessment and feedback to ensure it is reliable and valid and piloting it with people who are similar to or understand the targeted students are important stages in the development process. A good reporting system can help academics see and analyse the results (including student answers) and will help refine the assessment and feedback; It is important to prepare students to take the assessments that use technology enhancement by practising with similar levels of assessment using the same equipment and methods. This is similar to being able to practise on past papers; The reports generated by many technology-enhanced assessment systems are very helpful in checking the reliability and validity of each test item and the test as a whole. In conclusion, the references that were recommended to us are clearly having an impact on current practice and are found valuable. We would welcome more high-quality statistical studies that offer evidence to support the lessons that practitioners have learned from experience.
e-assessment, e-learning, feedback, technology-enhanced learning, technology-supported learning, technology-assisted learning, computer-aided assessment, computer-assisted assessment
Gilbert, Lester
a593729a-9941-4b0a-bb10-1be61673b741
Whitelock, Denise
f132dc6e-b8d7-4f7c-aab2-2086feb7fe47
Gale, Veronica
cdb5afba-3379-4821-b9d1-4afab91ac36f
Gilbert, Lester
a593729a-9941-4b0a-bb10-1be61673b741
Whitelock, Denise
f132dc6e-b8d7-4f7c-aab2-2086feb7fe47
Gale, Veronica
cdb5afba-3379-4821-b9d1-4afab91ac36f

Gilbert, Lester, Whitelock, Denise and Gale, Veronica (2011) Synthesis report on assessment and feedback with technology enhancement

Record type: Monograph (Project Report)

Abstract

This report has been commissioned by The Higher Education Academy (The Academy) in January 2010. The report addresses the main questions an academic audience is likely to have about assessment and feedback with technology enhancement by reference to evidence-based, widely available and peer-reviewed materials and papers available on technological support and enhancement for assessment and feedback. Of the 142 references which were recommended, the report reviews the 124 which were accessible, of which 67.7% were peer-reviewed, 28.2% provided quantitative data, and 18.5% provided experimental designs or statistical analyses. Most references focused on the reaction of students and teachers to the use of technology for assessment and feedback. Although it may be ideal to have high-quality evidence before implementing a new assessment approach, in the absence of this level of support, the insights and advice of reputable authors are valued by other practitioners and their messages have impact. The report provides a detailed introduction to and summary of references useful to practitioners on technology-enhanced assessment applicable to Higher (and Further) Education in a UK context. Some of the messages that are supported by evidence in the recommended literature include: Assessment for learning shows an effect size of between 0.34 and 0.46; Tutors can use technology-enhanced methods to implement effective learning designs that would not otherwise be possible because of factors such as time constraints, student numbers and geographical or temporal distribution; Effective regular, online testing can encourage student learning and improve their performance in tests; Student retention and inclusion can be increased by using technology-enhanced methods. Exam anxiety can also be reduced; Using technology-based methods does not disadvantage women or older students; Automated marking can be more reliable than human markers and there is no medium effect between paper and computerized exams; The success of assessment and feedback with technology enhancement lies with the pedagogy rather than the technology itself; technology is an enabler; Technology-enhanced assessment is not restricted to simple questions and clear-cut right and wrong answers, much more sophisticated questions are being used as well; Modern technology can be matched to the learning characteristics of the contemporary learner; The design of appropriate and constructive feedback plays a vital role in the success of assessment, especially assessment for learning. The literature offers detailed guidance on designing effective feedback such as conditions, research-backed principles and a typology, as well as specific advice for the design of audio feedback and peer assessment; What the characteristics of useful technologies to use for assessment and feedback are; Taking a team approach to the creation of technology-enhanced assessment and feedback is valuable because successful implementation requires skills in the application of technology and how to use to the technology itself as well as learning and the subject content; Staff development and support are vital when introducing and developing assessment and feedback with technology enhancement; Testing the assessment and feedback to ensure it is reliable and valid and piloting it with people who are similar to or understand the targeted students are important stages in the development process. A good reporting system can help academics see and analyse the results (including student answers) and will help refine the assessment and feedback; It is important to prepare students to take the assessments that use technology enhancement by practising with similar levels of assessment using the same equipment and methods. This is similar to being able to practise on past papers; The reports generated by many technology-enhanced assessment systems are very helpful in checking the reliability and validity of each test item and the test as a whole. In conclusion, the references that were recommended to us are clearly having an impact on current practice and are found valuable. We would welcome more high-quality statistical studies that offer evidence to support the lessons that practitioners have learned from experience.

Text
Synthesis report on assessment and feedback Final Report July 2011.pdf - Version of Record
Available under License Other.
Download (1MB)

More information

Published date: 13 July 2011
Keywords: e-assessment, e-learning, feedback, technology-enhanced learning, technology-supported learning, technology-assisted learning, computer-aided assessment, computer-assisted assessment
Organisations: Electronic & Software Systems

Identifiers

Local EPrints ID: 273221
URI: http://eprints.soton.ac.uk/id/eprint/273221
PURE UUID: 0dcb28c0-cb35-4d5e-9e62-bb5f1f37c35f

Catalogue record

Date deposited: 19 Feb 2012 16:23
Last modified: 14 Mar 2024 10:22

Export record

Contributors

Author: Lester Gilbert
Author: Denise Whitelock
Author: Veronica Gale

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×