Synthesis report on assessment and feedback with technology enhancement

Abstract

This report has been commissioned by The Higher Education Academy (The Academy) in January 2010. The report addresses the main questions an academic audience is likely to have about assessment and feedback with technology enhancement by reference to evidence-based, widely available and peer-reviewed materials and papers available on technological support and enhancement for assessment and feedback. Of the 142 references which were recommended, the report reviews the 124 which were accessible, of which 67.7% were peer-reviewed, 28.2% provided quantitative data, and 18.5% provided experimental designs or statistical analyses. Most references focused on the reaction of students and teachers to the use of technology for assessment and feedback. Although it may be ideal to have high-quality evidence before implementing a new assessment approach, in the absence of this level of support, the insights and advice of reputable authors are valued by other practitioners and their messages have impact. The report provides a detailed introduction to and summary of references useful to practitioners on technology-enhanced assessment applicable to Higher (and Further) Education in a UK context. Some of the messages that are supported by evidence in the recommended literature include: Assessment for learning shows an effect size of between 0.34 and 0.46; Tutors can use technology-enhanced methods to implement effective learning designs that would not otherwise be possible because of factors such as time constraints, student numbers and geographical or temporal distribution; Effective regular, online testing can encourage student learning and improve their performance in tests; Student retention and inclusion can be increased by using technology-enhanced methods. Exam anxiety can also be reduced; Using technology-based methods does not disadvantage women or older students; Automated marking can be more reliable than human markers and there is no medium effect between paper and computerized exams; The success of assessment and feedback with technology enhancement lies with the pedagogy rather than the technology itself; technology is an enabler; Technology-enhanced assessment is not restricted to simple questions and clear-cut right and wrong answers, much more sophisticated questions are being used as well; Modern technology can be matched to the learning characteristics of the contemporary learner; The design of appropriate and constructive feedback plays a vital role in the success of assessment, especially assessment for learning. The literature offers detailed guidance on designing effective feedback such as conditions, research-backed principles and a typology, as well as specific advice for the design of audio feedback and peer assessment; What the characteristics of useful technologies to use for assessment and feedback are; Taking a team approach to the creation of technology-enhanced assessment and feedback is valuable because successful implementation requires skills in the application of technology and how to use to the technology itself as well as learning and the subject content; Staff development and support are vital when introducing and developing assessment and feedback with technology enhancement; Testing the assessment and feedback to ensure it is reliable and valid and piloting it with people who are similar to or understand the targeted students are important stages in the development process. A good reporting system can help academics see and analyse the results (including student answers) and will help refine the assessment and feedback; It is important to prepare students to take the assessments that use technology enhancement by practising with similar levels of assessment using the same equipment and methods. This is similar to being able to practise on past papers; The reports generated by many technology-enhanced assessment systems are very helpful in checking the reliability and validity of each test item and the test as a whole. In conclusion, the references that were recommended to us are clearly having an impact on current practice and are found valuable. We would welcome more high-quality statistical studies that offer evidence to support the lessons that practitioners have learned from experience.

More information

Identifiers

Catalogue record

Export record