Assessment and Feedback

The QEI team has led various strategic projects at The Open University which have focused on understanding perspectives in the distance learning sector, adjustments to online assessment and advancements in technology.

Student Experience of Assessment, Feedback and Revision  

The aim of this programme was to understand the complex inter-relationships associated with the student experience of and demands for formative feedback, summative assessment and revision and identify opportunities for intervention and change. A dataset comprising responses to over 100 questions our Student Experience of Feedback, Assessment and Revision (SEFAR) survey has helped us answer teaching and retention questions such as:

• How do students of different ethnicities experience assessment?
• What are emerging opportunities for personalising assessment?
• How does exam ‘revision’ contribute to student learning?

We have developed the questionnaire since our initial survey in 2015 and used early data as baseline for comparison with insight collected during the Covid-19 pandemic.

Online Exams  

We have led the university in understanding the student experience of remote and online exams since Covid-19 and eliciting insight into future use, requirements, and expectations. Our pan-university Experience of Online Exams project designed and sent out questionnaires to students enrolled on 50 modules. Analysis is contributing to university policy and being shared internationally. Our research has received strong interest at leading international distance learning conferences.  

Automated Formative Feedback  

We led the development and evaluation of Open Essayist Optimiser; a tool designed to support essay-writing and reflective practice. This tool builds on prior EPSRC-funded research (Whitelock et al., 2018) and our recent focus has been on the pattern and degree of engagement and related this to prior assessment submissions (Edwards et al., 2022). Our expertise has also been called on for academic facilitation and analysis of focus group data to inform the development and implementation of next generation assessment and assignment management software.  

TeSLA - An Adaptive Trust-based e-assessment system for learning

TeSLA was an EU H2020-funded project, that developed an e-assessment authentication system to help ensure that learners in online assessments are who they say they are. It aimed to support any existing e-assessment model (diagnostic, formative, and summative), and address related pedagogical, ethical, legal and technological issues. The system developed is useful for fully online and blended learning and means that students avoid having to attend a face-to-face examination but are still able to trust their assessment results. TeSLA offered to educational institutions, accrediting agencies, and to society in general, an unambiguous proof of a learner’s academic progression and achievements.

Digital Badges for Assessment at Scale  

There is a pressing need to understand how digital badges can work at scale in the Global South and one such potential application is the support and promotion of Teacher Professional Development. Working with the Tata Institute of Social Sciences, the Digital Badges for Teacher Professional Development in India (DBTPDI) project, delivered short courses to teachers from over 220 schools in Assam, India. Our data enables us to understand how to conceptualise and deliver authentically assessed professional development at scale for teachers in the Global South (Cross et al., 2022). Project team members from IET and WELS were recently shortlisted for two categories in the OU’s Research Excellence Awards 2022 in recognition of their contribution to helping the university achieve global societal impact. We have hosted two international public events and our most recent publication has advanced understanding of the relationships between teacher professional values and the perceived value of digital badges for professional development (Cross, Wolfenden, et al., 2022).

Selected insights and findings

2022

• Edwards, Chris; Whitelock, Denise; Cross, Simon and Wild, Fridolin (2022). Developing automated ways to give essay writing feedback to students. In: INTED2022 Proceedings: 16th International Technology, Education and Development Conference, IATED Academy, Valencia, Spain, pp. 5446– 5450.
• Edwards, Chris; Whitelock, Denise; Cross, Simon; and Wild, Fridolin (2022). Towards autonomous, confident academic writing: supporting students in developing their writing through a stand-alone online tool. AHE2022.
• Edwards, Chris; Whitelock, Denise (2022). Was our pilot study successful? An approach when data are sparse? In: EDEN 2022 Proceedings (in press).
• Cross, S., Wolfenden, F., Charania, A., Adinolfi, L., Sen, S. & Sarkar, D. (2022). Supporting practice-based teacher professional learning and assessment at scale in the Global South. In: INTED2022 Proceedings: 16th International Technology, Education and Development Conference, IATED Academy, Valencia, Spain.
• Cross, S. & Brasher, A. (2022). How well does OU assessment practice take into account students' personal learning objectives and goals: insight for action from the SEFAR survey.
• Cross, S., Charania, A.,Wolfenden, F., Adinolfi, L., Sen, S. & Sarkar, D. (2022). Taking forward TPD@Scale in the Global South: what are we learning from open digital badge implementations in India?

2021

• Cross, S., Adinolfi, L., Charania, A., Wolfenden, F. et al. (2021). Teachers’ professional learning during and after Covid: A role for open digital badges. The Open University.
• Cross, S., and Brasher, A., (2021). Differences in assessment experience between Black, Asian and Minority Ethnic (BAME) students and White students: An initial review using IET’s SEFAR2 dataset. Available on the Scholarship Exchange.

2020

• Herodotou, C. and Gillespie, A. (2020). Explore: Marking and Feedback (Phase 1) (Final report). Open University.
• Wolfenden, F., Adinolfi, L. & & Cross, S. (2020). Exploring open digital badges in teacher education: a case study from India. Journal of Learning for Development - JL4D, 7(1) pp. 108–115.
• Whitelock, D., Edwards, C. and Okada, A. (2020). Can e-Authentication Raise the Confidence of Both Students and Teachers in Qualifications Granted through the e-Assessment Process? Journal of Learning for Development - JL4D, 7(1) pp. 46–60.

2018

• Whitelock, D., Twiner, A., Richardson, J. T. E., Field, D. and Pulman, S. (2018). What does a “good” essay look like? Rainbow diagrams representing essay quality . In: Technology Enhanced Assessment (TEA 2017). Communications in Computer and Information Science (Ras, E. and Guerrero Roldán, A. eds.), Springer, Cham, 829 pp. 1–12.

2016

• Cross, S., Whitelock, D., and Mittelmeier, J. (2016). Does the Quality and Quantity of Exam Revision Impact on Student Satisfaction and Performance in the Exam Itself?: Perspectives from Undergraduate Distance Learners. In: EDULEARN16. pp. 5052–5061.