Quality Enhancement and Innovation · Programme
Leading research, development, and dissemination of new data approaches and visualisations, as an internationally recognised expert in learning analytics.
IET uses its expertise to empower the successful use of data and learning analytics to improve the quality of teaching and learning and student outcomes. This is achieved through the development of new methodologies to enable leadership in evaluating and embedding predictive analytics in teaching practice, creating and establishing new ways to help disciplines at the OU understand and map their qualifications. It is also recognised for the role of supporting the OU to develop new innovative ways to evaluate teaching and learning interventions, as well as explore the potential of text analytics for summarising high volumes of student comments and providing data-rich student engagement to educators.
The HEFCE/Office for Students Learning Gains project explored an Affective-Behaviour-Cognition (ABC) model of learning, to broaden the concept of learning gain, and - more importantly - to develop, test, implement, and evaluate a range of measurements for learning gains at each of the ABC level. The focus of the project was on cognitive learning gains. Five different approaches to measure cognitive learning gains (i.e., within module, from one module to the next one, within a qualification, across qualifications, across institutions) were used. Project findings indicated that although assessment scores could potentially be used as proxies for cognitive learning gains, a large proportion of these gains were not related to individual student ability, effort, and/or cognition. How teachers designed courses, marked and graded assessments substantially influenced the grade trajectories of students, which were disciplinary, and institutionally influenced and determined.
The Open University offers a comprehensive curriculum for students, with flexibility in choice of modules, the order of study, and the length of time to complete a qualification. This flexibility frequently results in a large number of potential study paths to a qualification, each resulting in a different student experience. The Pathways project aimed to find a way to analyse this complexity and thereby develop approaches to provide meaningful insight to practitioners and researchers. This research took a novel approach by employing a multi-model database, to analyse, map and visualise pathways taken by students (Edwards & Gaved, 2020a, 2020b). The overall aim is to improve our understanding of the student study experience, in order that we then improve it through developing our provision, the advice, and the guidance that we provide. The approach is being used to support several projects, funded by OU Scholarship Centres, with detailed qualification insight (Edwards 2021; Edwards 2022). The Pathways Project has also informed the EU funded project Erasmus+ Complex Trajectories project.
A key challenge of teaching at a distance is knowing when a learner may require additional support and assistance. Since 2017, we have worked closely with Associate Lecturers (ALs) from across faculties to study, evidence and enable adoption and use of the Early Alert Indicators dashboard presenting predictions as to who of the OU students are at risk of failing in their next assignment (Herodotou et al., 2020a). This work has resulted in module-focused studies and an emphasis on using predictive analytics to support students from Black, Asian and Ethnic minority students, in particular. In addition, we used randomised control trials to test specific teacher-supported interventions that could enable students at risk to succeed. During the 2022-23, we conducted a pan-university survey about the use of predictive learning analytics (n>300) with insight showing that interventions are needed to raise awareness of the dashboard across more ALs, organised professional development and briefing sessions involving 100 staff, submitted contributions to the OU Retention initiative, and supported successful intervention activities in the OU's Faculty of Business and Law.
Students’ responses to the open questions of end-of-module surveys provide invaluable insight into the students’ learning experience, however time and coordination propose challenges, to systematic analysis at an institutional level. Our work with Text Analytics of Student Comments (TASC) aimed to use automated text analytics methods to summarise and review the written feedback of our students. The text analytics techniques developed have provided insight for informing module changes (Brasher & Ullmann, 2020; Ullmann & Brasher, 2020), support for institutional analytics programmes (Ullmann et al., 2021) and compare and contrast the comments made by students from ethnic and minority backgrounds (Ullmann, 2021a). Funding from the university's Test and Learn project has supported IET in contributing insight and ideas to the development of a staff dashboard and sharing case studies about topic detection, sentiment analysis, qualification differences, and insights into high and low performing students (Ullmann & Rienties, 2021). Building on this work, IET is presently leading a new Test and Learn funded project to explore how Artificial Intelligence can support students to write reflectively. This should be written in the past as the project has now finished.
IET is at the forefront of exploring how new methodologies can support the evaluation of teaching and learning. For example, the QEI team have been interested in how non-randomised studies can improve the quality of teaching and learning evaluation. Statistical matching techniques, such as propensity score matching and others, can be used to create a fairer comparison between groups of students. IET has led the analysis component of a project investigating the impact of student performance of a new voluntary peer assessment activity on student performance (Ullmann, 2020), the benefits of phone call interventions (Kubiak and Ullmann, 2020), the effects of one-to-one tutoring for skills development (Ullmann, 2021b) and the implementation of a virtual microscope. Also, working with colleagues in the OU’s Faculty of Wellbeing, Education and Language Studies (WELS), the team have explored the potential of 360-degree spherical video. Outputs from this work include a framework for the use of spherical video in educational research (Cross, Wolfenden & Adinolfi, 2022a), framing the affordances and use of spherical 360-degree video in teacher professional development (TPD); Cross, Wolfenden & Adinolfi, 2022b) and providing recommendations as to how to improve the Virtual Microscope design to better support learners when engaging with it. This work highlights the importance of four key dimensions: open observation, critical distance, spatial presence, and investigative space.
Editorial
OU steps-up efforts to improve student experience with innovative dashboard
An interactive dashboard at The Open University (OU) is using machine learning methods to provide tutors (Associate Lecturers) with greater visibility of student engagement and experience.
Editorial
Bart Rienties announced as President of SoLAR
Congratulations to IET's Professor Bart Rienties who in March 2023 was announced as the new President of the Society for Learning Analytics Research.
