The way teachers design courses directly impact student behaviour and success
Ground-breaking research at the Open University/Institute of Educational Technology has found that the way teachers design blended and online courses significantly influences students’ engagement, satisfaction and success. For example, investigations within this research programme has identified four common patterns of learning activities of how Open University (OU) teachers have developed distance learning modules. Perhaps surprisingly given the strong standardisation processes at the OU, we found four relatively distinct clusters of learning design: constructivist, assessment-driven, balanced-variety, and social constructivist.
Learning Design is described as a methodology for enabling teachers/designers to make more informed decisions in how they go about designing learning activities and interventions, which is pedagogically informed and makes effective use of appropriate resources and technologies. In other words, learning design is focused on ‘what students do’ as part of their learning, rather than on ‘what teachers do’ or on what will be taught. Within the OU, there is an increased recognition that learning design is an essential driver for learning.
Recent technological developments have allowed learning analytics researchers to capture the digital traces of learning activities of students and teachers in Virtual Learning Environments (VLEs). This rich and fine-grained data about actual learner behaviours offer educators potentially valuable insights into how students react to different learning designs. However, despite substantial progress in transferring learning design from implicit to explicit, there remains a paucity of evidence for how learners respond to different learning design.
For example, recent empirical research in this programme on learning designs of 74 modules over 30 weeks revealed that the workload on a weekly basis on other activities decreased when assessment activities were introduced. This implied that educators at the OU aimed to balance the total workload when designing computer-based assessments (CBA). Secondly, learning designs could explain 69% of weekly behaviour by students in the VLE, which gives teachers a clear mandate to think about the most effective strategies and designs to support students.
Based upon ten years of learning design research, and a recent surge in large-scale empirical research on learning analytics, in this research programme we identify four research questions that we aim to address in the next two to four years.
- How useful are the learning analytics and learning design data for teachers and students, and how they could be further improved?
- What is the optimum balance of learning design and learning analytics to improve retention rates?
- What is the students’ voice in learning analytics and learning design?
- How can we effectively support teachers and organisations to use learning analytics and learning design?
The Learning Analytics and Learning Design team
- Rogaten, Jekaterina; Rienties, Bart; Sharpe, Rhona; Cross, Simon; Whitelock, Denise; Lygo-Baker, Simon and Littlejohn, Allison. (). Reviewing affective, behavioural, and cognitive learning gains in higher education. Assessment & Evaluation in Higher Education, 44 (3) pp. 321-337.
- Ullmann, Thomas Daniel. (). Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches. International Journal of Artificial Intelligence in Education, 29 (2) pp. 217-257.
- Rienties, Bart; Lewis, Timothy; Mcfarlane, Ruth; Nguyen, Quan and Toetenel, Lisette. (). Analytics in online and offline language learning environments: the role of learning design to understand student online engagement. Journal of Computer Assisted Learning, 31 (3) pp. 273-293.
- Mittelmeier, Jenna; Rienties, Bart; Tempelaar, Dirk; Hillaire, Garron and Whitelock, Denise. (). The influence of internationalised versus local content on online intercultural collaboration in groups: A randomised control trial study in a statistics course. Computers & Education, 118 pp. 82-95.
- Nguyen, Quan; Huptych, Michal and Rienties, Bart. (). Linking students' timing of engagement to learning design and academic performance. In: Proceedings of the 8th International Conference on Learning Analytics and Knowledge, ACM, New York, 141-150.
- Rogaten, Jekaterina and Rienties, Bart Carlo. (). Which first-year students are making most learning gains in STEM subjects?. Higher Education Pedagogies, 3 (1) pp. 161-172.
- Rienties, Bart; Herodotou, Christothea; Olney, Tom; Schencks, Mat and Boroowa, Avi. (). Making sense of learning analytics dashboards: a technology acceptance perspective of 95 teachers. International Review of Research in Open and Distance Learning, 19 (5) pp. 186-202.
- Mittelmeier, Jenna; Edwards, Rebecca L.; Davis, Sarah K.; Nguyen, Quan; Murphy, Victoria L.; Brummer, Leonie and Rienties, Bart. (). ‘A double-edged sword. This is powerful but it could be used destructively’: Perspectives of early career education researchers on learning analytics. Frontline Learning Research, 6 (2) pp. 20-38.
- Coughlan, Tim; Ullmann, Thomas Daniel and Lister, Kate. (). Understanding Accessibility as a Process through the Analysis of Feedback from Disabled Students. In: W4A '17 Proceedings of the 14th Web for All Conference on The Future of Accessible Work, ACM, New York, USA.
- Ferguson, Rebecca and Clow, Doug. (). Where is the evidence? A call to action for learning analytics. In: LAK '17 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, ACM, New York, USA, 56-65.
- Herodotou, Christothea; Heiser, Sarah and Rienties, Bart. (). Implementing Randomised Control Trials in Open and Distance Learning: A Feasibility Study. Open Learning: The Journal of Open, Distance and e-learning, 32 (2) pp. 147-162.
- Knight, Simon; Rienties, Bart; Littleton, Karen; Mitsu, Matthew; Tempelaar, Dirk and Shah, Chirag. (). The relationship of (perceived) epistemic cognition to interaction with resources on the internet. Computers in Human Behavior, 73 pp. 507-518.
- Li, Nai; Marsh, Vicky; Rienties, Bart and Whitelock, Denise. (). Online learning experiences of new versus continuing learners: a large scale replication study. Assessment & Evaluation in Higher Education, 42 (4) pp. 657-672.
- Nguyen, Quan; Rienties, Bart and Toetenel, Lisette. (). Unravelling the dynamics of instructional practice: a longitudinal study on learning design and VLE activities. In: Proceedings of the Seventh International Learning Analytics & Knowledge Conference, ACM, New York, NY, USA, 168-177.
- Nguyen, Quan; Rienties, Bart; Toetenel, Lisette; Ferguson, Rebecca and Whitelock, Denise. (). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76 pp. 703-714.
- Tempelaar, Dirk T.; Rienties, Bart and Nguyen, Quan. (). Towards actionable learning analytics using dispositions. IEEE Transactions on Learning Technologies, 10 (1) pp. 6-16.
- Ullmann, Thomas Daniel. (). Reflective Writing Analytics - Empirically Determined Keywords of Written Reflection. In: LAK '17 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, ACM, New York, USA, 163-167.
- Rienties, Bart; Nguyen, Quan; Holmes, Wayne and Reedy, Katharine. (). A review of ten years of implementation and research in aligning learning design with learning analytics at the Open University UK. Interaction Design and Architecture(s), 33 pp. 134-154.
- Ferguson, Rebecca. (). Learning at Scale: Using an Evidence Hub To Make Sense of What We Know. In: Learning at Scale 2016, 25-26 Apr 2016, Edinburgh, Scotland.
- Ferguson, Rebecca; Brasher, Andrew; Clow, Doug; Cooper, Adam; Hillaire, Garron; Mittelmeier, Jenna; Rienties, Bart; Ullmann, Thomas and Vuorikari, Riina (). Research Evidence on the Use of Learning Analytics: Implications for Education Policy. Joint Research Centre, Seville, Spain.
- Ferguson, Rebecca; Brasher, Andrew; Clow, Doug; Griffiths, Dai and Drachsler, Hendrik. (). Learning Analytics: Visions of the Future. In: 6th International Learning Analytics and Knowledge (LAK) Conference, 25-29 Apr 2016, Edinburgh, Scotland.
- Ferguson, Rebecca and Clow, Doug. (). Learning Analytics Community Exchange: Evidence Hub. In: 6th International Learning Analytics and Knowledge (LAK) Conference, 25-29 Apr 2016, Edinburgh, Scotland.
- Nguyen, Quan; Tempelaar, Dirk; Rienties, Bart and Giesbers, Bas. (). What learning analytics based prediction models tell us about feedback preferences of students. Quarterly Review of Distance Education, 17 (3) pp. 13-33.
- Toetenel, Lisette and Rienties, Bart. (). Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision-making. British Journal of Educational Technology, 47 (5) pp. 981-992.