Photo of Dr Thomas Ullmann

People

Dr Thomas Ullmann SFHEA

Lecturer

About

I am a Lecturer in the Institute of Educational Technology at The Open University and a Senior Fellow of the Higher Education Academy (SFHEA). I was an Honorary Lecturer in the Department of Computer Science at the University of Warwick.

As Lecturer, I am responsible for teaching and for supporting the teaching and learning of the Open University. A major part of my role is to support the Quality Assurance and Quality Enhancement process at the Open University. In this role I investigate the latest online and distance teaching practices as well as contribute to the development of novel methods and techniques. I taught on the Master in Online and Distance Education (MAODE) at the OU. I was the deputy module chair of H817 'Openness and innovation in eLearning' and module team member of H817 'The network practitioner. For H817 I wrote the course materials and activities about reflective thinking and reflective writing for EdTech practitioners.

I am now working over a decade at the Open University. I joined the Institute of Educational Technology in 2015, but I have been with the Open University since 2008 working for the Knowledge Media Institute.

My research focus is on the future of education, especially on the space of possibilities created by new technologies for education. Currently, I am investigating the space of text analytics for improving teaching and learning. In my research I am exploring techniques to make outcomes of thinking processes visible in order to better understand teaching and learning. For example, in my work on reflection detection, I focus on techniques to automatically analyse texts regarding reflective thinking. I built ReflectR, a simple demonstrator of reflection detection in action, which you can try out here: http://qone.eu/reflectr. In my work as lead of the Text Analytics of Student Comments initiative, I investigate text analytics techniques to make visible the student experience based on large data sets.

I published over 30 publications in high-ranked journals, conferences, and workshops and over 45 research deliverables, internal research reports, quality enhancement reports, and scholarship work. Most of my publications are freely available as pre-print. You can find them under 'Publications'. I was a Co-Investigator of the LAEP project - 'The implications and opportunities of learning analytics for European educational policy' (Final report: http://dx.doi.org/10.2791/955210 - open access) and I worked for the following large-scale European Research projects STELLAR, Catalyst (both FP7), and X-Media (FP6). For several years, I co-organised the Workshop on Awareness and Reflection in Technology-Enhanced Learning at the European Conference on Technology Enhanced Learning (https://eduinf.eu/category/cfp/) and co-published several workshop proceedings and special issues on reflection and awareness in educational technology. I frequently review research for journals, conferences, and workshops.

I am currently supervising the following post-graduate student:

Simon Penn (EdD) researches innovative use of screencasting to support feedback processes in the context of assessments. Website: http://www.open.ac.uk/people/srp279.

I supervised the following post-gradulate student to completion:

Huda Alrashidi (PhD), who is now a lecturer at the Arab Open University in Kuwait, investigated the automation of the analysis of reflective writings in the context of Computer Science Education. Selected papers are here: http://oro.open.ac.uk/cgi/search/archive/simple?person=huda+alrashidi. Website: https://warwick.ac.uk/study/csde/gsp/eportfolio/directory/pg/u1663490/research/

I am interested in hearing from prospective PhD and EdD students on topics related to Educational Technology, Evaluation of Online and Distance Learning, Online Assessment, Learning Analytics, Educational Data Mining, Quality Enhancement, Automated text analytics (e.g. natural language processing, sentiment analysis, machine learning) for learning and teaching, Automated detection of reflective thinking in student writings, Artificial Intelligence (automated sense-making and decision-making techniques) for teaching and learning, Development and evaluation of intelligent automated analysis tools, and Automated text analytics methods to make sense of student comments.

In this wider context I am currently working on the following two specific topics:

  1. Automated assessment of thinking skills from academic student writings. The objective is to investigate the potential of natural language processing and text mining to automate the assessment of critical/reflective thinking from academic student writings.
  2. Applicability of natural language processing for the automated analysis of open-ended question data. The objective is to investigate natural language processing techniques to automatically derive insights from open-ended question data of large student experience surveys.

If you are interested in any of these two specific topics or interested in the outlined areas above and want to work with me, then please get in contact with me: My Email is firstname.lastname_its_two_nnsATopen.ac.uk. If you are looking for studentships, The Open University regularly announces fully funded PhD studentships on a competitive basis. You can ask me about this.

Some trivia about me. I was part of one of the winning teams of the Mozilla Jetpack for eLearning competition and I provide a free to use questionnaire generator, which was recently rated by the German magazine PC Welt as one of the best free online questionnaires generators.

See my biography for more information.

The permanent link to my latest biography is: http://qone.eu/ullmann.

(last updated 07/07/2023)

Biography

Dr Thomas Ullmann is a Lecturer in the Institute of Educational Technology at The Open University and a Senior Fellow of the Higher Education Academy (SFHEA). He was an Honorary Lecturer in the Department of Computer Science at the University of Warwick.

In his role as lecturer, he is responsible for teaching and for supporting the teaching and learning of the Open University. He supports Quality Assurance and Quality Enhancement processes at the Open University. Specifically, he investigates the latest online and distance teaching practices as well as contribute to the development of novel methods and techniques. He taught on the Master in Online and Distance Education (MAODE).

Thomas joined with Open University in 2008 and started working for the Institute of Educational Technology in 2015. With more than 15 years of experience inside and outside Higher Education, he is an expert in the evaluation of Educational Technology in Higher Education. His current research interests include the evaluation of online and distance teaching and learning, educational text analytics, and the automated analysis of reflective thinking in writings.

Thomas published over 30 publications in high-ranked journals, conferences, and workshops and over 45 research deliverables, internal research reports, quality enhancement reports, and scholarship work. Thomas co-organised the Workshop series on Awareness and Reflection in Technology-Enhanced Learning held in conjunction with the European Conference on Technology Enhanced Learning. He was member of the Programme Committee of six conferences and workshops. Thomas regularly reviews papers for journals, such as the Journal of Artificial Intelligence in Education, IEEE Transaction on Learning Technologies, Interactive Learning Environments, International Journal of Technology Enhanced Learning, Computers in Human Behaviour, and Open Learning: The Journal of Open and Distance Learning. Thomas regularly presents his work at conferences.

Thomas was a Co-Investigator of the LAEP project - 'The implications and opportunities of learning analytics for European educational policy'. LAEP investigated the state of the art of learning analytics in Europe and beyond and gave an outlook into the field during the next 10-15 years (Final report: http://dx.doi.org/10.2791/955210 - open access). He worked for several EU funded projects, such as the FP7 funded Catalyst project (http://catalyst-fp7.eu/), the STELLAR Network of Excellence project (http://www.stellarnet.eu/), the FP6 funded X-Media project (http://www.x-media-project.org/). For the OU, he also contributed to the social learning platform SocialLearn.

For the LAEP project he contributed to the evaluation of the field of learning analytics. For the CATALYST project he led the development and evaluation of the Collective Intelligence Dashboard. For the STELLAR project, he contributed to the development of the STELLAR Science 2.0 infrastructure for researchers in the field of Technology Enhanced Learning. He also developed the social networking platform for technology-enhanced learning TELeurope.eu (http://teleurope.eu, which is now defunct). For the X-Media project, he developed Knowledge Lenses for the SemSearch Semantic Search Engine to support information exploration tasks of knowledge engineers. For the SocialLearn project, he created the ELLIMent tool, supporting mentees in developing their lifelong learning skills within a mentor relationship. ELLIMent builds upon the Effective Lifelong Learning Inventory.

Since 2015 Thomas leads the pan-university wide Text Analytics of Student Comments Initiative. This initiative aims at improving the capability of the Open University to rapidly make sense of large quantities of student comments using automated text analytics methods.

Since 2019 Thomas is part of the OU-wide Quality Enhancement and Innovation (QEI) initiative spearheaded by Simon Cross. QEI investigates, supports and delivers innovative teaching and learning research, advice and practice with the aim of improving pan-university quality enhancement, strategic planning, curriculum delivery, and learning and teaching development.

From 2015 to 2019, Thomas was part of the pan-university Data Wrangler initiative spearheaded by Bart Rienties. He was part of the core team that steered the strategy of the whole Data Wrangler initiative. The purpose of the Data Wrangling initiative was to produce evidence-based and analytically informed reports that support faculties in their decision making and quality assurance and enhancement processes. The Data Wranglers offered several services, such as the bi-annual Scholarly Insights reports, which provided an in-depth analysis of OU-wide pedagogical problems and opportunities, the Key Metrics reports that summarise OU targets for the quality review process, faculty-specific bespoke requests, and general advice about learning analytics questions.

From 2015 to 2018, Thomas taught on the Master of Online and Distance Education until its teach-out. He was the deputy module chair of H817 'Openness and innovation in eLearning' and module team member of H817 'The network practitioner. For H817 I wrote the course materials and activities about reflective thinking and reflective writing for EdTech practitioners.

From 2014 to 2015, Thomas was a Research Assistant at the Knowledge Media Institute.

From 2009 to 2016, Thomas was a part-time PhD student at the Knowledge Media Institute.

From 2009 to 2014, Thomas worked as consultant and software developer for various enterprises in the educational area.

His full biography is available on request.

The permanent link to his latest biography is http://qone.eu/ullmann.

(last updated 07/07/2023)

Publications

2023

Leedham, Maria; Tuck, Jackie; Shrestha, Prithvi; Therova, Dana and Ullmann, Thomas. (). Student Writing and Assessment Practices: combining corpus linguistics and in-depth interviews to investigate assessment in a distance university context. In: Assessment in Higher Education, 22-23 Jun 2023, Manchester, UK.
Leedham, Maria; Tuck, Jackie; Shrestha, Prithvi; Therova, Dana and Ullmann, Thomas. (). Student Writing and Assessment Practices: combining corpus linguistics and in-depth interviews to investigate written assessment in a distance university context. In: The twelfth international Corpus Linguistics conference (CL2023), 3-6 Jul 2023, Lancaster, UK..

2022

Alrashidi, H.; Almujally, N.; Kadhum, M.; Ullmann, T. and Joy, M.. (). Evaluating an Automated Analysis using Machine Learning and Natural Language Processing Approaches to Classify Computer Science Students' Reflective Writing. In: Pervasive Computing and Social Networking. Lecture Notes in Networks and Systems, vol 475, 463-477.
Alrashidi, Huda; Almujally, Nouf; Kadhum, Methaq; Quqandi, Ebtihal; Ullmann, Thomas Daniel and Joy, Mike. (). How to Assess Reflective Writing in Computer Science Education? A Critical Analysis. In: Computers in Education Conference in conjunction with the 45th Jubilee International Convention on Information, Communication and Electronic Technology, 26 May 2022, Opatija, Croatia.

2021

Ullmann, Thomas and Rienties, Bart (). Using text analytics to understand open-ended student comments at scale: Insights from four case studies. In: Shah, Mahsood; Richardson, John T. E.; Pabel, Anja and Oliver, Beverley eds. Assessing and Enhancing Student Experience in Higher Education Cham: Palgrave Macmillan, pp. 211-233.
Charitonos, Koula; Littlejohn, Allison; Dawadi, Saraswati; Mcmullan, Rachel; MacQueen, Hilary; Goshtasbpour, Fereshte; Ullmann, Thomas and De Munari, Paola (). Tackling Antimicrobial Resistance through Professional Learning: The Development and Evaluation of the Global AMR Curriculum. The Open University.

2020

. (). An empirical evaluation of a Reflective Writing Framework (RWF) for Reflective Writing in Computer Science Education. In: IEEE Frontiers in Education Conference (FIE), 21-24 Oct 2020, Uppsala, Sweden.
Alrashidi, Huda; Joy, Mike; Ullmann, Thomas and Almujally, Nouf. (). Educators' Validation on a Reflective Writing Framework (RWF) for Assessing Reflective Writing in Computer Science Education. In: Intelligent Tutoring Systems, Springer International Publishing, Cham, 316-322.
Alrashidi, Huda; Joy, Mike; Ullmann, Thomas and Almujally, Nouf. (). Validating the Reflective Writing Framework (RWF) for Assessing Reflective Writing in Computer Science Education Through Manual Annotation. In: Intelligent Tutoring Systems, Springer International Publishing, Cham, 323-326.
Alrashidi, Huda; Ullmann, Thomas; Ghounaim, Samiah and Joy, Mike. (). A Framework for Assessing Reflective Writing Produced Within the Context of Computer Science Education. In: Companion Proceedings 10th International Conference on Learning Analytics & Knowledge (LAK20, 24/03/2020, Frankfurt, Germany.

2019

Alrashidi, Huda; Joy, Mike and Ullmann, Thomas. (). A Reflective Writing Framework for Computing Education. In: ITiCSE 2019: Proceedings of the 24th Annual ACM Conference on Innovation and Technology in Computer Science Education, 15 Jul - 17 July 2019, Aberdeen, UK.
Ullmann, Thomas Daniel. (). Automated Analysis of Reflection in Writing: Validating Machine Learning Approaches. International Journal of Artificial Intelligence in Education, 29 (2) pp. 217-257.
Clow, Doug; Coughlan, Tim; Cross, Simon; Edwards, Chris; Gaved, Mark; Herodotou, Christothea; Nguyen, Quan; Rienties, Bart; Thorne, Sam and Ullmann, Thomas (). Scholarly insight Winter 2019: a Data wrangler perspective. Open University, Milton Keynes.
Ullmann, Thomas Daniel. (). A Visualisation Dashboard for Contested Collective Intelligence. Learning Analytics to Improve Sensemaking of Group Discussion. RIED: Revista Iboeroamericana de Educación a Distancia (The Ibero-American Journal of Digital Education), 22 (1) pp. 41-80.
Ullmann, Thomas. (). Reflective writing analytics - Validating machine learning approaches. In: CALRG Annual Conference 2019, 17-18 Jun 2019, The Open University, Milton Keynes.

2018

Ullmann, Thomas; Lay, Stephanie; Cross, Simon; Edwards, Chris; Gaved, Mark; Jones, Edwina; Hidalgo, Rafael; Evans, Gerald; Lowe, Sue; Calder, Kathleen; Clow, Doug; Coughlan, Tim; Herodotou, Christothea; Mangafa, Chrysoula and Rienties, Bart (). Scholarly insight Spring 2018: a Data wrangler perspective. Open University UK, Milton Keynes.
Ullmann, Thomas; Lay, Stephanie; Coughlan, Tim; Lister, Katharine; Cross, Simon; Rienties, Bart and Whitelock, Denise. (). Understanding SEaM student comments from a Big(ger)Data perspective: what are students saying?. In: CALRG Annual Conference 2018, 18-19 Jun 2018, The Open University, Milton Keynes.

2017

Rienties, Bart; Clow, Doug; Coughlan, Tim; Cross, Simon; Edwards, Chris; Gaved, Mark; Herodotou, Christothea; Hlosta, Martin; Jones, Jan; Rogaten, Jekaterina and Ullmann, Thomas (). Scholarly insight Autumn 2017:a Data wrangler perspective. Open University UK.
Kravcik, Milos; Mikroyannidis, Alexander; Pammer, Viktoria; Prilla, Michael and Ullmann, Thomas (). Special Issue on: Awareness and Reflection in Technology Enhanced Learning. Vol.9, (2-3). Inderscience.
Rienties, Bart; Rogaten, Jekaterina; Nguyen, Quan; Edwards, Chris; Gaved, Mark; Holt, Deborah; Herodotou, Christothea; Clow, Doug; Cross, Simon; Coughlan, Tim; Jones, Jan and Ullmann, Thomas (). Scholarly insight Spring 2017: A Data Wrangler Perspective. Open University UK.
Coughlan, Tim; Ullmann, Thomas Daniel and Lister, Kate. (). Understanding Accessibility as a Process through the Analysis of Feedback from Disabled Students. In: W4A '17 Proceedings of the 14th Web for All Conference on The Future of Accessible Work, ACM, New York, USA.
Ullmann, Thomas Daniel. (). Reflective Writing Analytics - Empirically Determined Keywords of Written Reflection. In: LAK '17 Proceedings of the Seventh International Learning Analytics & Knowledge Conference, ACM, New York, USA, 163-167.
Rienties, Bart; Cross, Simon; Marsh, Vicky and Ullmann, Thomas. (). Making sense of learner and learning Big Data: reviewing 5 years of Data Wrangling at the Open University UK. Open Learning: The Journal of Open, Distance and e-learning, 32 (3) pp. 279-293.

2016

Rienties, Bart; Edwards, Chris; Gaved, Mark; Marsh, Vicky; Herodotou, Christothea; Clow, Doug; Cross, Simon; Coughlan, Tim; Jones, Jan and Ullmann, Thomas (). Scholarly insight 2016: a Data wrangler perspective. Open University UK.
Ferguson, Rebecca; Brasher, Andrew; Clow, Doug; Cooper, Adam; Hillaire, Garron; Mittelmeier, Jenna; Rienties, Bart; Ullmann, Thomas and Vuorikari, Riina (). Research Evidence on the Use of Learning Analytics: Implications for Education Policy. Joint Research Centre, Seville, Spain.

2015

Parent, Marc-Antoine; De Liddo, Anna; Ullmann, Thomas and Klein, Marc (). Project Testbed: Argument Mapping and Deliberation Analytics. CATALYST Project.
Ullmann, Thomas Daniel. (). Keywords of written reflection - a comparison between reflective and descriptive datasets. In: Proceedings of the 5th Workshop on Awareness and Reflection in Technology Enhanced Learning, 83-96.
Kravcik, Milos; Mikroyannidis, Alexander; Pammer, Viktoria; Prilla, Michael and Ullmann, Thomas Daniel (). Proceedings of the 5th Workshop on Awareness and Reflection in Technology Enhanced Learning. Aachen, Germany: CEUR Workshop Proceedings.
Ullmann, Thomas Daniel (). Automated detection of reflection in texts. A machine learning based approach. PhD thesis The Open University.

2014

Ullmann, Thomas; De Liddo, Anna and Bachler, Michelle (). Collective Intelligence Analytics Dashboard Usability Evaluation. CATALYST Project.
Kravcik, Milos; Mikroyannidis, Alexander; Pammer, Viktoria; Prilla, Michael; Ullmann, Thomas Daniel and Wild, Fridolin (). Proceedings of the 4th Workshop on Awareness and Reflection in Technology Enhanced Learning. In conjunction with the 9th European Conference on Technology Enhanced Learning: Open Learning and Teaching in Educational Communities. CEUR Workshop Proceedings.

2013

Meyer, Philip; Kelle, Sebastian; Ullmann, Thomas Daniel; Scott, Peter and Wild, Fridolin (). Interdisciplinary cohesion of TEL – an account of multiple perspectives. In: Hernández-Leo, Davinia; Ley, Tobias; Klamma, Ralf and Harrer, Andreas eds. Scaling up Learning for Sustained Impact Springer, pp. 219-232.
Ullmann, Thomas; Wild, Fridolin and Scott, Peter. (). Reflection - quantifying a rare good. In: ARTEL13 2013: Awareness and Reflection in Technology-Enhanced Learning, 29-40.
Kravcik, Milos; Krogstie, Birgit; Moore, Adam; Pammer, Viktoria; Pannese, Lucia; Prilla, Michael; Reinhardt, Wolfgang and Ullmann, Thomas Daniel (). Proceedings of The 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning in conjunction with The 8th European Conference on Technology Enhanced Learning: Scaling up learning for sustained impact (ECTEL 2013). CEUR-WS.org.

2012

Moore, Adam; Pammer, Viktoria; Pannese, Lucia; Prilla, Michael; Rajagopal, Kamakshi; Reinhardt, Wolfgang; Ullmann, Thomas and Voigt, Christian (). Proceedings of the 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning. CEUR-WS.
Sie, Rory L.L.; Ullmann, Thomas Daniel; Rajagopal, Kamakshi; Cela, Karina; Rijpkema, Marlies Bitter and Sloep, Peter B.. (). Social network analysis for technology-enhanced learning: review and future directions. International Journal of Technology Enhanced Learning, 4 (3/4) pp. 172-190.
Ullmann, Thomas Daniel; Wild, Fridolin and Scott, Peter. (). Comparing automatically detected reflective texts with human judgements. In: 2nd Workshop on Awareness and Reflection in Technology-Enhanced Learning, 18 Sep 2013, Saarbrucken, Germany.

2011

Wild, Fridolin; Ullmann, Thomas; Scott, Peter; Rebedea, Traian and Hoisl, Bernhard. (). Applicability of the technology acceptance model for widget-based personal learning environments. In: 1st Workshop on Exploring Fitness and Evolvability of Personal Learning Environments (EFEPLE'11), 30-31 Mar 2011, La Clusaz, France.
Ullmann, Thomas Daniel; Ferguson, Rebecca; Buckingham Shum, Simon and Deakin Crick, Ruth. (). Designing an online mentoring system for self-awareness and reflection on lifelong learning skills. In: 1st Workshop on Awareness and Reflection in Personal Learning Environments. In conjunction with the PLE Conference 2011., 11-13 Jul 2011, Southampton, UK.
Ullmann, Thomas Daniel. (). An architecture for the automated detection of textual indicators of reflection. In: 1st European Workshop on Awareness and Reflection in Learning Networks held in conjunction with the 6th European Conference on Technology Enhanced Learning: Towards Ubiquitous Learning 2011, 21 Sep 2011, Palermo, Italy.
Reinhardt, Wolfgang and Ullmann, Thomas (). Editorial: Awareness and reflection in personal learning environments. In: Reinhardt, Wolfgang and Ullmann, Thomas eds. Proceedings of the 1st Workshop on Awareness and Reflection in Personal Learning Environments. In Conjunction with the PLE Conference 2011. Southampton, United Kingdom, July 11, 2011 Not known, pp. 2-9.
Reinhardt, Wolfgang; Ullmann, Thomas Daniel; Scott, Peter; Pammer, Viktoria; Conlan, Owen and Berlanga, Adriana (). Editorial: Awareness and reflection in learning networks. In: Reinhardt, Wolfgang; Ullmann, Thomas Daniel; Scott, Peter; Pammer, Viktoria; Conlan, Owen and Berlanga, Adriana eds. Proceedings of the 1st European Workshop on Awareness and Reflection in Learning Networks. In conjunction with the 6th European Conference on Technology Enhanced Learning: Towards Ubiquitous Learning 2011. Palermo, Italy, September 21, 2011 CEUR-WS, pp. 2-7.

2010

Ullmann, Thomas Daniel; Wild, Fridolin; Scott, Peter; Duval, Erik; Vandeputte, Bram; Parra, Gonzalo and Reinhardt, Wolfgang. (). Components of a Research 2.0 infrastructure. In: 5th European Conference on Technology-Enhanced Learning (ECTEL'10), 28 Sep - 1 Oct 2010, Barcelona, Spain.
Wild, Fridolin; Ullmann, Thomas D. and Scott, Peter. (). The STELLAR science 2.0 mash-up infrastructure. In: Advanced Learning Technologies (ICALT), 2010 IEEE 10th International Conference on, IEEE, 621-623.
Wild, Fridolin and Ullmann, Thomas. (). The afterlife of 'living deliverables': angels or zombies?. In: Proceedings of the research 2.0 workshop at The 5th European Conference on Technology-Enhanced Learning (ECTEL'10), 28 Sep - 01 Oct 2010, Barcelona, Spain.
Duval, Erik; Wild, Fridolin; Scott, Peter; Ullmann, Thomas and Lindstaedt, Stefanie (). Proceedings of the 2nd International Workshop on Research 2.0. CEUR-WS.
Duval, Erik; Ullmann, Thomas Daniel; Wild, Fridolin; Lindstaedt, Stefanie and Scott, Peter (). Editorial: Research 2.0 for TEL: four challenges. In: Duval, Erik; Ullmann, Thomas Daniel; Wild, Fridolin; Lindstaedt, Stefanie and Scott, Peter eds. Proceedings of the 2nd International Workshop on Research 2.0. At the 5th European Conference on Technology-Enhanced Learning: Sustaining TEL, Barcelona, Spain, September 28, 2010 Publisher not known, pp. 2-4.
Camilleri, Anthony Fisher; Barak, Noaa; Burgos, Daniel and Ullmann, Thomas D.. (). Engaging the community in multidisciplinary TEL research: a case-study from networking in Europe. In: EDEN 2010 Annual Conference: Media Inspirations for Learning. What Makes the Impact?, 9-12 Jun 2010, Valencia, Spain.

2009

Ullmann, Thomas Daniel; Uren, Victoria and Nikolov, Andriy. (). The SemSearchXplorer - exploring semantic search results with semantic visualizations. In: AST 2009 Applications of Semantic Technologies, located at the Informatik 2009, 2 Oct 2009, Lübeck, Germany.