Predicting learning depth through dialogue : A design-based approach to SOLO classification using Naive Bayes in online MBA education

Abstract

This study explores the intersection of dialogic pedagogy, cognitive complexity and artificial intelligence (AI) in the context of asynchronous online learning within a postgraduate Executive MBA (EMBA) programme. The research applies a Naive Bayes classifier, developed in Statistical Package for the Social Sciences (SPSS), to predict student learning outcomes based on the Structure of Observed Learning Outcome (SOLO) taxonomy using dialogic indicators coded from forum posts. The study integrates the Toolkit for Systematic Educational Dialogue Analysis (Tech-SEDA) framework — a tool for analysing the quality of educational dialogue — with SOLO levels to examine whether machine learning (ML) can reliably infer cognitive depth from qualitative features of asynchronous student interactions. Using a hybrid design-based research (DBR) methodology informed by Bannan-Ritland’s Integrative Learning Design Framework (ILDF), the study was conducted over three instructional cycles in an online MBA course hosted on Moodle. Across 1,273 forum posts, dialogic features such as reasoning with invitation (REI), challenge (CH), elaboration (EL) and reference to wider context (RW) were identified and analysed alongside sentiment scores, lexical diversity and behavioural metadata. The classifier achieved a predictive accuracy of 73.4 per cent, with strongest performance at SOLO Levels 3 and 4. Feature importance analysis confirmed that dialogic richness, rather than superficial metrics such as word count or thread depth, was most strongly associated with deeper cognitive engagement. Findings highlight how structured prompts, AI-generated scaffolds (eg ChatGPT-facilitated chain-of-thought [CoT] questions) and dialogic design can significantly improve the quality of online student contributions. Moreover, the use of SOLO as a predictive outcome reframes it from a summative assessment tool to a dynamic indicator of dialogic development. This work demonstrates how AI-enhanced educational analytics can support real-time insights and adaptive feedback while maintaining alignment with humanistic values and pedagogical integrity. Although limited by its single institutional context, the study offers a replicable and scalable model for embedding learning analytics into adult online education. This paper is also included in The Business & Management Collection which can be accessed at https://hstalks.com/business/.