A Time-Synchronous Prequential Evaluation Framework for Next-Week Student Disengagement Prediction from Moodle LMS Logs

David Otoo-Arthur, Terence L. van Zyl

Retrospective evaluation of early-warning models may not reflect information available when predictions are issued and outcomes become observable. We developed a time-synchronous prequential framework for online prediction of next-week student disengagement, operationalised as inactivity onset, from Moodle logs. Ordered by calendar time, models were updated only with available outcomes and remained unchanged for all predictions at each prediction cutoff; development, calibration and evaluation were chronologically separate. We compared stochastic gradient descent logistic regression (SGD-LR), online sequential extreme learning machine (OSELM) and echo state network (ESN) across predictive performance, fixed, frozen and rolling threshold policies, calendar-time variation and four behavioural feature sets. Held-out evaluation comprised 49,329 predictions with 37.68% inactivity-onset prevalence. Discrimination was similar (average precision 0.665-0.679; ROC-AUC 0.755-0.774). OSELM had the highest recall, SGD-LR the highest precision and specificity, and ESN the highest Matthews correlation coefficient and balanced accuracy. Lower thresholds increased recall and alert volume, whereas…