One Hierarchy, Many Sports: Hierarchy-Aligned Multi-Task Learning for Next-Shot Prediction

Chao-Wei Cai, Min-Te Sun, Sheng Kuang Wu, Hung-Hsuan Chen

Next-shot prediction in racket sports requires learning from event data with different match structures, recorded attributes, and label vocabularies. Table tennis, badminton, and tennis nevertheless share a hierarchy in which shots form rallies and rallies form higher scoring units. We present MT-HTA, a hierarchy-aligned multi-task Transformer that uses this structure to jointly predict the attributes of the next shot. A temporal Transformer captures event progression, a feature-token path models interactions among semantic fields rather than along the time axis, and task-specific attention retrieves information from hierarchical summaries and player context. The architecture supports three- and four-level hierarchies and sport-dependent task sets. Across four datasets from three sports, the final MT-HTA achieves the highest mean task-averaged F1 among the compared main models. Ablations on table tennis support the combination of hierarchical encoding, task attention, and feature-token interaction. Additional modules alter the balance between classification scores and spatial error rather than improving every metric. These results support a reusable architecture for heterogeneous…