P2P-Former: Bridging the Supervision Gap Between Point and Proposal in Temporal Action Localization

Yunchuan Ma, Laiyun Qing, Sai Xu, Guorong Li, Qingming Huang

Recently, point-supervised temporal action localization has attracted increasing attention due to its low annotation cost (only one frame per action instance) and competitive detection performance. However, since point annotations are less complete than proposal annotations, existing methods mainly rely on multiple instance learning for snippet-level classification, which leads to a performance gap compared to fully supervised models. To bridge this gap, we propose P2P-Former, a novel point-to-proposal Transformer-based framework that effectively connects point-level supervision with proposal-level learning for temporal action localization. In this work, we identify two key challenges in transitioning from point-level supervision to full supervision: (1) how to generate reliable proposals as pseudo ground-truth, and (2) how to effectively integrate point-level cues into the fully supervised paradigm. To address the first challenge, we propose a Point-aware Proposal Generation (PPG) strategy that refines the confidence of proposals produced by a base locator using a point-aware importance estimation function, and then performs optimal matching between point annotations and…