SyncPro: Synchronous Reasoning and Responding for Proactive Streaming Video Understanding
Proactive streaming video understanding requires assistants to track unfolding events and respond when a user's requested condition is met. To make these decisions, assistants must accumulate relevant evidence and reassess response conditions as new observations arrive. Existing work offers complementary capabilities for this process: proactive video assistants learn when to respond, while streaming video reasoning methods generate scheduled intermediate text to organize observations and support subsequent answers. Together, these capabilities provide a foundation for proactive reasoning, but how to coordinate ongoing reasoning with timely responses remains underexplored. In a direct adaptation, fixed reasoning intervals can trigger updates when nothing relevant has changed or postpone useful updates until the next scheduled step. Sharing one generation path also couples two distinct requirements: preserving evidence for future decisions and producing only the response requested by the user. Moreover, responses generated after reasoning must wait for it to finish, even when sufficient evidence is already available. To address these challenges, we introduce SyncPro, a synchronous…
Synchronous reasoning and responding for proactive streaming video understanding.
- arXiv
- 2610.32132
- Authors
- Xueyun Tian, Bingbing Xu, Jiaming Liu, Jinpeng Yu, Yang Shi and 4 more
- Categories
- cs.CV · cs.CL · cs.LG