Seeing the Options: Typed Decisions over Visual Candidates with Reusable Option Memory
Typed decision-making tasks in agents, such as UI element selection, product recommendation, and chart matching, require models to choose from a set of discrete candidates. While prior work demonstrates that text-only System One models achieve four key properties including candidate-order invariance, scalability, and cache reuse via separate request-candidate encoding, extending them to multimodal settings where candidates are visual remains underexplored. We introduce Iris, a framework that accommodates image, text, or hybrid candidates while preserving the core decision module. We prove that order invariance and exact cache reuse hold for any per-candidate encoder under Iris, and derive the closed-form latency reduction. We construct a visual-candidate benchmark from Multimodal-Mind2Web (K=16, deduplicated, order-shuffled, three generalization splits). Independent encoding (Iris) achieves 78.4 / 72.1 / 74.8 percent accuracy with zero-flip rate under candidate reordering. Joint encoding reaches 90.6 / 89.5 / 91.6 percent but suffers an 8 to 10 percent answer-flip rate when shuffled. To bridge this gap, we propose a design that encodes each candidate conditioned on the request…
- arXiv
- 2610.33702
- Authors
- Fengxin Li
- Categories
- cs.AI · cs.CV