Learning to Report Unsafe Tasks in a Multi-Agent Game

Avyay M. Casheekar, Hariganesh Tangirala

When a team of agents shares a reward for completed tasks, reporting and stopping unsafe work reduces the team's reward. Random audits that penalize silence can make universal reporting optimal without ensuring that a silent team learns to report. We study this problem in a game where any witness can stop a task by reporting. When a task's k witnesses share a policy and draw actions independently, the derivative of expected reward with respect to the shared silence probability at universal silence counts its benefit k times, while the comparison between universal reporting and silence counts it once. A fully shared policy can therefore be locally attracted to silence even when reporting is optimal. For any grouping into shared policies, we give an audit condition sufficient for exact policy-gradient updates to reach universal reporting and, outside boundary cases, necessary from near universal silence. In a balanced family, meeting the condition with prescribed margins costs k times as much for a fully shared policy as for one policy per role. We test these predictions with PPO on 24 witness graphs, starting from learned silence. Witness groups, which place co-witnesses in…