Silent Failures in Agent-Tool Interaction: An Audit of ToolUniverse

Devansh Singh, Shreya Gopalan, Sundaraparipurnan Narayanan

Agentic AI systems are increasingly adopting automated pipelines that integrate multiple tools. While prior research have studied task success and task completion of such agentic systems, research on agent to tool interaction chain via a wrapper and API, specifically in biological agentic workflows, remains limited. This study investigates specific failures in agent to tool interaction where a tool invocation appears successful but some or all of the information or functionality from the tool via its API / wrapper is incomplete or missing and there are no communications to the user or the agent regarding the missing information. We call these 'silent failures' because the user or the agents are not aware that such a failure has occurred. For the purpose of this study, we developed an audit mechanism to identify such silent failures in agent to tool interaction chains, by examining 15 scientific tools (and their associated API documentation and tool documentations) integrated within the ToolUniverse environment (ToolUniverse serves as our experimental environment rather than the object of the study itself). We structure our study around 7 failure locus, characterising where the…