RoutingTrie: Metadata Routing for Constrained Generative Retrieval

Darshan Fofadiya, Yang Liu, Shuying Sun, Kyumin Park, Juhana Kangaspunta

Advances in large language models and semantic IDs (SIDs) have brought generative retrieval (GR) into industrial applications. These systems must return items that satisfy request metadata constraints as well as relevance requirements. A catalog trie ensures SID validity, but filtering metadata only after generation wastes computation on rejected candidates and may leave too few eligible results. We propose RoutingTrie to incorporate metadata constraints before beam selection. RoutingTrie maintains one shared SID trie and builds a compact index for each metadata field using Dense Bitset or Sparse List. We derive a storage cost formula to guide the choice between these index representations for different field distributions. During decoding, field probes determine whether requested values occur below candidate edges and produce a combined mask that guides beam selection. Experiments on ShopTrie, our configurable synthetic benchmark, demonstrate compact storage and efficient CPU and GPU probing across diverse metadata distributions. In a production replay, RoutingTrie raises Hit@100 from 67.5\% to 96.1\%, a gain of 28.6 percentage points, with only 1.1 ms latency increase. We will…