Multi-query retrieval with bounded cost

Different phrasings can surface evidence missed by one query, but unlimited variants multiply model calls, searches and duplicate context. Set a maximum and measure incremental value.

Execute a three-query bound

The retrieval evaluation lab runs BM25 for three authored refund variants.

from retrieval_cases import multi_query_case

result = multi_query_case()
assert result["maximum_variants"] == 3
assert result["calls_fixture"] == 3
assert result["within_bound"] is True
assert len(result["rankings"]) == 3
print(result["unique_candidates"])

The retriever executes locally. Variant generation is authored, so there is no model-generation cost measurement in this fixture.

Control variant quality

Require every variant to preserve protected numbers, entities, negation and authorization scope. Encourage complementary wording rather than three near duplicates. Reject empty, unsafe or intent-changing rewrites.

Deduplicate candidates by stable chunk/source identity and fuse ranks with a declared rule. Apply metadata eligibility to every search. Limit total candidate count and context budget so multi-query does not crowd the answer with repeated text.

Measure marginal gain

On labelled questions, compare one, two and three variants. Report recall gain per added search, latency, query-generation cost, duplicates and downstream support. Preserve cases where extra variants introduce a hard negative or stale version.

Use multi-query only on slices where it helps, such as ambiguous paraphrases. Exact identifiers may be stronger with the original lexical query. Stop early when the first query returns high-confidence eligible evidence if that rule is validated.

The Generative & Agentic AI course treats multi-query as an evaluated, budgeted workflow rather than an automatic quality boost.

Exercise

Generate up to three variants for 50 queries. Validate intent, fuse results and plot recall versus total searches. Define a routing rule and evaluate its false skips on a held-out set.

Continue learning

This article is part of the Retrieval quality and grounded answers sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Query rewriting: detect when the rewrite changes intent.
• Continue with Evaluate retrieval recall at k with labelled evidence.

Reference: LangChain multi-query retriever documentation.