Bedrock Rerank API: Improving RAG Result Quality Before Answer Generation

Integrate the Bedrock Rerank API between vector retrieval and answer generation, measure hit rate and relevance improvement, and choose the right threshold for a...

A vector search retrieves the passages most similar to a query embedding. That is not the same as retrieving the passages that best answer the query. Vector similarity measures geometric distance in embedding space, which correlates with semantic relatedness but does not directly measure answer quality. A reranker addresses this gap: it takes the top-k passages from vector search and scores them specifically for relevance to the query, using a cross-encoder model that sees both the query and the full passage simultaneously.

The Bedrock Rerank API provides this stage without requiring you to host your own reranker. You send the query and the retrieved passages; the API returns them with relevance scores. You take the top-N by reranker score and pass those to your generation model.

This article covers the integration pattern, measurement approach, threshold selection, and failure modes. The worked example uses an internal policy search.

Where reranking fits in the RAG pipeline

Standard RAG pipeline without reranking:

query → embedding → vector search (top-20) → generation (all 20 passages)

RAG pipeline with reranking:

query → embedding → vector search (top-20) → rerank (top-20 → top-5 by score) → generation (top-5 passages)

The reranker reduces the number of passages sent to the generation model. This has two effects: the prompt is smaller (lower token cost), and the passages in the prompt are more likely to contain the answer (higher answer quality if retrieval originally returned noise).

Reranking adds latency and cost. The reranker model must process each query-passage pair. If you retrieve 20 passages and rerank them, you are making a reranker call with 20 input pairs. This latency is acceptable for search interfaces where a user waits for results, but may not be acceptable for high-throughput real-time applications.

Bedrock Rerank API integration

The Rerank API accepts:

• A query (the user's question)
• A list of text sources (the retrieved passages)
• The model identifier for the reranker model (check the Rerank API documentation for supported models and their ARNs)

It returns the sources with relevance scores. The scores are on a 0-1 scale (the exact scale depends on the reranker model; verify against the model's documentation).

Illustrative integration in Python (not production code):

import boto3


# Amazon Rerank 1.0 is supported in us-west-2, ca-central-1, eu-central-1, ap-northeast-1.
# It is NOT supported in us-east-1. Use Cohere Rerank 3.5 (cohere.rerank-v3-5:0) if you need us-east-1.
# Source: https://docs.aws.amazon.com/bedrock/latest/userguide/rerank-supported.html, checked 2026-11-05.
bedrock_runtime = boto3.client("bedrock-runtime", region_name="us-west-2")

def rerank_passages(query: str, passages: list[str], top_n: int = 5) -> list[dict]:
    """
    passages: list of text strings from vector retrieval
    Returns top_n passages sorted by reranker relevance score
    """
    sources = [{"type": "INLINE", "inlineDocumentSource": {"type": "TEXT", "textDocument": {"text": p}}} for p in passages]

    response = bedrock_runtime.rerank(
        rerankingConfiguration={
            "type": "BEDROCK_RERANKING_MODEL",
            "bedrockRerankingConfiguration": {
                "modelConfiguration": {
                    "modelArn": "arn:aws:bedrock:us-west-2::foundation-model/amazon.rerank-v1:0"
                },
                "numberOfResults": top_n
            }
        },
        sources=sources,
        textSources=[{"type": "QUERY", "textQuery": {"text": query}}]
    )
    return response["rerankingResults"]

The return value contains the relevance scores and the ranked results. Use the top N results (as specified by numberOfResults) to construct the prompt for your generation model. The model ARN amazon.rerank-v1:0 is confirmed in AWS documentation. Regional support for Amazon Rerank 1.0 as of 2026-11-05: us-west-2, ca-central-1, eu-central-1, ap-northeast-1. It is not available in us-east-1; use cohere.rerank-v3-5:0 there. Source: Supported regions and models for reranking, checked 2026-11-05.

The reranker adds latency to each query. Measure the p50 and p99 added latency in your environment before releasing to production, and set a timeout for the reranker call that is consistent with your end-to-end response SLA. If the reranker call fails or times out, your fallback should return the original retrieval order rather than surfacing an error to the end user.

Worked example: internal policy search

An HR team uses a Bedrock Knowledge Base to answer policy questions from employees. The knowledge base contains 200 policy documents. Initial retrieval uses numberOfResults=20 to cast a wide net. The generation model is asked to answer using the retrieved passages.

Before reranking: observed problems:

The knowledge base returns 20 passages. Many are semantically related to the query but are from general policy sections that do not contain the specific answer. The generation model, faced with 20 passages, sometimes constructs an answer that blends content from multiple passages and misses the specific exception clause that actually answers the question.

Measurement method:

Construct a test set of 30 policy questions, each with a known answer from a specific passage. For each question:

1. Run retrieval (top-20 passages)
2. Check whether the answer-bearing passage is in the top-20 (hit@20)
3. Check whether the answer-bearing passage is in the top-3 (hit@3)
4. Run reranking on the top-20
5. Check whether the answer-bearing passage is in the top-5 by reranker score (hit@5-reranked)

Illustrative comparison (fictional scores, not actual measurements):

The reranker does not improve the recall (hit@20 is the same, because retrieval is the same). It improves precision by pushing the best passage into the positions the generation model uses most.

Threshold selection:

After reranking, you receive a relevance score per passage. If the top passage has a score below 0.3, the question probably falls outside the knowledge base. Consider refusing to answer (or returning a "not found" response) rather than generating from low-relevance passages. The right threshold depends on your domain and how the scores distribute for in-scope vs out-of-scope questions. Set it by reviewing the score distribution on your test set.

Cost and latency considerations

Reranking adds an API call per user query. The cost depends on the reranker model and the number of passages processed. Check the Bedrock pricing page for current reranker model pricing.

Latency impact: a reranker call adding 50-200ms is typically acceptable for a search interface. For a Bedrock Agent where the agent invokes retrieval as a tool and needs fast turnaround, evaluate whether reranking fits within your latency budget.

If latency is a concern, reduce the number of passages you send to the reranker. Retrieve 20 at the vector search stage but only send the top-10 by vector score to the reranker. This reduces reranker input volume at the cost of possibly missing a passage that ranked 11-20 in vector search but would have ranked first in the reranker.

Failure modes

Wrong reranker model ARN. The API returns a validation error if the ARN is invalid or the model is not available in your region. Test with the exact ARN before shipping.

Reranking scores not calibrated across domains. Reranker scores are relative within one query's passage set. A score of 0.5 for a policy question does not mean the same thing as a score of 0.5 for a legal contract question. If you use a threshold, evaluate it separately for each document domain.

Reranker call failing silently. If the Rerank API call fails and your code falls back to the original vector-ranked order, the pipeline degrades gracefully but you lose the quality improvement without an alert. Emit a metric when the reranker call fails so you can track degradation rate.

Passage too long for reranker. Some reranker models have input length limits per passage. If your retrieved chunks are very long, they may exceed the limit. Check the reranker model's documentation for token constraints.

What to do next

The next useful step is building the measurement method before integrating the reranker. Establish your baseline hit@k metrics. After integration, compare on the same test set. If the reranker provides a meaningful improvement, integrate it into your production pipeline and set up a monitoring metric for reranker call failures.

For the full RAG pipeline design and production deployment checklist, see the Bedrock RAG Pipeline Deployment playbook. For Ampity's work on retrieval systems, see the LLM and RAG Systems service page.

Primary references

• Bedrock Rerank API overview, source checked 2026-11-05
• Rerank API reference, source checked 2026-11-05