The outcome

Choose an embedding and reranking pipeline from labeled task evidence with explicit quality, latency, cost, and no-answer gates.

Step by step

A workflow you can repeat.

  1. 01

    Build labeled queries with relevant, partially relevant, irrelevant and forbidden documents, including typos, paraphrases, multilingual, long-tail, ambiguous, no-answer, stale, adversarial and duplicate cases.

  2. 02

    Embed queries with input_type=query using the exact corpus-compatible model, retrieve a generous candidate set, and measure recall at k before adding any reranker.

  3. 03

    Run a current reranker on only authorized candidates, set truncation deliberately, sweep candidate and top-k sizes, and measure ranking quality, no-answer behavior, tail latency, token use, errors and cost.

  4. 04

    Inspect failures against source text, especially relevant passages truncated from long documents, near-duplicate inflation, metadata mismatch, query-document role reversal, and scores treated as calibrated confidence.

  5. 05

    Set release thresholds and monitoring samples, canary the full pipeline, track model and index versions per result, and roll back both together when compatibility or quality changes.

Working standard

What good use looks like.

  • Measure retrieval recall before reranking.
  • Never treat relevance score as truth.
  • Roll back model and index together.

Official references

Check the current product documentation.