Retrieval and Quality Tuning

Hit Testing (the retrieval lab) runs real retrieval without touching the production configuration and shows the intermediate values of each stage: recall, fusion, rerank, and final ranking. Use it to prove that a chunking change, a mode switch, or a weight change actually improved recall. Open it from the knowledge base detail page → Hit Testing (requires the kb:test permission).

Retrieval modes

There are two layers of defaults: a search request without search_mode runs as hybrid, while the knowledge base setting search_mode defaults to empty (null) and only becomes the default for that KB's Agent/workflow retrieval once you set it explicitly. Field definitions are in the Knowledge Base settings reference.

Parameter reference

The hit-testing panel lets you override the following (all optional):

How hybrid retrieval fuses channels

After both channels recall, results are merged by weighted reciprocal rank fusion (RRF):

score = Σ weight × 1 / (rrf_k + rank)

• Each weight affects only its own channel: dense_weight for the vector channel, lexical_weight for the full-text channel; setting one to 0 disables that channel.
• In hybrid mode the two weights cannot both be 0, or the request is rejected (the UI also disables the search button).
• A smaller rrf_k favours the top-ranked results more strongly; a larger one flattens the ranking differences.

A/B comparison

Turn on A/B Comparison to run two configurations for the same query side by side:

1. Configure side A and side B independently (each with its own mode, top_k, threshold, weights, and rerank parameters).
2. The batch returns an independent result per configuration: a failure on one side never hides the other side's success, and the failing side reports its own reason.
3. The result panel shows each side's ranks, the number of overlapping results (Overlap: N/total), and per-stage scores.

The batch endpoint POST /api/v1/knowledge-bases/{kb_id}/search/batch accepts up to 10 configurations, and each configuration id must be unique.

Reading results and diagnostics

Every result carries the per-stage values, and final_score_stage tells you which stage decided the final order:

Stage timings timings are split into recall, rerank, context, and total, so you can tell whether recall or rerank is the slow part.

Diagnostics diagnostics may return:

Failed requests carry retrieval_error_category and stage so you can pick a direction:

Applying to production

A hit-testing configuration can be saved as a local preset (stored in the browser only, not synced across devices); then Apply to Production writes side A's configuration back into the knowledge base settings:

{
  "search_mode": "hybrid",
  "top_k": 5,
  "score_threshold": 0.0,
  "dense_weight": 1.0,
  "lexical_weight": 1.0,
  "rrf_k": 60,
  "rerank_enabled": true,
  "rerank_candidate_k": 10,
  "rerank_score_threshold": null
}

Before applying, confirm:

• The embedding model and index dimension have not changed (otherwise rebuild the index first).
• In hybrid mode at least one weight is greater than 0.
• The knowledge base is actually referenced — what you write back is the knowledge base level default, and an Agent's retrieval_top_k, score_threshold, and search_mode still override it.

Troubleshooting

Common failures and recovery steps are also covered in the Knowledge Base settings reference; for tuning trade-offs see Knowledge Base Optimization.