Best Rerankers for RAG in 2026: 7 Models Compared

Reranking in RAG is the second pass: a bi-encoder pulls the top 50 or 100 chunks from the whole corpus, then a reranker re-sorts that short list by scoring the query against each candidate. The best reranker models for RAG in late 2026 are Cohere Rerank 5 and Voyage rerank-3 as managed APIs, and BGE Reranker v2-m3, Qwen3 Reranker and mxbai-rerank-large-v2 for Apache 2.0 self-hosting.

The seven models below cover managed APIs and open weights, pointwise cross-encoders, listwise rerankers and token-level late interaction. Pick on four things: lift measured on your own corpus, end-to-end latency at top-50, license terms, and the language pairs in your data.

TL;DR by use case

If you only read one row: pick Cohere Rerank 5 for the lowest-friction managed path, BGE Reranker v2-m3 for an open-license self-host, and ColBERTv2 when token-level interaction matters more than the cross-encoder ceiling. Jina Reranker v2, v3 and v3.5 weights are CC-BY-NC-4.0; commercial use needs the Jina API, an official cloud-marketplace image, or Jina On-Prem.

How we evaluated this shortlist

These seven rerankers were picked against five axes that map to real RAG decisions:

1. Accuracy on RAG eval sets: nDCG@10, MRR@10, and Recall@10 lift on standard benchmarks (BEIR, MTEB, MIRACL) and on production reproductions you run yourself.
2. Latency at top-50: P50 and P95 reranking time for 50 candidate documents, measured on your hardware and document length.
3. License and self-hosting: OSI license string on the model card, commercial-use status, and whether the weights are downloadable.
4. Language coverage: published languages on the model card, performance on cross-lingual benchmarks, special-domain coverage (code, function-calling).
5. Operational cost: API price per million tokens or per thousand searches; GPU memory footprint for self-hosting.

Shortlisted but not in the top 7: ZeroEntropy zerank-2 (4B, Apache 2.0 on its model card), the ms-marco cross-encoders, and LLM rerankers that prompt a general model to rank. In mixedbread’s July 2026 test, GPT-5.6 rerankers at high reasoning effort took about 36 to 73 seconds per query versus about 0.65 seconds for its listwise model.

Cross-encoder vs bi-encoder vs LLM reranker

• Bi-encoder: embeds query and document separately with an embedding model; fast enough for the first pass.
• Cross-encoder: reads each query and document pair together, as BGE Reranker v2-m3 does.
• Listwise: reads the whole candidate list in one pass, as Jina Reranker v3.5 does.
• Late interaction: ColBERTv2 matches per-token embeddings at search time.
• LLM prompted to rank: a general LLM such as GPT-5.6 orders the list from a prompt, at far higher latency.

The 7 rerankers compared

1. Cohere Rerank 5: Best for managed accuracy with broad language coverage

Closed model. Hosted API. Cloud-partner availability varies by model version.

Use case: Production RAG pipelines that need managed accuracy without operating a GPU. Cohere released Rerank 5 Pro and Fast on October 21, 2026, both multilingual with a 32K context. Context Focus returns the most relevant passages inside each document, and Cohere recommends at most 1,000 documents per request.

Pricing: $0.05 per 1M tokens for Rerank 5 Pro and $0.02 for Fast. Model Vault Medium instances list at $4.00 per hour or $2,500 per month.

License: Closed. Rerank 5 runs on the Cohere API, Model Vault, Microsoft Foundry and Amazon SageMaker, or in your own VPC or on-premises. Amazon Bedrock still offers Cohere Rerank 3.5, not Rerank 5.

Best for: Teams running a multilingual RAG corpus where the procurement priority is “the reranker is managed” and the latency budget tolerates a network round-trip. Rerank 5 Fast targets latency-sensitive use cases; Rerank 5 Pro targets reasoning-heavy queries and long documents.

Worth flagging: Closed weights; on-prem runs as a Cohere private deployment. Network latency is added on top of model inference; measure on your region.

2. BGE Reranker v2-m3: Best for open-license self-hosted RAG

Apache 2.0. Self-hostable. Hosted on Hugging Face Inference Endpoints.

Use case: Self-hosted RAG with multilingual content where the team owns the GPU and needs an OSI-open license. BGE Reranker v2-m3 is the multilingual cross-encoder from BAAI, built on the BGE-M3 base model.

Pricing: Free for the model weights. Self-hosting throughput depends on hardware, batch size, and document length; benchmark on your workload.

License: Apache 2.0. 568M parameters; the smaller bge-reranker-base (278M, MIT) and the larger bge-reranker-v2-gemma (2.5B) are also published on Hugging Face; verify each variant’s license string on its model card before redistribution.

Best for: Teams that already operate ML infrastructure and want a strong open-license reranker without an API bill. BGE v2-m3 reports MIRACL multilingual results on its model card; verify against your own corpus before committing.

Worth flagging: GPU operations are on the team. Quantization (8-bit, 4-bit) can drop accuracy; verify on your eval set. Cross-lingual queries (English query, Chinese document) need explicit testing; performance varies by language pair.

3. Jina Reranker v3.5: Best for fast multilingual listwise reranking

CC-BY-NC-4.0 weights. Commercial use needs the Jina API, a cloud-marketplace image or Jina On-Prem.

Use case: Multilingual RAG over long or semi-structured documents. Jina Reranker v3.5 (August 2026) is a 0.6B listwise model with a 131K context that scores the whole candidate list in one pass.

Pricing: The Jina cloud API is priced per token.

License: The jinaai/jina-reranker-v3.5 model card lists cc-by-nc-4.0. Commercial use needs the Jina API, an official AWS, Azure or Google Cloud marketplace image, or Jina On-Prem, sold by Elastic since August 10, 2026.

Best for: Teams with long-document or semi-structured corpora that can use the Jina API or Jina On-Prem, or research/non-commercial workloads that fit the CC-BY-NC-4.0 terms.

Worth flagging: The CC-BY-NC-4.0 license is the most common confusion in this category; do not deploy these weights in a paid product without the API or a commercial contract. Verify any sibling variant’s license string on its own model card.

4. ColBERTv2: Best for token-level late interaction

MIT. Self-hostable.

Use case: RAG pipelines where token-level relevance matters more than the cross-encoder ceiling. ColBERTv2 uses late interaction: it stores per-token embeddings for each document, then computes maxsim across token pairs at query time. The model never sees query and document together at inference; the late interaction step does.

Pricing: Free. Self-hosting needs a vector index sized for per-token embeddings, which is far larger than a single-vector index. ColBERTv2 shrinks it with residual compression, and the PLAID engine speeds up search.

OSS status: MIT (Stanford).

Best for: Long-document corpora where token-level relevance signals matter (technical docs, legal, scientific). ColBERTv2 is the canonical late-interaction implementation and the foundation that many newer late-interaction models extend.

Worth flagging: Index size and operations cost are higher than a standard cross-encoder. ColBERTv2 base is English-only; ColBERT-XM exists for multilingual but is a smaller community. The 2024 ColPali extension targets visual document retrieval and is a separate stack.

5. Voyage rerank-3: Best for managed enterprise search

Closed. Hosted API.

Use case: Enterprise RAG where the team already uses Voyage embeddings. Voyage released rerank-3 and rerank-3-lite on September 30, 2026 as drop-in upgrades to the 2.5 series.

Pricing: $0.05 per 1M tokens for rerank-3 and $0.02 for rerank-3-lite; the first 200M tokens are free.

License: Closed. Available on the Voyage API and in MongoDB Atlas Vector Search.

Best for: Teams that already standardize on Voyage embeddings. Strong fit for enterprise search where the buyer prefers a single vendor for embed + rerank.

Worth flagging: Closed weights with no open-weight download. MongoDB acquired Voyage AI in February 2025. Test your own language pairs.

6. mixedbread mxbai-rerank-large-v2: Best for Apache 2.0 multilingual self-hosting

Apache 2.0. Self-hostable.

Use case: Multilingual RAG that self-hosts under Apache 2.0. mxbai-rerank-large-v2 is the second-generation reranker family from mixedbread.ai; the model card lists 100+ languages and reports BEIR, multilingual, Chinese and code-search results.

Pricing: Free for the weights. mixedbread retired its standalone rerank API docs; Mixedbread Search now defaults to mxbai-rerank-v3.1-listwise (July 2026).

License: Apache 2.0. The model is marketed as ProRank-1.5B; Hugging Face reports about 1.5B parameters. Smaller variants mxbai-rerank-base-v2 and mxbai-rerank-xsmall-v1 are also published.

Best for: Teams that want a mid-size open reranker between BGE v2-m3 (568M) and the 4B Qwen3. The variant range covers laptop-to-GPU deployments.

Worth flagging: Larger memory footprint than bge-reranker-base; right-size the variant. Verify the language matrix on the model card against your top language pairs.

7. Qwen3 Reranker: Best for multilingual Apache 2.0 family

Apache 2.0. Self-hostable.

Use case: Multilingual RAG with strict open-license requirements and a tunable size budget. Qwen3 Reranker is part of the Qwen3 family from Alibaba and ships in 0.6B, 4B, and 8B variants. The model card lists 100+ languages and a 32K context.

Pricing: Free for self-hosting. Fireworks AI hosts the 8B with separate metering.

License: Apache 2.0.

Best for: Teams with multilingual workloads (especially Chinese-English) and strict on-prem requirements where the weights need to stay inside the data perimeter. The size range supports laptop-to-GPU deployments.

Worth flagging: 8B model has higher inference latency than smaller cross-encoders; benchmark against your latency budget. Verify the model card’s published language list before relying on long-tail languages.

Decision framework by constraint

• Managed API with broad language coverage: Cohere Rerank 5 (Pro for accuracy, Fast for latency).
• Self-host with OSI-open license: BGE Reranker v2-m3 (Apache 2.0).
• Tight latency budget on self-host: ColBERTv2 (late interaction), or smaller BGE variants.
• Token-level relevance on long documents: ColBERTv2.
• Apache 2.0, mid-size or large self-host: mixedbread mxbai-rerank-large-v2 or Qwen3 Reranker.
• Already on Voyage embeddings: Voyage rerank-3 for the bundle.
• Multilingual, strict open license, on-prem: Qwen3 Reranker (4B or 8B).
• Long-context multilingual, can use Jina API: Jina Reranker v3.5 via the Jina API (CC-BY-NC-4.0 weights are not safe to deploy directly in a paid product).

Common mistakes when picking a reranker

• Picking on a public benchmark only. BEIR, MTEB, and MIRACL are useful starting points, but a real RAG corpus has its own term distribution and intent mix. Build a 100 to 500 query golden set and re-rank on it before committing.
• Ignoring downstream answer quality. A reranker can lift Recall@10 measurably and lift answer-quality evals by less than expected. Track both; the reranker serves the answer.
• Skipping the latency budget. Reranker latency stacks on top of retrieval, generation, and any guardrails. Verify p50 and p95 on your real document length and production hardware, not on the model card benchmark default.
• Choosing an English-only reranker for a multilingual corpus. ColBERTv2 base and most ms-marco cross-encoders are English-trained; running them on Chinese, Japanese, or Arabic queries reduces accuracy. Check the model card’s published language list.
• Treating the license as automatic. Jina Reranker v2, v3 and v3.5 weights are CC-BY-NC-4.0 and are not safe to deploy directly in a paid product. Sibling variants from any vendor can have different license strings. Verify each model card before shipping.
• Forgetting CI eval gates. A reranker swap can lift accuracy and break a long-tail of edge cases. Gate the change behind a CI eval set with deterministic thresholds.

Reranker leaderboards and benchmarks

Each public reranker board measures something different. Agentset scores models by Elo from GPT-5 pairwise judgments over FAISS top-50 results on datasets that include FiQA and Paul Graham essays. Its February 15, 2026 update put Zerank 2 first at 1638, just ahead of Cohere Rerank 4 Pro at 1629.

MTEB has a reranking task category but is built for embedding models. OpenRouter ranks rerank models by request count, a usage signal, and already lists Voyage rerank-3. Agentset’s last update predates both Cohere Rerank 5 and Voyage rerank-3.

Vendor numbers move faster. Cohere reports Rerank 5 Pro at 82.2 RCP-nDCG@10 on BRIGHT, 12.4 points above Rerank 4 Pro, and Voyage reports rerank-3 2.72% ahead of Cohere Rerank v4.0 Pro across 95 datasets. Treat both as vendor benchmarks and confirm on your own golden set.

How to evaluate a reranker for production

1. Build a real golden set. Pull 100 to 500 queries from production logs. Label the top-10 documents per query for relevance. Commit the labels to a versioned dataset.

2. Run two passes. Bi-encoder retrieve top-50 once; rerank with each candidate model on the same top-50. Compute nDCG@10, MRR@10, Recall@10, and downstream RAG metrics (groundedness, answer relevance) per model.

3. Cost-adjust. Compute end-to-end latency, GPU memory, and cost per query for each candidate. Pick the model that wins on accuracy and fits the latency and cost budgets, in that order if you must compromise.

4. Gate the change. Add the reranker eval to CI. A future reranker version, an embedding swap, or a chunking change should rerun the eval and surface a regression before merge. Future AGI scores this gate with built-in NDCG@K, Recall@K, MRR and Groundedness evals. Reranker is a built-in span kind in Future AGI tracing, so each rerank step records the documents and their new order, and the same evals run in CI/CD on every pull request.

Let your golden set pick the reranker

Leaderboards narrow the choice to three or four candidates; your own golden set picks the winner. Rerank the same top-50 with each, keep the model that lifts nDCG@10 and groundedness inside your latency and cost budget, and rerun the test when a vendor ships a new generation. Jina v3.5, Voyage rerank-3 and Cohere Rerank 5 all shipped between August and October 2026.

The Future AGI platform has an Apache-2.0 core you can self-host, and it scores each candidate with built-in retrieval evals. traceAI captures every retrieval step as an OpenTelemetry trace, so a reranker swap shows up as a measured change in your traces.

Start free on Future AGI and run your reranker bake-off on your own queries.

Sources

• Cohere Rerank docs
• Cohere Rerank 5 launch post
• BGE Reranker v2-m3 model card
• Jina Reranker models, licenses and On-Prem
• ColBERT GitHub repo
• Voyage rerank-3 release
• Voyage AI pricing
• mixedbread mxbai-rerank-large-v2 model card
• mixedbread mxbai-rerank-v3.1-listwise release
• Qwen3 Reranker model card
• Qwen3 Embedding and Reranker announcement
• MongoDB Voyage AI acquisition
• Agentset reranker leaderboard
• BEIR benchmark
• MTEB leaderboard