Pinecone shipped full-text search into public preview, and the headline reads like the end of a long-running annoyance: a single index now holds text fields, dense vectors, sparse vectors, and filterable metadata, all defined together in a schema you set at index creation. BM25 scoring across multiple text fields, Lucene query syntax, multi-language tokenization — the keyword half of retrieval, native, in the same system that already does your embeddings. For anyone who has been running Pinecone next to an Elasticsearch or OpenSearch cluster and reconciling two ID spaces by hand, that is genuinely good news.
It is also where most integration write-ups stop, and where the interesting constraint begins. "One index for keyword and vectors" is true. "One query that ranks by both" is not — and the gap between those two sentences is exactly the part you have to design around.
One request, one score#
A Pinecone search request scores results by a single score_by directive. Point it at a query_string and you get Lucene-syntax keyword search with BM25 ranking across your text fields. Point it at a vector and you get dense semantic ranking. What you cannot do is hand one request a query and ask it to produce a single ordering that blends BM25 relevance and cosine similarity into one number. The ranking function is singular by design.
Pinecone gives you two honest ways to combine the signals anyway, and it's worth being precise about what each one actually is, because they are not the "hybrid search" people usually picture.
- Lexical as a filter. The text-match operators — exact phrase, all-tokens, any-token (
$match_phrase,$match_all,$match_any) — act as a constraint on a vector search. The dense query runs only over documents that already satisfy the keyword condition. This is powerful and underrated: "semantically closest, among the docs that literally contain this error code." But notice what it is not. Keyword match here is a gate, not a co-ranker. A document that's a perfect lexical hit but a mediocre semantic match is admitted, then ranked purely by vector distance. BM25 never touches the final order.
- Two searches, fused in your code. If you want lexical and semantic to both influence rank — the thing "hybrid" usually means — you run a keyword search and a vector search and merge the two result lists yourself. And the merge is where the sharp edge is.
Why the fusion can't stay on the server#
You might expect Pinecone's single-index sparse+dense support to solve this for free — after all, it stores a sparse vector and a dense vector per record and can query them together in one request. But BM25 and sparse-lexical scores are not normalized to the dense vector's scale. Mix the raw numbers and the components fight on different axes; in practice the sparse/lexical side dominates the combined score unless you hand-tune weights that have no principled value. This is the oldest trap in hybrid retrieval, and putting both vectors in one index doesn't disarm it.
The standard answer is Reciprocal Rank Fusion, which sidesteps normalization entirely by throwing away the scores and combining rank positions — a document's contribution is 1 / (k + rank) in each list, summed. No score is compared to any other score, so BM25's units and cosine's units never have to be reconciled. It's robust, it's boring, and it's the right default. It is also, for the true-hybrid case, something you run after Pinecone returns — over two result sets, in your application code.
The single index consolidated your storage. It did not consolidate your ranking. Fusion is still a thing you own, and pretending otherwise is how "hybrid search" quietly becomes "vector search with a keyword prefilter."
What actually changed, and what to build#
Don't undersell the win: collapsing keyword, dense, sparse, and metadata into one schema-defined index removes a whole class of operational pain — no second cluster, no cross-system ID reconciliation, no separate tokenizer to keep in sync with your corpus. For a large fraction of real workloads, the lexical-as-filter pattern is not a consolation prize; it's the correct architecture. "Find me the semantically relevant passages that mention CVE-2026-55255" is a filtered vector search, and now it's one call.
But if your retrieval quality genuinely depends on a keyword-strong document and a semantically-strong document competing for the top slot, budget for the fusion layer. Fire the two searches concurrently, RRF the results, and — because you're now doing rank fusion anyway — consider a reranker as the final stage rather than trusting the fused order raw. The feature that looked like "Elasticsearch and vectors in one API call" is really "your two retrievers now live in one house." Introducing them to each other is still your job.
