Retrieval
Versioned docs in one RAG index: filter answers by version
Store a version field on every chunk of a versioned docs site, filter on it at query time, and know which questions a version filter cannot answer.
By Rose Donaldson, Context Retrieval team · Published
This article was drafted with AI assistance from the sources listed below and edited and published by the Context Retrieval team. Disclosure: the Context Retrieval API, which this article mentions, is this site's own product.
- 1Read the final URLTake the version slug from the path after redirects, not from the URL you requested.
- 2Keep aliases apartStore latest or stable as an alias; store a release number only when the page states it.
- 3Tag every chunkPut the version in the metadata, the chunk ID and the embedded header.
- 4Filter at query timeUse the version the user named, or a default, as a metadata filter.
- 5Route change questionsQuestions about what changed need both versions side by side, not one filter.
A documentation site that keeps old releases online publishes the same page many times. The install guide for 2.0, 2.1 and 3.0 share most of their sentences, so their embeddings sit close together. Ask "how do I configure retries?" and the retriever returns whichever copy scores a hair higher, which may be the one for a version the user has never run. The answer reads well, cites a real page and is still wrong.
The VersionRAG paper names this failure directly: standard retrieval picks semantically similar text without checking whether it is valid for the version in question.1 On the authors' VersionQA benchmark (100 hand-curated questions over 34 versioned technical documents), naive RAG answered 58% correctly, GraphRAG 64%, and their version-aware method 90%.1 Those are the authors' numbers on their own set. We have not reproduced them, and the method itself, a hierarchical graph of versions and changes, is more machinery than most teams need on day one. The useful lesson is cheaper: version has to be a field the retriever can see, not something the model is left to infer from prose.
Take the version from the final URL
Most versioned docs encode the version in the path. Read the Docs is a common example: each version gets a slug derived from its branch or tag name, and the project root redirects to the default version, which is latest unless the maintainers switch it to stable.2 That redirect is why the address you store matters. If you requested the root and stored that, every chunk would claim no version at all. The final URL after redirects carries the slug.
Two of those slugs are aliases, not releases. On Read the Docs, latest follows the default branch, and stable is created automatically when tags follow semantic versioning and points at the greatest stable release, skipping pre-releases.2 Both move over time. Store the alias as an alias, and store a concrete release number only when the page itself states one, for example in its title or a version heading. Leave that field empty when the page does not say; a guessed release number is worse than none.
import re
from urllib.parse import urlparse
ALIASES = {"latest", "stable"}
RELEASE = re.compile(r"^v?\d+(\.\d+){0,2}$")
def version_from_url(final_url: str) -> dict:
for segment in urlparse(final_url).path.split("/"):
if segment in ALIASES:
return {"version": segment, "version_kind": "alias"}
if RELEASE.match(segment):
return {"version": segment.lstrip("v"), "version_kind": "release"}
return {"version": "unversioned", "version_kind": "none"}
print(version_from_url("https://example.com/en/2.1/guide/retries/"))
print(version_from_url("https://example.com/en/stable/guide/retries/"))
Run it against a sample of your own URLs before you trust it. Sites differ: some put the version in a subdomain, some in a query string, some only in a dropdown.
Put the version in three places
Once a page has a version, every chunk of it should carry that version in three places, each for a different reader.
- Metadata, so the vector store can filter on it.
- The chunk ID, so the
2.1and3.0copies of the same section never overwrite each other when you re-index. - The embedded header, a line such as
Version 2.1 > Configuration > Retriesprepended to the chunk text. This helps when the user types a version number, because exact strings like2.1are where keyword search earns its place in hybrid retrieval.
The fetch step is where the final URL and title come from. If you fetch with the Context Retrieval API, results[].url is the final URL after redirects and title comes with it, so both can go straight into this metadata; the SDKs show the request. Everything after that, the parsing, chunking and filtering, is yours.
Filter at query time
With the field in place, the query needs a version to filter on. Take it from the question if the user named one, from the product context if your app knows it, and otherwise from a default you choose on purpose, usually the current stable release. Chroma, for example, accepts a where dictionary on query, where a bare value means equality and $in matches any value in a list.3
import chromadb
client = chromadb.Client()
docs = client.create_collection("docs")
docs.add(
ids=["2.1#retries#0", "3.0#retries#0"],
documents=[
"Version 2.1 > Retries: set max_retries in client.yaml.",
"Version 3.0 > Retries: pass a retry policy to the client.",
],
metadatas=[
{"version": "2.1", "version_kind": "release", "url": "https://example.com/en/2.1/retries/"},
{"version": "3.0", "version_kind": "release", "url": "https://example.com/en/3.0/retries/"},
],
)
hits = docs.query(
query_texts=["how do I configure retries?"],
n_results=1,
where={"version": "2.1"},
)
print(hits["documents"][0])
How the store applies the filter matters. Qdrant's write-up on filtering describes the trap of post-filtering: run the vector search, then drop results that fail the filter, and you can end up with fewer results than you asked for.4 With many versions in one index, that is the normal case, since most of the nearest neighbours belong to the wrong version. Check whether your store filters before or during the search. Qdrant, for its part, recommends a payload index on every field you filter on, created before ingestion.4
Measure the change on your golden question set, and add a few questions per version that only one version answers correctly. Those are the misses this filter exists to fix.
What a filter cannot answer
A version filter narrows retrieval to one version. Some questions need two: "what changed in retries between 2.1 and 3.0?" or "does this still work after the upgrade?". VersionRAG calls these implicit change questions, and on its benchmark the baselines answered 0–10% of them while the version-aware method reached 60%.1 A plain filter will not move you toward the higher number. Two practical options: retrieve from both versions with $in, label each chunk with its version in the prompt and ask the model to compare; or index changelogs and migration guides as documents in their own right, since they already state the differences in words.
When a version is retired from the site, its pages start to fail on re-fetch. Remove them deliberately rather than on the first failure; deleting stale vectors covers the rule.
If you want the fetch step handled with the final URL and title on every result, you can request API access. Companies and individual professionals can sign up. Only Business, Enterprise and Custom sign-ups are reviewed before checkout, usually within 1 business day.
Footnotes
Frequently asked questions
How do I stop a RAG system from answering with the wrong documentation version?
Store the version on every chunk as metadata, taken from the final URL path or the page itself, and filter retrieval on it. When the user names no version, filter on a default such as the current stable release instead of searching all versions at once.
Should I keep old documentation versions in the same RAG index?
Yes, if users still run those versions, as long as every chunk carries a version field and every query filters on it. Without the filter, near-identical pages from several versions compete for the same top-k slots.
Can a version filter answer questions about what changed between versions?
No. A filter narrows retrieval to one version, while a change question needs passages from two versions compared side by side. Retrieve from both versions with labels, or index changelog pages as their own documents.