Context retrieval guides for RAG over web pages
Each guide covers one step after the fetch: what to keep from a page, how to split it, how to retrieve and cite it, and how to keep the index true. Starting values to copy, sources listed at the end.
Ingestion
Web content ingestion for RAG: what to keep after the fetch
The source list, metadata schema, document IDs and failure records that decide what your retriever can find and what your model can cite.
0 related posts
Chunking
Chunking Markdown for RAG: structure first, size second
Split web Markdown on its headings, keep tables and code whole, and give every chunk the page title and heading path it needs to be found.
1 related post
Retrieval
RAG retrieval for web pages: evaluate, hybrid search, rerank
A 30-question golden set, then contextual chunk headers, hybrid search, reranking and context packing, each tested on your own pages.
1 related post
Citations
RAG citations from web sources: cite the passage, URL and date
Cite the final URL, the passage and the date it was read, verify every citation after generation, and handle pages that change or disappear.
0 related posts
Freshness
RAG index freshness: re-fetch, re-embed, delete what is gone
Refresh each source on its own schedule, re-embed only what changed, delete what is gone, and know how old every answer's sources are.
1 related post