· Chunking
Chunk size for RAG: a starting point and one tuning pass
There is no best chunk size for RAG. Start from headings with a cap, count in tokens, and change one setting at a time against real questions.
Notes on context retrieval for web RAG
One question per note, answered with sources you can check and starting values you can test. Each note links up to the guide that covers its pipeline step end to end.
RSS feedChunking
Retrieval
· Retrieval
BM25 vs embeddings on documentation pages: which queries each wins
BM25 finds exact strings such as error codes and versions; embeddings find paraphrases. On docs pages you usually need both, fused into one ranking.
Freshness
· Freshness
Delete stale vectors without deleting pages that only failed once
Removed pages keep answering until their vectors go. Delete on a confirmed 404 or 410, never on a timeout, and watch sync modes that delete what is missing.