Chunking architecture
H2- and H3-anchored sections at roughly 300 tokens, each one self-contained enough to stand alone as an answer. Enriched with product, version, and topic-type metadata so retrieval can filter before it ranks.
Enterprise documentation is written for people, which is why retrieval over it performs badly. My work is turning a large product corpus into something a model can query accurately, then measuring whether it actually worked.
Documentation now has two readers: the developer on the page, and the model retrieving from it. They fail differently. A person skims past a bad heading; a retriever returns the wrong chunk and the model answers confidently from it.
So I architect every layer for both: structure, metadata, chunking, and evaluation. I treat retrieval quality as a number I can move, not a feeling. Fourteen years to get here: DITA and single-source publishing, information architecture at Adobe HelpX scale, and now retrieval engineering at Actian.
The search box above is the argument. It indexes this page at load, scores your query with term frequency weighted by field, and ranks the sections. Same shape as the pipeline, minus the embeddings.
H2- and H3-anchored sections at roughly 300 tokens, each one self-contained enough to stand alone as an answer. Enriched with product, version, and topic-type metadata so retrieval can filter before it ranks.
Answer relevance, faithfulness, and context recall measured against a repeatable baseline. Evaluation is the part most RAG projects skip, and the reason most of them cannot be trusted in production.
sentence-transformers, text-embedding-3-small, and bge-m3 scored against the actual documentation corpus rather than a public leaderboard. The corpus decides, not the ranking table.
Exposing a corpus, its analytics, and its build status as tools any MCP client can discover and call. Static pages become a live source an assistant can query mid-conversation.
Documentation modelled as nodes and typed edges (REQUIRES, COVERS, BELONGS_TO) so retrieval can follow relationships instead of returning isolated chunks.
A GA4 to BigQuery pipeline surfacing zero-result searches and high-exit pages weekly. The corpus gets more answerable over time rather than just larger.
Five tool handlers exposing a documentation corpus, GA4 content gaps, and Jenkins CI status to Claude in one conversation. Runs with no credentials: mock fixtures, documented upgrade paths to pgvector and BigQuery.
View repository →The evolution of docs-mcp: adds a RAGAS-style evaluation loop measuring relevance, faithfulness, and recall, so retrieval quality is measured rather than assumed. Retrieval plus the loop that tells you it works.
View repository →A retrieval-augmented assistant grounding LLM answers in a documentation corpus through embeddings and semantic search, with retrieval guardrails on what it is allowed to answer from.
View repository →Turns a documentation set into a typed knowledge graph and retrieves across relationships rather than isolated chunks. Information architecture made executable. GraphRAG generation is in progress.
View repository →Writing standards as an automated check that runs on every change, callable over MCP. A style guide nobody reads becomes a gate nothing gets past.
View repository →My independent practice and published work on enterprise content retrievability: chunk integrity, knowledge graphs, and the shift from pageview KPIs to AI-readiness metrics.
Visit Knowlayer →Actian
Leading documentation engineering across the enterprise data products, and building the retrieval layer that makes the corpus queryable by AI systems.
Adobe
Content strategy and information architecture for Adobe HelpX, serving over a million users per quarter across developer and enterprise documentation.
ABB
Documentation for a full developer ecosystem: API guides, CLI references, and Swift and Kotlin mobile SDKs.
Aristocrat Technologies
Structured content foundations: DITA information modelling, XML publishing, and enterprise taxonomy.
From a standardised IA and navigation model at Actian
Metadata, URL structure, schema markup, and internal linking, in six months
Through the M&A documentation framework at Adobe
Jenkins pipeline with automated validation gates
REST and GraphQL reference with interactive testing
Adobe HelpX, the corpus the IA served
B.Tech Aeronautical Engineering, R. V. College of Engineering · Bengaluru, Karnataka
This page ships /llms.txt and schema.org Person markup, because a page arguing for machine-readable content should be machine-readable.
© Bipin Pandey