Graph-Based Document Model Instead of Vector RAG: Experience with Regulatory Documents
Standard RAG — splitting text into chunks, vectorizing, and searching — fails with regulatory documents. Semantic search finds relevant clauses but ignores related subclauses, notes, or cross-references. The result: incomplete answers where the model draws conclusions from isolated fragments, losing essential context.
For questions involving chains of regulations, retrieval brings noise or misses key elements. The problem isn't the LLM, but the flat representation of documents, where the minimal storage unit doesn't match the unit of meaning.
Structured Model: From Text to Node Hierarchy
Documents are modeled as a tree of addressable elements:
- Root node — the entire document.
- Sections, clauses, subclauses.
- Tables and formulas as separate nodes.
- Connections between nodes.
- A separate layer for terms with definitions.
Indexing is based on normalized structure, not raw chunks after parsing. This preserves hierarchy and allows precise access to elements.
Context Grouping: Why One Node Isn't Enough
Atomic nodes (single clauses) lose local context. For a question like "minimum emergency exits," you need not just the main clause but:
- Adjacent subclauses at the same level.
- Notes.
- Related tables.
- Cross-references to other sections or documents.
Grouping occurs by common parent and nesting level, with a size limit. There's no universal chunk size: small groups lose meaning, large groups add noise to retrieval. Structural grouping improves answer robustness by including meaningful blocks in the index.
Multi-Mode Retrieval: Beyond Vector Search
Semantic search works for free-form queries, but regulatory documents require precision:
- Point search by clause number (e.g., 6.2.2).
- Table search (e.g., Table 7.1).
- Lexical and hybrid search.
- Terminology layer.
- Cross-reference search.
A single query triggers multiple strategies in parallel: point search by clause + keyword + terms. Results are combined in a context builder by a search agent.
Terminology Layer for Synonymy
Users and documents use different phrasing: official terms, abbreviations, scattered definitions. Terms are extracted separately:
- Extraction from the document.
- Linking to definitions.
- Separate indexing.
- Use as an additional signal in retrieval.
This connects queries by concepts to relevant fragments, improving recall.
Graph of Connections: Mandatory Context
Hierarchy doesn't guarantee pulling related regulations. A graph adds typed relationships:
mandatory— obligatory companion to a node.cross— cross-reference.- Document ↔ node.
- Node ↔ table.
- Between documents.
When a relevant hit occurs, retrieval forcibly adds related nodes, even if they aren't in the top-k by semantics. This prevents incomplete answers.
Practical System Improvements
The graph-based model makes answers robust:
- Precise references to clauses with context.
- Handling tables without losing descriptions.
- Distinguishing terms from similar phrases.
- Chains of regulations instead of isolated fragments.
The system stops being a chat — it becomes a tool for verifiable answers.
Key Points:
- Documents aren't flat text but an addressable hierarchy with a graph of connections.
- Retrieval is multi-layered: vector + point + terms + connections.
- Grouping by structure, not tokens, to preserve context.
- Mandatory connections are explicitly expressed in data.
- One question — multiple searches with result composition.
Lessons for Complex Domains
RAG quality in regulatory contexts is determined by data, not the model. Key takeaways:
- Structure documents as a graph of nodes.
- Chunking by hierarchy, with adaptive size.
- Compose retrieval strategies.
- Extract terms and mandatory connections.
This scales to other domains with rigid structure: code, specifications, contracts.
— Editorial Team
No comments yet.