What is grounding in RAG?

Grounding (G) — a multiplier that checks the accuracy of links to document pages. Low G nullifies even a perfect answer.

Why is BM25 worse than semantic search?

BM25 relies on keywords, ignoring semantics, which leads to low grounding (0.390 vs 0.901 in hybrid).

Is Mac M3 Ultra suitable for RAG?

Yes for embeddings and parsing, but prefill-heavy tasks (context generation) require cloud due to delays of 15-20 sec/chunk.

RAG optimization for legal tasks

Optimizing RAG for Legal Documents: Lessons from the Agentic RAG Legal Challenge

The development team tested RAG on a corpus of legal documents from the Agentic RAG Legal Challenge. The primary focus was improving grounding and citation accuracy. Documents from DIFC (court rulings, laws, regulatory acts) were parsed from PDFs into Markdown using visual page rendering to images, followed by processing with Qwen 3.5-9b on a Mac Studio M3 Ultra.

Semantic chunking powered by FRIDA (dense) and GTE (sparse) embeddings split 30 documents into ~2,000 chunks. Each chunk includes:

text: a complete idea or fact
path: source document identifier
context: description of the chunk’s surrounding content within the document

Context is generated via LLM (Qwen), feeding both the chunk and the full document. This compensates for lost coherence during splitting. Embeddings are stored in Qdrant. Indexing 30 documents took ~2 hours, with the bottleneck being prefill latency during context generation (15–20 seconds per chunk locally).

Response Generation Pipeline

For each query:

Intent: LLM generates alternative search queries.
Select: Vectorized via FRIDA/GTE, searched in Qdrant using RRF, returning top-N chunks. LLM re-ranks and filters relevant ones.
Answer: Filtered chunks are fed to the LLM with instructions to respond and cite specific chunk numbers for grounding.

Competition metrics: Total = (0.7 × S_det + 0.3 × S_asst) × G × T × F. Key factor G (grounding) validates reference to page numbers. S_det measures precision of deterministic answers (boolean, date, name, number). S_asst evaluates free-text quality via LLM-based scoring.

Experiment Results and Benchmarks

15 warm-up runs (30 docs, 100 questions) showed progression from baseline RAG to optimized versions. Key iterations:

|--------|--------|---------|---|--------|

| v3 (GPT-5.4) | 0.843 | 0.667 | 0.451 | 0.362 |

| v5 (CAG Qwen 1M) | 0.971 | 0.553 | 0.563 | 0.403 |

| v6 (BM25) | 0.943 | 0.540 | 0.390 | 0.269 |

| v7 (BM25 + semantic RRF) | 0.871 | 0.567 | 0.717 | 0.468 |

| v11 (MORAG v3) | 1.000 | 0.680 | 0.757 | 0.664 |

| v15 (final) | 0.986 | 0.733 | 0.901 | 0.780 |

G improved from 0.45 to 0.90—this drove success. Pure BM25 performed worst (Total: 0.269). CAG (full context prompt with 350K tokens in Qwen 3.5-Flash) excelled in S_det (0.971) but failed grounding.

OpenAI’s built-in RAG on PDF delivered 0.362—acceptable accuracy, poor citation.

Approach Comparison: RAG vs Alternatives

BM25: Fast keyword search (TF-IDF), but ignores semantics—low G (0.390).
CAG: Full context in prompt (1M tokens), high S_det, but grounding suffers without explicit references.
Hybrid RAG (MORAG): FRIDA/GTE + RRF + LLM re-ranking + question-type-aware reasoning. Achieved 1.000 S_det, 0.901 G.

Agentic element: after selection, the model autonomously decides next steps (reasoning on/off, pinning maps).

MORAG (github.com/catonmoon/morag) outperformed OpenAI/Grok’s built-in RAG across all metrics.

Mac Studio M3 Ultra handled indexing efficiently (native embeddings), but struggled with prefill-heavy tasks—switched to OpenRouter.

Key Takeaways

Grounding (G) is the dominant factor: even perfect answers fail without precise page citations.
Hybrid retrieval (dense + sparse + RRF) + LLM re-ranking dramatically boosts chunk relevance.
Semantic chunking with context generation preserves coherence without the overhead of large LLMs.
Question type dictates mode: reasoning for free-text, direct for deterministic responses.

— Editorial Team

RAG optimization for legal tasks