PageIndex in RAG: replacement for vector search

PageIndex as an Alternative to Vector Search in RAG: Analysis and Local Setup

PageIndex proposes replacing vector search in RAG with a hierarchical document structure. The document is segmented into pages, and an LLM builds a TOC tree with node summaries. Search queries are directed to the LLM to select relevant nodes, followed by extracted pages as context. No chunking, embeddings, or vector databases are required.

Scaling Limitations

PageIndex is optimized for single documents. For collections, the following are suggested:

• Metadata filtering (in development, similar to vector stores).
• Search via generated descriptions (for small volumes).
• Semantic search (reverts to vector embeddings).

The TOC with summaries compresses context by 4–5 times. Alternatives like TF-IDF or BM25 require indexing. Storing TOC for thousands of documents remains unresolved. Here, the LLM replaces similarity search with relevance search, akin to navigating complex text via a table of contents.

Claimed accuracy of 98.7% on FinanceBench (vs. ~90% for vector search) is achieved through reasoning. Cost is tens of times higher. Tested on gpt-4o and gpt-5.4.

Experiments with Local Models

Tested on an academic article about Cyberpunk 2077 (PDF, ~17k tokens). The Markdown pipeline is similar to MarkdownTextSplitter from LangChain—splitting by headings.

The PDF pipeline extracts headings automatically but with issues:

• JSON output errors—text processing (removing bibliographies, quotes).
• Accuracy threshold of 60%—lowered via env ACCURACY_THRESHOLD.
• Low accuracy due to LLM matching nodes and fragments—increased MAX_TOKENS from 20k.

Models:

• gemma3:27b—hallucinates TOC (accuracy <60%).
• gpt-oss:20b—accuracy 57–63%, stable.
• qwen3:14b—accuracy 69.57%, but with JSON errors.

Time on 16GB GPU: ~30 min (error correction dominates).

Example TOC (gpt-oss:20b, accuracy 0.63):

{
  "doc_name": "2077.pdf",
  "structure": [
    {
      "title": "Introduction",
      "node_id": "0000",
      "start_index": 1,
      "end_index": 3,
      "summary": "The paper examines how the video game Cyberpunk 20...",
      "text": "A Cyberpunk 2077 perspective on the prediction and..."
    },
    {
      "title": "Literature review",
      "node_id": "0001",
      "start_index": 3,
      "end_index": 3,
      "summary": "The partial document presents a literature review ...",
      "text": "contextualizing Cyberpunk 2077 within the broader ...",
      "nodes": [
        // ... subnodes
      ]
    }
    // ... remaining nodes
  ]
}

Configuration and Local Setup

Installation: liteLLM==1.82.0 (newer versions are vulnerable). Do not use pip install pageindex—that's for the API.

Notebooks: MinimalPageindexLocalPDF.ipynb, MinimalPageindexLocalMD.ipynb.

Modifications to utils.py: OLLAMA_HOST, OLLAMA_TIMEOUT.

page_index.py: ACCURACY_THRESHOLD, MAX_TOKENS.

Process:

• Load document (PDF/MD).
• LLM builds TOC (find_toc_pages).
• Segmentation into pages/lines.
• Building node tree with summaries.
• Accuracy correction.
• Search: LLM selects nodes by query → page extraction → final RAG.

PageIndex integrates LLM into retrieval for traceable search with page/section references. Suitable for complex documents but resource-intensive for production.

Key Takeaways

• Scaling: No ready solution for large collections without vector search.
• Accuracy: Model-dependent; local (qwen3:14b) achieves 69% but with errors.
• Cost: +8–9% accuracy for x10–xN resources compared to vector RAG.
• Formats: PDF with auto-detection of headings; MD via markdown structure.
• Modifications: Env variables needed for thresholds and tokens on weaker hardware.

— Editorial Team