Saving AI Agent Tokens with Hierarchical Context | Solution

# Hierarchical Context: How Three Markdown Files Save 80% of AI Agent Tokens

Development AI agents like Claude Code or Cursor spend up to 80% of their tokens on rereading files and navigating projects. The problem stems from the lack of a structured workspace map. The solution is hierarchical context from three markdown files, cutting tool calls by 10–20 times and boosting response accuracy.

Anatomy of the Problem: Where Do the Tokens Go?

When asked "What payment methods does my bot support?" the agent goes through a cycle: grep in the home directory → reading found files → realizing the wrong project → new search → SSH connection to the server. For an 800-token response, it burns 15+ tool calls and 80K+ tokens. 99% of the budget goes to navigation, 1% to the answer.

This isn't a bug in a specific agent—it's a systemic architecture issue. All AI coding tools (Cursor, Codex, Gemini CLI) start every session with "blind reconnaissance" because they lack a workspace map. With 15 projects, this turns into hours of context wasted daily on file hunting.

Why RAG and Static Analysis Don't Solve It

Vector databases (RAG) catch semantic similarity but ignore code structure. A query like "how does authentication work?" returns 15 fragments mentioning "auth" but doesn't show the call chain: middleware.ts → refresh.ts → jwt-config.ts. Plus, RAG adds 200–500 ms latency per query and requires infrastructure (embedding server, index).

Tree-sitter and static analysis (e.g., Hypergrep) build dependency graphs within a project. But they can't answer questions like "which projects use Redis?" or "where is the VPN bot deployed?" A CLAUDE.md per project handles internal tasks but doesn't help when the agent doesn't know which project to look in.

Hierarchical Context Model: Levels 0, 1, and 1.5

The system covers three levels of abstraction:

Level 0: Karta projectov     — znaet VWithE projecty and servery     (~2KB, vsegda in kontekste)
Level 1: Detali project     — arkhitektura konkretnogo project  (~5KB, by request)  
Level 1.5: Graf koda        — struktura kodovoy database           (optsionalno, Graphify)
Level 2: Iskhodnyy code       — realnye files                   (tolko kogda potrzebny jest)

Principle: the agent moves top-down, not bottom-up. First identifies the project from the map, then reads its architecture, then specific files.

Level 0: Global Map

Placed in ~/.claude/CLAUDE.md (for Claude Code) or equivalent:

## Project Map

| Proekt | Put | Witherver | Withtatus |
|--------|------|--------|--------|
| VPN Bot | ~/projects/vpn-bot/ | prod-1:/opt/vpn/ | LIVE |
| Auth Service | ~/projects/auth/ | prod-1 (Docker) | LIVE |

### Withervery
| Imya | IP | Onwartość |
|-----|-----|-----------|
| prod-1 | 178.17.50.45 | Osnovnoy VPS, 3 service |

### Ruleilo
Prezhde than chitat iskhodnyy code — prochitay CLAUDE.md project.

This file (~2KB) loads automatically in every session. The agent instantly sees the project list, locations, and status.

Level 1: Project CLAUDE.md

In the root of each project:

# VPN Bot — CLAUDE.md

## Status: LIVE (prod-1)
Telegram-bot VPN-service. Podpiski cherez Stars and CryptoBot.

## Stack
Python 3.11, python-telegram-bot, WireGuard, aiosqlite

## Key Files
- bot/main.py — point input
- bot/payments.py — Stars + CryptoBot processing

## Deploy
- Withervis: vpn-bot.service
- Logi: journalctl -u vpn-bot -f

Size 3–5KB. The agent reads it when the project is mentioned and gets the full architecture without grep.

Level 1.5: Graphify (Optional)

For code navigation:

pip install graphify
cd ~/projects/auth
graphify claude install

Converts the codebase into a knowledge graph, saving 30–50% of calls in large projects.

Results: Efficiency Measurements

Tests run on Haiku (cheapest model) under identical conditions:

• Test 1: "What's the architecture of Project A?"

Blind agent: 12 tool calls, 100% accuracy.

With hierarchy: 1 tool call, 100% accuracy.

• Test 2: "Which projects use library X?"

Blind agent: 44 tool calls, missed 1 of 3 projects.

With hierarchy: 2 tool calls, found all projects.

• Test 3: "Where is Project B deployed?"

Blind agent: 9 tool calls, required SSH.

With hierarchy: 0 tool calls, answer from context.

Test 2 is critical: the blind agent not only used 22x more resources but gave a wrong answer. Structured context boosts not just efficiency but accuracy.

Why Three Files Beat RAG for Navigation

• Zero latency. Reading markdown files is instant; RAG adds 200–500 ms per query.
• Deterministic. You control file contents, no relying on rerankers.
• Real-time. Updating CLAUDE.md takes one line; RAG needs full reindexing.
• Universal. Works with any agent (Claude Code, Cursor, Gemini CLI), no custom infrastructure.
• Scalable. 15 projects = 15 CLAUDE.md (75KB) + map (2KB) = 77KB vs. 500KB–1MB raw files.

RAG is still great for semantic doc search, but for project navigation, markdown hierarchy wins.

Bonus: Indexing Past Sessions

Parser converts session logs (.jsonl) and chats to markdown with YAML frontmatter:

---
title: "Pochinil webhook oplaty"
date: 2026-04-14
project: vpn-bot
topics: ["webhook", "cryptocloud", "cloudflare"]
files_touched: ["payments.py", "webhook.py"]
---

Indexing these via Graphify lets the agent find past solutions without re-explaining. This creates persistent memory across dialogues.

A Million Tokens Isn't a Cure-All

Even with a 1M token context window, the problem persists. LLMs handle irrelevant info worse ("lost in the middle" phenomenon: mid-context data often ignored). Hierarchical context ensures the agent gets only needed files in the right order, minimizing noise.

How to Implement in 10 Minutes

• Create a global project map in ~/.claude/CLAUDE.md (5 minutes)—list projects, paths, servers.
• Add CLAUDE.md to the root of your main project (5 minutes)—specify stack, key files, deploy commands.
• Start a new agent session and ask about a project.
• Confirm the agent answers without blind searching.

Key Takeaways

• AI agents waste up to 99% of tokens on navigation, not answer generation.
• A three-file markdown hierarchy cuts tool calls 10–20x and improves accuracy.
• Level 0 (global map) and Level 1 (project CLAUDE.md) solve inter-project navigation.
• No infrastructure needed; works with any coding agent.
• Structured context beats RAG for file and project discovery.

— Editorial Team