RAG system for 1C: MCP server architecture

Multi-Level RAG Architecture for Searching 1C Documentation on MCP Servers

MCP servers documents1c and metadata1c implement a multi-level RAG system for precise search across 1C documentation and configurations. Each ranking layer addresses specific challenges: from chunk context to combining vector and full-text search. The system uses PostgreSQL with pgvector, Redis for caching, and custom tokenization for 1C terminology.

Why We Moved Away from Vector Databases with Hierarchies

Standard vector stores like Qdrant or Weaviate struggle with the dynamic nature of 1C documentation. Documents are not self-contained without context: an article titled "Setup" requires specifying the section, and an object like Reference.Employees needs descriptions of its attributes and modules.

Documentation is updated frequently—new BSP releases, platform methods. Rebuilding the tree in a vector database is costly. The solution: PostgreSQL with pgvector + heading_path embedded in each chunk. Embeddings are generated for heading_path + text, only the chunk text is stored.

Savings: A embedding halfvec(1024) column in the article_chunks table instead of a separate vector database.

Layer 1: Chunks with Heading Context

Splitting markdown into chunks considers structure: YAML is removed, division by headers, code blocks are not broken, overlap of 220 characters.

def split_markdown_into_chunks(
    text: str,
    max_len: int = 800,
    overlap: int = 220,
    include_heading_path: str = "full_path",
) -> list[str]:
    body = _strip_frontmatter(text)
    sections = _split_by_headers(body)
    chunks: list[str] = []

    for section, heading_path, current_heading in sections:
        context_heading = " > ".join(heading_path)
        section_with_context = f"{context_heading}\n\n{section}"

        if len(section_with_context) <= max_len:
            chunks.append(section_with_context)
        else:
            sub_chunks = _split_section_by_paragraphs(section_with_context, max_len)
            chunks.extend(sub_chunks)

    return _add_overlap(chunks, overlap)

Example chunk:

Payroll and HR Management > Payroll Calculation > Setting Up Personal Income Tax

To correctly calculate personal income tax, you must specify the tax rate...

In the database, meta stores heading_path separately for filters and links. Embedding: heading_path + chunk_text.

Layer 2: Embedding Cache in Redis

Generating embeddings (text-embedding-qwen3-embedding-4b) takes 50–200 ms. For repeated queries—Redis with TTL, key SHA256 of model:dimensions:text.

async def embed_text(text: str) -> list[float]:
    model = _get_embed_model_name()
    dims = _get_embed_dimensions()

    if _cache_enabled():
        cached = await cache_get(text, model, dims)
        if cached is not None:
            return cached

    response = await to_thread(client.embeddings.create, model=model, input=text)
    vector = response.data[0].embedding

    if _cache_enabled():
        await cache_set(text, model, dims, vector, _cache_ttl())

    return vector

The cache is invalidated automatically when the model changes.

Layer 3: Vector Search with halfvec

Cosine distance via <=> in pgvector, halfvec(1024) type saves memory.

SELECT
    ac.id, ac.article_id, ac.chunk_text, ac.meta,
    ac.embedding::halfvec(1024) <=> CAST(:query_embedding AS halfvec(1024)) AS distance
FROM article_chunks ac
INNER JOIN articles a ON ac.article_id = a.id
WHERE a.job_id IN (SELECT id FROM jobs WHERE load_mode IN ('hbk', 'docs'))
AND a.destination = :destination
ORDER BY distance
LIMIT :limit;

Score = 1 - distance.

Layer 4: Merging Rankings with RRF

Vector search is weak for precise terms like &OnClientOnServer. RRF fuses vector and FTS lists:

@staticmethod
def _rrf_merge(vector_results, fts_results, top_k: int, k: int = 60) -> list[dict]:
    scores: dict[int, float] = {}
    items: dict[int, dict] = {}

    for rank, item in enumerate(vector_results, start=1):
        chunk_id = item["id"]
        scores[chunk_id] = scores.get(chunk_id, 0.0) + 1.0 / (k + rank)
        items[chunk_id] = item

    for rank, item in enumerate(fts_results, start=1):
        chunk_id = item["id"]
        scores[chunk_id] = scores.get(chunk_id, 0.0) + 1.0 / (k + rank)
        if chunk_id not in items:
            items[chunk_id] = item

    sorted_ids = sorted(scores, key=lambda cid: scores[cid], reverse=True)
    # Normalization and return top_k

HBK boost: For queries without a filter, 1C built-in language chunks (load_mode='hbk') are added to the beginning of the list.

• Top HBK chunks by query_emb
• Duplicates are excluded
• Placed before main results

Layer 5: BM25 Reranking with 1C Tokenization

For 50–100 candidates from RRF, BM25 is applied with weights (vector 0.6, bm25 0.4). Effective for:

• Specific terms: CommonModule, InformationRegister, &OnServer
• Object names: Reference.Nomenclature
• Short technical queries

Custom tokenization:

GUID_RE = re.compile(r"\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b", re.I)
VERSION_RE = re.compile(r"\bv?8\.\d+(?:\.\d+){1,3}\b", re.I)
DIRECTIVE_RE = re.compile(r"[&#][And-IA-Z][And-YaYoA-Za-z0-9_]*", re.I)

def _tokenize(self, text: str, expand_query: bool = False) -> list[str]:
    text = _norm_text(text)
    out = []

    for g in GUID_RE.findall(text): out.append(g.lower())
    for v in VERSION_RE.findall(text): out.append(v.lower())
    for d in DIRECTIVE_RE.findall(text): out.append(d.lower())

    for t in TOKEN_RE.findall(text):
        if t in RU_STOP: continue
        for p in _split_identifier(t):
            out.append(p.lower())
    return out

Splits CamelCase, snake_case, extracts directives and GUIDs.

Key Takeaways

• heading_path in chunks provides context without a document tree
• Redis cache reduces embedding latency from 100+ ms to 0 ms
• RRF combines semantics and exact search without normalization
• BM25 with 1C tokenization boosts relevant technical terms
• halfvec(1024) saves memory in pgvector while maintaining cosine accuracy

— Editorial Team