Evolution of RAG: genomes and cache of MCP servers

Evolution of RAG Pipelines via Genetic Algorithms: Implementation on MCP Servers

RAG systems with static prompts lose effectiveness as queries change. Implementing controlled evolution uses LLMs to generate variants of "genomes"—pipeline configurations. A judge-evaluator tests them on a sample of queries, with the best candidates moving to pending_approval status for manual confirmation by an administrator. Applied to MCP servers documents1c (RAG for 1C documentation) with three layers: postprocess, routing, cache_policy.

Evolution is not autonomous but operates within the admin panel's laboratory loop. The focus is on improving system prompts for post-processing and caching policies in the 1C documentation domain, with potential expansion to finance, accounting, and legal fields.

Architecture of the Three-Layer Pipeline

The docsearch pipeline processes queries sequentially:

• Semantic chunk search (hybrid if needed).
• Routing: A classifier determines query_type (factual_lookup, explanation, bsl_help) and activates a profile with system_prompt, depth_budget, format.
• Postprocess: An LLM assembles the response according to a contract (answer, key_points, sources, warnings) based on the active genome.
• Cache_policy: Admission control before writing to cache (min_sources_count, block_if_warnings).

| Layer | Evolution Object | Impact |

|------|-----------------|---------|

| postprocess | system_prompt, chunk_count, max_tokens | Quality of text response |

| routing | classifier_prompt, profiles | Adaptation to query type |

| cache_policy | Admission thresholds (min_supportedness) | Cache filtering |

Postprocess and routing modify responses in real-time, while cache_policy only affects storage.

Data Model for Genomes and Evaluations

Genomes are stored in the evolution_genome table as JSONB with fields: layer, status (candidate → evaluated → pending_approval → active), genome, score.

class EvolutionGenome(Base):
    layer = Column(String(32), nullable=False, index=True)
    status = Column(String(32), nullable=False, default="candidate", index=True)
    genome = Column(JSONB, nullable=False)
    score = Column(Float, nullable=True)
    eval_results = relationship("EvolutionEvalResult", back_populates="genome")

Each eval_result records query, response (up to 2000 characters), judge_score (0–10), judge_reasoning from the LLM judge.

class EvolutionEvalResult(Base):
    genome_id = Column(Integer, ForeignKey("evolution_genome.id"), nullable=False)
    query = Column(Text, nullable=False)
    response = Column(Text, nullable=True)
    judge_score = Column(Float, nullable=True)
    judge_reasoning = Column(Text, nullable=True)

For routing, an eval_seed_prompt is generated—a test session of agent+MCP.

Evolution Cycle: From Mutations to Production

Cycle: generate_mutations() → eval_genome() → promote_best() → manual approve.

• Generation: Base genome (active or DEFAULT_GENOMES) + N query-response pairs from the MCP documents1c cache. LLM generates N JSON variants {"genome": {...}, "notes": "..."}.
• Evaluation: Judge (rag_postprocess_model) checks against criteria: completeness, accuracy, sources, conciseness. Average score across the sample.
• Promotion: Best candidate moves to pending_approval.
• Activation: Admin confirms in the UI.

Mutation Generation by Layer

Postprocess: Mutations in system_prompt (instructions for assembling responses, considering client/server context), system_prompt_reduce (merging without duplicates), chunk_count (15–20), max_tokens (4096–8192), citation_policy (always/inline), verbosity (adaptive/comprehensive).

Example postprocess genome:

{
  "verbosity": "adaptive",
  "max_tokens": 4096,
  "chunk_count": 15,
  "system_prompt": "You are a 1C expert. When answering, always consider the execution context (thin client, web client, server, mobile app). Respond in Russian. Cite sources.",
  "citation_policy": "always",
  "system_prompt_reduce": "Combine responses, clearly separating recommendations for different execution contexts. Remove duplication."
}

Routing: classifier_prompt + profiles by query_type (system_prompt, depth_budget, format).

Cache_policy: min_supportedness, require_sources, min_sources_count, block_if_warnings, confidence_key_points_min.

Real dialogues are added to the generator context to analyze weaknesses.

Evaluation and Judge Metrics

The judge evaluates based on 4 metrics (0–10):

• Completeness: Coverage of the query.
• Accuracy: Alignment with chunks.
• Sources: Presence/quality of citations.
• Conciseness: Absence of fluff.

Average score determines promotion. Eval_judge_response stores the raw LLM output.

Key Points

• Layer evolution is independent: postprocess improves text, cache_policy enhances cache quality.
• Manual approval prevents regressions in production.
• Context from real cache increases mutation relevance.
• SQLAlchemy models support cascade delete for eval_results.
• Scalable to multi-domain (1C + finance + coding).

Scaling and Best Practices

Expansion: Add layers for domains (finance: system_prompt considering reference data; coding: BSL syntax). Test on 10–20 queries from the cache. Monitor scores >8.5 for approval. Avoid overfitting—limit mutations per cycle.

— Editorial Team