Memory OS: reasoning paradox and SGR in agent memory

Memory OS Architecture: From Reasoning Paradoxes to Hierarchical Memory Graphs

Powerful models like o4-mini with high reasoning_effort break schema strictness during structured object extraction. Instead of sticking to Pydantic schemas, the model starts 'improving' the structure: merging objects, tweaking JSON fields, and blurring concept types. Testing over 300 prompts revealed an inverse relationship: more reasoning leads to less determinism.

SGR (Schema-Guided Reasoning) with response_format and strict StrictBase (extra='forbid', no Optional/default) solves the issue partially, but the model still fights the constraints. Introducing a Semantic Mapper cut reasoning_effort by shifting discipline to data contracts, validation, and post-processing.

Results:

• Token costs dropped 48%;
• Extraction quality rose from 5.2 to 7.11;
• Full traceability via message_id.

The validator recursively checks the JSON schema: additionalProperties: false, all fields in required. This serves as a core signal for agent memory: LLM intelligence during extraction is for schema execution, not creativity.

Why Flat RAG Fails on Large Datasets and the Shift to Managed Memory

Flat RAG shines in demos but crumbles on big corpora like Jocker (3,253 conversations, 24,639 messages): no lineage, temporal semantics, or update rules. Retrieval pulls similar chunks without source ties, leading to Graph Sludge — thousands of nodes lacking granularity (26,000+ in Jocker pre-refactor).

Layer separation is essential:

• Retrieval: top-k similar chunks;
• Memory: knowledge objects, links, temporal changes, revisions, degradation.

Without this, the system devolves into a chunk index riddled with hallucinations on complex queries.

Core Ontology: ConceptHypothesis as a First-Class Citizen

We start with a core ontology featuring typed objects and relations. Ditching loose entities/summaries for ConceptHypothesis:

from pydantic import BaseModel, ConfigDict, Field
from typing import List, Literal, Optional
from datetime import datetime

class StrictBase(BaseModel):
    model_config = ConfigDict(extra="forbid")

class EvidenceSpan(StrictBase):
    source_id: str
    message_id: str
    text: str
    confidence: float

class ConceptRelation(StrictBase):
    target_id: str
    relation: Literal["causes", "conflicts_with", "supports", "instance_of", "enables"]
    confidence: float

class ConceptHypothesis(StrictBase):
    concept_id: str
    title: str
    type: Literal["behavioral_pattern", "project", "event", "tension", "goal", "belief", "entity"]
    description: str
    state: Literal["active", "weak", "merged", "split", "archived"]
    confidence: float
    aliases: List[str]
    evidence_spans: List[EvidenceSpan]
    relations: List[ConceptRelation]
    revision_count: int
    first_seen: Optional[datetime]
    last_seen: Optional[datetime]

This object carries evidence spans, relations, confidence, and revision_count. Temporal reasoning (events, contradictions) is first-class, not a chunk byproduct.

Data Contracts and Staging: SQLite as the Single Source of Truth

Data contracts lock in structure at pipeline boundaries. SQLite acts as the source of truth for messages (UUID, text, role, parent_id, timestamp, conversation_id). Neo4j graph: nodes like Conversation/Message/Segment/Concept/Topic, edges like HAS_MESSAGE/NEXT/CONTAINS_MESSAGE/EVIDENCE_FOR/BELONGS_TO.

The key is EVIDENCE_FOR: concepts tie directly to message-level evidence. Chronological NEXT/PREVIOUS skeleton speeds up inference without database sorting.

Staging database (separate SQLite) holds pipeline intermediate states for full auditability.

Concept Memory Loop and Topic Clustering

Concept Memory Loop: create/attach/refine/reject — the hypothesis management cycle. Topic clustering via UMAP + HDBSCAN + Optuna for weighted links prevents over-compression and concept explosion.

LongMemEval and Stateful Multi-Agent Inference

LongMemEval (100M+ tokens): 4 debug phases, Exploration vs Execution. Stateful multi-agent: Planner/Scout/Synthesizer for hierarchical retrieval.

Agentic Gardener, Invalidation, and MCP

Agentic Gardener — dedicated graph maintenance pipeline. Invalidation/repair for bi-temporal facts. MCP — sovereign memory interface.

Scale: 106.7M tokens, 632,868 semantic units, 2.4M edges.

Key Takeaways:

• High reasoning breaks SGR; use Semantic Mapper and low effort;
• Flat RAG → Graph Sludge; demand evidence spans and temporal ontology;
• ConceptHypothesis with Pydantic StrictBase — foundation for managed compression;
• SQLite as source of truth + Neo4j for lineage and chronology;
• Multi-agent + Gardener for 100M+ token scale.

— Editorial Team