JuliaLM: RAG for knowledge bases without ChatGPT

JuliaLM: A Self-Contained RAG System for Personal Knowledge Bases

Developers often face the challenge of processing large volumes of documents: PDFs, YouTube lectures, web pages. Standard chatbots like ChatGPT lose context when working with multiple sources, and off-the-shelf RAG frameworks struggle with scanned documents and videos. JuliaLM solves this with a custom pipeline: one-time source loading, multi-strategy search, and context budgeting. The system is built on Nuxt.js, FastAPI, and SurrealDB, providing vector search, chat, and flashcard generation.

Tech Stack Architecture and SurrealDB Choice

The Nuxt.js frontend communicates with the backend via REST and SSE. FastAPI orchestrates AI with LangGraph, manages a task queue, and integrates with SurrealDB. The database combines relational tables, graph connections, and vector search in a single service, replacing Postgres + Redis + Pinecone.

Frontend (Nuxt.js)
    ↕ REST + SSE
Backend (FastAPI)
    ├── LangGraph (AI orchestration)
    ├── Background commands (task queue)
    ├── SurrealDB (data + vectors + graph)
    ├── Gotenberg (document conversion)
    └── MiniMax / Embedding API (LLM + vectorization)

The task queue is implemented on top of SurrealDB with retries (up to 5 attempts) and exponential backoff. For lightweight sources—synchronous processing (5–10 seconds); for heavy PDFs—asynchronous with status polling.

Cascading Source Parsing

Document processing uses fallbacks for reliability:

• PDF and office files: Docling for Markdown with structure preservation (headings, tables). Fallback—Gotenberg based on LibreOffice for PPTX/XLSX.
• YouTube: 1) youtube-transcript-api (subtitles by language: RU→EN→ES→PT); 2) pytubefix (Android client); 3) Firecrawl/Jina.
• Web pages: Playwright (JS rendering, UA/viewport randomization) + readability; fallbacks—Jina Reader, HTTP.
• Images: Vision model for handwritten text OCR.

This ensures text extraction even with YouTube API failures or CAPTCHAs.

Vectorization and Multi-Strategy Search

Text is split into chunks of ~500 tokens (15% overlap) with prioritized separators: double line breaks → single line breaks → periods. An empirical balance between accuracy and context for scientific texts.

Source search combines four strategies in parallel:

• Vector search—semantic via embeddings (cosine ≥ 0.15).
• Text search—BM25 over full text.
• Title search—over titles.
• Insight search—over summaries and insights.

Deduplication: score +0.05 per match. Top 5 results.

Context Budgeting for Long Documents

Overall limit—300,000 characters (75K tokens). Per-source: min 5,000, max 40,000.

• Short source: entirely.
• Long + known query: insights + relevant chunks (internal vector search).
• Long + unknown query: insights + beginning of text.

This approach prevents context overflow when working with books.

Flashcards with FSRS

Generation of "question-answer" pairs in JSON with validation. Spaced repetition via FSRS: stability, difficulty, state parameters (new/learning/review). Rating 1–4 recalculates schedule. Integration into the notebook eliminates export to Anki.

Advanced Database Search

Two basic modes: BM25 and vector search. "Ask the database"—LangGraph graph:

• Generation of up to 5 subqueries with instructions.
• Parallel search (up to 10 results/query).
• Synthesis of intermediate answers.

Key Takeaways

• Cascading parsing guarantees text extraction from any source, including YouTube without subtitles.
• Four search strategies improve recall by 20–30% compared to single vector search.
• Context budget allows processing corpora >1000 pages without losing relevance.
• FSRS integration simplifies learning by automating spaced repetition.
• SurrealDB simplifies deployment: one container instead of three databases.

— Editorial Team