GraphRAG and Ollama: Knowledge Graphs for Advanced Response Generation

GraphRAG and Ollama: Enhanced Response Generation with Knowledge Graphs

Modern enterprise RAG (Retrieval-Augmented Generation) systems are continuously evolving, striving to overcome the limitations of traditional vector search. Microsoft GraphRAG presents a promising solution that leverages knowledge graphs for a deeper understanding of context and information retrieval. In this article, we will explore practical experience with GraphRAG, its integration with local language models via Ollama, and demonstrate its capabilities using a text analysis example set in the cyberpunk world. The goal is to show how graph-based approaches can significantly improve the accuracy and relevance of responses in complex information systems, offering a new level of semantic search and data analysis.

From Vector Search to Knowledge Graphs

At the core of many RAG systems lies vector search, which, despite its effectiveness, has several limitations, especially when dealing with complex queries and ambiguous entities. GraphRAG offers an alternative approach by constructing a comprehensive knowledge graph based on source documents. This graph consists of entities (objects, persons, locations) and relationships between them, enabling the system to capture not only the presence of keywords but also their semantic relationships within the context. This approach significantly enhances the accuracy and depth of retrieved information.

The process of building a knowledge graph in GraphRAG involves extracting entities and relationships from text. Specialized tools, such as Gephi with the Leiden plugin, can be used to visualize and analyze the structure of such a graph. This not only allows for visualizing interconnections but also for discovering hidden communities or clusters of entities that GraphRAG automatically identifies and describes. It's important to note that during graph construction, challenges may arise with entity resolution when similar but not identical terms (e.g., "Molly Millions" and "Molly") are not automatically merged, requiring further algorithm refinement or manual correction.

GraphRAG also generates community descriptions, which are stored in community_reports.parquet files. These descriptions are critically important for global search, as they enable LLMs to gain a high-level understanding of thematic clusters within the data. This approach significantly surpasses simple text chunk searching, providing more structured and contextually enriched material for response generation.

Query Strategies in GraphRAG

GraphRAG offers several search strategies, each optimized for different query types. The system's flexibility is achieved through the ability to fine-tune prompts, for example, in local_search_system_prompt.txt and global_search_reduce_system_prompt.txt files. This allows for adapting the system's behavior to specific tasks and languages, ensuring response relevance even when questions are formulated in different languages, provided LLM and embedding support.

Global Search

Global search is designed to answer broad questions requiring an understanding of the entire dataset. It operates in a MapReduce fashion, reviewing community descriptions extracted from the knowledge graph. This allows for generalized, yet deep and contextually complete responses.

Example query:

graphrag query "What are the top cyberpunk elements in this story?"

The example response demonstrates how the system aggregates information from various parts of the graph, referencing specific elements using human_readable_id. This allows the user to delve deeper into the source data if needed, using these identifiers:

• Integration of Technology and Biology: The use of Lewis's neuroperturbator and Yakuza's prosthetic thumb demonstrates how technology penetrates the body and influences control and interaction with the environment. Such elements are a vivid manifestation of the cyberpunk theme, where the boundaries between organic and synthetic blur [Data: Reports (5, 7, 12, 2, +more)].
• Dystopian Urban Landscapes: The narrative describes dystopian urban spaces, such as Nighttown and Drome, which symbolize ruin, lawlessness, and underground crime networks. [Data: Reports (1, 8, 2, +more)].
• Corporate Competition and Illicit Deals: Elements of corporate espionage and illegal technological transactions, such as the microwave equipment deal between the Aryan reggae band and the Yakuza program, highlight the theme of corporate control and high-stakes conflicts. [Data: Reports (0, 12, 2, +more)].
• Contrast Between Low and High Technology: The symbolism of low technology, associated with LO TEK and their use of products like Yiheyuan filters, contrasts with advanced cybernetic enhancements. [Data: Reports (9, 2, +more)].
• Cyber-Organic Beings and Technical Skills: Characters such as Jones — a cyber-dolphin with a military past — and Molly Millions with her combat skills in using control modules, demonstrate the synthesis of organic and synthetic. [Data: Reports (11, 7, 2, +more)].

Local Search

Local search, in contrast to global search, focuses on specific interrelationships between entities. It combines relevant data from the knowledge graph with fragments of source documents, making it ideal for detailed questions about specific objects or persons. This mode is particularly useful when a deep understanding of the roles and relationships of individual elements within the system is required.

Example query:

graphrag query "Who is JONES and what are his main relationships?" -m local

The example response shows the detailed relationships:

Jones is a character described as a cybernetic dolphin with advanced sensory capabilities, associated with the object "Funland". His primary functions include managing a freshwater reservoir and collaborating with "Calm". Jones plays a key role in data retrieval operations, especially in collaboration with Molly and Johnny, where his unique physiology allows access to data from Johnny's chip [Data: Entities (28, 33); Relationships (44, 51, +more)].

Jones's main relationships include:

• With Molly Millions: Participation in the operation to extract data from Johnny's chip, demonstrating his technical abilities [Data: Entities (33); Relationships (44, 51, +more)].
• With Johnny: Interaction within the data extraction operation, where Jones uses his sensors to access information [Data: Entities (33); Relationships (44, 51, +more)].
• With "Squid" (Superconducting Quantum Interference Devices): Use of the "Squid" device to read traces, which is related to military technologies [Data: Entities (19); Relationships (25, +more)].
• With Yakuza: Indirect connection through the Yakuza's concern that "Squid" might extract traces of their program from Johnny's mind [Data: Relationships (25); Entities (19)].

The Role of Ollama and Other Search Modes

Using Ollama in conjunction with GraphRAG enables local deployment of language models, which is particularly relevant for projects with limited computational resources or strict data privacy requirements. Ollama demonstrates the ability to work effectively even with 4B models, providing RAG system functionality without the need for costly cloud-based LLMs. This significantly lowers the barrier to entry for experimenting with and deploying graph-based RAG systems in an enterprise environment.

In addition to global and local search, GraphRAG also offers BASIC and DRIFT modes:

• BASIC Search: This represents a standard vector search across document chunks. It serves as a baseline for comparing performance and response quality with more complex graph-based approaches.
• DRIFT Search: This feature is positioned as a key advantage of GraphRAG, resembling Query Expansion in traditional RAG systems. However, it's important to note that DRIFT Search is resource-intensive and takes significantly longer to execute, which can be critical for systems with high response time requirements.

Key Takeaways

• Overcoming Vector Search Limitations: GraphRAG uses knowledge graphs for deep semantic analysis and information retrieval, surpassing traditional vector methods in accuracy and context.
• Local Deployment with Ollama: Integration with Ollama allows GraphRAG to effectively use local language models, reducing computational costs and enhancing data privacy.
• Diverse Search Strategies: The system offers global, local, BASIC, and DRIFT search modes, each optimized for specific query types, from broad to detailed.
• Flexible Configuration and Adaptation: The ability to modify prompts and analyze generated knowledge graphs (graph.graphml, community_reports.parquet) provides a high degree of control and adaptation for specific project needs.
• Enhanced Response Relevance: By structuring knowledge as a graph and applying advanced query algorithms, GraphRAG significantly improves the quality and relevance of generated responses, making them more informative and accurate.

— Editorial Team