New System Enhances Retrieval-Augmented Generation

A new method for constructing a deterministic, multi-tier retrieval-augmented generation system has been introduced, focusing on utilizing knowledge graphs and vector databases. This system aims to enhance the accuracy of information retrieval by addressing the limitations of traditional vector databases, which often struggle with atomic facts and strict entity relationships.

The new architecture employs a quad store backend to implement a knowledge graph, allowing for precise retrieval of atomic facts. This setup is supported by a vector database to handle broader, fuzzy context. Instead of using complex algorithmic routing to select the appropriate database, the system queries all databases simultaneously. The results are then combined using prompt-enforced fusion rules to ensure that the language model resolves conflicts deterministically, which is expected to reduce relationship hallucinations.

The retrieval architecture is structured in three tiers, enforcing a strict data hierarchy. The first two tiers focus on absolute truths and broader statistics, utilizing a lightweight in-memory knowledge graph known as a quad store. This type of graph shifts away from semantic embeddings to a strict schema, which aids in efficient data retrieval and prevents memory issues.

The third tier incorporates a standard dense vector database, specifically ChromaDB, which stores text chunks that may not be captured by the knowledge graph. This multi-tiered approach allows for simultaneous querying of deterministic graphs and semantic vectors, bridging the gap through entity extraction.

By using named entity recognition techniques, the system can extract entities from user prompts and query the respective stores in parallel. This process separates retrieved information into distinct streams, facilitating clearer data management.

To resolve conflicts, the system employs a simpler method of prompt-enforced rules, allowing the language model to follow a structured hierarchy when outputting responses. This innovative approach is intended to combat the issue of hallucinations in generated data, providing a more reliable retrieval-augmented generation system for practical applications.