Programming With LightRAG Core

If you want to integrate LightRAG into your project, we recommend using the REST API provided by the LightRAG Server. LightRAG Core is intended for embedded applications or researchers conducting studies and evaluations.

A Simple Program

import os
import asyncio
from lightrag import LightRAG, QueryParam
from lightrag.llm.openai import gpt_4o_mini_complete, gpt_4o_complete, openai_embed
from lightrag.utils import setup_logger

setup_logger("lightrag", level="INFO")

WORKING_DIR = "./rag_storage"
if not os.path.exists(WORKING_DIR):
    os.mkdir(WORKING_DIR)

async def initialize_rag():
    rag = LightRAG(
        working_dir=WORKING_DIR,
        embedding_func=openai_embed,
        llm_model_func=gpt_4o_mini_complete,
    )
    # IMPORTANT: Both initialization calls are required!
    await rag.initialize_storages()  # Initialize storage backends
    return rag

async def main():
    try:
        # Initialize RAG instance
        rag = await initialize_rag()
        await rag.ainsert("Your text")

        # Perform hybrid search
        mode = "hybrid"
        print(
          await rag.aquery(
              "What are the top themes in this story?",
              param=QueryParam(mode=mode)
          )
        )

    except Exception as e:
        print(f"An error occurred: {e}")
    finally:
        if rag:
            await rag.finalize_storages()

if __name__ == "__main__":
    asyncio.run(main())

Notes:

• Export your OPENAI_API_KEY environment variable before running.
• All data is persisted to WORKING_DIR.

Important:

LightRAG requires explicit initialization before use. You must call await rag.initialize_storages() after creating a LightRAG instance, otherwise you will encounter errors.

LightRAG Init Parameters

Parameters

addon_params

addon_params is a live configuration mapping on each LightRAG instance. LightRAG currently reads the fields below; unknown custom keys may remain in the dict, but core LightRAG behavior does not use them.

Supported Fields

Compact chunker shape:

{
  "chunk_token_size": 1200,
  "fixed_token": {
    "chunk_token_size": 1200,
    "chunk_overlap_token_size": 100,
    "split_by_character": null,
    "split_by_character_only": false
  },
  "recursive_character": {
    "chunk_token_size": 1200,
    "chunk_overlap_token_size": 100,
    "separators": ["\n\n", "\n", "。", "！", "？", "；", "，", " ", ""]
  },
  "semantic_vector": {
    "chunk_token_size": 1200,
    "breakpoint_threshold_type": "percentile",
    "breakpoint_threshold_amount": null,
    "buffer_size": 1,
    // env/SDK only (CHUNK_V_SENTENCE_SPLIT_REGEX); the REST chunking.params
    // object rejects this key with 422 — see GHSA-32jh-39m7-8x84 (ReDoS)
    "sentence_split_regex": "(?<=[.?!])\\s+|(?<=[。？！])"
  },
  "paragraph_semantic": {
    "chunk_token_size": 2000,
    "chunk_overlap_token_size": 100
  }
}

Initialization

When you create a LightRAG object, addon_params is normalized before storage initialization:

• If addon_params is omitted, LightRAG builds defaults from SUMMARY_LANGUAGE, ENTITY_TYPE_PROMPT_FILE, and the chunker-related CHUNK_* environment variables.
• If you pass a partial dict, missing language, entity_type_prompt_file, and chunker values are still backfilled from the same env-backed defaults.
• entity_type_prompt_file and entity_types_guidance are resolved into a cached entity extraction prompt profile during construction.
• chunk_token_size and chunk_overlap_token_size constructor arguments are overlaid into addon_params["chunker"] only for slots that were not already set by explicit addon_params or strategy-specific env vars.

Example:

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=llm_model_func,
    embedding_func=embedding_func,
    addon_params={
        "language": "Chinese",
        "entity_type_prompt_file": "entity_type_prompt.sample.yml",
        "entity_types_guidance": "- Paper: academic papers, reports, and preprints",
        "chunker": {
            "chunk_token_size": 1000,
            "recursive_character": {
                "separators": ["\n\n", "\n", "。", "！", "？", " "]
            }
        },
    },
)
await rag.initialize_storages()

Updating After Creation

rag.addon_params is an observable mapping. Top-level updates mark the derived prompt cache dirty; the cache is refreshed the next time LightRAG builds runtime config for extraction or query work.

Update one field:

rag.addon_params["language"] = "Chinese"
rag.addon_params["entity_types_guidance"] = "- Dataset: structured research data"

Replace the whole mapping:

rag.addon_params = {
    "language": "German",
    "entity_type_prompt_file": "domain_profile.yml",
}

Replacing rag.addon_params creates a new observable mapping. If you kept an old reference, discard it and re-read rag.addon_params before making more changes.

Change F-strategy fixed-token splitting defaults for future documents:

rag.addon_params["chunker"]["fixed_token"]["split_by_character"] = "\n\n"
rag.addon_params["chunker"]["fixed_token"]["split_by_character_only"] = True

split_by_character pre-splits text by the given separator before token-window chunking. When split_by_character_only is True, an oversized segment raises an error instead of being split again by token size.

Change R-strategy recursive splitting defaults for future documents:

rag.addon_params["chunker"]["recursive_character"]["separators"] = [
    "\n\n",
    "\n",
    "###",
    "。",
    "！",
    "？",
    " ",
]

Nested chunker edits are read when future documents are enqueued. Documents already enqueued keep their persisted chunk_options snapshot.

semantic_vector.sentence_split_regex is the one exception: it is re-read from addon_params (seeded by CHUNK_V_SENTENCE_SPLIT_REGEX) on every processing run, and any value inside a persisted chunk_options snapshot is discarded and logged at WARNING. This also applies to an explicit chunk_options= passed to apipeline_enqueue_documents — a per-document splitter pattern is not supported. The pattern is applied by re.split to the document body while CPython holds the GIL, so an untrusted one can freeze the whole worker process; see GHSA-32jh-39m7-8x84.

Notes and Precedence

• Entity type guidance precedence is: addon_params["entity_types_guidance"] > entity_type_prompt_file profile > built-in default guidance.
• Chunker precedence is: explicit addon_params["chunker"] values > strategy-specific CHUNK_* env vars > legacy constructor fields (chunk_token_size, chunk_overlap_token_size) > legacy env vars (CHUNK_SIZE, CHUNK_OVERLAP_SIZE).
• Per-strategy chunk_token_size: every strategy reads chunk_token_size from its own sub-dict first and falls back to the top-level chunk_token_size when its sub-dict doesn't set one. F, R, and V can each seed their sub-dict value from a dedicated env var (CHUNK_F_SIZE / CHUNK_R_SIZE / CHUNK_V_SIZE) or set it explicitly in addon_params; when neither is set they inherit the top-level value.
• paragraph_semantic.chunk_token_size is the exception: unlike F/R/V it never inherits the top-level chunk_token_size; if not explicit it uses CHUNK_P_SIZE, then the built-in default 2000.
• enable_multimodal_pipeline is deprecated and ignored if passed in addon_params. Use per-document process_options such as i, t, and e to control multimodal processing.

QueryParam

Use QueryParam to control the behavior of your query:

class QueryParam:
    """Configuration parameters for query execution in LightRAG."""

    mode: Literal["local", "global", "hybrid", "naive", "mix", "bypass"] = "mix"
    """Specifies the retrieval mode:
    - "local": Focuses on context-dependent information.
    - "global": Utilizes global knowledge.
    - "hybrid": Combines local and global retrieval methods.
    - "naive": Performs a basic search without advanced techniques.
    - "mix": Integrates knowledge graph and vector retrieval.
    """

    only_need_context: bool = False
    """If True, only returns the retrieved context without generating a response."""

    only_need_prompt: bool = False
    """If True, only returns the generated prompt without producing a response."""

    response_type: str = "Multiple Paragraphs"
    """Defines the response format. Examples: 'Multiple Paragraphs', 'Single Paragraph', 'Bullet Points'."""

    stream: bool = False
    """If True, enables streaming output for real-time responses."""

    top_k: int = int(os.getenv("TOP_K", "60"))
    """Number of top items to retrieve. Represents entities in 'local' mode and relationships in 'global' mode."""

    chunk_top_k: int = int(os.getenv("CHUNK_TOP_K", "20"))
    """Number of text chunks to retrieve initially from vector search and keep after reranking.
    If None, defaults to top_k value.
    """

    max_entity_tokens: int = int(os.getenv("MAX_ENTITY_TOKENS", "6000"))
    """Maximum number of tokens allocated for entity context in unified token control system."""

    max_relation_tokens: int = int(os.getenv("MAX_RELATION_TOKENS", "8000"))
    """Maximum number of tokens allocated for relationship context in unified token control system."""

    max_total_tokens: int = int(os.getenv("MAX_TOTAL_TOKENS", "30000"))
    """Maximum total tokens budget for the entire query context (entities + relations + chunks + system prompt)."""

    # History messages are only sent to LLM for context, not used for retrieval
    conversation_history: list[dict[str, str]] = field(default_factory=list)
    """Stores past conversation history to maintain context.
    Format: [{"role": "user/assistant", "content": "message"}].
    """

    user_prompt: str | None = None
    """User-provided prompt for the query.
    Additional instructions for LLM. If provided, this will be injected into the prompt template.
    Its purpose is to let the user customize the way LLM generates the response.
    """

    disable_user_prompt_prefix: bool = False
    """If True, the server-side global prompt prefix is NOT prepended to `user_prompt`."""

    enable_rerank: bool = True
    """Enable reranking for retrieved text chunks. If True but no rerank model is configured, a warning will be issued.
    Default is True to enable reranking when rerank model is available.
    """

The default value of top_k can be changed by the environment variable TOP_K.

LLM and Embedding Injection

LightRAG requires LLM and Embedding models for document indexing and querying. During initialization, inject the relevant model functions into LightRAG.

Model Selection Requirements

• LLM: at least 32B parameters, 32KB context (64KB recommended). Avoid reasoning models during indexing; use stronger models at query time.
• Embedding: must be consistent across indexing and querying. Recommended: BAAI/bge-m3, text-embedding-3-large. Changing models requires clearing vector storage.
• Reranker: significantly improves retrieval. When enabled, set query mode to mix. Recommended: BAAI/bge-reranker-v2-m3, Jina rerankers.

Using OpenAI-like APIs

LightRAG supports OpenAI-like chat/embeddings APIs:

import os
import numpy as np
from lightrag.utils import wrap_embedding_func_with_attrs
from lightrag.llm.openai import openai_complete_if_cache, openai_embed

async def llm_model_func(
    prompt, system_prompt=None, history_messages=[], keyword_extraction=False, **kwargs
) -> str:
    return await openai_complete_if_cache(
        "solar-mini",
        prompt,
        system_prompt=system_prompt,
        history_messages=history_messages,
        api_key=os.getenv("UPSTAGE_API_KEY"),
        base_url="https://api.upstage.ai/v1/solar",
        **kwargs
    )

@wrap_embedding_func_with_attrs(embedding_dim=4096, max_token_size=8192, model_name="solar-embedding-1-large-query")
async def embedding_func(texts: list[str]) -> np.ndarray:
    return await openai_embed.func(
        texts,
        model="solar-embedding-1-large-query",
        api_key=os.getenv("UPSTAGE_API_KEY"),
        base_url="https://api.upstage.ai/v1/solar"
    )

async def initialize_rag():
    rag = LightRAG(
        working_dir=WORKING_DIR,
        llm_model_func=llm_model_func,
        embedding_func=embedding_func  # Pass the decorated function directly
    )
    await rag.initialize_storages()
    return rag

Important Note on Embedding Function Wrapping:

EmbeddingFunc cannot be nested. Functions decorated with @wrap_embedding_func_with_attrs (such as openai_embed, ollama_embed, etc.) cannot be wrapped again using EmbeddingFunc(). This is why we call xxx_embed.func (the underlying unwrapped function) instead of xxx_embed directly when creating custom embedding functions.

Using Hugging Face Models

See lightrag_hf_demo.py

from functools import partial
from transformers import AutoTokenizer, AutoModel

# Pre-load tokenizer and model
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")
embed_model = AutoModel.from_pretrained("sentence-transformers/all-MiniLM-L6-v2")

# Initialize LightRAG with Hugging Face model
rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=hf_model_complete,  # Use Hugging Face model for text generation
    llm_model_name='meta-llama/Llama-3.1-8B-Instruct',  # Model name from Hugging Face
    # Use Hugging Face embedding function
    embedding_func=EmbeddingFunc(
        embedding_dim=384,
        max_token_size=2048,
        model_name="sentence-transformers/all-MiniLM-L6-v2",
        func=partial(
            hf_embed.func,  # Use .func to access the unwrapped function
            tokenizer=tokenizer,
            embed_model=embed_model
        )
    ),
)

Using Ollama Models

Pull the model you plan to use and an embedding model, for example nomic-embed-text:

import numpy as np
from lightrag.utils import wrap_embedding_func_with_attrs
from lightrag.llm.ollama import ollama_model_complete, ollama_embed

@wrap_embedding_func_with_attrs(embedding_dim=768, max_token_size=8192, model_name="nomic-embed-text")
async def embedding_func(texts: list[str]) -> np.ndarray:
    return await ollama_embed.func(texts, embed_model="nomic-embed-text")

# Initialize LightRAG with Ollama model
rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=ollama_model_complete,
    llm_model_name='your_model_name',
    embedding_func=embedding_func,
)

Increasing context size

LightRAG requires at least 32k context tokens. Ollama defaults to 8k. Two approaches:

Approach 1: Edit Modelfile

ollama pull qwen2
ollama show --modelfile qwen2 > Modelfile
# Add this line to Modelfile:
# PARAMETER num_ctx 32768
ollama create -f Modelfile qwen2m

Approach 2: Set num_ctx via llm_model_kwargs

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=ollama_model_complete,
    llm_model_name='your_model_name',
    llm_model_kwargs={"options": {"num_ctx": 32768}},
    embedding_func=embedding_func,
)

Important Note on Embedding Function Wrapping:

EmbeddingFunc cannot be nested. Use xxx_embed.func to access the underlying unwrapped function.

Low RAM GPUs

For low-RAM GPUs (e.g. 6GB), select a small model and tune the context window. For example, gemma2:2b with num_ctx=26000 can find ~197 entities and 19 relations on book.txt.

LlamaIndex

LightRAG supports integration with LlamaIndex (llm/llama_index_impl.py):

import asyncio
from lightrag import LightRAG
from lightrag.llm.llama_index_impl import llama_index_complete_if_cache, llama_index_embed
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.llms.openai import OpenAI
from lightrag.utils import setup_logger

setup_logger("lightrag", level="INFO")

async def initialize_rag():
    rag = LightRAG(
        working_dir="your/path",
        llm_model_func=llama_index_complete_if_cache,
        embedding_func=EmbeddingFunc(
            embedding_dim=1536,
            max_token_size=2048,
            model_name=embed_model,
            func=partial(llama_index_embed.func, embed_model=embed_model)
        ),
    )
    await rag.initialize_storages()
    return rag

Further reading:

• LlamaIndex Documentation
• Direct OpenAI Example
• LiteLLM Proxy Example
• LiteLLM Proxy with Opik Example

Using Azure OpenAI Models

import os
import numpy as np
from lightrag.utils import wrap_embedding_func_with_attrs
from lightrag.llm.azure_openai import azure_openai_complete_if_cache, azure_openai_embed

async def llm_model_func(
    prompt, system_prompt=None, history_messages=[], keyword_extraction=False, **kwargs
) -> str:
    return await azure_openai_complete_if_cache(
        prompt,
        system_prompt=system_prompt,
        history_messages=history_messages,
        api_key=os.getenv("AZURE_OPENAI_API_KEY"),
        azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
        api_version=os.getenv("AZURE_OPENAI_API_VERSION"),
        deployment_name=os.getenv("AZURE_OPENAI_DEPLOYMENT_NAME"),
        **kwargs
    )

@wrap_embedding_func_with_attrs(
    embedding_dim=1536,
    max_token_size=8192,
    model_name=os.getenv("AZURE_OPENAI_EMBEDDING_MODEL")
)
async def embedding_func(texts: list[str]) -> np.ndarray:
    return await azure_openai_embed.func(
        texts,
        api_key=os.getenv("AZURE_OPENAI_API_KEY"),
        azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"),
        api_version=os.getenv("AZURE_OPENAI_API_VERSION"),
        deployment_name=os.getenv("AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME")
    )

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=llm_model_func,
    embedding_func=embedding_func
)

Using Google Gemini Models

import os
import numpy as np
from lightrag.utils import wrap_embedding_func_with_attrs
from lightrag.llm.gemini import gemini_model_complete, gemini_embed

async def llm_model_func(
    prompt, system_prompt=None, history_messages=[], keyword_extraction=False, **kwargs
) -> str:
    return await gemini_model_complete(
        prompt,
        system_prompt=system_prompt,
        history_messages=history_messages,
        api_key=os.getenv("GEMINI_API_KEY"),
        model_name="gemini-2.0-flash",
        **kwargs
    )

@wrap_embedding_func_with_attrs(
    embedding_dim=768,
    max_token_size=2048,
    model_name="models/text-embedding-004"
)
async def embedding_func(texts: list[str]) -> np.ndarray:
    return await gemini_embed.func(
        texts,
        api_key=os.getenv("GEMINI_API_KEY"),
        model="models/text-embedding-004"
    )

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=llm_model_func,
    llm_model_name="gemini-2.0-flash",
    embedding_func=embedding_func
)

Rerank Function Injection

To enhance retrieval quality, documents can be re-ranked based on a more effective relevance scoring model. The rerank.py file provides three Reranker provider driver functions:

• Cohere / vLLM: cohere_rerank
• Jina AI: jina_rerank
• Aliyun: ali_rerank

Inject one of these functions into the rerank_model_func attribute of the LightRAG object. For detailed usage, refer to examples/rerank_example.py.

Injecting a Custom Tokenizer

Any object with encode(str) -> list[int] and decode(list[int]) -> str can be wrapped in Tokenizer and passed as tokenizer=. Two requirements apply:

1. It must be safe to call concurrently from multiple threads. Token counting is CPU-bound, so LightRAG runs it in worker threads to keep the asyncio event loop responsive; several of them may enter your encode/decode at once. LightRAG deliberately does not serialize calls on your behalf — a lock owned by LightRAG would end up being waited on by the event loop behind a worker thread, which is exactly the stall the threading is there to avoid.
2. It must survive copy.deepcopy. LightRAG is a dataclass and builds its internal config with dataclasses.asdict, which deep-copies non-dataclass fields.

The two interact: if you achieve thread safety with an internal threading.Lock, deep-copying it raises TypeError: cannot pickle '_thread.lock' object. Declare __deepcopy__ returning self, which is sound precisely because a thread-safe tokenizer is safe to share:

class MyTokenizer:
    def __init__(self):
        self._lock = threading.Lock()

    def __deepcopy__(self, memo):
        return self  # thread-safe, therefore shareable

    def encode(self, content: str) -> list[int]: ...
    def decode(self, tokens: list[int]) -> str: ...

rag = LightRAG(..., tokenizer=Tokenizer("my-model", MyTokenizer()))

The built-in TiktokenTokenizer satisfies both. Note that copying it is not a way to get isolation: tiktoken caches encodings in a process-wide registry, so every TiktokenTokenizer for a given model — copies included — resolves to the same underlying BPE engine.

User Prompt vs. Query

When using LightRAG for content queries, avoid combining the search process with unrelated output processing, as this significantly impacts query effectiveness. The user_prompt parameter in QueryParam does not participate in the RAG retrieval phase — it guides the LLM on how to process the retrieved results after the query is completed.

"Does not participate in retrieval" means it does not influence what is found or how it is ranked: it is not used for keyword extraction, vector search, or reranking. It does still consume part of the token budget, because it genuinely occupies space in the final prompt alongside the retrieved context.

query_param = QueryParam(
    mode="hybrid",
    user_prompt="For diagrams, use mermaid format with English/Pinyin node names and Chinese display labels",
)

response_default = rag.query(
    "Please draw a character relationship diagram for Scrooge",
    param=query_param
)
print(response_default)

A Global User Prompt Prefix

user_prompt is supplied per request, so it cannot express an output policy that should hold for every caller. LightRAG.user_prompt_prefix is that policy: a server-side string prepended to each request's user_prompt.

rag = LightRAG(..., user_prompt_prefix="Answer in the language of the question.\n\n")

For the API server it comes from the environment instead — USER_PROMPT_PREFIX for a short value, or USER_PROMPT_PREFIX_FILE (a .md/.txt file name under PROMPT_DIR/user_prompt) when the text is long, multi-paragraph, or contains ${...}, which python-dotenv would otherwise interpolate away.

The two strings are concatenated verbatim, with no separator inserted — end the prefix with your own \n\n so it does not run into the caller's text. The prefix comes first because a model weights later instructions more heavily on conflict, so the per-request prompt wins.

An empty user_prompt does not disable the prefix. When a request sends no user_prompt — None, "", or the field omitted entirely — the prefix alone becomes the instructions sent to the LLM. This is the common deployment: the operator sets one policy and callers send nothing.

# All three send exactly "Answer in the language of the question." to the model.
rag.query("...", param=QueryParam(mode="hybrid"))
rag.query("...", param=QueryParam(mode="hybrid", user_prompt=None))
rag.query("...", param=QueryParam(mode="hybrid", user_prompt=""))

This matters for the WebUI in particular, which ships user_prompt: "" as its default: leaving the box blank applies the operator's policy rather than clearing it. The Additional Instructions section falls back to n/a only when both the prefix and the request's user_prompt are empty.

A request opts out with disable_user_prompt_prefix — the only way to suppress the prefix — which is what lets a front-end take full control of the final instruction text:

QueryParam(user_prompt="...", disable_user_prompt_prefix=True)

The prefix is configuration, not request data: a request can decline it but can never read or replace it. Three limits are worth knowing:

• bypass mode ignores it, as it ignores user_prompt entirely — empty or not. That path has no {user_prompt} slot and its system_prompt argument belongs to the caller, so this is not an exception to the rule above: bypass simply sends no user instructions at all.
• only_need_prompt=True returns the composed prompt, so any client that can set that debug flag can read the prefix verbatim.
• only_need_context and only_need_prompt are charged for the prefix, even though only_need_context returns before any prompt is sent. These switches preview the real request: if retrieval-only calls skipped the charge they would report more chunks than a live query retrieves, and context sized against that number would be truncated at answer time. /query/data (aquery_data) is retrieval-only and follows the same rule.
• A custom system_prompt without a {user_prompt} placeholder drops it, the same way it already drops user_prompt. The token budget accounts for this: the prefix is charged against the context allowance only when the template that will actually be rendered has somewhere to put it.

The prefix participates in the answer cache key, so editing it invalidates answers generated under the old one. With no prefix configured the key is unchanged, so existing cache entries keep hitting.

Storage Backends

Sotrage Types

LightRAG uses 4 types of storage for different purposes:

Supported Implementations

KV_STORAGE

JsonKVStorage        JsonFile (default)
PGKVStorage          Postgres
RedisKVStorage       Redis
MongoKVStorage       MongoDB
OpenSearchKVStorage  OpenSearch

GRAPH_STORAGE

NetworkXStorage          NetworkX (default)
Neo4JStorage             Neo4J
PGGraphStorage           PostgreSQL with AGE plugin
PGTableGraphStorage      PostgreSQL, plain tables (no AGE, no extensions)
MemgraphStorage          Memgraph
OpenSearchGraphStorage   OpenSearch

Testing has shown that Neo4J delivers superior performance in production environments compared to PostgreSQL with AGE plugin.

PGTableGraphStorage implements the graph layer on ordinary indexed tables plus JSONB, so it runs on any stock PostgreSQL 14+ — including managed instances (RDS, Cloud SQL, Supabase, Neon) where the AGE extension cannot be installed. It shares the same POSTGRES_* configuration and connection pool as the other PG storages. Choose PGGraphStorage only if you specifically need AGE/Cypher.

VECTOR_STORAGE

NoopVectorDBStorage         Disabled (graph-only ingestion)
NanoVectorDBStorage         NanoVector (default)
PGVectorStorage             Postgres
MilvusVectorDBStorage       Milvus
FaissVectorDBStorage        Faiss
QdrantVectorDBStorage       Qdrant
MongoVectorDBStorage        MongoDB
OpenSearchVectorDBStorage   OpenSearch

Graph-only ingestion

NoopVectorDBStorage is intended for an initial or offline corpus backfill where the graph and KV stores are authoritative and vector indexes can be materialized once from the final state. It avoids embedding and persisting intermediate entity, relationship, and chunk revisions during ingestion.

Do not use this workflow when newly inserted documents must become queryable immediately. Normal incremental ingestion should use the intended persistent vector backend from the beginning.

Configure the backfill process with the no-op backend. embedding_func=None is supported when no other configured component requires embeddings:

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=llm_model_func,
    embedding_func=None,
    vector_storage="NoopVectorDBStorage",
)

The backend accepts vector mutations without calling the embedding function or persisting vectors. Graph, full-document, text-chunk, LLM-cache, document-status, and graph-recovery writes continue normally.

While NoopVectorDBStorage is active, only bypass queries are available. local, global, hybrid, mix, and naive modes require vector indexes and raise an error that points to lightrag-rebuild-vdb.

If the semantic-vector (V) chunker is selected while embedding_func=None, it logs a warning and falls back to recursive-character chunking. Configure an embedding function during ingestion if semantic-vector chunk boundaries are required; this is separate from whether vectors are persisted.

Switching to persistent vectors

After the backfill, stop the server and all ingestion writers. Keep WORKING_DIR, WORKSPACE, graph storage, KV storage, and their connection settings unchanged. For example, switch from Noop to NanoVector with an OpenAI embedding model while pointing to the same graph and KV sources:

export WORKING_DIR=/data/lightrag/rag_storage
export WORKSPACE=project_a
export LIGHTRAG_GRAPH_STORAGE=NetworkXStorage
export LIGHTRAG_KV_STORAGE=JsonKVStorage
export LIGHTRAG_VECTOR_STORAGE=NanoVectorDBStorage
export EMBEDDING_BINDING=openai
export EMBEDDING_MODEL=text-embedding-3-small
export EMBEDDING_DIM=1536

lightrag-rebuild-vdb  # Select "Rebuild ALL vector storages"

Replace all example values with the backfill's actual storage settings and the production embedding model, dimension, host, and credentials. Backend-specific connection variables must remain available. If the same .env already contains these values, only the vector and embedding entries need to change. Run this in a new process or after a restart, then keep the persistent configuration for later queries and incremental ingestion.

Rebuild cost and memory grow with the final graph and chunk data. If rebuilding fails or is interrupted, keep writers stopped and rerun it with the same configuration; the graph and KV sources remain unchanged. Start the server only after the tool reports a successful rebuild, for example with lightrag-server, because LightRAG has no persisted vector-index readiness marker.

See lightrag/tools/README_REBUILD_VDB.md for rebuild options and operational details.

DOC_STATUS_STORAGE

JsonDocStatusStorage        JsonFile (default)
PGDocStatusStorage          Postgres
MongoDocStatusStorage       MongoDB
OpenSearchDocStatusStorage  OpenSearch

Example connection configurations for each storage type can be found in the repository's env.example file. The database instance in the connection string must be created beforehand — LightRAG only creates tables within the instance, not the instance itself.

Backend-Specific Setup

Using Neo4J Storage

For production level scenarios you will most likely want to leverage an enterprise solution for KG storage. Running Neo4J in Docker is recommended for seamless local testing. See: https://hub.docker.com/_/neo4j

export NEO4J_URI="neo4j://localhost:7687"
export NEO4J_USERNAME="neo4j"
export NEO4J_PASSWORD="password"
export NEO4J_DATABASE="neo4j"  # Required for community edition

from lightrag.utils import setup_logger

setup_logger("lightrag", level="INFO")

async def initialize_rag():
    rag = LightRAG(
        working_dir=WORKING_DIR,
        llm_model_func=gpt_4o_mini_complete,
        graph_storage="Neo4JStorage",
    )
    await rag.initialize_storages()
    return rag

See test_neo4j.py for a working example.

Using PostgreSQL Storage

PostgreSQL can provide a one-stop solution as KV store, VectorDB (pgvector), and GraphDB (PGTableGraphStorage on plain indexed tables, or PGGraphStorage on Apache AGE). PostgreSQL version 16.6 or higher is supported.

• PostgreSQL is lightweight; the whole binary distribution including all necessary plugins can be zipped to 40MB: Ref to Windows Release as it is easy to install for Linux/Mac.
• If you prefer Docker and graph storage is PGTableGraphStorage (the recommended choice, which needs no Apache AGE), the official pgvector image pgvector/pgvector:pg18 is all you need.
• Only PGGraphStorage requires an AGE-bundled image; to avoid hiccups there, start with https://hub.docker.com/r/gzdaniel/postgres-for-rag (published for linux/amd64 and linux/arm64). The latest image no longer ships hardcoded credentials; on first start it creates the user, password, and database from the POSTGRES_USER / POSTGRES_PASSWORD / POSTGRES_DB environment variables (these are set automatically when you deploy via the scripts/setup/setup.sh wizard, so you can pick any values).
• How to start: see examples/lightrag_gemini_postgres_demo.py
• For high-performance graph database requirements, Neo4j is recommended as Apache AGE's performance is not as competitive.

Using Faiss Storage

Before using Faiss, manually install faiss-cpu or faiss-gpu:

pip install faiss-cpu

async def embedding_func(texts: list[str]) -> np.ndarray:
    model = SentenceTransformer('all-MiniLM-L6-v2')
    embeddings = model.encode(texts, convert_to_numpy=True)
    return embeddings

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=llm_model_func,
    embedding_func=EmbeddingFunc(
        embedding_dim=384,
        max_token_size=2048,
        model_name="all-MiniLM-L6-v2",
        func=embedding_func,
    ),
    vector_storage="FaissVectorDBStorage",
    vector_db_storage_cls_kwargs={
        "cosine_better_than_threshold": 0.3
    }
)

Using Memgraph for Storage

Memgraph is a high-performance, in-memory graph database compatible with the Neo4j Bolt protocol. See: https://memgraph.com/download

export MEMGRAPH_URI="bolt://localhost:7687"

async def initialize_rag():
    rag = LightRAG(
        working_dir=WORKING_DIR,
        llm_model_func=gpt_4o_mini_complete,
        graph_storage="MemgraphStorage",
    )
    await rag.initialize_storages()
    return rag

Using Milvus for Vector Storage

Milvus is a high-performance, scalable vector database for production-level vector storage. For full configuration options including index types (HNSW, HNSW_SQ, IVF, DISKANN, etc.) and metric types, see docs/MilvusConfigurationGuide.md.

Quick setup via environment variables:

MILVUS_URI=http://localhost:19530
MILVUS_DB_NAME=lightrag
LIGHTRAG_VECTOR_STORAGE=MilvusVectorDBStorage

Quick setup via Python SDK:

rag = LightRAG(
    working_dir="./rag_storage",
    llm_model_func=...,
    embedding_func=...,
    vector_storage="MilvusVectorDBStorage",
    vector_db_storage_cls_kwargs={
        "milvus_uri": "http://localhost:19530",
        "milvus_db_name": "lightrag",
        "cosine_better_than_threshold": 0.2,
    },
)

Using MongoDB Storage

MongoDB provides a one-stop storage solution for LightRAG with native KV storage and vector storage. LightRAG uses MongoDB collections to implement a simple graph storage.

MongoVectorDBStorage requires a MongoDB deployment with Atlas Search / Vector Search support (e.g., MongoDB Atlas or Atlas local). The setup wizard's bundled local Docker MongoDB service is MongoDB Community Edition — it can be used for KV/graph/doc-status storage but not for MongoVectorDBStorage.

Using Redis Storage

LightRAG supports Redis as KV storage. Configure persistence and memory usage carefully. Recommended Redis configuration:

save 900 1
save 300 10
save 60 1000
stop-writes-on-bgsave-error yes
maxmemory 4gb
maxmemory-policy noeviction
maxclients 500

When the interactive setup manages a local Redis container, it stages a user-editable config at ./data/config/redis.conf and mounts it into the container. Setup preserves that file on reruns so local Redis tuning can be adjusted without losing manual edits.

Using OpenSearch Storage

OpenSearch provides a unified storage solution for all four LightRAG storage types (KV, Vector, Graph, DocStatus). It offers native k-NN vector search, full-text search, and horizontal scalability without cloud-only restrictions.

Requirements: OpenSearch 3.x or higher with k-NN plugin enabled.

Install with Docker (without plugins):

docker run -d -p 9200:9200 -e "discovery.type=single-node" \
  -e "OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password>" \
  opensearchproject/opensearch:latest

Install with Docker Compose (Recommended, with plugins):

curl -O https://raw.githubusercontent.com/opensearch-project/opensearch-build/main/docker/release/dockercomposefiles/docker-compose-3.x.yml
OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password> docker-compose -f docker-compose-3.x.yml up -d

Configuration (see env.example for full list):

export OPENSEARCH_HOSTS=localhost:9200
export OPENSEARCH_USER=admin
export OPENSEARCH_PASSWORD=<custom-admin-password>
export OPENSEARCH_USE_SSL=true
export OPENSEARCH_VERIFY_CERTS=false

Usage:

rag = LightRAG(
    working_dir=WORKING_DIR,
    llm_model_func=your_llm_func,
    embedding_func=your_embed_func,
    kv_storage="OpenSearchKVStorage",
    doc_status_storage="OpenSearchDocStatusStorage",
    graph_storage="OpenSearchGraphStorage",
    vector_storage="OpenSearchVectorDBStorage",
)

Graph Traversal: When the OpenSearch SQL plugin with PPL support is available, graph queries use server-side BFS via the graphlookup command for optimal performance. Otherwise, it falls back to client-side batched BFS. Auto-detected at startup, or force via OPENSEARCH_USE_PPL_GRAPHLOOKUP=true|false.

Integration Testing:

1. Start OpenSearch using Docker Compose:

OPENSEARCH_INITIAL_ADMIN_PASSWORD=<custom-admin-password> docker-compose -f docker-compose-3.x.yml up -d

2. Verify the cluster is running:

curl -sk -u admin:<custom-admin-password> https://localhost:9200
curl -sk -u admin:<custom-admin-password> https://localhost:9200/_cat/plugins?v

3. Run unit tests (no OpenSearch required — uses mocks):

python -m pytest tests/kg/opensearch_impl/test_opensearch_storage.py -v

4. Run the OpenSearch storage demo:

export OPENSEARCH_HOSTS=localhost:9200
export OPENSEARCH_USER=admin
export OPENSEARCH_PASSWORD=<custom-admin-password>
export OPENSEARCH_USE_SSL=true
export OPENSEARCH_VERIFY_CERTS=false
python examples/opensearch_storage_demo.py

5. Run the full OpenAI + OpenSearch demo (requires OPENAI_API_KEY):

export OPENAI_API_KEY=your-api-key
python examples/lightrag_openai_opensearch_graph_demo.py

6. Visualize the knowledge graph via LightRAG WebUI:

LIGHTRAG_KV_STORAGE=OpenSearchKVStorage \
LIGHTRAG_DOC_STATUS_STORAGE=OpenSearchDocStatusStorage \
LIGHTRAG_GRAPH_STORAGE=OpenSearchGraphStorage \
LIGHTRAG_VECTOR_STORAGE=OpenSearchVectorDBStorage \
LLM_BINDING=openai \
EMBEDDING_BINDING=openai \
EMBEDDING_MODEL=text-embedding-3-large \
EMBEDDING_DIM=3072 \
OPENAI_API_KEY=your-api-key \
lightrag-server

Data Isolation Between LightRAG Instances

The workspace parameter ensures data isolation between different LightRAG instances. Once initialized, the workspace is immutable.

Legacy compatibility: Default workspace for PostgreSQL non-graph storage is default; for PostgreSQL AGE graph storage is null; for Neo4j graph storage is base.

Storage-specific workspace environment variables override the common WORKSPACE variable: REDIS_WORKSPACE, MILVUS_WORKSPACE, QDRANT_WORKSPACE, MONGODB_WORKSPACE, POSTGRES_WORKSPACE, NEO4J_WORKSPACE, OPENSEARCH_WORKSPACE.

For a practical demonstration of managing multiple isolated knowledge bases, see Workspace Demo.

Insert

• Basic Insert

rag.insert("Text")

• Batch Insert

# Basic Batch Insert
rag.insert(["TEXT1", "TEXT2", ...])

# Batch Insert with custom batch size
rag = LightRAG(
    ...
    working_dir=WORKING_DIR,
    max_parallel_insert=4
)
rag.insert(["TEXT1", "TEXT2", "TEXT3", ...])  # Processed in batches of 4

The max_parallel_insert parameter determines the number of documents processed concurrently. Default is 3. Recommended to keep below 10, as the bottleneck typically lies with the LLM.

• Insert with ID

The number of documents and IDs must be the same.

# Single text with ID
rag.insert("TEXT1", ids=["ID_FOR_TEXT1"])

# Multiple texts with IDs
rag.insert(["TEXT1", "TEXT2", ...], ids=["ID_FOR_TEXT1", "ID_FOR_TEXT2"])

• Insert using Pipeline

apipeline_enqueue_documents and apipeline_process_enqueue_documents allow incremental insertion of documents in the background while the main thread continues executing.

rag = LightRAG(..)
await rag.apipeline_enqueue_documents(input)
# Your routine in loop
await rag.apipeline_process_enqueue_documents(input)

• Insert Multi-file Type Support

The textract library supports reading TXT, DOCX, PPTX, CSV, and PDF:

import textract

file_path = 'TEXT.pdf'
text_content = textract.process(file_path)
rag.insert(text_content.decode('utf-8'))

• Citation Functionality

By providing file paths, the system ensures sources can be traced back to their original documents:

documents = ["Document content 1", "Document content 2"]
file_paths = ["path/to/doc1.txt", "path/to/doc2.txt"]

rag.insert(documents, file_paths=file_paths)

Edit Entities and Relations

LightRAG supports comprehensive knowledge graph management: create, edit, and delete entities and relationships.

• Create Entities and Relations

# Create entity
entity = rag.create_entity("Google", {
    "description": "Google is a multinational technology company specializing in internet-related services and products.",
    "entity_type": "company"
})

product = rag.create_entity("Gmail", {
    "description": "Gmail is an email service developed by Google.",
    "entity_type": "product"
})

# Create relation
relation = rag.create_relation("Google", "Gmail", {
    "description": "Google develops and operates Gmail.",
    "keywords": "develops operates service",
    "source_id": "chunk-google-gmail",
    "weight": 1.5
})

• Edit Entities and Relations

# Edit entity attributes
updated_entity = rag.edit_entity("Google", {
    "description": "Google is a subsidiary of Alphabet Inc., founded in 1998.",
    "entity_type": "tech_company"
})

# Rename entity (with all its relationships properly migrated)
renamed_entity = rag.edit_entity("Gmail", {
    "entity_name": "Google Mail",
    "description": "Google Mail (formerly Gmail) is an email service."
})

# Edit relation
updated_relation = rag.edit_relation("Google", "Google Mail", {
    "description": "Google created and maintains Google Mail service.",
    "keywords": "creates maintains email service",
    "weight": 3.0
})

Entity names supplied to create_entity and new names supplied during edit_entity renames use the same normalization rules as extracted entity names. When editing an existing entity, LightRAG first preserves an exact legacy name match and otherwise falls back to the normalized name. insert_custom_kg applies the same rules to declared entity names and both endpoints of every relationship before writing any custom KG data. merge_entities resolves existing exact legacy source/target names first and otherwise uses normalized names. The target may be an existing entity or a new normalized name created by the merge.

Relation Weight Contract

Relation weight has an evidence-count floor. Each distinct real ID in the source_id field contributes one unit of evidence, and a larger explicit weight is an optional importance boost:

weight >= len(distinct real source IDs)

Multiple source IDs use the <SEP> separator. Empty values and the historical no-source placeholders manual_creation and UNKNOWN are not evidence. When a relation has no real source IDs, its evidence count is zero, so a non-negative fractional weight is valid. When creating a source-less relation, omit source_id; when editing an existing relation, set source_id to an empty string in the same edit that lowers the weight.

create_relation, edit_relation, and insert_custom_kg validate this contract before writing graph or vector data; invalid Python API inputs raise ValueError and the REST graph API returns HTTP 400. Relation edits validate the complete post-edit shape, so source_id and weight can be changed together. Existing legacy relations are repaired upward when extraction adds evidence, an entity rename rewrites their endpoints, an unrelated relation edit rewrites the row, or a relation is rebuilt from surviving chunks (document purge, resume, and custom-chunk rollback). lightrag-rebuild-vdb is not such a repair point: it mirrors each graph edge into the vector storage field for field, copying the stored weight verbatim without touching the graph.

A rebuild re-derives the relation from the extraction results cached for the surviving chunks, so — like the rebuilt description and keywords — the weight is recomputed rather than preserved: it becomes the summed fragment weights lifted to the surviving evidence count. Weight therefore follows evidence downward as a purge removes chunks, and an importance boost applied through edit_relation does not survive a rebuild that finds cached fragments. When no cached fragment survives, the rebuild keeps the stored weight.

When entity merging redirects multiple relations onto the same endpoint, the result is:

merged weight = max(all input weights, distinct merged real source IDs)

This preserves a larger manual boost while preventing the merged weight from falling below its evidence count.

All operations are available in both synchronous and asynchronous versions. Async versions have the prefix "a" (e.g., acreate_entity, aedit_relation).

• Insert Custom KG

custom_kg = {
    "chunks": [
        {
            "content": "Alice and Bob are collaborating on quantum computing research.",
            "source_id": "doc-1",
            "file_path": "test_file",
        }
    ],
    "entities": [
        {
            "entity_name": "Alice",
            "entity_type": "person",
            "description": "Alice is a researcher specializing in quantum physics.",
            "source_id": "doc-1",
            "file_path": "test_file"
        },
        {
            "entity_name": "Bob",
            "entity_type": "person",
            "description": "Bob is a mathematician.",
            "source_id": "doc-1",
            "file_path": "test_file"
        },
        {
            "entity_name": "Quantum Computing",
            "entity_type": "technology",
            "description": "Quantum computing utilizes quantum mechanical phenomena for computation.",
            "source_id": "doc-1",
            "file_path": "test_file"
        }
    ],
    "relationships": [
        {
            "src_id": "Alice",
            "tgt_id": "Bob",
            "description": "Alice and Bob are research partners.",
            "keywords": "collaboration research",
            "weight": 1.0,
            "source_id": "doc-1",
            "file_path": "test_file"
        },
        {
            "src_id": "Alice",
            "tgt_id": "Quantum Computing",
            "description": "Alice conducts research on quantum computing.",
            "keywords": "research expertise",
            "weight": 1.0,
            "source_id": "doc-1",
            "file_path": "test_file"
        },
        {
            "src_id": "Bob",
            "tgt_id": "Quantum Computing",
            "description": "Bob researches quantum computing.",
            "keywords": "research application",
            "weight": 1.0,
            "source_id": "doc-1",
            "file_path": "test_file"
        }
    ]
}

rag.insert_custom_kg(custom_kg)

• Other Entity and Relation Operations
  • create_entity: Creates a new entity with specified attributes
  • edit_entity: Updates an existing entity's attributes or renames it
  • create_relation: Creates a new relation between existing entities
  • edit_relation: Updates an existing relation's attributes

These operations maintain data consistency across both the graph database and vector database components.

Delete Functions

LightRAG provides comprehensive deletion capabilities.

Delete Entities

# Synchronous
rag.delete_by_entity("Google")

# Asynchronous
await rag.adelete_by_entity("Google")

When deleting an entity:

• Removes the entity node from the knowledge graph
• Deletes all associated relationships
• Removes related embedding vectors from the vector database
• Maintains knowledge graph integrity

Delete Relations

# Synchronous
rag.delete_by_relation("Google", "Gmail")

# Asynchronous
await rag.adelete_by_relation("Google", "Gmail")

When deleting a relationship:

• Removes the specified relationship edge
• Deletes the relationship's embedding vector
• Preserves both entity nodes and their other relationships

Delete by Document ID

# Asynchronous only (complex reconstruction process)
await rag.adelete_by_doc_id("doc-12345")

The deletion process:

1. Delete all text chunks related to the document
2. Identify and delete entities/relationships that belong only to this document
3. Rebuild entities/relationships that still exist in other documents
4. Update all related vector indexes
5. Clean up document status records

Important Reminders:

1. All deletion operations are irreversible — use with caution
2. Deleting large amounts of data may take time, especially deletion by document ID
3. Deletion operations automatically maintain consistency between the graph and vector databases
4. Consider backing up data before performing important deletions

Entity Merging

Merge Entities and Their Relationships

# Basic merge
rag.merge_entities(
    source_entities=["Artificial Intelligence", "AI", "Machine Intelligence"],
    target_entity="AI Technology"
)

# With custom merge strategy
rag.merge_entities(
    source_entities=["John Smith", "Dr. Smith", "J. Smith"],
    target_entity="John Smith",
    merge_strategy={
        "description": "concatenate",  # Combine all descriptions
        "entity_type": "keep_first",   # Keep the type from the first entity
        "source_id": "join_unique"     # Combine all unique source IDs
    }
)

# With custom target entity data
rag.merge_entities(
    source_entities=["New York", "NYC", "Big Apple"],
    target_entity="New York City",
    target_entity_data={
        "entity_type": "LOCATION",
        "description": "New York City is the most populous city in the United States.",
    }
)

# Advanced: combining both strategy and custom data
rag.merge_entities(
    source_entities=["Microsoft Corp", "Microsoft Corporation", "MSFT"],
    target_entity="Microsoft",
    merge_strategy={
        "description": "concatenate",
        "source_id": "join_unique"
    },
    target_entity_data={
        "entity_type": "ORGANIZATION",
    }
)

When merging entities:

• All relationships from source entities are redirected to the target entity
• Duplicate relationships are intelligently merged
• Self-relationships (loops) are prevented
• Source entities are removed after merging
• Relationship attributes are merged, and each resulting weight is the larger of every input weight and the distinct merged real-source count

Troubleshooting

Common Initialization Errors

1. AttributeError: __aenter__

   • Cause: Storage backends not initialized
   • Solution: Call await rag.initialize_storages() after creating the LightRAG instance

2. KeyError: 'history_messages'

   • Cause: Pipeline status not initialized
   • Solution: Call await rag.initialize_storages() after creating the LightRAG instance

3. Both errors in sequence

   • Solution: Always follow this pattern:
   pythonCopy

   rag = LightRAG(...)
   await rag.initialize_storages()
   

Model Switching Issues

When switching between different embedding models, you must clear the data directory to avoid errors. The only file you may want to preserve is kv_store_llm_response_cache.json if you wish to retain the LLM cache.