ContextQMD
Back to Llama Index

Accessing/Customizing Prompts within Higher-Level Modules

docs/examples/prompts/prompt_mixin.ipynb

0.14.216.0 KB
Original Source

<a href="https://colab.research.google.com/github/run-llama/llama_index/blob/main/docs/examples/prompts/prompt_mixin.ipynb" target="_parent"></a>

Accessing/Customizing Prompts within Higher-Level Modules

LlamaIndex contains a variety of higher-level modules (query engines, response synthesizers, retrievers, etc.), many of which make LLM calls + use prompt templates.

This guide shows how you can 1) access the set of prompts for any module (including nested) with get_prompts, and 2) update these prompts easily with update_prompts.

If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

python
%pip install llama-index
python
import os

os.environ["OPENAI_API_KEY"] = "sk-..."
python
from llama_index.core import Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

# Set the default embedding model and LLM
Settings.embed_model = OpenAIEmbedding(model_name="text-embedding-3-small")
Settings.llm = OpenAI(model="gpt-4o-mini")

Setup: Load Data, Build Index, and Get Query Engine

Here we build a vector index over a toy dataset (PG's essay), and access the query engine.

The query engine is a simple RAG pipeline consisting of top-k retrieval + LLM synthesis.

Download Data

python
!mkdir -p 'data/paul_graham/'
!wget 'https://raw.githubusercontent.com/run-llama/llama_index/main/docs/examples/data/paul_graham/paul_graham_essay.txt' -O 'data/paul_graham/paul_graham_essay.txt'
python
from llama_index.core import SimpleDirectoryReader

# load documents
documents = SimpleDirectoryReader("./data/paul_graham/").load_data()
python
from llama_index.core import VectorStoreIndex

index = VectorStoreIndex.from_documents(documents)
python
query_engine = index.as_query_engine(response_mode="tree_summarize")
python
from IPython.display import Markdown, display


# define prompt viewing function
def display_prompt_dict(prompts_dict):
    for k, p in prompts_dict.items():
        text_md = f"**Prompt Key**: {k}
" f"**Text:** 
"
        display(Markdown(text_md))
        print(p.get_template())
        display(Markdown("

"))

Accessing Prompts

Here we get the prompts from the query engine. Note that all prompts are returned, including ones used in sub-modules in the query engine. This allows you to centralize a view of these prompts!

python
prompts_dict = query_engine.get_prompts()
python
display_prompt_dict(prompts_dict)

Checking get_prompts on Response Synthesizer

You can also call get_prompts on the underlying response synthesizer, where you'll see the same list.

python
prompts_dict = query_engine.response_synthesizer.get_prompts()
display_prompt_dict(prompts_dict)

Checking get_prompts with a different response synthesis strategy

Here we try the default compact method.

We'll see that the set of templates used are different; a QA template and a refine template.

python
# set Logging to DEBUG for more detailed outputs
query_engine = index.as_query_engine(response_mode="compact")
python
prompts_dict = query_engine.get_prompts()
display_prompt_dict(prompts_dict)

Put into query engine, get response

python
response = query_engine.query("What did the author do growing up?")
print(str(response))

Customize the prompt

You can also update/customize the prompts with the update_prompts function. Pass in arg values with the keys equal to the keys you see in the prompt dictionary.

Here we'll change the summary prompt to use Shakespeare.

python
from llama_index.core import PromptTemplate

# reset
query_engine = index.as_query_engine(response_mode="tree_summarize")

# shakespeare!
new_summary_tmpl_str = (
    "Context information is below.\n"
    "---------------------\n"
    "{context_str}\n"
    "---------------------\n"
    "Given the context information and not prior knowledge, "
    "answer the query in the style of a Shakespeare play.\n"
    "Query: {query_str}\n"
    "Answer: "
)
new_summary_tmpl = PromptTemplate(new_summary_tmpl_str)
python
query_engine.update_prompts(
    {"response_synthesizer:summary_template": new_summary_tmpl}
)
python
prompts_dict = query_engine.get_prompts()
python
display_prompt_dict(prompts_dict)
python
response = query_engine.query("What did the author do growing up?")
print(str(response))

Accessing Prompts from Other Modules

Here we take a look at some other modules: query engines, routers/selectors, evaluators, and others.

python
from llama_index.core.agent.workflow import ReActAgent
from llama_index.core.selectors import LLMMultiSelector
from llama_index.core.evaluation import FaithfulnessEvaluator, DatasetGenerator
from llama_index.core.postprocessor import LLMRerank

Analyze Prompts: ReActAgent

python
agent = ReActAgent(tools=[])
python
prompts_dict = agent.get_prompts()
display_prompt_dict(prompts_dict)

Analyze Prompts: FLARE Query Engine

python
flare_query_engine = FLAREInstructQueryEngine(query_engine)
python
prompts_dict = flare_query_engine.get_prompts()
display_prompt_dict(prompts_dict)

Analyze Prompts: LLMMultiSelector

python
from llama_index.core.selectors import LLMSingleSelector

selector = LLMSingleSelector.from_defaults()
python
prompts_dict = selector.get_prompts()
display_prompt_dict(prompts_dict)

Analyze Prompts: FaithfulnessEvaluator

python
evaluator = FaithfulnessEvaluator()
python
prompts_dict = evaluator.get_prompts()
display_prompt_dict(prompts_dict)

Analyze Prompts: DatasetGenerator

python
dataset_generator = DatasetGenerator.from_documents(documents)
python
prompts_dict = dataset_generator.get_prompts()
display_prompt_dict(prompts_dict)

Analyze Prompts: LLMRerank

python
llm_rerank = LLMRerank()
python
prompts_dict = dataset_generator.get_prompts()
display_prompt_dict(prompts_dict)