Simple Vector Store - Async Index Creation

If you're opening this Notebook on colab, you will probably need to install LlamaIndex 🦙.

python

%pip install llama-index-readers-wikipedia

python

!pip install llama-index

python

import time

# Helps asyncio run within Jupyter
import nest_asyncio

nest_asyncio.apply()

# My OpenAI Key
import os

os.environ["OPENAI_API_KEY"] = "[YOUR_API_KEY]"

python

from llama_index.core import VectorStoreIndex, download_loader

from llama_index.readers.wikipedia import WikipediaReader

loader = WikipediaReader()
documents = loader.load_data(
    pages=[
        "Berlin",
        "Santiago",
        "Moscow",
        "Tokyo",
        "Jakarta",
        "Cairo",
        "Bogota",
        "Shanghai",
        "Damascus",
    ]
)

python

len(documents)

9 Wikipedia articles downloaded as documents

python

start_time = time.perf_counter()
index = VectorStoreIndex.from_documents(documents)
duration = time.perf_counter() - start_time
print(duration)

Standard index creation took 7.69 seconds

python

start_time = time.perf_counter()
index = VectorStoreIndex(documents, use_async=True)
duration = time.perf_counter() - start_time
print(duration)

Async index creation took 2.37 seconds

python

query_engine = index.as_query_engine()
query_engine.query("What is the etymology of Jakarta?")