🧩 Embedding models - Mem0

Overview

Embedchain supports several embedding models from the following providers:

OpenAI

To use OpenAI embedding function, you have to set the OPENAI_API_KEY environment variable. You can obtain the OpenAI API key from the OpenAI Platform.

Once you have obtained the key, you can use it like this:

python

import os
from embedchain import App

os.environ['OPENAI_API_KEY'] = 'xxx'

# load embedding model configuration from config.yaml file
app = App.from_config(config_path="config.yaml")

app.add("https://en.wikipedia.org/wiki/OpenAI")
app.query("What is OpenAI?")

yaml

embedder:
  provider: openai
  config:
    model: 'text-embedding-3-small'

</CodeGroup>

OpenAI announced two new embedding models: text-embedding-3-small and text-embedding-3-large. Embedchain supports both these models. Below you can find YAML config for both:

yaml

embedder:
  provider: openai
  config:
    model: 'text-embedding-3-small'

yaml

embedder:
  provider: openai
  config:
    model: 'text-embedding-3-large'

</CodeGroup>

Google AI

To use Google AI embedding function, you have to set the GOOGLE_API_KEY environment variable. You can obtain the Google API key from the Google Maker Suite

<CodeGroup> ```python main.py import os from embedchain import App

os.environ["GOOGLE_API_KEY"] = "xxx"

app = App.from_config(config_path="config.yaml")


```yaml config.yaml
embedder:
  provider: google
  config:
    model: 'models/embedding-001'
    task_type: "retrieval_document"
    title: "Embeddings for Embedchain"

</CodeGroup> <Note> For more details regarding the Google AI embedding model, please refer to the [Google AI documentation](https://ai.google.dev/tutorials/python_quickstart#use_embeddings). </Note>

AWS Bedrock

To use AWS Bedrock embedding function, you have to set the AWS environment variable.

<CodeGroup> ```python main.py import os from embedchain import App

os.environ["AWS_ACCESS_KEY_ID"] = "xxx" os.environ["AWS_SECRET_ACCESS_KEY"] = "xxx" os.environ["AWS_REGION"] = "us-west-2"

app = App.from_config(config_path="config.yaml")


```yaml config.yaml
embedder:
  provider: aws_bedrock
  config:
    model: 'amazon.titan-embed-text-v2:0'
    vector_dimension: 1024
    task_type: "retrieval_document"
    title: "Embeddings for Embedchain"

</CodeGroup> <Note> For more details regarding the AWS Bedrock embedding model, please refer to the [AWS Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html). </Note>

Azure OpenAI

To use Azure OpenAI embedding model, you have to set some of the azure openai related environment variables as given in the code block below:

python

import os
from embedchain import App

os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://xxx.openai.azure.com/"
os.environ["AZURE_OPENAI_API_KEY"] = "xxx"
os.environ["OPENAI_API_VERSION"] = "xxx"

app = App.from_config(config_path="config.yaml")

yaml

llm:
  provider: azure_openai
  config:
    model: gpt-35-turbo
    deployment_name: your_llm_deployment_name
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
    stream: false

embedder:
  provider: azure_openai
  config:
    model: text-embedding-ada-002
    deployment_name: you_embedding_model_deployment_name

</CodeGroup>

You can find the list of models and deployment name on the Azure OpenAI Platform.

GPT4ALL

GPT4All supports generating high quality embeddings of arbitrary length documents of text using a CPU optimized contrastively trained Sentence Transformer.

python

from embedchain import App

# load embedding model configuration from config.yaml file
app = App.from_config(config_path="config.yaml")

yaml

llm:
  provider: gpt4all
  config:
    model: 'orca-mini-3b-gguf2-q4_0.gguf'
    temperature: 0.5
    max_tokens: 1000
    top_p: 1
    stream: false

embedder:
  provider: gpt4all

</CodeGroup>

Hugging Face

Hugging Face supports generating embeddings of arbitrary length documents of text using Sentence Transformer library. Example of how to generate embeddings using hugging face is given below:

python

from embedchain import App

# load embedding model configuration from config.yaml file
app = App.from_config(config_path="config.yaml")

yaml

llm:
  provider: huggingface
  config:
    model: 'google/flan-t5-xxl'
    temperature: 0.5
    max_tokens: 1000
    top_p: 0.5
    stream: false

embedder:
  provider: huggingface
  config:
    model: 'sentence-transformers/all-mpnet-base-v2'
    model_kwargs:
        trust_remote_code: True # Only use if you trust your embedder

</CodeGroup>

Vertex AI

Embedchain supports Google's VertexAI embeddings model through a simple interface. You just have to pass the model_name in the config yaml and it would work out of the box.

python

from embedchain import App

# load embedding model configuration from config.yaml file
app = App.from_config(config_path="config.yaml")

yaml

llm:
  provider: vertexai
  config:
    model: 'chat-bison'
    temperature: 0.5
    top_p: 0.5

embedder:
  provider: vertexai
  config:
    model: 'textembedding-gecko'

</CodeGroup>

NVIDIA AI

NVIDIA AI Foundation Endpoints let you quickly use NVIDIA's AI models, such as Mixtral 8x7B, Llama 2 etc, through our API. These models are available in the NVIDIA NGC catalog, fully optimized and ready to use on NVIDIA's AI platform. They are designed for high speed and easy customization, ensuring smooth performance on any accelerated setup.

Usage

In order to use embedding models and LLMs from NVIDIA AI, create an account on NVIDIA NGC Service.

Generate an API key from their dashboard. Set the API key as NVIDIA_API_KEY environment variable. Note that the NVIDIA_API_KEY will start with nvapi-.

Below is an example of how to use LLM model and embedding model from NVIDIA AI:

python

import os
from embedchain import App

os.environ['NVIDIA_API_KEY'] = 'nvapi-xxxx'

config = {
    "app": {
        "config": {
            "id": "my-app",
        },
    },
    "llm": {
        "provider": "nvidia",
        "config": {
            "model": "nemotron_steerlm_8b",
        },
    },
    "embedder": {
        "provider": "nvidia",
        "config": {
            "model": "nvolveqa_40k",
            "vector_dimension": 1024,
        },
    },
}

app = App.from_config(config=config)

app.add("https://www.forbes.com/profile/elon-musk")
answer = app.query("What is the net worth of Elon Musk today?")
# Answer: The net worth of Elon Musk is subject to fluctuations based on the market value of his holdings in various companies.
# As of March 1, 2024, his net worth is estimated to be approximately $210 billion. However, this figure can change rapidly due to stock market fluctuations and other factors.
# Additionally, his net worth may include other assets such as real estate and art, which are not reflected in his stock portfolio.

</CodeGroup>

Cohere

To use embedding models and LLMs from COHERE, create an account on COHERE.

Generate an API key from their dashboard. Set the API key as COHERE_API_KEY environment variable.

Once you have obtained the key, you can use it like this:

python

import os
from embedchain import App

os.environ['COHERE_API_KEY'] = 'xxx'

# load embedding model configuration from config.yaml file
app = App.from_config(config_path="config.yaml")

yaml

embedder:
  provider: cohere
  config:
    model: 'embed-english-light-v3.0'

</CodeGroup>

Cohere has few embedding models: embed-english-v3.0, embed-multilingual-v3.0, embed-multilingual-light-v3.0, embed-english-v2.0, embed-english-light-v2.0 and embed-multilingual-v2.0. Embedchain supports all these models. Below you can find YAML config for all:

yaml

embedder:
  provider: cohere
  config:
    model: 'embed-english-v3.0'
    vector_dimension: 1024

yaml

embedder:
  provider: cohere
  config:
    model: 'embed-multilingual-v3.0'
    vector_dimension: 1024

yaml

embedder:
  provider: cohere
  config:
    model: 'embed-multilingual-light-v3.0'
    vector_dimension: 384

yaml

embedder:
  provider: cohere
  config:
    model: 'embed-english-v2.0'
    vector_dimension: 4096

yaml

embedder:
  provider: cohere
  config:
    model: 'embed-english-light-v2.0'
    vector_dimension: 1024

yaml

embedder:
  provider: cohere
  config:
    model: 'embed-multilingual-v2.0'
    vector_dimension: 768

</CodeGroup>

Ollama

Ollama enables the use of embedding models, allowing you to generate high-quality embeddings directly on your local machine. Make sure to install Ollama and keep it running before using the embedding model.

You can find the list of models at Ollama Embedding Models.

Below is an example of how to use embedding model Ollama:

python

import os
from embedchain import App

# load embedding model configuration from config.yaml file
app = App.from_config(config_path="config.yaml")

yaml

embedder:
  provider: ollama
  config:
    model: 'all-minilm:latest'

</CodeGroup>

Clarifai

Install related dependencies using the following command:

bash

pip install --upgrade 'embedchain[clarifai]'

set the CLARIFAI_PAT as environment variable which you can find in the security page. Optionally you can also pass the PAT key as parameters in LLM/Embedder class.

Now you are all set with exploring Embedchain.

python

import os
from embedchain import App

os.environ["CLARIFAI_PAT"] = "XXX"

# load llm and embedder configuration from config.yaml file
app = App.from_config(config_path="config.yaml")

#Now let's add some data.
app.add("https://www.forbes.com/profile/elon-musk")

#Query the app
response = app.query("what college degrees does elon musk have?")

Head to Clarifai Platform to explore all the State of the Art embedding models available to use. For passing LLM model inference parameters use model_kwargs argument in the config file. Also you can use api_key argument to pass CLARIFAI_PAT in the config.

yaml

llm:
 provider: clarifai
 config:
   model: "https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct"
   model_kwargs:
     temperature: 0.5
     max_tokens: 1000  
embedder:
 provider: clarifai
 config:
   model: "https://clarifai.com/clarifai/main/models/BAAI-bge-base-en-v15"

</CodeGroup>