Setting up MongoDB Atlas as the Datastore Provider

MongoDB Atlas is a multi-cloud database service made by the same people that build MongoDB. Atlas simplifies deploying and managing your databases while offering the versatility you need to build resilient and performant global applications on the cloud providers of your choice.

You can perform semantic search on data in your Atlas cluster running MongoDB v6.0.11, v7.0.2, or later using Atlas Vector Search. You can store vector embeddings for any kind of data along with other data in your collection on the Atlas cluster.

In the section, we set up a cluster, a database, test it, and finally create an Atlas Vector Search Index.

Deploy a Cluster

Follow the Getting-Started documentation to create an account, deploy an Atlas cluster, and connect to a database.

Retrieve the URI used by Python to connect to the Cluster

When you deploy the ChatGPT Retrieval App, this will be stored as the environment variable: MONGODB_URI
It will look something like the following. The username and password, if not provided, can be configured in Database Access under Security in the left panel.

export MONGODB_URI="mongodb+srv://<username>:<password>@chatgpt-retrieval-plugin.zeatahb.mongodb.net/?retryWrites=true&w=majority"

There are a number of ways to navigate the Atlas UI. Keep your eye out for "Connect" and "driver".

On the left panel, navigate and click 'Database' under DEPLOYMENT. Click the Connect button that appears, then Drivers. Select Python. (Have no concern for the version. This is the PyMongo, not Python, version.) Once you have got the Connect Window open, you will see an instruction to pip install pymongo. You will also see a connection string. This is the uri that a pymongo.MongoClient uses to connect to the Database.

Test the connection

Atlas provides a simple check. Once you have your uri and pymongo installed, try the following in a python console.

from pymongo.mongo_client import MongoClient
client = MongoClient(uri)  # Create a new client and connect to the server
try:
    client.admin.command('ping')  # Send a ping to confirm a successful connection
    print("Pinged your deployment. You successfully connected to MongoDB!")
except Exception as e:
    print(e)

Troubleshooting

• You can edit a Database's users and passwords on the 'Database Access' page, under Security.
• Remember to add your IP address. (Try curl -4 ifconfig.co)

Create a Database and Collection

As mentioned, Vector Databases provide two functions. In addition to being the data store, they provide very efficient search based on natural language queries. With Vector Search, one will index and query data with a powerful vector search algorithm using "Hierarchical Navigable Small World (HNSW) graphs to find vector similarity.

The indexing runs beside the data as a separate service asynchronously. The Search index monitors changes to the Collection that it applies to. Subsequently, one need not upload the data first. We will create an empty collection now, which will simplify setup in the example notebook.

Back in the UI, navigate to the Database Deployments page by clicking Database on the left panel. Click the "Browse Collections" and then "+ Create Database" buttons. This will open a window where you choose Database and Collection names. (No additional preferences.) Remember these values as they will be as the environment variables, MONGODB_DATABASE and MONGODB_COLLECTION. Though arbitrary, we suggest "SQUAD" and "Beyonce" as these describe the data that we will use in our example Jupyter Notebook.

Set Datastore Environment Variables

To establish a connection to the MongoDB Cluster, Database, and Collection, plus create a Vector Search Index, define the following environment variables. You can confirm that the required ones have been set like this: assert "MONGODB_URI" in os.environ

IMPORTANT It is crucial that the choices are consistent between setup in Atlas and Python environment(s).

The following will also be required to authenticate with OpenAI and Plugin APIs.

Create an Atlas Vector Search Index

The final step to configure MongoDB as the Datastore is to create a Vector Search Index. The procedure is described here.

Under Services on the left panel, choose Atlas Search > Create Search Index > Atlas Vector Search JSON Editor.

The Plugin expects an index definition like the following. To begin, choose numDimensions: 1536 along with the suggested EMBEDDING variables above. You can experiment with these later.

{
  "fields": [
    {
      "numDimensions": 1536,
      "path": "embedding",
      "similarity": "cosine",
      "type": "vector"
    }
  ]
}

Running MongoDB Integration Tests

In addition to the Jupyter Notebooks in examples/, a suite of integration tests is available to verify the MongoDB integration. The test suite needs the cluster up and running, and the environment variables defined above.

Then, launch the test suite with this command:

pytest ./tests/datastore/providers/mongodb_atlas/test_mongodb_datastore.py