docs/decisions/0050-updated-vector-store-design.md
Semantic Kernel has a collection of connectors to popular Vector databases e.g. Azure AI Search, Chroma, Milvus, ... Each Memory connector implements a memory abstraction defined by Semantic Kernel and allows developers to easily integrate Vector databases into their applications. The current abstractions are experimental and the purpose of this ADR is to progress the design of the abstractions so that they can graduate to non experimental status.
IMemoryStore interface has four responsibilities with different cardinalities. Some are schema aware and others schema agnostic.IMemoryStore interface only supports a fixed schema for data storage, retrieval and search, which limits its usability by customers with existing data sets.IMemoryStore implementations are opinionated around key encoding / decoding and collection name sanitization, which limits its usability by customers with existing data sets.Responsibilities:
| Functional Area | Cardinality | Significance to Semantic Kernel |
|---|---|---|
| Collection/Index create | An implementation per store type and model | Valuable when building a store and adding data |
| Collection/Index list names, exists and delete | An implementation per store type | Valuable when building a store and adding data |
| Data Storage and Retrieval | An implementation per store type | Valuable when building a store and adding data |
| Vector Search | An implementation per store type, model and search type | Valuable for many scenarios including RAG, finding contradictory facts based on user input, finding similar memories to merge, etc. |
interface IMemoryStore
{
// Collection / Index Management
Task CreateCollectionAsync(string collectionName, CancellationToken cancellationToken = default);
IAsyncEnumerable<string> GetCollectionsAsync(CancellationToken cancellationToken = default);
Task<bool> DoesCollectionExistAsync(string collectionName, CancellationToken cancellationToken = default);
Task DeleteCollectionAsync(string collectionName, CancellationToken cancellationToken = default);
// Data Storage and Retrieval
Task<string> UpsertAsync(string collectionName, MemoryRecord record, CancellationToken cancellationToken = default);
IAsyncEnumerable<string> UpsertBatchAsync(string collectionName, IEnumerable<MemoryRecord> records, CancellationToken cancellationToken = default);
Task<MemoryRecord?> GetAsync(string collectionName, string key, bool withEmbedding = false, CancellationToken cancellationToken = default);
IAsyncEnumerable<MemoryRecord> GetBatchAsync(string collectionName, IEnumerable<string> keys, bool withVectors = false, CancellationToken cancellationToken = default);
Task RemoveAsync(string collectionName, string key, CancellationToken cancellationToken = default);
Task RemoveBatchAsync(string collectionName, IEnumerable<string> keys, CancellationToken cancellationToken = default);
// Vector Search
IAsyncEnumerable<(MemoryRecord, double)> GetNearestMatchesAsync(
string collectionName,
ReadOnlyMemory<float> embedding,
int limit,
double minRelevanceScore = 0.0,
bool withVectors = false,
CancellationToken cancellationToken = default);
Task<(MemoryRecord, double)?> GetNearestMatchAsync(
string collectionName,
ReadOnlyMemory<float> embedding,
double minRelevanceScore = 0.0,
bool withEmbedding = false,
CancellationToken cancellationToken = default);
}
IMemoryStore should be split into different interfaces, so that schema aware and schema agnostic operations are separated.The separation between collection/index management and record management.
---
title: SK Collection/Index and record management
---
classDiagram
note for IVectorRecordStore "Can manage records for any scenario"
note for IVectorCollectionCreate "Can create collections and\nindexes"
note for IVectorCollectionNonSchema "Can retrieve/delete any collections and\nindexes"
namespace SKAbstractions{
class IVectorCollectionCreate{
<<interface>>
+CreateCollection
}
class IVectorCollectionNonSchema{
<<interface>>
+GetCollectionNames
+CollectionExists
+DeleteCollection
}
class IVectorRecordStore~TModel~{
<<interface>>
+Upsert(TModel record) string
+UpsertBatch(TModel record) string
+Get(string key) TModel
+GetBatch(string[] keys) TModel[]
+Delete(string key)
+DeleteBatch(string[] keys)
}
}
namespace AzureAIMemory{
class AzureAISearchVectorCollectionCreate{
}
class AzureAISearchVectorCollectionNonSchema{
}
class AzureAISearchVectorRecordStore{
}
}
namespace RedisMemory{
class RedisVectorCollectionCreate{
}
class RedisVectorCollectionNonSchema{
}
class RedisVectorRecordStore{
}
}
IVectorCollectionCreate <|-- AzureAISearchVectorCollectionCreate
IVectorCollectionNonSchema <|-- AzureAISearchVectorCollectionNonSchema
IVectorRecordStore <|-- AzureAISearchVectorRecordStore
IVectorCollectionCreate <|-- RedisVectorCollectionCreate
IVectorCollectionNonSchema <|-- RedisVectorCollectionNonSchema
IVectorRecordStore <|-- RedisVectorRecordStore
How to use your own schema with core sk functionality.
---
title: Chat History Break Glass
---
classDiagram
note for IVectorRecordStore "Can manage records\nfor any scenario"
note for IVectorCollectionCreate "Can create collections\nan dindexes"
note for IVectorCollectionNonSchema "Can retrieve/delete any\ncollections and indexes"
note for CustomerHistoryVectorCollectionCreate "Creates history collections and indices\nusing Customer requirements"
note for CustomerHistoryVectorRecordStore "Decorator class for IVectorRecordStore that maps\nbetween the customer model to our model"
namespace SKAbstractions{
class IVectorCollectionCreate{
<<interface>>
+CreateCollection
}
class IVectorCollectionNonSchema{
<<interface>>
+GetCollectionNames
+CollectionExists
+DeleteCollection
}
class IVectorRecordStore~TModel~{
<<interface>>
+Upsert(TModel record) string
+Get(string key) TModel
+Delete(string key) string
}
class ISemanticTextMemory{
<<interface>>
+SaveInformationAsync()
+SaveReferenceAsync()
+GetAsync()
+DeleteAsync()
+SearchAsync()
+GetCollectionsAsync()
}
}
namespace CustomerProject{
class CustomerHistoryModel{
+string text
+float[] vector
+Dictionary~string, string~ properties
}
class CustomerHistoryVectorCollectionCreate{
+CreateCollection
}
class CustomerHistoryVectorRecordStore{
-IVectorRecordStore~CustomerHistoryModel~ _store
+Upsert(ChatHistoryModel record) string
+Get(string key) ChatHistoryModel
+Delete(string key) string
}
}
namespace SKCore{
class SemanticTextMemory{
-IVectorRecordStore~ChatHistoryModel~ _VectorRecordStore
-IMemoryCollectionService _collectionsService
-ITextEmbeddingGenerationService _embeddingGenerationService
}
class ChatHistoryPlugin{
-ISemanticTextMemory memory
}
class ChatHistoryModel{
+string message
+float[] embedding
+Dictionary~string, string~ metadata
}
}
IVectorCollectionCreate <|-- CustomerHistoryVectorCollectionCreate
IVectorRecordStore <|-- CustomerHistoryVectorRecordStore
IVectorRecordStore <.. CustomerHistoryVectorRecordStore
CustomerHistoryModel <.. CustomerHistoryVectorRecordStore
ChatHistoryModel <.. CustomerHistoryVectorRecordStore
ChatHistoryModel <.. SemanticTextMemory
IVectorRecordStore <.. SemanticTextMemory
IVectorCollectionCreate <.. SemanticTextMemory
ISemanticTextMemory <.. ChatHistoryPlugin
A comparison of the different ways in which stores implement storage capabilities to help drive decisions:
| Feature | Azure AI Search | Weaviate | Redis | Chroma | FAISS | Pinecone | LLamaIndex | PostgreSql | Qdrant | Milvus |
|---|---|---|---|---|---|---|---|---|---|---|
| Get Item Support | Y | Y | Y | Y | Y | Y | Y | Y | ||
| Batch Operation Support | Y | Y | Y | Y | Y | Y | ||||
| Per Item Results for Batch Operations | Y | Y | Y | N | N | |||||
| Keys of upserted records | Y | Y | N<sup>3</sup> | N<sup>3</sup> | N<sup>3</sup> | Y | ||||
| Keys of removed records | Y | N<sup>3</sup> | N | N | N<sup>3</sup> | |||||
| Retrieval field selection for gets | Y | Y<sup>4<sup> | P<sup>2</sup> | N | Y | Y | Y | |||
| Include/Exclude Embeddings for gets | P<sup>1</sup> | Y | Y<sup>4,1<sup> | Y | N | P<sup>1</sup> | Y | N | ||
| Failure reasons when batch partially fails | Y | Y | Y | N | N | |||||
| Is Key separate from data | N | Y | Y | Y | Y | N | Y | N | ||
| Can Generate Ids | N | Y | N | N | Y | Y | N | Y | ||
| Can Generate Embedding | Not Available Via API yet | Y | N | Client Side Abstraction | N |
Footnotes:
Footnotes:
| Feature | Azure AI Search | Weaviate | Redis | Chroma | FAISS | Pinecone | LLamaIndex | PostgreSql | Qdrant | Milvus |
|---|---|---|---|---|---|---|---|---|---|---|
| Index allows text search | Y | Y | Y | Y (On Metadata by default) | Only in combination with Vector | Y (with TSVECTOR field) | Y | Y | ||
| Text search query format | Simple or Full Lucene | wildcard | wildcard & fuzzy | contains & not contains | Text only | wildcard & binary operators | Text only | wildcard | ||
| Multi Field Vector Search Support | Y | N | N (no multi vector support) | N | Unclear due to order by syntax | N | Y | |||
| Targeted Multi Field Text Search Support | Y | Y | Y | N (only on document) | N | Y | Y | Y | ||
| Vector per Vector Field for Search | Y | N/A | N/A | N/A | N/A | N/A | ||||
| Separate text search query from vectors | Y | Y | Y | Y | Y | Y | Y | Y | ||
| Allows filtering | Y | Y | Y (on TAG) | Y (On Metadata by default) | Y | Y | Y | Y | ||
| Allows filter grouping | Y (Odata) | Y | Y | Y | Y | Y | Y | |||
| Allows scalar index field setup | Y | Y | Y | N | Y | Y | Y | Y | ||
| Requires scalar index field setup to filter | Y | Y | Y | N | N (on by default for all) | N | N | N (can filter without index) |
Mapping between data models and the storage models can also require custom logic depending on the type of data model and storage model involved.
I'm therefore proposing that we allow mappers to be injectable for each VectorStoreCollection instance. The interfaces for these would vary depending
on the storage models used by each vector store and any unique capabilities that each vector store may have, e.g. qdrant can operate in single or
multiple named vector modes, which means the mapper needs to know whether to set a single vector or fill a vector map.
In addition to this, we should build first party mappers for each of the vector stores, which will cater for built in, generic models or use metadata to perform the mapping.
The different stores vary in many ways around how data is organized.
I'm proposing that we allow two ways in which to provide the information required to map data between the consumer data model and storage data model. First is a set of configuration objects that capture the types of each field. Second would be a set of attributes that can be used to decorate the model itself and can be converted to the configuration objects, allowing a single execution path. Additional configuration properties can easily be added for each type of field as required, e.g. IsFilterable or IsFullTextSearchable, allowing us to also create an index from the provided configuration.
I'm also proposing that even though similar attributes already exist in other systems, e.g. System.ComponentModel.DataAnnotations.KeyAttribute, we create our own. We will likely require additional properties on all these attributes that are not currently supported on the existing attributes, e.g. whether a field is or should be filterable. Requiring users to switch to new attributes later will be disruptive.
Here is what the attributes would look like, plus a sample use case.
sealed class VectorStoreRecordKeyAttribute : Attribute
{
}
sealed class VectorStoreRecordDataAttribute : Attribute
{
public bool HasEmbedding { get; set; }
public string EmbeddingPropertyName { get; set; }
}
sealed class VectorStoreRecordVectorAttribute : Attribute
{
}
public record HotelInfo(
[property: VectorStoreRecordKey, JsonPropertyName("hotel-id")] string HotelId,
[property: VectorStoreRecordData, JsonPropertyName("hotel-name")] string HotelName,
[property: VectorStoreRecordData(HasEmbedding = true, EmbeddingPropertyName = "DescriptionEmbeddings"), JsonPropertyName("description")] string Description,
[property: VectorStoreRecordVector, JsonPropertyName("description-embeddings")] ReadOnlyMemory<float>? DescriptionEmbeddings);
Here is what the configuration objects would look like.
abstract class VectorStoreRecordProperty(string propertyName);
sealed class VectorStoreRecordKeyProperty(string propertyName): Field(propertyName)
{
}
sealed class VectorStoreRecordDataProperty(string propertyName): Field(propertyName)
{
bool HasEmbedding;
string EmbeddingPropertyName;
}
sealed class VectorStoreRecordVectorProperty(string propertyName): Field(propertyName)
{
}
sealed class VectorStoreRecordDefinition
{
IReadOnlyList<VectorStoreRecordProperty> Properties;
}
All methods currently existing on IMemoryStore will be ported to new interfaces, but in places I am proposing that we make changes to improve consistency and scalability.
RemoveAsync and RemoveBatchAsync renamed to DeleteAsync and DeleteBatchAsync, since record are actually deleted, and this also matches the verb used for collections.GetCollectionsAsync renamed to GetCollectionNamesAsync, since we are only retrieving names and no other information about collections.DoesCollectionExistAsync renamed to CollectionExistsAsync since this is shorter and is more commonly used in other apis.| Criteria | Current SK Implementation | Proposed SK Implementation | Spring AI | LlamaIndex | Langchain |
|---|---|---|---|---|---|
| Support for Custom Schemas | N | Y | N | N | N |
| Naming of store | MemoryStore | VectorStore, VectorStoreCollection | VectorStore | VectorStore | VectorStore |
| MultiVector support | N | Y | N | N | N |
| Support Multiple Collections via SDK params | Y | Y | N (via app config) | Y | Y |
From GitHub Issue:
interface IVectorRecordStore<TRecord>
{
Task CreateCollectionAsync(CollectionCreateConfig collectionConfig, CancellationToken cancellationToken = default);
IAsyncEnumerable<string> ListCollectionNamesAsync(CancellationToken cancellationToken = default);
Task<bool> CollectionExistsAsync(string name, CancellationToken cancellationToken = default);
Task DeleteCollectionAsync(string name, CancellationToken cancellationToken = default);
Task UpsertAsync(TRecord data, CancellationToken cancellationToken = default);
IAsyncEnumerable<string> UpsertBatchAsync(IEnumerable<TRecord> dataSet, CancellationToken cancellationToken = default);
Task<TRecord> GetAsync(string key, bool withEmbedding = false, CancellationToken cancellationToken = default);
IAsyncEnumerable<TRecord> GetBatchAsync(IEnumerable<string> keys, bool withVectors = false, CancellationToken cancellationToken = default);
Task DeleteAsync(string key, CancellationToken cancellationToken = default);
Task DeleteBatchAsync(IEnumerable<string> keys, CancellationToken cancellationToken = default);
}
class AzureAISearchVectorRecordStore<TRecord>(
Azure.Search.Documents.Indexes.SearchIndexClient client,
Schema schema): IVectorRecordStore<TRecord>;
class WeaviateVectorRecordStore<TRecord>(
WeaviateClient client,
Schema schema): IVectorRecordStore<TRecord>;
class RedisVectorRecordStore<TRecord>(
StackExchange.Redis.IDatabase database,
Schema schema): IVectorRecordStore<TRecord>;
interface IVectorCollectionStore
{
virtual Task CreateChatHistoryCollectionAsync(string name, CancellationToken cancellationToken = default);
virtual Task CreateSemanticCacheCollectionAsync(string name, CancellationToken cancellationToken = default);
IAsyncEnumerable<string> ListCollectionNamesAsync(CancellationToken cancellationToken = default);
Task<bool> CollectionExistsAsync(string name, CancellationToken cancellationToken = default);
Task DeleteCollectionAsync(string name, CancellationToken cancellationToken = default);
}
class AzureAISearchVectorCollectionStore: IVectorCollectionStore;
class RedisVectorCollectionStore: IVectorCollectionStore;
class WeaviateVectorCollectionStore: IVectorCollectionStore;
// Customers can inherit from our implementations and replace just the creation scenarios to match their schemas.
class CustomerCollectionStore: AzureAISearchVectorCollectionStore, IVectorCollectionStore;
// We can also create implementations that create indices based on an MLIndex specification.
class MLIndexAzureAISearchVectorCollectionStore(MLIndex mlIndexSpec): AzureAISearchVectorCollectionStore, IVectorCollectionStore;
interface IVectorRecordStore<TRecord>
{
Task<TRecord?> GetAsync(string key, GetRecordOptions? options = default, CancellationToken cancellationToken = default);
Task DeleteAsync(string key, DeleteRecordOptions? options = default, CancellationToken cancellationToken = default);
Task<string> UpsertAsync(TRecord record, UpsertRecordOptions? options = default, CancellationToken cancellationToken = default);
}
class AzureAISearchVectorRecordStore<TRecord>(): IVectorRecordStore<TRecord>;
Vector store same as option 2 so not repeated for brevity.
interface IVectorCollectionCreate
{
virtual Task CreateCollectionAsync(string name, CancellationToken cancellationToken = default);
}
// Implement a generic version of create that takes a configuration that should work for 80% of cases.
class AzureAISearchConfiguredVectorCollectionCreate(CollectionCreateConfig collectionConfig): IVectorCollectionCreate;
// Allow custom implementations of create for break glass scenarios for outside the 80% case.
class AzureAISearchChatHistoryVectorCollectionCreate: IVectorCollectionCreate;
class AzureAISearchSemanticCacheVectorCollectionCreate: IVectorCollectionCreate;
// Customers can create their own creation scenarios to match their schemas, but can continue to use our get, does exist and delete class.
class CustomerChatHistoryVectorCollectionCreate: IVectorCollectionCreate;
interface IVectorCollectionNonSchema
{
IAsyncEnumerable<string> ListCollectionNamesAsync(CancellationToken cancellationToken = default);
Task<bool> CollectionExistsAsync(string name, CancellationToken cancellationToken = default);
Task DeleteCollectionAsync(string name, CancellationToken cancellationToken = default);
}
class AzureAISearchVectorCollectionNonSchema: IVectorCollectionNonSchema;
class RedisVectorCollectionNonSchema: IVectorCollectionNonSchema;
class WeaviateVectorCollectionNonSchema: IVectorCollectionNonSchema;
Variation on option 3.
interface IVectorCollectionCreate
{
virtual Task CreateCollectionAsync(string name, CancellationToken cancellationToken = default);
}
interface IVectorCollectionNonSchema
{
IAsyncEnumerable<string> ListCollectionNamesAsync(CancellationToken cancellationToken = default);
Task<bool> CollectionExistsAsync(string name, CancellationToken cancellationToken = default);
Task DeleteCollectionAsync(string name, CancellationToken cancellationToken = default);
}
// DB Specific NonSchema implementations
class AzureAISearchVectorCollectionNonSchema: IVectorCollectionNonSchema;
class RedisVectorCollectionNonSchema: IVectorCollectionNonSchema;
// Combined Create + NonSchema Interface
interface IVectorCollectionStore: IVectorCollectionCreate, IVectorCollectionNonSchema {}
// Base abstract class that forwards non-create operations to provided implementation.
abstract class VectorCollectionStore(IVectorCollectionNonSchema collectionNonSchema): IVectorCollectionStore
{
public abstract Task CreateCollectionAsync(string name, CancellationToken cancellationToken = default);
public IAsyncEnumerable<string> ListCollectionNamesAsync(CancellationToken cancellationToken = default) { return collectionNonSchema.ListCollectionNamesAsync(cancellationToken); }
public Task<bool> CollectionExistsAsync(string name, CancellationToken cancellationToken = default) { return collectionNonSchema.CollectionExistsAsync(name, cancellationToken); }
public Task DeleteCollectionAsync(string name, CancellationToken cancellationToken = default) { return collectionNonSchema.DeleteCollectionAsync(name, cancellationToken); }
}
// Collections store implementations, that inherit from base class, and just adds the different creation implementations.
class AzureAISearchChatHistoryVectorCollectionStore(AzureAISearchVectorCollectionNonSchema nonSchema): VectorCollectionStore(nonSchema);
class AzureAISearchSemanticCacheVectorCollectionStore(AzureAISearchVectorCollectionNonSchema nonSchema): VectorCollectionStore(nonSchema);
class AzureAISearchMLIndexVectorCollectionStore(AzureAISearchVectorCollectionNonSchema nonSchema): VectorCollectionStore(nonSchema);
// Customer collections store implementation, that uses the base Azure AI Search implementation for get, doesExist and delete, but adds its own creation.
class ContosoProductsVectorCollectionStore(AzureAISearchVectorCollectionNonSchema nonSchema): VectorCollectionStore(nonSchema);
Same as option 3 / 4, plus:
interface IVectorStore : IVectorCollectionStore, IVectorRecordStore
{
}
// Create a static factory that produces one of these, so only the interface is public, not the class.
internal class VectorStore<TRecord>(IVectorCollectionCreate create, IVectorCollectionNonSchema nonSchema, IVectorRecordStore<TRecord> records): IVectorStore
{
}
IVectorStore acts as a factory for IVectorStoreCollection, and any schema agnostic multi-collection operations are kept on IVectorStore.
public interface IVectorStore
{
IVectorStoreCollection<TKey, TRecord> GetCollection<TKey, TRecord>(string name, VectorStoreRecordDefinition? vectorStoreRecordDefinition = null);
IAsyncEnumerable<string> ListCollectionNamesAsync(CancellationToken cancellationToken = default));
}
public interface IVectorStoreCollection<TKey, TRecord>
{
public string Name { get; }
// Collection Operations
Task CreateCollectionAsync();
Task<bool> CreateCollectionIfNotExistsAsync();
Task<bool> CollectionExistsAsync();
Task DeleteCollectionAsync();
// Data manipulation
Task<TRecord?> GetAsync(TKey key, GetRecordOptions? options = default, CancellationToken cancellationToken = default);
IAsyncEnumerable<TRecord> GetBatchAsync(IEnumerable<TKey> keys, GetRecordOptions? options = default, CancellationToken cancellationToken = default);
Task DeleteAsync(TKey key, DeleteRecordOptions? options = default, CancellationToken cancellationToken = default);
Task DeleteBatchAsync(IEnumerable<TKey> keys, DeleteRecordOptions? options = default, CancellationToken cancellationToken = default);
Task<TKey> UpsertAsync(TRecord record, UpsertRecordOptions? options = default, CancellationToken cancellationToken = default);
IAsyncEnumerable<TKey> UpsertBatchAsync(IEnumerable<TRecord> records, UpsertRecordOptions? options = default, CancellationToken cancellationToken = default);
}
Option 1 is problematic on its own, since we have to allow consumers to create custom implementations of collection create for break glass scenarios. With a single interface like this, it will require them to implement many methods that they do not want to change. Options 4 & 5, gives us more flexibility while still preserving the ease of use of an aggregated interface as described in Option 1.
Option 2 doesn't give us the flexibility we need for break glass scenarios, since it only allows certain types of collections to be created. It also means that each time a new collection type is required it introduces a breaking change, so it is not a viable option.
Since collection create and configuration and the possible options vary considerable across different database types, we will need to support an easy to use break glass scenario for collection creation. While we would be able to develop a basic configurable create option, for complex create scenarios users will need to implement their own. We will also need to support multiple create implementations out of the box, e.g. a configuration based option using our own configuration, create implementations that re-create the current model for backward compatibility, create implementations that use other configuration as input, e.g. Azure-ML YAML. Therefore separating create, which may have many implementations, from exists, list and delete, which requires only a single implementation per database type is useful. Option 3 provides us this separation, but Option 4 + 5 builds on top of this, and allows us to combine different implementations together for simpler consumption.
Chosen option: 6
public class AzureAISearchVectorStoreCollection<TRecord> : IVectorStoreCollection<TRecord>
{
...
// On input.
var normalizedCollectionName = this.NormalizeCollectionName(collectionName);
var encodedId = AzureAISearchMemoryRecord.EncodeId(key);
...
// On output.
DecodeId(this.Id)
...
}
new KeyNormalizingAISearchVectorStoreCollection<MyModel>(
"keyField",
new AzureAISearchVectorStoreCollection<MyModel>(...));
public class AzureAISearchVectorStoreCollection<TRecord>(StoreOptions options);
public class StoreOptions
{
public Func<string, string>? EncodeKey { get; init; }
public Func<string, string>? DecodeKey { get; init; }
public Func<string, string>? SanitizeCollectionName { get; init; }
}
If developer wants to change any values they can do so by creating a custom mapper.
Chosen option 3, since it is similar to how we are doing mapper injection and would also work well in python.
Option 1 won't work because if e.g. the data was written using another tool, it may be unlikely that it was encoded using the same mechanism as supported here and therefore this functionality may not be appropriate. The developer should have the ability to not use this functionality or provide their own encoding / decoding behavior.
public class MyVectorStoreCollection()
{
public async Task<TRecord?> GetAsync(string collectionName, string key, GetRecordOptions? options = default, CancellationToken cancellationToken = default);
}
public class MyVectorStoreCollection(string defaultCollectionName)
{
public async Task<TRecord?> GetAsync(string key, GetRecordOptions? options = default, CancellationToken cancellationToken = default);
}
public class MyVectorStoreCollection(string defaultCollectionName)
{
public async Task<TRecord?> GetAsync(string key, GetRecordOptions? options = default, CancellationToken cancellationToken = default);
}
public class GetRecordOptions
{
public string CollectionName { get; init; };
}
Chosen option 2. None of the other options work with the decision outcome of Question 1, since that design requires the VectorStoreCollection to be tied to a single collection instance.
public async Task<TRecord?> GetAsync(string key, GetRecordOptions? options = default, CancellationToken cancellationToken = default)
{
var convertedKey = this.keyType switch
{
KeyType.Int => int.parse(key),
KeyType.GUID => Guid.parse(key)
}
...
}
public async Task<TRecord?> GetAsync(object key, GetRecordOptions? options = default, CancellationToken cancellationToken = default)
{
var convertedKey = this.keyType switch
{
KeyType.Int => key as int,
KeyType.GUID => key as Guid
}
if (convertedKey is null)
{
throw new InvalidOperationException($"The provided key must be of type {this.keyType}")
}
...
}
public async Task<TRecord?> GetAsync(string key, GetRecordOptions? options = default, CancellationToken cancellationToken = default)
{
var convertedKey = this.keyType switch
{
KeyType.Int => int.Parse(key),
KeyType.String => key,
KeyType.GUID => Guid.Parse(key)
}
}
public async Task<TRecord?> GetAsync(int key, GetRecordOptions? options = default, CancellationToken cancellationToken = default)
{
var convertedKey = this.keyType switch
{
KeyType.Int => key,
KeyType.String => key.ToString(),
KeyType.GUID => throw new InvalidOperationException($"The provided key must be convertible to a GUID.")
}
}
public async Task<TRecord?> GetAsync(GUID key, GetRecordOptions? options = default, CancellationToken cancellationToken = default)
{
var convertedKey = this.keyType switch
{
KeyType.Int => throw new InvalidOperationException($"The provided key must be convertible to an int.")
KeyType.String => key.ToString(),
KeyType.GUID => key
}
}
interface IVectorRecordStore<TRecord, TKey>
{
Task<TRecord?> GetAsync(TKey key, GetRecordOptions? options = default, CancellationToken cancellationToken = default);
}
class AzureAISearchVectorRecordStore<TRecord, TKey>: IVectorRecordStore<TRecord, TKey>
{
public AzureAISearchVectorRecordStore()
{
// Check if TKey matches the type of the field marked as a key on TRecord and throw if they don't match.
// Also check if keytype is one of the allowed types for Azure AI Search and throw if it isn't.
}
}
Chosen option 4, since it is forwards compatible with any complex key types we may need to support but still allows each implementation to hardcode allowed key types if the vector db only supports certain key types.
interface IVectorDBRecordService {}
interface IVectorDBCollectionUpdateService {}
interface IVectorDBCollectionCreateService {}
interface IMemoryRecordService {}
interface IMemoryCollectionUpdateService {}
interface IMemoryCollectionCreateService {}
interface IVectorRecordStore<TRecord> {}
interface IVectorCollectionNonSchema {}
interface IVectorCollectionCreate {}
interface IVectorCollectionStore {}: IVectorCollectionCreate, IVectorCollectionNonSchema
interface IVectorStore<TRecord> {}: IVectorCollectionStore, IVectorRecordStore<TRecord>
interface IVectorStore
{
IVectorStoreCollection GetCollection()
}
interface IVectorStoreCollection
{
Get()
Delete()
Upsert()
}
Chosen option 4. The word memory is broad enough to encompass any data, so using it seems arbitrary. All competitors are using the term vector store, so using something similar is good for recognition. Option 4 also matches our design as chosen in question 1.
class CacheEntryModel(string prompt, string result, ReadOnlyMemory<float> promptEmbedding);
class SemanticTextMemory(IVectorStore configuredVectorStore, VectorStoreRecordDefinition? vectorStoreRecordDefinition): ISemanticTextMemory
{
public async Task SaveInformation<TDataType>(string collectionName, TDataType record)
{
var collection = vectorStore.GetCollection<TDataType>(collectionName, vectorStoreRecordDefinition);
if (!await collection.CollectionExists())
{
await collection.CreateCollection();
}
await collection.UpsertAsync(record);
}
}
class CacheSetFunctionFilter(ISemanticTextMemory memory); // Saves results to cache.
class CacheGetPromptFilter(ISemanticTextMemory memory); // Check cache for entries.
var builder = Kernel.CreateBuilder();
builder
// Existing registration:
.AddAzureOpenAITextEmbeddingGeneration(textEmbeddingDeploymentName, azureAIEndpoint, apiKey, serviceId: "AzureOpenAI:text-embedding-ada-002")
// Register an IVectorStore implementation under the given key.
.AddAzureAISearch("Cache", azureAISearchEndpoint, apiKey, new Options() { withEmbeddingGeneration = true });
// Add Semantic Cache Memory for the cache entry model.
builder.Services.AddTransient<ISemanticTextMemory>(sp => {
return new SemanticTextMemory(
sp.GetKeyedService<IVectorStore>("Cache"),
cacheRecordDefinition);
});
// Add filter to retrieve items from cache and one to add items to cache.
// Since these filters depend on ISemanticTextMemory<CacheEntryModel> and that is already registered, it should get matched automatically.
builder.Services.AddTransient<IPromptRenderFilter, CacheGetPromptFilter>();
builder.Services.AddTransient<IFunctionInvocationFilter, CacheSetFunctionFilter>();
Need the following for all features: