docs/decisions/0047-azure-open-ai-connectors.md
It has recently been announced that OpenAI and Azure will each have their own dedicated SDKs for accessing their services. Previously, there was no official SDK for OpenAI, and our OpenAI Connector relied solely on the Azure SDK client for access.
With the introduction of the official OpenAI SDK, we now have access to more up-to-date features provided by OpenAI, making it advantageous to use this SDK instead of the Azure SDK.
Additionally, it has become clear that we need to separate the OpenAI connector into two distinct targets: one for OpenAI and another for Azure OpenAI. This separation will enhance code clarity and facilitate a better understanding of the usage of each target.
Although current Azure.AI.OpenAI and OpenAI SDK packages have its major versions updated (2.0.0), that change does not represent a SemanticKernel major breaking change. Any of the alternative options provided below take in consideration the that the new updated version of SemanticKernel.Connectors.OpenAI and SemanticKernel.Connectors.AzureOpenAI will be a minor version bump 1.N+1.0 for all SemanticKernel packages.
Currently the Microsoft.SemanticKernel package is a meta package that includes both SemanticKernel.Core and SemanticKernel.Connectors.OpenAI, with the new changes a new project will be added to the meta package SemanticKernel.Connectors.AzureOpenAI that will include the new Azure OpenAI connector.
A documentation guidance and samples/examples will be created to guide on how to upgrade from the current OpenAI connector to the new when needed.
The new OpenAI SDK introduce some limitations that need to be considered and potentially can introduce breaking changes if not remediated by our internal implementation.
Remediation: Internally make the multiple requests and combine them.
No remediation: Breaking change removing ResultsPerPrompt from OpenAIPromptExecutionSettings.
Remediation: Internally provide a HttpClient to be used against gpt-3.5-turbo-instruct for text generation modality. Same way was done for TextToImage, AudioToText service modalities.
No remediation: Breaking change removing any specific TextGeneration service implementations, this change don't impact ChatCompletion services that may still being used as ITextGenerationService implementations.
This also represents an opportunity to improve the current OpenAI connector by introducing the Configuration pattern to allow more flexibility and control over the services and their configurations.
// Before
builder.AddAzureOpenAIChatCompletion(deploymentName, endpoint, apiKey, httpClient);
// After
builder.AddAzureOpenAIChatCompletion(new
{
DeploymentName = modelId;
Endpoint = endpoint;
ApiKey = apiKey;
});
// Before
builder.AddAzureOpenAIChatCompletion(deploymentName, openAIClient, serviceId, modelId)
// After
builder.AddAzureOpenAIChatCompletion(new
{
DeploymentName = deploymentName;
ServiceId = serviceId;
ModelId = modelId;
}, openAIClient);
Since SemanticKernel.Connectors.AzureOpenAI and SemanticKernel.Connectors.OpenAI share same OpenAI 2.0.0 dependency, if the vestion of OpenAI 2.0.0 differ on each, that may create conflict when both connector packages are used together in a project.
If this happens:
Before updating our OpenAI connector package we will get in touch with Azure.AI.OpenAI team to align on the ETAs for their update.
Investigate if the most recent OpenAI package when used with a Azure.AI.OpenAI that initially was targeting an older version of OpenAI SDK will not cause any breaking changes or conflicts.
If There are conflicts and their ETA is small we may keep the OpenAI dependency on our SemanticKernel.Connectors.OpenAI similar to Azure's for a short period of time, otherwise we will evaluate moving forward with the OpenAI dependency version upgrade.
This is the least breaking approach where we keep the current legacy OpenAI and AzureOpenAI APIs temporarily in the connector using last Azure SDK Azure.AI.OpenAI 1.0.0-beta.17 and add new OpenAI specific APIs using the new OpenAI 2.0.0-beta.* SDK package.
This approach also implies that a new connector will be created on a second moment for Azure OpenAI services specifically fully dependent on the latest Azure.AI.OpenAI 2.0.0-beta.* SDK package.
In a later stage we will deprecate all the OpenAI and Azure legacy APIs in the SemanticKernel.Connectors.OpenAI namespace and remove Azure SDK Azure.AI.OpenAI 1.0.0-beta.17 and those APIs in a future release, making the OpenAI Connector fully dedicated for OpenAI services only depending on with the OpenAI 2.0.0-beta.* dependency.
graph TD
A[SemanticKernel.Connectors.OpenAI] --> B[OpenAI 2.0.0-beta.*]
A --> C[Azure.OpenAI 1.0.0-beta.17]
D[SemanticKernel.Connectors.AzureOpenAI] --> E[Azure.AI.OpenAI 2.0.0-beta.*]
The new Options pattern we be used as an improvement as well as a measure to avoid breaking changes with the legacy APIs.
Following this change the SemanticKernel.Connectors.OpenAI and a new SemanticKernel.Connectors.AzureOpenAI connector will be created for Azure specific services, using the new Azure SDK Azure.AI.OpenAI 2.0.0-beta.* with all new APIs using the options approach.
OpenAI connector pointing to new AzureOpenAI connectorAzureOpenAI connector to the Microsoft.SemanticKernel meta package.OpenAI APIs in the OpenAI connector pointing to new Options APIs.Pros:
Cons:
SemanticKernel.Connectors.AzureOpenAI and SemanticKernel.Connectors.OpenAI share a same dependency of different versions, both packages cannot be used in the same project and a strategy will be needed when deploying both connectors.Azure OpenAI 1.0-beta17 and OpenAI 2.0-beta1.Concepts and other projects that shares OpenAI and AzureOpenAI examples.Azure.AI.OpenAI.Legacy 1.0.0-beta.17 and update our SemanticKernel.Connectors.OpenAI to use this new namespace to avoid version clashing on the Azure.AI.OpenAI namespace.This option is focused on creating fully independent connectors for OpenAI and Azure OpenAI services since the start with all breaking changes needed to achieve that.
graph TD
D[SemanticKernel.Connectors.AzureOpenAI] --> E[Azure.AI.OpenAI 2.0.0-beta.*]
E --> B[OpenAI 2.0.0-beta.*]
A[SemanticKernel.Connectors.OpenAI] --> B[OpenAI 2.0.0-beta.*]
Impact:
Azure related logic will be removed from SemanticKernel.Connectors.OpenAI to avoid any clashes with same names introduced in the new SemanticKernel.Connectors.AzureOpenAI as well as sending a congruent message to developers that the OpenAI connector is focused on OpenAI services only moving forward.Pros:
Cons:
Azure.AI.OpenAI team does not update their package.This option is fully focused in the least impact possible, combining both Azure and OpenAI SDK dependencies in one single connector following the same approach as the current connector.
Changes:
Options pattern new APIs to the connector services and deprecate old ones.Pros:
Azure.AI.OpenAI and OpenAI SDKs.Cons:
Azure.AI.OpenAI package, which may not be the latest version available.This option is the faster approach on transitioning to a potential 1.0 general availability of OpenAI SDK.
This also option provides a clear separation of concerns between OpenAI and Azure OpenAI connectors from the start.
Prevents any confusion sending a clear message on our intentions on splitting OpenAI and AzureOpenAI components away.