dotnet/docs/OPENAI-CONNECTOR-MIGRATION.md
This manual prepares you for the migration of your OpenAI Connector to the new OpenAI Connector. The new OpenAI Connector is a complete rewrite of the existing OpenAI Connector and is designed to be more efficient, reliable, and scalable. This manual will guide you through the migration process and help you understand the changes that have been made to the OpenAI Connector.
If you are working with Azure and or OpenAI public APIs, you will need to change the package from Microsoft.SemanticKernel.Connectors.OpenAI to Microsoft.SemanticKernel.Connectors.AzureOpenAI,
[!IMPORTANT] The
Microsoft.SemanticKernel.Connectors.AzureOpenAIpackage depends on theMicrosoft.SemanticKernel.Connectors.OpenAIpackage so there's no need to add both to your project when usingOpenAIrelated types.
- // Before
- using Microsoft.SemanticKernel.Connectors.OpenAI;
+ After
+ using Microsoft.SemanticKernel.Connectors.AzureOpenAI;
When using Azure with OpenAI, before where you were using OpenAIClient you will need to update your code to use the new AzureOpenAIClient type.
All services below now belong to the Microsoft.SemanticKernel.Connectors.AzureOpenAI namespace.
AzureOpenAIAudioToTextServiceAzureOpenAIChatCompletionServiceAzureOpenAITextEmbeddingGenerationServiceAzureOpenAITextToAudioServiceAzureOpenAITextToImageServiceThe latest OpenAI SDK does not support text generation modality, when migrating to their underlying SDK we had to drop the support and removed TextGeneration specific services but the existing ChatCompletion ones still supports (implements ITextGenerationService).
If you were using any of the OpenAITextGenerationService or AzureOpenAITextGenerationService you will need to update your code to target a chat completion model instead, using OpenAIChatCompletionService or AzureOpenAIChatCompletionService instead.
[!NOTE] OpenAI and AzureOpenAI
ChatCompletionservices also implement theITextGenerationServiceinterface and that may not require any changes to your code if you were targeting theITextGenerationServiceinterface.
tags:
OpenAITextGenerationService,AzureOpenAITextGenerationService,
AddOpenAITextGeneration,AddAzureOpenAITextGeneration
The latest OpenAI SDK does not support multiple choices, when migrating to their underlying SDK we had to drop the support and removed ResultsPerPrompt also from the OpenAIPromptExecutionSettings.
tags: ResultsPerPrompt,results_per_prompt
The OpenAIFileService was deprecated in the latest version of the OpenAI Connector. We strongly recommend to update your code to use the new OpenAIClient.GetFileClient() for file management operations.
The OpenAIChatCompletionService experimental constructor for custom endpoints will not attempt to auto-correct the endpoint and use it as is.
We have the two only specific cases where we attempted to auto-correct the endpoint.
If you provided chat/completions path before. Now those need to be removed as they are added automatically to the end of your original endpoint by OpenAI SDK.
- http://any-host-and-port/v1/chat/completions
+ http://any-host-and-port/v1
If you provided a custom endpoint without any path. We won't be adding the v1/ as the first path. Now the v1 path needs to provided as part of your endpoint.
- http://any-host-and-port/
+ http://any-host-and-port/v1
To be retro compatible with the new OpenAI and AzureOpenAI Connectors, our Microsoft.SemanticKernel meta package changed its dependency to use the new Microsoft.SemanticKernel.Connectors.AzureOpenAI package that depends on the Microsoft.SemanticKernel.Connectors.OpenAI package. This way if you are using the metapackage, no change is needed to get access to Azure related types.
The Tools property type has changed from IReadOnlyList<ChatCompletionsToolCall> to IReadOnlyList<ChatToolCall>.
Inner content type has changed from ChatCompletionsFunctionToolCall to ChatToolCall.
Metadata type FunctionToolCalls has changed from IEnumerable<ChatCompletionsFunctionToolCall> to IEnumerable<ChatToolCall>.
FinishReason property type has changed from CompletionsFinishReason to FinishReason.ToolCallUpdate property has been renamed to ToolCallUpdates and its type has changed from StreamingToolCallUpdate? to IReadOnlyList<StreamingToolCallUpdate>?.AuthorName property is not initialized because it's not provided by the underlying library anymore.The meter s_meter = new("Microsoft.SemanticKernel.Connectors.OpenAI"); and the relevant counters still have old names that contain "openai" in them, such as:
semantic_kernel.connectors.openai.tokens.promptsemantic_kernel.connectors.openai.tokens.completionsemantic_kernel.connectors.openai.tokens.totalWith the new AzureOpenAIClient, you can now specify your datasource thru the options and that requires a small change in your code to the new type.
Before
var promptExecutionSettings = new OpenAIPromptExecutionSettings
{
AzureChatExtensionsOptions = new AzureChatExtensionsOptions
{
Extensions = [ new AzureSearchChatExtensionConfiguration
{
SearchEndpoint = new Uri(TestConfiguration.AzureAISearch.Endpoint),
Authentication = new OnYourDataApiKeyAuthenticationOptions(TestConfiguration.AzureAISearch.ApiKey),
IndexName = TestConfiguration.AzureAISearch.IndexName
}]
};
};
After
var promptExecutionSettings = new AzureOpenAIPromptExecutionSettings
{
AzureChatDataSource = new AzureSearchChatDataSource
{
Endpoint = new Uri(TestConfiguration.AzureAISearch.Endpoint),
Authentication = DataSourceAuthentication.FromApiKey(TestConfiguration.AzureAISearch.ApiKey),
IndexName = TestConfiguration.AzureAISearch.IndexName
}
};
Breaking glass scenarios are scenarios where you may need to update your code to use the new OpenAI Connector. Below are some of the breaking changes that you may need to be aware of.
Some of the keys in the content metadata dictionary have changed, you will need to update your code to when using the previous key names.
Created -> CreatedAtThe PromptFilterResults metadata type has changed from IReadOnlyList<ContentFilterResultsForPrompt> to ContentFilterResultForPrompt.
The ContentFilterResultsForPrompt type has changed from ContentFilterResultsForChoice to ContentFilterResultForResponse.
The FinishReason metadata string value has changed from stop to Stop
The ToolCalls metadata string value has changed from tool_calls to ToolCalls
The LogProbabilityInfo type has changed from ChatChoiceLogProbabilityInfo to IReadOnlyList<ChatTokenLogProbabilityInfo>.
All of above have been removed.
The Token usage naming convention from OpenAI changed from Completion, Prompt tokens to Output and Input respectively. You will need to update your code to use the new naming.
The type also changed from CompletionsUsage to ChatTokenUsage.
Example of Token Usage Metadata Changes
- Before
- var usage = FunctionResult.Metadata?["Usage"] as CompletionsUsage;
- var completionTokesn = usage?.CompletionTokens ?? 0;
- var promptTokens = usage?.PromptTokens ?? 0;
+ After
+ var usage = FunctionResult.Metadata?["Usage"] as ChatTokenUsage;
+ var promptTokens = usage?.InputTokens ?? 0;
+ var completionTokens = completionTokens: usage?.OutputTokens ?? 0;
totalTokens: usage?.TotalTokens ?? 0;
The OpenAIClient type previously was a Azure specific namespace type but now it is an OpenAI SDK namespace type, you will need to update your code to use the new OpenAIClient type.
When using Azure, you will need to update your code to use the new AzureOpenAIClient type.
The new OpenAI SDK uses a different pipeline configuration, and has a dependency on System.ClientModel package. You will need to update your code to use the new HttpClientPipelineTransport transport configuration where before you were using HttpClientTransport from Azure.Core.Pipeline.
Example of Pipeline Configuration
var clientOptions = new OpenAIClientOptions
{
- // Before: From Azure.Core.Pipeline
- Transport = new HttpClientTransport(httpClient),
+ // After: From OpenAI SDK -> System.ClientModel
+ Transport = new HttpClientPipelineTransport(httpClient),
};