docs/decisions/0005-kernel-hooks-phase1.md
A Kernel function caller needs to be able to handle/intercept any function execution in the Kernel before and after it was attempted. Allowing it to modify the prompt, abort the execution, or modify the output and many other scenarios as follows:
Pre-Execution / Function Invoking
Post-Execution / Function Invoked
Pre-Execution / Function Invoking
Post-Execution / Function Invoked
Pros:
Cons:
Pros:
Cons:
Expose events on both IKernel and ISKFunction that the call can can be observing to interact.
Pros:
Cons:
ISKFunction.InvokeAsyncSpecified on Kernel level, and would only be used using IKernel.RunAsync operation, this pattern would be similar to asp.net core middlewares, running the pipelines with a context and a requestdelegate next for controlling (Pre/Post conditions)
Pros:
Cons:
```csharp
class Kernel : IKernel
{
RunAsync() {
var functionInvokingArgs = await this.TriggerEvent<FunctionInvokingEventArgs>(this.FunctionInvoking, skFunction, context);
var functionResult = await skFunction.InvokeAsync(context, cancellationToken: cancellationToken);
var functionInvokedArgs = await this.TriggerEvent<FunctionInvokedEventArgs>(
this.FunctionInvoked,
skFunction,
context);
}
private TEventArgs? TriggerEvent<TEventArgs>(EventHandler<TEventArgs>? eventHandler, ISKFunction function, SKContext context) where TEventArgs : SKEventArgs
{
if (eventHandler is null)
{
return null;
}
if (function is ISKFunctionEventSupport<TEventArgs> supportedFunction)
{
var eventArgs = await supportedFunction.PrepareEventArgsAsync(context);
eventHandler.Invoke(this, eventArgs);
return eventArgs;
}
// Think about allowing to add data with the extra interface.
// If a function don't support the specific event we can:
return null; // Ignore or Throw.
throw new NotSupportedException($"The provided function \"{function.Name}\" does not supports and implements ISKFunctionHandles<{typeof(TEventArgs).Name}>");
}
}
public interface ISKFunctionEventSupport<TEventArgs> where TEventArgs : SKEventArgs
{
Task<TEventArgs> PrepareEventArgsAsync(SKContext context, TEventArgs? eventArgs = null);
}
class SemanticFunction : ISKFunction,
ISKFunctionEventSupport<FunctionInvokingEventArgs>,
ISKFunctionEventSupport<FunctionInvokedEventArgs>
{
public FunctionInvokingEventArgs PrepareEventArgsAsync(SKContext context, FunctionInvokingEventArgs? eventArgs = null)
{
var renderedPrompt = await this.RenderPromptTemplateAsync(context);
context.Variables.Set(SemanticFunction.RenderedPromptKey, renderedPrompt);
return new SemanticFunctionInvokingEventArgs(this.Describe(), context);
// OR Metadata Dictionary<string, object>
return new FunctionInvokingEventArgs(this.Describe(), context, new Dictionary<string, object>() { { RenderedPrompt, renderedPrompt } });
}
public FunctionInvokedEventArgs PrepareEventArgsAsync(SKContext context, FunctionInvokedEventArgs? eventArgs = null)
{
return Task.FromResult<FunctionInvokedEventArgs>(new SemanticFunctionInvokedEventArgs(this.Describe(), context));
}
}
public sealed class SemanticFunctionInvokedEventArgs : FunctionInvokedEventArgs
{
public SemanticFunctionInvokedEventArgs(FunctionDescription functionDescription, SKContext context)
: base(functionDescription, context)
{
_context = context;
Metadata[RenderedPromptKey] = this._context.Variables[RenderedPromptKey];
}
public string? RenderedPrompt => this.Metadata[RenderedPromptKey];
}
public sealed class SemanticFunctionInvokingEventArgs : FunctionInvokingEventArgs
{
public SemanticFunctionInvokingEventArgs(FunctionDescription functionDescription, SKContext context)
: base(functionDescription, context)
{
_context = context;
}
public string? RenderedPrompt => this._context.Variables[RenderedPromptKey];
}
```
Pros:
Kernel is not aware of SemanticFunction implementation details or any other ISKFunction implementationISKFunctions implementation, including prompts for semantic functionsISKFunctionEventSupport<NewEvent> interfaceISKFunctions can choose to implement it or notCons:
ISKFunctionEventSupport interface if they want to support events.Kernel will have to check if the function implements the interface or not, and if not, it will have to throw an exception or ignore the event.Q: Post Execution Handlers should execute right after the LLM result or before the end of the function execution itself? A: Currently post execution Handlers are executed after function execution.
Q: Should Pre/Post Handlers be many (pub/sub) allowing registration/deregistration? A: By using the standard .NET event implementation, this already supports multiple registrations as well as deregistrations managed by the caller.
Q: Setting Handlers on top of pre existing Handlers should be allowed or throw an error? A: By using the standard .NET event implementation, the standard behavior will not throw an error and will execute all the registered handlers.
Q: Setting Handlers on Plans should automatically cascade this Handlers for all the inner steps + overriding existing ones in the process? A: Handlers will be triggered before and after each step is executed the same way the Kernel RunAsync pipeline works.
Q: When a pre function execution handler intents to cancel the execution, should further handlers in the chain be called or not? A: Currently the standard .net behavior is to call all the registered handlers. This way function execution will solely depends on the final state of the Cancellation Request after all handlers were called.
Chosen option: 3. Event Base Registration (Kernel only)
This approach is the simplest and take the benefits of the standard .NET event implementation.
Further changes will be implemented to fully support all the scenarios in phase 2.