showcase/shell-docs/src/content/docs/integrations/ag2/frontend-tools.mdx
<video
src="https://cdn.copilotkit.ai/docs/copilotkit/images/frontend-actions-demo.mp4"
className="rounded-lg shadow-xl"
loop
playsInline
controls
autoPlay
muted
/>
<Callout>
This video shows the result of npx copilotkit@latest init with the
implementation section applied to it.
</Callout>
Frontend actions are powerful tools that allow your AI agents to directly interact with and update your application's user interface. Think of them as bridges that connect your agent's decision-making capabilities with your frontend's interactive elements.
<Callout type="info"> CopilotKit consumes AG-UI protocol events streamed by AG2 over{" "} <code>/chat</code>. See the{" "} <a href="https://docs.ag2.ai/latest/docs/user-guide/ag-ui/" target="_blank"> AG2 AG-UI integration docs </a> . </Callout>Frontend actions are essential when you want to create truly interactive AI applications where your agent needs to:
Without frontend actions, agents are limited to just processing and returning data. By implementing frontend actions, you can create rich, interactive experiences where your agent actively drives the user interface.
To use frontend actions, you'll need to setup CopilotKit first. For the sake of brevity, we won't cover it here.
Check out our [getting started guide](/ag2/quickstart) and come back here when you're setup.
</Step>
<Step>
### Create a frontend action
First, you'll need to create a frontend action using the [useFrontendTool](/reference/v1/hooks/useFrontendTool) hook. Here's a simple one to get you started
that says hello to the user.
```tsx title="page.tsx"
export function Page() {
// ...
// [!code highlight:13]
useFrontendTool({
name: "sayHello",
description: "Say hello to the user",
available: "remote", // optional, makes it so the action is *only* available to the AG2 backend over AG-UI
parameters: z.object({
name: z.string().describe("The name of the user to say hello to"),
}),
handler: async ({ name }) => {
alert(`Hello, ${name}!`);
return `Said hello to ${name}!`;
},
});
// ...
}
```
</Step>
<Step>
### Modify your agent
Now, we'll need to modify the agent to access these frontend actions. Open your AG2 backend file and continue from there.
</Step>
<Step>
### Setup your AG2 backend over AG-UI
AG2 can call frontend actions through AG-UI tool events with a standard `/chat` endpoint:
<Tabs groupId="language" items={['Python']} default="Python">
<Tab value="Python">
```python title="agent.py"
from fastapi import FastAPI, Header
from fastapi.responses import StreamingResponse
from autogen import ConversableAgent, LLMConfig
from autogen.ag_ui import AGUIStream, RunAgentInput
agent = ConversableAgent(
name="assistant",
system_message="You are a helpful assistant.",
llm_config=LLMConfig({"model": "gpt-5.4-mini"}),
)
stream = AGUIStream(agent)
app = FastAPI()
@app.post("/chat")
async def run_agent(
message: RunAgentInput,
accept: str | None = Header(None),
):
return StreamingResponse(
stream.dispatch(message, accept=accept),
media_type=accept or "text/event-stream",
)
```
</Tab>
</Tabs>
That's it. Your AG2 backend can now receive frontend action tool definitions emitted by CopilotKit through AG-UI.
</Step>
<Step>
### Give it a try
You've now given your agent the ability to directly call any CopilotActions you've defined. These actions will be available as tools to the agent where they can be used as needed.
</Step>