docs/scf/examples/ai-streaming.md
This example demonstrates how to build and deploy an Express-based AI fullstack streaming application that uses Server-Sent Events (SSE) to stream live AI responses from multiple providers (OpenAI and Anthropic). It includes both a front-end and a back-end. The Serverless Container Framework (SCF) enables this application to be deployed to either AWS Lambda or AWS ECS Fargate without rearchitecting, while providing local development, flexible compute configurations, and smooth AWS deployments.
Before getting started, make sure you have:
npm i -g serverless
For more information on setting up AWS credentials, see the SCF Getting Started guide.
At the project root, the serverless.containers.yml file defines the SCF configuration:
name: ai-streaming
deployment:
type: [email protected]
containers:
service:
src: ./service
routing:
pathPattern: /*
pathHealthCheck: /health
environment:
OPENAI_API_KEY: ${env:OPENAI_API_KEY}
ANTHROPIC_API_KEY: ${env:ANTHROPIC_API_KEY}
compute:
type: awsLambda # or awsFargateEcs
This file specifies:
./service directory./*) is used with a dedicated health check endpoint (/health).awsLambda by default (switchable to awsFargateEcs as needed).For more details on SCF configuration options, see the SCF Configuration documentation.
A typical project structure looks like this:
example-ai-streaming/
├── serverless.containers.yml # SCF project configuration file
└── service/
├── Dockerfile # Multi-stage Dockerfile for Lambda and Fargate
├── package.json # Node.js project configuration and dependencies
├── .env # Environment variables (not committed)
└── src/
├── index.js # Main Express application entrypoint
├── routes/ # API route definitions (including AI streaming endpoint)
├── middleware/ # Custom middleware (error handling, etc.)
└── public/ # Static assets (HTML, CSS, JS, images)
Serverless Container Framework provides a local development mode that emulates AWS routing, Lambda, and ECS Fargate environments, including AWS Application Load Balancer emulation. To start the development mode (with hot reloading on file changes), run:
serverless dev
Additionally, you can run the application directly using:
npm start
For more information on local development with SCF, see the SCF Development documentation.
Deploy your AI streaming application to AWS with:
serverless deploy
During deployment, SCF builds the container image (using the provided Dockerfile) and provisions AWS resources (ALB, VPC, Lambda function or ECS service) automatically.
For more details on deployment options and processes, see the SCF Deployment documentation.
Serverless Container Framework supports the Serverless Framework Variables system to reference infrastructure details, secrets, and more from various sources:
containers:
service:
environment:
# Simple static value
SERVICE_NAME: ai-streaming-service
# Environment variable reference
OPENAI_API_KEY: ${env:OPENAI_API_KEY}
# AWS Systems Manager Parameter Store reference
DATABASE_URL: ${aws:ssm:/path/to/database/url}
# AWS Secrets Manager reference
DATABASE_PASSWORD: ${aws:secretsmanager:MyDatabaseSecret.password}
# HashiCorp Vault reference
API_SECRET: ${vault:secret/data/api/credentials.secret}
# HashiCorp Terraform state reference
REDIS_ENDPOINT: ${terraform:outputs:redis_endpoint}
# S3 bucket value reference
CONFIG_JSON: ${aws:s3:config-bucket/config.json}
# CloudFormation stack output reference
VPC_ID: ${aws:cf:networking-stack.VpcIdOutput}
For more details on using variables, see the Serverless Framework Variables documentation.
To remove deployed AWS resources when they are no longer needed:
serverless remove
For complete cleanup (including shared infrastructure):
serverless remove --force --all
This example supports multiple AI models from both OpenAI and Anthropic:
GPT-4o Series
GPT-4 Series
GPT-3.5 Series
O-Series Models
Claude 3.7 Series
Claude 3.5 Series
Claude 3 Series