docs/doc/developer/backend/Backend_Setup.mdx
The Omi backend powers all AI capabilities including transcription, conversation processing, chat, and integrations. This guide will help you set up a local development environment.
flowchart LR
subgraph External["External Services"]
OAI[OpenAI]
DG[Deepgram]
Pine[Pinecone]
Redis[Redis]
end
subgraph Google["Google Cloud"]
FB[Firebase]
GCS[Cloud Storage]
end
subgraph Backend["Your Backend"]
API[FastAPI Server]
Push[Pusher Service]
end
subgraph App["Omi App"]
Flutter[Flutter Client]
end
Flutter <-->|Ngrok| API
API --> OAI
API --> DG
API --> Pine
API --> Redis
API --> FB
API --> GCS
API --> Push
Before starting, gather these API keys and credentials:
<Card title="Required Services" icon="key"> | Service | Purpose | Get Key | |---------|---------|---------| | OpenAI | AI language models | [Get Key](https://platform.openai.com/settings/organization/api-keys) | | Deepgram | Speech-to-text | [Get Key](https://console.deepgram.com/api-keys) | | Redis (Upstash) | Caching & sessions | [Get Key](https://console.upstash.com/) | | Pinecone | Vector database | [Get Key](https://app.pinecone.io/organizations/-/projects/-/api-keys) | | Hugging Face | Voice detection | [Get Key](https://huggingface.co/settings/tokens) | </Card> <Card title="Optional Services" icon="puzzle-piece"> | Service | Purpose | Get Key | |---------|---------|---------| | Modal | Serverless deployment | [Get Key](https://modal.com/settings#tokens) | | GitHub | Firmware updates | [Get Key](https://github.com/settings/tokens) | | Google Maps | Location features | [Get Key](https://console.cloud.google.com/google/maps-apis/credentials) | | Typesense | Search functionality | [Get Key](https://cloud.typesense.org/clusters) | | Stripe | Payment processing | [Get Key](https://dashboard.stripe.com/apikeys) | </Card> <Tip> New to backend development? Install [Homebrew](https://docs.brew.sh/Installation) (macOS/Linux) or [Chocolatey](https://chocolatey.org/install) (Windows) first - they make installing tools much easier. </Tip><CardGroup cols={2}>
<Card title="Cloud Resource Manager API" icon="sitemap" href="https://console.cloud.google.com/apis/library/cloudresourcemanager.googleapis.com">
Required for project management
</Card>
<Card title="Firebase Management API" icon="fire" href="https://console.cloud.google.com/apis/library/firebase.googleapis.com">
Required for Firebase integration
</Card>
</CardGroup>
```bash
gcloud auth login
gcloud config set project <project-id>
gcloud auth application-default login --project <project-id>
```
This generates credentials at `~/.config/gcloud/application_default_credentials.json`.
OAuth is required for user authentication. You need to configure both Google and Apple sign-in.
<Tabs> <Tab title="Google OAuth" icon="google"> <Steps> <Step title="Create OAuth 2.0 Client"> 1. Go to [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials) 2. Click **Create Credentials** → **OAuth 2.0 Client ID** 3. Select **Web application** as the application type 4. Set a name (e.g., "Omi Backend Auth") </Step> <Step title="Configure Authorized Origins"> Under **Authorized JavaScript origins**, add: - `https://your-domain.com` (production) - `https://your-ngrok-domain.ngrok-free.app` (local development) </Step> <Step title="Configure Redirect URIs"> Under **Authorized redirect URIs**, add: - `https://your-domain.com/v1/auth/callback/google` - `https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/google` </Step> <Step title="Save Credentials"> Click **Create** and save your **Client ID** and **Client Secret** for the `.env` file. </Step> <Step title="Configure Consent Screen"> Go to **APIs & Services → OAuth consent screen**: - Fill in required app information - Add your domain to **Authorized domains** - Add scopes: `openid`, `email`, `profile` </Step> </Steps> </Tab> <Tab title="Apple OAuth" icon="apple"> <Warning> Apple OAuth requires a paid Apple Developer account ($99/year). </Warning><Steps>
<Step title="Create App ID">
1. Go to [Apple Developer Console → Identifiers](https://developer.apple.com/account/resources/identifiers/list)
2. Create a new App ID with **Sign In with Apple** capability enabled
3. Note your Bundle ID
</Step>
<Step title="Create Services ID">
1. Create a new **Services ID** (this becomes your `APPLE_CLIENT_ID`)
2. Configure **Sign In with Apple** for this Services ID
3. Add authorized domains and return URLs:
- `https://your-domain.com/v1/auth/callback/apple`
- `https://your-ngrok-domain.ngrok-free.app/v1/auth/callback/apple`
</Step>
<Step title="Create Private Key">
1. Go to **Keys** in Apple Developer Console
2. Create a new key with **Sign In with Apple** enabled
3. Download the `.p8` file and note the **Key ID**
4. Note your **Team ID** from your Apple Developer account
</Step>
<Step title="Configure Firebase">
1. Go to [Firebase Console](https://console.firebase.google.com/) → Authentication → Sign-in method
2. Enable **Apple** and add your configuration:
- Client ID: Your Services ID
- Team ID: Your Apple Developer Team ID
- Key ID: From your private key
- Private Key: Contents of your .p8 file
</Step>
</Steps>
<Note>
Your Apple environment variables will be:
- `APPLE_CLIENT_ID`: Your Services ID
- `APPLE_TEAM_ID`: Your Apple Developer Team ID
- `APPLE_KEY_ID`: The Key ID from step 3
- `APPLE_PRIVATE_KEY`: The full contents of your .p8 file (including BEGIN/END lines)
</Note>
```bash
# Create virtual environment (use Python 3.9-3.12)
python -m venv venv
```
<Tabs>
<Tab title="macOS/Linux">
```bash
source venv/bin/activate
```
</Tab>
<Tab title="Windows">
```powershell
venv\Scripts\activate
```
</Tab>
</Tabs>
You should see `(venv)` at the beginning of your command prompt.
Edit `.env` and fill in your API keys (see [Environment Variables](#environment-variables) below).
```bash
cd pusher
cp .env.template .env
```
Edit the `.env` file and set `SERVICE_ACCOUNT_JSON` to your Google credentials string.
Start the service:
```bash
cd ..
uvicorn pusher.main:app --reload --env-file .env --port 8000
```
Optionally expose via Ngrok for external access.
1. Create an account on [Typesense](https://typesense.org/)
2. Create a collection named `conversations` using the schema in `typesense/conversations.schema`
3. Install the [Firebase Typesense extension](https://console.firebase.google.com/project/_/extensions/install?ref=typesense/[email protected])
Configuration for the extension:
| Setting | Value |
|---------|-------|
| Firestore Collection Path | `users/{userId}/conversations` |
| Firestore Collection Fields | `structured,created_at,discarded,started_at,id,finished_at,geolocation,userId` |
<Note>
If you have existing data, create a `typesense_sync` collection and add a document named `backfill` with `{'trigger': true}`.
</Note>
1. Sign up at [ngrok.com](https://ngrok.com/) and install Ngrok
2. Authenticate with your account
3. Start the tunnel:
```bash
ngrok http --domain=your-domain.ngrok-free.app 8000
```
Note the public URL (e.g., `https://your-domain.ngrok-free.app`).
| Flag | Purpose |
|------|---------|
| `--reload` | Auto-restart on code changes |
| `--env-file .env` | Load environment variables |
| `--host 0.0.0.0` | Listen on all interfaces (optional) |
| `--port 8000` | Port to listen on |
```
API_BASE_URL=https://your-domain.ngrok-free.app/
```
<Warning>
Don't forget the trailing `/` in the URL!
</Warning>
Make sure your OAuth redirect URIs in Google Cloud Console and Apple Developer Console include your Ngrok URL.
```bash
deactivate
```
To resume later, just activate the virtual environment again.
```python
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
```
```bash
pip install -r requirements.txt --upgrade --force-reinstall
```
Complete reference for all .env variables:
We use black for code formatting with a line length of 120 characters.
pip install black
To auto-format on commit, install the pre-commit hook from the repository root:
ln -s -f ../../scripts/pre-commit .git/hooks/pre-commit