Supabase Studio E2E Tests

Set up

Prerequisites

For Self-Hosted Tests

• Nothing is required, running with IS_PLATFORM=false should run the tests locally with a self hosted docker container

For Platform Tests

1. Create a platform account - You can authenticate using either:
   • Email and password
   • GitHub OAuth (requires TOTP 2FA)
2. Create an organization on the platform, this can be done if run locally through mise fullstack
3. Generate a Personal Access Token (PAT) for API access
4. Configure the environment variables below (see Authentication section for details on email vs GitHub auth)

Configure Environment

Choose the appropriate example file based on your testing scenario:

For self-hosted tests:

cp .env.local.self-hosted.example .env.local

For platform tests with email authentication:

cp .env.local.email.example .env.local

For platform tests with GitHub authentication:

cp .env.local.github.example .env.local

Edit .env.local and set the appropriate values based on your test environment (see Environment Variables section below).

Install the playwright browser

⚠️ This should be done in the e2e/studio directory

cd e2e/studio

pnpm exec playwright install

Environment Variables

Configure your tests by setting the following environment variables in .env.local. We have examples of what required on self hosted and platform:

Core Configuration

• STUDIO_URL: The URL where Studio is running (default: http://localhost:8082)
• API_URL: The Supabase API endpoint (default: https://localhost:8080)
• IS_PLATFORM: Set to true for platform tests, false for self-hosted (default: false)
  • When true: Tests run serially (1 worker) due to API rate limits
  • When false: Tests run in parallel (5 workers)

Authentication (Required for Platform Tests)

⚠️ Before running platform tests, you must create an account and organization on the platform you're testing.

Authentication is automatically enabled when either email/password OR GitHub credentials are configured.

Email Authentication

• EMAIL: Your platform account email
• PASSWORD: Your platform account password
• PROJECT_REF: Project reference (optional, will be auto-created if not provided)

When both EMAIL and PASSWORD are set, the tests will authenticate using email/password. HCaptcha is mocked during test setup. Note this only works on local and staging environments

GitHub Authentication

• GITHUB_USER: Your GitHub username
• GITHUB_PASS: Your GitHub password
• GITHUB_TOTP: Your GitHub TOTP secret for 2FA (required, as GitHub enforces 2FA)

When GITHUB_USER, GITHUB_PASS, and GITHUB_TOTP are all set, the tests will authenticate using GitHub OAuth with TOTP-based 2FA. The authentication flow handles:

• Clicking "Sign In with GitHub" button
• Filling GitHub credentials
• Generating and submitting TOTP codes
• Handling GitHub authorization prompts
• Automatic retry on failure (up to 3 attempts)

Getting your GitHub TOTP secret: When setting up 2FA on GitHub, you'll see a QR code. Click "enter this text code instead" to reveal the secret key. This is the value to use for GITHUB_TOTP.

Platform-Specific Variables (Required when IS_PLATFORM=true)

• ORG_SLUG: Organization slug (default: default)
• SUPA_REGION: Supabase region (default: us-east-1)
• SUPA_PAT: Personal Access Token for API authentication (default: test)
• BRANCH_NAME: Name for the test branch/project (default: e2e-test-local)

Optional Variables

• OPENAI_API_KEY: Required for the AI Assistant test (assistant.spec.ts). Without this variable, the assistant test will be skipped.
• VERCEL_AUTOMATION_BYPASS_SELFHOSTED_STUDIO: Bypass token for Vercel protection (default: false)

Setup Commands Based on Configuration

The test setup automatically runs different commands based on your environment:

• Platform + Localhost (IS_PLATFORM=true and STUDIO_URL=localhost): Runs pnpm run e2e:setup:platform
• Platform + Remote (IS_PLATFORM=true and remote STUDIO_URL): No web server setup
• Self-hosted (IS_PLATFORM=false): Runs pnpm run e2e:setup:selfhosted

Running the tests

Check the package.json for the available commands and environments.

pnpm run e2e

With Playwright UI:

pnpm run e2e -- --ui

Tips for development

• Read Playwright Best Practices
• Use pnpm run e2e -- --ui to get the playwright UI.
• Add the tests in examples/examples.ts to Cursor as context.
• Add messages to expect statements to make them easier to debug.

Example:

await expect(page.getByRole('heading', { name: 'Logs & Analytics' }), {
  message: 'Logs heading should be visible',
}).toBeVisible()

• Use the test utility instead of playwrights test.

import { test } from '../utils/test'

• Use the PWDEBUG environment variable to debug the tests.

PWDEBUG=1 pnpm run e2e -- --ui

What should I test?

• Can the feature be navigated to?
• Does the feature load correctly?
• Can you do the actions (filtering, sorting, opening dialogs, etc)?

API Mocks

Read here: https://playwright.dev/docs/mock#mock-api-requests

Example:

await page.route(`*/**/logs.all*`, async (route) => {
  await route.fulfill({ body: JSON.stringify(mockAPILogs) })
})