ContextQMD
Back to Langchaingo

httprr: HTTP Record and Replay for Testing

internal/httprr/README.md

0.1.1411.7 KB
Original Source

httprr: HTTP Record and Replay for Testing

The httprr package provides deterministic HTTP record and replay functionality for testing. It allows tests to record real HTTP interactions during development and replay them during CI/testing, ensuring consistent and fast test execution.

Quick Start

go
func TestMyAPI(t *testing.T) {
    // Skip test gracefully if no credentials and no recording exists
    httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")
    
    // Create recorder/replayer
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    defer rr.Close()
    
    // Use rr.Client() for all HTTP calls
    client := rr.Client()
    resp, err := client.Get("https://api.example.com/data")
    // ... test continues
}

Core Concepts

Recording vs Replay Modes

  • Recording Mode (-httprecord=.): Makes real HTTP requests and saves them to .httprr files
  • Replay Mode (default): Reads saved .httprr files and replays the responses

Command-Line Flags

  • -httprecord=<regexp>: Re-record traces for files matching the regexp pattern (use "." to match all)
  • -httprecord-delay=<ms>: Add delay in milliseconds between HTTP requests during recording (helps avoid rate limits)

File Management

  • Recording: Always creates uncompressed .httprr files for easier debugging
  • Replay: Automatically handles both .httprr and .httprr.gz files
  • Conflict Resolution: Chooses the newer file if both compressed and uncompressed exist
  • Auto-cleanup: Recording mode removes conflicting files automatically

API Reference

Core Functions

OpenForTest(t *testing.T, rt http.RoundTripper) (*RecordReplay, error)

The primary API for most test cases. Creates a recorder/replayer for the given test.

  • Recording mode: Creates testdata/TestName.httprr
  • Replay mode: Loads existing recording
  • File naming: Derived automatically from t.Name()
  • Directory: Always uses testdata/ subdirectory

SkipIfNoCredentialsOrRecording(t *testing.T, envVars ...string)

Gracefully skips tests when they cannot run (no API keys) and have no recorded data.

go
// Skip if OPENAI_API_KEY not set AND no recording exists
httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")

// Skip if neither API_KEY nor BACKUP_KEY is set AND no recording exists  
httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY", "BACKUP_KEY")

Open(file string, rt http.RoundTripper) (*RecordReplay, error)

Low-level API for custom file management. Most tests should use OpenForTest instead.

RecordReplay Methods

Client() *http.Client

Returns an HTTP client that routes through the recorder/replayer.

ScrubReq(scrubs ...func(*http.Request) error)

Adds request scrubbing functions to sanitize sensitive data before recording.

go
rr.ScrubReq(func(req *http.Request) error {
    req.Header.Set("Authorization", "Bearer test-api-key")
    return nil
})

ScrubResp(scrubs ...func(*bytes.Buffer) error)

Adds response scrubbing functions to sanitize sensitive data in responses.

Recording() bool

Reports whether the recorder is in recording mode.

Close() error

Closes the recorder/replayer. Use with defer for automatic cleanup.

Usage Patterns

Basic API Testing

go
func TestOpenAIChat(t *testing.T) {
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")
    
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    defer rr.Close()
    
    // Scrub sensitive data
    rr.ScrubReq(func(req *http.Request) error {
        req.Header.Set("Authorization", "Bearer test-api-key")
        return nil
    })
    
    // Create client with recording support
    llm, err := openai.New(openai.WithHTTPClient(rr.Client()))
    require.NoError(t, err)
    
    // Test continues with recorded/replayed HTTP calls
    response, err := llm.GenerateContent(ctx, messages)
    require.NoError(t, err)
}

Helper Functions for Multiple Tests

go
func createTestClient(t *testing.T) *MyAPIClient {
    t.Helper()
    httprr.SkipIfNoCredentialsOrRecording(t, "MY_API_KEY")
    
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    t.Cleanup(func() { rr.Close() })
    
    return NewMyAPIClient(WithHTTPClient(rr.Client()))
}

func TestFeatureA(t *testing.T) {
    client := createTestClient(t)
    // ... test continues
}

func TestFeatureB(t *testing.T) {
    client := createTestClient(t)
    // ... test continues
}

Multiple API Endpoints

go
func TestMultiAPIIntegration(t *testing.T) {
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY", "SERPAPI_KEY")
    
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    if err != nil {
        t.Fatal(err)
    }
    defer rr.Close()
    
    // Both clients will use the same recording
    openaiClient := openai.New(openai.WithHTTPClient(rr.Client()))
    searchClient := serpapi.New(serpapi.WithHTTPClient(rr.Client()))
    
    // All HTTP calls are recorded/replayed together
}

Command Line Usage

Recording New Interactions

bash
# Record all tests
go test ./... -httprecord=.

# Record specific test
go test ./pkg -httprecord=. -run TestSpecificFunction

# Record with pattern matching
go test ./... -httprecord="TestOpenAI.*"

Running with Recorded Data

bash
# Normal test run (uses recorded data)
go test ./...

# Skip tests that need credentials
OPENAI_API_KEY="" go test ./...  # Tests will skip gracefully

File Management

File Structure

testdata/
├── TestBasicFunction.httprr           # Uncompressed recording
├── TestWithSubtest-subcase.httprr     # Subtest recording  
├── TestOldFunction.httprr.gz          # Compressed recording
└── TestComplexAPI-setup.httprr        # Multi-part test

File Naming Rules

  • Test name: TestMyFunction → File: TestMyFunction.httprr
  • With subtests: TestMyFunction/subcase → File: TestMyFunction-subcase.httprr
  • Special chars: Replaced with hyphens for filesystem compatibility

Compression Management

bash
# Compress all recordings (for repository storage)
go run ./internal/devtools/rrtool pack -r

# Check compression status
go run ./internal/devtools/rrtool check

# Decompress for debugging
go run ./internal/devtools/rrtool unpack -r

Recording with Rate Limit Protection

When recording tests that make many API calls, use the delay flag to avoid hitting rate limits:

bash
# Record with 1 second delay between requests
go test -httprecord=. -httprecord-delay=1000 ./...

# Record specific test with 500ms delay
go test -httprecord=. -httprecord-delay=500 -run TestMyAPI ./mypackage

Best Practices

1. Always Use Graceful Skipping

go
// ✅ Good: Test skips gracefully when it can't run
httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")

// ❌ Bad: Test fails when API key missing
rr, err := httprr.OpenForTest(t, http.DefaultTransport)

2. Scrub Sensitive Data

go
// ✅ Good: Replace real API keys with test values
rr.ScrubReq(func(req *http.Request) error {
    req.Header.Set("Authorization", "Bearer test-api-key")
    return nil
})

// ❌ Bad: Real API keys recorded in files
// (No scrubbing - keys end up in repository)

3. Use Helper Functions

go
// ✅ Good: Reusable test setup
func createTestLLM(t *testing.T) *openai.LLM {
    t.Helper()
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")
    // ... setup code
}

// ❌ Bad: Duplicate setup in every test
func TestA(t *testing.T) {
    httprr.SkipIfNoCredentialsOrRecording(t, "OPENAI_API_KEY")
    rr, err := httprr.OpenForTest(t, http.DefaultTransport)
    // ... repeated setup
}

4. Handle Cleanup Properly

go
// ✅ Good: Automatic cleanup
defer rr.Close()

// or
t.Cleanup(func() { rr.Close() })

// ❌ Bad: Manual cleanup (can be forgotten)
// (No defer or cleanup)

Troubleshooting

Common Issues

"cached HTTP response not found"

Problem: Test is trying to make an HTTP request not in the recording.

Solutions:

bash
# Re-record the test
go test ./pkg -httprecord=. -run TestName

# Check if you have required environment variables
export OPENAI_API_KEY="your-key-here"
go test ./pkg -httprecord=. -run TestName

"gzip: invalid header"

Problem: .httprr.gz file is corrupted or not actually compressed.

Solutions:

bash
# Check and fix compression
go run ./internal/devtools/rrtool check
go run ./internal/devtools/rrtool pack -r

# Or remove the corrupted file and re-record
rm testdata/TestName.httprr.gz
go test ./pkg -httprecord=. -run TestName

Test skipped unexpectedly

Problem: Test is skipping when you expect it to run.

Debug steps:

bash
# Check if environment variables are set
echo $OPENAI_API_KEY

# Check if recording exists
ls testdata/TestName.httprr*

# Run with verbose output
go test ./pkg -run TestName -v

File Conflicts

The system automatically handles conflicts, but you can resolve manually:

bash
# Check which file is newer
ls -la testdata/TestName.httprr*

# Remove older file (system will warn and use newer)
rm testdata/TestName.httprr.gz  # if .httprr is newer

# Or compress the newer one
gzip testdata/TestName.httprr

Migration Guide

From OpenForTestWithSkip (Old API)

go
// ❌ Old API (removed)
rr := httprr.OpenForTestWithSkip(t, http.DefaultTransport, "API_KEY")
defer rr.Close()

// ✅ New API
httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")

rr, err := httprr.OpenForTest(t, http.DefaultTransport)
if err != nil {
    t.Fatal(err)
}
defer rr.Close()

Benefits of New API

  1. Consistent Error Handling: All httprr operations return errors
  2. Clear Separation: Skip logic separate from file operations
  3. Single Responsibility: Each function has one clear purpose
  4. Better Documentation: Self-documenting function names

Advanced Usage

Custom File Locations

go
// For custom file management (rarely needed)
rr, err := httprr.Open("custom/path/recording.httprr", http.DefaultTransport)
if err != nil {
    t.Fatal(err)
}
defer rr.Close()

Conditional Recording

go
func TestWithConditionalRecording(t *testing.T) {
    // Only record if we have credentials
    if os.Getenv("API_KEY") != "" {
        // Will record new interactions
        rr, err := httprr.OpenForTest(t, http.DefaultTransport)
        // ...
    } else {
        // Will only replay existing recordings
        httprr.SkipIfNoCredentialsOrRecording(t, "API_KEY")
        rr, err := httprr.OpenForTest(t, http.DefaultTransport)
        // ...
    }
}

Complex Scrubbing

go
rr.ScrubReq(func(req *http.Request) error {
    // Remove API keys
    req.Header.Set("Authorization", "Bearer test-key")
    
    // Scrub request body
    if req.Body != nil {
        body := req.Body.(*httprr.Body)
        bodyStr := string(body.Data)
        bodyStr = strings.ReplaceAll(bodyStr, "real-secret", "test-secret")
        body.Data = []byte(bodyStr)
    }
    
    return nil
})

rr.ScrubResp(func(buf *bytes.Buffer) error {
    // Remove sensitive data from responses
    content := buf.String()
    content = strings.ReplaceAll(content, "sensitive-data", "redacted")
    buf.Reset()
    buf.WriteString(content)
    return nil
})

Contributing

When adding new tests that use external APIs:

  1. Always use SkipIfNoCredentialsOrRecording for graceful degradation
  2. Include appropriate scrubbing to avoid committing secrets
  3. Record with real credentials initially, then scrub the results
  4. Compress recordings before committing to save repository space
  5. Document required environment variables in test comments

For questions or issues with the httprr system, see the main project documentation or open an issue.