ContextQMD
Back to Picoclaw

Provider Architecture Refactoring - Test Suite Summary

docs/design/provider-refactoring-tests.md

0.2.87.7 KB
Original Source

Provider Architecture Refactoring - Test Suite Summary

This document summarizes the complete test suite designed for the Provider architecture refactoring.

Test File Structure

pkg/
├── config/
│   ├── model_config_test.go      # US-001, US-002: ModelConfig struct and GetModelConfig tests
│   └── migration_test.go         # US-003: Backward compatibility and migration tests
├── providers/
│   ├── factory_test.go           # US-004, US-005: Provider factory tests
│   └── factory_provider_test.go  # Factory provider integration tests

Test Case Checklist

1. pkg/config/model_config_test.go - Configuration Parsing Tests

Test NamePurposePRD Reference
TestModelConfig_ParsingVerify ModelConfig JSON parsingUS-001
TestModelConfig_ModelListInConfigVerify model_list parsing in ConfigUS-001
TestModelConfig_ValidationVerify required field validationUS-001
TestConfig_GetModelConfig_FoundVerify GetModelConfig finds modelUS-002
TestConfig_GetModelConfig_NotFoundVerify GetModelConfig returns errorUS-002
TestConfig_GetModelConfig_EmptyModelListVerify empty model_list handlingUS-002
TestConfig_BackwardCompatibility_ProvidersToModelListVerify old config conversionUS-003
TestConfig_DeprecationWarningVerify deprecation warningUS-003
TestModelConfig_ProtocolExtractionVerify protocol prefix extractionUS-004
TestConfig_ModelNameUniquenessVerify model_name uniquenessUS-001

2. pkg/config/migration_test.go - Migration Tests

Test NamePurposePRD Reference
TestConvertProvidersToModelList_OpenAIOpenAI config conversionUS-003
TestConvertProvidersToModelList_AnthropicAnthropic config conversionUS-003
TestConvertProvidersToModelList_MultipleProvidersMultiple provider conversionUS-003
TestConvertProvidersToModelList_EmptyProvidersEmpty providers handlingUS-003
TestConvertProvidersToModelList_GitHubCopilotGitHub Copilot conversionUS-003
TestConvertProvidersToModelList_AntigravityAntigravity conversionUS-003
TestGenerateModelName_*Model name generationUS-003
TestHasProvidersConfig_*Detect old config existenceUS-003
TestValidateMigration_*Migration validationUS-003
TestMigrateConfig_DryRunDry run migrationUS-003
TestMigrateConfig_ActualActual migrationUS-003

3. pkg/providers/registry_test.go - Load Balancing Tests

Test NamePurposePRD Reference
TestModelRegistry_SingleConfigSingle config returns same resultUS-006
TestModelRegistry_RoundRobinSelection3-config round-robin selectionUS-006
TestModelRegistry_RoundRobinTwoConfigs2-config round-robin selectionUS-006
TestModelRegistry_ConcurrentAccessConcurrent access thread safetyUS-006
TestModelRegistry_RaceDetectionData race detectionUS-006
TestModelRegistry_ModelNotFoundModel not found errorUS-006
TestModelRegistry_EmptyRegistryEmpty registry handlingUS-006
TestModelRegistry_MultipleModelsMultiple model registrationUS-006
TestModelRegistry_MixedSingleAndMultipleSingle/multiple config mixUS-006
TestModelRegistry_CaseSensitiveModelNamesCase sensitivityUS-006

4. pkg/providers/factory/factory_test.go - Provider Factory Tests

Test NamePurposePRD Reference
TestCreateProviderFromConfig_OpenAICreate OpenAI providerUS-004
TestCreateProviderFromConfig_OpenAIDefaultDefault openai protocolUS-004
TestCreateProviderFromConfig_AnthropicCreate Anthropic providerUS-004
TestCreateProviderFromConfig_AntigravityCreate Antigravity providerUS-004
TestCreateProviderFromConfig_ClaudeCLICreate Claude CLI providerUS-004
TestCreateProviderFromConfig_CodexCLICreate Codex CLI providerUS-004
TestCreateProviderFromConfig_GitHubCopilotCreate GitHub Copilot providerUS-004
TestCreateProviderFromConfig_UnknownProtocolUnknown protocol error handlingUS-004
TestCreateProviderFromConfig_MissingAPIKeyMissing API key errorUS-004
TestExtractProtocolProtocol prefix extractionUS-004
TestCreateProvider_UsesModelListCreate using model_listUS-005
TestCreateProvider_FallbackToProvidersFallback to providersUS-005
TestCreateProvider_PriorityModelListOverProvidersmodel_list priorityUS-005

5. pkg/providers/integration_test.go - E2E Integration Tests

Test NamePurposePRD Reference
TestE2E_OpenAICompatibleProvider_NoCodeChangeZero-code provider additionGoal
TestE2E_LoadBalancing_RoundRobinLoad balancing actual effectUS-006
TestE2E_BackwardCompatibility_OldProvidersConfigOld config compatibilityUS-003
TestE2E_ErrorHandling_ModelNotFoundModel not foundFR-30
TestE2E_ErrorHandling_MissingAPIKeyMissing API keyFR-31
TestE2E_ErrorHandling_InvalidAPIBaseInvalid API baseFR-30
TestE2E_ToolCalls_OpenAICompatibleTool call support-
TestE2E_AntigravityProviderAntigravity providerUS-004
TestE2E_ClaudeCLIProviderClaude CLI providerUS-004

6. Performance Tests

Test NamePurpose
BenchmarkCreateProviderFromConfigProvider creation performance
BenchmarkGetModelConfigModel lookup performance
BenchmarkGetModelConfigParallelConcurrent lookup performance

Running Tests

bash
# Run all tests
go test ./pkg/... -v

# Run with data race detection
go test ./pkg/... -race

# Run specific package tests
go test ./pkg/config -v
go test ./pkg/providers -v

# Run E2E tests
go test ./pkg/providers -run TestE2E -v

# Run performance tests
go test ./pkg/providers -bench=. -benchmem

PRD Acceptance Criteria Mapping

PRD Acceptance CriteriaTest Cases
US-001: Add ModelConfig structTestModelConfig_Parsing, TestModelConfig_Validation
US-001: model_name uniqueTestConfig_ModelNameUniqueness
US-002: GetModelConfig methodTestConfig_GetModelConfig_*
US-003: Auto-convert providersTestConvertProvidersToModelList_*
US-003: Deprecation warningTestConfig_DeprecationWarning
US-003: Existing tests pass(existing test files unchanged)
US-004: Protocol prefix factoryTestExtractProtocol, TestCreateProviderFromConfig_*
US-004: Default prefix openaiTestCreateProviderFromConfig_OpenAIDefault
US-005: CreateProvider uses factoryTestCreateProvider_*
US-006: Round-robin selectionTestModelRegistry_RoundRobin*
US-006: Thread-safe atomicTestModelRegistry_RaceDetection

  1. Phase 1: Configuration Structure (US-001, US-002)

    • Implement ModelConfig struct
    • Implement GetModelConfig method
    • Run model_config_test.go

  2. Phase 2: Protocol Factory (US-004)

    • Implement CreateProviderFromConfig
    • Implement ExtractProtocol
    • Run factory_test.go

  3. Phase 3: Load Balancing (US-006)

    • Implement ModelRegistry
    • Implement round-robin selection
    • Run registry_test.go (with -race)

  4. Phase 4: Backward Compatibility (US-003, US-005)

    • Implement ConvertProvidersToModelList
    • Refactor CreateProvider
    • Run migration_test.go
    • Verify existing tests pass

  5. Phase 5: E2E Verification

    • Run integration_test.go
    • Manual testing with config.example.json