analyze

analyze

Uses AI to analyze test failures and provide insights

This plugin analyzes failed tests using AI to provide detailed explanations and group similar failures. When enabled with --ai flag, it generates reports after test execution.

Usage

// in codecept.conf.js
exports.config = {
  plugins: {
    analyze: {
      enabled: true,
      clusterize: 5,
      analyze: 2,
      vision: false
    }
  }
}

Configuration

• clusterize (number) - minimum number of failures to trigger clustering analysis. Default: 5
• analyze (number) - maximum number of individual test failures to analyze in detail. Default: 2
• vision (boolean) - enables visual analysis of test screenshots. Default: false
• categories (array) - list of failure categories for classification. Defaults to:
  • Browser connection error / browser crash
  • Network errors (server error, timeout, etc)
  • HTML / page elements (not found, not visible, etc)
  • Navigation errors (404, etc)
  • Code errors (syntax error, JS errors, etc)
  • Library & framework errors
  • Data errors (password incorrect, invalid format, etc)
  • Assertion failures
  • Other errors
• prompts (object) - customize AI prompts for analysis
  • clusterize - prompt for clustering analysis
  • analyze - prompt for individual test analysis

Features

• Groups similar failures when number of failures >= clusterize value
• Provides detailed analysis of individual failures
• Analyzes screenshots if vision=true and screenshots are available
• Classifies failures into predefined categories
• Suggests possible causes and solutions

Parameters

• config Object Plugin configuration

Returns void