dev/AIRFLOW_ERROR_GUIDE.md
Table of Contents generated with DocToc
<!-- END doctoc generated TOC please keep comment here to allow auto update -->| Error Code | Exception Type | User-facing Error Message | Description | First Steps | Documentation |
|---|---|---|---|---|---|
| AERR001 | AirflowException | Dynamic task mapping exceeded limit | Happens when dynamically mapped tasks exceed the maximum number of tasks allowed. | Check the task count limit in the configuration. Consider increasing the task limit or optimizing task mapping logic. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dynamic-task-mapping.html |
| AERR002 | AirflowException | Task instance not found | Happens when the scheduler or webserver cannot locate a task instance in the database. | Verify that the database connection is stable. Check if the task instances exist in the metadata database and consider re-running the DAG. | https://airflow.apache.org/docs/apache-airflow/stable/administration/task-instances.html |
| AERR003 | AirflowException | Task is in 'None' state | Indicates a task instance has not been assigned a proper state, often due to missing execution context. | Ensure that the task is correctly configured and the execution context is provided, especially for dynamic tasks or task dependencies. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dag-run.html#execution-context |
| AERR004 | AirflowWebServerException | Webserver 502 Bad Gateway | Triggered when the webserver encounters an upstream issue or fails to proxy requests. | Check the webserver logs for more details. Ensure all upstream systems are working properly, and restart the webserver if necessary. | https://airflow.apache.org/docs/apache-airflow/stable/webserver.html#troubleshooting |
| AERR005 | KeyError | KeyError in Variable retrieval | Occurs when a requested Airflow Variable is not found in the metadata database. | Check if the Airflow variable exists. Ensure the correct database is used, and verify if the Airflow variable is defined in the Airflow UI or through code. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/variables.html |
| AERR006 | PermissionError | Access denied for SSH hook | Triggered when the SSH hook cannot authenticate or connect to the target server. | Verify the SSH credentials and network access to the target server. Test the connection using a manual SSH client before troubleshooting in Airflow. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/ssh.html |
| AERR007 | AirflowXComException | TaskInstance not recognized in XCom | Happens when a task's XCom entry is missing or corrupted in the metadata database. | Check if the XCom data is being pushed correctly. Inspect the DAG code for issues in data push logic, and consider clearing any corrupted XCom entries. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html |
| AERR008 | AirflowDatabaseException | Duplicate XCom entry detected | Occurs when the same XCom key-value pair is inserted multiple times into the database. | Ensure that the XCom key-value pair is unique for each task. Modify the DAG logic to avoid reusing keys or overwriting XCom entries. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html#avoiding-duplicate-keys |
| AERR009 | AirflowDatabaseException | Error creating database session | Triggered when Airflow cannot create a new database session. | Check the database connection settings and ensure the database is running. Verify user permissions and the number of concurrent connections allowed. | https://airflow.apache.org/docs/apache-airflow/stable/howto/set-up-database.html |
| AERR010 | AirflowConfigException | Strict validation in Dataset URI breaks existing DAGs | Happens when a dataset URI is not compliant with stricter validation rules introduced in newer versions. | Review the dataset URI format for compliance with new validation rules. Update the URI if necessary to meet the required standards. | https://airflow.apache.org/docs/apache-airflow/stable/datasets.html#uri-validation |
| AERR011 | AirflowException | Failed to upload logs to remote storage | Occurs when Airflow cannot push task logs to a configured remote storage backend. | Check the configuration for the remote storage backend. Ensure that the connection credentials are correct and the backend is accessible. | https://airflow.apache.org/docs/apache-airflow/stable/logging.html#remote-logging |
| AERR012 | AirflowDatabaseException | Cannot connect to airflow database | Happens when the metadata database is unreachable due to network or configuration issues. | Verify the network connection to the metadata database and check for any misconfigurations. Restart Airflow components if necessary. | https://airflow.apache.org/docs/apache-airflow/stable/administration/metadata-database.html |
| AERR013 | KeyError | KeyError in retrieving XCom value | Occurs when a requested XCom value is not found or incorrectly defined. | Ensure the XCom key and value are properly defined and passed between tasks. Double-check task execution order and parameter passing. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/xcoms.html#pushing-and-pulling-xcoms |
| AERR014 | ImportError | Missing dependency for KubernetesExecutor | Occurs when the required dependencies for the KubernetesExecutor are not installed. | Verify that all required dependencies for the KubernetesExecutor are installed. Use pip or your environment management tool to install missing packages. | https://airflow.apache.org/docs/apache-airflow/stable/executors/kubernetes.html#kubernetes-executor |
| AERR015 | AirflowDagPausedException | DAG is paused and not running | Indicates the DAG is manually paused and will not trigger scheduled runs. | Check the DAG status in the Airflow UI and unpause the DAG if needed. Verify the DAG configuration and dependencies. | https://airflow.apache.org/docs/apache-airflow/stable/dag-run.html#paused-dags |
| AERR016 | AirflowTaskTimeout | Task execution delayed indefinitely | Happens when a task does not start execution within the specified timeout. | Review the task timeout settings in the DAG configuration. Increase the timeout if necessary and check for system performance issues. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#timeouts |
| AERR017 | AirflowConfigException | Can't find executor class | Happens when the executor specified in the configuration file is not recognized or available. | Verify the executor configuration in the airflow.cfg file. Ensure the specified executor is installed and supported by your Airflow version. | https://airflow.apache.org/docs/apache-airflow/stable/executors/index.html |
| AERR018 | AirflowConfigException | Invalid value in airflow.cfg file | Triggered when airflow.cfg contains an invalid or unsupported value. | Review the airflow.cfg configuration file for errors or unsupported values. Consult the Airflow documentation for valid settings. | https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html |
| AERR019 | AirflowCliException | Airflow CLI authentication failed | Occurs when CLI commands cannot authenticate with the Airflow backend. | Check if the Airflow CLI has the proper credentials to access the backend. Ensure the correct connection configurations and environment variables are set. | https://airflow.apache.org/docs/apache-airflow/stable/cli-and-commands.html#authentication |
| AERR020 | AirflowTriggerException | Error triggering external API | Happens when a trigger for an external API fails to execute. | Check the API endpoint and ensure it is reachable. Review the Airflow logs to see if there is an error in the API trigger logic. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/external-task.html |
| AERR021 | AirflowDatabaseException | Database deadlock detected | Occurs when multiple processes are locked in conflicting database operations. | Investigate the database for any deadlocks or conflicting operations. Optimize queries or increase database capacity to avoid locking issues. | https://airflow.apache.org/docs/apache-airflow/stable/administration/database-optimizations.html |
| AERR022 | PermissionError | Permission error in KubernetesPodOperator | Occurs when the KubernetesPodOperator lacks permissions to perform required actions. | Review the KubernetesPodOperator configuration and ensure it has the necessary permissions to perform actions in your Kubernetes cluster. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/kubernetes.html#permissions |
| AERR023 | AirflowSchedulerException | Scheduler loop error | Triggered when the scheduler encounters an unexpected condition during its loop. | Check the scheduler logs for specific error messages. Review recent changes to the Airflow environment or DAGs that could affect scheduler behavior. | https://airflow.apache.org/docs/apache-airflow/stable/administration/scheduler.html#troubleshooting-scheduler |
| AERR024 | AirflowParseException | Broken DAG: syntax error | Occurs when a syntax error in the DAG file prevents it from being parsed. | Review the DAG file for any syntax errors and correct them. Use a linter or Python syntax checker to help identify issues. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#dags |
| AERR025 | AirflowDatabaseException | DagRun state update failed | Occurs when Airflow fails to update the state of a DAG run in the database. | Check the database connection and permissions. Review any database constraints or performance issues that could prevent state updates. | https://airflow.apache.org/docs/apache-airflow/stable/administration/metadata-database.html |
| AERR026 | AirflowTaskTimeout | Task marked as failed due to timeout | Happens when a task exceeds its maximum allowable execution time. | Increase the task execution timeout in the DAG configuration. Investigate the task logic for inefficiencies or external system delays. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#timeouts |
| AERR027 | FileNotFoundError | Task log not found | Happens when task logs are missing from the local or remote storage. | Check the task log configuration in Airflow. Ensure that logging paths are correctly configured and accessible. Verify permissions for remote storage. | https://airflow.apache.org/docs/apache-airflow/stable/logging.html#task-logging |
| AERR028 | ImportError | Cannot import module in BashOperator | Triggered when a script run by BashOperator references missing Python modules. | Ensure all required Python modules are installed in the environment. Check the requirements.txt or virtual environment for missing dependencies. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/bash.html#bashoperator |
| AERR029 | AirflowConfigException | Error loading connections from secret | Happens when Airflow fails to load connection credentials from a secret backend. | Check the secret backend configuration and credentials. Ensure the secret backend is accessible and correctly set up in Airflow. | https://airflow.apache.org/docs/apache-airflow/stable/security/secrets.html |
| AERR030 | AirflowWorkerException | Worker not responding | Happens when a worker node becomes unresponsive or fails to report its status. | Check the worker node logs for errors. Restart the worker node if necessary and verify network connectivity between the scheduler and worker. | https://airflow.apache.org/docs/apache-airflow/stable/administration/worker.html |
| AERR031 | AirflowException | Resource not found in GCP hook | Triggered when a GCP hook is unable to locate the specified resource. | Verify the resource exists in GCP. Check the GCP credentials and ensure that the hook is correctly configured to access the resource. | https://airflow.apache.org/docs/apache-airflow-providers-google/stable/index.html |
| AERR032 | AirflowExecutorException | Backend not reachable for Celery | Occurs when the CeleryExecutor cannot connect to the configured Celery backend. | Check the Celery backend configuration in airflow.cfg. Verify network access and that the Celery worker is properly connected and running. | https://airflow.apache.org/docs/apache-airflow/stable/executors/celery.html |
| AERR033 | AirflowException | Invalid cron expression | Occurs when the cron schedule provided in the DAG is invalid or unparsable. | Check the cron expression syntax for errors. Use a cron expression validator tool to confirm it is valid and properly formatted. | https://airflow.apache.org/docs/apache-airflow/stable/scheduler.html#scheduling-dags |
| AERR034 | UnpicklingError | UnpicklingError while running task | Happens when Airflow cannot deserialize data, often due to incompatible Python versions or corrupted data. | Ensure that Airflow and its dependencies are compatible with the Python version in use. Review any corrupted data and clear invalid entries. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/serialization.html |
| AERR035 | AirflowWorkerException | Task instance killed by external system | Occurs when an external system terminates a task instance during execution. | Review the external system logs to determine why it terminated the task. Modify task handling to account for external termination events. | https://airflow.apache.org/docs/apache-airflow/stable/operators.html#external-task-sensor |
| AERR036 | AirflowWorkerException | Worker died during task execution | Happens when the worker process handling a task crashes or is terminated. | Investigate the worker logs to identify the cause of the crash. Ensure that the worker environment has adequate resources and is properly configured. | https://airflow.apache.org/docs/apache-airflow/stable/administration/worker.html#troubleshooting |
| AERR037 | AirflowDatabaseException | Failed to fetch task state | Occurs when the metadata database does not return a valid state for a task instance. | Check the metadata database for data consistency and ensure that the database is responsive. Investigate for any database corruption or misconfigurations. | https://airflow.apache.org/docs/apache-airflow/stable/administration/metadata-database.html |
| AERR038 | ValueError | Cron interval parsing failed | Occurs when the cron expression for a DAG's schedule interval cannot be parsed. | Review the cron expression in the DAG configuration. Use a cron expression validator to confirm the format is correct and supported by Airflow. | https://airflow.apache.org/docs/apache-airflow/stable/scheduler.html#scheduling-dags |
| AERR039 | AirflowSchedulerException | Scheduler throttled due to excessive DAGs | Happens when the scheduler takes too long to process a large number of DAGs. | Consider optimizing DAG execution and task scheduling. Check the scheduler logs for any performance bottlenecks, and adjust the scheduler settings to handle large DAGs efficiently. | https://airflow.apache.org/docs/apache-airflow/stable/administration/scheduler.html#scheduler-performance |
| AERR040 | AirflowDatabaseException | DagRun execution_date conflicts | Occurs when there is a mismatch in execution_date for a DAG run in the database. | Ensure the execution_date is correctly defined and consistent across task instances. Check for any issues with time zone settings or manual overrides. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dag-run.html |
| AERR041 | AirflowTaskTimeout | Task is stuck in queued state | Occurs when a task remains queued without being picked up by an executor. | Check the executor configuration and ensure that sufficient worker nodes are available. Verify the queue settings and make sure the task is being routed to the correct executor. | https://airflow.apache.org/docs/apache-airflow/stable/administration/task-instance.html#queued |
| AERR042 | AirflowParseException | Error while parsing DAG file | Triggered when the scheduler encounters syntax errors or invalid code in a DAG file. | Review the DAG file for any code errors or invalid syntax. Use a Python linter to catch issues before loading the DAG into Airflow. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#dags |
| AERR043 | AirflowDagCycleException | Task dependency cycle detected | Triggered when task dependencies in a DAG create an infinite loop. | Check task dependencies in the DAG and ensure that there are no circular references. Modify dependencies to prevent infinite loops. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#task-dependencies |
| AERR044 | AirflowTaskException | Task failed due to retries exceeded | Triggered when a task exhausts its retry limit without succeeding. | Increase the retry limit in the DAG configuration or modify the task logic to handle failure scenarios more gracefully. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#retries |
| AERR045 | ValueError | ValueError in task arguments | Happens when a task operator is provided with invalid or incompatible arguments. | Review the operator arguments and ensure they are correctly specified. Check the Airflow documentation for valid arguments for the operator used. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator.html |
| AERR046 | AirflowTaskException | Task queue not found | Occurs when the task's queue is not recognized by the executor. | Ensure the specified queue exists in the Airflow configuration. Review the executor settings to confirm it can process tasks from the specified queue. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/queues.html |
| AERR047 | AirflowSchedulerException | Executor cannot retrieve task instance | Happens when the executor fails to fetch task instance details from the database. | Verify the database connection and ensure the task instance details are present in the metadata database. Check the executor logs for any errors. | https://airflow.apache.org/docs/apache-airflow/stable/administration/task-instance.html#status |
| AERR048 | AirflowWebServerException | Webserver connection refused | Occurs when the webserver process is not running or accessible. | Check the webserver logs for errors. Restart the webserver and verify that it is running and accessible through the configured URL. | https://airflow.apache.org/docs/apache-airflow/stable/webserver.html |
| AERR049 | AirflowTaskException | Invalid return type from PythonOperator | Happens when a PythonOperator returns a value of an unexpected type. | Ensure the Python function used in the PythonOperator returns the expected type. Review the function implementation for type correctness. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html |
| AERR050 | AirflowConfigException | Config error: executor not defined | Triggered when the Airflow configuration does not specify a valid executor. | Review the airflow.cfg file and verify that the executor is correctly specified. Ensure that the specified executor is installed and properly configured. | https://airflow.apache.org/docs/apache-airflow/stable/executors/index.html |
| AERR051 | AirflowException | Error in task failure hook execution | Triggered when the failure hook defined for a task encounters an error. | Check the failure hook configuration for errors. Review the task and hook logs to identify the cause of the failure and resolve any issues. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator.html#failure-hooks |
| AERR052 | AirflowTemplateException | Failed to resolve template variable | Triggered when a task's templated field contains errors or undefined variables. | Review the task templated fields for any errors or undefined variables. Ensure all variables are defined and passed correctly in the DAG. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operators.html#templating |
| AERR053 | AirflowSchedulerException | Scheduler process killed unexpectedly | Triggered when the scheduler process is terminated due to resource or system issues. | Investigate the system logs for resource-related issues. Increase system resources or adjust scheduler configurations to prevent termination. | https://airflow.apache.org/docs/apache-airflow/stable/administration/scheduler.html |
| AERR054 | AirflowTaskTimeout | Task failed with exit code 1 | Occurs when a task script or subprocess exits with a non-zero code indicating failure. | Check the task script or subprocess logs for the error code and cause of failure. Debug the script and address any underlying issues. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/bash.html |
| AERR055 | AirflowDatabaseException | TaskInstance already exists in database | Occurs when a duplicate TaskInstance entry is created in the metadata database. | Review task instance scheduling and ensure that duplicate entries are not being created. Investigate DAG scheduling logic for potential issues. | https://airflow.apache.org/docs/apache-airflow/stable/administration/metadata-database.html |
| AERR056 | AirflowDatabaseException | Could not reach the database | Indicates connectivity issues with the metadata database, often due to misconfiguration. | Verify the connection settings in airflow.cfg for the metadata database. Ensure that the database is accessible and network connectivity is working. | https://airflow.apache.org/docs/apache-airflow/stable/administration/metadata-database.html#troubleshooting |
| AERR057 | AirflowExecutorException | Task stuck in 'deferred' state | Triggered when a task using deferrable operators remains in 'deferred' longer than expected. | Review the deferrable operator configuration and execution logic. Ensure that the task is properly resumed and that there are no long delays in task deferral. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/deferrable.html |
| AERR058 | AirflowException | Error loading custom operator | Indicates a problem in importing or defining a custom operator in the DAG. | Check the custom operator import path and ensure it is available. Verify that the custom operator class is correctly defined and instantiated. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator.html#creating-custom-operators |
| AERR059 | AirflowTriggerException | Trigger timeout for external task | Occurs when an ExternalTaskSensor exceeds its timeout waiting for an external task. | Increase the timeout setting for the ExternalTaskSensor. Check if the external task is delayed and adjust the DAG dependencies accordingly. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/external-task.html |
| AERR060 | AirflowDagImportException | DagBag import errors | Happens when the scheduler encounters issues loading DAG files into the DagBag. | Check the DAG files for syntax or configuration errors. Inspect the DagBag loading process to ensure no invalid DAGs are being included. | https://airflow.apache.org/docs/apache-airflow/stable/scheduler.html#dagbag |
| AERR061 | AirflowDagNotFound | DAG not found in trigger DAG run | Triggered when the DAG specified in a TriggerDagRunOperator does not exist. | Check the name of the target DAG in the TriggerDagRunOperator. Ensure that the DAG exists and is correctly spelled in the configuration. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/trigger-dag.html |
| AERR062 | AirflowDagCycleException | Circular dependencies in DAG | Triggered when tasks in a DAG form a circular dependency loop. | Review the task dependencies and remove any circular references. Modify the DAG structure to avoid infinite dependency loops. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#task-dependencies |
| AERR063 | AirflowException | Error in on_failure_callback for DAG | Occurs when the on_failure_callback for a DAG raises an exception. | Check the on_failure_callback function for errors. Ensure it handles exceptions correctly and does not raise additional errors during task failure. | https://airflow.apache.org/docs/apache-airflow/stable/advanced/alerts.html |
| AERR064 | AirflowTriggerException | TriggerDagRunOperator failed | Happens when the TriggerDagRunOperator cannot trigger the specified target DAG. | Verify the configuration of the TriggerDagRunOperator. Ensure the target DAG exists and the trigger parameters are correct. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/trigger-dag.html |
| AERR065 | AirflowTaskException | DAG execution failed | Indicates that the overall execution of a DAG run failed due to errors in one or more tasks. | Check the logs for each task to determine which task failed. Review task dependencies, configurations, and retries. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dag-run.html#status |
| AERR066 | AirflowSchedulerException | Scheduler heartbeat failed | Occurs when the scheduler process stops sending heartbeats, often due to resource issues. | Investigate the system resources, such as CPU and memory, and increase them if necessary. Restart the scheduler and monitor system performance. | https://airflow.apache.org/docs/apache-airflow/stable/administration/scheduler.html |
| AERR067 | AirflowConfigException | Scheduler crashes when passing invalid value to argument in default_args | Occurs when default_args in a DAG contains an invalid or incompatible parameter. | Review the default_args section in the DAG definition. Ensure all parameters are valid and compatible with the Airflow version in use. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#default-args |
| AERR068 | AirflowApiException | HTTP error while connecting to API | Occurs when Airflow encounters connectivity issues or invalid responses from an external API. | Check the API connection settings and ensure that the external service is reachable. Verify that the API is returning valid responses. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/http.html |
| AERR069 | AirflowSchedulerException | Scheduler backlog due to excessive retries | Happens when too many tasks are retried, overloading the scheduler. | Adjust the task retry logic and retry limits. Review task logs for patterns that lead to excessive retries and optimize DAG logic. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#retries |
| AERR070 | AirflowException | DAG folder not found | Triggered when the directory specified for DAGs does not exist or is inaccessible. | Ensure that the DAG directory exists and is accessible by the Airflow scheduler and webserver. Check directory permissions and paths. | https://airflow.apache.org/docs/apache-airflow/stable/administration/dags-directory.html |
| AERR071 | AirflowException | Max active tasks for DAG exceeded | Triggered when the number of concurrent tasks for a DAG exceeds the configured limit. | Increase the task concurrency settings in the DAG configuration or adjust the task dependencies to limit the number of concurrent tasks. | https://airflow.apache.org/docs/apache-airflow/stable/scheduler.html#concurrency |
| AERR072 | AirflowException | Cannot find DAG run in the database | Happens when the specified DAG run is missing or deleted from the metadata database. | Verify the DAG run's existence in the metadata database. Ensure the run has not been manually deleted or corrupted. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dag-run.html |
| AERR073 | AirflowException | Airflow CLI not recognized | Happens when the Airflow CLI command is not installed or available in the PATH. | Ensure that Airflow is properly installed and that the CLI is accessible in the environment PATH. Reinstall or reconfigure Airflow if necessary. | https://airflow.apache.org/docs/apache-airflow/stable/cli-and-commands.html |
| AERR074 | AirflowDatabaseException | SQLAlchemy database connection error | Happens when the connection string for the metadata database is invalid or unreachable. | Check the connection string in the Airflow configuration. Ensure the database is reachable and verify the credentials and network configuration. | https://airflow.apache.org/docs/apache-airflow/stable/administration/database-setup.html |
| AERR075 | PermissionError | Permission denied for Airflow logs | Occurs when Airflow does not have the required permissions to read or write logs. | Check the file permissions for the Airflow logs directory. Ensure that the user running Airflow has proper read and write access. | https://airflow.apache.org/docs/apache-airflow/stable/logging.html |
| AERR076 | AirflowException | Task marked as 'up for retry' without retries available | Happens when a task is incorrectly marked for retry but has exceeded its limit. | Check the task's retry configuration in the DAG. Ensure that the retry count is properly set in the task's arguments and verify if retries are needed. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#retries |
| AERR077 | AirflowConfigException | Invalid section in airflow.cfg | Triggered when airflow.cfg contains an unrecognized configuration section. | Review your airflow.cfg file to ensure that all sections are correctly defined and recognized. Consult the official Airflow configuration reference to verify your settings. | https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html |
| AERR078 | AirflowException | Task dependencies are not met | Indicates a task cannot start due to upstream dependencies not being completed. | Ensure that all upstream tasks have been successfully completed before the task is triggered. Review the task dependencies in your DAG to make sure they are properly defined. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/task-instance.html |
| AERR079 | AirflowException | Task not found in serialized DAG | Occurs when the task referenced is missing from the serialized DAG in the database. | Check your DAG serialization settings and verify that all tasks are properly serialized. Ensure that the task is defined correctly in the DAG file. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#dag-serialization |
| AERR080 | AirflowException | Worker node not available | Triggered when an executor cannot find a suitable worker to run a task. | Check if there are any worker nodes available and verify if the executor configuration is correct. Ensure the workers are running and properly configured in the Airflow setup. | https://airflow.apache.org/docs/apache-airflow/stable/executors/index.html |
| AERR081 | ModuleNotFoundError | Python dependency not installed | Occurs when a Python package required by a DAG is missing. | Verify that all required Python packages are installed in the environment where Airflow is running. Install any missing dependencies using pip or a virtual environment manager. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html#python-packages |
| AERR082 | AirflowExecutorException | Celery task not acknowledged | Occurs when a Celery task is not acknowledged by a worker. | Check the Celery worker logs to determine why the task was not acknowledged. Ensure that Celery is properly configured and running in your Airflow setup. | https://airflow.apache.org/docs/apache-airflow/stable/executors/celery.html |
| AERR083 | AirflowTaskTimeout | Task duration exceeds timeout | Happens when a task exceeds its specified execution timeout. | Review the task's execution timeout setting. If necessary, increase the timeout or optimize the task to ensure it completes within the allowed duration. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#timeouts |
| AERR084 | AirflowException | Error migrating database to Airflow version 2.9.3 | Indicates issues during metadata database migration, often due to schema mismatches. | Check the database schema and migration logs for errors. You may need to run manual database migrations or troubleshoot database connectivity and compatibility issues. | https://airflow.apache.org/docs/apache-airflow/stable/administration/database-migrations.html |
| AERR085 | AirflowException | Error writing metrics to statsd | Occurs when Airflow fails to send metrics data to the configured statsd server. | Verify the connection settings to your statsd server. Ensure that the statsd server is reachable and properly configured in Airflow's settings. | https://airflow.apache.org/docs/apache-airflow/stable/metrics.html |
| AERR086 | AirflowException | DAG import timeout | Occurs when DAG files take too long to parse, possibly due to large files or inefficient code. | Review your DAG files to identify any large or inefficient code. Optimize DAG parsing and consider splitting large DAGs into smaller ones to improve performance. | https://airflow.apache.org/docs/apache-airflow/stable/scheduler.html#dag-parsing |
| AERR087 | AirflowTemplateException | Invalid template in BashOperator command | Happens when the command template in BashOperator contains errors. | Check the BashOperator template for syntax errors. Ensure all placeholders and variables are correctly defined and are available in the context of the task. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/bash.html |
| AERR088 | AirflowDagCycleException | Multiple DAGs with same ID | Triggered when two or more DAGs have the same ID, causing conflicts. | Ensure that each DAG has a unique DAG ID. Review your DAG files to identify any duplicate DAG IDs and resolve the conflict. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html#dag-id |
| AERR089 | AttributeError | AttributeError in PythonOperator | Occurs when the Python callable in a PythonOperator is improperly defined or missing required attributes. | Check the Python callable in your PythonOperator. Ensure that all required arguments and attributes are defined and passed correctly. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html |
| AERR090 | AirflowException | Error in callback execution for TaskInstance | Occurs when a failure callback defined for a task raises an error. | Review the callback function for errors. Ensure that the function is correctly defined and handles exceptions as expected. | https://airflow.apache.org/docs/apache-airflow/stable/advanced/alerts.html |
| AERR091 | AirflowTaskTimeout | Operator execution exceeded SLA | Happens when a task operator runs longer than its Service Level Agreement (SLA). | Review the task's execution time and optimize the code to ensure it completes within the specified SLA. Adjust the SLA if necessary to account for task duration. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#sla |
| AERR092 | AirflowDagNotFound | DAG not found in scheduler | Occurs when a DAG file is not loaded into the scheduler, often due to parsing errors or missing files. | Check for errors in the DAG parsing logs. Ensure that the DAG file exists in the DAGs folder and is properly formatted. | https://airflow.apache.org/docs/apache-airflow/stable/scheduler.html#dagbag |
| AERR093 | AirflowException | Error in email notification for task | Triggered when an email notification for a task failure cannot be sent. | Verify the email configuration in Airflow. Check if the SMTP server is correctly configured and reachable from the Airflow instance. | https://airflow.apache.org/docs/apache-airflow/stable/advanced/alerts.html#email-alerts |
| AERR094 | AirflowWebServerException | Airflow webserver won't start | Happens when the webserver process fails, often due to misconfiguration or missing dependencies. | Check the webserver logs for errors. Ensure that all required dependencies are installed and the configuration is correct. | https://airflow.apache.org/docs/apache-airflow/stable/webserver.html |
| AERR095 | ModuleNotFoundError | No module named '...' in DAG file | Occurs when the DAG imports a Python module that is not installed or available. | Install the missing Python module using pip or ensure the module is available in the environment where Airflow is running. | https://airflow.apache.org/docs/apache-airflow/stable/howto/operator/python.html |
| AERR096 | AirflowTemplateException | Missing template field in operator | Happens when a required templated field is not defined for a task operator. | Review the task operator and ensure all required templated fields are defined. Add any missing fields to the operator's parameters. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operators.html#templating |
| AERR097 | AirflowException | Error syncing DAGs to remote storage | Happens when DAG files fail to sync with a remote storage backend like S3 or GCS. | Verify the connection to the remote storage backend. Ensure that the correct credentials are configured and that the storage service is accessible. | https://airflow.apache.org/docs/apache-airflow/stable/storage.html#remote-storage-backends |
| AERR098 | AirflowTaskTimeout | Subprocess in task exceeded timeout | Happens when a subprocess spawned by a task runs longer than its allowed time. | Review the subprocess configuration and ensure it completes within the task's timeout. If needed, increase the timeout or optimize the subprocess execution. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/operator.html#timeouts |
| AERR099 | AirflowDatabaseException | Inconsistent database schema | Occurs when the metadata database schema is outdated or corrupted. | Check the database schema and run any necessary migrations. Ensure the database is in sync with the current Airflow version. | https://airflow.apache.org/docs/apache-airflow/stable/administration/database-setup.html |
| AERR100 | AirflowException | Invalid DAG structure | Triggered when a DAG's dependencies or attributes are incorrectly defined. | Review the DAG structure for errors. Ensure that all dependencies are correctly defined and that no circular dependencies exist. | https://airflow.apache.org/docs/apache-airflow/stable/concepts/dags.html |