task-sdk/docs/api.rst
.. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
This page documents the full public API exposed in Airflow 3.0+ via the Task SDK python module.
If something is not on this page it is best to assume that it is not part of the public API and use of it is entirely at your own risk -- we won't go out of our way break usage of them, but we make no promises either.
.. autoapiclass:: airflow.sdk.DAG
The conf object is available as part of the Task SDK. It provides an interface to the
configurations, allowing you to read and interact with Airflow configuration values.
The macros module is available as part of the Task SDK. It provides builtin utility functions
for date manipulation and other common operations in Jinja templates and task code.
Available functions include:
ds_add(ds, days) - Add or subtract days from a date stringds_format(ds, input_format, output_format) - Format datetime stringsds_format_locale(ds, input_format, output_format, locale) - Format datetime strings with locale supportdatetime_diff_for_humans(dt, since) - Human-readable datetime differencesThe module also provides direct access to commonly used standard library modules:
json, time, uuid, dateutil, and random.
.. autoapifunction:: airflow.sdk.dag .. autoapifunction:: airflow.sdk.task
Task Decorators:
@task.run_if(condition, skip_message=None)
Run the task only if the given condition is met; otherwise the task is skipped. The condition is a callable
that receives the task execution context and returns either a boolean or a tuple (bool, message).@task.skip_if(condition, skip_message=None)
Skip the task if the given condition is met, raising a skip exception with an optional message.@task.<provider>, e.g. @task.python, @task.docker, etc., dynamically loaded from registered providers... autoapifunction:: airflow.sdk.task_group
.. autoapifunction:: airflow.sdk.setup
.. autoapifunction:: airflow.sdk.teardown
.. autofunction:: airflow.sdk.task .. autofunction:: airflow.sdk.setup .. autofunction:: airflow.sdk.teardown .. autofunction:: airflow.sdk.asset
.. autoapiclass:: airflow.sdk.BaseAsyncOperator
.. autoapiclass:: airflow.sdk.BaseBranchOperator
.. autoapiclass:: airflow.sdk.BaseOperator
.. autoapiclass:: airflow.sdk.BaseSensorOperator
.. autoapiclass:: airflow.sdk.BaseNotifier
.. autoapiclass:: airflow.sdk.BaseOperatorLink
.. autoapiclass:: airflow.sdk.BaseXCom
.. autoapiclass:: airflow.sdk.BranchMixIn
.. autoapiclass:: airflow.sdk.PokeReturnValue
.. autoapiclass:: airflow.sdk.SkipMixin
.. autoapiclass:: airflow.sdk.BaseHook
.. autoclass:: airflow.sdk.AsyncCallback
.. autoclass:: airflow.sdk.SyncCallback
.. autoclass:: airflow.sdk.DeadlineAlert
.. autoclass:: airflow.sdk.DeadlineReference
.. autoapiclass:: airflow.sdk.Connection
.. autoapiclass:: airflow.sdk.Variable
.. autoapiclass:: airflow.sdk.TaskGroup
.. autoclass:: airflow.sdk.TaskInstance
.. autoapiclass:: airflow.sdk.XComArg
.. autoapifunction:: airflow.sdk.literal
.. autoapiclass:: airflow.sdk.Param
.. autoclass:: airflow.sdk.ParamsDict
.. autoclass:: airflow.sdk.TriggerRule
.. autoapifunction:: airflow.sdk.get_current_context
.. autoapifunction:: airflow.sdk.get_parsing_context
.. autoclass:: airflow.sdk.TaskInstanceState
.. autoclass:: airflow.sdk.DagRunState
.. autoclass:: airflow.sdk.WeightRule
Setting Dependencies
.. autoapifunction:: airflow.sdk.chain
.. autoapifunction:: airflow.sdk.chain_linear
.. autoapifunction:: airflow.sdk.cross_downstream
Edges & Labels
~~~~~~~~~~~~~~
.. autoapiclass:: airflow.sdk.EdgeModifier
.. autoapiclass:: airflow.sdk.Label
Assets
------
.. autoapiclass:: airflow.sdk.Asset
.. autoapiclass:: airflow.sdk.AssetAlias
.. autoapiclass:: airflow.sdk.AssetAll
.. autoapiclass:: airflow.sdk.AssetAny
.. autoapiclass:: airflow.sdk.AssetWatcher
.. autoapiclass:: airflow.sdk.Metadata
Timetables
----------
.. autoapiclass:: airflow.sdk.AssetOrTimeSchedule
.. autoapiclass:: airflow.sdk.CronDataIntervalTimetable
.. autoapiclass:: airflow.sdk.CronTriggerTimetable
.. autoapiclass:: airflow.sdk.CronPartitionTimetable
.. autoapiclass:: airflow.sdk.DeltaDataIntervalTimetable
.. autoapiclass:: airflow.sdk.DeltaTriggerTimetable
.. autoapiclass:: airflow.sdk.EventsTimetable
.. autoapiclass:: airflow.sdk.MultipleCronTriggerTimetable
.. autoapiclass:: airflow.sdk.PartitionedAssetTimetable
Partition Mapper
----------------
.. autoapiclass:: airflow.sdk.PartitionMapper
.. autoapiclass:: airflow.sdk.ChainMapper
.. autoapiclass:: airflow.sdk.IdentityMapper
.. autoapiclass:: airflow.sdk.StartOfHourMapper
.. autoapiclass:: airflow.sdk.StartOfDayMapper
.. autoapiclass:: airflow.sdk.StartOfWeekMapper
.. autoapiclass:: airflow.sdk.StartOfMonthMapper
.. autoapiclass:: airflow.sdk.StartOfQuarterMapper
.. autoapiclass:: airflow.sdk.StartOfYearMapper
.. autoapiclass:: airflow.sdk.ProductMapper
.. autoapiclass:: airflow.sdk.AllowedKeyMapper
I/O Helpers
-----------
.. autoapiclass:: airflow.sdk.ObjectStoragePath
Execution Time Components
-------------------------
.. rubric:: Context
.. autoapiclass:: airflow.sdk.Context
The ``Context`` object represents the execution-time context available to tasks.
It corresponds to the same context that is exposed to Jinja templates during task execution.
For a complete list of available context variables (such as ``dag_run``,
``task_instance``, ``logical_date``, etc.), see the
:ref:`Templates reference <templates-ref>`.
.. rubric:: Logging
.. autofunction:: airflow.sdk.log.mask_secret
Observability
-------------
.. autoclass:: airflow.sdk.Trace
Everything else
---------------
.. autoapimodule:: airflow.sdk
:members:
:special-members: __version__
:exclude-members: BaseAsyncOperator, BaseOperator, DAG, dag, asset, Asset, AssetAlias, AssetAll, AssetAny, AssetWatcher, TaskGroup, TaskInstance, XComArg, get_current_context, get_parsing_context
:undoc-members:
:imported-members:
:no-index: