providers/google/docs/operators/cloud/workflows.rst
.. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
You can use Workflows to create serverless workflows that link series of serverless tasks together in an order you define. Combine the power of Google Cloud's APIs, serverless products like Cloud Functions and Cloud Run, and calls to external APIs to create flexible serverless applications.
For more information about the service visit
Workflows production documentation <Product documentation <https://cloud.google.com/workflows/docs/overview>__.
.. include:: /operators/_partials/prerequisite_tasks.rst
.. _howto/operator:WorkflowsCreateWorkflowOperator:
To create a workflow use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsCreateWorkflowOperator.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_create_workflow] :end-before: [END how_to_create_workflow]
The workflow should be define in similar why to this example:
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 0 :start-after: [START how_to_define_workflow] :end-before: [END how_to_define_workflow]
For more information about authoring workflows check official
production documentation <Product documentation <https://cloud.google.com/workflows/docs/overview>__.
.. _howto/operator:WorkflowsUpdateWorkflowOperator:
To update a workflow use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsUpdateWorkflowOperator.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_update_workflow] :end-before: [END how_to_update_workflow]
.. _howto/operator:WorkflowsGetWorkflowOperator:
To get a workflow use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsGetWorkflowOperator.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_get_workflow] :end-before: [END how_to_get_workflow]
.. _howto/operator:WorkflowsListWorkflowsOperator:
To list workflows use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsListWorkflowsOperator.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_list_workflows] :end-before: [END how_to_list_workflows]
.. _howto/operator:WorkflowsDeleteWorkflowOperator:
To delete a workflow use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsDeleteWorkflowOperator.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_delete_workflow] :end-before: [END how_to_delete_workflow]
.. _howto/operator:WorkflowsCreateExecutionOperator:
To create an execution use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsCreateExecutionOperator.
This operator is not idempotent due to API limitation.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_create_execution] :end-before: [END how_to_create_execution]
The create operator does not wait for execution to complete. To wait for execution result use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowExecutionSensor.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_wait_for_execution] :end-before: [END how_to_wait_for_execution]
.. _howto/operator:WorkflowsGetExecutionOperator:
To get an execution use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsGetExecutionOperator.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_get_execution] :end-before: [END how_to_get_execution]
.. _howto/operator:WorkflowsListExecutionsOperator:
To list executions use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsListExecutionsOperator.
By default this operator will return only executions for last 60 minutes.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_list_executions] :end-before: [END how_to_list_executions]
.. _howto/operator:WorkflowsCancelExecutionOperator:
To cancel an execution use
:class:~airflow.providers.google.cloud.operators.dataproc.WorkflowsCancelExecutionOperator.
.. exampleinclude:: /../../google/tests/system/google/cloud/workflows/example_workflows.py :language: python :dedent: 4 :start-after: [START how_to_cancel_execution] :end-before: [END how_to_cancel_execution]