providers/apache/livy/docs/changelog.rst
.. Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
.. http://www.apache.org/licenses/LICENSE-2.0
.. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
.. NOTE TO CONTRIBUTORS: Please, only add notes to the Changelog just below the "Changelog" header when there are some breaking changes and you want to add an explanation to the users on how they are supposed to deal with them. The changelog is updated and maintained semi-automatically by release manager.
apache-airflow-providers-apache-livy
4.5.3 .....
Misc
* ``Use common provider's get_async_connection in other providers (#56791)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
4.5.2
.....
Misc
New year means updated Copyright notices (#60344)Source conf from 'airflow.sdk.configuration.conf' for apache providers (#59966).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
TaskInstance unused method cleanup (#59835)4.5.1 .....
Misc
* ``Add backcompat for exceptions in providers (#58727)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
4.5.0
.....
.. note::
This release of provider is only available for Airflow 2.11+ as explained in the
Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>_.
Misc
Bump minimum Airflow version in providers to Airflow 2.11.0 (#58612).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Updates to release process of providers (#58316)4.4.5 .....
Misc
* ``Convert all airflow distributions to be compliant with ASF requirements (#58138)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Delete all unnecessary LICENSE Files (#58191)``
* ``Enable PT006 rule to 13 files in providers (apache) (#57998)``
* ``Enable ruff PLW0129 rule (#57516)``
4.4.4
.....
Misc
Migrate Apache providers & Elasticsearch to ''common.compat'' (#57016)Doc-only
* ``fix(livy): replace connection type (#55984)``
* ``Remove placeholder Release Date in changelog and index files (#56056)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Enable PT011 rule to provider tests (#56578)``
4.4.3
.....
Bug Fixes
Fix: use merged headers in LivyAsyncHook API call (#55072).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Prepare release for Sep 2025 1st wave of providers (#55203)Fix Airflow 2 reference in README/index of providers (#55240)Make term Dag consistent in providers docs (#55101)Switch pre-commit to prek (#54258)Prepare release for Aug 2025 1st wave of providers (#54193)Conn docs extra fix (#53942)4.4.2 .....
Misc
* ``Add Python 3.13 support for Airflow. (#46891)``
* ``Remove type ignore across codebase after mypy upgrade (#53243)``
* ``Remove upper-binding for "python-requires" (#52980)``
* ``Temporarily switch to use >=,< pattern instead of '~=' (#52967)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
4.4.1
.....
Misc
Move 'BaseHook' implementation to task SDK (#51873)Disable UP038 ruff rule and revert mandatory 'X | Y' in insintance checks (#52644)Replace models.BaseOperator to Task SDK for apache/livy (#52499)Drop support for Python 3.9 (#52072)Use BaseSensorOperator from task sdk in providers (#52296).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Separate out creation of default Connections for tests and non-tests (#52129)Remove pytest db markers from apache livy provider (#52099)Introducing fixture to create 'Connections' without DB in provider tests (#51930)4.4.0 .....
.. note:: This release of provider is only available for Airflow 2.10+ as explained in the Apache Airflow providers support policy https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers_.
Features
* ``add root parent information to OpenLineage events (#49237)``
Misc
~~~~
* ``Bump min Airflow version in providers to 2.10 (#49843)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Update description of provider.yaml dependencies (#50231)``
* ``Avoid committing history for providers (#49907)``
4.3.2
.....
Misc
~~~~
* ``remove superfluous else block (#49199)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
4.3.1
.....
Misc
~~~~
* ``Remove sqlite provider from preinstalled and use aiosqlite as core dep (#48966)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Remove unnecessary entries in get_provider_info and update the schema (#48849)``
* ``Remove fab from preinstalled providers (#48457)``
* ``Improve documentation building iteration (#48760)``
4.3.0
.....
Features
add OpenLineage configuration injection to LivyOperator (#47564).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Simplify tooling by switching completely to uv (#48223)Prepare docs for Mar 2nd wave of providers (#48383)Upgrade providers flit build requirements to 3.12.0 (#48362)Move airflow sources to airflow-core package (#47798)Remove links to x/twitter.com (#47801)4.2.1 .....
Misc
* ``Upgrade flit to 3.11.0 (#46938)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Move tests_common package to devel-common project (#47281)``
* ``Improve documentation for updating provider dependencies (#47203)``
* ``Add legacy namespace packages to airflow.providers (#47064)``
* ``Remove extra whitespace in provider readme template (#46975)``
4.2.0
.....
.. note::
This version has no code changes. It's released due to yank of previous version due to packaging issues.
4.1.0
.....
Features
Add 'endpoint_prefix' to 'LivyHook' (#45811)Misc
* ``Satisfy Mypy for Livy Hook (#46157)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Move provider_tests to unit folder in provider tests (#46800)``
* ``Removed the unused provider's distribution (#46608)``
* ``Fix doc issues found with recent moves (#46372)``
* ``Move Apache Livy to new provider structure (#46131)``
4.0.0
.....
.. note::
This release of provider is only available for Airflow 2.9+ as explained in the
`Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>`_.
Breaking changes
.. warning:: All deprecated classes, parameters and features have been removed from the Livy provider package. The following breaking changes were introduced:
Operators
get_hook method from LivyOperator. Use hook property insteadRemove Provider Deprecations in Apache Livy (#44631)
Misc
* ``Bump minimum Airflow version in providers to Airflow 2.9.0 (#44956)``
* ``Update DAG example links in multiple providers documents (#44034)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Use Python 3.9 as target version for Ruff & Black rules (#44298)``
* ``Update path of example dags in docs (#45069)``
3.9.2
.....
Misc
Set lower bound to asgiref>=2.3.0 (#43001).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Split providers out of the main "airflow/" tree into a UV workspace project (#42505)3.9.1 .....
Bug Fixes
* ``bugfix/livy-set-base-url (#41658)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
3.9.0
.....
.. note::
This release of provider is only available for Airflow 2.8+ as explained in the
`Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>`_.
Misc
~~~~
* ``Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Prepare docs for Aug 1st wave of providers (#41230)``
* ``Prepare docs 1st wave July 2024 (#40644)``
* ``Enable enforcing pydocstyle rule D213 in ruff. (#40448)``
3.8.1
.....
Misc
~~~~
* ``Faster 'airflow_version' imports (#39552)``
* ``Simplify 'airflow_version' imports (#39497)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Reapply templates for all providers (#39554)``
3.8.0
.....
.. note::
This release of provider is only available for Airflow 2.7+ as explained in the
`Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>`_.
Bug Fixes
Fix bug in LivyOperator when its trigger times out (#38916)Fix 'polling_interval' parameter docs in LivyOperator (#38979)Misc
* ``Bump minimum Airflow version in providers to Airflow 2.7.0 (#39240)``
* ``Update docstring 'LivyOperator' retry_args and deferrable docs (#39266)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Prepare docs 1st wave (RC1) April 2024 (#38863)``
* ``Bump ruff to 0.3.3 (#38240)``
3.7.3
.....
Bug Fixes
Fix rendering 'LivyOperator.spark_params' (#37361).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Add comment about versions updated by release manager (#37488)3.7.2 .....
Misc
* ``Bump aiohttp min version to avoid CVE-2024-23829 and CVE-2024-23334 (#37110)``
3.7.1
.....
Bug Fixes
Fix assignment of template field in '__init__' in 'livy.py' (#36490).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Speed up autocompletion of Breeze by simplifying provider state (#36499)Add documentation for 3rd wave of providers in Deember (#36464)Re-apply updated version numbers to 2nd wave of providers in December (#36380)3.7.0 .....
.. note::
This release of provider is only available for Airflow 2.6+ as explained in the
Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>_.
Misc
* ``Bump minimum Airflow version in providers to Airflow 2.6.0 (#36017)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Fix and reapply templates for provider documentation (#35686)``
* ``Prepare docs 3rd wave of Providers October 2023 - FIX (#35233)``
* ``Prepare docs 2nd wave of Providers November 2023 (#35836)``
* ``Use reproducible builds for providers (#35693)``
* ``Prepare docs 1st wave of Providers November 2023 (#35537)``
* ``Prepare docs 3rd wave of Providers October 2023 (#35187)``
* ``Pre-upgrade 'ruff==0.0.292' changes in providers (#35053)``
3.6.0
.....
.. note::
This release of provider is only available for Airflow 2.5+ as explained in the
`Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>`_.
Misc
Bump min airflow version of providers (#34728)Consolidate hook management in LivyOperator (#34431).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Refactor: consolidate import time in providers (#34402)3.5.4 .....
Misc
* ``Refactor regex in providers (#33898)``
* ``Improve modules import in Airflow providers by some of them into a type-checking block (#33754)``
3.5.3
.....
Misc
Refactor: Remove useless str() calls (#33629)Refactor: Simplify code in Apache/Alibaba providers (#33227)Simplify conditions on len() in providers/apache (#33564).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
D401 Support - Providers: Airbyte to Atlassian (Inclusive) (#33354)3.5.2 .....
Bug Fixes
* ``Fix 'LivyHook' TypeError exception on 'session_id' log format (#32051)``
Misc
~~~~
* ``Add default_deferrable config (#31712)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``D205 Support - Providers: Apache to Common (inclusive) (#32226)``
* ``Improve provider documentation and README structure (#32125)``
* ``Remove spurious headers for provider changelogs (#32373)``
* ``Prepare docs for July 2023 wave of Providers (#32298)``
3.5.1
.....
.. note::
This release dropped support for Python 3.7
Bug Fixes
Push Spark appId to XCOM for LivyOperator with deferrable mode (#31201)Misc
* ``Optimize deferred mode execution (#31685)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Apache provider docstring improvements (#31730)``
* ``Add discoverability for triggers in provider.yaml (#31576)``
* ``Add D400 pydocstyle check - Apache providers only (#31424)``
* ``Add note about dropping Python 3.7 for providers (#32015)``
3.5.0
.....
.. note::
This release of provider is only available for Airflow 2.4+ as explained in the
`Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>`_.
Misc
Bump minimum Airflow version in providers (#30917).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Upgrade ruff to 0.0.262 (#30809)Add full automation for min Airflow version for providers (#30994)Add cli cmd to list the provider trigger info (#30822)Add mechanism to suspend providers (#30422)Use '__version__' in providers not 'version' (#31393)Fixing circular import error in providers caused by airflow version check (#31379)Prepare docs for May 2023 wave of Providers (#31252)3.4.0 .....
Features
* ``Add non login-password auth support for SimpleHttpOpeator (#29206)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``adding trigger info to provider yaml (#29950)``
3.3.0
.....
Features
Add Livy Operator with deferrable mode (#29047).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Prepare docs for Jan 2023 mid-month wave of Providers (#28929)Fix providers documentation formatting (#28754)3.2.0 .....
.. note::
This release of provider is only available for Airflow 2.3+ as explained in the
Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>_.
Misc
* ``Move min airflow version to 2.3.0 for all providers (#27196)``
Features
Add template to livy operator documentation (#27404)Add Spark's 'appId' to xcom output (#27376)add template field renderer to livy operator (#27321).. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Update old style typing (#26872)Enable string normalization in python formatting - providers (#27205)Update docs for September Provider's release (#26731)Apply PEP-563 (Postponed Evaluation of Annotations) to non-core airflow (#26289)3.1.0 .....
Features
* ``Add auth_type to LivyHook (#25183)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Add documentation for July 2022 Provider's release (#25030)``
* ``Move provider dependencies to inside provider folders (#24672)``
* ``Remove 'hook-class-names' from provider.yaml (#24702)``
3.0.0
.....
Breaking changes
.. note::
This release of provider is only available for Airflow 2.2+ as explained in the
Apache Airflow providers support policy <https://github.com/apache/airflow/blob/main/PROVIDERS.rst#minimum-supported-version-of-airflow-for-community-managed-providers>_.
.. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Add explanatory note for contributors about updating Changelog (#24229)AIP-47 - Migrate livy DAGs to new design #22439 (#24208)Prepare docs for May 2022 provider's release (#24231)Update package description to remove double min-airflow specification (#24292)2.2.3 .....
Bug Fixes
* ``Fix: Add extra headers to all livy API requests instead of only to post_batch (#22510)``
2.2.2
.....
Bug Fixes
Fix mistakenly added install_requires for all providers (#22382)2.2.1 .....
Misc
* ``Add Trove classifiers in PyPI (Framework :: Apache Airflow :: Provider)``
2.2.0
.....
Features
Added retries to LivyHook #19384 (#21550)Misc
* ``Support for Python 3.10``
* ``add how-to guide for livy operator (#21529)``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Fixed changelog for January 2022 (delayed) provider's release (#21439)``
* ``Fix K8S changelog to be PyPI-compatible (#20614)``
* ``Fix template_fields type to have MyPy friendly Sequence type (#20571)``
* ``Fix MyPy errors in Apache Providers (#20422)``
* ``Add documentation for January 2021 providers release (#21257)``
* ``Remove ':type' lines now sphinx-autoapi supports typehints (#20951)``
* ``Update documentation for provider December 2021 release (#20523)``
* ``Update documentation for November 2021 provider's release (#19882)``
* ``Cleanup of start_date and default arg use for Apache example DAGs (#18657)``
* ``Prepare documentation for October Provider's release (#19321)``
* ``More f-strings (#18855)``
2.1.0
.....
Features
Fetching and logging livy session logs for LivyOperrator (#17393)Misc
* ``Optimise connection importing for Airflow 2.2.0``
.. Below changes are excluded from the changelog. Move them to
appropriate section above if needed. Do not delete the lines(!):
* ``Update description about the new ''connection-types'' provider meta-data (#17767)``
* ``Import Hooks lazily individually in providers manager (#17682)``
* ``Prepares docs for Rc2 release of July providers (#17116)``
* ``Prepare documentation for July release of providers. (#17015)``
* ``Removes pylint from our toolchain (#16682)``
2.0.0
.....
Breaking changes
Auto-apply apply_default decorator (#15667).. warning:: Due to apply_default decorator removal, this version of the provider requires Airflow 2.1.0+.
If your Airflow version is < 2.1.0, and you want to install this provider version, first upgrade
Airflow to at least version 2.1.0. Otherwise your Airflow package version will be upgraded
automatically and you will have to manually run airflow upgrade db to complete the migration.
.. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!):
Updated documentation for June 2021 provider release (#16294)More documentation update for June providers release (#16405)Synchronizes updated changelog after buggfix release (#16464)1.1.0 .....
Features
* ``Extend HTTP extra_options to LivyHook and operator (#14816)``
1.0.1
.....
* ``Updated documentation and readme files.``
1.0.0
.....
* ``Initial version of the provider.``