ContextQMD
Back to Datafusion

Apache DataFusion 54.1.0 Changelog

dev/changelog/54.1.0.md

55.0.04.6 KB
Original Source
<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->

Apache DataFusion 54.1.0 Changelog

This release consists of 19 commits from 9 contributors. See credits at the end of this changelog for more information.

See the upgrade guide for information on how to upgrade from previous versions.

Documentation updates:

  • [branch-54] Add datafusion.execution.enable_file_stream_work_stealing config #23296 (andygrove)

Other:

  • [branch-54] fix: preserve null_aware on logical JoinNode proto round-trip (backport #22104) #22785 (mithuncy)
  • [branch-54]: backport #22811 (bugfix: changed return type of spark's width_bucket to i64) #23087 (mbutrovich)
  • [branch-54] backport #22857 (Skip loading Parquet page index when row-group statistics already prove it cannot prune) #23088 (mbutrovich)
  • [branch-54] backport #23192 array_compact handle edge case with NULLs #23196 (comphead)
  • [branch-54] fix: Avoid panicing when stats are not available for a file group split (backport #23277) #23340 (mkleen)
  • [branch-54] fix: approx_distinct over-counts for utf8view (backport #22815, adapted) #23576 (mbutrovich)
  • [branch-54] fix: isolate anonymous file statistics cache (backport #22950, adapted) #23573 (mbutrovich)
  • [branch-54] fix: = ANY (SELECT ...) / <> ALL (SELECT ...) schema error (backport #22915) #23575 (mbutrovich)
  • [branch-54] fix: NestedLoopJoinExec emits spurious unmatched-left rows with multiple probe partitions (backport #22791) #23577 (mbutrovich)
  • [branch-54] fix: regex simplification of anchored patterns produces wrong results (backport #22727) #23578 (mbutrovich)
  • [branch-54] fix: Correctly compute nullability in recursive CTE schemas (backport #22552) #23579 (mbutrovich)
  • [branch-54] fix: handle IS TRUE correctly in EliminateOuterJoin (backport #22444) #23580 (mbutrovich)
  • [branch-54] fix: preserve no-filter SMJ matches across pending outer batches (backport #23049) #23574 (mbutrovich)
  • [branch-54] perf: avoid intermediate slice allocation in Spark slice function (backport #23481) #23582 (mbutrovich)
  • [branch-54] fix: don't duplicate volatile expressions when pushing projection into file scan (backport #23395, adapted) #23585 (fordN)
  • [branch-54] chore: fix cargo audit #23607 (alamb)
  • [branch-54] Resolve lost wakeup in SpillPoolReader with multiple conc… #23654 (pepijnve)
  • [branch-54] Handle nulls in type coercion of higher-order UDFs, map_extract, spark array_repeat (backport #23071) #23629 (gstvg)

Credits

Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.

    11	Matt Butrovich
     1	Andrew Lamb
     1	Andy Grove
     1	Ford
     1	Michael Kleen
     1	Mithun Chicklore Yogendra
     1	Oleks V
     1	Pepijn Van Eeckhoudt
     1	gstvg

Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.