ContextQMD
Back to Datafusion

Apache DataFusion 50.0.0 Changelog

dev/changelog/50.0.0.md

53.1.041.7 KB
Original Source
<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->

Apache DataFusion 50.0.0 Changelog

This release consists of 315 commits from 79 contributors. See credits at the end of this changelog for more information.

See the upgrade guide for information on how to upgrade from previous versions.

Breaking changes:

  • Support multiple ordered array_agg aggregations #16625 (findepi)
  • Make AsyncScalarUDFImpl::invoke_async_with_args consistent with ScalarUDFImpl::invoke_with_args #16902 (geetanshjuneja)
  • Derive WindowUDFImpl equality, hash from Eq, Hash traits #17081 (findepi)
  • Remove redundant plan from extension's check_invariants #17199 (findepi)
  • feat: Make parquet_encryption a non-default feature #17137 (miroim)
  • chore: fix typos #17135 (waynexia)
  • Use a struct for ProjectionExpr #17398 (adriangb)
  • Use DataFusionError instead of ArrowError in FileOpenFuture #17397 (adriangb)
  • Use return_field instead of return_type for calling aggregates via FFI #17407 (timsaucer)

Performance related:

  • feat: improve LiteralGuarantee for the case like (a=1 AND b=1) OR (a=2 AND b=3) #16762 (haohuaijin)
  • optimize initcap function by avoiding memory allocation #16878 (waynexia)
  • speedup date_trunc (~7x faster) in some cases #16859 (waynexia)
  • Feature: Improve hash Expr performance #16977 (tobixdev)
  • Perf: Port arrow-rs optimization for get_buffer_memory_size and add fast path for no buffer for gc string view #17008 (zhuqi-lucas-001)
  • Simplify comparisons and binary operations involving NULL #17088 (findepi)
  • Eliminate all redundant aggregations #17139 (findepi)

Implemented enhancements:

  • feat: Allow tree explain format width to be customizable #16827 (nuno-faria)
  • feat(spark): Implement Spark string function luhn_check #16848 (Standing-Man)
  • feat(spark): implement Spark datetime function last_day #16828 (Standing-Man)
  • feat: Add ScalarValue::{new_one,new_zero,new_ten,distance} support for Decimal128 and Decimal256 #16831 (theirix)
  • feat: support distinct for window #16925 (zhuqi-lucas-001)
  • feat: add multi level merge sort that will always fit in memory #15700 (rluvaton)
  • feat: [datafusion-spark] Implement next_day function #16780 (petern48)
  • feat: Support distinct window for sum #16943 (zhuqi-lucas-001)
  • feat(spark): implement Spark math function rint #16924 (chenkovsky)
  • feat(spark): implement Spark string function like/ilike #16962 (chenkovsky)
  • feat: Cache Parquet metadata in built in parquet reader #16971 (nuno-faria)
  • feat: Add Arc<ConfigOptions> to ScalarFunctionArgs, don't copy ConfigOptions on each query #16970 (Omega359)
  • feat(spark): implement spark hash function crc32/sha1 #17032 (chenkovsky)
  • feat: Limit the memory used in the file metadata cache #17031 (nuno-faria)
  • feat: Dynamic Parquet encryption and decryption properties #16779 (adamreeve)
  • feat: Use Cached Metadata for ListingTable Statistics #17022 (shehabgamin)
  • feat(spark): implement Spark math function mod/pmod #16829 (chenkovsky)
  • feat(spark): implement Spark math function bit_get/bit_count #16942 (chenkovsky)
  • feat: add isodow (ISO day-of-week) support to date_part (Monday = 0) #17112 (ayemjay)
  • feat(spark): implement spark datetime function date_add/date_sub #17024 (chenkovsky)
  • feat: Add the ability to review the contents of the Metadata Cache #17126 (nuno-faria)
  • feat: add datafusion-physical-adapter, implement predicate adaptation missing fields of structs #16589 (adriangb)
  • feat: implement QUALIFY clause #16933 (haohuaijin)
  • feat: allow to spawn/spawn_blocking on a provided runtime in RecordBatchReceiverStreamBuilder #17239 (rluvaton)
  • feat: Support SortMergeJoin proto serde #17296 (milenkovicm)
  • feat(spark): implement Spark bitmap function bitmap_count #17179 (SparkApplicationMaster)
  • feat: Track peak value in tracked consumer #17327 (wForget)
  • feat(spark): implement Spark conditional function if #16946 (chenkovsky)
  • feat(spark): implement Spark width_bucket function #17331 (davidlghellin)
  • feat: Make Parquet EncryptionFactory async #17342 (adamreeve)
  • feat: Support FILTER clause in aggregate window functions #17378 (geoffreyclaude)
  • feat: Support binary data types for SortMergeJoin on clause #17431 (stuartcarnie)

Fixed bugs:

  • fix: The inconsistency between scalar and array on the cast decimal to timestamp #16539 (chenkovsky)
  • fix: unit test for object_storage #16824 (chenkovsky)
  • fix(docs): Update broken links to TableProvider docs #16830 (jcsherin)
  • fix: PlaceholderRowExec::partition_statistics #16851 (crepererum)
  • fix: skip predicates on struct unnest in PushDownFilter #16790 (akoshchiy)
  • fix: regex bench #16890 (chenkovsky)
  • fix: ComposedPhysicalExtensionCodec does not use the same codec as encoding when decoding #16986 (Thearas)
  • fix: Remove datafusion.execution.parquet.cache_metadata config #17062 (jonathanc-n)
  • fix: Add missing member to visitor for ConfigFileEncryptionProperties #17103 (corwinjoy)
  • fix(ci): update datafusion-physical-expr-adapter version to 49.0.1in Cargo.lock #17209 (miroim)
  • fix: respect inexact flags in row group metadata #16412 (CookiePieWw)
  • fix: deserialization error for FilterExec (predicates with inlist) #17224 (haohuaijin)
  • FFI_RecordBatchStream was causing a memory leak #17190 (timsaucer)
  • fix: Windows paths crashing core tests #17231 (nuno-faria)
  • fix: sort should always output batches with batch_size rows #17244 (rluvaton)
  • fix: align array_has null buffer for scalar #17272 (comphead)
  • fix: dataframe function count_all with alias #17282 (Loaki07)
  • fix: correct readme field in Cargo.toml #17310 (Weijun-H)
  • fix(doc): update the link of deprecation guidelines (#17328) #17329 (ivila)
  • fix: lazy case else evaluation #17311 (chenkovsky)
  • fix: set distinct_count to Absent when merging statistics #17385 (adriangb)
  • fix: Remove duplicate filter from CrossJoin unparsing #17382 (jonathanc-n)
  • fix: set IPC alignment based on schema #17363 (ding-young)
  • fix: return ALL constants in EquivalenceProperties::constants #17404 (crepererum)
  • fix: align map_keys nullability flag #17454 (comphead)

Documentation updates:

  • docs: Fix broken links #16839 (2010YOUY01)
  • Add note to upgrade guide about MSRV update #16845 (alamb)
  • [main] Update version to 49.0.0, add 49.0.0 changelog #16855 (alamb)
  • Improve async_udf example and docs #16846 (alamb)
  • Docs: Update Upgrading.md to reflect 49.0.0 is released #16853 (alamb)
  • docs: Remove references to DataFusion for Ray sub project #16966 (andygrove)
  • Add temp_directory and max_temp_directory_size runtime config variables #16934 (delamarch3)
  • Add sql_parser.default_null_ordering config option to customize the default null ordering #16963 (goldmedal)
  • Added Example for Statistical Functions in Docs #16927 (Adez017)
  • Fix window_functions docs formatting #17005 (mattmatravers)
  • docs: Fix 'Analaysis' typo in query optimizer docs #17015 (petern48)
  • docs: Fix random extra bullet for 'Analytical Functions' #17014 (petern48)
  • docs: Fix failing documentation check in CI #17026 (adamreeve)
  • Upgrade arrow/parquet to 56.0.0 #16690 (alamb)
  • fix error result in execute&pre_selection #16930 (acking-you)
  • docs: Fix failing CI #17041 (liamzwbao)
  • Docs: Add Examples to Config Options page #17039 (alamb)
  • Docs: Add Tuning Guide for small data / short queries #17040 (alamb)
  • Docs: Update the crate configuration / build settings page #17038 (alamb)
  • Support centroids config for approx_percentile_cont_with_weight #17003 (liamzwbao)
  • Add ExecutionPlan::reset_state #17028 (adriangb)
  • Docs: Add Tuning Guide for larger-than-memory queries #17069 (2010YOUY01)
  • Link UdfEq and PtrEq to help understand relationship #17082 (findepi)
  • Derive AggregateUDFImpl equality, hash from Eq, Hash traits #17130 (findepi)
  • chore: Clarify EmptyRelation description #17157 (comphead)
  • Update dev env documentation to reflect pinned rust version #17107 (Jefffrey)
  • Differentiate 0-row and 1-row EmptyRelation in EXPLAIN #17145 (findepi)
  • (Re)Support old syntax for approx_percentile_cont and approx_percentile_cont_with_weight #16999 (alamb)
  • Derive ScalarUDFImpl equality, hash from Eq, Hash traits #17164 (findepi)
  • #17128 Add support for chr(0) #17131 (pepijnve)
  • [main] Update version to 49.0.1 and add changelog (#17175) #17191 (alamb)
  • Docs: Consolidate feature proposal content into roadmap #17156 (alamb)
  • Doc: Update upgrade guide for the rewritten NLJ operator #17202 (2010YOUY01)
  • Support serializing generate_series in datafusion-proto #17200 (cetra3)
  • Fix broken links in user docs #17228 (AdamGS)
  • Format Date32 to string given timestamp specifiers #15361 (friendlymatthew)
  • Improve documentation for Signature, Volatility, and TypeSignature #17264 (alamb)
  • [main] Forward port 49.0.2 version and changelog (#17277) #17287 (alamb)
  • Document schema merging. #17249 (wiedld)
  • Support from-first SQL syntax #17295 (simonvandel)
  • Add cfg(feature = "avro") attribute to Avro example in SQL API docs #17142 (kosiew)
  • Push the limits past window functions #17347 (avantgardnerio)
  • Refactor DataSourceExec::try_swapping_with_projection to simplify and remove abstraction leakage #17395 (adriangb)
  • doc: Document caveats of swap_inputs() interface in join executors #17373 (2010YOUY01)
  • Fix syntax error in DDL documentation example #17412 (pepijnve)
  • Add MSRV change to upgrade guide #17406 (findepi)
  • Add PhysicalExpr::is_volatile_node to upgrade guide #17443 (adriangb)
  • docs: Render -- properly in profiling docs #17430 (petern48)

Other:

  • chore: use equals_datatype for BinaryExpr #16813 (comphead)
  • chore: add tests for out of bounds for NullArray #16802 (comphead)
  • Refactor binary.rs tests into modular submodules under binary/tests #16782 (kosiew)
  • cache generation of dictionary keys and null arrays for ScalarValue #16789 (adriangb)
  • refactor(examples): remove redundant call to create directory in parquet_embedded_index.rs #16825 (jcsherin)
  • Add benchmark for ByteViewGroupValueBuilder #16826 (zhuqi-lucas-001)
  • Simplify try cast expr evaluation #16834 (lewiszlw)
  • Fix flaky test case in joins.slt #16849 (findepi)
  • chore(deps): bump sysinfo from 0.35.2 to 0.36.1 #16850 (dependabot[bot])
  • chore(deps): bump aws-credential-types from 1.2.3 to 1.2.4 #16815 (dependabot[bot])
  • fix(build-wasm): put arrow-ipc/zstd dep under compression feature #16844 (chrisvander)
  • chore(deps): bump serde_json from 1.0.140 to 1.0.141 #16863 (dependabot[bot])
  • chore(deps): bump aws-config from 1.8.1 to 1.8.2 #16864 (dependabot[bot])
  • test: Fix flaky join tests #16860 (2010YOUY01)
  • chore(deps): bump rand from 0.9.1 to 0.9.2 #16882 (dependabot[bot])
  • Report error when SessionState::sql_to_expr_with_alias does not consume all input #16811 (pepijnve)
  • test: fix more flaky join tests #16880 (2010YOUY01)
  • MINOR: add unit tests for chr function #16856 (waynexia)
  • remove deprecated methods from FileScanConfig / DataSourceExec #16901 (adriangb)
  • Support utf8view for spark hex #16885 (xudong963)
  • Fixes 3 bugs during serialization and deserialization of physical plans #16858 (NGA-TRAN)
  • chore(deps): bump aws-config from 1.8.2 to 1.8.3 #16912 (dependabot[bot])
  • Derive UDF equality from PartialEq, Hash #16842 (findepi)
  • Ensure Substrait consumer can handle expressions in VirtualTable #16857 (lorenarosati)
  • Mutable Join Unwind #16883 (berkaysynnada)
  • fix(datafusion-proto): support serializing/deserilizing ArrowFormat tables #16875 (colinmarc)
  • ScalarValue Default + Min + Max #16891 (berkaysynnada)
  • minor: add is_superset() method for Interval's #16895 (berkaysynnada)
  • minor: implement with_new_expressions for AggregateFunctionExpr #16897 (berkaysynnada)
  • minor: Rename add_spm_on_top as add_merge_on_top #16913 (berkaysynnada)
  • Implement Helpers for ScopedTimerGuard and Time Structs #16911 (berkaysynnada)
  • Fix Partial Sort Get Slice Point Between Batches #16881 (berkaysynnada)
  • Fix schema_adapter integration tests not running #16835 (kosiew)
  • Update release process #16929 (xudong963)
  • Fix next_up and next_down behavior for zero float values #16745 (liamzwbao)
  • Add Fetch Property to OutputRequirementExec #16892 (berkaysynnada)
  • chore(deps): bump tokio from 1.46.1 to 1.47.0 #16952 (dependabot[bot])
  • chore(deps): bump serde_json from 1.0.140 to 1.0.141 #16951 (dependabot[bot])
  • chore: Remove attributes to allow dead_code that aren't relevant anymore #16953 (AdamGS)
  • chore(deps): bump rand from 0.9.1 to 0.9.2 #16960 (dependabot[bot])
  • chore(deps): bump ctor from 0.4.2 to 0.4.3 #16961 (dependabot[bot])
  • disallow pushdown of volatile functions #16861 (adriangb)
  • remove warning from every file open #16968 (adriangb)
  • Pin github actions to commit sha #16964 (gopidesupavan)
  • Enable physical filter pushdown for hash joins #16954 (adriangb)
  • Fix [Bug] Aggregate + TopK fails when asc = false #16972 (avantgardnerio)
  • Use tokio::task::coop::poll_proceed by default in CooperativeStream #16748 (pepijnve)
  • Add benchmark utility to profile peak memory usage #16814 (ding-young)
  • chore(deps): bump indicatif from 0.17.11 to 0.18.0 #16992 (dependabot[bot])
  • test(datafusion-cli): migrate tests to insta in print_format.rs #16993 (Thearas)
  • Chore: remove 'spill_record_batch_by_size' api #16958 (ding-young)
  • chore(deps): bump serde_json from 1.0.141 to 1.0.142 #17006 (dependabot[bot])
  • Add tests for yielding in SpillManager::read_spill_as_stream #16616 (ding-young)
  • #16994 Ensure CooperativeExec#maintains_input_order returns a Vec of the correct size #16995 (pepijnve)
  • test: Add logic tests for string_agg with order #17033 (nuno-faria)
  • Implement From<Option<String>>' for ScalarValue` #17043 (findepi)
  • chore(deps): bump tokio-util from 0.7.15 to 0.7.16 #17030 (dependabot[bot])
  • Add missing Substrait to DataFusion function name mappings #16950 (lorenarosati)
  • refactor: use upstream arrow-rs inline_key_fast #17044 (zhuqi-lucas-001)
  • Implement spark array function array #16936 (Standing-Man)
  • Address memory over-accounting in array_agg #16816 (gabotechs)
  • chore(deps): bump aws-credential-types from 1.2.4 to 1.2.5 #17053 (dependabot[bot])
  • Support Substrait functions and_not, xor, and between in consumer built-in expression builder #16984 (lorenarosati)
  • Derive UDWF equality from PartialEq, Hash #17057 (findepi)
  • fix return field for is_not_null expression #17056 (davidhewitt)
  • chore(deps): bump tokio from 1.47.0 to 1.47.1 #17063 (dependabot[bot])
  • Optimize char expression #16076 (ajita-asthana)
  • Fix equality of parametrizable ArrayAgg function #17065 (findepi)
  • Implement Spark url function parse_url #16937 (Standing-Man)
  • Derive UDAF equality from Eq, Hash #17067 (findepi)
  • Remove elements deprecated since v 45 #17075 (findepi)
  • Deprecate ScalarUDF::is_nullable #17074 (findepi)
  • Re-export object_store crate via DataFusion Core and Common #17070 (kosiew)
  • Fix hash/equality issues for ScalarFunctionExpr #17078 (findepi)
  • Fill missing methods in aliased UDF impls #17080 (findepi)
  • Improve Hash speed for ScalarFunctionExpr #17099 (findepi)
  • chore(deps): bump clap from 4.5.42 to 4.5.43 #17079 (dependabot[bot])
  • minor: remove unused import in docstring of datafusion_common::record_batch #17106 (Jefffrey)
  • Make macros in common::test_util hygenic and not dependent on user dependencies #17102 (AdamGS)
  • minor: remove unnecessary clippy:large_enum_variant allows #17108 (Jefffrey)
  • minor: Improve equivalence handling of joins #16893 (berkaysynnada)
  • Fix incorrect NULL IN () optimization #17092 (findepi)
  • Add prettier to the devcontainer (GitHub codespaces) #17019 (alamb)
  • Set a lower threshold for clippy to flag large error variants #17109 (Jefffrey)
  • chore(deps): bump rustyline from 16.0.0 to 17.0.0 #17116 (dependabot[bot])
  • Add dynamic filter (bounds) pushdown to HashJoinExec #16445 (adriangb)
  • Remove the "run extended tests" github PR commend action #17119 (alamb)
  • chore(deps): bump sysinfo from 0.36.1 to 0.37.0 #17124 (dependabot[bot])
  • chore(deps): bump libc from 0.2.174 to 0.2.175 #17121 (dependabot[bot])
  • ff: Preserve cached plan information when pushing projection #17129 (friendlymatthew)
  • chore: Enforce checks for RC branches #17132 (comphead)
  • chore(deps): bump actions/checkout from 4.2.2 to 5.0.0 #17149 (dependabot[bot])
  • minor: enhance comment in SortPreservingMergeStream.abort #17115 (mapleFU)
  • Update workspace to use Rust 1.89 #17100 (shruti2522)
  • chore(deps): bump on-headers and compression in /datafusion/wasmtest/datafusion-wasm-app #16812 (dependabot[bot])
  • chore(deps): bump slab from 0.4.10 to 0.4.11 #17161 (dependabot[bot])
  • refactor character_length impl by unifying null handling logic #16877 (waynexia)
  • chore(deps): bump clap from 4.5.43 to 4.5.44 #17148 (dependabot[bot])
  • Pass the input schema to stats_projection for ProjectionExpr #17123 (hareshkh)
  • Fix extended tests failure on main by updating datafusion-testing pin #17176 (alamb)
  • Minor: display filter in HashJoin's tree explain #17170 (2010YOUY01)
  • add test for multi-column topk dynamic filter pushdown #17162 (adriangb)
  • Test: Add checks to sqllogictest temporary file creations #17017 (2010YOUY01)
  • Deprecate unused ScalarUDF::display_name #17168 (findepi)
  • CI: Fix extended test failure by updating datafusion-testing submodule #17187 (2010YOUY01)
  • Normalize NUL to \0 in sqllogictests #17181 (Jefffrey)
  • Simplify GetFieldFunc's display_name, schema_name #17167 (findepi)
  • Rewrite Nested Loop Join executor for 5× speed and 1% memory usage #16996 (2010YOUY01)
  • Minor: Fix compiler warning when compiling datafusion-cli #17205 (2010YOUY01)
  • Refactor: Do not silently ignore errors in stats_projection #17154 (alamb)
  • Miscellaneous cleanups #17189 (findepi)
  • [Parquet Metadata Cache] Document the ListingTable cache #17133 (alamb)
  • Fix: Show backtrace for ArrowError #17204 (2010YOUY01)
  • minor: clean up distinct window code #17215 (zhuqi-lucas-001)
  • chore: Add drop table test on create_drop.rs #17219 (caicancai)
  • chore(deps): bump async-trait from 0.1.88 to 0.1.89 #17203 (dependabot[bot])
  • Bump MSRV to 1.86.0 #17230 (adriangb)
  • Minor: improve error message when file creation failed #17217 (2010YOUY01)
  • Fix dynamic filter pushdown in HashJoinExec #17201 (adriangb)
  • Fix Analyze Exec protobuf roundtrip #17234 (cetra3)
  • Preserve distinct and ignore_nulls in window expressions during proto serde #17235 (cetra3)
  • chore(deps): bump serde_json from 1.0.142 to 1.0.143 #17240 (dependabot[bot])
  • chore(deps): bump syn from 2.0.105 to 2.0.106 #17243 (dependabot[bot])
  • Push dynamic pushdown through cooperative and projection execs #17238 (jackkleeman)
  • Configure cli test that requires backtrace to be optional #17236 (Jefffrey)
  • chore(deps): Update sqlparser to 0.58 #16456 (Dimchikkk)
  • chore(deps): bump rustyline from 17.0.0 to 17.0.1 #17252 (dependabot[bot])
  • chore(deps): bump thiserror from 2.0.14 to 2.0.16 #17257 (dependabot[bot])
  • Fix HashJoinExec sideways information passing for partitioned queries #17197 (adriangb)
  • Fix HashJoinExec test snapshot under force_hash_collisions=true #17265 (adriangb)
  • Deprecate confusingly named UserDefinedFunctionPlanner #17247 (alamb)
  • Fix: ListingTableFactory paths with dots #17233 (BlakeOrth)
  • chore(deps): bump tempfile from 3.20.0 to 3.21.0 #17268 (dependabot[bot])
  • Fix PartialOrd for ScalarUDF #17182 (findepi)
  • chore(deps): bump url from 2.5.4 to 2.5.6 #17283 (dependabot[bot])
  • Make dynamic filter creation in HashJoinExec deterministic against partition evaluation order #17280 (adriangb)
  • Consolidate Parquet Metadata handling into its own module and struct DFParquetMetadata #17127 (alamb)
  • Only update TopK dynamic filters if the new ones are more selective #16433 (adriangb)
  • Add documentation for UNION schema handling. #17248 (wiedld)
  • Replace π-related bound constants with next_up/next_down #16823 (rthummaluru)
  • chore: add example for how to use TrackConsumersPool #17213 (wiedld)
  • minor: Remove extra line break in explain physical plan #17303 (nuno-faria)
  • Support avg(distinct) for float64 type #17255 (Jefffrey)
  • chore: check the error message log #17308 (caicancai)
  • Expand sql_planner benchmark for benchmarking physical and logical optimization. #17276 (Omega359)
  • Encapsulate early File pruning in parquet opener in its own stream #17293 (alamb)
  • Implement partition_statistics API for RepartitionExec #17061 (liamzwbao)
  • chore: replace Schema with SchemaRef in PruningExpressionBuilder #17216 (etolbakov)
  • chore(deps): bump regex-syntax from 0.8.5 to 0.8.6 #17320 (dependabot[bot])
  • chore(deps): bump indexmap from 2.10.0 to 2.11.0 #17316 (dependabot[bot])
  • refactor: Split SortMergeJoin into multiple modules #17304 (jonathanc-n)
  • MINOR: add missing examples to example list #17333 (waynexia)
  • chore: split hash join to smaller modules #17300 (2010YOUY01)
  • chore(deps): bump url from 2.5.6 to 2.5.7 #17324 (dependabot[bot])
  • chore(deps): bump regex from 1.11.1 to 1.11.2 #17325 (dependabot[bot])
  • add a ci job for typo checking #17339 (waynexia)
  • chore(deps): bump clap from 4.5.45 to 4.5.46 #17338 (dependabot[bot])
  • chore(deps): bump korandoru/hawkeye from 6.1.1 to 6.2.0 #17321 (dependabot[bot])
  • chore: avoid very cheap copy in SchemaMapping #17344 (rluvaton)
  • chore(deps): bump actions/checkout from 4.2.2 to 5.0.0 #17345 (dependabot[bot])
  • chore(deps): bump libmimalloc-sys from 0.1.43 to 0.1.44 #17343 (dependabot[bot])
  • fix EquivalenceProperties calculation in DataSourceExec #17323 (adriangb)
  • chore(deps): bump mimalloc from 0.1.47 to 0.1.48 #17353 (dependabot[bot])
  • chore(deps): bump tracing-subscriber from 0.3.19 to 0.3.20 #17355 (dependabot[bot])
  • refactor: simplify json_shredding example by using ListingTable #17369 (waynexia)
  • Fix incorrect memory accounting for sliced StringViewArray #17315 (ding-young)
  • chore(deps): bump aws-credential-types from 1.2.5 to 1.2.6 #17368 (dependabot[bot])
  • minor: use debug level log for physical optimizer #17383 (waynexia)
  • chore(deps): bump uuid from 1.18.0 to 1.18.1 #17384 (dependabot[bot])
  • chore(deps): bump aws-config from 1.8.5 to 1.8.6 #17386 (dependabot[bot])
  • minor: make dict_from_values public #17376 (parthchandra)
  • chore: add memory catalog test to handle table removal before schema deregistration #17307 (caicancai)
  • chore(deps): bump actions/setup-node from 4.4.0 to 5.0.0 #17410 (dependabot[bot])
  • chore(deps): bump actions/stale from 9.1.0 to 10.0.0 #17409 (dependabot[bot])
  • chore(deps): bump actions/labeler from 5.0.0 to 6.0.0 #17408 (dependabot[bot])
  • Avoid panic when 'with order' expression could not be converted to a logical expression #17394 (pepijnve)
  • chore(deps): bump apache-avro from 0.17.0 to 0.20.0 #16092 (dependabot[bot])
  • chore(deps): bump actions/setup-python from 5.6.0 to 6.0.0 #17413 (dependabot[bot])
  • Test grouping by FixedSizeList #17415 (findepi)
  • re-export physical_expr_adapter #17414 (adriangb)
  • Benchmark window function with multiple partitioning columns #17402 (findepi)
  • Fix PartialOrd for Window #17393 (findepi)
  • Memory datasource protobuf support #17290 (lewiszlw)
  • fix bounds accumulator reset in HashJoinExec dynamic filter pushdown #17371 (adriangb)
  • Unimplement PartialOrd for TDigest's Centroid #17440 (findepi)
  • Unimplement PartialEq, PartialOrd from ToRepartition, RePartition #17441 (findepi)
  • chore(deps): bump insta from 1.43.1 to 1.43.2 #17436 (dependabot[bot])
  • chore(deps): bump actions/labeler from 6.0.0 to 6.0.1 #17433 (dependabot[bot])
  • chore(deps): bump clap from 4.5.46 to 4.5.47 #17435 (dependabot[bot])
  • Add PhysicalExpr::is_volatile #17351 (adriangb)
  • refactor: Use BufferedBatchState enum for SMJ spilling #17429 (jonathanc-n)
  • Re-enable page index for encrypted Parquet #17426 (adamreeve)
  • Re-export apache-avro when avro feature flag is set #17388 (shivbhatia10)
  • Improved experience when remote object store URL does not end in / #17364 (xiedeyantu)

Credits

Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.

    51	dependabot[bot]
    31	Piotr Findeisen
    24	Adrian Garcia Badaracco
    21	Andrew Lamb
    14	Yongting You
    11	Chen Chongchen
     9	Berkay Şahin
     9	Ruihang Xia
     7	Jeffrey Vo
     7	Nuno Faria
     6	Oleks V
     6	Pepijn Van Eeckhoudt
     6	Qi Zhu
     5	ding-young
     4	Adam Reeve
     4	Alan Tang
     4	Jonathan Chen
     4	Liam Bao
     4	Peter Nguyen
     4	Raz Luvaton
     4	kosiew
     3	Adam Gutglick
     3	Cancai Cai
     3	Huaijin
     3	Peter L
     3	lorenarosati
     3	wiedld
     2	Brent Gardner
     2	Bruce Ritchie
     2	Marco Neumann
     2	Matthew Kim
     2	Sherin Jacob
     2	Thearas
     2	Tim Saucer
     2	miro
     2	xudong.w
     2	张林伟
     1	Ajeeta Asthana
     1	Alex Huang
     1	Andrey Koshchiy
     1	Andy Grove
     1	Blake Orth
     1	Christian van der Loo
     1	Colin Marc
     1	Corwin Joy
     1	David Hewitt
     1	David López
     1	Dima
     1	Eugene Tolbakov
     1	Evgenii Glotov
     1	GPK
     1	Gabriel
     1	Geetansh Juneja
     1	Geoffrey Claude
     1	Haresh Khanna
     1	Jack Kleeman
     1	Jax Liu
     1	Jensen
     1	LB7666
     1	Loakesh Indiran
     1	Marko Milenković
     1	Matt Matravers
     1	Nga Tran
     1	Parth Chandra
     1	Ronit Thummaluru
     1	Shehab Amin
     1	Shiv Bhatia
     1	Shruti Sharma
     1	Simon Vandel Sillesen
     1	Stuart Carnie
     1	Tobias Schwarzinger
     1	Yuhan Wang
     1	ZC
     1	Zhen Wang
     1	aditya singh rathore
     1	ayemjay
     1	delamarch3
     1	mwish
     1	theirix

Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.