ContextQMD
Back to Datafusion

Apache DataFusion 52.0.0 Changelog

dev/changelog/52.0.0.md

53.1.076.4 KB
Original Source
<!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->

Apache DataFusion 52.0.0 Changelog

This release consists of 549 commits from 121 contributors. See credits at the end of this changelog for more information.

See the upgrade guide for information on how to upgrade from previous versions.

Breaking changes:

  • Force FileSource to be constructed with a Schema #18386 (adriangb)
  • Support Arrow IPC Stream Files #18457 (corasaurus-hex)
  • Change default of AggregateUDFImpl::supports_null_handling_clause to false #18441 (Jefffrey)
  • [Minor] Remove RawTableAllocExt #18748 (Dandandan)
  • Change CacheAccessor::remove to take &self rather than &mut self #18726 (alchemist51)
  • Move statistics handling into FileScanConfig #18721 (adriangb)
  • chore: remove pyarrow feature #18528 (timsaucer)
  • Limit visibility of internal impl functions in function crates #18877 (Jefffrey)
  • FFI: return underlying trait type when converting from FFI structs #18672 (timsaucer)
  • Refactor crypto functions code #18664 (Jefffrey)
  • move projection handling into FileSource #18627 (adriangb)
  • Add PhysicalOptimizerRule::optimize_plan to allow passing more context into optimizer rules #18739 (adriangb)
  • Optimize planning / stop cloning Strings / Fields so much (2-3% faster planning time) #18415 (alamb)
  • Adds memory-bound DefaultListFilesCache #18855 (BlakeOrth)
  • Allow Logical expression ScalarVariable to represent an extension type or metadata #18243 (batmnnn)
  • feat: Implement the statistics_cache function #19054 (nuno-faria)
  • Move newlines_in_values from FileScanConfig to CsvSource #19313 (adriangb)
  • Remove SchemaAdapter #19345 (adriangb)
  • feat: hash partitioning satisfies subset #19304 (gene-bordegaray)
  • feat: update FFI TableProvider and ExecutionPlan to use FFI Session and TaskContext #19281 (timsaucer)
  • Allow logical optimizer to be run without evaluating now() & refactor SimplifyInfo #19505 (adriangb)
  • Make default ListingFilesCache table scoped #19616 (jizezhang)

Performance related:

  • Normalize partitioned and flat object listing #18146 (BlakeOrth)
  • perf: Improve NLJ for very small right side case #17562 (2010YOUY01)
  • Consolidate EliminateNestedUnion and EliminateOneUnion optimizer rules' #18678 (alamb)
  • perf: improve performance of vectorized_equal_to for PrimitiveGroupValueBuilder in multi group by aggregation #17977 (rluvaton)
  • optimizer: Support dynamic filter in MIN/MAX aggregates #18644 (2010YOUY01)
  • perf: use new_repeated when converting scalar to an array #19018 (rluvaton)
  • perf: optimize CASE WHEN lookup table (2.5-22.5 times faster) #18183 (rluvaton)
  • add specialized InList implementations for common scalar types #18832 (adriangb)
  • Add hashing microbenchmark with_hashes #19373 (alamb)
  • Optimize muti-column grouping with StringView/ByteView (option 2) - 25% faster #19413 (alamb)
  • Optimize hashing for StringView and ByteView (15-70% faster) #19374 (alamb)
  • perf: Improve performance of to_hex (> 2x) #19503 (andygrove)
  • perf: improve performance of string repeat #19502 (andygrove)
  • perf: Optimize starts_with and ends_with for scalar arguments #19516 (andygrove)
  • perf: improve performance of string replace #19530 (viirya)
  • perf: improve performance of levenshtein by reusing cache buffer #19532 (viirya)
  • perf: improve performance of translate by reusing buffers #19533 (viirya)
  • perf: Optimize contains for scalar search arg #19529 (andygrove)
  • perf: improve performance of lpad/rpad by reusing buffers #19558 (viirya)
  • perf: optimize regexp_count to avoid String allocation when start position is provided #19553 (viirya)
  • perf: Improve performance of md5 #19568 (andygrove)
  • perf: optimize strpos by eliminating double iteration for UTF-8 #19572 (viirya)
  • perf: optimize factorial function performance #19575 (getChan)
  • perf: Improve performance of ltrim, rtrim, btrim #19551 (andygrove)
  • perf: optimize HashTableLookupExpr::evaluate #19602 (UBarney)
  • perf: Improve performance of split_part #19570 (andygrove)
  • Optimize Nullstate / accumulators #19625 (Dandandan)

Implemented enhancements:

  • feat: Enhance array_slice functionality to support ListView and LargeListView types #18432 (Weijun-H)
  • feat: support complex expr for prepared statement argument #18383 (chenkovsky)
  • feat: Implement SessionState::create_logical_expr_from_sql_expr #18423 (petern48)
  • feat: added clippy::needless_pass_by_value lint rule to datafusion/expr #18532 (Gohlub)
  • feat: support nested key for get_field #18394 (chenkovsky)
  • feat: Add ansi enable parameter for execution config #18635 (comphead)
  • feat: Add evaluate_to_arrays function #18446 (EmilyMatt)
  • feat: support named variables & defaults for CREATE FUNCTION #18450 (r1b)
  • feat: Add new() constructor for CachedParquetFileReader #18575 (petern48)
  • feat: support decimal for math functions: power #18032 (theirix)
  • feat: selectivity metrics (for Explain Analyze) in Hash Join #18488 (feniljain)
  • feat: Handle edge case with corr with single row and NaN #18677 (comphead)
  • feat: support spark csc #18642 (psvri)
  • feat: support spark sec #18728 (psvri)
  • feat(parquet): Implement scan_efficiency_ratio metric for parquet reading #18577 (petern48)
  • feat: Enhance map handling to support NULL map values #18531 (Weijun-H)
  • feat: add RESET statement for configuration variabless #18408 (Weijun-H)
  • feat: add human-readable formatting to EXPLAIN ANALYZE metrics #18689 #18734 (T2MIX)
  • feat: support Spark-compatible abs math function part 1 - non-ANSI mode #18205 (hsiang-c)
  • feat: Support Show runtime settings #18564 (Weijun-H)
  • feat(small): Support <slt:ignore> marker in sqllogictest for non-deterministic expected parts #18857 (2010YOUY01)
  • feat: allow custom caching via logical node #18688 (jizezhang)
  • feat: add array_slice benchmark #18879 (dqkqd)
  • feat: Support recursive queries with a distinct 'UNION' #18254 (Tpt)
  • feat: Makes error macros hygienic #18995 (Tpt)
  • feat: Add builder API for CreateExternalTable to reduce verbosity #19066 (AryanBagade)
  • feat(spark): Implement Spark functions url_encode, url_decode and try_url_decode #17399 (anhvdq)
  • feat: Move DefaultMetadataCache into its own module #19125 (AryanBagade)
  • feat: Add remove_optimizer_rule to SessionContext #19209 (nuno-faria)
  • feat: integrate batch coalescer with repartition exec #19002 (jizezhang)
  • feat: Preserve File Partitioning From File Scans #19124 (gene-bordegaray)
  • feat: Add constant column extraction and rewriting for projections in ParquetOpener #19136 (Weijun-H)
  • feat: Support sliding window queries for MedianAccumulator by implementing retract_batch #19278 (petern48)
  • feat: add compression level configuration for JSON/CSV writers #18954 (Smotrov)
  • feat(spark): implement Spark try_sum function #18569 (davidlghellin)
  • feat: Support log for Decimal32 and Decimal64 #18999 (Mark1626)
  • feat(proto): Add protobuf serialization for HashExpr #19379 (adriangb)
  • feat: Add decimal support for round #19384 (kumarUjjawal)
  • Support nested field access in get_field with multiple path arguments #19389 (adriangb)
  • feat: fix matching for named parameters with non-lowercase signatures #19378 (bubulalabu)
  • feat: Add per-expression evaluation timing metrics to ProjectionExec #19447 (2010YOUY01)
  • feat: Improve sort memory resilience #19494 (EmilyMatt)
  • feat: Add DELETE/UPDATE hooks to TableProvider trait and to MemTable implementation #19142 (ethan-tyler)
  • feat: implement partition_statistics for WindowAggExec #18534 (0xPoe)
  • feat: integrate batch coalescer with async fn exec #19342 (feniljain)
  • feat: output statistics for constant columns in projections #19419 (shashidhar-bm)
  • feat: to_time function #19540 (kumarUjjawal)
  • feat: Implement Spark functions hour, minute, second #19512 (andygrove)
  • feat: plan-time SQL expression simplifying #19311 (theirix)
  • feat: Implement Spark function space #19610 (kazantsev-maksim)
  • feat: Implement partition_statistics API for SortMergeJoinExec #19567 (kumarUjjawal)
  • feat: add list_files_cache table function for datafusion-cli #19388 (jizezhang)
  • feat: implement metrics for AsyncFuncExec #19626 (feniljain)
  • feat: split BatchPartitioner::try_new into hash and round-robin constructors #19668 (mohit7705)
  • feat: add Time type support to date_trunc function #19640 (kumarUjjawal)
  • feat: Allow log with non-integer base on decimals #19372 (Yuvraj-cyborg)

Fixed bugs:

  • fix: Eliminate consecutive repartitions #18521 (gene-bordegaray)
  • fix: with_param_values on LogicalPlan::EmptyRelation returns incorrect schema #18286 (dqkqd)
  • fix: Nested arrays should not get a field in lookup #18745 (EmilyMatt)
  • fix: update schema's data type for LogicalPlan::Values after placeholder substitution #18740 (dqkqd)
  • fix: Pick correct columns in Sort Merge Equijoin #18772 (tglanz)
  • fix: remove WorkTableExec special case in reset_plan_states #18803 (geoffreyclaude)
  • fix: display the failed sqllogictest file and query that failed in case of a panic #18785 (rluvaton)
  • fix: preserve byte-size statistics in AggregateExec #18885 (Tamar-Posen)
  • fix: Track elapsed_compute metric for CSV scans #18901 (Nithurshen)
  • fix: Implement Substrait consumer support for like_match, like_imatch, and negated variants #18929 (Nithurshen)
  • fix: Initialize CsvOptions::double_quote from proto_opts.double_quote #18967 (martin-g)
  • fix: rstest is a DEV dependency #19014 (crepererum)
  • fix: partition pruning stats pruning when multiple values are present #18923 (Mark1626)
  • fix: deprecate data_type_and_nullable and simplify API usage #18869 (BipulLamsal)
  • fix: pre-warm listing file statistics cache during listing table creation #18971 (bharath-techie)
  • fix: log metadata differences when comparing physical and logical schema #19070 (erratic-pattern)
  • fix: fix panic when lo is greater than hi #19099 (tshauck)
  • fix: escape underscores when simplifying starts_with #19077 (willemv)
  • fix: custom nullability for length (#19175) #19182 (skushagra)
  • fix: inverted null_percent logic in in_list benchmark #19204 (geoffreyclaude)
  • fix: Ensure column names do not change with expand_views_at_output #19019 (nuno-faria)
  • fix: bitmap_count should report nullability correctly #19195 (harshitsaini17)
  • fix: bit_count function to report nullability correctly #19197 (harshitsaini17)
  • fix: derive custom nullability for spark bit_shift #19222 (kumarUjjawal)
  • fix: spark elt custom nullability #19207 (EeshanBembi)
  • fix: array_remove/array_remove_n/array_remove_all not using the same nullability as the input #19259 (rluvaton)
  • fix: typo in sql/ddl #19276 (mag1c1an1)
  • fix: flaky cache test #19140 (xonx4l)
  • fix: Add custom nullability for Spark ILIKE function #19206 (Eshaan-byte)
  • fix: derive custom nullability for spark map_from_arrays #19275 (kumarUjjawal)
  • fix: derive custom nullability for spark map_from_entries #19274 (kumarUjjawal)
  • fix: derive custom nullable for spark make_dt_interval #19236 (kumarUjjawal)
  • fix: derive custome nullable for the spark last_day #19232 (kumarUjjawal)
  • fix: derive custom nullable for spark date_sub #19225 (kumarUjjawal)
  • fix: Fix a few minor issues with join metrics #19283 (linhr)
  • fix: derive nullability for spark bit_get #19220 (kumarUjjawal)
  • fix: pow() with integer base and negative float exponent returns error #19303 (adriangb)
  • fix(concat): correct nullability inference (nullable only if all arguments nullable) #19189 (ujjwaltwri)
  • fix: Added nullable return from date_add(#19151) #19229 (manishkr)
  • fix: spark sha1 nullability reporting #19242 (shashidhar-bm)
  • fix: derive custom nullability for the spark next_day #19253 (kumarUjjawal)
  • fix: preserve ListFilesCache TTL when not set in config #19401 (shashidhar-bm)
  • fix: projection for CooperativeExec and CoalesceBatchesExec #19400 (haohuaijin)
  • fix: spark crc32 custom nullability #19271 (watanaberin)
  • fix: Fix skip aggregate test to cover regression #19461 (kumarUjjawal)
  • fix: [19450]Added flush for tokio file(substrait) write #19456 (manishkr)
  • fix: csv schema_infer_max_records set to 0 return null datatype #19432 (haohuaijin)
  • fix: Add custom nullability for Spark LIKE function #19218 (KaranPradhan266)
  • fix: implement custom nullability for spark abs function #19395 (batmnnn)
  • fix: custom nullability for format_string (#19173) #19190 (skushagra)
  • fix: Implement reset_state for LazyMemoryExec #19362 (nuno-faria)
  • fix: CteWorkTable: properly apply TableProvider::scan projection argument #18993 (Tpt)
  • fix: Median() integer overflow #19509 (kumarUjjawal)
  • fix: Reverse row selection should respect the row group index #19557 (zhuqi-lucas)
  • fix: emit empty RecordBatch for empty file writes #19370 (nlimpid)
  • fix: handle invalid byte ranges in calculate_range for single-line files #19607 (vigimite)
  • fix: NULL handling in arrow_intersect and arrow_union #19415 (feniljain)
  • fix(doc): close #19393, make upgrading guide match v51 api #19648 (mag1c1an1)
  • fix(spark): Use wrapping addition/subtraction in SparkDateAdd and SparkDateSub #19377 (mzabaluev)
  • fix(functions): Make translate function postgres compatible #19630 (devanshu0987)
  • fix: Return Int for Date - Date instead of duration #19563 (kumarUjjawal)
  • fix: DynamicFilterPhysicalExpr violates Hash/Eq contract #19659 (kumarUjjawal)

Documentation updates:

  • [main] Update version to 51.0.0, add Changelog (#18551) #18565 (alamb)
  • refactor: include metric output_batches into BaselineMetrics #18491 (nmbr7)
  • chore(deps): bump maturin from 1.9.6 to 1.10.0 in /docs #18590 (dependabot[bot])
  • Update release download links on download page #18550 (alamb)
  • docs: fix rustup cmd for adding rust-analyzer #18605 (Jefffrey)
  • Enforce explicit opt-in for WITHIN GROUP syntax in aggregate UDAFs #18607 (kosiew)
  • docs: fix broken catalog example links #18765 (nlimpid)
  • doc: Add documentation for error handling #18762 (2010YOUY01)
  • docs: Fix the examples for char_length() and character_length() #18808 (martin-g)
  • chore: Support 'untake' for unassigning github issues #18637 (petern48)
  • chore: Add filtered pending PRs link to main page #18854 (comphead)
  • Docs: Enhance contributor guide with testing section #18852 (alamb)
  • Docs: Enhance testing documentation with examples and links #18851 (alamb)
  • chore(deps): bump maturin from 1.10.0 to 1.10.2 in /docs #18905 (dependabot[bot])
  • Update links in documentation to point at new example locations #18931 (alamb)
  • Add Kubeflow Trainer to known users #18935 (andreyvelich)
  • Add PGO documentation section to crate configuration #18959 (jatinkumarsingh)
  • Add upgrade guide for PhysicalOptimizerRule::optimize_plan #19030 (adriangb)
  • doc: add FilterExec metrics to user-guide/metrics.md #19043 (2010YOUY01)
  • Add force_filter_selections to restore pushdown_filters behavior prior to parquet 57.1.0 upgrade #19003 (alamb)
  • Implement FFI task context and task context provider #18918 (timsaucer)
  • Minor: fix link errors in docs #19088 (alamb)
  • Cut Parquet over to PhysicalExprAdapter, remove SchemaAdapter #18998 (adriangb)
  • Update Committer / PMC list #19105 (alamb)
  • Revert adding PhysicalOptimizerRule::optimize_plan #19186 (adriangb)
  • Push down InList or hash table references from HashJoinExec depending on the size of the build side #18393 (adriangb)
  • Move partition handling out of PhysicalExprAdapter #19128 (adriangb)
  • Push down projection expressions into ParquetOpener #19111 (adriangb)
  • Track column sizes in Statistics; propagate through projections #19113 (adriangb)
  • Improve ProjectionExpr documentation and comments #19263 (alamb)
  • Update README occording to the new examples (#18529) #19257 (cj-zhukov)
  • Add make_time function #19183 (Omega359)
  • Update to_date udf function to support a consistent set of argument types #19134 (Omega359)
  • Add library user guide for extending SQL syntax #19265 (geoffreyclaude)
  • Add runtime config options for list_files_cache_limit and list_files_cache_ttl #19108 (delamarch3)
  • Minor: clean up titles and links n extending operators and optimizer pages #19317 (alamb)
  • Establish the high level API for sort pushdown and the optimizer rule and support reverse files and row groups #19064 (zhuqi-lucas)
  • Add Decimal support to Ceil and Floor #18979 (kumarUjjawal)
  • doc: add example for cache factory #19139 (jizezhang)
  • chore(deps): bump sphinx-reredirects from 1.0.0 to 1.1.0 in /docs #19455 (dependabot[bot])
  • Add:arrow_metadata() UDF #19435 (xonx4l)
  • Update date_bin to support Time32 and Time64 data types #19341 (Omega359)
  • Update to_unixtime udf function to support a consistent set of argument types #19442 (kumarUjjawal)
  • docs: Improve config tables' readability #19522 (nuno-faria)
  • Introduce TypeSignatureClass::Any #19485 (Jefffrey)
  • Enables DefaultListFilesCache by default #19366 (BlakeOrth)
  • Fix typo in contributor guide architecture section #19613 (cdegroc)
  • docs: fix typos in PartitionEvaluator trait documentation #19631 (SolariSystems)
  • Respect execution timezone in to_timestamp and related functions #19078 (Omega359)
  • perfect hash join #19411 (UBarney)

Other:

  • chore(deps): bump taiki-e/install-action from 2.62.46 to 2.62.47 #18508 (dependabot[bot])
  • Consolidate builtin functions examples (#18142) #18523 (cj-zhukov)
  • refactor: update cmp and nested data in binary operator #18256 (sunng87)
  • Fix: topk_aggregate benchmark failing #18502 (randyli)
  • refactor: Add assert_or_internal_err! macro for more ergonomic internal invariant checks #18511 (2010YOUY01)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-physical-optimizer #18555 (foskey51)
  • chore: enforce clippy lint needless_pass_by_value for datafusion-sql #18554 (foskey51)
  • chore: enforce clippy lint needless_pass_by_value to physical-expr-common #18556 (foskey51)
  • chore: Enforce lint rule clippy::needless_pass_by_value to datafusion-physical-expr #18557 (corasaurus-hex)
  • Fix out-of-bounds access in SLT runner #18562 (theirix)
  • Make array_reverse faster for List and FixedSizeList #18500 (vegarsti)
  • Consolidate custom data source examples (#18142) #18553 (cj-zhukov)
  • chore(deps): bump taiki-e/install-action from 2.62.47 to 2.62.49 #18581 (dependabot[bot])
  • chore: Remove unused tokio dependency and clippy #18598 (comphead)
  • minor: enforce clippy::needless_pass_by_value for crates that don't require code changes. #18586 (2010YOUY01)
  • refactor: merge CoalesceAsyncExecInput into CoalesceBatches #18540 (Tim-53)
  • Enhance the help message for invalid command in datafusion-cli #18603 (klion26)
  • Update Release README.md with latest process #18549 (alamb)
  • Add timezone to date_trunc fast path #18596 (hareshkh)
  • Coalesce batches inside FilterExec #18604 (Dandandan)
  • Fix misleading boolean 'null' interval tests #18620 (pepijnve)
  • Clarify tests for Interval::and, Interval::not, and add Interval::or tests #18621 (pepijnve)
  • bugfix: correct regression on TableType for into_view #18617 (timsaucer)
  • Separating Benchmarks for physical sorted union over large columns in SQL planner based on Datatype #18599 (logan-keede)
  • Add RunEndEncoded type coercion #18561 (vegarsti)
  • minor: refactor with assert_or_internal_err!() in datafusion/sql #18614 (2010YOUY01)
  • chore: ASF tracking process on .asf.yaml #18636 (comphead)
  • Refactor bit aggregate functions signature #18593 (Jefffrey)
  • chore(deps): bump taiki-e/install-action from 2.62.49 to 2.62.50 #18645 (dependabot[bot])
  • bugfix: select_columns should validate column names #18623 (timsaucer)
  • Consolidate data io examples (#18142) #18591 (cj-zhukov)
  • Correct implementations of NullableInterval::and and NullableInterval::or. #18625 (pepijnve)
  • chore: ASF tracking process on .asf.yaml #18652 (comphead)
  • Refactor Spark bitshift signature #18649 (Jefffrey)
  • chore(deps): bump crate-ci/typos from 1.39.0 to 1.39.1 #18667 (dependabot[bot])
  • Update docs for aggregate repartition test #18650 (xanderbailey)
  • chore: Enforce lint rule clippy::needless_pass_by_value to datafusion-catalog #18638 (Standing-Man)
  • [main] Update Changelog (#18592) #18616 (alamb)
  • Refactor distinct aggregate implementations to use common buffer #18348 (Jefffrey)
  • chore: enforce lint rule clippy::needless_pass_by_value to datafusion-datasource-avro #18641 (Standing-Man)
  • Refactor Spark expm1 signature #18655 (Jefffrey)
  • chore(core): Enforce lint rule clippy::needless_pass_by_value to datafusion-core #18640 (Standing-Man)
  • Refactor substr signature #18653 (Jefffrey)
  • minor: Use allow->expect to explicitly suppress Clippy lint checks #18686 (2010YOUY01)
  • chore(deps): bump taiki-e/install-action from 2.62.50 to 2.62.51 #18693 (dependabot[bot])
  • chore(deps): bump crate-ci/typos from 1.39.1 to 1.39.2 #18694 (dependabot[bot])
  • Remove FilterExec from CoalesceBatches optimization rule, add fetch support #18630 (Dandandan)
  • minor: refactor with assert_or_internal_err!() in datafusion/datasource #18697 (kumarUjjawal)
  • chore: Enforce lint rule clippy::needless_pass_by_value to datafusion-datasource #18682 (AryanBagade)
  • [main] Update changelog for 51.0.0 RC2 #18710 (alamb)
  • Refactor Spark crc32/sha1 signatures #18662 (Jefffrey)
  • CI: try free up space in Rust / cargo test (amd64) action #18709 (Jefffrey)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-proto #18715 (foskey51)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-spark #18714 (foskey51)
  • minor: refactor with assert_or_internal_err!() in datafusion/optimizer #18699 (kumarUjjawal)
  • minor: refactor with assert_or_internal_err!() in datafusion/functions #18700 (kumarUjjawal)
  • minor: refactor with assert_or_internal_err!() in datafusion/expr-common #18702 (kumarUjjawal)
  • minor: refactor with assert_or_internal_err!() in datafusion/functions-aggregate #18716 (kumarUjjawal)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-execution #18723 (foskey51)
  • minor: refactor with assert_or_internal_err!() in datafusion/functions-nested #18724 (kumarUjjawal)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-substrait #18703 (foskey51)
  • chore: Refactor with assert_or_internal_err!() in datafusion/spark. #18674 (codetyri0n)
  • Minor: Add docs to release/README.md about rate limits #18704 (alamb)
  • Consolidate query planning examples (#18142) #18690 (cj-zhukov)
  • minor: refactor with assert_or_internal_err!() in datafusion/physical-expr-common #18735 (kumarUjjawal)
  • minor: refactor with assert_or_internal_err!() in datafusion/physical-expr #18736 (kumarUjjawal)
  • Consolidate ArrowFileSource and ArrowStreamFileSource #18720 (adriangb)
  • minor: refactor with assert_or_internal_err!() in datafusion/physical-optimizer #18732 (kumarUjjawal)
  • refactor: reduce duplication in make_udf_function macro #18733 (shashidhar-bm)
  • minor: refactor with assert_or_internal_err!() in datafusion/physical-plan #18730 (kumarUjjawal)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-functions-aggregate-common #18741 (foskey51)
  • Optimize NullState::build #18737 (Dandandan)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-datasource-parquet #18695 (foskey51)
  • minor: refactor with assert_or_internal_err!() in datafusion/expr #18731 (kumarUjjawal)
  • minor: Fix an example in the PruningPredicate documentation #18742 (2010YOUY01)
  • chore(deps): bump indicatif from 0.18.2 to 0.18.3 #18756 (dependabot[bot])
  • Fix map_query_sql benchmark duplicate key error #18427 (atheendre130505)
  • minor: enforce lint rule clippy::needless_pass_by_value to datafusion-ffi #18764 (Standing-Man)
  • Rename boolean Interval constants to match NullableInterval #18654 (pepijnve)
  • chore(deps): bump bytes from 1.10.1 to 1.11.0 #18755 (dependabot[bot])
  • CI: Fix main branch CI test failure #18792 (2010YOUY01)
  • chore: Enforce 'clippy::needless_pass_by_value' to datafusion-expr-common #18775 (petern48)
  • chore: Finish refactor with assert_or_internal_err!() #18790 (2010YOUY01)
  • Switch from xz2 to liblzma to reduce duplicate dependencies #17509 (timsaucer)
  • chore(deps): bump taiki-e/install-action from 2.62.51 to 2.62.53 #18796 (dependabot[bot])
  • chore(deps): bump actions/checkout from 5.0.0 to 5.0.1 #18797 (dependabot[bot])
  • Misc improvements to ProjectionExprs #18719 (adriangb)
  • Fix incorrect link for sql_query.rs example in README #18807 (kondamudikarthik)
  • Adds prefix filtering for table URLs #18780 (BlakeOrth)
  • Refactor InListExpr to support structs by re-using existing hashing infrastructure #18449 (adriangb)
  • chore: Add script to protect RC branches during the release #18660 (comphead)
  • Prevent overflow and panics when casting DATE to TIMESTAMP by validating bounds #18761 (kosiew)
  • chore(deps): bump taiki-e/install-action from 2.62.53 to 2.62.54 #18815 (dependabot[bot])
  • CI : Enforce clippy: :needless_pass_by_value rule to datafusion-functions-aggregate #18805 (codetyri0n)
  • Consolidate sql operations examples (#18142) #18743 (cj-zhukov)
  • Move GuaranteeRewriter to datafusion_expr #18821 (pepijnve)
  • Refactor state management in HashJoinExec and use CASE expressions for more precise filters #18451 (adriangb)
  • Refactor avg & sum signatures away from user defined #18769 (Jefffrey)
  • Hash UnionArrays #18718 (friendlymatthew)
  • CI: add clippy::needless_pass_by_value rule to datafusion-functions-window crate #18838 (codetyri0n)
  • Add field to DynamicPhysicalExpr to indicate when the filter is complete or updated #18799 (LiaCastaneda)
  • #17801 Improve nullability reporting of case expressions #17813 (pepijnve)
  • Consolidate execution monitoring examples (#18142) #18846 (cj-zhukov)
  • Implement CatalogProviderList in FFI #18657 (timsaucer)
  • Removed incorrect union check in enforce_sorting and updated tests #18661 (gene-bordegaray)
  • chore(deps): bump actions/checkout from 5.0.1 to 6.0.0 #18865 (dependabot[bot])
  • Remove unnecessary bit counting code from spark bit_count #18841 (pepijnve)
  • Fix async_udf batch size behaviour #18819 (shivbhatia10)
  • Fix Partial AggregateExec correctness issue dropping rows #18712 (xanderbailey)
  • chore: Add missing boolean tests to bit_count Spark function #18871 (comphead)
  • Consolidate proto examples (#18142) #18861 (cj-zhukov)
  • Use logical null count in case_when_with_expr #18872 (pepijnve)
  • chore: enforce clippy::needless_pass_by_value to datafusion-physical-plan #18864 (2010YOUY01)
  • Refactor spark bit_get() signature away from user defined #18836 (Jefffrey)
  • minor: enforce lint rule clippy::needless_pass_by_value to datafusion-functions #18768 (Standing-Man)
  • chore: enforce clippy lint needless_pass_by_value to datafusion-functions-nested #18839 (foskey51)
  • chore: fix CI on main #18876 (Jefffrey)
  • chore: update Repartition DisplayAs to indicate maintained sort order #18673 (ruchirK)
  • implement sum for durations #18853 (logan-keede)
  • Consolidate dataframe examples (#18142) #18862 (cj-zhukov)
  • Avoid the need to rewrite expressions when evaluating logical case nullability #18849 (pepijnve)
  • Avoid skew in Roundrobin repartition #18880 (Dandandan)
  • Add benchmark for array_has/array_has_all/array_has_any #18729 (zhuqi-lucas)
  • chore(deps): bump taiki-e/install-action from 2.62.54 to 2.62.56 #18899 (dependabot[bot])
  • chore(deps): bump indicatif from 0.18.0 to 0.18.3 #18897 (dependabot[bot])
  • chore(deps): bump tokio-util from 0.7.16 to 0.7.17 #18898 (dependabot[bot])
  • Support Non-Literal Expressions in Substrait VirtualTable Values and Improve Round-Trip Robustness #18866 (kosiew)
  • chore(deps): bump indexmap from 2.12.0 to 2.12.1 #18895 (dependabot[bot])
  • chore(deps): bump aws-config from 1.8.7 to 1.8.11 #18896 (dependabot[bot])
  • chore(deps): bump flate2 from 1.1.4 to 1.1.5 #18900 (dependabot[bot])
  • Add iter() method to Extensions #18887 (gabotechs)
  • chore: Enforce clippy::needless_pass_by_value globally across the workspace #18904 (2010YOUY01)
  • Consolidate external dependency examples (#18142) #18747 (cj-zhukov)
  • Optimize planning for projected nested union #18713 (logan-keede)
  • chore(deps): bump taiki-e/install-action from 2.62.56 to 2.62.57 #18927 (dependabot[bot])
  • chore(deps): bump actions/setup-python from 6.0.0 to 6.1.0 #18925 (dependabot[bot])
  • Fix map function alias handling in SQL planner #18914 (friendlymatthew)
  • minor: add builder setting NdJsonReadOptions::schema_infer_max_records #18920 (Jefffrey)
  • Implement Substrait Support for GROUPING SET CUBE #18798 (kosiew)
  • chore: unify common dependencies as workspace dependencies #18665 (Jefffrey)
  • Fix bug where binary types were incorrectly being casted for coercible signatures #18750 (Jefffrey)
  • Refactor approx_median signature & support f16 #18647 (Jefffrey)
  • Refactor to_local_time() signature away from user_defined #18707 (Jefffrey)
  • chore(deps-dev): bump node-forge from 1.3.1 to 1.3.2 in /datafusion/wasmtest/datafusion-wasm-app #18958 (dependabot[bot])
  • Support LikeMatch, ILikeMatch, NotLikeMatch, NotILikeMatch operators in protobuf serialization #18961 (zhuqi-lucas)
  • chore: cargo fmt to fix CI #18969 (Jefffrey)
  • chore(deps): bump Swatinem/rust-cache from 2.8.1 to 2.8.2 #18963 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.62.57 to 2.62.58 #18964 (dependabot[bot])
  • chore(deps): bump crate-ci/typos from 1.39.2 to 1.40.0 #18965 (dependabot[bot])
  • [Minor] Refactor traverse_chain macro to function #18951 (Dandandan)
  • Enable clippy::allow_attributes lint for datafusion-catalog #18973 (chakkk309)
  • chore: update group of crates to rust 2024 edition #18915 (timsaucer)
  • chore(deps): bump taiki-e/install-action from 2.62.58 to 2.62.59 #18978 (dependabot[bot])
  • Simplify percentile_cont for 0/1 percentiles #18837 (kumarUjjawal)
  • chore: enforce clippy::allow_attributes for functions-* crates #18986 (carlosahs)
  • chore: enforce clippy::allow_attributes for common crates #18988 (chakkk309)
  • Fix predicate_rows_pruned & predicate_rows_matched metrics #18980 (xudong963)
  • Allocate a buffer of the correct length for ScalarValue::FixedSizeBinary in ScalarValue::to_array_of_size #18903 (tobixdev)
  • Fix error planning aggregates with duplicated names in select list #18831 (tshauck)
  • chore: remove denys of needless_pass_by_value in lib.rs files #18996 (Jefffrey)
  • Add Explicit Error Handling for Unsupported SQL FETCH Clause in Planner and CLI #18691 (kosiew)
  • chore(deps): bump criterion from 0.7.0 to 0.8.0 #19009 (dependabot[bot])
  • chore(deps): bump syn from 2.0.108 to 2.0.111 #19011 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.62.59 to 2.62.60 #19012 (dependabot[bot])
  • chore: remove redundant clone code #18997 (Smith-Cruise)
  • Update to arrow, parquet to 57.1.0 #18820 (alamb)
  • deny on allow_attributes lint in physical-plan #18983 (YuraLitvinov)
  • Add additional test coverage of multi-value PartitionPruningStats #19021 (alamb)
  • Fix tpch benchmark harness #19033 (alamb)
  • Fix data for tpch_csv and tpch_csv10 #19034 (alamb)
  • chore: update group of 3 crates to rust 2024 edition #19001 (timsaucer)
  • chore(deps-dev): bump express from 4.21.2 to 4.22.1 in /datafusion/wasmtest/datafusion-wasm-app #19040 (dependabot[bot])
  • Allow repartitioning on files with ranges #18948 (Samyak2)
  • Support simplify not for physical expr #18970 (xudong963)
  • dev: Add typos check to the local dev/rust_lint.sh #17863 (2010YOUY01)
  • Implement FFI_PhysicalExpr and the structs it needs to support it. #18916 (timsaucer)
  • chore(deps): bump actions/setup-node from 6.0.0 to 6.1.0 #19063 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.62.60 to 2.62.61 #19062 (dependabot[bot])
  • chore(deps): bump actions/stale from 10.1.0 to 10.1.1 #19061 (dependabot[bot])
  • chore: merge make_array and spark array #19006 (jizezhang)
  • chore(deps): bump actions/checkout from 6.0.0 to 6.0.1 #19060 (dependabot[bot])
  • Add documentation example for PartitionPruningStatistics #19020 (alamb)
  • chore: upgrade expr and execution crates to rust 2024 edition #19047 (timsaucer)
  • refactor: Refactor spark make_interval signature away from user defined #19027 (kumarUjjawal)
  • Fix: Align sort_merge_join filter output with join schema to fix right-anti panic #18800 (kumarUjjawal)
  • Support Substrait Round-Trip of EmptyRelation Including produce_one_row Semantics #18842 (kosiew)
  • chore(deps): bump taiki-e/install-action from 2.62.61 to 2.62.62 #19081 (dependabot[bot])
  • chore: enforce clippy::allow_attributes for datasource crates #19068 (chakkk309)
  • common: Add hashing support for REE arrays #18981 (brancz)
  • Use tpchgen-cli to generate tpch data in bench.sh #19035 (alamb)
  • Update aggregate probe to be locked only if skipping aggregation #18766 (hareshkh)
  • Fix function doc CI check #19093 (alamb)
  • Fix Schema Duplication Errors in Self‑Referential INTERSECT/EXCEPT by Requalifying Input Sides #18814 (kosiew)
  • run cargo fmt to fix after #18998 #19102 (adriangb)
  • bench: set test_util as required feature for aggregate_vectorized #19101 (rluvaton)
  • use ProjectionExprs:project_statistics in FileScanConfig #19094 (adriangb)
  • Temporarily ignore test_cache_with_ttl_and_lru test #19115 (alamb)
  • refactor: move human readable display utilities to datafusion-common crate #19080 (2010YOUY01)
  • Always remove unecessary software from github runners for all jobs (fix intermittent out of space on runners) #19122 (alamb)
  • [datafusion-spark]: Refactor make_dt_interval's signature away from user defined #19083 (codetyri0n)
  • fix deprecation notes with incorrect versions from #13083 #19135 (adriangb)
  • Run the examples in the new format #18946 (cj-zhukov)
  • Add constant expression evaluator to physical expression simplifier #19130 (adriangb)
  • Fix shuffle function to report nullability correctly #19184 (harshitsaini17)
  • chore: enforce clippy::allow_attributes for physical crates #19185 (carlosahs)
  • Update 5 crates to rust 2024 edition #19091 (timsaucer)
  • Coalesce batches inside hash join, reuse indices buffer #18972 (Dandandan)
  • slt test coverage for CASE exprs with constant value lookup tables #19143 (alamb)
  • Fix fmt after logical conflict #19208 (alamb)
  • chore: Add TPCDS benchmarks #19138 (comphead)
  • Arc partition values in TableSchema #19137 (adriangb)
  • Add sorted data benchmark. #19042 (zhuqi-lucas)
  • Refactor PhysicalExprSimplfier to &self instead of &mut self #19212 (adriangb)
  • chore(deps): bump uuid from 1.18.1 to 1.19.0 #19199 (dependabot[bot])
  • chore(deps): bump async-compression from 0.4.34 to 0.4.35 #19201 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.62.62 to 2.62.63 #19198 (dependabot[bot])
  • chore(deps): bump tracing-subscriber from 0.3.20 to 0.3.22 #19200 (dependabot[bot])
  • chore(deps): bump wasm-bindgen-test from 0.3.55 to 0.3.56 #19202 (dependabot[bot])
  • bench: add dedicated Utf8View benchmarks for InList #19211 (geoffreyclaude)
  • Fix PruningPredicate interaction with DynamicFilterPhysicalExpr that references partition columns #19129 (adriangb)
  • Implement physical and logical codecs in FFI #19079 (timsaucer)
  • refactor: Refactor spark width bucket signature away from user defined #19065 (kumarUjjawal)
  • Sort Merge Join: Reduce batch concatenation, use BatchCoalescer, new benchmarks (TPC-H Q21 SMJ up to ~4000x faster) #18875 (mbutrovich)
  • Add relation planner extension support to customize SQL planning #17843 (geoffreyclaude)
  • Add additional tests for InListExpr #19050 (adriangb)
  • chore(deps): bump taiki-e/install-action from 2.62.63 to 2.62.64 #19226 (dependabot[bot])
  • Use strum in the examples (#19126) #19205 (cj-zhukov)
  • [Proto]: Serialization support for AsyncFuncExec #19118 (mach-kernel)
  • chore: add test case for decimal overflow #19255 (Jefffrey)
  • chore(deps): bump taiki-e/install-action from 2.62.64 to 2.62.65 #19251 (dependabot[bot])
  • chore: update 6 crates to rust edition 2024 #19196 (timsaucer)
  • Implement FFI_Session #19223 (timsaucer)
  • Feat: Add an option for fast tests by gating slow tests to extended_tests feature #19237 (Yuvraj-cyborg)
  • chore: enforce clippy::allow_attributes for 7 crates #19133 (chakkk309)
  • dev: Add CI doc prettier check to local rust_lint.sh #19254 (2010YOUY01)
  • bug: Eliminate dead round-robin insertion in enforce distribution #19132 (gene-bordegaray)
  • Automatically download tpcds benchmark data to the right place #19244 (alamb)
  • [datafusion-spark]: Refactor hex's signature away from user_defined #19235 (codetyri0n)
  • fix : correct nullability propagation for spark.bitwise_not #19224 (shifluxxc)
  • added custom nullability for char #19268 (skushagra)
  • replace HashTableLookupExpr with lit(true) in proto serialization #19300 (adriangb)
  • chore: fix return_field_from_args doc #19307 (xumingming)
  • chore: enforce clippy::allow_attributes for spark,sql,sustrait #19309 (kumarUjjawal)
  • Simplify make_date & fix null handling #19296 (Jefffrey)
  • Allow base64 encoding of fixedsizebinary arrays #18950 (maxburke)
  • chore: update 11 crates to Rust 2024 edition #19258 (timsaucer)
  • Minor: remove unnecessary unit tests for fixed size binary #19318 (alamb)
  • Populate partition column statistics for PartitionedFile #19284 (adriangb)
  • refactor: move metrics module to datafusion-common crate #19247 (2010YOUY01)
  • chore(deps): bump taiki-e/install-action from 2.62.65 to 2.62.67 #19295 (dependabot[bot])
  • chore(deps): bump ctor from 0.6.1 to 0.6.3 #19328 (dependabot[bot])
  • Refactor power() signature away from user defined #18968 (Jefffrey)
  • chore: enforce clippy::allow_attributes for optimizer and macros #19310 (kumarUjjawal)
  • chore(deps): bump taiki-e/install-action from 2.62.67 to 2.63.3 #19349 (dependabot[bot])
  • chore(deps): bump clap from 4.5.50 to 4.5.53 #19326 (dependabot[bot])
  • chore(deps): bump insta from 1.43.2 to 1.44.3 #19327 (dependabot[bot])
  • remove repartition exec from coalesce batches optimizer #19239 (jizezhang)
  • minor: cleanup unnecessary config in decimal.slt #19352 (Jefffrey)
  • Fix panic for GROUPING SETS(()) and handle empty-grouping aggregates #19252 (kosiew)
  • Update datafusion-core crate to Rust 2024 edition #19332 (timsaucer)
  • Update 4 crates to rust 2024 edition #19357 (timsaucer)
  • preserve Field metadata in first_value/last_value #19335 (adriangb)
  • Fix flaky SpillPool channel test by synchronizing reader and writer tasks #19110 (kosiew)
  • [minor] Upgrade rust version #19363 (Dandandan)
  • Minor: fix cargo fmt #19368 (zhuqi-lucas)
  • chore: enforce clippy::allow_attributes for proto, pruning, session #19350 (kumarUjjawal)
  • Update remaining crates to rust 2024 edition #19361 (timsaucer)
  • Minor: Make ProjectionExpr::new easier to use with constants #19343 (alamb)
  • Feat: DefaultListFilesCache prefix-aware for partition pruning optimization #19298 (Yuvraj-cyborg)
  • Extend in_list benchmark coverage #19376 (geoffreyclaude)
  • [datafusion-cli] Implement average LIST duration for object store profiling #19127 (peterxcli)
  • chore(deps): bump taiki-e/install-action from 2.63.3 to 2.64.0 #19382 (dependabot[bot])
  • update insta snapshots #19381 (kosiew)
  • Fix regression for negative-scale decimal128 in log #19315 (shifluxxc)
  • Fix input handling for encoding functions & various refactors #18754 (Jefffrey)
  • Fix ORDER BY positional reference regression with aliased aggregates #19412 (adriangb)
  • Implement disk spilling for all grouping ordering modes in GroupedHashAggregateStream #19287 (pepijnve)
  • refactor: add ParquetOpenerBuilder to reduce test code duplication #19405 (shashidhar-bm)
  • bench: add range_and_generate_series #19428 (rluvaton)
  • chore: use extend instead of manual loop in multi group by #19429 (rluvaton)
  • chore(deps): bump taiki-e/install-action from 2.64.0 to 2.64.2 #19399 (dependabot[bot])
  • Add recursive protection on planner's create_physical_expr #19299 (rgehan)
  • chore(deps): bump aws-config from 1.8.11 to 1.8.12 #19453 (dependabot[bot])
  • chore(deps): bump log from 0.4.28 to 0.4.29 #19452 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.64.2 to 2.65.1 #19451 (dependabot[bot])
  • chore(deps): bump insta from 1.44.3 to 1.45.0 #19454 (dependabot[bot])
  • added support for negative scale for log decimal32/64 and power #19409 (shifluxxc)
  • Remove core dependency from ffi #19422 (timsaucer)
  • bench: increase in_list benchmark coverage #19443 (geoffreyclaude)
  • Use SortMergeJoinExec name consistently in physical plan outputs #19246 (xavlee)
  • Fix panic during spill to disk in clickbench query #19421 (alamb)
  • Optimize memory footprint of view arrays from ScalarValue::to_array_of_size #19441 (Jefffrey)
  • minor: refactoring of some ScalarValue code #19439 (Jefffrey)
  • Refactor Spark crc32 & sha1 to remove unnecessary scalar argument check #19466 (Jefffrey)
  • Add link to arrow-rs ticket in comments #19479 (alamb)
  • chore(deps): bump taiki-e/install-action from 2.65.1 to 2.65.2 #19474 (dependabot[bot])
  • Improve plan_to_sql handling of empty projections with dialect-specific SELECT list support #19221 (kosiew)
  • examples: replace sql_dialect with custom_sql_parser example #19383 (geoffreyclaude)
  • Replace custom merge operator with arrow-rs implementation #19424 (pepijnve)
  • Implement nested recursive CTEs #18956 (Tpt)
  • Add: PI upper/lower bound f16 constants to ScalarValue #19497 (xonx4l)
  • chore: enforce clippy::allow_attributes for datafusion-ffi crate #19480 (chakkk309)
  • Add CI check to ensure examples are documented in README #19371 (cj-zhukov)
  • fix : snapshot to the modern multiline format #19517 (Nachiket-Roy)
  • chore(deps): bump taiki-e/install-action from 2.65.2 to 2.65.3 #19499 (dependabot[bot])
  • docs : clarify unused test utility #19508 (Nachiket-Roy)
  • Date / time / interval arithmetic improvements #19460 (Omega359)
  • Preserve ORDER BY in Unparser for projection -> order by pattern #19483 (adriangb)
  • Redesign the try_reverse_output to support more cases #19446 (zhuqi-lucas)
  • refactor: Spark ascii signature away from user_defined #19513 (kumarUjjawal)
  • Fix: SparkAscii nullability to depend on input nullability #19531 (Yuvraj-cyborg)
  • chore(deps): bump tracing from 0.1.41 to 0.1.43 #19543 (dependabot[bot])
  • chore(deps): bump substrait from 0.62.0 to 0.62.2 #19542 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.65.3 to 2.65.6 #19541 (dependabot[bot])
  • minor: run all examples by default #19506 (theirix)
  • Refactor TopKHashTable to use HashTable API #19464 (Dandandan)
  • Revert Spark Elt nullability change #19510 (Jefffrey)
  • minor: implement more arms for get_data_types() for NativeType #19449 (Jefffrey)
  • Upgrade hashbrown to 0.16 #19554 (Dandandan)
  • minor : add crypto function benchmark #19539 (getChan)
  • chore(deps): bump taiki-e/install-action from 2.65.6 to 2.65.8 #19559 (dependabot[bot])
  • bugfix: preserve schema metadata for record batch in FFI #19293 (timsaucer)
  • refactor: extract the data generate out of aggregate_topk benchmark #19523 (haohuaijin)
  • Compute Dynamic Filters only when a consumer supports them #19546 (LiaCastaneda)
  • Various refactors to string functions #19402 (Jefffrey)
  • Implement partition_statistics API for NestedLoopJoinExec #19468 (kumarUjjawal)
  • Replace deprecated structopt with clap in datafusion-benchmarks #19492 (Yuvraj-cyborg)
  • Refactor duplicate code in type_coercion/functions.rs #19518 (Jefffrey)
  • chore(deps): bump taiki-e/install-action from 2.65.8 to 2.65.10 #19578 (dependabot[bot])
  • perf: Improve performance of hex encoding in spark functions #19586 (shashidhar-bm)
  • Add left function benchmark #19600 (viirya)
  • chore: Add TPCDS benchmark comparison for PR #19552 (comphead)
  • chore(deps): bump taiki-e/install-action from 2.65.10 to 2.65.11 #19601 (dependabot[bot])
  • chore: bump testcontainers-modules to 0.14 and remove testcontainers dep #19620 (Jefffrey)
  • Validate parquet writer version #19515 (AlyAbdelmoneim)
  • chore(deps): bump insta from 1.45.0 to 1.46.0 #19643 (dependabot[bot])
  • chore(deps): bump taiki-e/install-action from 2.65.11 to 2.65.13 #19646 (dependabot[bot])
  • chore(deps): bump tracing from 0.1.43 to 0.1.44 #19644 (dependabot[bot])
  • chore(deps): bump syn from 2.0.111 to 2.0.113 #19645 (dependabot[bot])
  • Refactor percentile_cont to clarify support input types #19611 (Jefffrey)
  • Add a protection to release candidate branch 52 #19660 (xudong963)
  • Downgrade aws-smithy-runtime, update rust_decimal, ignore RUSTSEC-2026-0001 to get clean CI #19657 (alamb)
  • Update dependencies #19667 (alamb)
  • Refactor PartitionedFile: add ordering field and new_from_meta constructor #19596 (adriangb)
  • Remove coalesce batches rule and deprecate CoalesceBatchesExec #19622 (feniljain)
  • Perf: Optimize substring_index via single-byte fast path and direct indexing #19590 (lyne7-sc)
  • refactor: Use Signature::coercible for isnan/iszero #19604 (kumarUjjawal)
  • Parquet: Push down supported list predicates (array_has/any/all) during decoding #19545 (kosiew)
  • Remove dependency on rust_decimal, remove ignore of RUSTSEC-2026-0001 #19666 (alamb)
  • Store example data directly inside the datafusion-examples (#19141) #19319 (cj-zhukov)
  • minor: More comments to ParquetOpener::open() #19677 (2010YOUY01)
  • Feat: Allow pow with negative & non-integer exponent on decimals #19369 (Yuvraj-cyborg)
  • chore(deps): bump taiki-e/install-action from 2.65.13 to 2.65.15 #19676 (dependabot[bot])
  • Refactor cache APIs to support ordering information #19597 (adriangb)
  • Record sort order when writing Parquet with WITH ORDER #19595 (adriangb)
  • implement var distinct #19706 (thinh2)

Credits

Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.

    67	dependabot[bot]
    38	Andrew Lamb
    36	Jeffrey Vo
    35	Kumar Ujjawal
    34	Adrian Garcia Badaracco
    22	Tim Saucer
    19	Yongting You
    13	Sergey Zhukov
    11	Pepijn Van Eeckhoudt
    11	kosiew
    10	Daniël Heres
    10	Dhanush
    10	Oleks V
     8	Geoffrey Claude
     8	Raz Luvaton
     7	Andy Grove
     7	Liang-Chi Hsieh
     7	Qi Zhu
     6	Peter Nguyen
     6	Shashidhar B M
     5	Alan Tang
     5	Alex Huang
     5	Bruce Ritchie
     5	Gene Bordegaray
     5	Nuno Faria
     5	Sriram Sundar
     4	Blake Orth
     4	Thomas Tanon
     4	Yuvraj
     4	theirix
     3	Aryan Bagade
     3	Chakkk
     3	Emily Matheys
     3	Huaijin
     3	Khanh Duong
     3	Kushagra S
     3	Vedic Chawla
     3	feniljain
     3	harshit saini
     3	jizezhang
     3	shifluxxc
     3	xonx
     3	xudong.w
     2	Carlos Hurtado
     2	Chen Chongchen
     2	Cora Sutton
     2	Haresh Khanna
     2	Lía Adriana
     2	Manish Kumar
     2	Martin Grigorov
     2	Matthew Kim
     2	Namgung Chan
     2	Nimalan
     2	Nithurshen
     2	Rosai
     2	Shubham Yadav
     2	Trent Hauck
     2	Vegard Stikbakke
     2	Vrishabh
     2	Xander
     2	chakkk309
     2	mag1c1an1
     2	nlimpid
     2	yqrz
     1	Adam Curtis
     1	Aly Abdelmoneim
     1	Andrey Velichkevich
     1	Arpit Bandejiya
     1	Bharathwaj G
     1	Bipul Lamsal
     1	Clement de Groc
     1	Congxian Qiu
     1	David López
     1	David Stancu
     1	Devanshu
     1	Dongpo Liu
     1	EeshanBembi
     1	Eshaan Gupta
     1	Ethan Urbanski
     1	Frederic Branczyk
     1	Gabriel
     1	Gohlub
     1	Heran Lin
     1	James Xu
     1	Jatin Kumar singh
     1	Karan Pradhan
     1	Karthik Kondamudi
     1	Kazantsev Maksim
     1	Marco Neumann
     1	Matt Butrovich
     1	Max Burke
     1	Michele Vigilante
     1	Mikhail Zabaluev
     1	Mohit rao
     1	Ning Sun
     1	Peter Lee
     1	Quoc Anh
     1	Ram
     1	Randy
     1	Renan GEHAN
     1	Ruchir Khaitan
     1	Samyak Sarnayak
     1	Shiv Bhatia
     1	Smith Cruise
     1	Smotrov Oleksii
     1	Solari Systems
     1	Suhail
     1	T2MIX
     1	Tal Glanzman
     1	Tamar
     1	Tim-53
     1	Tobias Schwarzinger
     1	Ujjwal Kumar Tiwari
     1	Willem Verstraeten
     1	YuraLitvinov
     1	bubulalabu
     1	delamarch3
     1	hsiang-c
     1	r1b
     1	rin
     1	xavlee

Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.