ContextQMD
Back to Datafusion

12.0.0

dev/changelog/12.0.0.md

53.1.030.4 KB
Original Source
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->

12.0.0 (2022-09-12)

Full Changelog

Breaking changes:

Implemented enhancements:

  • support cast inside values #3446
  • update TPCH test schemas to use Decimal128 from Float #3435
  • Include Bitwise operators in the documentation #3434
  • How to read excel file with datafusion? #3433
  • Pass return type to the accumulator state factory in aggregates #3427
  • Support bitwise XOR operator (#) #3420
  • support InList with datatype Date32 #3412
  • add simplification for between expression during logical plan optimization #3402
  • Replace From trait with TryFrom trait for datafusion-proto crate #3401
  • update TPC-H benchmark to Decimal types from Float #3392
  • Use usize to represent Limit::skip #3369
  • Avoid coping in LogicalPlan::expressions #3368
  • Upgrade to Arrow 22 #3362
  • Eliminate OFFSET 0 in the logical plan optimization #3355
  • Add ability to get unoptimized logical plan from DataFrame #3340
  • Allow IDEs to recognize generated code #3332
  • CAST should not change the name of an expression #3326
  • add SQL support for unsigned integers #3325
  • Review use of panic in datafusion-proto crate #3318
  • Review use of panic in datafusion-sql crate #3315
  • Review use of panic in datafusion-optimizer crate #3314
  • Review use of panic in datafusion-expr crate #3312
  • Support registration of custom TableProviders through SQL #3310
  • Support binary data in sha hash functions #3308
  • add SQL support for tinyint and unsigned versions of all INTs #3307
  • Support binary types in InList expression #3300
  • Physical planner should map IsTrue and similar expressions to IsDistinctFrom #3288
  • Introduce physical plan version of Operator enum #3269
  • Introduce Expr variants for IS [NOT] TRUE / FALSE / UNKNOWN #3268
  • Add support for non-correlated subqueries #3266 [sql]
  • (Re-)add support for glob patterns in ListingTableUrl #3261
  • PreCastLitInComparisonExpressions should use ExprRewriter and supported nested expressions #3259
  • implement DROP VIEW #3251
  • Upgrade to Arrow 21 #3224
  • Add TypeCoercion optimizer rule #3221
  • Create bench for approx_percentile_cont aggregate #3217
  • Add SQL query planner support for DISTRIBUTED BY #3207
  • Support "IS [NOT] UNKNOWN" syntax #3195
  • sqlparser 0.21 upgrade #3192
  • Re-implement parsing/planning for SHOW TABLES due to sqlparser changes #3188
  • Support SUM AVG, MIN, MAX on Time columns. #3166
  • Support "IS TRUE/FALSE" syntax #3159
  • Support number of histogram bins in approx_percentile_cont #3145
  • Support create ApproxPercentileAccumulator with TDigest max_size #3142
  • Remove support for array function and only support array[] style postgres syntax #3115
  • Allow inline column aliases for create view #3108 [sql]
  • Add support for Postgres SIMILAR TO and ILIKE syntax #3099 [sql]
  • Update SQL reference in user guide to cover all supported syntax #3091
  • DataFusion prelude should import all logical expression functions #3068
  • Proposal: Add similar to operator #3016 [sql]
  • Release DataFusion 11.0.0 #3012
  • Implement "SHOW CREATE TABLE" for external tables #2848
  • Change java package names in protobuf files #2513
  • When creating DFField from Expr we should provide input plan not input schema #2456
  • Support "IS NOT TRUE/FALSE" syntax #2265
  • RFC: Spill-To-Disk Object Storage Download #2205
  • Support for BitwiseAnd &, BitOr | binary operators #1619
  • [Question] Usage of async object store APIs in consuming code #1313
  • Allow User Defined Aggregates to return multiple values / structs #600
  • Implement vectorized hashing for dictionary types #331

Fixed bugs:

  • Intermittent build error when changing selected features #3366
  • sql::timestamp::timestamp_add_interval_months failing since September 1st #3327
  • sql::timestamp::timestamp_add_interval_months test fails #3322
  • test case timestamp_add_interval_months failed on master branch #3321
  • datafusion-proto does not support untyped null scalar values #3302
  • ConfigOptions creation is slow #3295
  • FilterPushDown optimization through UNION ALL results in SchemaError #3281
  • Execute LogicalPlans after building for TPCH Benchmarks #3273
  • CREATE TABLE should return empty DataFrame #3265 [sql]
  • CREATE EXTERNAL TABLE from CSV creates a table with no columns if there is just a header row #3263
  • View TableProvider ignores projections, resulting in invalid plans #3240
  • CREATE VIEW should return an empty dataframe on success #3236
  • DISTRIBUTE BY expressions get removed during optimization #3234
  • datafusion cannot recognize chinese charactors. #3203
  • Panicked at 'byte index 1 is out of bounds on invalid query #3190
  • like_nlike_with_null_lt fails with latest sqlparser code #3187
  • Interval Literal output inconsistent date_type #3180
  • array function allows different data types #3123
  • eq operator doesn't work on binary data #3117
  • incorrect where clause comparison while using table alias #3073
  • Some functions are incorrectly declared as unary #3069
  • once now() is called in a statement, it forever returns the same value #3057
  • single_distinct_to_groupby panic when group by expr is a binaryExpr #2994
  • Cannot have order by expression that references complex group by expression #2360
  • Fix some bugs in TypeCoercion rule #3407 (andygrove)
  • MINOR: Stop ignoring AggregateFunction::distinct in protobuf serde code #3250 (andygrove)
  • Add assertion for invariant in create_physical_expression and fix ViewTable projection #3242 (andygrove)
  • Fix bug where optimizer was removing Partitioning::DistributeBy expressions #3229 (andygrove)

Documentation updates:

Closed issues:

  • Add \i command to datafusion-cli #1906
  • TPC-H Query 15 #166

Merged pull requests: