ContextQMD
Back to Datafusion

8.0.0

dev/changelog/8.0.0.md

53.1.042.4 KB
Original Source
<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->

8.0.0 (2022-05-12)

Full Changelog

Breaking changes:

  • Add SQL planner support for ROLLUP and CUBE grouping set expressions #2446 (andygrove)
  • Make ExecutionPlan::execute Sync #2434 (tustvold)
  • Introduce new DataFusionError::SchemaError type #2371 (andygrove)
  • Add Expr::InSubquery and Expr::ScalarSubquery #2342 (andygrove)
  • Add Expr::Exists to represent EXISTS subquery expression #2339 (andygrove)
  • Move LogicalPlan enum to datafusion-expr crate #2294 (andygrove)
  • Remove dependency from LogicalPlan::TableScan to ExecutionPlan #2284 (andygrove)
  • Move logical expression type-coercion code from physical-expr crate to expr crate #2257 (andygrove)
  • feat: 2061 create external table ddl table partition cols #2099 [sql] (jychen7)
  • Reorganize the project folders #2081 (yahoNanJing)
  • Support more ScalarFunction in Ballista #2008 (Ted-Jiang)
  • Merge dataframe and dataframe imp #1998 (vchag)
  • Rename ExecutionContext to SessionContext, ExecutionContextState to SessionState, add TaskContext to support multi-tenancy configurations - Part 1 #1987 (mingmwang)
  • Add Coalesce function #1969 (msathis)
  • Add Create Schema functionality in SQL #1959 [sql] (matthewmturner)
  • omit some clone when converting sql to logical plan #1945 [sql] (doki23)
  • [split/16] move physical plan expressions folder to datafusion-physical-expr crate #1889 (Jimexist)
  • remove sync constraint of SendableRecordBatchStream #1884 (doki23)
  • [split/15] move built in window expr and partition evaluator #1865 (Jimexist)

Implemented enhancements:

  • Include Expr to datafusion::prelude #2347
  • Implement Serialization API for DataFusion #2340
  • Implement power function #1493
  • allow lit python function to support boolean and other types #1136
  • Automate dependency updates #37
  • Add CREATE VIEW #2279 (matthewmturner)
  • [Ballista] Support Union in ballista. #2098 (Ted-Jiang)
  • Change the DataFusion explain plans to make it clearer in the predicate/filter #2063 (Ted-Jiang)
  • Add write_json, read_json, register_json, and JsonFormat to CREATE EXTERNAL TABLE functionality #2023 (matthewmturner)
  • Qualified wildcard #2012 [sql] (doki23)
  • support bitwise or/'|' operation #1876 [sql] (liukun4515)
  • Introduce JIT code generation #1849 (yjshen)

Fixed bugs:

  • CASE expr with NULL literals panics 'WHEN expression did not return a BooleanArray' #1189
  • Function calls with NULL literals do not work #1188
  • Add SQL planner support for calling round function with two arguments #2503 (andygrove)
  • nested query fix #2402 (comphead)
  • fix issue#2058 file_format/json.rs attempt to subtract with overflow #2066 (silence-coding)
  • fix bug the optimizer rule filter push down #2039 (jackwener)
  • fix: replace ExecutionContex and ExecutionConfig with SessionContext and SessionConfig #2030 (xudong963)
  • Fixed parquet path partitioning when only selecting partitioned columns #2000 (pjmore)
  • Fix ambiguous reference error in filter plan #1925 (jonmmease)
  • platform aware partition parsing #1867 (korowa)
  • Fix incorrect aggregation in case that GROUP BY contains duplicate column names #1855 (alex-natzka)

Documentation updates:

Performance improvements:

Closed issues:

  • Make expected result string in unit tests more readable #2412
  • remove duplicated fn aggregate() in aggregate expression tests #2399
  • split distinct_expression.rs into count_distinct.rs and array_agg_distinct.rs #2385
  • move sql tests in context.rs to corresponding test files in datafustion/core/tests/sql #2328
  • Date32/Date64 as join keys for merge join #2314
  • Error precision and scale for decimal coercion in logic comparison #2232
  • Support Multiple row layout #2188
  • TPC-H Query 18 #169
  • TPC-H Query 16 #167
  • Implement Sort-Merge Join #141
  • Split logical expressions out into separate source files #114

Merged pull requests: