23.0.0

<!--- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. -->

23.0.0 (2023-04-21)

Full Changelog

Breaking changes:

• Add new physical rule CombinePartialFinalAggregate #5837 (mingmwang)
• feat: Remove compact row since it's no longer used #6021 (yjshen)

Implemented enhancements:

• feat: extend substrait type support, including type variations #5775 (waynexia)
• feat: hyperbolic functions #5884 (izveigor)
• feat: Support SQL filter clause for aggregate expressions, add SQL dialect support #5868 (yjshen)
• feat: Implement the bitwise_not in NotExpr #5902 (RTEnzyme)
• feat: implementation of the constant "Pi" #5965 (izveigor)
• feat: allow the customization of analyzer rules #5963 (waynexia)
• feat: extra sqllogictests for scalar functions #5887 (izveigor)
• feat: degrees and radians #6023 (izveigor)
• feat: allow TableScan without projection #6032 (waynexia)
• feat: support month and year interval for date_bin on constant data #5982 (NGA-TRAN)

Fixed bugs:

• fix: type coercion for expr/subquery in InSubquery #5883 (jackwener)
• fix: binaryExpr not supported for LargeUtf8 #5896 (jackwener)
• fix: match boolean null for substrait #5947 (waynexia)
• fix: return NotImplemented when execute SELECT INTO syntax #5945 (r4ntix)
• fix: support date - timestamp and timestamp - date #5960 (jackwener)
• fix: do not depend on time 0.1 #5972 (crepererum)
• fix: largeUtf8 don't support is distinct from #5993 (jackwener)
• fix: Do not panic on invalid placeholders #5998 (Weijun-H)
• fix: Allow values in subqueries #6018 (scsmithr)
• fix: split "union" and "interleave" #6045 (crepererum)

Documentation updates:

• docs: improve expressions.md #5978 (izveigor)
• minor: add developer's guide link in README.md #6055 (jackwener)

Merged pull requests:

• fix: type coercion for expr/subquery in InSubquery #5883 (jackwener)
• Minor: fix the architecture slide url #5920 (gaoxinge)
• Count agg support multiple expressions #5908 (allenma)
• feat: extend substrait type support, including type variations #5775 (waynexia)
• fix: binaryExpr not supported for LargeUtf8 #5896 (jackwener)
• Clean up SortExec creation and add doc comments #5889 (alamb)
• Improve file scan time opening metric to include start_next_file #5900 (alamb)
• Remove unsafe code (transmute) in datafusion-proto #5946 (alamb)
• support date_trunc with millisecond and microsecond and keep the same type as the input #5818 (Weijun-H)
• fix: match boolean null for substrait #5947 (waynexia)
• feat: hyperbolic functions #5884 (izveigor)
• Planner: normalize_ident only when enable_ident_normalization is enabled #5785 (ayushdg)
• Improve contributor guide and main API landing page #5921 (alamb)
• minor: Refactor row_hash implementation #5936 (mustafasrepo)
• Specialize Primitive Cursor -- make sorts / merges on a single primitive column faster #5897 (tustvold)
• just match for BinaryExpr Date/Time +/- Interval #5932 (jackwener)
• Improve avg/sum Aggregator performance for Decimal #5866 (mingmwang)
• fix: return NotImplemented when execute SELECT INTO syntax #5945 (r4ntix)
• Update arrow 37 #5782 (tustvold)
• Scalar arithmetic should return error when overflows. #5811 (zhzy0077)
• feat: Support SQL filter clause for aggregate expressions, add SQL dialect support #5868 (yjshen)
• test: add Between UT for type_coercion #5929 (jackwener)
• feat: Implement the bitwise_not in NotExpr #5902 (RTEnzyme)
• Minor: Add DFField.with_nullable(bool) #5966 (comphead)
• Move error check from pipeline fixer to pipeline checker #5938 (mustafasrepo)
• when inferring the schema of compressed CSV, decompress before newline-delimited chunking #5860 (jiangzhx)
• Update sqlparser requirement from 0.32 to 0.33 #5957 (dependabot[bot])
• fix: support date - timestamp and timestamp - date #5960 (jackwener)
• minor: Add Expr::between to clean up boilerplate #5967 (alamb)
• Add new physical rule CombinePartialFinalAggregate #5837 (mingmwang)
• Specialized Cursor for StringArray and BinaryArray #5964 (tustvold)
• add an example of using DataFrame to create a subquery #5961 (jiangzhx)
• feat: implementation of the constant "Pi" #5965 (izveigor)
• fix: do not depend on time 0.1 #5972 (crepererum)
• Minor: fix wrong code comment #5979 (viirya)
• [DOCS]: consolidate doc site content simplify navbar #5962 (alamb)
• minor: port some expr tests to sqllogictests, improve error message #5968 (alamb)
• minor: fix doctest that runs accidentally #5989 (yjshen)
• Row AVG accumulator support Decimal type #5973 (mingmwang)
• Treat Partition by columns as set for window functions #5951 (mustafasrepo)
• feat: allow the customization of analyzer rules #5963 (waynexia)
• feat: extra sqllogictests for scalar functions #5887 (izveigor)
• docs: improve expressions.md #5978 (izveigor)
• Minor: Improve doc comments in FileStream #5898 (alamb)
• Update prost-build requirement from =0.11.8 to =0.11.9 #5987 (dependabot[bot])
• Don't use parquet file offset for file range pruning #5997 (tustvold)
• Streaming Memory Reservation in SHJ #5937 (metesynnada)
• Temporal datatype support for interval arithmetic #5971 (berkaysynnada)
• Remove optimize_children and replace with map_children #5984 (2010YOUY01)
• doc: fix site sidebar logo position #5990 (Jefffrey)
• fix: largeUtf8 don't support is distinct from #5993 (jackwener)
• Proper resolution for old name in with_column_renamed #5992 (Jefffrey)
• minor:update subquery example #6011 (jiangzhx)
• fix: Do not panic on invalid placeholders #5998 (Weijun-H)
• fix: Allow values in subqueries #6018 (scsmithr)
• chore: make JsonOpener and CsvOpener public #6004 (WenyXu)
• update count_wildcard_rule for more scenario #6010 (jiangzhx)
• Add analyzer output to verbose explain #6020 (Jefffrey)
• Minor: Fix compilation error #6029 (viirya)
• Open/Closed bounds for interval arithmetic #6007 (berkaysynnada)
• minor:move log_plan to utils #6030 (jiangzhx)
• Use OwnedTableReference for subquery aliases #6022 (scsmithr)
• Update sql doc #6025 (Jefffrey)
• feat: Remove compact row since it's no longer used #6021 (yjshen)
• Add test for handling precision overflow when casting from integer to decimal #6041 (viirya)
• feat: degrees and radians #6023 (izveigor)
• feat: allow TableScan without projection #6032 (waynexia)
• refactor: make FirstSelector not to have update and merge #6038 (Weijun-H)
• Port math.rs to sqllogictest #6037 (2010YOUY01)
• Row accumulator support update Scalar values #6003 (mingmwang)
• Minor: tweak docuement of sort enforcement optimizer rule #6054 (waynexia)
• Decimal multiply kernel should not cause precision loss #5980 (viirya)
• Clean up rustdoc and add doc lint #6044 (alamb)
• Minor: port more create_drop table tests to sqllogictests #6031 (jiangzhx)
• minor feat: impl FromStr for JoinType enum #6033 (nkarpov)
• Float support on interval artihmetics #6048 (metesynnada)
• minor: add developer's guide link in README.md #6055 (jackwener)
• feat: support month and year interval for date_bin on constant data #5982 (NGA-TRAN)
• Update rust_lint.sh to include newly added cargo doc check #6070 (alamb)
• Add support for UDAF in physical plan serialization #6063 (thinkharderdev)
• fix: split "union" and "interleave" #6045 (crepererum)
• Minor: assorted schema handling debug improvements #6076 (crepererum)