docs/changelogs/v25.9.2.1-stable.md
allow_dynamic_metadata_for_data_lakes. Now all iceberg tables try to fetch up-to-date table schema from storage before executing of each query. #86366 (Daniil Ivanik).arrayExcept that subtracts one array as a set from another. #82368 (Joanna Hulboj).database_replicas with information about database replicas. #83408 (Konstantin Morozov).nats_stream and nats_consumer for the NATS engine. #84799 (Dmitry Novikov).system.aggregated_zookeeper_log table. The table contains statistics (e.g. number of operations, average latency, errors) of ZooKeeper operations grouped by session id, parent path and operation type, and periodically flushed to disk. #85102 (Miсhael Stetsyuk).isValidASCII function to check if string contains only ASCII characters. Close #85377. #85786 (rajat mohan).1 (e.g. SELECT ... SETTINGS use_query_cache is equivalent to use_query_cache = 1). #85800 (thraeka).logger.startupLevel & logger.shutdownLevel allow for overriding the log level during the startup & shutdown of Clickhouse respectively. #85967 (Lennard Eijsackers).timeSeriesChangesToGrid and timeSeriesResetsToGrid. Behaves similarly to timeSeriesRateToGrid, accepting parameters for start timestamp, end timestamp, step, and look back window, as well as two arguments for the timestamps and values, but requiring at least 1 sample per window instead of 2. Calculates a PromQL changes/resets, counting the number of times the sample value changes or decreases in the specified window for each timestamp in the time grid defined by the parameters. The return type is Array(Nullable(Float64)). #86010 (Stephen Chi).ALTER UPDATE for Iceberg table engine. #86059 (scanhex12).iceberg_metadata_log to retrieve Iceberg metadata files during SELECT statements. #86152 (scanhex12).allowed_disks_for_table_engines which allows specific disks to use for Iceberg. Example: CREATE TABLE test ENGINE = Iceberg('path/inside/disk') SETTING datalake_disk_name = '<some_user_disk>'; ### Documentation entry for user-facing changes. #86778 (scanhex12).system.warnings table. #86838 (Bharat Nallan).func(primary_column) = 'xx' and column in (xxx). #85529 (李扬).LEFT ANY JOIN and RIGHT ANY JOIN to SEMI/ANTI JOIN if the filter condition is always false for matched or non-matched rows. This optimization is controlled by a new setting query_plan_convert_any_join_to_semi_or_anti_join. 2. Convert FULL ALL JOIN to LEFT ALL or RIGHT ALL JOIN if the filter condition is always false for non-matched rows from one side. #86028 (Dmitry Novik).HashJoin performance optimised slightly in the case of LEFT/RIGHT join having a lot of unmatched rows. #86312 (Nikita Taranov).allow_statistics_optimize = 1 and query_plan_optimize_join_order_limit = 10. #86822 (Han Fei).MarkRanges by using devector over deque). #86933 (Azat Khuzhin).query_plan_optimize_join_order_limit setting). Note that the join order optimization currently has limited statistics support and primarily relies on row count estimates from storage engines - more sophisticated statistics collection and cardinality estimation will be added in future releases. If you encounter issues with JOIN queries after upgrading, you can temporarily disable the new implementation by setting SET query_plan_use_new_logical_join_step = 0 and report the issue for investigation. Note about resolution of identifiers from USING clause: Changed resolving of the coalesced column from OUTER JOIN ... USING clause to be more consistent: previously, when selecting both the USING column and qualified columns (a, t1.a, t2.a) in a OUTER JOIN, the USING column would incorrectly be resolved to t1.a, showing 0/NULL for rows from the right table with no left match. Now identifiers from USING clause are always resolved to the coalesced column, while qualified identifiers resolve to the non-coalesced columns, regardless of which other identifiers are present in the query. For example: ```sql SELECT a, t1.a, t2.a FROM (SELECT 1 as a WHERE 0) t1 FULL JOIN (SELECT 2 as a) t2 USING (a) -- Before: a=0, t1.a=0, t2.a=2 (incorrect - 'a' resolved to t1.a) -- After: a=2, t1.a=0, t2.a=2 (correct - 'a' is coalesced). #80848 (Vladimir Cherkasov).use_skip_indexes_on_data_read (disabled by default). This addresses #75774. This includes some common groundwork shared with #81021. #81526 (Amos Bird).best_effort format. #84730 (Yarik Briukhovetskyi).TO when creating a materialized view, for example: CREATE MATERIALIZED VIEW mv TO {to_table:Identifier} AS SELECT * FROM src_table. #84899 (Diskein).jemalloc_enable_global_profiler. Sampled global allocations and deallocations can be stored in system.trace_log under JemallocSample type by enabling config jemalloc_collect_global_profile_samples_in_trace_log. Jemalloc profiling can now be enabled for each query independently using setting jemalloc_enable_profiler. Storing samples in system.trace_log can be controlled per query using setting jemalloc_collect_profile_samples_in_trace_log. #85438 (Antonio Andelic).backup_slow_all_threads_after_retryable_s3_error to S3 disks and renames it to the more general s3_slow_all_threads_after_retryable_error. #85918 (Julia Kartseva).use_persistent_processing_nodes (changeable by ALTER TABLE MODIFY SETTING). #85995 (Kseniia Sumarokova).query_condition_cache_selectivity_threshold (default value: 1.0) which excludes scan results of predicates with low selectivity from insertion into the query condition cache. This allows to reduce the memory consumption of the query condition cache at the cost of a worse cache hit rate. #86076 (zhongyuankai).full_url directive) in http_handlers (including schema and host:port). #86155 (Azat Khuzhin).allow_experimental_delta_lake_writes, disabled by default. #86180 (Kseniia Sumarokova).startup_scripts_failure_reason dimensional metric. This metric is needed to distinguish between different error types that result in failing startup scripts. In particular, for alerting purposes, we need to distinguish between transient (e.g., MEMORY_LIMIT_EXCEEDED or KEEPER_EXCEPTION) and non-transient errors. #86202 (Miсhael Stetsyuk).identity() function for partition for Iceberg table. #86314 (scanhex12).logger.formatting.channel to one of syslog/console/errorlog/log. #86331 (Azat Khuzhin).delta_lake_insert_max_rows_in_data_file and delta_lake_insert_max_bytes_in_data_file. #86357 (Kseniia Sumarokova).WHERE. They are already allowed to be arguments of logical functions. This simplifies filter-push-down and move-to-prewhere optimizations. #86390 (Nikolai Kochetov).SYSTEM DROP REPLICA against a Catalog with corrupted metadata. #86391 (Nikita Mikhaylov).skip_access_check=0) in Azure because it may be provisioning access for quite a long time. #86419 (Alexander Tokmakov).evaluation_time to promql_evaluation_time. #86459 (Vitaly Baranov).today() function case insensitive to make it consistent with other date/time related functions like NOW(). #86561 (Kaviraj Kanagaraj).timeSeries*() functions left-open and right-closed. #86588 (Vitaly Baranov).FailedInternal*Query profile events. #86627 (Shane Andrade).QueriesMemoryUsage and QueriesPeakMemoryUsage). #86669 (Azat Khuzhin).clickhouse-benchmark --precise flag for more precise reporting of QPS and other per-interval metrics. It helps to get consistent QPS in case if durations of queries are comparable to the reporting interval --delay D. #86684 (Sergei Trifonov).EXPLAIN query. Do not calculate the description for queries other than EXPLAIN. Added a steeing query_plan_max_step_description_length. #86741 (Nikolai Kochetov).query_profiler_real_time_period_ns/query_profiler_cpu_time_period_ns). And also collect SigQ from the /proc/self/status for introspection (if ProcessSignalQueueSize is near to ProcessSignalQueueLimit, then you will likely get CANNOT_CREATE_TIMER errors). #86760 (Azat Khuzhin).prefix.path when directory is removed for plain rewriteable disk. #86908 (alesapin).yesterday() function case insensitive and consistent with today() function. #86914 (Kaviraj Kanagaraj)..xml performance testing against remote ClickHouse instances, including ClickHouse Cloud. Usage example: tests/performance/scripts/perf.py tests/performance/math.xml --runs 10 --user <username> --password <password> --host <hostname> --port <port> --secure. #86995 (Raufs Dunamalijevs).network_compression_method is not a supported generic codec. #87097 (Robert Schulze).system.query_cache now returns all query result cache entries, whereas it previously returned only shared entries or non-shared entries of the same user and role. That is okay as non-shared entries are supposed to not reveal query results, whereas system.query_cache returns query strings. This makes the behavior of the system table more similar to system.query_log. #87104 (Robert Schulze).arrowFlight() table function. #87120 (Vitaly Baranov).S3 table engine and s3 table function named storage_class_name which allows to specify intelligent tiring supported by AWS. Supported both in key-value format and in positional (deprecated) format). #87122 (alesapin).alter table ... materialize statistics all will materialize all the statistics of a table. #87197 (Han Fei).s3_slow_all_threads_after_retryable_error by default. #87198 (Nikita Mikhaylov).system.aggregated_zookeeper_log table. The table contains statistics (e.g. number of operations, average latency, errors) of ZooKeeper operations grouped by session id, parent path and operation type, and periodically flushed to disk. #87208 (Miсhael Stetsyuk).statistics in system.parts_columns. #87259 (Han Fei).BackgroundSchedulePool. Avoid situations when all slots are occupied by task of one type, while other tasks are starving. Also avoids deadlocks when tasks wait for each other. This is controlled by background_schedule_pool_max_parallel_tasks_per_type_ratio server setting. #84008 (Alexander Tokmakov).IN functions sometimes resulting in error when array mapping is used. #85546 (Yakov Olkhovskiy).5.10 release. #85837 (Konstantin Bogdanov).NonZeroUInt64 for logs_to_keep in DatabaseReplicatedSettings. #86142 (Tuan Pham Anh).FINAL query with skip index if the table (e.g ReplacingMergeTree) was created with settingindex_granularity_bytes = 0. That exception has been fixed now. #86147 (Shankar Iyer)./etc/metrika.xml as a default when creating disks from SQL. #86232 (alekar).416 The range specified is invalid for the current size of the resource. The range specified is invalid for the current size of the resource when reading empty blobs from Azure blob storage for plain_rewritable disk. #86400 (Julia Kartseva).MergesMutationsMemoryTracking due to Buffer tables and fix query_views_log for streaming from Kafka (and others). #86422 (Azat Khuzhin).searchAny and searchAll when called with empty needles now return true (aka. "matches everything"). Previously, they returned false. (issue #86300). #86500 (Elmi Ahmadov).timeSeriesResampleToGridWithStaleness() when the first bucket has no value. #86507 (Vitaly Baranov).merge_tree_min_read_task_size being set to 0. #86527 (yanglongwei).listen_host for ArrowFlight. #86664 (Vitaly Baranov).ArrowFlight handler. This PR fixes #86596. #86665 (Vitaly Baranov).describe_compact_output=1. #86676 (Azat Khuzhin).Partition strategy wildcard can not be used without a '_partition_id' wildcard. when creating a table with PARTITION BY, but without partition wildcard, which used to work in versions before 25.8. Closes https://github.com/ClickHouse/clickhouse-private/issues/37567. #86748 (Kseniia Sumarokova).system.distributed_ddl_queue in cases where tasks are missing certain Keeper nodes. #86848 (Antonio Andelic).TTL with WHERE. #86965 (Anton Popov).uniqExact function with ROLLUP and CUBE modifiers. #87014 (Nikita Taranov).url() table function when parallel_replicas_for_cluster_functions setting is set to 1. #87029 (Konstantin Bogdanov).ON CLUSTER clause. #87043 (Anton Popov).max_temporary_data_on_disk_size limit tracking, close #87118. #87140 (JIaQi).EXISTS. It was broken with execute_exists_as_scalar_subquery=1 which was introduced in https://github.com/ClickHouse/ClickHouse/pull/85481 and affects 25.8. Fixes #86415. #87207 (Nikolai Kochetov).LOGICAL_ERROR in JOIN due to mixed const and non-const blocks. #87231 (Azat Khuzhin).s3_plain_rewritable (which may lead to all sort of troubles). #87426 (Azat Khuzhin).vmull_p64 instruction. #86521 (Pablo Marcos).openldap 2.6.10. #86623 (Konstantin Bogdanov).memalign on darwin. #86769 (Konstantin Bogdanov).krb5 1.22.1-final. #86836 (Konstantin Bogdanov).list-licenses.sh. #87305 (Konstantin Bogdanov).chcache in CI. #85686 (Konstantin Bogdanov).02443_detach_attach_partition. #86128 (Nikita Mikhaylov).max_digestion_size_per_segment back, mark as obsolete. #86170 (Robert Schulze).chcache: add basic stats. #86177 (Konstantin Bogdanov).chcache: collect more telemetry. #86212 (Konstantin Bogdanov).testcase-database can help with flaky checks. #86267 (Mikhail Artemenko).use_skip_indexes_on_data_read by default. https://github.com/ClickHouse/ClickHouse/pull/81526 is merged with this setting enabled to do more thorough testing. cc @CurtizJ. #86273 (Amos Bird).ErrorCodes::TIMEOUT_EXCEEDED when it fails. #86321 (Tuan Pham Anh).03580_improve_prewhere. #86388 (Nikita Taranov).-Wmissing-noreturn. #86497 (Konstantin Bogdanov).test_keeper_reconfig_remove. #86617 (Antonio Andelic).SerializationMap deserialization. #86798 (Raufs Dunamalijevs).AVX_FUNCTION_SPECIFIC_ATTRIBUTE macro. #86881 (Raufs Dunamalijevs).isIPAddressInRange: Rename file after the function it contains. #87017 (Robert Schulze).Clickhouse -> ClickHouse. #87023 (Konstantin Bogdanov).datalake_disk_name to disk. #87388 (Kseniia Sumarokova).