What's new in the v2.25 preview release series

What follows are the release notes for the YugabyteDB v2.25 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.25 release series.

Release announcements

• Doubling Down on PostgreSQL Compatibility: YugabyteDB Levels Up with PG15 Features

v2.25.2.0 - May 20, 2025 {#v2.25.2.0}

Build: 2.25.2.0-b359

Downloads

Docker:

docker pull yugabytedb/yugabyte:2.25.2.0-b359

Change log

New features

• yugabyted UI now displays xCluster replication details for the source and destination universe. {{<tags/feature/tp idea="1763">}}

Improvements

YSQL

• Enhances nested loop joins by rechecking pushability of conditions and renames relevant function to reduce confusion. {{<issue 25994>}}
• Restores CREATE permission on the public schema to yb_db_admin. {{<issue 26218>}}
• Exempts walsender from YSQL backend check to prevent index creation delays. {{<issue 26587>}}
• Enables viewing TCMalloc heap snapshots for PG backend processes via new YSQL functions. {{<issue 21851>}}
• Enhances yb_servers function to include universe_uuid for better cluster identification. {{<issue 25818>}}
• Fixes comment linting issues to handle non-word characters. {{<issue 25810>}}
• Enhances ASH data retrieval in query diagnostics using the SPI framework. {{<issue 25865>}}
• Allows customization of ybhnsw index creation options in YSQL. {{<issue 25859>}}
• Integrates new data types and functions from pgvector 0.8.0 into YSQL. {{<issue 25990>}}
• Enables on-demand logging and enhanced catalog cache statistics tracking. {{<issue 26023>}}
• Enables conditional checks for role existence in ysql_dump outputs with the dump_role_checks flag. {{<issue 25877>}}
• Removes the check that the first operation in a plain session must set the read time. {{<issue 26151>}}
• Enhances code consistency in ybgate_api.h by matching PostgreSQL style. {{<issue 26176>}}
• Consolidates multiple suppression flags into one for cleaner pg_regress outputs. {{<issue 26197>}}
• Refactors PgDocReadOp to enhance modularity by isolating sampling logic into PgDocSampleOp. {{<issue 26165>}}
• Enables ALTER TYPE ... SET SCHEMA support for orafce extension upgrades. {{<issue 26215>}}
• Enhances pg_stat_get_progress_info by adding new fields. {{<issue 26273>}}
• Eliminates unnecessary workaround in ALTER TABLE operations related to constraint handling. {{<issue 26315>}}
• Reinstates checks for ash_metadata in PgClient RPC requests with added code explanations. {{<issue 26268>}}
• Re-adds bitmap_scans_distinct test to ensure consistent behavior. {{<issue 26277>}}
• Adds support for datetime and UUID type pushdown in mixed mode. {{<issue 26510>}},{{<issue 26511>}}
• Organizes YSQL code by splitting function definitions into a new file. {{<issue 26300>}}
• Enables expression pushdown for MOD, LIKE, ASCII, SUBSTRING in mixed mode upgrades. {{<issue 26512>}}
• Disables AutoAnalyze during the entire PG15 upgrade to ensure stability. {{<issue 26412>}}
• Enforces naming conventions for distinguishing YugabyteDB-specific files. {{<issue 26457>}}
• Aligns CurrentMemoryContext handling more closely with PostgreSQL updates. {{<issue 26496>}}
• Enhances compatibility with PostgreSQL numeric tests by refining data ordering and simplifying table structures. {{<issue 26563>}}
• Maintains workaround in pg_dump to support upgrades with pg_stat_statements. {{<issue 26575>}}
• Ensures consistent transaction path settings for single-shard operations. {{<issue 26720>}}
• Merges PostgreSQL 15.12 improvements into YugabyteDB, enhancing database compatibility and stability. {{<issue 26746>}}
• Allows users to adjust ybhnsw.ef_search for HNSW index searches in YSQL. {{<issue 26629>}}
• Automatically maps hnsw to ybhnsw in CREATE INDEX statements for seamless index creation. {{<issue 26549>}}
• Recommends changing isolation level to read committed to avoid errors during concurrent inserts. {{<issue 26648>}}
• Excludes PostgreSQL owned code from bad_variable_declaration_spacing lint rule. {{<issue 26651>}}
• Adds server_type option to differentiate foreign servers in postgres_fdw. {{<issue 25500>}}
• Renames switch_fallthrough to yb_switch_fallthrough for consistency. {{<issue 26816>}}
• Enables the PostgreSQL anonymizer extension via the enable_pg_anonymizer flag. {{<issue 26804>}}
• Enhances error reporting by including index names for missing rows. {{<issue 26819>}}
• Displays rows removed by YugabyteDB index recheck in execution plans. {{<issue 26827>}}
• Aligns get_relation_constraint_attnos function to use correct flag. {{<issue 26846>}}
• Disallow setting ysql_select_parallelism to zero to prevent errors. {{<issue 26837>}}
• Removes pg_strtouint64 and adopts strtoull or strtou64 for consistency. {{<issue 26870>}}
• Aligns YSQL more closely with upstream PostgreSQL, reducing discrepancies and streamlining changes. {{<issue 26881>}}
• Logs now detail the cause and context of read restart errors for better troubleshooting. {{<issue 24431>}}
• Limits output buffer to 8kB to ensure compatibility with certain clients. {{<issue 26744>}}
• Enhances TServer by adding support for garbage collection of invalidation messages, reducing memory usage. {{<issue 26940>}}
• Increases the timeout for detecting pg_yb_catalog_version mode from 10 seconds to 20 seconds. {{<issue 26990>}}
• Enhances pg_stats with length and bounds histograms for better query planning. {{<issue 6237>}}
• Fixes build failures and enhances memory usage reporting with TCMalloc stats. {{<issue 3190>}}

YCQL

• Allows setting NULL in YCQL JSONB column values using UPDATE statements. {{<issue 5907>}}

DocDB

• Enables placing intermediate CA certificates directly in the server cert file for node-to-node encryption. {{<issue 25972>}}
• Tracks ByteBuffer memory usage with MemTracker. {{<issue 26875>}}
• Allows dynamic adjustment of rocksdb_compact_flush_rate_limit_bytes_per_sec across all tablets. {{<issue 25611>}}
• Selects geographically closest TServer for faster clone operations. {{<issue 26788>}}
• Switches most builds to clang 19, enhancing code safety and addressing new warnings. {{<issue 25826>}}
• Introduces block-based data organization in YbHnsw for efficient memory management during data loading and unloading. {{<issue 25571>}}
• Enables manual compaction of vector index chunks in Vector LSM. {{<issue 24071>}}
• Ensures vector index backfill reads from the indexed table at the correct time. {{<issue 26046>}}
• Upgrades protobuf to version 21.12 for better C++23 compatibility. {{<issue 26128>}}
• Updates codebase to C++23, enhancing compatibility and performance. {{<issue 26346>}}
• Enables sequence replication in xCluster by default, removing the need for a flag. {{<issue 26029>}}
• Adds logging for vector index search stats when vector_index_dump_stats flag is true. {{<issue 26072>}}
• Ensures consistent bootstrapping of vector indexes after a TServer restart. {{<issue 26087>}}
• Enhances handling of expired snapshots by retrying deletion tasks automatically. {{<issue 25628>}}
• Ensures vector index contains all entries from the indexed table. {{<issue 26150>}}
• Adds detailed cluster balancer warnings to the master UI page. {{<issue 26159>}}
• Adds tombstones to obsolete vector IDs, reducing queries to the main table. {{<issue 24064>}}
• Displays cluster balancer tasks on the master UI page. {{<issue 26157>}}
• Adds annotations to prevent compiler reordering in shared memory interactions. {{<issue 26261>}}
• Uses non-concurrent mode by default for creating vector indexes to streamline processes. {{<issue 26402>}}
• Adds safeguard to pause replication after repeated DDL failures. {{<issue 26093>}}
• Enhances xCluster DDL replication by adjusting yb_read_time usage and silencing related warnings. {{<issue 25629>}}
• Renames docdb::VectorIndex to docdb::DocVectorIndex to eliminate name confusion. {{<issue 26304>}}
• Allows specific compaction and flush for vector indexes via yb-admin commands. {{<issue 26332>}}
• Adds yb-ts-cli compact_vector_index command for tablet-specific vector index compaction. {{<issue 26338>}}
• Adds automatic_mode flag to create_checkpoint for simpler xCluster setup. {{<issue 26355>}}
• Enables dropping vector indexes in DocDB. {{<issue 26359>}}
• Displays replication mode in the master UI Outbound Replication Groups section. {{<issue 26368>}}
• Enhances vector index compaction with a new deletion API and clearer naming conventions. {{<issue 26433>}}
• Automatic mode now always requires bootstrapping to ensure OID consistency. {{<issue 24030>}}
• Reuse threads to enhance connection efficiency in shared memory communication. {{<issue 26501>}}
• Enhances vector index query stats logging and adds new metrics tracking. {{<issue 26597>}}
• Enhances monitoring by using thread pool names for thread categorization. {{<issue 26656>}}
• Simplifies navigation and modification of master async RPC tasks code. {{<issue 26685>}}
• Introduces idle timeouts in rpc::ThreadPool to automatically adjust thread counts based on activity, enhancing resource efficiency. {{<issue 26708>}}
• Switches to MPSCQueue for enhanced single-consumer performance and simpler maintenance. {{<issue 26774>}}
• Adds support for the DocumentDB extension v0.102-0 to enhance database functionality. {{<issue 26749>}}
• Simplifies xCluster DDL replication tests by removing bidirectional roles. {{<issue 26762>}}
• Allows setting ybhnsw.ef_search to customize search expansion factor. {{<issue 26883>}}
• Adds paginated querying for vector index operations. {{<issue 26884>}}
• Cancels vector index compaction during VectorLSM shutdown. {{<issue 26489>}}
• Enables cloning of vector indexes in databases. {{<issue 26872>}}
• Enables consistent backup and restore for vector indexes. {{<issue 26965>}}
• Speeds up ExternalMiniCluster tests by directly triggering master elections. {{<issue 26193>}}
• Deprecates the load_balancer_count_move_as_add flag to simplify cluster balancing. {{<issue 26259>}}
• Removes master_replication from master_fwd.h to optimize file parsing times. {{<issue 26543>}}

CDC

• Enhances CDC streaming by advancing restart time in idle periods, supported by the new flag cdcsdk_update_restart_time_interval_secs. {{<issue 25562>}}
• Reduces logging frequency for certain CDC errors to avoid clutter. {{<issue 26148>}}
• Sets wal_status in pg_replication_slots based on CDC consumption timing. {{<issue 26272>}}
• Corrects flag value conversion to ensure accurate update intervals for CDC restart times. {{<issue 25562>}}
• Blocks table drops if they are part of a publication to prevent replication issues. {{<issue 26659>}}
• Reduces the default yb_walsender_poll_sleep_duration_empty_ms flag value to 10 ms to speed up replication in sparse workloads. {{<issue 26733>}}
• Increases log visibility for netty errors by changing levels from DEBUG to WARN. {{<issue 26779>}}

yugabyted

• Removes psutil dependency in yugabyted for better compatibility. {{<issue 26350>}}

Bug fixes

YSQL

• Reduces XID usage by generating one per REFRESH MATERIALIZED VIEW CONCURRENTLY operation. {{<issue 26205>}}
• Renames on unique constraints now update associated DocDB table names. {{<issue 26276>}}
• Reduces read restart errors during concurrent disjoint key writes. {{<issue 25214>}}
• Avoids unnecessary catalog version bumps during in-place materialized view refreshes. {{<issue 26154>}}
• Disables index-only scans on copartitioned indexes. {{<issue 26344>}}
• Introduces custom SQL error codes for better error handling across processes. {{<issue 22353>}}
• Fixes crashes when using yb_get_range_split_clause with partitioned tables. {{<issue 25104>}}
• Fixes incorrect error message related to "INSERT ON CONFLICT" under concurrent transactions. {{<issue 25105>}}
• Corrects batched read behavior for mixed immediate and deferred FK constraints. {{<issue 25431>}}
• Reduces latency after DDL changes by using catalog version for cache invalidation. {{<issue 25799>}}
• Refines cost model tuning using server-side execution times for more accurate query optimization. {{<issue 23709>}}
• Removes redundant yb_cdc_snapshot_read_time field, simplifying snapshot management. {{<issue 26303>}}
• Enables geolocation costing in the new cost model using the yb_enable_geolocation_costing flag. {{<issue 25967>}}
• Fixes flaky behavior in Connection Manager when handling prepared statements. {{<issue 25958>}}
• Disables fast-path transactions for bulk loads on colocated tables by default. {{<issue 26191>}}
• Refactors the FK cache handling in YSQL for cleaner code structure. {{<issue 25431>}}
• Optimizes cost modeling for primary index scans to assume sequential disk block fetching. {{<issue 26235>}}
• Ensures accurate detection of duplicate entries during fast-path transactions. {{<issue 26204>}}
• Enables setting follower reads YSQL parameters at connection time. {{<issue 25635>}}
• Resolves multiple issues in tuple-to-string utility functions. {{<issue 26288>}}
• Restore support for yb_terminated_queries() for PG15. {{<issue 26081>}}
• Ensures stable operation of refresh materialized view during major upgrades. {{<issue 26297>}}
• Uses auto-generated OID symbols for pg_proc entries. {{<issue 26301>}}
• Displays the initdb log file path on stdout for easier debugging. {{<issue 26316>}}
• Ensures consistent data during fast-path COPY operations on tables with unique indexes. {{<issue 26207>}}
• Organizes tests into separate files for better clarity and maintenance. {{<issue 25452>}}
• Enhances query planning for inherited and partitioned tables with more efficient path usage. {{<issue 25988>}}
• Ensures PostgreSQL compilation only executes necessary tasks by correctly handling MAKELEVEL. {{<issue 26337>}}
• Prevents database crashes by blocking index creation on dimensionless vector columns. {{<issue 26345>}}
• Fixes upgrade issues for partitioned tables by reverting relam settings. {{<issue 26317>}}
• Prevents crash by excluding NULL values from vector indices. {{<issue 26378>}}
• Enhances index scans and partition pruning for BOOLEAN conditions. {{<issue 26266>}}
• Ensures correct behavior of YbBitmapIndexScan upon rescan by updating pushdown expressions. {{<issue 26210>}}
• Eliminates erroneous colocation data in indexdef for copartitioned indexes. {{<issue 26398>}}
• Adds unit test to handle SELECT errors in incremental cache refresh. {{<issue 26399>}}
• Fixes regression bug in handling incremental cache refresh across concurrent DDL sessions. {{<issue 26399>}}
• Restores and repositions a critical statement to its intended location in the planner. {{<issue 26363>}}
• Enables selective whole row retrieval for DELETE on partitioned tables with varying configurations. {{<issue 26464>}}
• Corrects estimations for inner table scans in Batched Nested Loop Joins. {{<issue 26462>}}
• Fixes integer overflow in BNL cost estimation, preventing negative values. {{<issue 26463>}}
• Prevents incorrect sharing of query limits in subplan executions. {{<issue 26485>}},{{<issue 26418>}}
• Adds a YSQL configuration parameter to customize negative catalog caching. {{<issue 26311>}},{{<issue 26358>}}
• Ensures the vmodule flag is respected by the postgres process. {{<issue 26521>}}
• Adds liblz4.1.dylib to macOS yugabyte-client package for successful deployment. {{<issue 26523>}}
• Enables ANALYZE to collect accurate stats for parent-level of partitioned tables. {{<issue 23592>}}
• Prevents crashes by handling non-variable expressions in single-row updates or deletes. {{<issue 26536>}}
• Adds a safeguard to prevent crashes during NULL vector index scans. {{<issue 26382>}}
• Enhances stability by initializing vector index scan costs to prevent undefined behavior. {{<issue 26588>}}
• Prevents relcache reference leaks in yb_get_ybctid_width. {{<issue 24819>}}
• Fixes port conflict issue when setting pgsql_proxy_bind_address in dual NIC setups. {{<issue 26443>}}
• Addresses "Duplicate table" error by ensuring unique OID allocation during restores. {{<issue 26456>}}
• Ensures YSQL dumps set colocated = false for non-colocated databases during backups. {{<issue 26569>}}
• Reduces default RPC message size limit for better data handling. {{<issue 26375>}}
• Enhances yb_index_check to verify materialized view indexes' consistency. {{<issue 26639>}}
• Ensures ysql_dump maintains enum sort order during backup and restore. {{<issue 26645>}}
• Ensures accurate data return from index scans by correctly fetching needed values for rechecks. {{<issue 26717>}}
• Ensures path->rows reflects accurate row estimates in scans, avoiding incorrect overwrites. {{<issue 25710>}}
• Prevents "Duplicate table" errors by not computing relfilenode_htab during initdb. {{<issue 26456>}}
• Switches from now to clock_timestamp for recording invalidation message time. {{<issue 26789>}}
• Ignores the SPLIT option when creating a partitioned table. {{<issue 26807>}}
• Renames YSQL metric prefixes for clarity and maintains compatibility with old names. {{<issue 26854>}}
• Updates description for ysql_yb_enable_ash flag. {{<issue 26838>}}
• Allows restoration of old backups with enum types without errors by reverting to warnings and auto-generated OIDs. {{<issue 26859>}}
• Logs odd pg_enum OID during restore if sortorder is missing, enhancing troubleshooting. {{<issue 26859>}}
• Restores the call to ScheduleCheckObjectIdAllocators inadvertently removed. {{<issue 26902>}}
• Fixes a use-after-free bug in ysql_dump by copying tablegroup_name. {{<issue 26547>}}
• Allows yb_binary_restore to be set by ybdbadmin for vector extension creation. {{<issue 26941>}}

DocDB

• Resolves issue where tables could get indefinitely stuck in HIDING state. {{<issue 22159>}}
• Prevents creation of tablespaces with duplicate placement blocks. {{<issue 23406>}}
• Prevents crashes by ensuring non-null frontiers during transaction apply after a TServer restart. {{<issue 396>}}
• Fixes load balancing for rewritten tables in colocated databases. {{<issue 26262>}}
• Prevents deadlocks in PG sessions when using shared memory, enhancing stability. {{<issue 26672>}}
• Fixes crashes in ProcessSupervisor when unable to restart a process. {{<issue 26731>}}
• Ensures yb-admin commands respect user-specified timeouts for table flushes and compactions. {{<issue 19957>}}
• Enhances transactional xCluster accuracy by using majority replicated OpId for ApplySafeTime calculations. {{<issue 26043>}}
• Ensures accurate WaitForReplicationDrain behavior by not mislabeling tablets as drained. {{<issue 25457>}}
• Enables cloning databases to any time in the snapshot schedule retention period. {{<issue 26293>}}
• Fixes handling of db_max_flushing_bytes to properly limit memory usage under high write loads. {{<issue 26916>}}
• Prevents unbounded growth of the recently_applied_map by not adding read-only transactions, conserving memory. {{<issue 26666>}}
• Enables cloning databases with sequences to earlier states without errors. {{<issue 25929>}}
• Prevents false conflict detection in snapshot isolation operations. {{<issue 26142>}}
• Fixes lock order for vector index creation to prevent deadlocks. {{<issue 26119>}}
• Allows xCluster to handle UNKNOWN state TableInfos gracefully. {{<issue 26019>}}
• Disables -Wmisleading-indentation warnings in GCC to prevent increased compile times. {{<issue 26177>}}
• Adjusts shared memory address range to 0x350000000000-0x3f0000000000 to avoid conflicts. {{<issue 26200>}}
• Ensures continuous leader lease revocation by supporting multiple old leader leases. {{<issue 19447>}}
• Prevents potential deadlocks by ensuring table locks follow ID order during namespace copies. {{<issue 26164>}}
• Separates thread pools for vector index backfill and inserts to avoid deadlocks. {{<issue 26343>}}
• Escapes non-printable characters in UI and file outputs. {{<issue 26413>}}
• Fixes logging of partition keys for new child tablets post-split. {{<issue 26409>}}
• Fixes flush failure reporting during shutdown to prevent data loss. {{<issue 26468>}}
• Ensures accurate data tracking during master leader transitions by handling chunked tablet reports efficiently. {{<issue 26554>}}
• Ensures system tables are correctly removed during deletions or upgrades. {{<issue 26595>}}
• Reduces thread contention by using a lock-free queue for thread management. {{<issue 26635>}}
• Enhances local RPC call handling to execute in the same thread when possible. {{<issue 26697>}}
• Stops tracking transactions when use_bootstrap_intent_ht_filter is set to false, preventing memory leaks. {{<issue 26860>}}
• Ensures Slice::ToDebugString respects the max_len setting for hex outputs. {{<issue 26874>}}
• Prevents yb-admin crashes by correctly handling argument count for create_database_snapshot. {{<issue 26899>}}
• Improves error handling for shared memory operations in DocDB. {{<issue 26740>}}
• Removes 60-second timeout upper bound on admin RPCs and adds new yb_client_admin_rpc_timeout_sec flag. {{<issue 26722>}}
• Prevents deadlocks during background compaction and transaction loading. {{<issue 26880>}}
• Fixes the issue of recording "query id 0" in Active Session History samples. {{<issue 26808>}}
• Blocks nonconcurrent index creation on xCluster replicated tables. {{<issue 27006>}}
• Prevents master process crashes by fixing an iteration modification issue in TriggerDdlVerificationIfNeeded. {{<issue 26967>}}
• Reverts non-printable character handling to fix tests and API scraping issues. {{<issue 26913>}}
• Prevents crashes during vector index flush on shutdown. {{<issue 26486>}}

CDC

• Ensures only relevant COMMIT records are streamed, fixing gRPC connector crashes. {{<issue 26861>}}
• Prevents CDC crashes by logging a warning for dropped indexes on colocated tables. {{<issue 26834>}}
• Prevents data loss by not streaming records during transaction load. {{<issue 26835>}}
• Ensures reliable CDC stream functionality during index creation, preventing schema packing errors. {{<issue 26891>}}

yugabyted

• Checks for chrony service before enabling clockbound during node start. {{<issue 26255>}}
• Preserves the universe key locally after enabling EAR for recovery scenarios. {{<issue 26552>}}

v2.25.1.0 - March 21, 2025 {#v2.25.1.0}

Build: 2.25.1.0-b381

Downloads

Docker:

docker pull yugabytedb/yugabyte:2.25.1.0-b381

Highlights

We're excited to announce the Technical Preview of the following powerful new features in YugabyteDB aimed at simplifying operations, enhancing functionality, and improving performance.

Automatic DDL Replication {{<tags/feature/tp idea="153">}}

xCluster replication currently requires manual execution of DDL statements on both the source and target clusters to ensure schema consistency.

Automatic DDL Replication eliminates this complexity by seamlessly replicating YSQL DDL changes from the source to the target without requiring manual intervention. This ensures:

• Effortless schema synchronization across clusters.
• Continuous replication without pauses due to schema mismatches.
• Improved reliability by handling DDL execution automatically.

By automating DDL replication, this feature simplifies database management, reduces operational overhead, and enhances the resilience of xCluster replication.

Advisory locks {{<tags/feature/tp idea="812">}}

YugabyteDB's latest release introduces support for PostgreSQL advisory locks, enabling efficient application-level locking mechanisms. These locks allow developers to coordinate concurrent processes without blocking transactions, improving workload management and resource synchronization in distributed environments. For more information on PostgreSQL advisory locks, refer to Advisory locks in the PostgreSQL documentation.

Time-travel Queries {{<tags/feature/tp idea="1110">}}

YugabyteDB now supports time-travel queries, allowing users to run queries as if they were executed at a specified point in the past. This capability enables historical data analysis, auditing, and debugging, without requiring separate backups or snapshots, enhancing data visibility in a running database.

PgVector HNSW Index Support {{<tags/feature/tp idea="1111">}}

YugabyteDB now supports HNSW (Hierarchical Navigable Small World) indexes via the pgvector extension, enabling efficient approximate nearest neighbor (ANN) search for high-dimensional data. This addition expands YugabyteDB's vector indexing capabilities, providing another option alongside existing indexing methods for vector similarity search.

Upgrade from PostgreSQL 11 to PostgreSQL 15 {{<tags/feature/tp idea="1746">}}

YugabyteDB now supports in-place zero downtime upgrade from a PostgreSQL 11-compatible (v2024.2.2) release to a PostgreSQL 15-compatible (v2.25.1) release. This capability allows PostgreSQL major version upgrades without any downtime or interruption to the application.

New features

• PostgreSQL Anonymizer extension, which masks or replaces personally identifiable information (PII) or commercially sensitive data, is now bundled with YugabyteDB. {{<tags/feature/tp idea="1497">}} <!-- IDEA-1497 -->

• YugabyteDB now supports the ability to compute multivariate statistics using the CREATE STATISTICS command. Part of PostgreSQL 15, this feature allows the query planner to use cross-column correlation statistics to generate better query plans. {{<tags/feature/tp>}} {{<issue 13598>}}

Change log

Improvements

YSQL

• Enhances initdb logging, correctly handles process termination, and fixes memory leaks on abort signals. {{<issue 24813>}}
• Ensures cached plans correctly depend on user roles, addressing {{<cve "CVE-2024-10976">}}. {{<issue 25330>}}
• Adds flag ysql_conn_mgr_sequence_support_mode for sequence management options. {{<issue 23660>}}
• Removes the enable_ysql_conn_mgr flag from preview status. {{<issue 25578>}}
• Allows setting enable_ysql_conn_mgr to true for general availability. {{<issue 25578>}}
• Enhances error transparency when dropping databases or users. {{<issue 21438>}}
• Simplifies handling of ybctids from multiple sources in PgDml. {{<issue 25162>}}
• Enables faster LZ4 compression for large attributes in YSQL with upgraded third party support. {{<issue 24290>}}
• Enhances INSERT ON CONFLICT by introducing efficient read batching and adding RETURNING support. {{<issue 24648>}}
• Adds ybctid as a reserved, fully queryable system column for improved row identification. {{<issue 1284>}}
• Simplifies index requests for copartitioned vector indexes, enhancing efficiency. {{<issue 25047>}}
• Fixes segmentation fault in CREATE OR REPLACE TRIGGER by ensuring ybctid is set. {{<issue 24941>}}
• Simplifies DML read pushdown binding for enhanced coding efficiency. {{<issue 25151>}}
• Ensures ybcModifyTable.h meets style guidelines to pass arc lint checks. {{<issue 25172>}}
• Adds new lint rules for better PostgreSQL code syntax checking {{<issue 25187>}}
• Resolves style issues in yb_scan.c and yb_scan.h, addressing lint errors and warnings. {{<issue 25198>}}
• Enhances YSQL code consistency with additional linting rules. {{<issue 25298>}}
• Adds line breaks for each command in pipelines for better readability. {{<issue 25297>}}
• Reduces CPU usage by optimizing ExecLockRows function. {{<issue 25268>}}
• Adds a flag ysql_block_dangerous_roles to prevent privilege escalation. {{<issue 25348>}}
• Adds new lint rules for comment styles in YSQL code. {{<issue 25360>}}
• Removes outdated comments that are no longer applicable in the current codebase. {{<issue 25400>}}
• Ensures YSQL code follows leading whitespace rules and blacklists non-compliant files. {{<issue 25375>}}
• Streamlines initialization of YsqlAdvisoryLocksTable using std::shared_future. {{<issue 25373>}}
• Removes unused parameters from YBInitPostgresBackend for cleaner code. {{<issue 25417>}}
• Simplifies ysql_dump by removing outdated references to pg_tablegroup. {{<issue 25419>}}
• Adds a helper class to manage YsqlCatalogConfig updates efficiently. {{<issue 25418>}}
• Enhances code quality in pg_yb_utils.c by applying lint warnings/errors. {{<issue 25420>}}
• Enhances INSERT ON CONFLICT performance by decoupling batch processing. {{<issue 25488>}}
• Ensures trailing whitespace issues are consistently addressed in the codebase. {{<issue 25438>}}
• Allows setting yb_insert_on_conflict_read_batch_size to zero to disable batching, aligning closer with PostgreSQL behavior. {{<issue 25489>}}
• Adjusts ysqlsh -V output for accurate command name representation. {{<issue 25429>}}
• Resolves code style issues by fixing spacing after control statements. {{<issue 25447>}}
• Streamlines code clarity in YSQL by renaming and conditional updates. {{<issue 25446>}}
• Streamlines datatype mapping by consolidating multiple code blocks into PgTypeInfo. {{<issue 25554>}}
• Enforces new lint rule bad_opening_paren for cleaner code alignment. {{<issue 25455>}}
• Enhances YSQL logging with new and updated linter rules to reduce errors and increase clarity. {{<issue 25482>}}
• Enforces new spacing rules around comments in YSQL code. {{<issue 25487>}}
• Re-enables the check_for_tables_with_oids for consistency with PostgreSQL. {{<issue 25498>}}
• Standardizes type naming across YSQL to eliminate inconsistencies. {{<issue 25587>}}
• Enables batching for INSERT ON CONFLICT with foreign key triggers. {{<issue 24649>}}
• Consolidates YSQL configuration parameter management for clarity and consistency. {{<issue 25586>}}
• Removes unnecessary needs_pg_session_transaction field to streamline advisory lock handling. {{<issue 25591>}}
• Adds "yb" prefix to custom YB type names for clarity and consistency. {{<issue 25631>}}
• Enables lz4 compression by default for large YSQL tuples, ensuring high performance. {{<issue 25613>}}
• Reduces PgClientSession's size and streamlines code for better readability and easier modifications. {{<issue 25652>}}
• Enhances YSQL linter scripts to handle Mac compatibility and prevent failures. {{<issue 25662>}}
• Simplifies running pgindent for cleaner code formatting and resolving issues with function names and comments. {{<issue 25663>}}
• Removes unnecessary sidecar usage from YBPgsqlLockOp for cleaner code. {{<issue 25855>}}
• Enhances thread safety by isolating client_ field storage in PgClientSession. {{<issue 25652>}}
• Enhances code readability by running yb_pgindent and manual formatting. {{<issue 25725>}}
• Enables INSERT ON CONFLICT read batching by default to increase test coverage. {{<issue 25985>}}
• Enhances lint rule messages for clarity and stricter comment spacing validation. {{<issue 25816>}}
• Refactors PgClientSession using the pimpl design pattern for better organization and cleanup. {{<issue 25847>}}
• Enhances YSQL index backfill tests to detect and verify index consistency issues more effectively. {{<issue 25854>}}
• Allows customizing ef_construction and m parameters for ybhnsw indexes during creation. {{<issue 25859>}}
• Reduces code duplication in query optimization checks for colocated/copartitioned tables. {{<issue 25959>}}
• Restores the ItemPointer static assert following upstream PostgreSQL code. {{<issue 25997>}}
• Fixes a TSAN issue by adjusting error reports in YugabyteDB. {{<issue 25998>}}
• Enhances code style in src/backend/tcop/ as per linting reports. {{<issue 25189>}}
• Adds query ID and leader PID to the /rpcz endpoint response. {{<issue 25603>}}

YCQL

• Adds keyspace field to <tserver-ip>:12000/statements output. {{<issue 25368>}}

DocDB

• Enables xCluster to support and replicate table rewrite DDL operations effectively. {{<issue 23955>}}
• Enables the creation of indexable vector columns in DocDB by introducing a new DataType Vector. {{<issue 24483>}}
• Reduces Prometheus metric scrape time from 15 seconds to 2 seconds. {{<issue 24565>}}
• Enables building with third party PR artifacts using specific GitHub IDs. {{<issue 25089>}}
• Enables file determination at compaction start by default. {{<issue 25113>}}
• Enables merging vector index search results across multiple tablets. {{<issue 25212>}}
• Enhances stability by fixing a missing null check in tablet lookup processes. {{<issue 25145>}}
• Reduces CPU usage by 3-8% during statistics collection. {{<issue 25160>}}
• Enables DDL replication for common PostgreSQL extensions in YugabyteDB. {{<issue 25052>}}
• Clarifies error messages during yb-master initialization. {{<issue 25176>}}
• Reduces CPU usage by only setting stmt_max_mem_base_bytes during EXPLAIN ANALYZE. {{<issue 25174>}}
• Reduces TServer CPU usage by using serialized_request size instead of SpaceUsedLong. {{<issue 25175>}}
• Enables monitoring of long tasks in Reactor threads with rpc_reactor_task_timeout_ms. {{<issue 25180>}}
• Speeds up database creation by batching table lock acquisitions. {{<issue 25203>}}
• Switches to GetTableSchema to check table existence, reducing timeouts. {{<issue 25197>}}
• Enables explicit release of advisory locks in transactions. {{<issue 25195>}}
• Enables re-creation of vector indexes on tablet restarts. {{<issue 25272>}}
• Speeds up certain tests by 1-2 minutes by halting retryable RPC calls during shutdown. {{<issue 25252>}}
• Disables Redis proxy by default to reduce overhead. {{<issue 25278>}}
• Optimizes RWCLock for faster read operations. {{<issue 25282>}}
• Disables tablet splitting for tables with vector indexes. {{<issue 25288>}}
• Removes unnecessary libraries from YB tools, reducing link times and flag count. {{<issue 25304>}}
• Adds automatic_ddl_mode argument to yb-admin create_xcluster_checkpoint. {{<issue 25302>}}
• Enables TSAN for RWCLock, fixing potential deadlocks. {{<issue 25391>}}
• Refactors lock management code to use static dispatch for compatibility with shared memory. {{<issue 25322>}}
• Ensures vector indexes are updated during tablet bootstrap if the TServer stops unexpectedly. {{<issue 25325>}}
• Enhances search by filtering vectors on TServer based on deletion, updates, and criteria. {{<issue 25357>}}
• Enables remote bootstrap for vector index storage. {{<issue 25369>}}
• Reduces the need for global mutex locks by introducing per-transaction LockTracker state. {{<issue 25379>}}
• Ensures DDL unlock operations follow the reverse order of lock operations. {{<issue 25380>}}
• Enables point-in-time restore support for vector indexes. {{<issue 25389>}}
• Adds flag rocksdb_iterator_sequential_disk_reads_factor to enhance readahead during non-sequential scans. {{<issue 25397>}}
• Reduces boilerplate for defining hash functions with new helper macros. {{<issue 25403>}}
• Ensures xCluster safe time service remains operational when the master loses lease. {{<issue 25505>}}
• Removes unnecessary //NOLINT comments to enhance code readability. {{<issue 25536>}}
• Replaces heuristic checks with VECTOR data type for column identification. {{<issue 25581>}}
• Speeds up TServer insert times by enabling variable bloom filtering on RocksDB iterators. {{<issue 25600>}}
• Speeds up write operations by using ScopedStatistics, reducing TServer insert time. {{<issue 25602>}}
• Enables backfilling existing table data when creating vector indexes. {{<issue 25676>}}
• Removes unnecessary key-value callbacks from vector LSM. {{<issue 25687>}}
• Allows the creation of colocated tables in xCluster configurations. {{<issue 22318>}}
• Sets the minimum xCluster safe time lag value to 0 and adds logging for debugging. {{<issue 25733>}}
• Preserves enum label OIDs during DDL replication across universes. {{<issue 24077>}}
• Enables YCQL support for GetTablesEligibleForXClusterReplication. {{<issue 25732>}}
• Supports custom vector labels in HNSW library for enhanced indexing flexibility. {{<issue 25041>}}
• Enables use of variable bloom filter during index scans for enhanced query performance. {{<issue 25795>}}
• Enables logging of YSQL write RPCs with the print_ysql_write_requests flag. {{<issue 25786>}}
• Reduces compile time for transaction_participant.cc by removing unused boost includes. {{<issue 25792>}}
• Simplifies PgSamplePicker by removing its dependency on PgSelectIndex. {{<issue 25858>}}
• Supports multiple vector distance functions in ybhnsw index access method. {{<issue 25807>}}
• Enables chunked vector index backfill and continuation after restart. {{<issue 25844>}}
• Removes unused ScanForward functionality for streamlined performance. {{<issue 25915>}}
• Ensures correct classification of temporary objects in xCluster automatic mode. {{<issue 25885>}}
• Enables better replication of concurrent DDLs with unique, ordered timestamps. {{<issue 24235>}}
• Enhances the user message for setting yb_read_time to clarify read-only restrictions. {{<issue 26027>}}
• Enables sequence replication in xCluster by default, removing the need for a flag. {{<issue 26029>}}
• Adds logging for vector index search stats when vector_index_dump_stats flag is true. {{<issue 26072>}}
• Ensures vector index backfill reads from the indexed table at the correct time. {{<issue 26046>}}
• Enhances handling of expired snapshots by retrying deletion tasks automatically. {{<issue 25628>}}
• Ensures consistent bootstrapping of vector indexes after a TServer restart. {{<issue 26087>}}
• Changes xcluster_enable_ddl_replication to a preview flag for user testing. {{<issue 26224>}}
• Adds safeguard to pause replication after repeated DDL failures. {{<issue 26093>}}
• Adds automatic_mode flag to create_checkpoint for simpler xCluster setup. {{<issue 26355>}}
• Streamlines tablespace validation by centralizing logic, enhancing future feature support. {{<issue 25202>}}
• Enhances maintainability by consolidating tablespace validation logic into TablespaceParser. {{<issue 25202>}}

CDC

• Blocks creation of IMPLICIT streams by default; use flag to override. {{<issue 24023>}}
• Enhances CDC accuracy by using the minimum of last WAL OP timestamp and transaction start time in segment footers. {{<issue 25163>}}
• Disables automatic tablet splitting for CDC stream tables by default. {{<issue 25148>}}
• Removes unused code related to transaction status checks in CDC. {{<issue 25351>}}
• Enables dynamic table addition in logical replication by default. {{<issue 25413>}}
• Increases default CDC retention flags to 8 hours for better data durability. {{<issue 25633>}}
• Enables automatic tablet splitting with logical replication by fixing hidden tablet deletion bugs. {{<issue 24918>}}
• Reduces unnecessary cleanup requests for logically replicated tables. {{<issue 25752>}}
• Adds parallel logical replication with consistent tablet subsetting under the ysql_yb_enable_consistent_replication_from_hash_range flag. {{<issue 25897>}}
• Reduces cdcsdk_publication_list_refresh_interval_secs to 15 minutes by default. {{<issue 25793>}}
• Adds cdcsdk_flush_lag metric and fixes bugs in other CDC metrics. {{<issue 19445>}},{{<issue 25819>}}
• Limits the number of VirtualWAL instances per TServer with cdc_max_virtual_wal_per_tserver flag. {{<issue 25896>}}
• Adds the cdc_cache_intent_block flag to control caching of CDC intent blocks. {{<issue 24909>}}
• Renames the flag to cdc_enable_caching_db_block and sets its default to true for enhanced caching. {{<issue 25573>}}
• Limits CDC GetConsistentChanges response by byte size, allowing toggling with cdc_use_byte_threshold_for_vwal_changes. {{<issue 25572>}}
• Reduces log verbosity in virtual WAL to prevent large log files. {{<issue 25739>}}
• Enables tests for optimized WAL reads in CDC for TSAN builds. {{<issue 24374>}}

yugabyted

• Removes psutil dependency in yugabyted for better compatibility. {{<issue 26350>}}
• Increases row count capacity in metrics API by using int64 instead of int32. {{<issue 25196>}}
• Fixes yb-voyager install instructions layout in yugabyted UI. {{<issue 25445>}}
• Displays universe UUID instead of a blank in the "Cluster Name" field on the yugabyted-ui settings page. {{<issue 25831>}}

Bug fixes

YSQL

• Reduces memory usage during YugabyteDB connection startup. {{<issue 24925>}}
• Removes an erroneous log entry that appeared in crashes or errors. {{<issue 24533>}}
• Enhances protocol flow for consistency in batched query executions using a new flag ysql_conn_mgr_optimized_extended_query_protocol. {{<issue 24898>}}
• Ensures Batched Nested Loop joins correctly check for hashable join clauses. {{<issue 25917>}}
• Enables creating non-colocated leaf partitions on colocated partitioned tables. {{<issue 24542>}}
• Enables pg_authid table prefetching during login to reduce RPCs and prevent errors. {{<issue 25776>}}
• Standardizes function usage for ATExecSetTableSpaceNoStorage to match PG's approach. {{<issue 25019>}}
• Removes an unnecessary code block to streamline backend selection in connection management. {{<issue 25154>}}
• Corrects tuple ID construction for primary keys with out-of-order columns. {{<issue 25070>}}
• Restores functionality by removing explicit packet parsing for yb_is_client_ysqlconnmgr. {{<issue 25220>}}
• Enables seamless execution of unnamed prepared statements across different backends. {{<issue 25577>}}
• Fixes data race in regex pushdown by using thread-local variables. {{<issue 24237>}}
• Ensures correct row order for queries with IN clauses. {{<issue 25559>}}
• Tracks sticky connection metrics on the Prometheus endpoint. {{<issue 25775>}}
• Eliminates segmentation faults caused by a bug in Odyssey's list iteration macro, which incorrectly handled NULL pointers and led to core dumps. {{<issue 25846>}}
• Disables index-only scans on copartitioned indexes. {{<issue 26344>}}
• Fixes crash by skipping internal savepoints for SET/BEGIN in RC isolation. {{<issue 23730>}}
• Automatically restarts the postmaster to prevent stuck processes when a background worker crashes. {{<issue 24706>}}
• Enhances accuracy of seek-next estimation for index scans and paged results. {{<issue 22087>}}
• Fixes memory leak in ysql_dump by clearing PG result in all scenarios. {{<issue 24934>}}
• Fixes issues with INSERT ON CONFLICT DO UPDATE to prevent secondary index corruption. {{<issue 25075>}}
• Clears session state on retrying a query to avoid errors and conflicts. {{<issue 25105>}}
• Eliminates background worker restart loops and enhances error handling. {{<issue 25192>}}
• Fixes "Duplicate table" error by adjusting OID allocation to prevent collisions. {{<issue 24320>}}
• Replaces fatal calls with pg_fatal in pg_dump.c for better error handling. {{<issue 25320>}}
• Allows Batch Nested Loop (BNL) joins on partitioned tables in outer joins. {{<issue 25080>}},{{<issue 25078>}}
• Ensures correct handling of INSERT ... ON CONFLICT DO NOTHING with multiple arbiter indexes. {{<issue 25240>}}
• Fixes a missing return statement in MasterTabletServiceImpl::Write. {{<issue 25249>}}
• Ensures all non-fast path UPDATEs correctly fetch the entire row to prevent foreign key violations. {{<issue 24701>}}
• Fixes inconsistency in index entries during concurrent writes in the backfill phase. {{<issue 25250>}}
• Corrects initialization of primaryKeyIndex in ysql_dump. {{<issue 25340>}}
• Removes a faulty assertion in INSERT ON CONFLICT to prevent failures. {{<issue 25296>}}
• Addresses a pending review comment for YSQL style corrections. {{<issue 25385>}}
• Adds a tablet state column to yb_local_tablets for clarity on tablet statuses. {{<issue 24250>}}
• Enhances session management by moving set_force_catalog_modifications handling to PgSession. {{<issue 25434>}}
• Reduces scope of ALTER DATABASE impact to only the targeted database. {{<issue 25392>}}
• Enhances code structuring and formatting for query diagnostics files. {{<issue 25058>}}
• Ensures correct column mapping for partitioned tables during INSERT ON CONFLICT updates. {{<issue 14895>}},{{<issue 25521>}}
• Fixes a compile error for GCC builds. {{<issue 25658>}}
• Fixes incorrect role name quoting in ysql_dumpall. {{<issue 25701>}}
• Ensures DDL operations continue smoothly by retrying background tasks on polling errors. {{<issue 25708>}}
• Corrects a bug in YugabyteDB where using yb_enable_expression_pushdown on aggregates in trivial subqueries with OFFSET 0 returned values from incorrect columns. {{<issue 24512>}}
• Modifies ysql_dumpall to combine CREATE ROLE and ALTER ROLE for effective role management. {{<issue 25608>}}
• Fixes an error in UPDATE queries setting the primary key to itself. {{<issue 21160>}}
• Fixes issues with INSERT ON CONFLICT queries using SPI_execute_with_args. {{<issue 25773>}}
• Reverts ALTER DATABASE RENAME and ALTER DATABASE OWNER to global impact. {{<issue 25742>}}
• Fixes memory leaks and segmentation faults in YSQL Connection Manager. {{<issue 25718>}}
• Enables testing of SECURITY LABEL with dummy_seclabel. {{<issue 25805>}}
• Fixes memory leaks in the PostgreSQL Anonymizer extension and optimizes test scheduling. {{<issue 25928>}}
• Fixes potential negative cost calculations for large table scans by using double for estimates. {{<issue 25862>}}
• Fixes \d command for indexes with spaces in their names. {{<issue 25711>}}
• Prevents returning corrupt data for INSERT ON CONFLICT with RETURNING when read batching is enabled. {{<issue 25836>}}
• Ensures CREATE OR REPLACE VIEW refreshes the catalog for immediate update visibility. {{<issue 25901>}}
• Enhances linter hints for Mac users by prioritizing brew over apt. {{<issue 25925>}}
• Sets query ID for background workers earlier to ensure correct tracking. {{<issue 25952>}}
• Enhances estimation of result tuple size in edge cases, preventing division by zero errors. {{<issue 21828>}}
• Prevents database crashes by blocking index creation on dimensionless vector columns. {{<issue 26345>}}
• Fixes upgrade issues for partitioned tables by reverting relam settings. {{<issue 26317>}}
• Eliminates erroneous colocation data in indexdef for copartitioned indexes. {{<issue 26398>}}

YCQL

• Allows users to control error suppression for GROUP BY in YCQL using ycql_ignore_group_by_error flag. {{<issue 25030>}}

DocDB

• Prevents crashes when calling ListMasterServers RPC on a master by returning an error. {{<issue 20372>}}
• Ensures continuous xCluster task operation even if the master temporarily loses the lease but retains leadership. {{<issue 25480>}}
• Enhances transactional xCluster accuracy by using majority replicated OpId for ApplySafeTime calculations. {{<issue 26043>}}
• Prevents server deadlocks by skipping Finished method during certain task aborts. {{<issue 23356>}}
• Ensures consistency between Slice::Less and Slice::compare methods. {{<issue 25319>}}
• Prevents crash during transaction abort by adjusting lock handling. {{<issue 25689>}}
• Ensures accurate WaitForReplicationDrain behavior by not mislabeling tablets as drained. {{<issue 25457>}}
• Enhances vector index merging for better search efficiency. {{<issue 24069>}}
• Clarifies the use of "IncludeInactive" and "IncludeHidden" flags in Master RPCs. {{<issue 25086>}}
• Enables direct mapping from row keys to vector IDs in vector indexes. {{<issue 24232>}}
• Removes a problematic import to prevent errors during direct script executions in GitHub CI. {{<issue 25155>}}
• Fixes crashes and errors during vector index creation on non-first column IDs. {{<issue 25276>}}
• Enhances search efficiency by enabling the Merge function to take vector indexes as input for compactions. {{<issue 24069>}}
• Fixes crashes when copying large rows with a vector index. {{<issue 25335>}}
• Updates Snappy to version 1.1.8.4 for better M1 Mac support. {{<issue 25359>}}
• Fixes snapshot deletion during unfinished operations. {{<issue 25405>}}
• Enhances XClusterSafeTimeService by integrating it with the upgraded Poller class. {{<issue 25253>}}
• Changes xCluster metrics from kSum to kMax to prevent inaccuracies. {{<issue 25422>}}
• Eliminates potential crashes when fetching status of sealed transactions. {{<issue 25439>}}
• Ensures vector indexes are flushed before deleting intents SST files to prevent data loss. {{<issue 25454>}}
• Enables block-based sampling by default using new YSQL configuration parameters. {{<issue 25557>}}
• Prevents xCluster failures caused by same-name table re-creation during non-DBScoped DR. {{<issue 25709>}}
• Fixes a deadlock issue during index creation and enhances conflict detection. {{<issue 25737>}}
• Ensures correct database IDs are used for xCluster replication setup. {{<issue 25770>}}
• Disallows write DML operations in time-traveling sessions set with yb_read_time. {{<issue 25834>}}
• Disables pushdown of MD5 and SHA256 to prevent TServer crashes. {{<issue 25889>}}
• Corrects the namespace of TSLocalLockManager to yb::tserver. {{<issue 25941>}}
• Ensures builds with third party PR artifacts now function correctly. {{<issue 25970>}}
• Upgrades tcmalloc and abseil to prevent crashes from malloc_usable_size(nullptr). {{<issue 25948>}}
• Disables the unstable fast backward scan feature by default in 2.25.1. {{<issue 26074>}}
• Allows xCluster to handle UNKNOWN state TableInfos gracefully. {{<issue 26019>}}
• Separates thread pools for vector index backfill and inserts to avoid deadlocks. {{<issue 26343>}}

CDC

• Reduces log frequency for certain CDC warnings to every 10 minutes. {{<issue 23148>}}
• Fixes CDC safe time update to ensure data consistency across tablets. {{<issue 25120>}}
• Ensures accurate history retention for various replica identities in CDC. {{<issue 25193>}}
• Reduces untracked memory growth in CDC by eliminating unnecessary request scopes. {{<issue 25311>}}
• Fixes metric lag by adding dummy entries for expired tablets. {{<issue 25550>}}
• Stops log cache eviction slowdown due to CDC to prevent untracked memory spikes. {{<issue 25311>}}
• Ensures newly created tables in logical replication are retained correctly during cleanup. {{<issue 25206>}}
• Handles missing CDC streams gracefully during polling by logging warnings. {{<issue 25605>}}
• Ensures schema checks during table eligibility evaluations for CDC stream cleanups. {{<issue 25206>}}
• Reduces log verbosity by changing the log level for virtual WAL destroy statements. {{<issue 25547>}}

yugabyted

• Checks for chrony service before enabling clockbound during node start. {{<issue 26255>}}
• Adds use_client_to_server_encryption flag to secure login data. {{<issue 25332>}}

v2.25.0.0 - January 17, 2025 {#v2.25.0.0}

Build: 2.25.0.0-b489

Downloads

Docker:

docker pull yugabytedb/yugabyte:2.25.0.0-b489

Highlights

We're excited to announce the Technical Preview of two foundational features as part of this release.

PostgreSQL 15 Support {{<tags/feature/tp>}}

As part of this release, we have upgraded our PostgreSQL fork from version 11.2 to 15.0, enabling you to leverage the many key capabilities introduced in PostgreSQL between these two versions. This upgrade brings YSQL API support for numerous features, including stored generated columns, foreign keys on partitioned tables, and non-distinct NULLs in unique indexes. It also introduces query execution optimizations like incremental sort and memoization, along with various observability and security enhancements. For more information, refer to PostgreSQL 15 features.

Query Diagnostics {{<tags/feature/tp idea="2078">}}

This feature significantly simplifies tuning poorly performing SQL queries by allowing you to capture and export detailed diagnostic information, including bind variables and constants, pg_stat_statements statistics, schema details, active session history, and execution plans. Refer to Query diagnostics.

Active session history {{<tags/feature/ea idea="830">}}

In addition, the Active Session History, which provides real-time and historical views of system activity, is now enabled by default. For more information, refer to Active Session History.

Change log

Improvements

YSQL

• Adds pg_hint_plan syntax and functionality to control batched nested loop joins, allows setting hints YbBatchedNL(t1 t2) and NoYbBatchedNL, and modifies yb_prefer_bnl handling. Also, it removes BNL's dependency on enable_nestloop and adjusts cost model. {{<issue 19494>}}
• Enables control over in-place index updates with yb_enable_inplace_index_update. {{<issue 20908>}}
• Added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables for yb_db_admin role. {{<issue 23786>}}
• Allows use of UNLOGGED tables syntax, handling it as LOGGED with a warning. {{<issue 23895>}},{{<issue 23889>}}
• Adds pg_yb_logical_client_version table to manage session configurations. {{<issue 23871>}}
• Introduces a dedicated auth-backend for YSQL to handle connection authentication, enhancing security and streamlining the process. {{<issue 24147>}}
• Ensures correct error handling by adding a missing InvalidOid check in YSQL. {{<issue 24147>}}
• Enables authentication via a specialized backend, enhancing security and reliability. {{<issue 24147>}},{{<issue 24427>}}
• Enables toggling authentication method in YSQL using the ysql_conn_mgr_use_auth_backend flag. {{<issue 24147>}}
• Sets yb_bnl_batch_size to 1024 and yb_prefer_bnl to true by default, ensuring BNL's replace nested loop joins without altering non-NL join plans. {{<issue 19273>}}
• Allows preloading of foreign key lists in relcache, avoiding on-demand master fetches, controlled by yb_enable_fkey_catcache. {{<issue 23686>}}
• Adds notes on pinned objects and script backporting to the README. {{<issue 24334>}}
• Prevents catalog version bumps on no-op ALTER ROLE commands, saving system resources. {{<issue 24390>}}
• Enhances object creation efficiency by using static factory methods in PgDML classes. {{<issue 24412>}}
• Disables JIT in YB PG15 by default for stability. {{<issue 24479>}}
• Enables successful major version upgrades of functions containing semicolons. {{<issue 24571>}}
• Introduces GetStatementAs<...> to simplify PgStatement* casts. {{<issue 24754>}}
• Fixes style issues and lint errors in ybcModifyTable.c. {{<issue 24882>}}
• Simplifies ybcSetupTargets by using BMS only, enhancing target column handling. {{<issue 25007>}}
• Excludes third-party extensions from PostgreSQL linter checks. {{<issue 25054>}}
• Enables creation of indexes that are covered by the main table. {{<issue 24123>}}
• Enables ASH by default for better monitoring and diagnostics. {{<issue 24686>}}
• Restores performance hints in prepared statements to ensure consistent execution. {{<issue 24055>}}
• Updates YSQL upgrade tests for compatibility with pg15. {{<issue 24289>}}
• Simplifies descriptions for inherited PostgreSQL wait events. {{<issue 24972>}}
• Enables users to control YB query execution plans using pg_hint_plan. {{<issue 6720>}}
• Adds Oracle-like functions to YSQL with the Orafce extension. {{<issue 3460>}}

YCQL

• Resolves timestamp precision mismatches in YCQL, aligning inserts and index scans to millisecond precision. Enables users to optionally revert to microsecond precision using the flag cql_revert_to_partial_microsecond_support=false. {{<issue 11052>}},{{<issue 23476>}}
• Tracks microsecond precision timestamps in YCQL with new metric cql_microseconds_timestamps_used. This tracking occurs if the cql_revert_to_partial_microsecond_support flag is true. {{<issue 24776>}}

DocDB

• Allows reduction of thread stack size to avoid Linux hugepage backing. {{<issue 23927>}}
• Reduces unnecessary attribute map creations during Prometheus metric aggregation. {{<issue 24405>}}
• Enhances load balancer to prioritize under-replicated tablets first. {{<issue 20263>}}
• Limits concurrent remote bootstraps per TServer using load_balancer_max_inbound_remote_bootstraps_per_tserver flag. {{<issue 2426>}}
• Prevents overloading by correctly categorizing bootstrapping tablets in load balancing. {{<issue 23487>}}
• Upgrades non-FIPS OpenSSL to 3.0.15 and removes CentOS 7 builds. {{<issue 24436>}}
• Enhances RBS throttling by focusing on active fetch sessions and adjusts expiration logic. {{<issue 21563>}},{{<issue 24031>}}
• Enables persisting TServer registry entries to the sys catalog, enhancing failover recovery. {{<issue 22925>}}
• Initial implementation of Vector LSM supports multi-threaded batch inserts. {{<issue 24020>}}
• Updates and organizes third-party dependencies like usearch for better maintainability. {{<issue 23998>}}
• Simplifies HNSW library structure by using a factory-based approach. {{<issue 24085>}}
• Allows explicit addition of flags to gflag_allowlist.txt for secure callhome data collection, plus version_info in TServer data. {{<issue 24103>}}
• Enables xCluster to handle non-colocated ALTER TABLE commands without pausing replication. {{<issue 23951>}}
• Passes automatic_ddl_mode to xCluster pollers for enhanced replication handling. {{<issue 24091>}}
• Adds a metric to alert users when the master follower heartbeat delay is too high. {{<issue 21178>}}
• Removes the need to set set_cdc_min_replicated_index during xCluster bootstrap for improved log retention. {{<issue 24105>}}
• Enables SimSIMD in Usearch for enhanced search operations. {{<issue 23998>}}
• Reduces thread usage by removing CDC-specific yb_client from PgClientServiceImpl. {{<issue 24145>}}
• Adds a simple test to verify Vector LSM functionality during and post-insertions. {{<issue 24242>}}
• Adds a yb-admin command to remove non-active, blacklisted TServers. {{<issue 24044>}}
• Skips schema validation for newly added tables in xCluster replication to handle schema changes. {{<issue 23078>}}
• Standardizes log prefixes for retryable requests with tablet and peer IDs. {{<issue 19516>}}
• Adds a yb-admin command to remove tablet servers from the registry. {{<issue 24044>}}
• Enables persistent TServer registry updates via autoflag configuration. {{<issue 24045>}}
• Enables persistent storage for vector LSM via separate index chunk files and meta files. {{<issue 24371>}}
• Enhances Vector LSM handling to maintain data integrity after unexpected restarts. {{<issue 24466>}}
• Issues NOTICE when creating indexes in bi-directional xCluster to ensure concurrent execution. {{<issue 24362>}}
• Reduces Python virtual environment changes using yugabyte-bash-common for top-level management. {{<issue 19478>}}
• Renames the vector library to vector_index and updates the namespace accordingly. {{<issue 24636>}}
• Enables selecting compaction files at task start with rocksdb_determine_compaction_input_at_start flag. {{<issue 24541>}}
• Creates vector index tablets colocated with the indexed table for effective data retrieval. {{<issue 24696>}}
• Enables direct replication of specific DDL commands in xCluster. {{<issue 23953>}}
• Reverts a change that issued NOTICE on index creation in bi-directional xCluster setups. {{<issue 24362>}}
• Enables creation of Vector LSM for tablets with vector indices. {{<issue 24892>}}
• Reduces argument redundancy in PgsqlReadOperation executions. {{<issue 24855>}}
• Enables safe index creation on bidirectional xCluster tables without halting operations. {{<issue 24884>}}
• Replaces ProducerSchemaPB with SchemaVersionsPB for xCluster configurations. {{<issue 24901>}}
• Supports DDL replication for additional PostgreSQL objects including foreign tables and text search configurations. {{<issue 25051>}}
• Enables ALTER operations on colocated tables with active replication. {{<issue 24910>}}
• Adds packed row support for vector indexes during transaction apply. {{<issue 24912>}}
• Fetches intents DB for vector index queries to account for unapplied committed transactions. {{<issue 24947>}}
• Enables building with third party PR artifacts using specific GitHub IDs. {{<issue 25089>}}
• Enables vector indexes on colocated tables. {{<issue 24994>}}
• Allows vector index inserts to use a dedicated thread pool, enhancing overall system performance. {{<issue 25029>}}
• Replaces vector_index::VertexId with StronglyTypedUuid as VectorId. {{<issue 25038>}}
• Limits concurrent reads on usearch vector index to avoid exceptions. {{<issue 25069>}}
• Enables file determination at compaction start by default. {{<issue 25113>}}
• Enhances stability by fixing a missing null check in tablet lookup processes. {{<issue 25145>}}
• Removes deprecated vector index storage in RocksDB. {{<issue 24854>}}
• Removes outdated code to streamline heartbeat processing. {{<issue 24872>}}
• Simplifies load balancer placement info management. {{<issue 25065>}}
• Updates lz4 library to fix a specific compression test failure. {{<issue 24965>}}
• Increases RollbackYsqlMajorVersionUpgrade RPC timeout to 3 minutes. {{<issue 24361>}}
• Adds race conditions in the odyssey library to the TSAN suppressions list. {{<issue 24735>}}
• Reduces data race when changing RocksDB options during auto-compactions. {{<issue 24238>}}
• Fixes deadlock issue by adjusting lock acquisition order in HierarchicalNSW. {{<issue 25017>}}
• Ensures tasks don't hang during shutdown by removing unnecessary checks. {{<issue 17898>}}
• Rejects alter table requests during table preparation to prevent state inconsistencies. {{<issue 25130>}}

CDC

• Enables asynchronous removal of user tables from CDC streams by adjusting how background threads process and persist stream metadata. {{<issue 23700>}}
• Adds a tag for the slot name attribute in the CDC metrics. {{<issue 24307>}}
• Enables tablet splitting on tables under CDCSDK stream by default using the GFLAG enable_tablet_split_of_cdcsdk_streamed_tables. {{<issue 24190>}}
• Avoids caching IntentDB block reads for CDC to prevent redundant data access. {{<issue 24909>}}
• Automatically removes metrics when a CDC stream is deleted. {{<issue 24876>}}
• Speeds up tablet bootstrap under CDC by skipping initial WAL segments, controlled by skip_wal_replay_from_beginning_with_cdc flag. {{<issue 24516>}}
• Enables kNoCacheQueryId to prevent crashes during RocksDB operations. {{<issue 24516>}}
• Speeds up tablet bootstrap under CDC by fixing WAL replay. {{<issue 24516>}}

yugabyted

• Enables better handling of complex YSQL configuration parameters using sticky connections. {{<issue 21637>}}
• Stops sending flag details in the callhome diagnostics to eliminate redundant data. {{<issue 24029>}}
• Ensures the user interface accurately displays disk size, even when multiple data directories are used. {{<issue 23810>}}
• Ensures join flag in config files is properly validated to prevent node misconfiguration. {{<issue 23007>}}
• Updates column names in pg_stat_statements to fix slow queries page on yugabyted UI. {{<issue 25094>}}
• Enables tserver_master_addresses to update immediately in multi-node deployments using Yb-ts-cli. {{<issue 24659>}}
• Adds fully qualified names for non-table SQL objects in the UI. {{<issue 24330>}},{{<issue 24331>}}
• Lists manually refactorable objects by type in the YugabyteD UI. {{<issue 24329>}}

Bug fixes

YSQL

• Introduces a per-database PG new OID allocator, ensuring OID uniqueness within the database and enhancing horizontal scalability in multi-node and multi-tenancy environments. This new mechanism mitigates OID collisions and allows OID consistency in backup-restore scenarios across clusters. A new GFlag ysql_enable_pg_per_database_oid_allocator is provided to return to old OID allocator behavior if necessary. {{<issue 16130>}}
• Resets current_hint_retrieved after query execution to ensure consistent hint parsing. {{<issue 12741>}}
• Modifies backup/restore process to skip column name checks for indexes, allowing for successful restoration even with renamed columns. {{<issue 24207>}}
• Enables ALTER SEQUENCE commands to change sequence name, schema, and owner without errors. {{<issue 17271>}}
• Enhances PostgreSQL restart stability by refining shutdown handling and lock file management. Includes killing only Postgres-associated PIDs, using SIGQUIT for immediate shutdowns to reduce hang-ups, and monitoring for stuck processes on macOS {{<issue 24396>}}
• Fixes crashes caused by using a RowComparisonExpression on a reordered primary key index. {{<issue 23824>}}
• Disables in-place index updates by default to address index inconsistency issues. {{<issue 24672>}}
• Increases YSQL connection manager's thread stack size to 512 KB to prevent crashes on Alma 9 machines. {{<issue 24979>}}
• Enhances read performance on colocated tables by skipping redundant tombstone checks. {{<issue 21561>}}
• Fixes flag overlap to differentiate sticky settings from explain in configurations. {{<issue 24954>}}
• Renames pipe type to orafce_pipe to resolve naming conflicts. {{<issue 12990>}}
• Eliminates unnecessary file creation for views on temporary tables by checking if storage is actually needed. {{<issue 19522>}}
• Reduces maintenance time by switching to a less complex implementation of SideBySideDiff.java, thereby eliminating errors from SideBySideDiff.sanityCheckLinesMatch. {{<issue 19690>}}
• Prevents segmentation faults in parallel index scans with aggregate pushdown. {{<issue 21427>}}
• Increases performance by batching index reads during INSERT ON CONFLICT, with customizable batch sizes using yb_insert_on_conflict_read_batch_size. {{<issue 24179>}}
• Improves cache re-invalidation for ALTER TABLE commands to avoid schema version mismatch errors in the same session. {{<issue 23882>}}
• Fixes UPDATE errors on partitioned tables by correctly mapping column numbers. {{<issue 23857>}}
• Ensures restart read time does not exceed the global limit to avoid inconsistent reads. {{<issue 24017>}}
• Fixes the flaw in the current DDL atomicity workflow where only the first table's schema was compared, even if it doesn't change after a schema version increment. {{<issue 23988>}}
• Streamlines ysql_dump process by eliminating faulty statements for colocated table unique constraint, enhancing database reliability. {{<issue 24057>}}
• Introduces batching for the INSERT ON CONFLICT command to enhance performance by minimizing alternating read and write operations. Control the batch size with the new YSQL configuration parameter yb_insert_on_conflict_read_batch_size, which, when default at 1, disables batching. {{<issue 24179>}}
• Enhances index relation checks with YBIsCoveredByMainTable method. {{<issue 24239>}}
• Alters encoding setup order to ensure UTF8 use instead of SQL_ASCII and adds pg_collation to pre-loaded tables to prevent specific errors. {{<issue 24149>}}
• Disables LZ4 build option to prevent PostgreSQL build failures. {{<issue 24286>}}
• Fixes crashes in creating tables with over 585 columns. {{<issue 24304>}}
• Adds support for ALTER TABLE VALIDATE CONSTRAINT in YugabyteDB. {{<issue 24291>}}
• Prevent incorrect loading of pg_collation catalog caches, eliminating associated performance issues. {{<issue 24149>}}
• Fixes out-of-bound memory access during index scan cost estimation ensuring stable query execution on system catalogs and expression-based indexes when using the yb_enable_base_scans_cost_model configuration {{<issue 24246>}}
• Blocks REINDEX and TRUNCATE on system tables using table rewrite. {{<issue 24356>}}
• Blocks the "NULLS NOT DISTINCT" command until supported. {{<issue 24452>}}
• Enhances consistency and correctness of INSERT...ON CONFLICT queries via YBCTID infrastructure integration and index management. {{<issue 24692>}},{{<issue 24784>}}
• Assumes indexes contain 1000 rows when statistics are unavailable, ensuring better query planning. {{<issue 24496>}}
• Enhances performance during nested loop joins by ensuring correct limit resets in colocated index scans. {{<issue 24560>}}
• Removes time sync service requirement from the prerequisites. {{<issue 24758>}}
• Enhances UPDATE query execution on generated columns by including them in updatedCols calculations. {{<issue 24641>}}
• Fixes foreign key checks to avoid errors with unique indexes. {{<issue 24663>}}
• Ensures accurate catalog version tracking during concurrent index creation. {{<issue 24775>}}
• Ensures YB master reads from valid catalog tables during upgrades. {{<issue 24647>}}
• Re-enables in-place index updates by default after fixing related inconsistencies. {{<issue 24849>}}
• Prevents DML operations on sequence data during upgrades. {{<issue 24943>}}
• Fixes ysql_dump crashes related to pg_catalog schema dumps. {{<issue 24907>}}
• Accurately estimates the cost of partial index scans by including index conditions. {{<issue 24916>}}
• Reduces memory usage spikes by using a new memory context for RelationBuildTriggers. {{<issue 24951>}}
• Adds debugging logs to track new OIDs for resolving table OID collisions. {{<issue 24320>}}
• Ensures successful cache invalidation after a failed ALTER TABLE to prevent "schema mismatch" errors. {{<issue 24981>}}
• Disables index update optimization by default to fix ON CONFLICT UPDATE issues. {{<issue 25075>}}
• Reverts changes related to yb_amiscoveredbymaintable for updated index co-partitioning design. {{<issue 25119>}}

YCQL

• Allows users to control error suppression for GROUP BY in YCQL using ycql_ignore_group_by_error flag. {{<issue 25030>}}
• Eliminates unexpected error during shell type definition update across multiple sessions. {{<issue 24217>}}

DocDB

• Allows the packed row size to exceed its limit during repacking, preventing TServer crash after alter table add column commands with a default value. {{<issue 24050>}}
• Prevents server crashes during table truncation by safely shutting down iterators before releasing locks. {{<issue 23243>}}
• Prevents resource waste by checking disk space before remote bootstrap operations. {{<issue 23987>}}
• TServers now reject remote bootstrap requests for previously deleted tablets. {{<issue 24574>}}
• Sets a default 10MB max size for RocksDB MANIFEST files, now configurable via flags. {{<issue 11117>}}
• Adds mtime and checksum to block cache key prefix to prevent conflicts. {{<issue 20852>}}
• Enhances visibility for unexpected rate limiter sleep times in RBS threads. {{<issue 24342>}}
• Eliminates inconsistencies by not including inactive tablets in OpenTable operations. {{<issue 24860>}}
• Adds a null check to prevent a system crash when cleaning up a recently applied transaction state after a flush operation. {{<issue 24026>}}
• Prevents crashes during compaction when ysql_enable_packed_row is off and existing packed rows are present. {{<issue 24545>}}
• Updates YSQL migration script to align with PostgreSQL 15 catalog changes. {{<issue 24593>}}
• Switches pg_cron back to using standard SIGTERM handling for cleaner shutdowns. {{<issue 24658>}}
• Ensures tablets remain effectively managed during transition from RF1 to higher replication factors. {{<issue 24575>}}
• Ensures consistent tablegroup assignments for colocated table backups without using explicit tablespaces. {{<issue 24809>}}
• Fixes error for select queries in large tables by adjusting backward scan bounds. {{<issue 24824>}}
• Eliminates segmentation faults by ensuring CleanupHiddenTables holds a write lock during Upsert. {{<issue 24929>}}
• Prevents fatal errors during multiple splits and PITR restores. {{<issue 25103>}}
• Ensures "create table" requests for colocated tables don't fail due to mistaken tablet limit checks. {{<issue 23922>}}
• Reduces unnecessary checks when adding tables in alter replication requests. {{<issue 24164>}}
• Enhances stability by replacing raw pointers with std::weak_ptr in tablet server handling. {{<issue 24182>}}
• Enhances index parameter consistency between Usearch and Hnswlib for balanced performance. {{<issue 24465>}}
• Ensures correct metadata sequencing for vector index chunks, preventing data inconsistencies. {{<issue 24511>}}
• Speeds up vector LSM loading by using a thread pool. {{<issue 24588>}}
• Adds a flag to limit multiple pending compactions for priority pools. {{<issue 24540>}}
• Enables merging of small vector index chunks to enhance search efficiency. {{<issue 24069>}}
• Resolves scheduling of contentious waiters to the correct, larger RPC thread pool. {{<issue 24478>}}
• Fixes shutdown path issues to prevent fatal errors during RPC completion. {{<issue 24524>}}
• Ensures pg_txn_start_us is set for restarted transactions, fixing priority issues in waiter resumption. {{<issue 24778>}}
• Fixes mini cluster restarts to ensure master leaders track all TServer re-connections. {{<issue 24772>}}
• Reduces latency by enhancing early resume of RPCs blocked on aborted transactions. {{<issue 24799>}}
• Fixes memory leak in MasterPathHandlers by properly managing shared pointers. {{<issue 24823>}}
• Ensures callback triggers after loading tablet server data if the count meets requirements. {{<issue 23744>}}
• Ensures consistent reads by verifying restart conditions before reporting false corruption. {{<issue 25072>}}

CDC

• Allows sending a null "before image" for CDC if cdc_send_null_before_image_if_not_exists flag is true. {{<issue 18381>}}
• Enhances accuracy of transaction start time logging for better consistency in CDC. {{<issue 24521>}}
• Fixes memory leaks in the walsender's handling of transactions and file nodes. {{<issue 24788>}}
• Fixes memory leaks in the walsender's handling of transactions and file nodes. {{<issue 24788>}}
• Ensures null values are now correctly captured in CDC before image records. {{<issue 22983>}}
• Enables removal of dropped tables from the replica identity map to prevent replication errors. {{<issue 24308>}}
• Excludes expired or irrelevant entries from metrics calculations for more accurate reporting. {{<issue 24975>}}

yugabyted

• Nodes now start using private IP as advertise_address instead of hostname. {{<issue 25037>}}