What's new in the YugabyteDB v2024.1 STS release series

What follows are the release notes for the YugabyteDB 2024.1 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB 2024.1 release series.

For an RSS feed of all release series, point your feed reader to the RSS feed for releases.

{{<warning title="Changes to supported operating systems">}} YugabyteDB 2024.1.0.0 and newer releases do not support v7 Linux versions (CentOS7, Red Hat Enterprise Linux 7, Oracle Enterprise Linux 7.x), Amazon Linux 2, and Ubuntu 18. If you're currently using one of these Linux versions, upgrade to a supported OS version before installing YugabyteDB v2024.1.0. Refer to Operating system support for the complete list of supported operating systems. {{</warning>}}

{{< tip title="New memory division settings available" >}}

YugabyteDB uses memory division flags to specify how memory should be divided between different processes (for example, YB-TServer versus YB-Master) on a YugabyteDB node as well as within processes. Using these flags, you can better allocate memory for PostgreSQL, making it more suitable for a wider range of use cases.

For new v2024.1.x universes, if you are expecting to use any nontrivial amount of YSQL, it is strongly recommended to set ‑‑use_memory_defaults_optimized_for_ysql. This changes the memory division defaults to better values for YSQL usage, and optimizes memory for the node size.

If you are upgrading a universe, you may want to instead review your memory division settings and adjust them if desired; see best practices.

In future releases, the memory division settings will be used to determine how many tablet replicas a YB-TServer can safely support; this information will power new alerts warning you about overloading nodes with too many tablet replicas and allow blocking operations that would create too many tablet replicas.

{{< /tip >}}

Release announcements

• Introducing YugabyteDB with Enhanced Postgres Compatibility: Postgres Without Limits

v2024.1.6.1 - July 23, 2025 {#v2024.1.6.1}

Build: 2024.1.6.1-b2

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.6.1-b2

Bug fix

DocDB

• Ensures accurate replication for transactional writes on xCluster range partitioned tables. {{<issue 27380>}}

v2024.1.6.0 - April 25, 2025 {#v2024.1.6.0}

Build: 2024.1.6.0-b53

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.6.0-b53

Change log

Improvements

YSQL

• Adds new flags for role management in YSQL backups and restores. {{<issue 20972>}}
• Enables viewing TCMalloc heap snapshots for PG backend processes via new YSQL functions. {{<issue 21851>}}
• Enhances yb_servers function to include universe_uuid for better cluster identification. {{<issue 25818>}}

DocDB

• Enhances handling of expired snapshots by retrying deletion tasks automatically. {{<issue 25628>}}

CDC

• Reduces logging frequency for certain CDC errors to avoid clutter. {{<issue 26148>}}

Bug fixes

YSQL

• Ensures Batched Nested Loop joins correctly check for hashable join clauses. {{<issue 25917>}}
• Reduces XID usage by generating one per REFRESH MATERIALIZED VIEW CONCURRENTLY operation. {{<issue 26205>}}
• Fixes potential negative cost calculations for large table scans by using double for estimates. {{<issue 25862>}}
• Enhances estimation of result tuple size in edge cases, preventing division by zero errors. {{<issue 21828>}}
• Fixes incorrect role name quoting in ysql_dumpall. {{<issue 25701>}}
• Fixes \d command for indexes with spaces in their names. {{<issue 25711>}}
• Modifies ysql_dumpall to combine CREATE ROLE and ALTER ROLE for effective role management. {{<issue 25608>}}
• Enhances index scans and partition pruning for BOOLEAN conditions. {{<issue 26266>}}
• Enhances accuracy of seek-next estimation for index scans and paged results. {{<issue 22087>}}
• Refines cost model tuning using server-side execution times for more accurate query optimization. {{<issue 23709>}}
• Prevents crashes by handling non-variable expressions in single-row updates or deletes. {{<issue 26536>}}
• Prevents relcache reference leaks in yb_get_ybctid_width. {{<issue 24819>}}
• Optimizes cost modeling for primary index scans to assume sequential disk block fetching. {{<issue 26235>}}
• Enhances yb_index_check to verify materialized view indexes' consistency. {{<issue 26639>}}
• Corrects estimations for inner table scans in Batched Nested Loop Joins. {{<issue 26462>}}
• Fixes integer overflow in BNL cost estimation, preventing negative values. {{<issue 26463>}}

DocDB

• Adds mtime and checksum to block cache key prefix to prevent conflicts. {{<issue 20852>}}
• Enhances transactional xCluster accuracy by using majority replicated OpId for ApplySafeTime calculations. {{<issue 26043>}}
• Ensures accurate WaitForReplicationDrain behavior by not mislabeling tablets as drained. {{<issue 25457>}}
• Prevents false conflict detection in snapshot isolation operations. {{<issue 26142>}}

yugabyted

• Preserves the universe key locally after enabling EAR for recovery scenarios. {{<issue 26552>}}

v2024.1.5.0 - February 18, 2025 {#v2024.1.5.0}

Build: 2024.1.5.0-b26

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.5.0-b26

Change log

Improvements

YSQL

• Enhances initdb logging, correctly handles process termination, and fixes memory leaks on abort signals. {{<issue 24813>}}

yugabyted

• Ensures cached plans correctly depend on user roles, addressing {{<cve "CVE-2024-10976">}}. {{<issue 25330>}}
• Adds support for IPv6. {{<issue 24665>}}

CDC

• Limits CDC GetConsistentChanges response by byte size, allowing toggling with cdc_use_byte_threshold_for_vwal_changes. {{<issue 25572>}}

Bug fixes

YSQL

• Reduces memory usage during YugabyteDB connection startup. {{<issue 24925>}}
• Removes an erroneous log entry that appeared in crashes or errors. {{<issue 24533>}}
• Enables pg_authid table pre-fetching during login to reduce RPC calls and prevent errors. {{<issue 25776>}}
• Ensures correct row order for queries with IN clauses. {{<issue 25559>}}
• Fixes data race in regex pushdown by using thread-local variables. {{<issue 24237>}}
• Automatically restarts the postmaster to prevent stuck processes when a background worker crashes. {{<issue 24706>}}
• Reduces scope of ALTER DATABASE impact to only the targeted database. {{<issue 25392>}}
• Ensures DDL operations continue smoothly by retrying background tasks on polling errors. {{<issue 25708>}}
• Reverts ALTER DATABASE RENAME and ALTER DATABASE OWNER to global impact. {{<issue 25742>}}
• Fixes incorrect aggregate column references in flattened subqueries. {{<issue 24512>}}

DocDB

• Prevents crash during transaction abort by adjusting lock handling. {{<issue 25689>}}
• Prevents crashes during catalog reloads by safely handling null cluster_config_. {{<issue 21775>}}
• Prevents xCluster failures caused by same-name table re-creation during non-DBScoped DR. {{<issue 25709>}}

CDC

• Ensures newly created tables in logical replication are retained correctly during cleanup. {{<issue 25206>}}

v2024.1.4.0 - January 9, 2025 {#v2024.1.4.0}

Build: 2024.1.4.0-b108

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.4.0-b108

New features

CDC

• Enables tablet splitting on tables under CDCSDK stream by default using the flag enable_tablet_split_of_cdcsdk_streamed_tables. {{<issue 24190>}}

Change log

Improvements

YSQL

• Updates advisory lock error message with a workaround hint instead of not yet implemented. {{<issue 23881>}}
• Allows preloading of foreign key lists in relcache, avoiding on-demand master fetches, controlled by yb_enable_fkey_catcache. {{<issue 23686>}}
• Streamlines sys table prefetching by automatically resetting catalog read time. {{<issue 23502>}}
• Prevents catalog version bumps on no-op ALTER ROLE commands, saving system resources. {{<issue 24390>}}

YCQL

• Resolves timestamp precision mismatches in YCQL, aligning inserts and index scans to millisecond precision. Enables users to optionally revert to microsecond precision using the flag cql_revert_to_partial_microsecond_support=false. {{<issue 11052>}},{{<issue 23476>}}
• Tracks microsecond precision timestamps in YCQL with new metric cql_microseconds_timestamps_used. This tracking occurs if the cql_revert_to_partial_microsecond_support flag is true. {{<issue 24776>}}

DocDB

• Blocks writes based on total bytes being flushed, not just memtable count. {{<issue 22571>}}
• Enhances load balancer to prioritize under-replicated tablets first. {{<issue 20263>}}
• Limits concurrent remote bootstraps per TServer using load_balancer_max_inbound_remote_bootstraps_per_tserver flag. {{<issue 2426>}}
• Reduces unnecessary attribute map creations during Prometheus metric aggregation. {{<issue 24405>}}
• Disables db_max_flushing_bytes logic to enhance write-stalling based on memtable size. {{<issue 22571>}}
• Upgrades non-FIPS OpenSSL to 3.0.15 and removes CentOS 7 builds. {{<issue 24436>}}
• Prevents overloading by correctly categorizing bootstrapping tablets in load balancing. {{<issue 23487>}}
• Enhances RBS throttling by focusing on active fetch sessions and adjusts expiration logic. {{<issue 21563>}},{{<issue 24031>}}
• Enhances stability by fixing a missing null check in tablet lookup processes. {{<issue 25145>}}

CDC

• Disables automatic tablet splitting for CDC stream tables by default. {{<issue 25148>}}
• Enables dynamic table addition in logical replication by default. {{<issue 25413>}}
• Automatically removes metrics when a CDC stream is deleted. {{<issue 24876>}}

yugabyted

• Ensures accurate CPU usage metrics in the YCQL table by updating prev_ticks_ each iteration. {{<issue 22910>}}

Bug fixes

YSQL

• Modifies backup/restore process to skip column name checks for indexes, allowing for successful restoration even with renamed columns. {{<issue 24207>}}
• Enhances Postgres restart stability by refining shutdown handling and lock file management. Includes killing only Postgres-associated PIDs, using SIGQUIT for immediate shutdowns to reduce hang-ups, and monitoring for stuck processes on macOS {{<issue 24396>}}
• Streamlines ysql_dump process by eliminating faulty statements for colocated table unique constraint, enhancing database reliability. {{<issue 24057>}}
• Fixes the flaw in the current DDL atomicity workflow where only the first table's schema was compared, even if it doesn't change after a schema version increment. {{<issue 23988>}}
• Alters encoding setup order to ensure UTF8 use instead of SQL_ASCII and adds pg_collation to preloaded tables to prevent specific errors. {{<issue 24149>}}
• Prevent incorrect loading of pg_collation catalog caches, eliminating associated performance issues. {{<issue 24149>}}
• Adds HELP and TYPE metadata to each metrics onto the YSQL Prometheus metrics endpoint for better metrics accessibility. {{<issue 23578>}}
• Blocks REINDEX and TRUNCATE on system tables using table rewrite. {{<issue 24356>}}
• Fixes out-of-bound memory access during index scan cost estimation ensuring stable query execution on system catalogs and expression-based indexes when using the yb_enable_base_scans_cost_model configuration {{<issue 24246>}}
• Ensures parent partitioned tables correctly rewrite during operations, preventing backup/restore errors. {{<issue 24458>}}
• Assumes indexes contain 1000 rows when statistics are unavailable, ensuring better query planning. {{<issue 24496>}}
• Enhances performance during nested loop joins by ensuring correct limit resets in colocated index scans. {{<issue 24560>}}
• Enhances indexing by allowing batching of PlaceHolderVar in parameterized conditions. {{<issue 24186>}}
• Reduces assert strictness in YSQL, preventing crashes when only some index key columns have unbatchable clauses. {{<issue 22911>}}
• Fixes foreign key checks to avoid errors with unique indexes. {{<issue 24663>}}
• Accurately estimates the cost of partial index scans by including index conditions. {{<issue 24916>}}
• Reduces memory usage spikes by using a new memory context for RelationBuildTriggers. {{<issue 24951>}}
• Allows Batch Nested Loop (BNL) joins on partitioned tables in outer joins. {{<issue 25080>}},{{<issue 25078>}}
• Ensures accurate catalog version tracking during concurrent index creation. {{<issue 24775>}}
• Fixes inconsistency in index entries during concurrent writes in the backfill phase. {{<issue 25250>}}

YCQL

• Allows users to control error suppression for GROUP BY in YCQL using ycql_ignore_group_by_error flag. {{<issue 25030>}}
• Eliminates unexpected error during shell type definition update across multiple sessions. {{<issue 24217>}}
• Prevents TServer crashes on executing specific queries by handling NULL pointers effectively. {{<issue 23004>}}

DocDB

• Prevents tablet bootstrap from getting stuck when replaying a truncate operation. {{<issue 23243>}}
• Allows the packed row size to exceed its limit during repacking, preventing TServer crash after alter table add column commands with a default value. {{<issue 24050>}}
• Ensures heartbeat processing now checks sequence numbers, avoiding overwrites that could corrupt metadata during remote bootstrapping of new tablets. This change prevents loss of leader tracking and enhances data consistency. {{<issue 21836>}}
• TServers now reject remote bootstrap requests for previously deleted tablets. {{<issue 24574>}}
• Sets a default 10MB max size for RocksDB MANIFEST files, now configurable via flags. {{<issue 11117>}}
• Enhances visibility for unexpected rate limiter sleep times in RBS threads. {{<issue 24342>}}
• Prevents crashes when calling ListMasterServers RPC on a master by returning an error. {{<issue 20372>}}
• Ensures continuous xCluster task operation even if the master temporarily loses the lease but retains leadership. {{<issue 25480>}}
• Prevents crashes during compaction when ysql_enable_packed_row is off and existing packed rows are present. {{<issue 24545>}}
• Fixes error for select queries in large tables by adjusting backward scan bounds. {{<issue 24824>}}
• Ensures consistency between Slice::Less and Slice::compare methods. {{<issue 25319>}}
• Fixes shutdown path issues to prevent fatal errors during RPC completion. {{<issue 24524>}}
• Resolves scheduling of contentious waiters to the correct, larger RPC threadpool. {{<issue 24478>}}
• Reduces latency by enhancing early resume of RPCs blocked on aborted transactions. {{<issue 24799>}}
• Enables ASAN builds to use a specific older third party version to ensure compatibility and unblock backports. {{<issue 17404>}}

CDC

• Fixes memory leaks in the walsender's handling of transactions and file nodes. {{<issue 24788>}}
• Enables removal of dropped tables from the replica identity map to prevent replication errors. {{<issue 24308>}}
• Excludes expired or irrelevant entries from metrics calculations for more accurate reporting. {{<issue 24975>}}
• Ensures accurate history retention for various replica identities in CDC. {{<issue 25193>}}

v2024.1.3.1 - November 20, 2024 {#v2024.1.3.1}

Build: 2024.1.3.1-b8

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.3.1-b8

Bug fixes

YSQL

• Enhances Postgres restarts by better handling lock files, identifying stuck processes and uses SIGQUIT for immediate shutdowns {{<issue 24396>}}
• Fixes a crash during cost estimation of indexes on system columns and expressions when using the new cost model {{<issue 24246>}}

v2024.1.3.0 - October 8, 2024 {#v2024.1.3.0}

Build: 2024.1.3.0-b105

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.3.0-b105

New features

• Bitmap scan support. Combine multiple indexes for more efficient scans. {{<tags/feature/ea>}}

• Voyager assessment visualisation in yugabyted UI. Updates to the migrations screens - "Schema Migrate page" to better capture the schema issues, and "Data Migrations UX" to readily provide voyager commands to users.

• Disk full scenarios. Fail writes on nodes, if the free disk space on the nodes becomes low.

Change log

Improvements

YSQL

• Introduces the yb_make_next_ddl_statement_nonincrementing configuration parameter to reduce catalog version increments during many consecutive DDLs, lowering memory and latency spikes. {{<issue 23786>}}
• Added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables for yb_db_admin role. {{<issue 23786>}}

YCQL

• Updates YCQL tables system_schema.functions and system_schema.aggregates to align with Cassandra, aiding Cortex migration. {{<issue 3893>}}

DocDB

• Enables reduction of duplicate code and custom flag filtering through GetFlagInfos relocation. {{<issue 23632>}}
• Allows reduction of thread stack size to avoid Linux hugepage backing. {{<issue 23927>}}
• Adds emergency_repair_mode flag and yb-admin commands dump_sys_catalog_entries and write_sys_catalog_entry to address CatalogEntity data corruption issues. {{<issue 23098>}}
• Introduces a new flag max_disk_throughput_mbps for automated control of write rejections when disk is full, replacing reject_writes_min_disk_space_aggressive_check_mb. {{<issue 23373>}}
• Allows explicit addition of flags to gflag_allowlist.txt for secure call home data collection, plus version_info in TServer data. {{<issue 24103>}}

CDC

• Allows dynamic table addition alongside table removal from CDC streams without needing to disable or restart. {{<issue 23581>}}
• Adds a yb-admin option to disable dynamic table addition when creating new streams. {{<issue 23482>}}
• Allows automatic cleanup for ineligible tables on CDC streams and improves the removal process by fetching tablet list from a local copy instead of the state table, improving performance. Updates include removal of flag protection for Step-1 and the replacement of the preview flag. {{<issue 23806>}}
• Syncs table removal from a CDC stream to a more performant async process, allowing both manual and automatic removal via the RemoveUserTableFromCDCSDKStream and RemoveTablesFromCDCSDKStream RPCs respectively. This change utilizes a background thread to enhance user table deletion while preserving data consistency. {{<issue 23700>}}
• Enables asynchronous removal of user tables from CDC streams by adjusting how background threads process and persist stream metadata. {{<issue 23700>}}
• Disables wal_level checks for logical replication in YSQL as PG WAL is not used. {{<issue 23661>}}
• Changes the default wal_level to logical to ensure compatibility with logical replication clients. {{<issue 23661>}}

yugabyted

• Enables bitmap scans and removes size-based fetching in the PostgreSQL Parity feature by updating flags. {{<issue 23777>}}
• Enhances yugabyted configure_read_replica commands with checks to gracefully handle failures when data_placement_constraint lacks :<num_of_replicas>. {{<issue 23273>}}
• Allows yugabyted UI metrics to display correctly when deployed with Kubernetes OSS operator. {{<issue 22532>}}
• Reduces data redundancy by stopping the sending of flag information from yugabyted in callhome diagnostics. {{<issue 24029>}}

Bug fixes

YSQL

• Fixes an issue related to regular expression pushdown causing inconsistency and crashes by introducing thread local cache. {{<issue 22989>}}
• Resets catalog read time after table prefetching to prevent outdated readings for on-demand loading from a master. {{<issue 23421>}}
• Corrects IndexScan's indextlist to improve accuracy of variable references in subquery scenarios. {{<issue 22533>}}
• Fixes crashes caused by using a RowComparisonExpression on a reordered primary key index. {{<issue 23824>}}
• Fixes repeated calls in DeleteTableInternal, preventing YbAdminSnapshotScheduleTest.SysCatalogRetention failure on asan builds. {{<issue 23459>}}
• Resolves potential memory corruption by mutating pushdown expressions outside the per tuple context. {{<issue 23461>}}
• Eliminates the publishing of empty metrics from the statementType enum on the YSQL Metrics endpoint. {{<issue 23557>}}
• Revamped Storage Read Requests metric now better reflects true RPC count in PrecastRequestSender usage. {{<issue 23396>}},{{<issue 18785>}}
• Adds more logs to aid in debugging a transaction status mismatch issue. {{<issue 23669>}}
• Addresses a bug in YSQL that caused crashes while costing an index scan on a hash index with included columns using CBO. {{<issue 22772>}}
• Fixes the hanging issue when creating a colocated table by correcting the unconditional reading of the relkind column. {{<issue 23708>}}
• Fixes a bug that caused incorrect deducing of commit/abort status for transactions which do not change the table's schema. {{<issue 23669>}}
• Replaces deprecated function shared_ptr::unique to resolve compilation errors on macOS. {{<issue 24012>}}
• Improves cache re-invalidation for ALTER TABLE commands to avoid schema version mismatch errors in the same session. {{<issue 23882>}}
• Adds a new flag TEST_check_catalog_version_overflow that can detect and prevent catalog version overflow issues. {{<issue 24021>}}
• Removes unnecessary DCHECK for smoother rolling upgrades from release 2.20, enhancing ysql_enable_db_catalog_version_mode flag functionality. {{<issue 23462>}}
• Enables a node to successfully join using the join flag by implementing a retry mechanism. {{<issue 23898>}}

DocDB

• Limits response size by omitting per-object details from SysSnapshotEntryPB in ListSnapshotSchedules RPC. {{<issue 23518>}}
• Reduces the risk of deadlocks when making active read/write requests to a table right after a tablet-split. {{<issue 23747>}}
• Simplifies handling of large RPC responses by catching errors earlier and appropriately using narrow_cast. {{<issue 22301>}}
• Switches write query duration from CoarseMonoClock to MonoClock for precise nanosecond granularity. {{<issue 20335>}}
• Refactors to remove an assumption that every TServer hosts tablets, preventing crashes in certain circumstances. {{<issue 20230>}}
• Sets the close_timestamp_micros field for partially copied log segments to prevent time-based policy violation. {{<issue 23335>}}
• Eliminates persistent logging issue after dropping a colocated table by removing unnecessary logs. {{<issue 22037>}}
• Corrects leader balancing for geopartitioned tables to prevent early exits and ensure accurate global leader sorting. {{<issue 23886>}}
• Enables batching of metric updates for YCQL reads to prevent performance slowdowns. {{<issue 21832>}}
• Ensures "create table" requests for colocated tables don't fail due to mistaken tablet limit checks. {{<issue 23922>}}
• Ensures node restarts successfully with secure enabled by introducing a timeout framework. {{<issue 23447>}}
• Promotes cluster start-up when master_webserver_port is customized by considering user input during the validate_and_set_configs step. {{<issue 23629>}}

CDC

• Prevents TServer crash by giving each thread its own copy of SchemaDetailsMap during concurrent GetChanges calls. {{<issue 23394>}},{{<issue 22962>}}
• Refactors CDCSDK to prevent data loss during workload with single shard transactions by ensuring WAL messages are committed before being replicated. {{<issue 19294>}}
• Reduces disk IO overhead by optimizing the tablet-meta update process in CDCService. {{<issue 22805>}}
• Introduces three new yb-admin commands for managing user tables in a CDC stream and addresses a race condition issue by removing an optimization in the UpdateCheckpointForTabletEntriesInCDCState method. {{<issue 22876>}},{{<issue 22835>}},{{<issue 22773>}}
• Obsoletes expired or unpolled tables from CDC stream and state table, reducing unnecessary resource retention and potential blocking of split tablet deletion. {{<issue 23367>}}
• Filters out records related to index tables in colocated databases to prevent GetChanges call failures. {{<issue 23809>}}
• Ensures old-tuples for updates are only sent when replica identity is FULL, matching PostgreSQL behavior. {{<issue 23353>}}

yugabyted

• Offers corrected DB OID in the function calculate_table_size, now matching YBCGetDatabaseOid(rel) for improved reliability. {{<issue 20030>}}

v2024.1.2.0 - September 3, 2024 {#v2024.1.2.0}

Build: 2024.1.2.0-b77

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.2.0-b77

New features

• Semi-automatic transactional xCluster setup. Provides operationally simpler setup and management of YSQL transactional xCluster replication, as well as simpler steps for performing DDL changes. {{<tags/feature/ea idea="1186">}}

• Kubernetes readiness probe. Added readiness probes for TServer and Master pods in YugabyteDB, supporting custom or default configurations, thereby enhancing stability by ensuring YSQL/YCQL and YB-Master pods are ready before traffic is routed. {{<tags/feature/ea>}}

• yugabyted

  • Voyager assessment visualisation in yugabyted UI. Yugabyted UI provides a dashboard to allow the users to effectively plan the migrations based on the complexity and also be able to monitor the progress of each migration <!-- IDEA-1075 -->

  • Backup/restore support with TLS enabled. In secure mode, yugabyted cluster supports taking full backup/restores. {{<tags/feature/ea>}}

  • xCluster support. yugabyted enables native support for setting up xCluster between two yugabyted deployed clusters. {{<tags/feature/ea>}}

Change log

Improvements

YSQL

• Avoids renaming DocDB tables during legacy rewrite operations to prevent issues with backup/restore and improves handling of failed ADD/DROP primary key, ALTER TYPE operations. {{<issue 22802>}}
• Adds Save/Restore state functionality to ConsistentReadPoint using a new Momento class. {{<issue 22597>}}

DocDB

• Allows asynchronous DNS cache updating and resolution retry upon failure to reduce RPC call delays and prevent unexpected leadership changes. {{<issue 22930>}},{{<issue 22311>}}
• Enhances stack trace tracking endpoints usability by adding an access button, a reset tracking link, and timestamp details. {{<issue 22842>}}
• Introduces a new flag enable_rwc_lock_debugging to control slow lock debugging and fixes a bug in rwc_lock.cc. {{<issue 22807>}}

yugabyted

• Enables better handling of multi-valued flags in yugabyted without duplication, making the system more maintainable. {{<issue 22091>}}
• Enables xCluster replication management between database clusters using new yugabyted commands. {{<issue 22349>}}
• Reduces collect_logs command failures by removing the yugabyted running check even when the yugabyted process is not running. {{<issue 23210>}}
• Introduces upgrade_finalize command for easy YugabyteDB version upgrades using yugabyted CLI and includes an upgrade_ysql_timeout flag. {{<issue 21888>}}
• Enhances yugabyted configure_read_replica commands with checks to gracefully handle failures when data_placement_constraint lacks :<num_of_replicas>. {{<issue 23273>}}

Bug fixes

YSQL

• Allows YB Admins to run pg_locks without requiring superuser status. {{<issue 23266>}}
• Allows faster data inserts into tables with identity columns. {{<issue 22837>}}
• Prevents a crash related to memory release associated with TupleTableSlots in SubPlans during a Values Scan. {{<issue 22967>}}
• Reduces prefix length in the index when using distinct index scan with included columns. {{<issue 22822>}}
• Allows resetting of stats collected by the ANALYZE command, including reltuples, pg_statistic rows, and pg_statistic_ext values. {{<issue 22028>}}
• Eliminates unnecessary waiting for concurrent transactions in the DEFERRABLE mode for READ ONLY serializable transactions. {{<issue 23120>}}
• Streamlines the "drop column" operation process, preventing hindrance even if the alter schema RPC is missed. {{<issue 23100>}}
• Fixes TServer crash when pushing down certain SAOP operations like string_to_array. {{<issue 23287>}}
• Allows pg_replication_slots to return an empty response instead of an error when ysql_yb_enable_replication_commands flag is false. {{<issue 23096>}}
• Ensures the node restarts properly even with secure mode enabled by adding a timeout framework. {{<issue 23447>}}

DocDB

• Allows restoring a snapshot schedule from a time just before the oldest snapshot, improving usability. {{<issue 21269>}}
• Fixes issue of scans not honoring timeouts, preventing indefinite reads and reducing CPU usage. {{<issue 21829>}}
• Adds validation to RPCs DeleteSnapshot and RestoreSnapshot to prevent deletion or use of snapshots involved in ongoing processes. {{<issue 23055>}}
• Prevents unexpected leader fatal errors by updating cached leader terms immediately after a leader change. {{<issue 21808>}}
• Allows the persistent mapping of source-target schema versions when a new table is added to a colocated database, thus preventing replication from stalling after TServer restarts. {{<issue 23188>}}
• Updates cotable IDs in flushed frontiers during a snapshot restore, preventing potential post-restore issues. {{<issue 23047>}}
• Ensures object drops correctly cascade to dependent columns in the DocDB table, preventing inconsistencies. {{<issue 22874>}}
• Allows large bytes to be requested on RateLimiter, preventing indefinite call stalling. {{<issue 23173>}}
• Restores the previously missing home icon in the master user interface. {{<issue 23275>}}
• Removes extra reads during the processing of INSERT INTO ...RETURNS STATUS AS ROW for CQL tables with a specific primary key, improving system load and efficiency. {{<issue 23330>}}

CDC

• Introduces new auto flag cdcsdk_enable_identification_of_non_eligible_tables and three yb-admin commands to manage tables in CDC stream, enhancing control and reducing unnecessary resource usage. {{<issue 22876>}},{{<issue 22835>}},{{<issue 22773>}}
• Fixes the issue of getting either 0 or a random time as the server's system clock in XLogData from the logical replication stream. {{<issue 22929>}}
• Simple: Adds null checks and removes table removal optimization for CDC stream, ensuring proper arrangement and functioning of user-created tables. {{<issue 22876>}},{{<issue 22835>}},{{<issue 22773>}}
• Enables support for dynamically allotted OID data types in CDC to prevent system crashes. {{<issue 23179>}}
• Allows handling of non-eligible table cleanup in CDC stream loading even after table drop, preventing master crash. {{<issue 23278>}}

v2024.1.1.0 - July 31, 2024 {#v2024.1.1.0}

Build: 2024.1.1.0-b137

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.1.0-b137

Highlight

PostgreSQL Logical Replication Protocol Support {{<tags/feature/ea idea="805">}}

We're excited to announce in the 2024.1.1.0 release support for the PostgreSQL Logical Replication Protocol for Change Data Capture (CDC), in addition to the existing native gRPC Replication protocol. This feature allows you to manage CDC streams using Publications and Replication Slots, similar to native PostgreSQL. Additionally, a new connector is introduced that utilizes the logical replication protocol to consume the CDC streams via Replication slots.

For more information, refer to logical replication protocol. <!-- IDEA-805 -->

New features

• Automated SQL/CQL Shell binary. Along with full binary, added separate downloadable SQL/CQL Shell binary. <!-- IDEA-1526 -->

• Voyager assessment visualisation in yugabyted UI. Yugabyted UI provides a dashboard to allow the users to effectively plan the migrations based on the complexity and also be able to monitor the progress of each migration <!-- IDEA-1075 --> {{<tags/feature/tp idea="IDEA-1075">}}

Change log

Improvements

YSQL

• Eradicates misleading log message during table creation with DDL atomicity enabled. {{<issue 22459>}}
• Removes the unused keep_order field from YbctidGenerator for cleaner results processing. {{<issue 21944>}}
• Updates read time for each operation to simplify code and avoid applying used read time from obsolete operations. {{<issue 21623>}}
• Enhances logging for DDL atomicity and corrects a loop index in cross-db unit test for clearer test logs. {{<issue 22736>}}

DocDB

• Offers a new stack trace tracking framework for improved visibility of disk I/O operations, configurable through the track_stack_traces flag. {{<issue 17993>}}
• Introduces utility to dump top heap stacks when memory limit is exceeded for enhanced debugging. {{<issue 21396>}}
• Adds SCHECK_PB_FIELDS_NOT_EMPTY macro for validating non-empty fields, with individual checks on list elements. {{<issue 22182>}}
• Disables the unused and unreliable NamespaceReplication tests temporarily for seamless operation. {{<issue 22457>}}
• Eliminates potential deadlock issue in DeleteOrHideTabletsAndSendRequests by removing unnecessary shared lock, and introduces a test for bug 22608. {{<issue 22618>}}
• Refreshes stack trace tracking UI endpoints to enable per-column sorting and optimize sorting script. {{<issue 22841>}}
• Introduces two new SOURCE_UNREACHABLE and SYSTEM_ERROR enums to enable more detailed error reporting from the Poller. {{<issue 22996>}}
• Refreshes stack trace tracking UI endpoints to enable per-column sorting and optimize sorting script. {{<issue 22841>}}
• Introduces two new SOURCE_UNREACHABLE and SYSTEM_ERROR enums to enable more detailed error reporting from the Poller. {{<issue 22996>}}
• Shifts tcmalloc profiling code to the Utils folder. {{<issue 22258>}}
• Enhances visibility of the Hidden state in Master/Tserver Tables UI by shifting its position more prominently to the State column. {{<issue 22521>}}

CDC

• Prevents addition of tables with enum array column to the CDC stream to avoid crashes during consumption. {{<issue 22897>}}
• Transforms the flag yb_enable_cdc_consistent_snapshot_streams from a preview into a default true auto flag. {{<issue 22984>}}
• Introduces three new yb-admin commands to remove a user table from a CDCSDK stream, descend dynamic table addition in a CDC stream, and validate CDC state for a particular stream, enhancing control over CDC streams. {{<issue 22876>}},{{<issue 22773>}}
• Adds a TServer flag, ysql_yb_default_replica_identity, for customizing default replica identity at table creation. {{<issue 22326>}}
• Introduces cdcsdk_enable_dynamic_table_addition flag to manage dynamic table additions in replication slot consumption model. {{<issue 22406>}}
• Introduces replication slot name for internal distinction between two consumption models in the code. {{<issue 22810>}}
• Allows creating an old model stream via yb-admin in upgraded environments, ensuring only one stream type per database. {{<issue 22894>}}
• Eliminates unnecessary NOTICE messages when setting yb_read_time from walsender, reducing message clutter. {{<issue 22379>}}

yugabyted

• Allows to specify multiple data directories using the new additional_data_dir configuration. {{<issue 22126>}}

Bug fixes

YSQL

• Fixes an error that occurs when decoding null values from a boolean column sorted as NULLS LAST in a secondary index. {{<issue 22121>}}
• Fixes YSQL upgrade single connection mode error preventing new connection attempts before the old ones are released. {{<issue 22283>}}
• Fixes memory leaks in pg_constraint/pg_attrdef local cache by adding a missing hash_destroy call in YbCleanupTupleCache. {{<issue 22262>}}
• Fixes a memory leak in catalog cache refresh during upgrade that led to higher memory consumption. {{<issue 22262>}}
• Resolves remaining memory leaks in CacheMemoryContext to stabilize cache memory after every catalog cache refresh. {{<issue 22262>}}
• Stops Batched Nest Loop (BNL) crashes by ensuring better indexing condition checks. {{<issue 21954>}}
• Refines the logic to accurately push down join clauses to batched index scans without causing conflicts. {{<issue 21878>}}
• Re-enables rechecking for RowCompareExpressions to accurately handle NULL inputs in scan bound calculations. {{<issue 22075>}}
• Grants BNL hashtable its own expression context to prevent data overwrites during query execution. {{<issue 21266>}}
• Allows for an enhanced readability and performance of yb_cost_index code, aiding in merging with the pg15 branch. {{<issue 21672>}}
• Fixes a bug in the index tuple width calculation for better YB base scans cost model. {{<issue 21892>}}
• Makes yb_get_range_split_clause robust using PG TRY CATCH block, ensuring YB backup doesn't fail. {{<issue 22356>}}
• Prevents coredumps by ensuring YSQL webserver destruction upon receiving a termination signal. {{<issue 18948>}}
• Reduces test iterations of PgCatalogVersionTest.SqlCrossDBLoadWithDDL in tsan/asan build to prevent timeouts. {{<issue 22475>}}
• Fixes memory leaks in ybcFetchNextHeapTuple by properly freeing the YBCStatus. {{<issue 22396>}}
• Replaces TRUNCATE with DELETE FROM in TestPgTransparentRestarts, enhancing its stability and reducing flakiness. {{<issue 22522>}}
• Allows CreateNamespaceIfNotExists function to retry on "already exists" error, preventing race conditions. {{<issue 22512>}}
• Removes the unused function Catalocustomeranager::WaitForDdlVerificationToFinish for clarity. {{<issue 22649>}}
• Resolves potential database OID collision with system_postgres by excluding reserved OID 65535 in allocation. {{<issue 22598>}}
• Prevents the IN expressions on single column from wrongly taking the tuple path, ensuring correct data processing. {{<issue 22704>}}
• Ensures bitmap scans correctly recheck all results and avoid excluding rows, improving accuracy of outcomes. {{<issue 22622>}}
• Fixes incorrect access to the scan plan's bind descriptor during tuple IN condition rechecks. {{<issue 22800>}}
• Allows the creation of new shared relations during YSQL upgrade to have a global impact by incrementing the catalog version across every database. {{<issue 22830>}}
• Corrects the YbGetOrdinaryColumnsNeedingPgRecheck function to return table column numbers instead of index numbers, preventing unnecessary data fetches and potential crashes or errors after dropping a column. {{<issue 22832>}}
• Fixes the issue when a separately created and later attached partition does not properly inherit the parent's primary key using ALTER TABLE ...ATTACH PARTITION. {{<issue 22562>}}
• Now allows for correct backward prefix-based scanning by eliminating the problematic kGroupEnd marker that was leading to inaccurate seek results. {{<issue 22615>}}
• Reduces unexpected log messages by not invoking YsqlDdlTxnCompleteCallback if all table 'pb_txn_id's in the DDL transaction verifier state are already cleared, avoiding potential deadlock situations in DDL atomicity. {{<issue 22882>}}
• Reduces sequence cache collision by incorporating both database and sequence OIDs as the entry key. {{<issue 22935>}}
• Resolves a detected deadlock during ALTER TABLE operations, enhancing test stability. {{<issue 22882>}}
• Prevents a crash related to memory release associated with TupleTableSlots in SubPlans during a Values Scan. {{<issue 22967>}}
• Eliminates the occurrence of "schema version mismatch" error following a DROP INDEX statement by introducing a delay in index deletion. {{<issue 22637>}}

DocDB

• Eliminates possible deadlock during setup replication by fixing the order in which locks are acquired. {{<issue 22376>}}
• Removes unnecessary flush during snapshot deletion, preventing write blocks. {{<issue 22369>}}
• Resolves a heartbeat metrics issue ensuring full xCluster error information is sent to the new master even during a leader failover, and makes tserver_heartbeat_metrics_interval_ms runtime updatable. {{<issue 22624>}}
• Converted the ysql_skip_row_lock_for_update to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{<issue 22057>}}
• Prevents duplicate entries in the sequence data filter used by Snapshot schedules, ensuring successful operations. {{<issue 22663>}}
• Resolves the issue of pg_locks query failure due to missing host node UUID in distributed transactions. {{<issue 22181>}}
• Clarifies memory division flags to reflect they are percentage of the process's hard memory limit, not total available memory. {{<issue 22423>}}
• Eliminates latency spikes in conflicting workloads by preventing redundant ProbeTransactionDeadlock RPCs. {{<issue 22426>}}
• Prevents premature metric destruction during Prometheus scrapes, resolving non-UTF8 character issues. {{<issue 22767>}}
• Ensures failed xCluster setup if the xCluster stream state update to ACTIVE does not occur. {{<issue 22601>}}

CDC

• Ensures deletion of MemoryContext after each GetChanges RPC to prevent memory leaks. {{<issue 22328>}}
• Prevents newly created indexes, materialized views, and non-user tables from being added to the Chang Data Capture (CDC) stream metadata. {{<issue 22808>}}
• Reduces resource usage by removing non-eligible tables, like indexes, from existing CDC SDK stream metadata, and releasing retention barriers. This change requires the master flag enable_cleanup_of_non_eligible_tables_from_cdcsdk_stream and limits processing to two non-eligible tables per namespace per run with cdcsdk_table_processing_limit_per_run. Introduces three yb-admin commands for managing CDC streams. {{<issue 22876>}},{{<issue 22835>}},{{<issue 22773>}}
• Fixes the issue of getting either 0 or a random time as the server's system clock in XLogData from the logical replication stream. {{<issue 22929>}}
• Fixes the segmentation fault in walsender for dynamic table addition by refreshing stored replica identities and preventing a race condition when creating dynamic tables. {{<issue 22273>}}
• Solves an issue where CDCSDK incorrectly deduces tablets as not interesting for stream before reaching the configured time limit. {{<issue 22383>}}
• Refines the logic to remove BEGIN record when no DML records are added, preventing potential virtual WAL crashes. {{<issue 21646>}}
• Addresses a race condition in dynamic table creation, enhancing stability during table and tablet initialization. {{<issue 22408>}}
• Increases timeout for deletion of slot entry from the state table to correct test failures in TSAN builds. {{<issue 22560>}}
• Resolves "could not open relation" error by updating slot creation method and simplifying yb_read_time logic. {{<issue 22398>}}
• Adds support for data types with dynamic OIDs in CDC to prevent crashes and allows tables with such columns to join the stream. {{<issue 23179>}}

yugabyted

• Eliminates inconsistent failures in TestSessionParameters.java by resetting roles to default instead of dropping them. {{<issue 21523>}}

v2024.1.0.0 - June 4, 2024 {#v2024.1.0.0}

Build: 2024.1.0.0-b129

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker:

docker pull yugabytedb/yugabyte:2024.1.0.0-b129

Highlights

Enhanced Postgres Compatibility Mode {{<tags/feature/ea>}}

We're pleased to announce the early access of the new Enhanced Postgres Compatibility Mode in the 2024.1.0.0 release. This mode enables you to take advantage of many new improvements in both PostgreSQL compatibility and performance parity, making it even easier to lift and shift your applications from PostgreSQL to YugabyteDB. When this mode is turned on, YugabyteDB uses the Read-Committed isolation mode, the Wait-on-Conflict concurrency mode for predictable P99 latencies, and the new Cost Based Optimizer {{<tags/feature/ea>}} that takes advantage of the distributed storage layer architecture and includes query pushdowns, LSM indexes, and batched nested loop joins to offer PostgreSQL-like performance.

Rollback after upgrade

Rolling back to the pre-upgrade version if you're not satisfied with the upgraded version is now {{<tags/feature/ga>}}. Refer to the Rollback phase for more information.

New features

• yugabyted

  • Set preferred regions. The preferred region handles all read and write requests from clients. Use the yugabyted configure data_placement command to specify preferred regions for clusters.
  • Connection management integration. With connection management enabled, the Nodes page of yugabyted UI displays the split of physical and logical connections.
  • Docker-based deployments. Improves the yugabyted Docker user experience for RF-3 deployments and docker container/host restarts.
  • Simplified PITR configuration.
  • Perform all admin operations using a pass through mechanism to execute yb-admin commands.

• DocDB Availability

  • Speed up local bootstrap. Faster rolling upgrades and restarts by minimizing table bootstrap time.
  • Hardening Raft. Reduced time window for re-tryable requests by honoring write RPC timeouts.

• Batched nested loop joins. A join execution strategy that is an improvement on Nested Loop joins that sends one request to the inner table per batch of outer table tuples instead of once per individual outer table tuple.

• Tablet splitting on range-sharded tables. Optimized the tablet split thresholds to speed up data ingestion.

• Catalog Caching. Reduce master requests during PostgreSQL system catalog refresh by populating YB-TServer catalog cache. {{<tags/feature/ea>}}

• Catalog Caching. Use TOAST compression to reduce PG catalog cache memory. Compressed catalog tuples when storing in the PG catalog cache to reduce the memory consumption. {{<tags/feature/ea>}}

• Index backfill stability improvements. Ensure timely notification to all nodes and PostgreSQL backends before initiating index backfill to prevent missing entries during index creation.

• Support for CDC with atomic DDL. In case of DDL being rolled back, CDC will not send records with rolled back schema.

• Wait-On Conflict Concurrency Control. Cross-tablet fairness in resuming "waiters". Resume waiters in a consistent order across tablets, when a set of transactions simultaneously wait on more than one intent/lock on various tablets.

• YSQL

  • Cost-based optimizer. Added support for cost based optimizer for YSQL. {{<tags/feature/ea>}}
  • DDL concurrency. Support for isolating DDLs per database. Specifically, a DDL in one database does not cause catalog cache refreshes or aborts transactions due to breaking change in another database.
  • DDL atomicity. Ensures that YSQL DDLs are fully atomic between YSQL and DocDB layers, that is in case of any errors, they are fully rolled back, and in case of success they are applied fully. Currently, such inconsistencies are rare but can happen.
  • ALTER TABLE support. Adds support for the following variants of ALTER TABLE ADD COLUMN:
    • with a SERIAL data type
    • with a volatile DEFAULT
    • with a PRIMARY KEY
  • Lower latency for large scans with size-based fetching. A static size based fetch limit value to control how many rows can be returned in one request from DocDB. {{<tags/feature/ea>}}

• Tablet limits. Depending on the available nodes and resources such as memory and CPU, YugabyteDB can automatically limit the total number of tables that can be created to ensure that the system can be stable and performant.

• Truncate support with PITR. The TRUNCATE command is now allowed for databases with PITR enabled.

• DocDB memory tracking enhancements. Memory tracking in DocDB to account for 90% of memory used.

• Enhanced Explain Analyze output. Explain Analyze when used with DIST option will also show the rows read from the storage layer, which can help diagnosing the query performance.

• Upgrade OpenSSL to 3.0.8 from 1.1.1. OpenSSL 1.1.1 is out of support. This feature upgrades YugabyteDB to FIPS compliant OpenSSL version 3.0.8.

Change log

Improvements

YSQL

• Enhances logging for DDL transaction conflicts and PG catalog version mismatches by including the DDL command tag and specific log details outside of the log_ysql_catalog_versions flag. {{<issue 20084>}}
• Displays distinct prefix keys explicitly in the explain output, enhancing the clarity of indexing for users. {{<issue 20831>}}
• Adds auto flag ysql_yb_enable_ddl_atomicity_infra to control DDL atomicity feature during the upgrade phase. {{<issue 21535>}}
• Allows YbInitPinnedCacheIfNeeded to only load the shared pinned cache, enhancing concurrent handling of DDLs in various databases. {{<issue 21635>}}
• Now logs global-impact DDL statements that increment all database catalog versions. {{<issue 21826>}}
• Adds a new YSQL view for YCQL statement metrics, allowing it to be joined with YCQL wait events in the yb_active_universe_history table. {{<issue 20616>}}
• Reduces per-backend memory consumption by reinstating TOAST compression for catalogue tables. {{<issue 21040>}}
• Avoids schema version mismatch errors during ALTER TABLE operations in cases where DDL atomicity is enabled. {{<issue 21787>}}
• Resolves schema version mismatch errors that occur after an ALTER TABLE operation due to DDL transaction verification in non-debug builds. {{<issue 21787>}}
• Introduces a new YSQL configuration parameter yb_enable_parallel_append to disable the unannounced feature parallel append. {{<issue 21934>}}
• Adds new columns to localhost:13000/statements for more comprehensive database management, including user and database IDs along with varied block level statistics. {{<issue 21735>}}
• Enables DDL atomicity feature by default by altering ysql_yb_ddl_rollback_enabled, report_ysql_ddl_txn_status_to_master, and ysql_ddl_transaction_wait_for_ddl_verification flags' defaults. {{<issue 22097>}}

YCQL

• Now throws an error when using the unsupported GROUP BY clause in YCQL with autoflag ycql_suppress_group_by_error available for compatibility issues. {{<issue 13956>}}

DocDB

• Introduces a new flag to toggle on or off recommended memory defaults for increased control over individual memory settings. {{<issue 22161>}}
• Boosts error messaging clarity when preview flags are not set in allowed_preview_flags_csv. {{<issue 21484>}}
• Shifts xCluster-related functions from Catalocustomeranager to XClusterSourceManager for easier management. {{<issue 21325>}}
• Adjusts TServer memory percentage from 50% to 48% for (4,8]GiB boxes and sets new recommendations for boxes over 16 GiB. {{<issue 20664>}}
• Returns the original error messages from CreateTable and CreateTablegroup instead of an incorrect "Database not found" message. {{<issue 21760>}}
• Updates AreNodesSafeToTakeDown to return earlier and deliver a readable error message when hitting a timeout, using a control flag, are_nodes_safe_to_take_down_timeout_buffer_ms with a default setting of 2 seconds. {{<issue 21855>}}
• Allows for faster failover in xCluster DR by skipping the cleanup process when skip_producer_stream_deletion is set on DeleteUniverseReplicationRequestPB. Safe for upgrades and rollbacks. {{<issue 22050>}}
• Adds a flag FLAGS_tablet_split_min_size_ratio to control tablet splitting based on SST file sizes, ensuring better control over tablet size imbalance. {{<issue 21458>}}

CDC

• Preserves CDC stream even when all associated tables are dropped, tying its lifecycle to the database. {{<issue 21419>}}
• Allows modification of the publication refresh interval using the cdcsdk_publication_list_refresh_interval_secs flag. {{<issue 21796>}}
• Introduces replica identity in CDC to populate before image records, allowing table-level before image information fetching and retaining in stream metadata. {{<issue 21314>}}
• Eliminates unnecessary NOTICE messages when setting yb_read_time from walsender, reducing message clutter. {{<issue 22379>}}
• Enables transaction state to be cleared promptly after a table is deleted, preventing table deletion from getting stuck and resulting in faster functionality. {{<issue 22095>}}

yugabyted

• Directly enables yb_enable_read_committed_isolation and ysql_enable_read_request_caching on yb-master and yb-tserver processes. {{<issue 22061>}}
• Simplifies yugabyted by dropping Python2 support and transitioning the script to use Python3, replacing deprecated distutils package with shutil. {{<issue 22072>}},{{<issue 21409>}}

Bug fixes

YSQL

• Avoids failure when upgrading from version 2.14/2.16 to 2.20 by introducing a check to ensure pggate can handle RPC metrics sidecar before sending Scanned Rows count. {{<issue 21229>}}
• Safeguards against an out-of-bounds memory write error in HashAggregate plan node with duplicate columns by rectifying array size allocation. Imported tests ensure the fix's robustness. {{<issue 21122>}}
• Disables bitmap scan by default to prevent unwarranted selection due to lower CBO costs. {{<issue 21479>}}
• Allows YSQL DDL operations to wait for rollback/roll-forward operations to finish before proceeding. {{<issue 20033>}}
• Reduces unnecessary log messages when catalog_version_table_in_perdb_mode is set to true. {{<issue 21481>}}
• Corrects an issue where certain unbatchable filters weren't detected during indexpath formation when indexpath accepted batched values from multiple relations. Requires backports to 2.20 and 2.18. {{<issue 21292>}}
• Adds network latency cost to startup cost, yielding more accurate cost calculations in small tables. {{<issue 20898>}}
• Allows more accurate modeling of base scan costs by taking into account the impact of storage index filters on secondary index. {{<issue 20635>}}
• Renames the YSQL configuration parameter ddl_rollback_enabled to yb_ddl_rollback_enabled for specificity. {{<issue 21480>}}
• Resolves colocation option issues in table creation linked to table rewriting and partitioning. Enhances the defGetBoolean function to parse string values "0" and "1" as false and true respectively, and shifts a verification step to a earlier spot in the CREATE TABLE execution path, ensuring successful table partition creation. {{<issue 20302>}},{{<issue 20914>}}
• Exposes the YSQL configuration parameter yb_enable_optimizer_statistics as a flag ysql_yb_enable_optimizer_statistics. {{<issue 21650>}}
• Corrects checks in YbIsScanCompatible to ensure the right-hand side (RHS) of all bound index conditions, not just inequalities, fits into the left-hand side (LHS) datatype. {{<issue 21758>}}
• The deadlock issue occurring when both a table and its index are deleted concurrently in yb-master has been resolved. {{<issue 21663>}}
• Fixes a bug that caused incorrect setting of global catalog version mode on TServer start. {{<issue 21850>}}
• Deprecates the unused enable_pg_savepoints flag to prevent irregular system behaviour. {{<issue 20918>}}
• Refines the YbGetOrdinaryColumnsNeedingPgRecheck condition to align with the ybIsTupMismatch implementation, ensuring Postgres rechecks index conditions when the "preliminary check" is skipped due to an invalid target key attnum. {{<issue 21451>}}
• Reduces the frequency of schema version mismatch errors during consecutive DDL operations by ensuring the up-to-date schema is fetched. {{<issue 21706>}}
• Allows usage of YsqlDdlRollbackEnabled in pggate C++ code by correctly passing the result of YbDdlRollbackEnabled, reducing DDL atomicity g-flag issues in RF 1 clusters. {{<issue 21706>}}
• Ensures pushed down RowCompareExpressions correctly enforce non-null column references, rectifying previous behavior and enhancing data accuracy. {{<issue 21847>}}
• The postgres process no longer crashes when running a "show all" command due to correct placement of the yb_enable_ddl_atomicity_infra description. {{<issue 21947>}}
• Adds a new flag ysql_min_new_version_ignored_count to prevent a TServer crash caused by the downward shift in yb-master's catalog version, often surfacing post a PITR restore operation. {{<issue 21776>}}
• Adjusts the value of YB_AT_REWRITE_ALTER_PRIMARY_KEY to prevent flag clashes and accommodate future upstream PG flags. {{<issue 22086>}}
• Fills in the "relation" column in pg_locks with the correct table OID after a table rewrite. {{<issue 22081>}}
• Prevents query layer retries for multi-statement queries to avoid redoing whole queries, ensuring idempotence. {{<issue 21361>}}
• Prevents unnecessary CPU cycles and log flooding by not reading pg_yb_catalog_version when enable_ysql=false. {{<issue 22213>}}
• Corrects the log message for successful column drop operations, ensuring accurate representation of DDL operations. {{<issue 22243>}}

YCQL

• Allows the deletion of the Cassandra role in ycqlsh without it regenerating upon cluster restart, by adding a flag to mark if the role was previously created. {{<issue 21057>}}

DocDB

• Ensures Create Table operation fails if Alter Replication encounters an error, enhancing the reliability of replication setup. {{<issue 21732>}}
• Converted the ysql_skip_row_lock_for_update to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{<issue 22057>}}
• Modifies memory consumption calculations for pending operations to ensure accurate rejection of new writes at bootstrap, preventing loading failures. {{<issue 21254>}}
• Trims large error messages in AsyncRpc::Failed to prevent hitting memory limit and resulting unavailability. {{<issue 21402>}}
• Renames and updates the description of the flag min_secustomerent_size_to_rollover_at_flush for clarity. {{<issue 21691>}}
• Changes the class of enable_automatic_tablet_splitting flag from kLocalPersisted (class 2) to kExternal (class 4) to eliminate setup issues with xCluster configurations. {{<issue 22088>}}
• Fixes a segmentation fault in yb-master by checking for a null pointer before dereferencing it, addressing an issue in the CDC run on 2.23.0.0-b37-arm. {{<issue 21648>}}
• Allows DML operations on non-replicated databases and blocks DML only on databases in transactional xCluster replication STANDBY mode. Now only databases part of an inbound transactional xCluster replication group in the xCluster safe time map will have DML operations blocked. Also, certain attributes are moved from TServer to TserverXClusterContext. {{<issue 21245>}}
• Adds a TSAN suppression to manage the apparent race condition in the function boost::regex_match. {{<issue 21585>}}
• Eliminates potential FATAL errors during reported tabletPB creation by ensuring retrieval of schema version is atomic. {{<issue 21340>}}
• Enables the session to outlive the callback by holding a shared pointer to it, preventing potential crashes during concurrent DML queries. {{<issue 21103>}}
• Prevents yb-master crash by ensuring background task isn't deleted before the callback is invoked. {{<issue 21773>}}
• Enables callback completion wait in PollTransactionStatusBase during shutdown to prevent unexpected process termination. {{<issue 21773>}}
• Allows viewing of the RPC bind addresses in the master leader UI, especially beneficial in cases like k8s where the RPC bind address with the pod DNS is more useful than the broadcast address. {{<issue 21959>}}
• Reduces unnecessary logging during checkpoint operations by lowering INFO level logs to DEBUG_LEVEL, enhancing log readability. {{<issue 21658>}}
• Prevents fatal errors by skipping ReserveMarker/AsyncAppend if the tablet peer has already been shut down. {{<issue 21769>}}
• Enhances YSQL operation by refining task shutdown procedures and avoiding unnecessary task aborts. {{<issue 21917>}}
• Enhances load balancer efficiency by refining validation logic to block tablet replica additions only for those with a pending delete in progress on the same server, avoiding potential slowdowns during mass tablet replica moves. {{<issue 21806>}}
• Avoids multiple destruction of the same database connection, preventing system crashes due to simultaneous connection failures. {{<issue 21738>}}
• Stops fatal errors caused by the re-use of remote log anchor session during remote bootstrap from a non-leader peer. This fix ensures shared pointers are accurately tracked for tablet_peer objects using the = operator, preventing unintentional destruction of underlying objects. {{<issue 22007>}}
• Corrects a bug causing some tablet metrics to display incorrect metric_type attribute. {{<issue 21608>}}
• Enables the skip_table_tombstone_check for colocated tables to prevent errors. {{<issue 22115>}}
• Initializes prev_op to UNKNOWN to prevent AlmaLinux 8 fastdebug gcc11 compilation failures. {{<issue 21811>}}
• Delays min_running_ht initialization until after the successful completion of tablet bootstrap to prevent unexpected behaviors. {{<issue 22099>}}
• Resolves the issue of pg_locks query failure due to missing host node UUID in distributed transactions. {{<issue 22181>}}
• Eliminates latency spikes in conflicting workloads by preventing redundant ProbeTransactionDeadlock RPCs. {{<issue 22426>}}

CDC

• Ensures deletion of MemoryContext after each GetChanges RPC to prevent memory leaks. {{<issue 22328>}}
• Fixes a memory leakage issue in the walsender process by deep freeing the cached record batch after streaming to the client. {{<issue 21530>}}
• Adds more debug logs in the walsender to aid in investigating issues like linked data loss. {{<issue 21465>}}
• Adds more debug logs for stress run debugging, skips RollbackToSubTransaction RPC to local TServer if not needed, and enhances debugging of the ListReplicationSlots function. {{<issue 21780>}},{{<issue 21519>}},{{<issue 21652>}}
• Removes table level attributes from CDCSDK metrics to avoid TServer crash due to failed DCHECK assertion. {{<issue 22142>}}
• Fixes the segmentation fault in walsender for dynamic table addition by refreshing stored replica identities and preventing a race condition when creating dynamic tables. {{<issue 22273>}}
• Solves an issue where CDCSDK incorrectly deduces tablets as not interesting for stream before reaching the configured time limit. {{<issue 22383>}}
• Enables support for streaming update operations via walsender, enhancing PG compatible logical replication support. Now executes schema changes in the logical replication protocol and maintains a record of changes in each table's read_time_ht hybrid time in the PG catalog. Includes handling late ALTER TABLE responses and addressing incomplete cleanup in the case of a stream creation failure. This feature is disabled under test flag ysql_TEST_enable_replication_slot_consumption. {{<issue 20725>}}
• Prevents failures in decoding change events by refreshing cached_schema_details when executing a new GetChanges request if the client indicates a necessity for the schema. {{<issue 20698>}}

yugabyted

• Prevents SyntaxWarning and exceptions when incorrect advertise_address is given by adjusting string literals and adding check for errors. {{<issue 22552>}},{{<issue 22210>}},{{<issue 22230>}}

Other

• Fixes hidden split parent tablets wrongly appearing as leaderless in the master's leaderless tablet endpoint, ensuring accurate load balance status and preventing potential issues with Point-in-Time Recovery (PITR) operations. {{<issue 21371>}}