What's new in the YugabyteDB v2.20 LTS release series

Release announcements

• YugabyteDB 2.20 Unveiled: Elevating Change Data Capture for Cloud Native Applications

Release notes

{{< tip title="LTS - Feature availability" >}} All features in stable releases are considered to be {{<tags/feature/ga>}} unless marked otherwise. {{< /tip >}}

What follows are the release notes for the YugabyteDB v2.20 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.20 release series.

For an RSS feed of all release series to track the latest product updates, point your feed reader to the RSS feed for releases.

{{< warning title="Technical Advisories" >}}

• Impacts: v2.20.0.0 to 2.20.1.2 - {{<ta 20648>}} - Index update can be wrongly applied on batch writes
• Impacts: v2.20.1.0 to 2.20.1.3 - {{<ta 20827>}} - Correctness issue for queries using SELECT DISTINCT

{{< /warning >}}

v2.20.12.0 - October 6, 2025 {#v2.20.12.0}

Build: 2.20.12.0-b30

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.12.0-b30

Improvements

YSQL

• Ensures all COPY FROM commands in a transaction block use one transaction unit. {{<issue 27061>}}

DocDB

• Adds data block consistency check to sst_dump tool. {{<issue 27233>}}
• Adds flag to allow tablet writes after a compaction failure. {{<issue 27269>}}
• Adds remove_corrupt_data_blocks_unsafe flag for yb-ts-cli compact_tablet command. {{<issue 27381>}}
• Ensures only voter peers are considered for leadership during a leader stepdown. {{<issue 27568>}}
• Fixes overflow in FormatBytesAsStr to prevent large memory allocations. {{<issue 27924>}}
• Upgrades OpenSSL to version 3.0.16 and enhances aarch64 support. {{<issue 27491>}}

Bug fixes

YSQL

• Ensures secondary index scans remain consistent during concurrent deletes in YSQL. {{<issue 17159>}}
• Reverts a previous update to stabilize INSERT ON CONFLICT features on partitioned tables. {{<issue 25147>}}
• Ensures accurate EXPLAIN output for forward scans with aggregate pushdown. {{<issue 27642>}}
• Eliminates spurious "Attempted to restart" errors during multi-page reads. {{<issue 22821>}}
• Ensures the correct renew strategy for IsRenewRequired works as expected. {{<issue 28226>}}

DocDB

• Ensures accurate replication for transactional writes on xCluster range partitioned tables. {{<issue 27380>}}
• Ensures post-split compactions complete even if background compactions are in progress. {{<issue 27426>}}
• Ensures consistent data reads by setting propagated safe time only after successful operation replication. {{<issue 23696>}}
• Prevents incorrect query results due to invalid seek keys in internal iterator operations. {{<issue 27641>}}
• Enhances the stability of document reads by validating key prefixes before processing. {{<issue 27923>}}
• Adjusts DocRowwiseIterator to correctly include nulls for NULLS FIRST sorting. {{<issue 27031>}}
• Ensures tablets remain effectively managed during transition from RF1 to higher replication factors. {{<issue 24575>}}
• Replaces the deprecated exec_program command with execute_process in CMake, resolving issue 20481 and eliminating potential warning CMP0153 for developers. {{<issue 20481>}}

v2.20.11.1 - July 14, 2025 {#v2.20.11.1}

Build: 2.20.11.1-b2

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.11.1-b2

Bug fix

YSQL

• Eliminates spurious ""Attempted to restart"" errors during multi-page reads. {{<issue 22821>}}

v2.20.11.0 - July 2, 2025 {#v2.20.11.0}

Build: 2.20.11.0-b34

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.11.0-b34

Improvements

YSQL

• Enhances error reporting by including index names for missing rows. {{<issue 26819>}}

YCQL

• Allows setting NULL in YCQL JSONB column values using UPDATE statements. {{<issue 5907>}}

DocDB

• Upgrades OpenSSL to version 3.0.16, maintaining FIPS at 3.0.8. {{<issue 27491>}}
• Deprecates the load_balancer_count_move_as_add flag to simplify cluster balancing. {{<issue 26259>}}

Bug fixes

YSQL

• Reduces XID usage by generating one per REFRESH MATERIALIZED VIEW CONCURRENTLY operation. {{<issue 26205>}}
• Ensures consistent collation behavior between YSQL and DocDB by aligning default settings and adjusting expression pushdown rules. {{<issue 26385>}}
• Ensures correct column mapping for partitioned tables during INSERT ON CONFLICT updates. {{<issue 14895>}},{{<issue 25521>}}
• Prevents crashes by handling non-variable expressions in single-row updates or deletes. {{<issue 26536>}}
• Enhances yb_index_check to verify materialized view indexes' consistency. {{<issue 26639>}}
• Ensures ysql_dump maintains enum sort order during backup and restore. {{<issue 26645>}}
• Allows restoration of old backups with enum types without errors by reverting to warnings and auto-generated OIDs. {{<issue 26859>}}
• Logs odd pg_enum OID during restore if sortorder is missing, enhancing troubleshooting. {{<issue 26859>}}

DocDB

• Ensures accurate replication for transactional writes on xCluster range partitioned tables. {{<issue 27380>}}
• Ensures all transaction batches update schema versions, preventing data readability issues from aggressive metadata cleanup. {{<issue 25106>}}
• Fixes handling of db_max_flushing_bytes to properly limit memory usage under high write loads. {{<issue 26916>}}
• Prevents data loss by ensuring databases are flushed before updating flush markers. {{<issue 26910>}}
• Prevents deadlocks during background compaction and transaction loading. {{<issue 26880>}}
• Prevents duplicate connection destruction attempts during concurrent failures. {{<issue 21738>}}

CDC

• Ensures only relevant COMMIT records are streamed, fixing gRPC connector crashes. {{<issue 26861>}}
• Prevents CDC crashes by logging a warning for dropped indexes on colocated tables. {{<issue 26834>}}

yugabyted

• Preserves the universe key locally after enabling EAR for recovery scenarios. {{<issue 26552>}}

v2.20.10.0 - April 1, 2025 {#v2.20.10.0}

Build: 2.20.10.0-b29

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.10.0-b29

Improvements

YSQL

• Ensures cached plans correctly depend on user roles, addressing {{<cve "CVE-2024-10976">}}. {{<issue 25330>}}

Bug fixes

YSQL

• Ensures Batched Nested Loop joins correctly check for hashable join clauses. {{<issue 25917>}}
• Fixes data race in regex pushdown by using thread-local variables. {{<issue 24237>}}
• Fixes incorrect aggregate column references in flattened subqueries. {{<issue 24512>}}
• Enhances regex to correctly identify test names with dots in Java PG regress tests. {{<issue 26129>}}
• Enhances index scans and partition pruning for BOOLEAN conditions. {{<issue 26266>}}

DocDB

• Adds mtime and checksum to block cache key prefix to prevent conflicts. {{<issue 20852>}}
• Enhances transactional xCluster accuracy by using majority replicated OpId for ApplySafeTime calculations. {{<issue 26043>}}
• Prevents crash during transaction abort by adjusting lock handling. {{<issue 25689>}}
• Prevents false conflict detection in snapshot isolation operations. {{<issue 26142>}}

v2.20.9.0 - January 28, 2025 {#v2.20.9.0}

Build: 2.20.9.0-b47

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.9.0-b47

Improvements

YSQL

• Enhances initdb logging, correctly handles process termination, and fixes memory leaks on abort signals. {{<issue 24813>}}
• Prevents catalog version bumps on no-op ALTER ROLE commands, saving system resources. {{<issue 24390>}}

YCQL

• Tracks microsecond precision timestamps in YCQL with new metric cql_microseconds_timestamps_used. This tracking occurs if the cql_revert_to_partial_microsecond_support flag is true. {{<issue 24776>}}

DocDB

• Limits concurrent remote bootstraps per TServer using load_balancer_max_inbound_remote_bootstraps_per_tserver flag. {{<issue 2426>}}
• Reduces unnecessary attribute map creations during Prometheus metric aggregation. {{<issue 24405>}}
• Enhances RBS throttling by focusing on active fetch sessions and adjusts expiration logic. {{<issue 21563>}},{{<issue 24031>}}
• Introduces limits on Prometheus metric entries per scrape to prevent server overload. {{<issue 18089>}}
• Enhances stability by fixing a missing null check in tablet lookup processes. {{<issue 25145>}}

CDC

• Automatically removes metrics when a CDC stream is deleted. {{<issue 24876>}}

Bug fixes

YSQL

• Reduces memory usage during YugabyteDB connection startup. {{<issue 24925>}}
• Fixes foreign key checks to avoid errors with unique indexes. {{<issue 24663>}}
• Reduces memory usage spikes by using a new memory context for RelationBuildTriggers. {{<issue 24951>}}
• Ensures accurate catalog version tracking during concurrent index creation. {{<issue 24775>}}
• Fixes inconsistency in index entries during concurrent writes in the backfill phase. {{<issue 25250>}}
• Automatically restarts the postmaster to prevent stuck processes when a background worker crashes. {{<issue 24706>}}

YCQL

• Allows users to control error suppression for GROUP BY in YCQL using ycql_ignore_group_by_error flag. {{<issue 25030>}}

DocDB

• TServers now reject remote bootstrap requests for previously deleted tablets. {{<issue 24574>}}
• Sets a default 10MB max size for RocksDB MANIFEST files, now configurable via flags. {{<issue 11117>}}
• Enhances visibility for unexpected rate limiter sleep times in RBS threads. {{<issue 24342>}}
• Prevents crashes when calling ListMasterServers RPC on a master by returning an error. {{<issue 20372>}}
• Ensures continuous xCluster task operation even if the master temporarily loses the lease but retains leadership. {{<issue 25480>}}
• Prevents crashes during compaction when ysql_enable_packed_row is off and existing packed rows are present. {{<issue 24545>}}
• Fixes error for select queries in large tables by adjusting backward scan bounds. {{<issue 24824>}}
• Ensures consistency between Slice::Less and Slice::compare methods. {{<issue 25319>}}
• Fixes shutdown path issues to prevent fatal errors during RPC completion. {{<issue 24524>}}
• Fixes metric attribute collisions and ensures correct metric types for tablets. {{<issue 21608>}}
• Prevents TServer crashes on executing specific queries by handling NULL pointers effectively. {{<issue 23004>}}

CDC

• Excludes expired or irrelevant entries from metrics calculations for more accurate reporting. {{<issue 24975>}}
• Ensures accurate history retention for various replica identities in CDC. {{<issue 25193>}}

v2.20.8.1 - December 16, 2024 {#v2.20.8.1}

Build: 2.20.8.1-b2

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.8.1-b2

This is a YugabyteDB Anywhere-only release, with no significant changes to YugabyteDB.

v2.20.8.0 - December 2, 2024 {#v2.20.8.0}

Build: 2.20.8.0-b53

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.8.0-b53

New features

CDC

• Enables tablet splitting on tables under CDCSDK stream by default using the GFLAG enable_tablet_split_of_cdcsdk_streamed_tables. {{<issue 24190>}}

Improvements

YSQL

• Allows preloading of foreign key lists in relcache, avoiding on-demand master fetches, controlled by yb_enable_fkey_catcache. {{<issue 23686>}}
• Reduces the YSQL catalog lease period to 10 seconds with a new flag master_ts_ysql_catalog_lease_ms improving response times during master leader changes and minimizing disruptions. {{<issue 21249>}}

YCQL

• Resolves timestamp precision mismatches in YCQL, aligning inserts and index scans to millisecond precision. {{<issue 11052>}},{{<issue 23476>}}

DocDB

• Allows reduction of thread stack size to avoid Linux hugepage backing. {{<issue 23927>}}
• Blocks writes based on total bytes being flushed, not just memtable count. {{<issue 22571>}}
• Enhances load balancer to prioritize under-replicated tablets first. {{<issue 20263>}}
• Prevents overloading by correctly categorizing bootstrapping tablets in load balancing. {{<issue 23487>}}
• Upgrades non-FIPS OpenSSL to 3.0.15 and removes CentOS 7 builds. {{<issue 24436>}}
• Allows explicit addition of gFlags to gflag_allowlist.txt for secure callhome data collection, plus version_info in TServer data. {{<issue 24103>}}

Bug fixes

YSQL

• Modifies backup/restore process to skip column name checks for indexes, allowing for successful restoration even with renamed columns. {{<issue 24207>}}
• Enhances Postgres restart stability by refining shutdown handling and lock file management. Includes killing only Postgres-associated PIDs, using SIGQUIT for immediate shutdowns to reduce hang-ups, and monitoring for stuck processes on macOS {{<issue 24396>}}
• Fixes crashes caused by using a RowComparisonExpression on a reordered primary key index. {{<issue 23824>}}
• Streamlines ysql_dump process by eliminating faulty statements for colocated table unique constraint, enhancing database reliability. {{<issue 24057>}}
• Alters encoding setup order to ensure UTF8 use instead of SQL_ASCII and adds pg_collation to preloaded tables to prevent specific errors. {{<issue 24149>}}
• Adds HELP and TYPE metadata for each metric in the YSQL Prometheus metrics endpoint for better metrics accessibility. {{<issue 23578>}}
• Prevents a possible system crash in YSQL backends manager by ensuring essential checks are in place before using the job database object. {{<issue 20060>}}
• Reduces PostgreSQL connection startup timeouts in geo-distributed clusters with a new wait_for_ysql_backends_catalog_version_master_tserver_rpc_timeout_ms GFlag, increasing the default timeout value to 60s from 30s. This alteration only impacts one specific RPC - WaitForYsqlBackendsCatalogVersion, not all RPCs, which should diminish time-out incidents. {{<issue 18228>}}
• Minimizes CREATE INDEX delays during master leader changes or cluster startups by ensuring the new master leader waits 10 seconds before serving YSQL Backends Catalog requests after startup or leader change. {{<issue 21251>}}
• Enhances performance during nested loop joins by ensuring correct limit resets in colocated index scans. {{<issue 24560>}}
• Reduces memory usage spikes by using a new memory context for RelationBuildTriggers. {{<issue 24951>}}

YCQL

• Eliminates unexpected error during shell type definition update across multiple sessions. {{<issue 24217>}}

DocDB

• Prevents tablet bootstrap from getting stuck when replaying a truncate operation. {{<issue 23243>}}
• Allows the packed row size to exceed its limit during repacking, preventing TServer crash after alter table add column commands with a default value. {{<issue 24050>}}
• Delays min_running_ht initialization until after tablet bootstrap completes. {{<issue 22099>}}
• Prevents incorrect marking of tablets as over-replicated during startup. {{<issue 21135>}}

v2.20.7.2 - November 22, 2024 {#v2.20.7.2}

Build: 2.20.7.2-b1

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.7.2-b1

Bug fix

YSQL

• Reduces memory usage spikes by using a new memory context for RelationBuildTriggers. {{<issue 24951>}}

v2.20.7.1 - October 16, 2024 {#v2.20.7.1}

Build: 2.20.7.1-b10

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.7.1-b10

Improvements

DocDB

• Allows reduction of thread stack size to avoid Linux huge page backing. {{<issue 23927>}}

Bug fixes

DocDB

• Allows the packed row size to exceed its limit during repacking, preventing TServer crash after alter table add column commands with a default value. {{<issue 24050>}}

v2.20.7.0 - September 30, 2024 {#v2.20.7.0}

Build: 2.20.7.0-b58

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.7.0-b58

Improvements

YSQL

• Introduces the yb_make_next_ddl_statement_nonincrementing configuration parameter to reduce catalog version increments during many consecutive DDLs, lowering memory and latency spikes. {{<issue 23786>}}
• Added yb_make_next_ddl_statement_nonincrementing to YbDbAdminVariables for yb_db_admin role. {{<issue 23786>}}

YCQL

• Updates YCQL tables system_schema.functions and system_schema.aggregates to align with Cassandra, aiding Cortex migration. {{<issue 3893>}}

DocDB

• Enables reduction of duplicate code and custom flag filtering through GetFlagInfos relocation. {{<issue 23632>}}
• Adds emergency_repair_mode flag and yb-admin commands dump_sys_catalog_entries and write_sys_catalog_entry to address CatalogEntity data corruption issues. {{<issue 23098>}}

CDC

• Allows dynamic table addition alongside table removal from CDC streams without needing to disable or restart. {{<issue 23581>}}
• Adds a yb-admin option to disable dynamic table addition when creating new streams. {{<issue 23482>}}
• Allows automatic cleanup for ineligible tables on CDC streams and improves the removal process by fetching tablet list from a local copy instead of the state table, improving performance. Updates include removal of flag protection for Step-1 and the replacement of the preview flag. {{<issue 23806>}}
• Enables asynchronous removal of user tables from CDC streams by adjusting how background threads process and persist stream metadata. {{<issue 23700>}}

yugabyted

• Stops sending flag details in the callhome diagnostics to eliminate redundant data. {{<issue 24029>}}

Other

• Updates the callhome URL to use HTTPS for enhanced security. {{<issue 23947>}}

Bug fixes

YSQL

• Fixes an issue related to regular expression pushdown causing inconsistency and crashes by introducing thread local cache. {{<issue 22989>}}
• Resets catalog read time after table prefetching to prevent outdated readings for on-demand loading from a master. {{<issue 23421>}}
• Corrects IndexScan's indextlist to improve accuracy of variable references in subquery scenarios. {{<issue 22533>}}
• Prevents retry of schema mismatch errors in batched execution mode to maintain transaction atomicity. {{<issue 23843>}}
• Resolves potential memory corruption by mutating pushdown expressions outside the per tuple context. {{<issue 23461>}}
• Replaces deprecated function shared_ptr::unique to resolve compilation errors on macOS. {{<issue 24012>}}
• Removes unnecessary DCHECK for smoother rolling upgrades from release 2.20, enhancing ysql_enable_db_catalog_version_mode flag functionality. {{<issue 23462>}}

DocDB

• Limits response size by omitting per-object details from SysSnapshotEntryPB in ListSnapshotSchedules RPC. {{<issue 23518>}}
• Reduces the risk of deadlocks when making active read/write requests to a table right after a tablet-split. {{<issue 23747>}}
• Simplifies handling of large RPC responses by catching errors earlier and appropriately using narrow_cast. {{<issue 22301>}}
• Switches write query duration from CoarseMonoClock to MonoClock for precise nanosecond granularity. {{<issue 20335>}}
• Refactors to remove an assumption that every TServer hosts tablets, preventing crashes in certain circumstances. {{<issue 20230>}}
• Sets the close_timestamp_micros field for partially copied log segments to prevent time-based policy violation. {{<issue 23335>}}
• Eliminates persistent logging issue after dropping a colocated table by removing unnecessary logs. {{<issue 22037>}}
• Corrects leader balancing for geo-partitioned tables to prevent early exits and ensure accurate global leader sorting. {{<issue 23886>}}

CDC

• Prevents TServer crash by giving each thread its own copy of SchemaDetailsMap during concurrent GetChanges calls. {{<issue 23394>}},{{<issue 22962>}}
• Reduces disk IO overhead by optimizing the tablet-meta update process in CDCService. {{<issue 22805>}}
• Refactors CDCSDK to prevent data loss during workload with single shard transactions by ensuring WAL messages are committed before being replicated. {{<issue 19294>}}
• Introduces three new yb-admin commands for managing user tables in a CDC stream and addresses a race condition issue by removing an optimization in the UpdateCheckpointForTabletEntriesInCDCState method. {{<issue 22876>}},{{<issue 22835>}},{{<issue 22773>}}
• Obsoletes expired or unpolled tables from CDC stream and state table, reducing unnecessary resource retention and potential blocking of split tablet deletion. {{<issue 23367>}}
• Eliminates occurrence of master crashes after table drop by handling non-eligible tables cleanup in CDC stream loading. {{<issue 23278>}}

yugabyted

• Offers corrected DB OID in the function calculate_table_size, now matching YBCGetDatabaseOid(rel) for improved reliability. {{<issue 20030>}}

v2.20.6.0 - August 23, 2024 {#v2.20.6.0}

Build: 2.20.6.0-b66

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.6.0-b66

Improvements

YSQL

• Enhances logging for DDL transaction conflicts and PG catalog version mismatches by including the DDL command tag and specific log details outside of the log_ysql_catalog_versions flag. {{<issue 20084>}}
• Avoids renaming DocDb tables during legacy rewrite operations to prevent issues with backup/restore and improves handling of failed ADD/DROP primary key, ALTER TYPE operations. {{<issue 22802>}}

DocDB

• Allows asynchronous DNS cache updating and resolution retry upon failure to reduce RPC call delays and prevent unexpected leadership changes. {{<issue 22930>}},{{<issue 22311>}}

CDC

• Transforms the flag yb_enable_cdc_consistent_snapshot_streams from a preview into a default true auto flag. {{<issue 22984>}}

yugabyted

• Updates the cqlsh version to v3.10-yb-19 for enhanced functionality. {{<issue 22982>}}
• Reduces collect_logs command failures by removing the yugabyted running check even when the yugabyted process is not running. {{<issue 23210>}}

Bug fixes

YSQL

• Allows YB Admins to run pg_locks without requiring superuser status. {{<issue 23266>}}
• Now allows for correct backward prefix-based scanning by eliminating the problematic kGroupEnd marker that was leading to inaccurate seek results. {{<issue 22615>}}
• Reduces sequence cache collision by incorporating both database and sequence OIDs as the entry key. {{<issue 22935>}}
• Allows faster data inserts into tables with identity columns. {{<issue 22837>}}
• Prevents a crash related to memory release associated with TupleTableSlots in SubPlans during a Values Scan. {{<issue 22967>}}
• Reduces prefix length in the index when using distinct index scan with included columns. {{<issue 22822>}}
• Eliminates unnecessary waiting for concurrent transactions in the DEFERRABLE mode for READ ONLY serializable transactions. {{<issue 23120>}}

DocDB

• Allows restoring a snapshot schedule from a time just before the oldest snapshot, improving usability. {{<issue 21269>}}
• Fixes issue of scans not honoring timeouts, preventing indefinite reads and reducing CPU usage. {{<issue 21829>}}
• Adds validation to RPCs DeleteSnapshot and RestoreSnapshot to prevent deletion or use of snapshots involved in ongoing processes. {{<issue 23055>}}
• Allows the persistent mapping of source-target schema versions when a new table is added to a colocated database, thus preventing replication from stalling after TServer restarts. {{<issue 23188>}}
• Updates cotable IDs in flushed frontiers during a snapshot restore, preventing potential post-restore issues. {{<issue 23047>}}
• Eliminates latency spikes in conflicting workloads by preventing redundant ProbeTransactionDeadlock RPCs. {{<issue 22426>}}
• Ensures object drops correctly cascade to dependent columns in the DocDB table, preventing inconsistencies. {{<issue 22874>}}
• Allows large bytes to be requested on RateLimiter, preventing indefinite call stalling. {{<issue 23173>}}
• Removes extra reads during the processing of INSERT INTO ...RETURNS STATUS AS ROW for CQL tables with a specific primary key, improving system load and efficiency. {{<issue 23330>}}

CDC

• Resolves an issue preventing newly created tables from being added to the stream metadata and CDC state table after an existing table is dropped, by considering streams in DELETING_METADATA state as well as ACTIVE state during dynamic table addition. {{<issue 20428>}}
• Prevents newly created indexes, materialized views, and non-user tables from being added to the CDC stream metadata. {{<issue 22808>}}
• Introduces three new yb-admin commands to manage CDC streams, including disabling dynamic table addition, removing specific user tables from CDC streams, and validating CDC state table entries. {{<issue 22876>}},{{<issue 22835>}},{{<issue 22773>}}
• Allows removal of expired or unpolled tables from CDC stream metadata and state table, preventing potential blockages. {{<issue 23367>}}
• Conversion of cdcsdk_enable_cleanup_of_expired_table_entries to an auto flag now allows more consistent handling and cleanup of expired tables. {{<issue 23367>}}
• Refines the logic to remove BEGIN record when no DML records are added, preventing potential virtual WAL crashes. {{<issue 21646>}}
• Allows handling of non-eligible table cleanup in CDC stream loading even after table drop, preventing master crash. {{<issue 23278>}}

v2.20.5.0 - July 10, 2024 {#v2.20.5.0}

Build: 2.20.5.0-b72

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.5.0-b72

Improvements

YSQL

• Adds new columns to localhost:13000/statements for more comprehensive database management, including user and database IDs along with varied block level statistics. {{<issue 21735>}}

DocDB

• Introduces utility to dump top heap stacks when memory limit is exceeded for enhanced debugging. {{<issue 21396>}}
• Solves the potential deadlock in DeleteTabletListAndSendRequests, while also preventing xrepl tablets deletion due to multiple calls to the function. {{<issue 22618>}},{{<issue 22608>}}
• Shifts tcmalloc profiling code to the Utils folder. {{<issue 22258>}}

CDC

• Promotes the flag yb_enable_cdc_consistent_snapshot_streams from a preview flag to a default true auto flag. {{<issue 22984>}}
• Introduces three new yb-admin commands for managing user tables in a CDCSDK stream, enabling you to disable dynamic table addition, remove specific user tables from the CDC stream, and validate CDC state table entries for a stream. {{<issue 22876>}},{{<issue 22773>}}

yugabyted

• Updates the cqlsh version to v3.10-yb-19 for enhanced functionality. {{<issue 22982>}}

Bug fixes

YSQL

• Fixes an error that occurs when decoding null values from a boolean column sorted as NULLS LAST in a secondary index. {{<issue 22121>}}
• Fixes YSQL upgrade single connection mode error preventing new connection attempts before the old ones are released. {{<issue 22283>}}
• Stops Batched Nest Loop (BNL) crashes by ensuring better indexing condition checks. {{<issue 21954>}}
• Refines the logic to accurately push down join clauses to batched index scans without causing conflicts. {{<issue 21878>}}
• Re-enables rechecking for RowCompareExpressions to accurately handle NULL inputs in scan bound calculations. {{<issue 22075>}}
• Grants BNL hashtable its own expression context to prevent data overwrites during query execution. {{<issue 21266>}}
• Makes yb_get_range_split_clause robust using PG TRY CATCH block, ensuring YB backup doesn't fail. {{<issue 22356>}}
• Prevents coredumps by ensuring YSQL webserver destruction upon receiving a termination signal. {{<issue 18948>}}
• Fixes memory leaks in ybcFetchNextHeapTuple by properly freeing the YBCStatus. {{<issue 22396>}}
• Allows CreateNamespaceIfNotExists function to retry on "already exists" error, preventing race conditions. {{<issue 22512>}}
• Fixes database OID collision issue by skipping 65535 for database OID allocation. {{<issue 22598>}}
• Prevents the IN expressions on single column from wrongly taking the tuple path, ensuring correct data processing. {{<issue 22704>}}
• Corrects the YbGetOrdinaryColumnsNeedingPgRecheck function to return table column numbers instead of index numbers, preventing unnecessary data fetches and potential crashes or errors after dropping a column. {{<issue 22832>}}
• Reduces sequence cache collision by incorporating both database and sequence OIDs as the entry key. {{<issue 22935>}}

DocDB

• Fix ensures heartbeat processing doesn't blindly overwrite tablet replica state, avoiding potential data corruption. {{<issue 21836>}}
• Eliminates possible deadlock during setup replication by fixing the order in which locks are acquired. {{<issue 22376>}}
• Removes unnecessary flush during snapshot deletion, preventing write blocks. {{<issue 22369>}}
• Resolves a heartbeat metrics issue ensuring full xCluster error information is sent to the new master even during a leader failover, and makes tserver_heartbeat_metrics_interval_ms runtime updatable. {{<issue 22624>}}
• Converted the ysql_skip_row_lock_for_update to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{<issue 22057>}}
• Changes the class of enable_automatic_tablet_splitting gflag from kLocalPersisted (class 2) to kExternal (class 4) to eliminate setup issues with xCluster configurations. {{<issue 22088>}}
• Prevents duplicate entries in the sequence data filter used by Snapshot schedules, ensuring successful operations. {{<issue 22663>}}
• Resolves the issue of pg_locks query failure due to missing host node UUID in distributed transactions. {{<issue 22181>}}
• Prevents premature metric destruction during Prometheus scrapes, resolving non-UTF8 character issues. {{<issue 22767>}}
• Ensures failed xCluster setup if the xCluster stream state update to ACTIVE does not occur. {{<issue 22601>}}

CDC

• Ensures deletion of MemoryContext after each GetChanges RPC to prevent memory leaks. {{<issue 22328>}}
• Resolves an issue preventing newly created tables from being added to the stream metadata and CDC state table after an existing table is dropped, by considering streams in DELETING_METADATA state as well as ACTIVE state during dynamic table addition. {{<issue 20428>}}
• Prevents newly created indexes and materialized views from automatically being added to the CDC stream metadata. {{<issue 22808>}}
• Enables removal of non-eligible tables from Change Data Capture (CDC) streams to prevent unnecessary resource usage. Includes a background thread for dynamic table addition in CDC streams and introduces three new yb-admin commands with the auto flag enable_cdcsdk_dynamic_tables_disable_option for better CDC stream management. {{<issue 22876>}},{{<issue 22835>}},{{<issue 22773>}}
• Solves an issue where CDCSDK incorrectly deduces tablets as not interesting for stream before reaching the configured time limit. {{<issue 22383>}}
• Corrects the logic to remove the BEGIN record when no DML records are added to the GetChanges response. {{<issue 21646>}}

v2.20.4.1 - June 21, 2024 {#v2.20.4.1}

Build: 2.20.4.1-b5

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.4.1-b5

Improvements

DocDB

• Solves the potential deadlock in DeleteTabletListAndSendRequests, while also preventing xrepl tablets deletion due to multiple calls to the function. {{<issue 22618>}},{{<issue 22608>}}

Bug fixes

YSQL

• Makes yb_get_range_split_clause robust using PG TRY CATCH block, ensuring YB backup doesn't fail. {{<issue 22356>}}

DocDB

• Removes unnecessary flush during snapshot deletion, preventing write blocks. {{<issue 22369>}}

CDC

• Ensures deletion of MemoryContext after each GetChanges RPC to prevent memory leaks. {{<issue 22328>}}

v2.20.4.0 - May 23, 2024 {#v2.20.4.0}

Build: 2.20.4.0-b50

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.4.0-b50

Improvements

YSQL

• Adds new columns to localhost:13000/statements for more comprehensive database management, including user and database IDs along with varied block level statistics. {{<issue 21735>}}

YCQL

• Now throws an error when using the unsupported GROUP BY clause in YCQL with autoflag ycql_suppress_group_by_error available for compatibility issues. {{<issue 13956>}}

DocDB

• Updates AreNodesSafeToTakeDown to return earlier and deliver a readable error message when hitting a timeout, using a control flag, are_nodes_safe_to_take_down_timeout_buffer_ms with a default setting of 2 seconds. {{<issue 21855>}}
• Allows for faster failover in xCluster DR by skipping the cleanup process when skip_producer_stream_deletion is set on DeleteUniverseReplicationRequestPB. Safe for upgrades and rollbacks. {{<issue 22050>}}

yugabyted

• Allows a smooth restart of the second node in a cluster using the join flag without throwing any errors. {{<issue 20684>}}

Bug fixes

YSQL

• Re-enables the Storage Rows Scanned for EXPLAIN (ANALYZE, DIST) functionality in 2.20 after a successful upgrade fix, enhancing user understanding of data selectivity in filters and index conditions. {{<issue 21229>}},{{<issue 12676>}}
• Safeguards against an out-of-bounds memory write error in HashAggregate plan node with duplicate columns by rectifying array size allocation. Imported tests ensure the fix's robustness. {{<issue 21122>}}
• Resolves colocation option issues in table creation linked to table rewriting and partitioning. Enhances the defGetBoolean function to parse string values "0" and "1" as false and true respectively, and shifts a verification step to a earlier spot in the CREATE TABLE execution path, ensuring successful table partition creation. {{<issue 20302>}},{{<issue 20914>}}
• Corrects checks in YbIsScanCompatible to ensure the right-hand side (RHS) of all bound index conditions, not just inequalities, fits into the left-hand side (LHS) datatype. {{<issue 21758>}}
• Refines the YbGetOrdinaryColumnsNeedingPgRecheck condition to align with the ybIsTupMismatch implementation, ensuring Postgres rechecks index conditions when the "preliminary check" is skipped due to an invalid target key attnum. {{<issue 21451>}}
• Ensures pushed down RowCompareExpressions correctly enforce non-null column references, rectifying previous behavior and enhancing data accuracy. {{<issue 21847>}}
• Delivers consistent error messages for aborted transactions that align with those from serialization and deadlock errors. {{<issue 21043>}}

YCQL

• Allows the deletion of the Cassandra role in YCQLsh without it regenerating upon cluster restart, by adding a flag to mark if the role was previously created. {{<issue 21057>}}

DocDB

• Ensures Create Table operation fails if Alter Replication encounters an error, enhancing the reliability of replication setup. {{<issue 21732>}}
• Converted the ysql_skip_row_lock_for_update to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{<issue 22057>}}
• Modifies memory consumption calculations for pending operations to ensure accurate rejection of new writes at bootstrap, preventing loading failures. {{<issue 21254>}}
• Allows DML operations on non-replicated databases and blocks DML only on databases in transactional xCluster replication STANDBY mode. Now only databases part of an inbound transactional xCluster replication group in the xCluster safe time map will have DML operations blocked. Also, certain attributes are moved from TServer to TserverXClusterContext. {{<issue 21245>}}
• Allows viewing of the RPC bind addresses in the master leader UI, especially beneficial in cases like k8s where the RPC bind address with the pod DNS is more useful than the broadcast address. {{<issue 21959>}}
• Reduces unnecessary logging during checkpoint operations by lowering INFO level logs to DEBUG_LEVEL, enhancing log readability. {{<issue 21658>}}
• Enhances load balancer efficiency by refining validation logic to block tablet replica additions only for those with a pending delete in progress on the same server, avoiding potential slowdowns during mass tablet replica moves. {{<issue 21806>}}
• Restores original order of rpcs_.Shutdown in TransactionParticipant::Impl::CompleteShutdown that was altered due to a bad backport commit. {{<issue 22011>}}
• Stops fatal errors caused by the re-use of remote log anchor session during remote bootstrap from a non-leader peer. This fix ensures shared pointers are accurately tracked for tablet_peer objects using the = operator, preventing unintentional destruction of underlying objects. {{<issue 22007>}}
• Fixes hidden split parent tablets wrongly appearing as leaderless in the master's leaderless tablet endpoint, ensuring accurate load balance status and preventing potential issues with Point-in-Time Recovery (PITR) operations. {{<issue 21371>}}
• Addresses an issue where tablet shutdown may be blocked due to a race condition between shutdown request and transaction loader thread. The fix ensures a smoother tablet shutdown process when errors occur during transaction loading or when there is a premature exit from the loader thread. {{<issue 20587>}},{{<issue 20977>}}

CDC

• Prevents failures in decoding change events by refreshing cached_schema_details when executing a new GetChanges request if the client indicates a necessity for the schema. {{<issue 20698>}}

v2.20.3.1 - May 14, 2024 {#v2.20.3.1}

Build: 2.20.3.1-b2

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.3.1-b2

Bug fix

DocDB

• Converted the ysql_skip_row_lock_for_update to an auto-flag to resolve compatibility issues during upgrade, preventing incorrect DB record creations that can affect row visibility and integrity. {{<issue 22057>}}

v2.20.3.0 - April 9, 2024 {#v2.20.3.0}

Download

{{< warning title="Use v2.20.3.1 or later.">}} {{< /warning >}}

New feature

• Roll back upgrades. Ability to roll back a database upgrade in-place and restore the cluster to its state before the upgrade. You can roll back a database upgrade only to the pre-upgrade release. {{<tags/feature/ea>}}

• Added support for Red Hat Enterprise Linux 9.3 on x86-based systems. Refer to Operating system support for the complete list of supported operating systems.

Improvements

YSQL

• Alters temporary namespace naming in YB to pg_temp_<tserver-uuid>_<backend_id> from pg_temp_<backend_id>, making them unique across nodes and preventing temp tables overwriting or deletion. {{<issue 19255>}}
• Treats REFRESH MATERIALIZED VIEW as a non-disruptive change, preventing unnecessary transaction terminations. The default option, REFRESH MATERIALIZED VIEW NONCONCURRENTLY, modifies metadata but without making a disruptive alteration. {{<issue 20420>}}
• Displays distinct prefix keys explicitly in the explain output, enhancing the clarity of indexing for users. {{<issue 20831>}}
• Allows correct initialization and propagation of ybDataSent and ybDataSentForCurrQuery flags for both parent and child transactions, for enhanced error handling and retry mechanism. {{<issue 18638>}}
• Optimizes the get_tablespace_distance function, enhancing the speed of the yb_is_local_table YSQL function. Reduces query time by caching GeolocationDistance value. {{<issue 20860>}}

DocDB

• Limits the number of rows returned per transaction per tablet in pg_locks to avoid potential memory issues during batch inserts, and includes additional fields to indicate partial lock info. {{<issue 20765>}}
• Introduces a new YSQL configuration parameter yb_locks_txn_locks_per_tablet to limit the number of rows returned by pg_locks, preventing the system from running out of memory during large transactions. {{<issue 19934>}}
• Introduces a new metric for untracked memory which eliminates the need to hardcode child memory trackers. Now users can effortlessly monitor untracked memory for better resource management. {{<issue 18683>}}
• Limits the number of tablets per node, and hastens reaching the desired number of tablets by lowering the values of FLAGS_tablet_split_low_phase_shard_count_per_node to 1 and FLAGS_tablet_split_low_phase_size_threshold_bytes to 128MB. {{<issue 20579>}}
• Adds a new retrying master-to-master task, allowing for the API AreNodesSafeToTakeDown to check if it's safe to remove or upgrade certain nodes without disrupting overall cluster health. {{<issue 17562>}}
• Introduces AreNodesSafeToTakeDown API that ensures safe node removals during cluster upgrades or maintenance operations by checking tablet health and follower lag, facilitating seamless and risk-free updates. {{<issue 17562>}}
• Enables monitoring of master leader heartbeat delays through a new RPC in the MasterAdmin, ensuring undesired lags can be readily detected and mitigated. {{<issue 18788>}}
• Introduces two new metrics to alert users when CreateTable function tries to exceed the tablet count limits and to highlight when a tablet server is managing more tablets than its capacity allows. {{<issue 20668>}}
• Adjusts the tablet guardrail mechanism to wait for TServer registrations when master leadership changes, avoiding false-positive CREATE TABLE rejections due to incomplete TServer data. A new flag now controls this wait duration. {{<issue 20667>}}
• Adds a flag FLAGS_tablet_split_min_size_ratio to control tablet splitting based on SST file sizes, ensuring better control over tablet size imbalance. {{<issue 21458>}}

CDC

• Added a test to certify the safe_time set during GetChanges call, reducing data loss during network failures. Ensures consistent safe_hybrid_time in multiple GetChanges calls. {{<issue 21240>}}

Bug fixes

YSQL

• Eliminates unnecessary computation of range bounds in Index-Only Scan precheck condition, preventing crashes for certain queries and improving performance. {{<issue 21004>}}
• Eliminates risk of data loss by ensuring only the first statement in SQL mutation batches is retried in the event of a transaction conflict, when using PostgreSQL extended query protocol. {{<issue 21297>}}
• Temporarily reverts new field additions to the PgsqlResponsePB proto to address upgrade failures encountered when transitioning from versions 2.14/2.16 to 2.20.2. {{<issue 21229>}}
• Fixes table rewrite issue on non-colocated tables/matviews in colocated DB, ensuring the new table uses the original table's colocation setting. Includes a workaround for GH issue 20914. {{<issue 20856>}}
• Reduces excessive storage metric updates during EXPLAIN ANALYZE operation, enhancing performance by incorporating storage_metrics_version in YBCPgExecStats and YbInstrumentation. {{<issue 20917>}}
• Prevents simultaneous send of read and write operations in the same RPC request that could lead to inconsistent read results, by ensuring that, in case of multiple operations, all buffered ones are flushed first. {{<issue 20864>}}
• Redesigned expression tree walkers to properly handle NULL nodes in queries with subselects used in an index condition, preventing planner crashes. {{<issue 21133>}}
• Enforces stricter locking mechanisms during concurrent updates on different columns of the same row, to maintain data consistency and prevent 'write-skew anomaly within a row'. Adds a new flag ysql_skip_row_lock_for_update to toggle the new row-level locking behavior. {{<issue 15196>}}
• Adjusts heartbeat mechanism to shut down when an "Unknown Session" error occurs, reducing log alerts. This benefits idle connections with expired sessions. {{<issue 21264>}}
• Allows BNL's on outer and inner tables, even if the inner table has "unbatchable" join restrictions that can't accept batches of inputs, enhancing queries with complex join conditions. {{<issue 21366>}}
• Incorporates checks of inequality filters in the YSQL layer to avoid transmitting trivially false inequalities, preventing undesired behavior from DocDB iterators. {{<issue 21383>}}
• Allows ModifyTable EXPLAIN statements to run as a single row transactions, decreasing latency. Also enables logging for transaction types when yb_debug_log_docdb_requests is enabled. {{<issue 19604>}}
• Corrects an issue where certain unbatchable filters weren't detected during indexpath formation when indexpath accepted batched values from multiple relations. {{<issue 21292>}}

DocDB

• Fixes a race condition on kv_store_.colocation_to_table to prevent undefined behavior and re-enables packed row feature for colocated tables, enhancing data writing and compaction processes. {{<issue 20638>}}
• Clears pending_deletes_ on failed delete tasks thus preventing tablets from being incorrectly retained after task failure or completion. This rectifies a race condition and allows the Load Balancer to perform operations on specific tablets and Tablet Servers. {{<issue 13156>}}
• Enhances universe upgrade process by incrementing ClusterConfig version during an update and adds checks to prevent universe_uuid modification. Also introduces a yb-ts-cli to clear universe UUID if necessary, improving troubleshooting capabilities. {{<issue 21491>}}
• Reflects the actual columns locked in conflict resolution instead of the shared in-memory locks in pg_locks, providing more accurate output for waiting transactions. {{<issue 18399>}}
• Modifies the DocDB system by shifting the acquirement of submit_token_ of the WriteQuery to the post-conflict resolution phase to prevent DDL requests from being blocked, thus optimizing both reads and writes for continued performance and enhanced data consistency. {{<issue 20730>}}
• Corrects transaction queue behavior allowing multiple waiters for a single transaction per tablet, thereby resolving conflicts and enhancing transaction handling capability. {{<issue 18394>}}
• Incorporates detection of recently aborted transactions into the transaction coordinator with a new flag clear_deadlocked_txns_info_older_than_seconds. {{<issue 14165>}},{{<issue 19257>}}
• Disables the packed row feature for colocated tables, effectively preventing a possible encounter with the underlying issue in 21218 during debugging. {{<issue 21218>}}
• Prevents TServers from crashing due to duplication of tablets in two drives, occurring after repairing a faulty drive, by preventing the creation of new tablets during Raft-based system (RBS) process. {{<issue 20754>}}
• Eliminates potential issues with colocated tables during heavy DDL operations and compaction, reducing the risk of crashing on newer builds (2.20.0+) where the packed row feature is default. {{<issue 21244>}}
• Allows database drop operations to proceed smoothly by ignoring missing streams errors and skipping replication checks for already dropped tables. {{<issue 21070>}}
• Allows ListTabletServers to handle heartbeats older than 24 days by adjusting the setting to the maximum int32 value, avoiding system crash. {{<issue 21096>}}
• Includes the indexed_table_id with the index in table listings, eliminating the need for a second lookup to associate a main table with an index. {{<issue 21159>}}
• Corrects RPATH setting for OpenLDAP libraries that prevents system libraries being picked up or not found. Also refactors library_packager.py for improved library dependency categorization. {{<issue 21236>}}
• Reduces TPCC NewOrder latency by replacing the ThreadPoolToken with a Strand within a dedicated rpc::ThreadPool in PeerMessageQueue's NotifyObservers functions, enhancing speed and efficiency. {{<issue 20912>}}
• Early aborts transactions that fail during the promotion process, enhancing throughput in geo-partitioned workloads and offering stability in geo-partitioned tests. {{<issue 21328>}}
• Corrects block cache metrics discrepancy by ensuring Statistics object passes into LRUCache from TableCache for accurate updates. {{<issue 21407>}}
• Fixes a segmentation fault in yb-master by checking for a null pointer before dereferencing it, addressing an issue in the CDC run on 2.23.0.0-b37-arm. {{<issue 21648>}}
• Validates the use of two arguments for disable_tablet_splitting, addressing a previous condition where only one was required, thereby enhancing backup process reliability. {{<issue 8744>}}

CDC

• Introduces a fix for data loss issue caused by faulty update of cdc_sdk_safe_time during explicit checkpointing, along with tests to ensure validity. {{<issue 15718>}}
• Fixed the decoding of NUMERIC value in CDC records to prevent precision loss by ensuring that the decoded string is not converted to scientific notation if its length is more than 20 characters. {{<issue 20414>}}
• Fixes issue with CDC packed rows, now ensures a single record for large insert operations, providing consistent data regardless of row size. {{<issue 20310>}}
• Ensures consistency in CDCSDKYsqlTest.TestLargeTxnWithExplicitStream test by setting FLAGS_cdc_max_stream_intent_records value from 40 to 41, overcoming the issue of multiple records for a single insert when packed row size exceeds ysql_packed_row_size_limit. {{<issue 20310>}}

Other

• Updates the condition for HT lease reporting to ensure accurate leaderless tablet detection in RF-1 setup, preventing false alarms. {{<issue 20919>}}
• Reduces disruptions by throttling the master process log messages related to "tablet server has a pending delete" into 20-second intervals. {{<issue 19331>}}

v2.20.2.4 - September 12, 2024 {#v2.20.2.4}

Build: 2.20.2.4-b3

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.2.4-b3

Bug fix

DocDB

• Reduces the risk of deadlocks when making active read/write requests to a table right after a tablet-split. {{<issue 23747>}}

v2.20.2.3 - June 27, 2024 {#v2.20.2.3}

Build: 2.20.2.3-b2

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.2.3-b2

This is a YugabyteDB Anywhere-only release, with no changes to YugabyteDB.

v2.20.2.2 - April 1, 2024 {#v2.20.2.2}

Build: 2.20.2.2-b1

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.2.2-b1

This is a YugabyteDB Anywhere-only release, with no changes to the database.

v2.20.2.1 - March 22, 2024 {#v2.20.2.1}

Build: 2.20.2.1-b3

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.2.1-b3

Bug fixes

DocDB

• Enhances universe upgrade process by incrementing ClusterConfig version during an update and adds checks to prevent universe_uuid modification. Also, introduces a yb-ts-cli to clear universe UUID if necessary, improving troubleshooting capabilities. {{<issue 21491>}}

CDC

• Ensures numeric values are decoded without precision loss by utilizing string representation with no length limit and the PostgreSQL numeric_out method. {{<issue 20414>}}

v2.20.2.0 - March 4, 2024 {#v2.20.2.0}

Build: 2.20.2.0-b145

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.2.0-b145

New features

• Added support for read-only mode for DR replica, currently supported for YSQL tables.

• Tablet splitting now can be enabled on tables with CDC configured. {{<issue 18479>}}

• Transactional CDC now supports consistent snapshots, ensuring data integrity during replication to a sink. Snapshots, ordered by commit time across all tables and tablets, establish a reliable replication order. {{<issue 19682>}}

Improvements

YSQL

• Introduces yb_silence_advisory_locks_not_supported_error as a temporary solution, avoiding disruption while users transition from the use of advisory locks. {{<issue 19974>}}
• Shifts from the test flag "FLAGS_TEST_enable_db_catalog_version_mode" to the {{<tags/feature/tp>}} flag "FLAGS_ysql_enable_db_catalog_version_mode", enhancing user control over concurrent DDL execution across different databases. {{<issue 12417>}}
• Issues a notice for unsafe ALTER TABLE operations, including for ADD COLUMN...DEFAULT, to indicate existing rows won't be backfilled with the default value, enhancing user awareness. Suppression possible by setting ysql_suppress_unsafe_alter_notice flag to true. {{<issue 19360>}}
• Added sorting capabilities to BatchNestedLoopJoin to return the rows in the same order as NestedLoopJoin. {{<issue 19589>}}
• Replaces the ysql_max_read_restart_attempts and ysql_max_write_restart_attempts flags with yb_max_query_layer_retries, applies limit to Read Committed isolation statement retries, and adjusts retry_backoff_multiplier and retry_min_backoff defaults. {{<issue 20359>}}
• Disallows the creation of a temporary index with a tablespace, preventing client hangs and providing a clear error message for temporary index creation with set tablespace. {{<issue 19368>}}
• Enables index tablespace modification through the ALTER INDEX SET TABLESPACE command and regulates column statistics using the ALTER INDEX ALTER COLUMN SET STATISTICS command. Also, allows the creation and alteration of materialized views with the specified tablespace. Suppress the beta feature warning by enabling ysql_beta_feature_tablespace_alteration flag. {{<issue 6639>}}
• Adds function to log the memory contexts of a specified backend process, enhancing memory usage monitoring and allowing users to troubleshoot memory-related issues more effectively. {{<issue 14025>}}

DocDB

• Allows customizing retryable request timeouts to respect client-side YCQL and YSQL timings, optimizing log replay and preventing the TServer from rejecting requests that exceed durations. Adjusts default retryable request timeout to 660 seconds and offers a configuration to eliminate server-side retention of retryable requests with FLAGS_retryable_request_timeout_secs =0. {{<issue 18736>}}
• Speeds up TServer Init by optimizing the handling of deleted and tombstoned tablets. It ensures faster startup by using a new flag num_open_tablets_metadata_simultaneously, which sets the number of threads for opening tablet metadata. This allows for parallel opening of metadata, improving response times even in cases with large numbers of tablets. Additionally, the handling of tablets marked as Deleted or Tombstoned is managed asynchronously, marking tombstoned tablets as dirty for inclusion in the next heartbeat. {{<issue 15088>}}
• Logs all instances of tablet metadata creation/updation, enabling additional insights for troubleshooting in cases of multiple meta records for the same tablet. {{<issue 20042>}}
• Adds a 10-second delay (auto_flags_apply_delay_ms) between AutoFlag configuration updates and their application, allowing all TServers to receive the new configuration before applying it. This change enhances configuration consistency and update safety. {{<issue 19932>}}
• Enhances thread safety by setting Wthread-safety-reference to check when guarded members are passed by reference and resolving all build errors resulting from this change. {{<issue 19365>}}
• Automates recovery of index tables impacted by a bug, preventing performance degradation and disk size leak, by ensuring schema.table_properties.retain_delete_markers is reset to false when index backfilling is done. {{<issue 19731>}}
• Enhances the demote_single_auto_flag yb-admin command by returning distinct error messages for invalid process_name, flag_name, or non-promoted flag, thereby aiding easier identification of errors. Replaces HasSubstring with ASSERT_STR_CONTAINS in the AutoFlags yb-admin test. {{<issue 20004>}}
• Introduces support for Upgrade and Downgrade of universes with xCluster links, enhancing compatibility checks for AutoFlags during these operations. Data replication between two universes is only catered if AutoFlags of the Target universe are compatible with the Source universe. The compatibility check, stored in ProducerEntryPB, triggers if the AutoFlag configuration changes during upgrades and rollbacks. This prevents unnecessary RPC calls if no AutoFlag configurations have changed. Also includes fixes for cds initialization bugs in TestThreadHolder. {{<issue 19518>}}
• Offers redesigned server level aggregation for metrics, thus introducing more metrics for enhanced debugging. Removes several unused URL parameters and makes the new output compatible with YugabyteDB Anywhere and YugabyteDB Aeon, preventing double-counting issues in charts. Drops unused JSON and Prometheus callbacks from MetricEntity for a cleaner design. {{<issue 18078>}}
• Allows a limit on the addition of new tablet replicas in a cluster to conserve CPU resources, with safeguards for downscaling. Introduces test flags for controlling memory reservation and tablet replica per core limits. {{<issue 16177>}}
• Introduces verbose logging for global and per table state transitions in the load balancer to facilitate easier debugging. {{<issue 20289>}}
• Reduces server initialization time by eliminating the accumulation of deleted tablet superblocks during startup, through a modification in the DeleteTablet operation. {{<issue 19840>}}
• Enables automatic recovery of index tables affected by a bug, verifying their backfilling status and correcting the retain_delete_markers property to enhance performance. {{<issue 20247>}}
• Includes single shard (fast-path) transactions in the pg_locks by querying single shard waiters registered with the local waiting transaction registry at the corresponding TServer, ensuring more complete transaction tracking and lock status reporting. {{<issue 18195>}}
• Creates Prometheus metrics for server hard and soft memory limits which allow detailed insight into TServer or master memory usage regardless of Google flag defaults; aids in creating dashboards charts to monitor utilization close to soft limit or TServer TCMalloc overhead. {{<issue 20578>}}
• Enables control over the batching of non-deferred indexes during backfill via a new flag, improving index management. {{<issue 20213>}}

yugabyted

• Corrects the CPU usage reporting in the sankey diagram by filtering nodes based on region selection on the performance page. {{<issue 19991>}}

• Repairs yugabyted-ui functionality when using custom YSQL and YCQL ports by passing values to yugabyted-ui, ensuring the correct operation of the user interface. {{<issue 20406>}}

Bug fixes

YSQL

• Reduces Proc struct consumption by enabling cleanup for killed background workers, preventing webserver start-up failure after 8 attempts. {{<issue 20154>}}
• Frees up large volumes of unused memory from the webserver after processing queries, enhancing periodic workloads by reallocating held memory to the OS more effectively. {{<issue 20040>}}
• Introduces two new boolean flags for YSQL webserver debugging and failure identification.The first will log every endpoint access and its list of arguments before running the path handler. The second will print basic tcmalloc stats after the path handler and garbage collection have run. {{<issue 20157>}}
• Rectifies an issue causing segmentation faults when the postmaster acquires already owned LWLock, enhancing stability during process cleanup. Uses KilledProcToCleanup instead of MyProc if acquiring locks under the postmaster. {{<issue 20166>}}
• Introduces two new boolean flags for YSQL webserver debugging and failure identification.The first will log every endpoint access and its list of arguments before running the path handler. The second will print basic tcmalloc stats after the path handler and garbage collection have run. {{<issue 20157>}}
• Restarts postmaster in critical sections.This applies to backend PG code; defines and improves handling of errors in these sections. {{<issue 20255>}}
• Introduces the pg_stat_statements.yb_qtext_size_limit flag to control the maximum file size read into memory, preventing system crashes due to oversized or corrupt qtext files. {{<issue 20211>}}
• Caps the number of attempts to read inconsistent backend entries to 1000 for safer operation and better visibility, limiting potential infinite loops in /rpcz calls and logging every hundredth attempt. {{<issue 20274>}}
• Resolves segmentation faults in the webserver SIGHUP handler during cleanup. {{<issue 20309>}}
• Display consistent wait-start times in pg_locks view for waiting transactions. {{<issue 18603>}},{{<issue 20120>}}
• Reduces excessive memory consumption during secondary index scans. {{<issue 20275>}}
• Allows the finalize_plan function to now apply finalize_primnode on PushdownExprs, ensuring Parameter nodes in subplans transfer values accurately, especially in Parallel Queries. {{<issue 19694>}}
• Return correct results when Batch Nested Loop join is used for queries involving Nested LEFT JOINs on LATERAL views. {{<issue 19642>}},{{<issue 19946>}}
• Upgrades "Unknown session" error to FATAL permitting drivers to instantly terminate stale connections, minimizing manual user intervention. {{<issue 16445>}}
• Rectifies correctness issues when BatchNestedLoop join is used and the join condition contained a mix of equality and non-equality filters. {{<issue 20531>}}
• Mitigates incorrect data entry into the default partition by incrementing the schema version when creating a new partition, enhancing data consistency across all connections. {{<issue 17942>}}
• Rectifies correctness issue when join on inequality condition and join columns contains NULL values. {{<issue 20642>}}
• Rectifies correctness issue when queries involving outer joins and aggregate use BNL. {{<issue 20660>}}
• Corrects the Batch Nested Loop's optimization logic for proper handling of cases where the given limit matches the outer table's exact size, ensuring accurate query results. {{<issue 20707>}}
• Prevents "Not enough live tablet servers to create table" error during ALTER TABLE SET TABLESPACE by correctly supplying the placement_uuid, even when creating a table in the same tablespace. {{<issue 14984>}}
• Addresses a bug that caused backup failure due to the absence of yb_catalog_version, by ensuring the function's existence post-normal migration. {{<issue 18507>}}
• Corrects an error in the aggregate scans' pushdown eligibility criteria to prevent wrong results from being returned when PG recheck is not expected, but YB preliminary check is required to filter additional rows. {{<issue 20709>}}
• Ensures the Linux PDEATH_SIG mechanism signals child processes of their parent process's exit, by correctly configuring all PG backends immediately after their fork from the postmaster process. {{<issue 20396>}}
• Fixes a MISMATCHED_SCHEMA error when upgrading from version 2.16 to 2.21 by introducing a 2-second delay for catalog version propagation when a breaking DDL statement is detected. {{<issue 20842>}}
• Return correct result for join queries using BatchNestedLoop join and DISTINCT scan using the range Index during inner table scan. {{<issue 20827>}}
• Renders a fix for memory corruption issue that caused failure in creating a valid execution plan for SELECT DISTINCT queries. Enables successful execution of queries without errors and prevents server connection closures by disabling distinct pushdown. This fix improves the stability and effectiveness of SELECT DISTINCT queries. {{<issue 20893>}}
• Eliminates unnecessary computation of range bounds in Index-Only Scan precheck condition, preventing crashes for certain queries. {{<issue 21004>}}

DocDB

• Resolves potential WriteQuery leak issue in CQL workloads, ensuring proper execution and destruction of queries, while preventing possible tablet shutdown blockages during conflict resolution failure. {{<issue 19919>}}
• Mitigates the issue of uneven tablet partitions and multiple pollers writing to the same consumer tablet by only applying intents on the consumer that match the producer tablet's key range. If some keys/values are filtered out from a batch, it will not delete the consumer's intents, as they may be needed by subsequent applications. Also guarantees idempotency, even if some apply records stutter and fetch older changes. {{<issue 19728>}}
• Reduces chances of transaction conflicts upon promotion by delaying the sending of UpdateTransactionStatusLocation RPCs until after the first PROMOTED heartbeat response is received, enhancing transaction consistency and accuracy. {{<issue 17319>}}
• Sets kMinAutoFlagsConfigVersion to 1 to resolve flag configuration mismatch issue. {{<issue 19985>}}
• Unblocks single shard waiters once a blocking subtransaction rolls back, by applying identical conflict check logic for both distributed transactions and single shard transactions. {{<issue 20113>}}
• Eliminates a race condition that can occur when simultaneous calls to SendAbortToOldStatusTabletIfNeeded try to send the abort RPC, thus preventing avoidable FATALs for failed geo promotions. {{<issue 17113>}}
• Fixes behavior of Tcmalloc sampling; deprecates the enable_process_lifetime_heap_sampling flag in favor of directly setting profiler_sample_freq_bytes for controlling tcmalloc sampling, enhancing control over sampling process. {{<issue 20236>}}
• Resolves instances of the leaderless tablet endpoint incorrectly reporting a tablet as leaderless post-leader change, by tweaking the detection logic to depend on the last occurrence of a valid leader, ensuring more accurate tablet reporting. {{<issue 20124>}}
• Allows early termination of ReadCommitted transactions with a kConflict error, enhancing overall system throughput by eliminating unnecessary blockages without waiting for the next RPC restart. {{<issue 20329>}}
• Fixes FATAL errors occurring during tablet participant shutdown due to in-progress RPCs by ensuring rpcs_.Shutdown is invoked after all status resolvers have been shut down. {{<issue 19823>}}
• Modifies SysCatalog tablet's retryable request retention duration to consider both YQL and YSQL client timeouts, reducing the likelihood of request is too old errors during YSQL DDLs. {{<issue 20330>}}
• Handles backfill responses gracefully even when they overlap across multiple operations, reducing risks of crashes and errors due to network delays or slow masters. {{<issue 20510>}}
• Reintroduces bloom filters use during multi-row insert, improving conflict resolution and rectifying missing conflict issues, while also addressing GH 20648 problem. {{<issue 20398>}},{{<issue 20648>}}
• Fixes handling of duplicate placement blocks in under-replication endpoint for better compatibility and correct replica counting, preventing misrepresentation of under-replicated tablespaces. {{<issue 20657>}}
• Logs the first failure during setup replication instead of the last error, facilitating better error diagnosis. {{<issue 20689>}}
• Implements a retry mechanism to acquire the shared in-memory locks during waiter resumption. Rather than failing after a single attempt, it now schedules retrying until the request's deadline, reducing request failures due to heavy contention. {{<issue 20651>}},{{<issue 19032>}},{{<issue 19859>}}
• Reduces log warnings in normal situations by downgrading repeated waiter resumption alerts to VLOG(1), benefiting from the direct signaling of transaction resolution. {{<issue 19573>}}
• Disables the packed row feature for colocated tables to prevent possible write failures post complications, as a workaround while investigating issue 20638. {{<issue 21047>}}
• Resolves database deletion failures by skipping replication checks for dropped tables during the database drop process, addressing errors related to missing streams. {{<issue 21070>}}
• Addresses a race condition on kv_store_.colocation_to_table reading and permits packed row features for colocated tables, tackling undefined behaviors and failed table writes. {{<issue 20638>}}
• Disables the packed row feature for colocated tables to prevent the issue 20638 that causes subsequent write failures after certain compactions. {{<issue 21047>}}
• Corrects RPATH setting for OpenLDAP libraries, preventing the system from picking up wrong versions or failing to find them at all. {{<issue 21236>}}

CDC

• Corrects the computation of the cdcsdk_sent_lag metric to prevent steep, disproportionate increases by updating the last_sent_record_time when a SafePoint record is spotted, in addition to DMLs and READ ops. {{<issue 15415>}}

Other

• Adjusts tserver start and tserver stop scripts to successfully terminate all running PG processes, irrespective of their PID digit count. {{<issue 19817>}}
• Updates the condition for HT lease reporting to ensure accurate leaderless tablet detection in RF-1 setup, preventing false alarms. {{<issue 20919>}}

v2.20.1.3 - January 25, 2024 {#v2.20.1.3}

Build: 2.20.1.3-b3

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.20.1.3-b3

Bug fixes

YSQL

• Fixed index scans where a query with join on inequality condition incorrectly returns rows with NULL in the key column.{{<issue 20642>}}

DocDB

• Restore bloom filter usage during multi-row insert. {{<issue 20398>}},{{<issue 20648>}}

v2.20.1.2 - January 17, 2024 {#v2.20.1.2}

Downloads

{{< warning title="Use 2.20.1.3 or later.">}} {{< /warning >}}

Bug fixes

YSQL

• Fix BNL local join lookup equality function. {{<issue 20531>}}

CDC

• Fix addition of new tables to stream metadata after drop table. {{<issue 20428>}}

v2.20.1.1 - January 11, 2024 {#v2.20.1.1}

Downloads

{{< warning title="Use 2.20.1.3 or later.">}} {{< /warning >}}

Improvements

YSQL

• In Read Committed Isolation, limit the number of retry attempts when the aborted query is retried. {{<issue 20359>}}

Bug fixes

YSQL

• Return correct results when Batch Nested Loop join is used for queries involving Nested LEFT JOINs on LATERAL views. {{<issue 19642>}},{{<issue 19946>}}

• Added a regression test for nested correlated subquery. {{<issue 20316>}}

CDC

• Fixed decimal type precision while decoding CDC record. {{<issue 20414>}}

DocDB

• In Read Committed Isolation, immediately abort transactions when conflict is detected. {{<issue 20329>}}

v2.20.1.0 - December 27, 2023 {#v2.20.1.0}

Downloads

{{< warning title="Use 2.20.1.3 or later.">}} {{< /warning >}}

Improvements

YSQL

• Adjusts ysql_dump when run with --include-yb-metadata argument, to circumvent unnecessary automatic index backfilling. This avoids unnecessary RPC calls. {{<issue 19457>}}
• Imports the pgcrypto: Check for error return of px_cipher_decrypt upstream PostgreSQL patch; OpenSSL 3.0+ prerequisite. {{<issue 19732>}}
• Imports the upstream PG commit Disable OpenSSL EVP digest padding in pgcrypto; OpenSSL 3.0+ prerequisite. {{<issue 19733>}}
• Imports an upstream PostgreSQL commit that adds an alternative output when OpenSSL 3 doesn't load legacy modules; OpenSSL 3.0+ prerequisite. {{<issue 19734>}}
• Removes the need for a table scan when adding a new column with a NOT NULL constraint and a non-volatile DEFAULT value, enhancing ADD COLUMN operations. {{<issue 19355>}}
• Mitigates CVE-2023-39417 by importing a security-focused upstream Postgres commit from REL_11_STABLE for YSQL users. {{<issue 14419>}}
• Fixes CVE-2020-1720 by importing upstream Postgres commit from REL_11_STABLE, enabling support for ALTER <object>DEPENDS ON EXTENSION in future for objects like function, procedure, routine, trigger, materialized view, and index. {{<issue 14419>}}
• Modifies the planner to allow ordered index scans with IN conditions on lower columns; this takes advantage of the YB LSM indexes, which maintain index order. {{<issue 19576>}}
• Increase the oom_score_adj (Out of Memory score adjustment) of the YSQL webserver to 900, the same as PG backends, prioritizing its termination when it significantly consumes memory. {{<issue 20028>}}
• Blocks the use of advisory locks in YSQL, as they are currently unsupported. {{<issue 18954>}}
• Introduces yb_silence_advisory_locks_not_supported_error as a temporary solution, avoiding disruption while users transition from the use of advisory locks. {{<issue 19974>}}

DocDB

• Enable checks for trailing zeros in SST data files to help identify potential corruption. {{<issue 19691>}}
• Includes version information in the error message when the yb process crashes due to AutoFlags being enabled on an unsupported older version, enhancing the ease of identifying the issue. {{<issue 16181>}}
• Prints long trace logs with a continuation marker when a trace is split into multiple LOG(INFO) outputs. This is for log readability. {{<issue 19532>}},{{<issue 19808>}}
• Upgrades OpenSSL to version 3.0.8, disabling Linuxbrew builds and updating glog for stack unwinding based on the backtrace function. {{<issue 19736>}}
• Expands debug information to help investigate SELECT command errors that imply faults in read path processing or provisional record writing. {{<issue 19876>}}
• Adds support for rocksdb_max_sst_write_retries flag: maximum allowed number of attempts to write SST file in case of detected corruption after write (by default 0 which means no retries). Implemented for both flushes and compactions. {{<issue 19730>}}
• Balances load during tablet creation across all drives, preventing bottlenecks and underutilized drives during remote-bootstrapping; uses total number of tablets assigned for tie-breaking. {{<issue 19846>}}
• Adds TServer page to display all ongoing Remote Bootstrap sessions, and shows the source of bootstrap sessions in the Last Status field, for improved visibility. {{<issue 19568>}}
• Enhances error reporting from XClusterPollers by storing just the error code instead of detailed status, making it safer against master and TServer restarts and reducing memory usage. {{<issue 19455>}}
• Incorporates a JoinStringsLimitCount utility for displaying only the first 20 elements in a large array, saving memory and space while logging or reporting tablet IDs. {{<issue 19527>}}
• Adds a sanity check to prevent restoration of the initial sys catalog snapshot when the master_join_existing_universe flag is set. {{<issue 19357>}}
• Parallelizes RPCs in DoGetLockStatus to fetch locks swiftly, enhancing the database's response time while retrieving old transactions. {{<issue 18034>}}
• Simplifies xCluster replication failure debugging via a new get_auto_flags_config yb-admin command, returning the current AutoFlags config. {{<issue 20046>}}
• Enables automatic recovery of index tables affected by {{<issue 19544>}}, improving performance and disk size management by removing excess tombstones in the SST files. {{<issue 19731>}}
• Introduces verbose logging for global and per table state transitions in the load balancer to facilitate easier debugging. {{<issue 20289>}}

CDC

• Allows the catalog manager to eliminate erroneous entries from the cdc_state table for newly split tablets; prevents race conditions by reversing the order of operations during the CleanUpCDCStreamsMetadata process. {{<issue 19746>}}

yugabyted

• Resolves 0.0.0.0 to 127.0.0.1 as the IP for master, TServer and yugabyted-UI when 0.0.0.0 is specified as the advertise_address in yugabyted start command. {{<issue 18580>}}
• Ensures the yugabyted UI can fetch and display Alert messages from different nodes by adjusting the /alerts endpoint in the API server to include a node_address parameter. {{<issue 19972>}}
• Eliminates the deprecated use_initial_sys_catalog_snapshot flag in yugabyted, thus reducing log warning messages. {{<issue 20056>}}

Other

• Better indentation of multi-line remote trace entry output, for readability. {{<issue 19758>}}
• Implements a preventative safeguard in TServer operations by adding a check that a tablet has explicitly been deleted before issuing a DeleteTablet command, minimizing data loss risks and enhancing reliability. This feature, enabled with master_enable_deletion_check_for_orphaned_tablets=true, is upgrade and downgrade safe. {{<issue 18332>}}

Bug fixes

YSQL

• Allows the postprocess script in pg_regress to run on alternative expected files alongside default_expectfile and platform_expectfile, fixing unintended mismatches. {{<issue 19737>}}
• Prevents stuck PostgreSQL processes and ensures successful acquisition of locks in the future. This enhancement particularly aids in preventing deadlocks when creating replication slots with duplicate names. {{<issue 19509>}}
• Ensures OID uniqueness within a database in YB environment by introducing a new per-database OID allocator, avoiding risk of collisions across multiple nodes or tenants. This change is safe for upgrades and rollbacks, enabled by a new flag ysql_enable_pg_per_database_oid_allocator. {{<issue 16130>}}
• Prevents postmaster crashes during connection termination cleanup by using the killed process's ProcStruct to wait on a lock. {{<issue 18000>}}
• Fixes the distinct query iteration to distinguish between deleted and live keys; this prevents missing live rows during distinct queries on tables with dead tuples. {{<issue 19911>}}
• Fixes bug when type-checking bound tuple IN conditions involving binary columns like UUID. {{<issue 19753>}}
• Ensures PK modification does not disable Row-Level Security (RLS). {{<issue 19815>}}
• Tracks spinlock acquisition during process initialisation, and reduce the time to detect spinlock deadlock. {{<issue 18272>}},{{<issue 18265>}}
• Reverts to PG restart in rare event of process termination during initiation or cleanup; this avoids needing to determine what subset of shared memory items to clean. {{<issue 19945>}}
• Prevents unexpected crashes during plan creation when joining two varchar columns. Handles the derivation of element_typeid in Batched Nested Loop (BNL) queries; casts both sides to text to ensure valid equality expression. {{<issue 20003>}}
• Reduces Proc struct consumption by enabling cleanup for killed background workers, preventing webserver start-up failure after 8 attempts. {{<issue 20154>}}
• Frees up large volumes of unused memory from the webserver after processing queries, enhancing periodic workloads by reallocating held memory to the OS more effectively. {{<issue 20040>}}
• Introduces two new boolean flags for YSQL webserver debugging and failure identification.The first will log every endpoint access and its list of arguments before running the path handler. The second will print basic tcmalloc stats after the path handler and garbage collection have run. {{<issue 20157>}}
• Rectifies an issue causing segmentation faults when the postmaster acquires already owned LWLock, enhancing stability during process cleanup. Uses KilledProcToCleanup instead of MyProc if acquiring locks under the postmaster. {{<issue 20166>}}
• Improves insight into webserver memory usage :13000/memz and :13000/webserver-heap-prof, which print tcmalloc stats and display currents or peak allocations respectively. {{<issue 20157>}}
• Restarts postmaster in critical sections.This applies to backend PG code; defines and improves handling of errors in these sections. {{<issue 20255>}}
• Caps the number of attempts to read inconsistent backend entries to 1000 for safer operation and better visibility, limiting potential infinite loops in /rpcz calls and logging every hundredth attempt. {{<issue 20274>}}
• Resolves segmentation faults in the webserver SIGHUP handler during cleanup. {{<issue 20309>}}
• Allows the finalize_plan function to now apply finalize_primnode on PushdownExprs, ensuring Parameter nodes in subplans transfer values accurately, especially in Parallel Queries. {{<issue 19694>}}

DocDB

• Enables Masters to decrypt universe key registry correctly when Master turns into a leader following Remote Bootstrap. correctly when it turns into a leader. {{<issue 19513>}}
• Fixes metric retryable requests mem-tracker metric so it's aggregated at table-level {{<issue 19301>}}
• Ensure data consistency by checking if there is pending apply record, preventing the removal of large transactions that are still applying intents during bootstrap. {{<issue 19359>}}
• Prevents a write to freed memory; RefinedStream::Connected returns status instead of calling context_->Destroy(status). {{<issue 19727>}}
• Allows serving remote bootstrap from a follower peer, i.e. initial remote log anchor request is at the follower's last logged operation id index, reducing probability of fallback to bootstrap from the leader; this could increase system efficiency under high load. {{<issue 19536>}}
• Corrects Master's tablet_overhead memory tracker to correctly display consumption, ensuring accurate reflection of data in Memory Breakdown UI page, and aligns MemTracker metric names between TServer and Master. {{<issue 19904>}}
• Resolves intermittent index creation failure for empty YCQL tables by validating the is_running state instead of checking the index state directly; prevents inaccurate values in old indexes and ensuring retain_delete_markers are correctly set to false. {{<issue 19933>}}
• Addresses a deadlock issue during tablet shutdown with wait-queues enabled by modifying the Wait-Queue shutdown path to set shutdown flags during StartShutdown and process callbacks during CompleteShutdown. {{<issue 19867>}}
• Reduces race conditions in MasterChangeConfigTest.TestBlockRemoveServerWhenConfigHasTransitioningServer by letting asynchronous threads operate on ExternalMaster* copies, not current_masters. {{<issue 19927>}}
• Allows only a single heap profile to run at a time, restricting concurrent profile runs to avoid misinterpretation of sampling frequency reset values. {{<issue 19841>}}
• Eliminates misleading Transaction Metadata Missing errors during a deadlock by replacing them with more accurate deadlock-specific messages. {{<issue 20016>}}
• Fixes behavior of Tcmalloc sampling; deprecates the enable_process_lifetime_heap_sampling flag in favor of directly setting profiler_sample_freq_bytes for controlling tcmalloc sampling, enhancing control over sampling process. {{<issue 20236>}}

CDC

• Corrects a deadlock during the deletion of a database with CDC streams by ensuring namespace-level CDC stream does not add the same stream per table more than once. {{<issue 19879>}}

yugabyted

• Adds a check for a join flag to avoid error when starting two different local RF-1 YugabyteDB instances on mac {{<issue 20018>}}

Other

• Rectifies the CentOS GCC11 and background worker asan build issues by utilizing stack allocation and eliminating the unsupported std::string::erase with const iterators. {{<issue 20147>}},{{<issue 20087>}}
• Fixes segmentation fault in stats collector after Postmaster reset by cleaning up after terminated backends and properly resetting shared memory and structures, resulting in improved stability and reliability of the system. {{<issue 19572>}}
• Allows custom port configurations in multi-region/zone YugabyteDB clusters to prevent connectivity problems during cluster setup. {{<issue 15334>}}

v2.20.0.2 - December 15, 2023 {#v2.20.0.2}

Downloads

{{< warning title="Use 2.20.1.3 or later.">}} {{< /warning >}}

This is a YugabyteDB Anywhere-only release, with no changes to the database.

v2.20.0.1 - December 5, 2023 {#v2.20.0.1}

Downloads

{{< warning title="Use 2.20.1.3 or later.">}} {{< /warning >}}

Bug fixes

• [19753] [YSQL] Fix tuple IN condition type-checking in the presence of binary columns
• [19904] [DocDB] Fix Master's tablet_overhead mem_tracker zero consumption issue
• [19911] [DocDB] Avoid AdvanceToNextRow seeking past live rows upon detecting a deleted row during distinct iteration
• [19933] [DocDB] Backfill done may not be triggered for empty YCQL table

v2.20.0.0 - November 13, 2023 {#v2.20.0.0}

Downloads

{{< warning title="Use 2.20.1.3 or later.">}} {{< /warning >}}

Highlights

• Support for OIDC token-based authentication via Azure AD. This allows YSQL database users to sign in to YugabyteDB universes using their JSON Web Token (JWT) as their password.
• Support for Transactional CDC.
• Wait-on conflict concurrency control with pg_locks. This feature is especially important to users that have workloads with many multi-tablet statements, which may have multiple sessions contending for the same locks concurrently. Wait-on conflict concurrency control helps avoid a deadlock scenario as well as ensure fairness so a session is not starved by new sessions. {{<tags/feature/ea>}}
• Catalog metadata caching to improve YSQL connection scalability and performance with faster warm up times. {{<tags/feature/ea>}}
• Support for ALTER COLUMN TYPE with on-disk changes is now {{<tags/feature/ga>}} for production deployments. This feature expands ALTER TABLE capabilities with a broader range of data type modifications, including changes to the type itself, not just size increments.
• Index backfill stability improvements. Ensure timely notification to all nodes and PostgreSQL backends before initiating index backfill to prevent missing entries during index creation. {{<tags/feature/ea>}}

New Features

• [13686] [DocDB] Introduce the ability to Rollback And Demote an AutoFlag
• [18748] [CDCSDK] Change default checkpointing type to EXPLICIT while stream creating

Improvements

• [4906] [YSQL] Uncouple variables is_single_row_txn from is_non_transactional
• [9647] [DocDB] Add metric for the number of running tablet peers on a YB-TServer
• [12751] [CDCSDK] Made maxAttempts and sleepTime for retrying RPCs configurable in AsyncClient
• [13358] [YSQL] YSQL DDL Atomicity Part 5: YB-Backup support
• [16177] [DocDB] Mechanisms to reject table creation requests based on cluster resources, switch the tablet limit gflags from test flags to runtime flags
• [16785] [DocDB] Add tracking for event stats metrics
• [17904] [DocDB] Prevent YB-TServer heartbeats to master leader in a different universe
• [18055] [DocDB] Resume waiters in consistent order across tablets
• [18335] [DocDB] Reject ChangeConfig requests for sys catalog when another server is amidst transition
• [18384] [14114] [DocDB] Throw consistent error on detected deadlocks
• [18522] [YSQL] Change default unit for yb_fetch_size_limit to bytes from kilobytes
• [18940] [DocDB] Metric to track active WriteQuery objects
• [19071] [DocDB] Add estimated bytes/count to pprof memory pages
• [19099] [YSQL] [DocDB] Adopt the trace correctly in pg_client_session
• [19203] [xCluster] Add xCluster YB-TServer UI page, Switch to throughput in the YB-TServer UI pages from MiBps to KiBps
• [19221] [DocDB] Improved timeout handling for YCQL index scan
• [19272] [DocDB] Clear ResumedWaiterRunner queue during WaitQueue Shutdown
• [19274] [xCluster] Update xCluster apply safe time if there are no active transactions
• [19292] [CDCSDK] Publish snapshot_key in GetCheckpointResponse only if it is present
• [19295] [yugabyted] Password Authentication showing enabled in UI even when it's not turned on.
• [19301] [DocDB] Make re-tryable requests mem-tracker metric to be aggregated for Prometheus
• [19351] Harden more YSQL backends manager tests
• [19353] [xCluster] Introduce XClusterManager, Cleanup code on master and XClusterConsumer
• [19417] [DocDB] Support tracing UpdateConsensus API
• [19454] build: Move macOS support off of mac11
• [19482] [CDCSDK] Add GFlag to disable tablet split on tables part of CDCSDK stream
• [19532] [DocDB] Allow Traces to get past Glog's 30k limit
• [YSQL] Disable backends manager by default
• [YSQL] Some extra logging in Postgres Automatically determine upstream branch name from local branch name

Bug fixes

• [17025] [xCluster] Calculate replication lag metrics for split tablet children
• [17229] [DocDB] Prevent GC of any schema packings referenced in xCluster configuration
• [18081] [19535] [DocDB] Update TabletState only for tablets that are involved in write, Ignore statuses from old status tablet on promotion
• [18157] [DocDB] Don't pack rows without liveness column
• [18540] [xCluster] [TabletSplitting] Fix setting opid in cdc_state for split children
• [18711] [YSQL] Fix pg_stat_activity taking exclusive lock on t-server session
• [18732] [DocDB] Don't update cache in LookupByIdRpc if the fetched tablet has been split
• [18909] [YSQL] Fix ALTER TYPE on temporary tables
• [18911] [19382] [YSQL] Fix ALTER TYPE null constraint violation and failure for range key tables
• [19021] [YSQL] Fix bug in computation of semi/anti join factors during inner unique joins
• [19033] [DocDB] PGSQL operation triggered during parent tablet shutting down may be not retried
• [19063] [YSQL] Fix table rewrite on a table referenced through a partitioned table's foreign key
• [19308] [YSQL] Fix bug where tuple IN filters were not bound to the request
• [19316] [19314] [yugabyted] Bugs regarding --join flag provided.
• [19329] [YSQL] Avoid O(n^2) complexity in list traversal while generating ybctid for IN operator
• [19348] [CDCSDK] Only delete removed tablets from cdc_state in CleanUpCDCStreamsMetadata
• [19384] [YSQL] Fix handling of case where a given RowCompareExpression cannot be bound
• [19385] [CDCSDK] Fix WAL GC issue for tables added after stream creation
• [19394] [CDCSDK] Only populate key to GetChangesRequest when it is not null, Set snapshot key from correct parameter in explicit checkpointing
• [19407] [YSQL] Single shard operations should pick read time only after conflict resolution
• [19414] [yugabyted] UI bugs
• [19434] [CDCSDK] Do not completely fail while fetching tablets if one table hits error
• [19440] [DocDB] Fix bug where invalid filter key was passed to Iterator initialization in backwards scans
• [19450] [xCluster] Fix CDCServiceTestMultipleServersOneTablet.TestUpdateLagMetrics
• [19497] [DocDB] Collect end-to-end traces only if the parent trace is non-null
• [19514] [DocDB] Do not ignore replicas on dead YB-TServer in under-replicated endpoint
• [19523] [xCluster] Fix TSAN error in xClusterConfig
• [19544] [DocDB] Hash and Range Indexes grow in size even after rows are deleted, leading to slower queries
• [19546] [DocDB] Add missing call to Prepare in PlayChangeMetadataRequest
• [19605] [YSQL] Fix test failures encountered upon enabling DDL Atomicity
• [19663] [DocDB] Fix AutoFlagsMiniClusterTest.PromoteOneFlag test