What's new in the YugabyteDB v2.16 STS release series

Release announcements

• Announcing the General Availability of YugabyteDB 2.16

Release notes

What follows are the release notes for the YugabyteDB v2.16 series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.16 series.

{{< warning title="Technical Advisory: Use database version 2.16.1.1 or later" >}} YugabyteDB releases in the v2.16 series prior to v2.16.1.1 contain a backup issue: if a database contains a pre-split range-partitioned table whose primary key constraint column ordering isn't the same as the table's column ordering.

If you're using a previous v2.16 database version, you won't be able to successfully perform backups until you upgrade to YugabyteDB 2.16.1.1 or later. {{< /warning >}}

v2.16.9.0 - January 23, 2024 {#v2.16.9.0}

Build: 2.16.9.0-b67

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.9.0-b67

Improvements

YSQL

• Enables the modification of is_single_row_txn for finer control over non-transactional writes required by COPY, index backfill, or when yb_disable_transactional_writes is set, preventing issues during non-bufferable operations for single row transactions. {{<issue 4906>}}
• Verifies error return of px_cipher_decrypt in pgcrypto, a crucial step towards supporting OpenSSL 3.0+. {{<issue 19732>}}
• Enables a prerequisite import of upstream PG commit Disable OpenSSL EVP digest padding in pgcrypto, paving the way for OpenSSL 3.0+ upgrade. {{<issue 19733>}}
• Allows for the import of upstream PG commit Add alternative output for OpenSSL 3 without legacy loaded, aiding OpenSSL 3.0+ upgrade. {{<issue 19734>}}
• Introduces enhanced security by enabling CVE-2023-39417 mitigation, implementing upstream Postgres changes from REL_11_STABLE. {{<issue 14419>}}
• Mitigates potential security issues by importing upstream Postgres commit from REL_11_STABLE for future support of ALTER <object> DEPENDS ON EXTENSION, although it's not currently available in YugabyteDB. {{<issue 14419>}}
• Adjusts webserver out-of-memory score to match PG backends, avoiding unnecessary termination and prioritizing problem-solving if excessive memory use occurs. {{<issue 20028>}}

DocDB

• Introduces profiler_sample_freq_bytes to enable allocation sampling for gperftools tcmalloc, boosts default sampling from 100KiB to 1MiB, and adds Abseil symbolizer for improved yb_prof speed. {{<issue 17758>}},{{<issue 18561>}}
• Enables distinct history retentions for the master's DocDB metadata table and other tables, enhancing the logic of fast PITR by adjusting snapshot schedules and restoration techniques. {{<issue 18207>}}
• Reduces master leader startup time and avoids deadlocks by moving migration writes out of the catalog loading phase into a catalog manager background thread pool. {{<issue 15909>}}
• Prevents tservers from communicating with master leaders in different universe clusters averting possible data loss, by introducing a new universe_uuid field and an autoflag master_enable_universe_uuid_heartbeat_check to manage the tserver heartbeat checks. {{<issue 17904>}}
• Introduces support for rocksdb_check_sst_file_tail_for_zeros flag, enabling verification of SST data file trailing zeros and reporting errors for anomalies detected. {{<issue 19691>}}
• Includes version info in the error message when the yb process is incorrectly started on an older version post-AutoFlags activation, enhancing error identification ease. {{<issue 16181>}}
• Rejects ConfigChange requests for system catalog while another server is transitioning, preventing potential data loss from mistaken quorum formation by new peers. {{<issue 18335>}}
• Enables SST write retries on detected corruption after write with the rocksdb_max_sst_write_retries flag, enhancing data reliability. {{<issue 19730>}}
• Upgrades OpenSSL to 3.0.8 from 1.1.1, reflecting its end-of-life status, while also disabling Linuxbrew builds and updating glog for stack unwinding based on backtrace function. {{<issue 19736>}}
• Ensures the master_join_existing_universe flag value check before restoring the initial sys catalog snapshot to avoid incorrect operations. {{<issue 19357>}}
• Reduces time window for retryable requests to match Write RPC timeouts, preventing the replay of unnecessary logs in YCQL tablets and offering protection against expired requests. {{<issue 18736>}}
• Logs tablet metadata creation/updation instances, giving additional insights and reducing crash loops on TServer startup by avoiding duplicate tablet metadata records. {{<issue 20042>}}
• Enables tracing for the UpdateConsensus API; activate the collect_update_consensus_traces flag to collect traces from remote followers, enhancing local log trace messages. {{<issue 19417>}}
• Enables automatic recovery of index tables for bugs detected in the context of all new index tables created from 2.14; prevents retention of tombstones in SST files, mitigates performance degradation and disk size leak. {{<issue 19731>}}
• Enhances debugging experience by adding verbose logs for load balancer state changes, updating the global state in verbose logs, and logging state transitions per table state. {{<issue 20289>}}
• Enables faster server initialization by deleting the tablet's superblock during the DeleteTablet operation if the delete_type is TABLET_DATA_DELETED, reducing the number of DELETED tablet superblocks at next server startup. {{<issue 19840>}}

CDC

• Introduces a GFlag to toggle automatic tablet splitting for tables in a CDCSDK stream, enhancing user control over replication processes. {{<issue 19482>}}

Bug fixes

YSQL

• Allows ALTER TABLE DROP CONSTRAINT IF EXISTS command to execute without throwing an error, even when the table lacks a constraint, enhancing the user experience. {{<issue 17659>}}
• Trims down the probability of inaccurate behaviour involving conflicts between single shard INSERT operations by ensuring read times are chosen after conflict resolution, enhancing data consistency. {{<issue 19407>}}
• Addresses mismatches that occurred due to non-execution of the postprocess script on alternate expected files and default_expectfile in pg_regress. {{<issue 19737>}}
• Resolves a bug that hindered consecutive lock acquisition due to not releasing LWLocks during AbortTransaction, preventing PostgreSQL processes from getting stuck. {{<issue 19509>}}
• Prevents potential postmaster crashes during cleanup of killed connections by using the killed process's ProcStruct to wait on an unavailable LWLock. {{<issue 18000>}}
• Preserves table grants, policies, dependent views, and statistical objects after modifications to a table's primary key. {{<issue 15245>}}
• Reduces spinlock deadlock detection time by 75% for prompt handling of potential freezes and restarts Postmaster when a process holding a spinlock is killed, ensuring successful initiation of new connections. {{<issue 18272>}},{{<issue 18265>}}
• Preserves the Row-Level Security (RLS) behavior during table rewrites by correctly copying relforcerowsecurity metadata from the old table. {{<issue 19815>}}
• Restarts postmaster for easier and more effective cleanup when a process is accidentally killed during its own initialization or cleanup, avoiding potential issues with shared memory items handling. {{<issue 19945>}}
• Refines background worker processes structure to prevent loss of Proc struct after killing a background worker, ensuring continued webserver functionality. {{<issue 20154>}}
• Enhances debugging in YSQL webserver by introducing two new flags for better logs and by enabling the handling of SIGHUP signals. Additionally, it provides deeper insight into memory usage through newly created endpoints. {{<issue 20157>}}
• Resolves the segmentation fault issue in postmaster by utilizing the KilledProcToCleanup instead of MyProc when acquiring locks, eliminating process restart root causes. {{<issue 20166>}}
• Restart the postmaster whenever a process is killed while writing in a critical section, preventing corrupted state and CPU consumption due to infinite loops. {{<issue 20255>}}
• Caps the number of attempts to read inconsistent back-end entries at 1000 to prevent indefinite waiting, proceeding with potentially incomplete yet safe results if consistently fails. {{<issue 20274>}}
• Sets yb_pg_batch_detection_mechanism to DETECT_BY_PEEKING by default for safer usage on older versions, mitigates "Illegal state: Used read time is not set" error with INSERTs. {{<issue 20099>}},{{<issue 16446>}}
• Eradicates segmentation faults in the webserver SIGHUP handler during cleanup by using MyLatch throughout the process life cycle, enhancing stability. {{<issue 20309>}}
• Reduces memory usage during secondary index scan operations by introducing a separate, periodically reset arena for batch operations, thereby preventing system memory overload. {{<issue 20275>}}

DocDB

• Mitigates instances of hybrid time too low error by requiring new leaders to only accept writes when the previous leader's lease has expired. {{<issue 18121>}}
• Allows tracing of outgoing calls only if the current RPC is being traced, reducing excessive memory consumption and logging. {{<issue 19497>}}
• Overcomes issues with YCQL, snapshots, and metrics by migrating the TableInfo namespace_name field through the master migration framework, first introduced in version 2.3, with the namespace loader now preceding the table loader. {{<issue 17713>}}
• Restores the original behavior that doesn't count tablets on dead tservers towards the replica count in the under-replicated endpoint after a master leader failover. {{<issue 19514>}}
• Introduces retry logic to synchronize metadata and checkpoint creation during remote bootstrap initialization, reducing inconsistency risks associated with schema packing. {{<issue 19546>}}
• Blocks schema Garbage Collection while creating a snapshot to avoid loss of schema packing between a checkpoint creation and saving tablet metadata. {{<issue 18379>}}
• Reduces index size growth and enhances query speed by properly updating the retain_delete_markers field on index backfilling, addressing an issue where delete markers persisted in SST files. {{<issue 19544>}}
• Increases tablet_creation_timeout_ms from 30 seconds to 600 seconds to prevent tablet replacement during table creation, improving restore operations in network latency sensitive environments. {{<issue 13030>}}
• Refines meta cache updates to avoid overwriting child tablets and consequently causing stale data, ensuring more accurate partition map refreshes. {{<issue 18732>}}
• Prevents potential lags and errors in the WaitOnConflict workload testing by ensuring in-memory locks get properly resettled, even in the event of a failed conflict resolution sequence. This fix helps avoid lock-related issues that might occur with any leaked or stuck WriteQuery objects. {{<issue 18770>}}
• Resolves a use-after-free issue in RefinedStream's failure path to enhance system stability and prevent memory corruption. {{<issue 19727>}}
• Fixes an intermittent index creation failure by checking the result of is_running, ensuring stability for empty YCQL tables, and preventing retain_delete_markers from being incorrectly set. {{<issue 19933>}}
• Resolves a concurrency issue in MasterChangeConfigTest by operating on a copy of ExternalMaster* to prevent timing out when removing a follower with a transitioning server. {{<issue 19927>}}
• Streamlines handling of single shard waiters by replicating existing conflict checking logic typically used for distributed transactions, ensuring waiters unblock properly after a blocking subtransaction rollback. {{<issue 20113>}}
• Deprecates the enable_process_lifetime_heap_sampling flag, allowing control over tcmalloc sampling only with the profiler_sample_freq_bytes flag for better accuracy. {{<issue 20236>}}
• Adjusts SysCatalog tablet's retryable requests retain duration to prevent the request is too old error in YSQL DDL after 60 seconds, enhancing user's cross-database operation execution. {{<issue 20330>}}

CDC

• Eliminates deadlocks during the deletion of a YSQL database with namespace-level CDC streams by improving the marking code logic. {{<issue 19879>}}
• Corrects the problem where new tables were not added to the stream metadata after a table was dropped, by allowing tables to be added to streams in both the ACTIVE and DELETING_METADATA states. {{<issue 20428>}}

Other

• Eliminates potential crashes by adding nullity checks to CDCServiceImpl::GetChanges that previously could return a null pointer. {{<issue 19524>}}
• Avails a more robust stats collector to prevent crashes when a query is terminated while holding a LWLock, ensuring uninterrupted analytics data gathering. {{<issue 19572>}}

v2.16.8.0 - October 12, 2023 {#v2.16.8.0}

Build: 2.16.8.0-b16

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.8.0-b16

Improvements

• [17270] [DST] Fast PITR: Add ability to set cotables filter map
• [17464] [DST] Fast PITR - Use cotables filter for master restore
• [17716] [DocDB] Move tcmalloc utility code out of MemTracker
• [18744] [DocDB] Add ability to recover from follower lag caused by stuck OutboundCall
• [18770] [DocDB] Stricter memory order in PreparerImpl::Run to prevent re-ordering of instructions
• [18998] [DocDB] New flag to force shell mode for empty masters regardless of master_addresses.
• [19221] [DocDB] Improved timeout handling for YCQL index scan
• Easy Deployment of K8s cluster, changes to rbac rules.
• Adding debug hooks to add to container startup
• Add argument to enable/disable pex
• Adding svc to SAN of cert-manager
• Added dnsConfig dnsPolicy for YugabyteDB
• Allow cluster role to access all resources under operator.yugabyte.io for operator
• Remove thirdparty-deps references from values.yaml since this isn't used

Bug fixes

• [11975] [DocDB] Fix the TEST_docdb_log_write_batches flag
• [12574] [xCluster] Hold reference count of YBTable in XClusterWriteRpc
• [13128] [YSQL] Fix ClearSeparateDdlTxnMode error handling
• [18668] [DocDB] Block stack collection during thread creation
• [18879] [YSQL] Disable pushdown for TextSearch functions that are not thread safe
• [18744] [DocDB] Fixed issue with last_rpc_start_time_ reset which resulted in tsan race in connection
• [19097] [DocDB] Escape output of master and tserver UI endpoints
• Fix kubernetes operator config in the helm chart to work with latest runtime config changes
• Fix generate_kubeconfig to work against specified namespace

Known issues

N/A

v2.16.7.0 - September 15, 2023 {#v2.16.7.0}

Build: 2.16.7.0-b59

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.7.0-b59

Improvements

• [17075] [17078] Send full universe key registry to Tablet Server on Startup
• [17559] [DST] Adding loop call during tablet server init to wait on master leader to get encryption keys at rest
• [17570] [DocDB] Improve leaderless tablet endpoint
• [17683] [DocDB] Invalidate YCQL system.partitions cache on table creation
• [17805] [17965] [YSQL] Enhance TServer PG daemon robustness
• [18064] [DocDB] enhance tserver consensus state page
• [18465] [DocDB] Dump PeerManager to tserver debug ui consensus page
• [18518] [DocDB] removed unused MemTracker::rand_ field
• [18586] [YCQL] Return reserved keywords in double quotes
• [18826] Build: thirdparty_tool --list-compilers option enhancement

Bug fixes

• [13171] [DocDB] reject expired retryable requests
• [14210] [DocDB] Fixed YSQL tablet routing
• [16432] [17080] xCluster: Disallow N:1 replication topology
• [16446] [YSQL] Peek message following Execute from JDBC driver to identify batch execution and make yb_pg_batch_detection_mechanism configurable at runtime for session
• [16963] [18838] [DocDB] Add verbose logging to the load balancer.
• [17146] [xCluster] Fix missing pgschema_name during SetupUniverseReplication
• [17148] [DocDB] Fixed wrong metric_type of prometheus metric
• [17225] [YSQL] Use consistent read snapshot in DEFERRABLE READ ONLY mode
• [17400] [17488] [16665] [14308] [DocDB] fix crash due to speculatively decoding a slice ending in a Typeset
• [17558] [DocDB] Issue with picking local_limit during paging read
• [17682 [DocDB] fix hybrid time too low due to race between TransactionParticipant::Poll and RaftConsensus::Start in rf-1 setup
• [17848] [DocDB] Fix race on RemoteTablet::replicas_ between RemoteTablet::GetRemoteTabletServers and RemoteTablet::Refresh
• [17867] [DocDB] Account for table placement and read replicas in under replicated endpoint
• [17961] [18008] [YSQL] Fix max connection limit errors with additional locking
• [18001] [DocDB] Skip key registry decryption in EncryptionManager::GetFullUniverseKeyRegistry() when EAR is disabled.
• [18164] [DocDB] Ignore deleted tablets in leaderless endpoint
• [18227] [DocDB] Fix log reader missing max_replicate_index
• [18281] [YSQL] Fix query layer's read restart retries in Read Committed isolation
• [18328] [CDCSDK] Fix bug in GetTabletIdsToPoll when only 1 child tablet has been polled
• [18344] [YCQL] Fix TOJSON() to also unpack nested FROZENs
• [18519] [CDCSDK] Stop sending checkpoint details for unpolled child tablets in UpdatePeersAndMetrics
• [18592] [DocDB] Fix log message format in WritePostgresConfig
• [18652] [YSQL] YB Sequential Scans don't take locks in SERIALIZABLE isolation

Known issues

N/A

v2.16.6.1 - August 7, 2023 {#v2.16.6.1}

Build: 2.16.6.1-b1

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.6.1-b1

Bug fixes

This release includes a fix for YugabyteDB Anywhere and no changes to YugabyteDB (the core database).

v2.16.6.0 - July 20, 2023 {#v2.16.6.0}

Build: 2.16.6.0-b43

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.6.0-b43

New features

• [16794] [DocDB] Make full compactions non-exclusive by default

Improvements

• [14473] [DocDB] Accept registrations from tservers on the same node with greater sequence numbers.
• [15186] [YSQL] expression pushdown forward compatibility - part2
• [16513] Add statistics to track DocDB keys found (total, obsolete, and obsolete past history cutoff)
• [17038] [DocDB] Async open table while updating indexes
• [17398] [CQL] Account for user query and keyspace name for prepared statements
• [17404] Use LLVM URL from the third-party dir for ASAN/TSAN
• [17608] [DocDB] Add HTTP security headers
• [17629] [YSQL] Add waitpid status code decoded messages to log
• [17633] [YCQL] Introduce GFlag to control local calls on RPC thread
• [17966] [DocDB] Unify ModifyPlacementInfo string parsing

Bug fixes

• [6597] [DocDB] fixed PartitionSchema::CreateHashPartitions to provide even hash codes distribution
• [13395] [YCQL] Return error for unset bind PK value
• [14284] [13041] [YSQL] Avoid setting read time when using serializable isolation level
• [14762] [DocDB] Prevent crashing the master if trying to restore a deleted table
• [14926] [DocDB] Prevent intent clean-up while reading YSQL system tables
• [15166] [DocDB] Fix crashes due to tablet shutdown timeout
• [15911] [15912] [DocDB] Fix transaction promotion/conflict resolution abort handling
• [16137] [DST] PITR + Sequences - Ensure generated sequence values are monotonic after restore
• [16157] [YSQL] Set PgClient session timeout value
• [16267] [17755] [DocDB] Fix issues in RBS from closest peer codepath
• [16367] [DocDB] Delete Snapshot keeps on retrying indefinitely after master failover
• [16508] [YSQL] Fix incorrect tablet count assignment on user tables created within geo-partitoned tablespace
• [16605] [17526] [DocDB] Fix write operation CHECK fail due to old schema version
• [16790] [CDCSDK] GetCheckpoint will return invalid checkpoint for unpolled tablets
• [17074] [DocDB] Root memory consumption should not include pageheap_free_bytes
• [17261] [DST] PITR - Single file compaction does not reset Hybrid Time filter
• [17318] [YSQL] Import 'Replace last PushOverrideSearchPath() call with set_config_option().' from PG
• [17455] [xCluster] Log when UpdateCdcMetrics is delayed
• [17574] [DocDB] Add EncryptedWritableFile::Size() to fix incorrect WAL footer start offset
• [17592] [17760] [DocDB] Rename WAL file after writing the file header on the disk
• [17603] [DocDB] Tablet splitting should respect FLAGS_max_create_tablets_per_ts.
• [17663] [DocDB] Clear drive load / leader load maps before appending to them
• [17712] [DocDB] Fix system partitions query to not return empty namespace name when its not set at table level
• [17726] [DocDB] A workaround for memory leak from InboundCall weak pointers
• [18139] [YSQL] Fix file filter to correctly use bound inclusiveness information
• 16009] [YSQL] Fix the system catalog versioning behaviour on CREATE TABLE with FK reference command

Known issues

N/A

v2.16.5.0 - May 30, 2023 {#v2.16.5.0}

Build: 2.16.5.0-b24

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.5.0-b24

Improvements

• [16849] [DocDB] [Tablet splitting] Lower the thresholds for splits on low core machines
• [17041] [DocDB] Optimize YCQL JSONB attribute-level upsert performance
• [17194] [DocDB] Add memtracker to track CQL requests memory usage
• [17276] [xCluster] cleanup the transaction to write_id map whenever transaction is applied
• [17358] [DocDB] Add tracking for memory used by YBMetadataCache

Bug fixes

• [14268] [Xcluster] Fix GetChangesOnSplitParentTablet
• [15769] [YSQL] Fix functional indexes returning wrong results
• [16240] [YSQL] Use per-tuple memory context in relcache preloading
• [16863] [DocDB] Ignore missing YSQL indices when importing a snapshot.
• [16976] [YSQL] Fix postgres crash when pg_stat_progress_create_index has an index from a different database
• [16999] [DocDB] Retry read if read buffer was full on previous step
• [17026] [Xcluster] Fix metrics jump when regaining leadership
• [17107] [YCQL] Prevent DROP TYPE for the used YCQL type
• [17163] [YCQL] Use Metadata Cache by default in PREPARE
• [17317] [YSQL] Import Handle RLS dependencies in inlined set-returning functions properly.
• [17362] Avoid double memory tracking of request data and fix tracking for plain response buffer size
• [17380] [YSQL] Make sure ybctid_column_value is still sent in index scans for upgrade compatibility
• [17480] [DocDB] Update to NULL could be lost after compaction when packed row is enabled
• [17502] [DocDB] Exclude yugabyted-ui from sed binary manipulation.
• [17502] Fix the shared library interpreter used for yugabyted-ui

Known issues

N/A

v2.16.4.0 - May 10, 2023 {#v2.16.4.0}

Build: 2.16.4.0-b32

Docker

docker pull yugabytedb/yugabyte:2.16.4.0-b32

New features

• [17165] [CDCSDK] Added serve_as_proxy member to GetCheckpointRequestPB

Improvements

• [506] [YCQL] Disable alter table rename on columns with index
• [3090] [YCQL] Limit topology change events to subscribed connections
• [13718] [DocDB] Remove const modifier for iterator HasNext and GetNextReadSubDocKey
• [15362] [DocDB] Reuse regular RocksDB upper bound for intents RocksDB
• [15746] [DocDB] harden master leaderless tablet endpoint
• [15987] [YSQL] Use tablespace leader preference as a cost component in planner
• [16058] [CDCSDK] Use safe time from request rather than response
• [16108] [DocDB] Start transaction promotion immediately even if not ready yet
• [16571] [DocDB] Protect InboundCall::trace_ with the mutex/atomic_ptr
• [16581] [DocDB] Remove outstanding tablet split limit by default.
• [16666] [YSQL] Switch to new query context for get_tablespace_distance() operations
• [16697] [DocDB] Disallow multiple OpenTable calls in YBMetaDataCache::GetTable
• [16789] [YSQL] Assume that CAPABILITY_PickReadTimeAtTabletServer is enabled all the time
• [16868] [DocDB] Sanitizes debug messages

Bug fixes

• [13862] [YSQL] Fix metric for total number of connections made
• [15090] [DocDB] Tablet splitting: Some queries fail with "The tablet is shutting down" during the workload
• [15449] [CDCSDK] Send snapshot end marker when snapshot has no data
• [16131] [DST] B/R, PITR - Fix invalid enum value of restoration state object
• [16329] [YSQL] Fix metric for rejections due to connections limit
• [16565] [DocDB] Fixed missing data in case of RocksDB iterator error
• [16583] [YSQL] Fix IndexOnlyScan with no targets returning empty rows
• [16631] [DocDB] Fix delete_snapshot sending delete RPCs indefinitely for deleted tablets
• [16632] [DocDB] Fix TestIndex.testDropDuringWrite
• [16669] [DocDB] Fix tablet splitting / PITR deadlock
• [16875] [DST] PITR - Extend master's sys catalog retention based on snapshot schedule
• [16964] [DocDB] Compilation failure for Apple clang 14.0.3

Known issues

N/A

v2.16.3.0 - April 18, 2023 {#v2.16.3.0}

Build: 2.16.3.0-b43

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.3.0-b43

New features

• [15499] [DocDB] Enable lease revocation
• [15735] [DocDB] Add Raft Config info to consensus information in YB-Tserver UI
• [16124] [DocDB] Add mem_tracker for RunningTransaction objects in TransactionParticipant
• [16253] [YSQL] Support expression pushdown for ybgin index scans

Improvements

• [10696] [YSQL] Avoid reading sys catalog from followers
• [13163] [CDCSDK] Making CDC checkpoint logs on tablet peer periodic
• [14063] [DocDB] Support packed row in sst_dump
• [15629] [DocDB] Update last_received_current_leader sent from replica to leader by including de-duped messages as well
• [15794] [DocDB] Refactor AreLeadersOnPreferredOnly to Account for Geo Partitioned Table
• [15883] [DocDB] Check permission status of YCQL indices in CatalogManagerIsDeleteTableDone
• [15937] [DocDB] Don't initiate packing with tombstoned column
• [15952] [YSQL] Reduce verbosity of a Postgres log line in read committed isolation level codepath
• [16006] [DocDB] execute callbacks for local RPC calls the same way as for remote calls
• [16034] [15933] [YSQL] Avoid picking hybrid timestamps in YSQL backends
• [16091] [DocDB] Changing post-split compaction to run non-exclusively
• [16138] [DocDB] Tablet splitting should be blocked on system_postgres.sequences_data
• [16147] Correctly set cdc_sdk_min_checkpoint_op_id to -1.-1 in upgrade scenarios
• [16163] [DocDB] Change IsDeleteTableInProgress treatment of alive Index
• [16474] [DocDB] Change PgClientService session_expiration_queue_ to min heap
• [16679] [CDCSDK] Correcting safe_time bug for batches with only commit record
• [DocDB] Bump RBS session default timeout
• [DocDB] Release RequestScope in WriteQueryExecuteDone()

Bug fixes

• [13379] [DocDB] Fix BlockBasedTable memtracker being unregistered in Tablet destructor
• [13467] [YSQL] Fix ANALYZE error after ALTER TABLE DROP COLUMN
• [14519] [YSQL] Prevent 'Restart read required' error in case of sys catalog cache refresh
• [14716] [DocDB] LDB tool is crashing while trying to dump manifest file
• [15220] [15786] [15947] [DocDB] Fix apply-operation timeout during schema changes
• [15286] [YSQL] Fix premature return in PG_TRY block in relcache.c
• [15528] [DocDB] Fix usage of Connectionoutbound_data_being_processed_ during Shutdown
• [15740] [CDCSDK] Failed to decode Before image record with packed row enabled and multiple table alter
• [16048] [YSQL] Fix regression in performance of queries returning large columns
• [16151] [DocDB] Fix incorrect iteration over modified set in CleanupAbortsTaskRun
• [16173] [DocDB] Cluster creation is failing on Mac
• [16444] [DocDB] Ignore heartbeats from peers not in reported config
• [16457] [YSQL] Fix segmentation fault after rollback of subtransaction with deferred
• [16507] DST PITR + Packed - Fix sys catalog restoration logic
• [16637] [CDCSDK] Fix safe_time issues around large transactions
• [DB-5855] xCluster Configuration - Data Type Missing in the Type.Java List
• [DocDB] Fix deadlock with YqlPartitionsVTable
• [DocDB] fix leader ht lease exp under leader only mode
• fix Added missing name attribute for replication factor field

Known issues

N/A

v2.16.2.0 - March 7, 2023 {#v2.16.2.0}

Build: 2.16.2.0-b41

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.2.0-b41

New features

• [10595] [YSQL] Add progress reporting for create index commands
• [13123] [YSQL] Push down nextval() validation
• [13295] [YCQL] Disable PAGING & LIMIT for aggregate functions
• [15735] [DocDB] Add Raft Config info to consensus information in YB-Tserver UI
• [15842] [DocDB] Count time from transaction init for tracing thresholds
• [16047] [CDCSDK] Mark stream as active on ns with no table with PK
• [16135] [YCQL] Use Metadata Cache in IsYBTableAltered

Improvements

• [12631] [DocDB] [YSQL] Update PGTableDesc table cache when partition list version is outdated
• [14961] [14962] [15840] [YSQL] Further reduce first-query metadata RPCs
• [15761] [YSQL] Avoid caching responses for auth process
• [15807] [DocDB] Ensure IntraTxnWriteId is properly decoded even in weak lock intents
• [15952] [YSQL] Reduce verbosity of a PostgreSQL log line in read committed isolation level codepath
• [16034] [15933] [YSQL] Avoid picking hybrid timestamps in YSQL backends
• [16147] [CDCSDK] Checkpoint set to 0.0 from raft superblock in upgrade scenarios
• [16147] [CDCSDK] Correctly set cdc_sdk_min_checkpoint_op_id to -1.-1 in upgrade scenarios

Bug fixes

• [13379] [DocDB] Fix BlockBasedTable memtracker being unregistered in Tablet destructor
• [13891] [Backfill] Do not update index permissions when table is being deleted
• [14556] [DocDB] Prevent momentary data masking during IntentAwareIterator scans in index backfill
• [15286] [YSQL] Fix premature return in PG_TRY block in relcache.c
• [15626] Move mac release builds to mac12 nodes
• [15649] fix auto re-running cmake
• [15820] [YSQL] fix ybgin index scan after DROP COLUMN
• [15849] [DocDB] Don't hold mutex when starting TransactionStatusResolver
• [15863] [Backfill] Fixed race in Tablet::FlushWriteIndexBatch
• [15874] [YCQL] Fix wrong metadata version with pagination on SELECT using Index
• [15881] [YSQL] unpushable SET clause expressions caused unnecessary index updates
• [15937] [DocDB] Don't initiate packing with tombstoned column
• [15969] [YSQL] Control inclusiveness info being sent over in QL_OP_BETWEEN with an AutoFlag
• [16020] [YSQL] Fix yb_get_range_split_clause primary key constraint wrong assumption
• [16048] [YSQL] Fix regression in performance of queries returning large columns
• [16082] [YSQL] ysql_dump cannot correctly dump primary key with INCLUDE clause for table schema

Known issues

N/A

v2.16.1.1 - February 13, 2023 {#v2.16.1.1}

Build: 2.16.1.1-b3

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.1.1-b3

Bug fixes

• [16020] [YSQL] Fix yb_get_range_split_clause primary key constraint wrong assumption
• [15969] [YSQL] Disable inclusiveness info being sent over in QL_OP_BETWEEN

Known issues

N/A

v2.16.1.0 - February 2, 2023 {#v2.16.1.0}

Build: 2.16.1.0-b50

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.1.0-b50

New features

• [14939] [YSQL] Allow admin to limit the number of successive failed login attempts by a role
• [15368] [YCQL] Add mechanism to reset password as a non-authenticated user

Improvements

• [14886] [DocDB] Add compaction types to compaction task metrics

Bug fixes

• [14417] [DocDB] Tserver Registration hazard - UUID can be blank
• [14489] [DocDB] Fix term mismatch caused by stale consensus meta data passed to InitTabletPeer
• [14614] [YSQL] Cleanup after terminated connections
• [14789] [CDCSDK] Do not send checkpoint to tablet peers for tablets in uninitialized state
• [14913] [DocDB] Start hinted leader election only on running replicas
• [15016] [DocDB] YSQL select max() returns inconsistent result after tablet splitting
• [15051] [DocDB] Retain schema packing for non flushed entries
• [15051] [DocDB] Take intents into account for retaining schema packing
• [15051] [DocDB] Use schema version during apply intents
• [15087] [xCluster] Fix ranged keys + uneven tablet counts
• [15172] [YSQL] Move the ybRunContext to be a child of TopPortalContext
• [15230] [15386] [CDCSDK] setcheckpoint never fails with non-primary key table
• [15309] [CDCSDK] With larger transaction we miss before image.
• [15384] [CDCSDK] Ignore cases where safe_time not set in PopulateTabletCheckPointInfo
• [15418] [YSQL] Fix rolprffailedloginattempts counter behaviour
• [15433] [CDCSDK] Only delete parent tablet after a successful GetChanges call on both children
• [15435] [YSQL] Exclude some auth methods from profile handling
• [15443] [DocDB] Fix setting cotable id for non primary ysql catalog tables
• [15516] [CDCSDK] Data loss with CDC snapshot plus streaming
• [15559] [CDCSDK] Prevent updating LastReplicationTime when reporting TabletSplit error
• [15562] [CDCSDK] In cases of not using the before image remove safe time after a successful snapshot
• [15698] [DocDB] Handle null placement info due to faulty restores
• [15705] [CDCSDK] Update CDCSDK checkpoint non-actionable replicated messages in WAL
• [15763] [DocDB] Fix reading packed row with column updated to NULL

Known issues

N/A

v2.16.0.1 - January 6, 2023 {#v2.16.0.1}

Build: 2.16.0.1-b7

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.0.1-b7

Bug fixes

• [14614] [YSQL] Clean up after terminated connections

Known issues

N/A

v2.16.0.0 - December 14, 2022 {#v2.16.0.0}

Build: 2.16.0.0-b90

Third-party licenses: YugabyteDB, YugabyteDB Anywhere

Downloads

Docker

docker pull yugabytedb/yugabyte:2.16.0.0-b90

New features

• [7614] [DocDB] Add an option to run a major compaction at a fixed interval
• [10935] [CDCSDK] Provide tablet split support to CDCSDK Service
• [12744] [CDCSDK] Composite type support
• [13970] [CDCSDK] Alter table support with CDCSDK
• [14069] [YSQL] Allow Batched Nested Loop joins for clauses that have expressions on outer variables
• [15215] [yugabyted] The yugabyted UI is now enabled by default
• [DB-2705] [YSQL] Introduce GFlag to control local file access
• [Metrics] Add a metric "ql_write_latency" to track write latency, similar to ql_read_latency

Improvements

• [12953] [YSQL] Populate query termination entry in pg_stat with OOM and seg fault errors
• [13211] [YSQL] Integrate READ COMMITTED isolation with wait queues
• [13371] [CDCSDK] CDCSDK metrics aggregation with stream level
• [13609] [YSQL] Adding Geo-distribution helper functions
• [13750] Add explicit GFlag macros for DEFINE_RUNTIME and DEFINE_NON_RUNTIME
• [13807] [xCluster] Add APIs for changing role
• [13815] [YSQL] Enhance passwordcheck with more checks
• [14106] [CDCSDK] Add Java level APIs to fetch tablets from the cdc_state table
• [14123] [DocDB] Propagate new AutoFlag configs to Tservers
• [14128] [DocDB] Track ql_read_latency for SQL too
• [14140] [YSQL] Import Make contrib modules' installation scripts more secure
• [14265] [DocDB] speedup split child tablet local bootstrap by copying in-memory retryable requests structure from parent tablet
• [14280] [DocDB] Avoid decoding doc hybrid time during compaction when possible
• [14352] [YSQL] Allow for yb_db_admin users to access all yb_terminated_queries
• [14378] [YSQL] Output current ysql_max_connections as a metric
• [14474] [YSQL] Deprioritize expression pushdown in single line updates
• [14498] [YSQL] Move recently added YB NodeTags to the end of the list
• [14532] [DocDB] Add tcmalloc malloc/free stack trace sampling
• [14574] [YSQL] Complete support for PARAM_EXEC pushdown
• [14823] [CDCSDK] Change log level of line which indicates fetching next block from WAL in cdcsdk_producer
• [14876] [CDCSDK] Change logging level for CDCSDK related client errors
• [15091] [DocDB] Automatic tablet splitting is disabled by default in 2.16
• [CDCSDK] Before image support for CDC with only record types Change and All

Bug fixes

• [10735] Fix TransactionLoader race on tablet bootstrap vs tablet delete
• [10821] [YSQL] Cache sys catalog read request on local TServer side
• [10921] [CDCSDK] Populate CDC stream cache with newly added table details
• [10921] [CDCSDK] Support for newly added table to active stream's namespace
• [11019] [yugabyted] Secure deployment of YugabyteDB cluster using yugabyted CLI (Phase 2). Added cert generation experience through yugabyted.
• [11645] [YSQL] Wait for pending Perform RPCs to complete before aborting YB DDL transaction
• [11849] [YSQL] Add distributed storage access statistics to EXPLAIN ANALYZE
• [12068] [14047] [xCluster] Fix shutdown path of CDCConsumer
• [12400] Obtain Tablet from TabletPeer as a shared_ptr to avoid incorrect memory access
• [12442] [YSQL] Add colocation_id to pg_dump for materialized views
• [13102] [DocDB] Fix deadlock in DoSplitTablet.
• [13168] [YSQL] TPCC tests with restore failing with "Query error: schema version mismatch for table"
• [13367] [YSQL] Avoid using plain text LDAP password via environment variable
• [13446] [DocDB] Combine AutomaticTabletSplittingMovesToNextPhase and AutomaticTabletSplittingMultiPhase.
• [13465] [YSQL] Batch upserts in CatalogManager::CopyPgsqlSysTables when creating namespace
• [13540] [DocDB] Move row lock from primitive key type to primitive value type
• [13575] [DocDB] Fixed negative metric value from thread_running category
• [13779] [DocDB] Fix master UI on non-leader nodes when HTTPS is enabled
• [13808] [DocDB] Backups fail when two same-name namespaces exist, one RUNNING and another DELETED
• [13961] [DocDB] Add a URL param to reset histograms
• [13988] [YSQL] Multibyte chars in pushdown expressions
• [14005] [DocDB] fix potential duplicate WRITE_OP due to split and RPC timeout interference
• [14056] [DocDB] Support xCluster + Packed columns when Producer/Consumer schema_versions don't match
• [14109] [DocDB] Fix wait for safe time timeout by cleanup aborts task
• [14112] [yugabyted] [QA] Hiding DB password from the logs
• [14136] [YSQL] Enable yb_db_admin to ALTER/DROP/CREATE triggers like a superuser
• [14153] [YCQL] Allow CQL calls to request tracing on a per-session/rpc basis
• [14159] [CDCSDK] Intents get GCed during universe upgrade
• [14163] [YQL] Improve logging in case of duplicate index error
• [14184] [YSQL] Fix the partial write issue of writev/pwritev by adding looping logic
• [14201] [YCQL] Fixing partition_hash() with multiple indexes
• [14202] [YSQL] Avoid RPC from YSQL to local tserver when switching sub-txns
• [14210] [DocDB] Add Guard Around Txn Coordinator Retry Logic for Failed Apply
• [14212] [YSQL] Optimize relation's rd_replidindex preloading
• [14225] [DocDB] disallow advance max_safe_time_returned_with_lease_ before NoOp of current term is replicated
• [14289] [DocDB] Include subtransaction_id in value of weak intents
• [14293] [CDCSDK] upgrading the universe from 2.15.3 to 2.15.4, causing intent GCed
• [14296] [DST] Packed + Backup/Restore - Merge schema packing history on restore
• [14298] [DST] [PITR] MetaCache flips replica state from failed to not failed for hidden tablets
• [14309] [YSQL] Store TServerSharedObject by value in PgApiImpl
• [14335] [YSQL] Release YbScanDesc on scan end
• [14349] [xCluster] Fix get_xcluster_estimated_data_loss
• [14365] [YSQL] Handle backups of range-partitioned indexes with null values in partition bounds
• [14377] [YSQL] Do not pushdown expressions with collation
• [14379] [DocDB] Fix Flags generated files for LTO builds
• [14394] [yugabyted] gen_certs.sh file not present in the release versions.
• [14407] [ysql] Fix missing NULL check for MyBEEntry in pgstat_report_query_termination()
• [14424] [YCQL] Persist where predicate in IndexInfo for PartialIndexes
• [14426] [yugabyted] [QA] Added validation for hostnames provided through CLI.
• [14446] [YSQL] Enable yb_db_admin to SELECT/INSERT/UPDATE/DELETE rows in tables like a superuser
• [14479] [CDCSDK] Add bool parameter to error out of UpdateCdcReplicatedIndex RPC fails on any TServer
• [14533] [DocDB] Colocated table creation optimization
• [14552] [DST] [PITR] Do not crash if table not found from colocated list
• [14554] [DocDB] Fixed RBS file downloader trace
• [14555] [xCluster] Use xCluster Role to determine when xCluster safe time is computed
• [14561] [DocDB] Consolidate the use of RegisterFlagValidator and DEFINE_validator
• [14582] [DocDB] Fix livelock issue in pessimistic locking
• [14595] [CDCSDK] tserver crash during snapshot mode
• [14596] [DocDB] JSON escape the GFlags in CallHome
• [14632] [DocDB] disallow packed row for colocated tables
• [14643] [DocDB] Fix strongly typed bool constants
• [14652] Fix xcluster_safe_time_service Init
• [14695] [YSQL] YBCRestartTransaction has no parameter
• [14713] [yugabyted] [QA] 'yugabyted status' command shows Stopped even when the node is destroyed.
• [14746] [CDCSDK] Fixing the erroneous behaviour of "cdcsdk_last_sent_physicaltime" metric
• [14788] [CDCSDK] Fixes DDL messages for colocated tables
• [14797] [DocDB] Prevent repeat resumption of waiting operation if first resumption was error
• [14825] [YSQL] Fix ysql_dump use-after-free in getYbSplitClause
• [14826] [DocDB] Rename enable_wait_queue_based_pessimistic_locking to enable_wait_queues
• [14846] [CDCSDK] Add support for tablet split for newly added tables
• [14855] [yugabyted] Java install check is no longer needed for yugabyted start command
• [14881] [CDCSDK] Fix issue where "cdcsdk_sent_lag_micros" metric shows sudden peaks.
• [14909] [CDCSDK] Fix issue where 'cdcsdk_change_event_count' metric is incremented even on empty batches
• [14931] [CDCSDK] YBClient unable to find a TabletClient for child tablets which are split further
• [14950] [CDCSDK] Add tablet_id param to GetTabletListToPollForCDC RPC
• [15004] [CDCSDK] Data loss when streaming transaction intents for large tables
• [15006] [DocDB] set same error to operations with same request id when retrying a batcher
• [15025] [YCQL] Fixing Index updating by DELETE for a deleted row
• [15031] [CDCSDK] Restrict compaction during the CDC snapshot
• [15047] [CDCSDK] Stop hiding tablets when table deleted
• [15051] [DocDB] Log schema packing versions
• [15062] [CDCSDK] Differentiate between Update and Insert records in case of packed row
• [15064] [15033] [CDCSDK] Restrict compaction only for CDC snapshot mode and for before image type
• [15067] [CDCSDK] Java level changes in the API to create CDC stream for before image
• [15100] [CDCSDK] Populate the old/new tuples correctly in the context of single shard update
• [15124] [CDCSDK] Create rows for child tablets for all tablets of stream
• [15124] [CDCSDK] Fix bug in GetTabletListToPollForCDC
• [15136] [CDCSDK] Fix for crash when running CDCSDK with before image
• [15154] [CDCSDK] Before image failure with packed row
• [15157] [CDCSDK] Ignore intents with kRowLock in GetIntentsBatch

Known issues

N/A