What's new in the v2.19 release series

Release announcements

v2.19.0

• Part 1: Dream Big, Go Bigger: Turbocharging PostgreSQL

v2.19.2

• Part 2: Built-in Connection Manager Turns Key PostgreSQL Weakness into a Strength
• Part 3: Simplify The Lift and Shift of PostgreSQL Apps With Familiar Read Committed Behavior

Release notes

What follows are the release notes for the YugabyteDB v2.19 release series. Content will be added as new notable features and changes are available in the patch releases of the YugabyteDB v2.19 release series.

v2.19.3.0 - October 25, 2023 {#v2.19.3.0}

Build: 2.19.3.0-b140

Downloads

Docker

docker pull yugabytedb/yugabyte:2.19.3.0-b140

New features

• [18241] [YSQL] Enable switching of 'user/role' context along with session parameter context
• [18583] [YSQL] Publishing YSQL Connection Manager stats on REST API
• [18723] [YSQL] Enable one pool per database in YSQL Connection Manager
• [19049] [YSQL] Use appropriate database for control connection in YSQL Connection Manager

Improvements

• [12574] [xCluster] Hold reference count of YBTable in XClusterWriteRpc
• [12751] [CDCDK] Made maxAttempts and sleepTime for retrying RPCs configurable in AsyncClient
• [16224] [xCluster] Deprecate cdc_ybclient_reactor_threads
• [16265] [xCluster] Rename to xCluster RPCs and consumer registry service
• [16309] [YSQL] DROP SEQUENCE is not atomic
• [16783] [DocDB] Display all per-rpc metrics changes in EXPLAIN (ANALYZE, DIST, DEBUG)
• [16784] [DocDB] Emit changed metrics in RPC tracing
• [17302] [YSQL] Re-enable wait-on-conflict concurrency control in TestPgTransparentRestarts
• [17511] Use Clang 16 for TSAN
• [17560] [DocDB] Increase load balancer leader moves per run
• [18078] [DocDB] Reconstruct Prometheus metric aggregation
• [18082] [YSQL] Stop using ForeignScan for YB relations
• [18105] [xCluster] Define CDCStreamInfoPtr
• [18216] [YSQL] Implement creating/managing client context in shared memory for YSQL Connection Manager
• [18482] [yugabyted] Improve logging of yugabyted-ui, and changes to make it more robust against node failures
• [18491] [DocDB] Handle Split tablets for pg_locks
• [18503] Convert a few remaining non-gtest tests to gtest
• [18510] [CDCDK] Support for new BeforeImage modes on yb-admin CLI
• [18510] [CDCDK] Support partial before image modes
• [18532] [YSQL] Create WeakRead intent instead of StrongRead in case only key columns are read
• [18561] [DocDB] Enable allocation sampling endpoint for gperftools tcmalloc.
• [18584] [DocDB] Allows writes during ysql_upgrade on STANDBY cluster
• [18606] [xCluster] Remove ExternalTxnIntentsState map
• [18627] [DocDB] New strategy for MemTracker's metric name creation
• [18632] [xCluster] Schedule Poller delay work on rector instead of sleeping on the thread pool thread
• [18642] [DocDB] Tablet Splitting: Gracefully handle splits of large tablets to ensure writes don't get throttled during splitting
• [18671] yb-admin get_xcluster_info should return valid JSON
• [18679] [YSQL] Refactor YbBindSearchArray to better determine if recheck is needed
• [18685] Track stuck OutboundCalls and enforce timeouts at Reactor level
• [18686] [YSQL] Import pgvector extension into yugabyte-db
• [18698] [xCluster] Check for existence of data in table for IsBootstrapRequired check of a new stream
• [18744] [DocDB] Add ability to recover from follower lag caused by stuck OutboundCall
• [18757] [DocDB] Make WAL reuse always happen
• [18760] [YSQL] Encode blocked_by in yb_lock_status() as UUIDARRAY
• [18770] [DocDB] Ensure in-memory doc path locks get reset in failed conflict resolution path as well
• [18770] [DocDB] Stricter memory order in PreparerImpl::Run to prevent re-ordering of instructions
• [18772] [DocDB] Improve Preview flag description
• [18782] [yugabyted] API changes to send verify phase when data import is complete.
• [18782] Voyager migrations visualisation API implementation.
• [18790] [18647] [DocDB] Skip acquiring weak lock on primary table for locks of colocated tables
• [18793] DocDB tools: Sys-catalog reader tool
• [18793] Tools: Clean-up in sys-catalog-tool
• [18802] [YSQL] Integrate pgvector as an extension
• [18808] A write-once lock-free weak pointer class
• [18810] Move thirdparty regress tests to proper place
• [18816] [CDCDK] Enhance the stream mode FULL_ROW_NEW_IMAGE mode
• [18818] [yugabyted] Cloud based fault tolerance is supported with the yugabyted start command.
• [18823] Refactor ConsensusMetadata::Create to use Result.
• [18826] Build: thirdparty_tool --list-compilers option enhancement
• [18827] [YSQL] ORDER BY support for pgvector extension
• [18848] [YSQL] Preserve order in case of conversion IN operator into list of ybctids
• [18854] [YSQL] SELECT INTO and CREATE TABLE AS cause catalog version increase
• [18855] Add functions for formatting a CoarseTimePoint relative to the current time
• [18860] [DocDB] Improve packed row v2 decoding
• [18875] Enable Odyssey by default in yb_release for core database Linux releases
• [18877] Log invalid OutboundCall state transitions and other OutboundCall improvements
• [18880] [xCluster] Set XClusterAsyncExecutor vlogs to level 5
• [18903] Use Clang 16 for the release build
• [18917] [YSQL] Pass the database max connection setting to the YSQL connection manager layer
• [18923] [DocDB] Disable retryable requests flusher temporarily
• [18941] [DocDB] Add debug view for SharedLockManager
• [18998] [DocDB] New flag to force shell mode for empty masters regardless of master_addresses.
• [18999] [YSQL] Avoid abort transactions in backfill get safe time
• [19003] [YSQL] The tablespace caching task in YB-Master must skip deleted namespaces
• [19017] [yugabyted] yugabyted UI redirects should work with page refresh
• [19019] [DocDB] Promote remote_bootstrap_from_leader_only to an AutoFlag
• [19020] [DocDB] Change the enable_automatic_tablet_splitting autoflag to kLocalPersisted
• [19021] [YSQL] Fix bug in computation of semi/anti join factors during inner unique joins
• [19035] DocDB Add debug UI for tablet preparer state
• [19051] [DocDB] Fallback to RPC when shared memory exchange is not yet ready
• [19055] [YSQL] Adjust index scan costing to only charge DOCDB context switch overhead once per tuple
• [19069] [YSQL] Build exact tlist for YB relations
• [19075] [YSQL] Introduce the PGConn::FetchAll helper method
• [19097] [DocDB] Escape output of master and tserver UI endpoints
• [19108] [YSQL] Client to server encryption support for YSQL Connection Manager
• [19109] [YSQL] Publish Prometheus metrics for YSQL Connection Manager
• [19115] [DocDB] Cache best child in BinaryHeap
• [19124] [YSQL] Revert 'Fix bug in computation of semi/anti join factors during inner unique joins'
• [19151] Refactor - remove GetInt32, GetInt64, GetString
• [19176] [DocDB] Add URL parameter to disable Prometheus metrics help and type metadata
• [19182] [YSQL] refine the condition of "switching to per-db mode"
• [19208] Refactor stack trace related code
• [19240] Support building with Clang 17 and GCC 13
• [19292] [CDCDK] Publish snapshot_key in GetCheckpointResponse only if it is present

Bug fixes

• [10429] [18653] [YSQL] Plans created at one isolation level can crash or execute incorrectly when executed at other isolation levels
• [13030] [DocDB] Increase tablet_creation_timeout_ms to avoid replacing tablets behavior
• [15532] [CDCDK] Fixed issues for running CDC on YCQL tables
• [16212] [YSQL] Fix conflict detection in serializable isolation in case of explicit row locking
• [16556] [CDCDK] Only ignore unpolled tablets depending on context
• [17148] [DocDB] Fixed wrong metric_type of Prometheus metric
• [17682] [DocDB] Fix hybrid time too low due to race between TransactionParticipant::Poll and RaftConsensus::Start in rf-1 setup
• [17745] [DocDB] Fix max_nexts_to_avoid_seek flag
• [17822] [DocDB] Disable packed row for org.yb.pgsql.TestPgColumnReadEfficiency.testScans
• [18121] [DocDB] respond VoteGranted with majority replicated lease and ht lease from the old leader
• [18219] [YSQL] Avoid sending ybctid target in select *
• [18281] [YSQL] Fix query layer's read restart retries in Read Committed isolation
• [18343] [DocDB] Part fix for yb_lock_status isolation tests
• [18387] [DocDB] Select statement fails with TABLET_DATA_SPLIT_COMPLETED
• [18513] [YSQL] Fix estimated seeks and nexts in base table for Index Scan
• [18658] [xCluster] Skip local peer during bootstrap if not ready/valid
• [18668] [DocDB] Block stack collection during thread creation
• [18689] [YSQL] Fix occasional SIGSEGV in yb_cost_index
• [18711] [YSQL] Fix pg_stat_activity taking exclusive lock on t-server session
• [18744] [DocDB] Fixed issue with last_rpc_start_time_ reset which resulted in tsan race in connection
• [18751] [YCQL] Miscellaneous fixes for YCQL statement metrics
• [18761] [YSQL] Prevent READ COMMITTED semantics in DDL statements
• [18782] Fixing /api/migration_assesment to get issues from Report
• [18786] [DocDB] Fix HdrHistogram::CountAt data race
• [18831] [DocDB] Make MemTracker's collapse button visible
• [18838] [DocDB] Fix incorrect load balancing of dead node tablets
• [18879] [YSQL] Disable pushdown for TextSearch functions that are not thread safe
• [18918] [DocDB] Fix heap-use-after-free in PgLeaderChangeWaitQueuesTest.AddTwoServers
• [18919] [DocDB] Memory leak in yb::[DocDB]:DocDBRocksDBUtil::CotablePacking in DocOperationTest.MaxFileSizeWithWritesTrigger
• [18928] [YSQL] Fixed multiple bugs for YSQL Connection Manager
• [18939] [YSQL] SimulateUpgradeToPerdbMode fails if tserver cache is turned on
• [18946] [DocDB] Fix potential raft deadlock caused by persisting retryable requests
• [18995] [DocDB] TSAN race in PgRangePartitionedTableSplit tests
• [18997] [YCQL] GRANT does not need access to recipient_role
• [19021] [YSQL] Fix bug in computation of semi/anti join factors during inner unique joins
• [19043] [YSQL] Remove incorrect references to MyProc in RemoveLockGroupLeader
• [19090] Disable reactor-based outbound call expiration by default
• [19090] Fix a memory leak in tracked outbound calls in Reactor
• [19116] [DocDB] Remove enable_deadlock_detection flag and introduce enable_wait_on_conflict
• [19210] [DocDB] Fix spacing before args in yb-admin commands
• [19295] [yugabyted] Password Authentication showing enabled in UI even when it's not turned on.
• [19308] [YSQL] Fix bug where tuple IN filters were not bound to the request
• [19316] [19314] [yugabyted] Bugs regarding --join flag provided.
• [19348] [CDCDK] Only delete removed tablets from cdc_state in CleanUpCDCStreamsMetadata
• [19385] [CDCDK] Fix WAL GC issue for tables added after stream creation
• [19394] [CDCDK] Set snapshot key from correct parameter in explicit checkpointing; Only populate key to GetChangesRequest when it is not null
• [19434] [CDCSDK] Do not completely fail while fetching tablets if one table hits error
• [19440] [DocDB] Fix bug where invalid filter key was passed to Iterator initialization in backwards scans
• [19482] [CDCSDK] Add Flag to disable tablet split on tables part of CDCSDK stream
• [19502] Do not DFATAL when failing to queue an inbound connection heartbeat
• [YSQL] Fix CREATE INDEX for ysql_conn_mgr

Known issues

N/A

v2.19.2.0 - September 11, 2023 {#v2.19.2.0}

Build: 2.19.2.0-b121

Downloads

Docker

docker pull yugabytedb/yugabyte:2.19.2.0-b121

Highlights

• YSQL Connection Manager {{<tags/feature/tp>}}, a built-in server side connection pooling mechanism that overcomes SQL support limitations of some popular pooling mechanisms such as PgBouncer and Odyssey.

New features

• [16862] [YSQL] Pre-creation of server connection in YSQL Connection Manager
• [18216] [YSQL] Implement creating/managing client context in shared memory for YSQL Connection Manager
• [18583] [YSQL] Publishing YSQL Connection Manager stats on REST API
• [18723] [YSQL] Enable one pool per database in YSQL Connection Manager

Improvements

• [4332] [YSQL] Remove Physical Files for Drop Temp Relation
• [4415] [YSQL] Use default value for existing rows after ALTER TABLE ADD COLUMN ... DEFAULT
• [4971] [DocDB] Track memory consumed by retryable requests
• [10429] [18316] [YSQL] Turn off the yb_lock_pk_single_rpc flag by default
• [10429] [18589] [YSQL] Do single-RPC locking planning without propagation
• [12417] [YSQL] add per-db catalog version mode upgrade support
• [13358] [YSQL] Part 4 - Enable DDL Atomicity for Indexes/Matviews/ColocatedTables/Tablegroups
• [14164] [DocDB] tidy up some redundant function signatures
• [14203] [DocDB] Implement encoding/decoding for a new Postgres-like packed row format
• [14203] [DocDB] Use packed row v2
• [14308] [DocDB] update flag description now that savepoints work with xCluster
• [14659] [yugabyted] Give option to specify ENV variables in config file.
• [14682] [DocDB] Remove TEST_allow_duplicate_flag_callbacks
• [14744] [yugabyted] Give option to collect_logs at specified directory
• [15857] [YSQL] support IndexScan aggregate pushdown
• [16078] [17756] [YSQL] Preserve split properties during table rewrite
• [16196] [DocDB] Add per-tablet's metrics memory usage to MemTracker
• [16782] [DocDB] Update tablet counters at the end of the RPC
• [17087] [18054] [YSQL] Exposes YB Transaction IDs in Postgres.
• [17342] [18456] [YSQL] Update RestartReadHt when control exits early from ReadYSQLDBCatalogVersionImpl
• [17384] [YCQL] Statements reset endpoint
• [17527] [DocDB] Expose coarse histograms min value to the prometheus metrics
• [17570] [DocDB] Improve leaderless tablet endpoint
• [17696] [YSQL] Add yb_latency_histogram to <server>/statements
• [17713] [DocDB] Add master side migration for TableInfo namespace_name
• [17730] [YSQL] Import Improve pruning of a default partition
• [17751] [CDCSDK] Turn flag 'enable_update_local_peer_min_index' to true by default
• [17805] [17965] [YSQL] Enhance TServer PG daemon robustness
• [17826] [17823] [YSQL] Use a more granular clock for RPC timing on Linux builds.
• [17863] [DocDB] Abort Snapshot Restore command
• [17872] [YSQL] Planner support for distinct pushdown
• [17882] yugabyted-ui: Making changes to /activities API to make it faster.
• [17891] [DocDB] Refactor mocked load balancer tests.
• [17905] [YSQL] Pick read time on tserver for distributed txn too whenever possible
• [17940] [DocDB] Include node information in pg_locks output
• [17949] [YSQL] Respect yb_fetch_size_limit for index scans
• [17952] [yugabyted] [UI] Alerts and activities integration
• [17959] [yugabyted] Use new /api/v1/tablets tserver api in go
• [17985] [DocDB] Move event stats (coarse histograms) to lightweight implementation without buckets
• [18014] [DocDB] Adding parallel flow to RBS that handles file transfers
• [18014] [YB-TServer] API to start remote snapshot transfer
• [18042] [xCluster] Speed up xCluster Poller shutdown
• [18051] Updating yugabyte-db-thirdparty release version
• [18069] [YSQL] Add support for JWT based authentication
• [18069] [YSQL] Introduce jwtcpp_util as a wrapper over JWT-CPP
• [18069] [YSQL] Update third-party dependencies & add CMake configs for JWT-CPP
• [18106] [DocDB] Break up the RegisterCommandHandlers function in yb-admin
• [18136] [xCluster] Async tasks to download snapshot files, new endpoint to bootstrap and setup replication automatically, `SetupNamespaceReplicationWithBootstrap`` create snapshot on producer and import on consumer
• [18160] [YSQL] pass exec_params through for YB Seq Scan
• [18171] [YSQL] Implement stickiness for unsupported sql in transaction pooling mode
• [18176] [YSQL] Add attnum to yb_lock_status
• [18178] [YSQL] GUCs/RUNTIME_PG_FLAGs for lock status variables
• [18196] [18197] [DocDB] Populate waitstart and blocking transactions in GetLockStatus RPC
• [18207] [DST] Fast PITR - Custom history retention for docdb metadata table from other tables of sys catalog tablet
• [18241] [YSQL] Enable switching of 'user/role` context along with session parameter context
• [18252] [YSQL] Populate the 'node' field in yb_lock_status()
• [18255] [DocDB] Get rid of prefix in IntentAwareIterator
• [18256] [DocDB] Improve WriteBuffer performance
• [18289] [DocDB] Invoke waiter callback outside the scope of waiter's mutex
• [18293] refactor: simplify function ybcingettuple
• [18294] [DocDB] Refactor PgGetLockStatusResponsePB format
• [18319] [xCluster] Handle OpId 0 in IsBootstrapRequired
• [18321] [YSQL] Enable preloading additional tables by specifying catalog list in flags
• [18321] [YSQL] restore --ysql_catalog_preload_additional_tables
• [18323] [DocDB] Clean code related to replication of transaction status table
• [18341] [YSQL] Ensure blocked_by field of pg_locks is null if lock is acquired
• [18346] [YSQL] Decouple YBIsDBCatalogVersionMode() from multi-tenancy use case
• [18348] test: remove unnecessary tenk1_local table
• [18355] [DocDB] Populate subtransaction_id for waiters in pg_locks
• [18379] [DocDB] Retain schema packing while creating snapshot
• [18391] [17864] [YSQL] RPC timers in write path default to granular clocks in Linux.
• [18397] refactor: make ybcin funcs static
• [18398] [DocDB] Enforce timeout on YB Session
• [18398] [DocDB] Set deadline on Pg Client Session
• [18400] Build: Turn BUILD_VALGRIND option off in odyssey build
• [18404] [xCluster] Refactor xcluster tests
• [18407] [DocDB] Adding test to measure PgKeyDecoder performance
• [18407] [DocDB] Build decoder for key columns
• [18413] [YSQL] Inconsistent schema version mismatch reported for SELECT and INSERT
• [18415] [DocDB] Filter locks corresponding to aborted subtransactions from pg_locks
• [18418] [CDCSDK] Set cdc_populate_safepoint_record default to true
• [18429] [DocDB] Annotate expected race within pg_locks_test_base library
• [18445] [CDCSDK] Call ResolveIntents if consistent_safe_time is stuck
• [18449] [YSQL] Support handling of system columns in pg_doc_op row width estimation
• [18452] [YSQL] Remove post-test logging from rewrite_test_log.py
• [18457] [YSQL] Tserver flag for the number of worker threads
• [18465] [DocDB] Dump PeerManager to tserver debug ui consensus page
• [18467] [CDCSDK] Do not send safepoint records in snapshot phase
• [18468] [DocDB] Refactor transaction_locks in TabletLockInfoPB
• [18480] [DocDB] New serialization for postgres read response
• [18484] [YSQL] Cost model for Sequential and Index scan
• [18485] [YSQL] YbSeqScan support for tableoid
• [18490] [xCluster] Test to make sure cdc checkpoint is updated on new peers
• [18491] [DocDB] Handle Split tablets for pg_locks
• [18519] [CDCSDK] Stop sending checkpoint details for unpolled child tablets in UpdatePeersAndMetrics
• [18521] [CDCSDK] Make fields in CdcSdkCheckpoint protected
• [18530] [YSQL] Expression pushdown in index scan on partitioned tables
• [18533] Add tablet-level wait queue time histogram metric
• [18543] [YSQL] Allow to use std::optional with GetValue for all available types
• [18547] [xCluster] Make SetupReplicationWithBootstrap Async
• [18547] [xCluster] yb-admin setup_namespace_replication_with_bootstrap
• [18549] [DocDB] Load rows of MemTracker UI page lazily
• [18553] [DocDB] Build decoder for packed row
• [18555] [CDCSDK] Refactored cdcsdk_ysql_tests to add waiting mechanism for GetChanges() call
• [18570] [YSQL] Distinct pushdown support for join queries, Separate distinct pushdown related API into a separate file
• [18586] [YCQL] Return reserved keywords in double quotes
• [18607] [DocDB] Add intentsdb metrics changes to RPC response
• [18617] [YSQL] optimize EXPLAIN index agg on non-YB rel
• [18631] [DocDB] wait retryable_requests_flusher to be quiet at TabletPeer::CompleteShutdown
• [18640] refactor: remove ASSERT_NOT_OK, EXPECT_NOT_OK
• [18642] [DocDB] Code refactoring to match usage of has_been_fully_compacted to a proper name.
• [18663] [YSQL] Make ysql_ident_conf_csv a non-preview GFlag
• [18679] [YSQL] Refactor YbBindSearchArray to better determine if recheck is needed
• [18686] [YSQL] Import pgvector extension into yugabyte-db
• [18708] [YSQL] Reset row limit for subsequent requests of queries with LIMIT when using size-based limits
• [18721] [YSQL] Port assignment of ysql connection manager and backend db
• [18802] [YSQL] Integrate pgvector as a yugabyte extension
• [18818] [yugabyted] Cloud based fault tolerance is supported with the yugabyted start command.
• [18827] [YSQL] ORDER BY support for pgvector extension
• [18875] [YSQL] Enable Odyssey by default in yb_release for core database Linux releases
• 18941] [DocDB] Add debug view for SharedLockManager

Bug fixes

• [10429] [18653] [YSQL] Plans created in one isolation level can crash or be incorrect when executed at other levels
• [14114] [18266] [DocDB] Fix upgrade issue with GetTransactionStatusResponsePB proto message due to recent regression
• [14114] [YSQL] Avoid retrying deadlock errors in read committed isolation
• [14164] [DocDB] fail sys catalog writes with stale tablet or table info to fix heartbeat & PITR race
• [14210] [DocDB] fixed YSQL tablet routing
• [14495] [DocDB] Cleanup failed namespace from by name map in ProcessPendingNamespace, Fix heap-use-after-free in ProcessPendingNamespace
• [14658] [yugabyted] Deleting the conf directory when yugabyted destroy command is used.
• [16196] [DocDB] Fix ASAN build
• [16441] [YSQL] Retry CREATE DATABASE when oid collision happens
• [16446] [YSQL] Make yb_pg_batch_detection_mechanism configurable at runtime for session
• [16630] [YCQL] Fixing data race on Executor::ResetAsyncCalls::num_async_calls_
• [16711] [DocDB] Only display AutoFlags in UI that are relevent to the process
• [16813] [DocDB] Remove SegmentedQueue from LockfreeTest.QueuePerformance
• [16906] [CDCSDK] Removed the creation of second unused stream in the test CDCSDKYSQLTest.TestCDCLagMetric
• [16974] [YSQL] Fixing inconsistent state in pg_database when namespace creation fails
• [17043] [YSQL] Fix check to see when to attempt binding hash key options in scanspec
• [17468] Retry ysqlsh in the Docker-based package check
• [17659] [YSQL] fix ALTER TABLE DROP CONSTRAINT IF EXISTS
• [17936] [YSQL] Ensure tserver retries row-locking as best-effort if no read time is specified by the query layer
• [17938] [xCluster] minor fixes for xCluster architecture page (#18414)
• [17961] [18008] [YSQL] Fix max connection limit errors with additional locking
• [18018] [YSQL] clean up index scan recheck code
• [18023] [DocDB] Log the result of chronyd when tserver crashes with a high clock skew
• [18149] [DocDB] Fix error while decoding subdockey from lock entries in wait-queue
• [18184] [DocDB] Fix crash in case of IN handling
• [18200] [YSQL] Fix bug disabling row prefetching when using size based fetching
• [18205] [DocDB] Return correct table_id in GetLockStatus
• [18219] [YSQL] Avoid sending ybctid target in select *
• [18227] [DocDB] Fix log reader missing max_replicate_index
• [18281] [YSQL] Fix query layer's read restart retries in Read Committed isolation
• [18286] [14114] [18435] [YSQL] Fix regression caused by error code set in reported deadlocks
• [18296] [YSQL] fix aggregate pushdown indexqual recheck
• [18298] [YSQL] Fix conflict detection in query with collocated tables
• [18307] [DocDB] Do not overwrite tserver CancelTransactionResponsePB node responses in pg_client_service
• [18308] [DocDB] Fix undefined behavior in WaitQueue::GetLockStatus
• [18309] [DocDB] Disallow use of deadlock detection if enable_wait_queues=false
• [18322] [YSQL] fix index ScanKey bugs
• [18323] [xCluster] Don't allow Bootstrap of transaction status table
• [18328] [CDCSDK] Fix bug in GetTabletIdsToPoll when only 1 child tablet has been polled
• [18343] [DocDB] Part fix for yb_lock_status isolation tests
• [18344] [YCQL] Fix TOJSON() to also unpack nested FROZENs
• [18352] [YSQL] Fix ALTER TABLE with multiple ALTER COLUMN ... TYPE failures
• [18358] [DocDB] Flush regular db immediate if intents db write is blocked by immutable mem tables
• [18387] [DocDB] Select statement fails with TABLET_DATA_SPLIT_COMPLETED
• [18388] [yugabyted] Fixing yugabyted start --secure command
• [18393] [DocDB] Fix unknown int64 flags incorrectly getting tagged as runtime
• [18404] [xCluster] Remove duplicate GetTestParam in xcluster-test
• [18408] [yugabyted-ui] Fixing /api/tables to divide tables and indexes
• [18411] [DocDB] fix error message for proto dump tool
• [18412] [DocDB] fix tablet bootstrap when persisted retryable requests file is ahead of committed ops stored in the last write batch
• [18422] [DocDB] fixed DBImpl::GetMutableMemTableFrontier failure when no frontier
• [18428] [DocDB] Fixes a bug while prioritizing older transactions for pg_locks
• [18482] [yugabyted] Fix total disk usage on overview page
• [18482] [yugabyted] Fixes and updates on the yugabyted UI (#18430)
• [18492] [YSQL] Suppress NULL error fields during catalog version mismatch errors
• [18504] [YSQL] fix index agg pushdown double-count bug
• [18513] [YSQL] Fix estimated seeks and nexts in base table for Index Scan
• [18518] [DocDB] removed unused MemTracker::rand_ field
• [18531] [DocDB] Skip master tablet while processing involved tablets for pg_locks
• [18546] [YSQL] Fix row estimation and costing underestimation for BNL
• [18571] [YSQL] Move read time picking into PgClientSession in case of flushing non-transactional buffered writes
• [18592] [DocDB] Fix log message format in WritePostgresConfig
• [18615] [DocDB] Skip processing transaction meta record while loading intents of the transaction on Tablet::Open
• [18626] [YSQL] fix scan recheck perf regression
• [18635] [xCluster] Fix ordering in GetLatestEntryOpId during bootstrap
• [18645] [YSQL] Mitigate upgrade failure on yb_backend_xid add column.
• [18652] [YSQL] YB Sequential Scans don't take locks in SERIALIZABLE isolation
• [18673] test: fix bad merge on yb_aggregates.sql
• [18689] [YSQL] Fix occasional SIGSEGV in yb_cost_index
• [18710] [DocDB] Fix seg-fault in LocalWaitingTxnRegistry in case of delayed coordinator response
• [18738] [yugabyted] Corrected the spelling error in the cert command description when displaying the help output.
• [18756] [xCluster] Only Throttle XCluster Traffic from Source Side
• [18757] [DocDB] Make WAL reuse always happen
• [18760] [YSQL] Encode 'blocked_by' in yb_lock_status() as UUIDARRAY
• [18770] [DocDB] Ensure in-memory doc path locks get reset in failed conflict resolution path as well
• [18786] [DocDB] Fix HdrHistogram::CountAt data race
• [18790] [18647] [DocDB] Skip acquiring weak lock on primary table for locks of colocated tables
• [18831] [DocDB] Make MemTracker's collapse button visible
• [18917] [YSQL] Pass the database max connection setting to the ysql connection manager layer
• [18923] [DocDB] disable retryable requests flusher temporarily
• [18928] [YSQL] Fixed multiple bugs for YSQL Connection Manager
• [DB-2613] [YSQL] Remove unused log4j reference from the java modules.
• [DocDB] fix logging of collision detection when verbose logging
• [YSQL] fix CREATE INDEX for ysql_conn_mgr

Known issues

N/A

v2.19.0.0 - June 20, 2023 {#v2.19.0.0}

Build: 2.19.0.0-b190

Downloads

Docker

docker pull yugabytedb/yugabyte:2.19.0.0-b190

New features

• [1013] [YSQL] Alter Column Type: Support altering column types, Refactor add/drop primary keys in preparation for altering column type, Re-arrange functions related to cloning a table.
• [2172] [YSQL] support ALTER AGGREGATE RENAME TO
• [13488] [xCluster] Create index in transactionally consistent xCluster
• [14308] [xCluster] Support Savepoints with xCluster transactional replication

Improvements

• [506] [YCQL] Disable alter table rename on columns with index
• [10584] [DocDB] Update precompiled headers
• [13474] [DocDB] Update stable auto flags for 2.19.0.0 release
• [13494] Add Support for making /tmp directory customizable
• [14316] [DocDB] Hold SCOPED_LEADER_SHARED_LOCK during ProcessPendingNamespace.
• [14369] [DocDB] Add a flag to ignore NULL sys catalog entries during enumeration
• [14926] [DocDB] Prevent intent clean-up while reading YSQL system tables
• [15036] [xCluster] Add transactional flag to setup_universe_replication
• [15060] [DocDB] Makes backoff loop aggressive in colocated table creation path
• [15449] [CDCSDK] Send snapshot end marker when snapshot has no data
• [15644] [YSQL] Add flag to run Postgres under Valgrind
• [16116] [DocDB] Lazily flush superblock on colocated table creation
• [16230] [DocDB] compaction_status Shows Last Full Compaction Time of Table and show_tablets Option
• [16267] [DocDB] Enable RBS the gflag to bootstrap from the closest follower
• [16343] [DocDB] Include TableType in the table and cdc metrics
• [16366] [xCluster] Print missing safe time and slow safe time to the log
• [16432] [xCluster] Disallow N:1 replication topology
• [16466] [DocDB] Flush superblock before reading in RBS with lazy superblock flush
• [16561] [YCQL] Restart cluster in TestCreateTable.testCreateTableNumTablets (#16897)
• [16581] [DocDB] Remove outstanding tablet split limit by default.
• [16590] [DocDB] Add test which ensures deadlocks are detected across partitions
• [16604] [DocDB] Renames last_change_metadata_op_id to last_flushed_change_metadata_op_id in superblock
• [16673] [YSQL] Remove advanced and hidden tag for ysql_enable_profile
• [16697] [DocDB] Disallow multiple OpenTable calls in YBMetaDataCache::GetTable
• [16742] Adding additional logging for API server.
• [16746] [xCluster] Compute and propagate apply_safe_time from producer
• [16754] [DocDB] Avoid unnecessary seek during iteration
• [16756] [xCluster] Move yb-admin get_xcluster_estimated_data_loss to get_xcluster_safe_time
• [16757] [xCluster] Add transactional flag on Producer stream
• [16763] [xCluster] Reduce logging noise during xcluster errors
• [16770] [DocDB] Update YSQL serialisation
• [16778] [DocDB] Background flush task when lazy superblock flush is enabled
• [16779] [DocDB] Reuse schema packing information during scan
• [16789] [YSQL] Assume that CAPABILITY_PickReadTimeAtTabletServer is enabled all the time
• [16794] [DocDB] Make full compactions non-exclusive by default
• [16807] [DocDB] Enable automatic_tablet_splitting for new clusters using AutoFlags
• [16817] [DocDB] Add support to read and write to the underlying tables of a StatefulService
• [16849] [DocDB] Tablet splitting: Lower the thresholds for splits on low core machines
• [16875] [DST] PITR - Extend master's sys catalog retention based on snapshot schedule
• [16879] [DocDB] Extract encoding related part from docdb to dockv
• [16891] ccmd: a command-line tool utilizing the compilation database file
• [16892] [DocDB] AutoFlags should provide the capability to enable flags on Install scenarios only, but not upgrade
• [16911] [DocDB] Augment GetLockStatus tserver endpoint to handle transaction_id list
• [16991] [YSQL] Update pruning quals when nested loop batching is on
• [16993] [YSQL] Restructure type checking when forming batched expressions
• [17003] [DocDB] Use ReaderProjection by DocRowwiseIterator
• [17013] [DocDB] Add colocation info to ListTables output.
• [17023] [YSQL] Improve drop database in per-database catalog version mode
• [17028] [DocDB] Introduce qlexpr library
• [17029] [xCluster] Aggregate xcluster metrics on stream_id for 1:N support
• [17041] [YCQL] Add cache in FindMemberForIndex for jsonb
• [17050] [DocDB] Optimize zero encoded string encoding/decoding
• [17067] [YSQL] Disallow hash batching if there is only one batch element
• [17075] [17078] Send full universe key registry to Tablet Server on Startup
• [17090] [YSQL] Add yb_increment_all_db_catalog_versions for global ddl
• [17094] [DocDB] Introduce PgTableRow class
• [17104] [DocDB] Shuts downs superblock_flush_bg_task_ on tserver shutdown
• [17115] [YSQL] Restructure batched indexqual creation
• [17118] [CDCSDK] Remove tables with no active streams from namespace_to_cdcsdk_unprocessed_table_map_
• [17119] [DocDB] Create valid (non empty) IntentTypeSet for ancestor doc keys
• [17134] [YSQL] CREATE INDEX add timeout on wait for backends
• [17141] [YSQL] Allow using PgsqlConditionPB for pgsql remote filter
• [17150] [YSQL] Allow BNL on expressions with multiple relations on one side
• [17150] [YSQL] Disable batching on join clauses that involve more than one base relation on a side
• [17163] [YCQL] Use Metadata Cache by default in PREPARE
• [17165] [CDCSDK] Added serve_as_proxy member to GetCheckpointRequestPB
• [17178] [DocDB] increase max_len when printing Slice to 32 bytes
• [17181] [YSQL] Improve data parsing
• [17194] [CQL] Add memtracker to track CQL requests memory usage
• [17228] Reduce recompilation after Postgres headers are installed
• [17268] Consolidate type-checked Python-based build scripts into the python/yugabyte directory
• [17502] [DocDB] Exclude yugabyted-ui from sed binary manipulation
• [17565] [CDCSDK] Adding the column type for backward compatibility
• [16979] [DocDB] Mark UNKNOWN flags as NON_RUNTIME in tools and tests
• [DB-6011] [DB-5926] [DB-6134] YCQLSH: Updating yugabyte-db-thirdparty release version
• [yugabyted][UI] yugabyted UI updates (#16945)
• Improve handling for LDAP strings in ysql_hba_conf_csv
• Increase the character length for image bundle name
• Modify volume API only passes needed args

Bug fixes

• [7376] [YSQL] fix state waits for online CREATE INDEX
• [11307] [DocDB] Master UI Hides Deleted Tablets on the Table Page
• [13128] [YSQL] Fix ClearSeparateDdlTxnMode error handling
• [13786] [DocDB] Auto full compactions based on deletes in SST files
• [13862] [YSQL] Fix metric for total number of connections made
• [14284] [13041] [YSQL] Avoid setting read time when using serializable isolation level
• [14308] [xCluster] fix debug string info to include missing subtransaction info
• [14308] [xCluster] fix xCluster replication so it correctly handles SAVEPOINTs
• [14392] [YSQL] Fix ysql dependencies, Add a response size-based fetch limit
• [15090] [DocDB] Tablet splitting: Some queries fail with "The tablet is shutting down" during the workload
• [15166] [DocDB] Fix crashes due to tablet shutdown timeout
• [15769] [YSQL] Fix functional indexes returning wrong results
• [16075] Fix GCC 12 specific compilation errors
• [16126] [DocDB] Fix aborted transactions metric overcount issue
• [16230] [DocDB] TServer Heartbeat Sends Back Tablet Peer Full Compaction States
• [16329] [YSQL] Fix metric for rejections due to connections limit
• [16367] [DocDB] Delete Snapshot keeps on retrying indefinitely after master failover
• [16492] [YSQL] Make yb_table_properties return more up-to-date value
• [16513] [DocDB] Correcting statistics for docdb keys found
• [16551] [xCluster] Account for xCluster safe time for history retention on consumer
• [16612] [DocDB] Fix transaction promotion race condition resulting in unnecessary abort
• [16631] [DocDB] Fix delete_snapshot sending delete RPCs indefinitely for deleted tablets
• [16632] [DocDB] Fix TestIndex.testDropDuringWrite
• [16669] [DocDB] Fix tablet splitting / PITR deadlock
• [16680] [CDCSDK] Fix colocated before image test with packed row
• [16748] [DocDB] Fix colocation parent table's metrics update when a colocated table is altered
• [16773] [xCluster] Fix TSAN race on StreamTabletMetadata
• [16790] [CDCSDK] GetCheckpoint will return invalid checkpoint for unpolled tablets
• [16851] Fix DB UT for learner-TS due to broadcast address changes in yb_backup.py
• [16861] Remove unnecessary "using" declarations
• [16863] Ignore missing YSQL indices when importing a snapshot.
• [16868] [DocDB] Sanitizes debug messages
• [16900] [CDCSDK] Remove cdcsdk_unprocessed_tables from CDCStreamInfo
• [16902] [DocDB] Avoid decoding entry key when possible
• [16944] [16715] [DocDB] Fix 2 bugs with xcluster + packed
• [16953] [DocDB] Deleted tablets will not be displayed on Underreplicated Tablets page
• [16964] [DocDB] Compilation failure for Apple clang 14.0.3
• [16976] [YSQL] Fix postgres crash when pg_stat_progress_create_index has an index from a different database
• [16992] [YSQL] Fix scan upperbound calculation during batched IN query on hash columns
• [16994] Fixing the demo connect creating tables to default database.
• [16995] [CDCSDK] Removed duplicated access to CDC Stream table
• [17013] [DocDB] Fix DFATAL when ListTables called with a hidden tablegroup.
• [17026] [xCluster] Fix metrics jump when regaining leadership
• [17050] [DocDB] fix D24845: fix DecodeEncodedStr so it correctly handles all malformed input
• [17055] [refactor] remove some code related to YSQL syscache
• [17060] [YSQL] Describing a colocated Materialized View does not show colocation information
• [17080] [xCluster] Fix XClusterAdminCliTest.TestRenameUniverseReplication
• [17096] [DocDB] Avoid decoding key prefix while extracting key columns
• [17107] [YCQL] Prevent DROP TYPE for the used YCQL type
• [17147] [refactor] remove pg_yb_utils.h from execnodes.h
• [17184] [CDCSDK] Fix negative tests expecting failure on SetCDCCheckpoint
• [17203] Fix post_install.sh failure related to pg_stat_statements.so
• [17244] [17177] [DocDB] Fix test failures caused by master shutdown
• [17253] [DocDB] Fix RPC error handling in pg_client
• [17261] [DST] PITR - Single file compaction does not reset Hybrid Time filter
• [17323] yb-admin: revert bad change to list_tablets usage instruction
• [17395] [17494] [DocDB] Fix for LoadBalancerPlacementPolicyTest + FullCompactionManager memory cleanup
• [17480] [DocDB] Update to NULL could be lost after compaction when packed row is enabled
• [17502] Fix the shared library interpreter used for yugabyted-ui
• Fix a bug in Docker-based package sanity check
• Fix build on arm64 macOS: use system Python
• Fix shellcheck: add missing double quotes around $YB_TARGET_ARCH on the right side of a comparison
• Fix the issue [TIMEOUT] issue with Slf4jLogger
• Packaging: Fix missing source dir for new packaging sync command.
• Patched to put k8s operator behind configuration

Known issues

N/A