Opensearch.Release Notes 2.16.0

2024-07-24 Version 2.16.0 Release Notes

[2.16.0]

Added

• Add fingerprint ingest processor (#13724)
• [Remote Store] Rate limiter for remote store low priority uploads (#14374)
• Apply the date histogram rewrite optimization to range aggregation (#13865)
• [Writable Warm] Add composite directory implementation and integrate it with FileCache (12782)
• [Workload Management] Add QueryGroup schema (13669)
• Add batching supported processor base type AbstractBatchingProcessor (#14554)
• Fix race condition while parsing derived fields from search definition (14445)
• Add strict_allow_templates dynamic mapping option (#14555)
• Add allowlist setting for ingest-common and search-pipeline-common processors (#14439)
• [Workload Management] add queryGroupId header propagator across requests and nodes (#14614)
• Create SystemIndexRegistry with helper method matchesSystemIndex (#14415)
• Print reason why parent task was cancelled (#14604)
• Add matchesPluginSystemIndexPattern to SystemIndexRegistry (#14750)
• Add Plugin interface for loading application based configuration templates ((#14659))
• Refactor remote-routing-table service inline with remote state interfaces(#14668)
• Add shard-diff path to diff manifest to reduce number of read calls remote store ((#14684))
• Add SortResponseProcessor to Search Pipelines ((#14785))
• Add prefix mode verification setting for repository verification ((#14790))
• Add SplitResponseProcessor to Search Pipelines ((#14800))
• Optimize TransportNodesAction to not send DiscoveryNodes for NodeStats, NodesInfo and ClusterStats call (14749)
• Reduce logging in DEBUG for MasterService:run (#14795)
• Refactor remote-routing-table service inline with remote state interfaces(#14668)
• Add rest, transport layer changes for hot to warm tiering - dedicated setup ((#13980)
• Enabling term version check on local state for all ClusterManager Read Transport Actions (#14273)
• Optimize Cluster Stats Indices to precomute node level stats (#14426)
• Create listener to refresh search thread resource usage (#14832)
• Add logic to create index templates (v2) using context field (#14811)

Dependencies

• Update to Apache Lucene 9.11.1 (#14042, #14576)
• Bump netty from 4.1.110.Final to 4.1.111.Final (#14356)
• Bump org.wiremock:wiremock-standalone from 3.3.1 to 3.6.0 (#14361)
• Bump reactor from 3.5.17 to 3.5.19 (#14395, #14697)
• Bump reactor-netty from 1.1.19 to 1.1.21 (#14395, #14697)
• Bump commons-net:commons-net from 3.10.0 to 3.11.1 (#14396)
• Bump com.nimbusds:nimbus-jose-jwt from 9.37.3 to 9.40 (#14398)
• Bump org.apache.commons:commons-configuration2 from 2.10.1 to 2.11.0 (#14399)
• Bump com.gradle.develocity from 3.17.4 to 3.17.5 (#14397)
• Bump opentelemetry from 1.36.0 to 1.40.0 (#14457, #14674)
• Bump opentelemetry-semconv from 1.25.0-alpha to 1.26.0-alpha (#14674)
• Bump azure-identity from 1.11.4 to 1.13.0, Bump msal4j from 1.14.3 to 1.15.1, Bump msal4j-persistence-extension from 1.2.0 to 1.3.0 (#14506)
• Bump com.azure:azure-storage-common from 12.21.2 to 12.25.1 (#14517)
• Bump com.microsoft.azure:msal4j from 1.15.1 to 1.16.0 (#14610)
• Bump com.github.spullara.mustache.java:compiler from 0.9.13 to 0.9.14 (#14672)
• Bump net.minidev:accessors-smart from 2.5.0 to 2.5.1 (#14673)
• Bump jackson from 2.17.1 to 2.17.2 (#14687)
• Bump net.minidev:json-smart from 2.5.0 to 2.5.1 (#14748)

Changed

• [Tiered Caching] Move query recomputation logic outside write lock (#14187)
• unsignedLongRangeQuery now returns MatchNoDocsQuery if the lower bounds are greater than the upper bounds (#14416)
• Make the class CommunityIdProcessor final (#14448)
• Updated the indices.query.bool.max_clause_count setting from being static to dynamically updateable (#13568)
• Allow @InternalApi annotation on classes not meant to be constructed outside of the OpenSearch core (#14575)
• Add @InternalApi annotation to japicmp exclusions (#14597)
• Allow system index warning in OpenSearchRestTestCase.refreshAllIndices (#14635)
• Make reroute iteration time-bound for large shard allocations (#14848)

Deprecated

• Deprecate batch_size parameter on bulk API (#14725)

Removed

• Remove query categorization changes (#14759)

Fixed

• Fix bug in SBP cancellation logic (#13259)
• Fix handling of Short and Byte data types in ScriptProcessor ingest pipeline (#14379)
• Switch to iterative version of WKT format parser (#14086)
• Fix match_phrase_prefix_query not working on text field with multiple values and index_prefixes (#10959)
• Fix the computed max shards of cluster to avoid int overflow (#14155)
• Fixed rest-high-level client searchTemplate & mtermVectors endpoints to have a leading slash (#14465)
• Write shard level metadata blob when snapshotting searchable snapshot indexes (#13190)
• Fix aggs result of NestedAggregator with sub NestedAggregator (#13324)
• Fix fs info reporting negative available size (#11573)
• Add ListPitInfo::getKeepAlive() getter (#14495)
• Fix FuzzyQuery in keyword field will use IndexOrDocValuesQuery when both of index and doc_value are true (#14378)
• Fix file cache initialization (#14004)
• Handle NPE in GetResult if "found" field is missing (#14552)
• Fix create or update alias API doesn't throw exception for unsupported parameters (#14719)
• Refactoring FilterPath.parse by using an iterative approach (#14200)
• Refactoring Grok.validatePatternBank by using an iterative approach (#14206)
• Fix NPE when creating index with index.number_of_replicas set to null (#14812)
• Update help output for _cat (#14722)
• Fix bulk upsert ignores the default_pipeline and final_pipeline when auto-created index matches the index template (#12891)
• Fix NPE in ReplicaShardAllocator (#14385)
• Use circuit breaker in InternalHistogram when adding empty buckets (#14754)
• Create new IndexInput for multi part upload (#14888)
• Fix searchable snapshot failure with scripted fields (#14411)