ContextQMD
Back to Opensearch

Opensearch.Release Notes 2.12.0

release-notes/opensearch.release-notes-2.12.0.md

3.6.024.6 KB
Original Source

2024-02-09 Version 2.12.0 Release Notes

[2.12.0]

Added

  • [Admission control] Add Resource usage collector service and resource usage tracker (#9890)
  • [Admission control] Add enhancements to FS stats to include read/write time, queue size and IO time (#10541)
  • [Remote cluster state] Change file names for remote cluster state (#10557)
  • [Search Pipelines] Add request-scoped state shared between processors (and three new processors) (#9405)
  • Per request phase latency (#10351)
  • [Remote Store] Add repository stats for remote store(#10567)
  • [Remote cluster state] Upload global metadata in cluster state to remote store(#10404)
  • [Remote cluster state] Download functionality of global metadata from remote store (#10535)
  • [Remote cluster state] Restore global metadata from remote store when local state is lost after quorum loss (#10404)
  • [Remote cluster state] Make index and global metadata upload timeout dynamic cluster settings (#10814)
  • Add search query categorizer (#10255)
  • Per request phase latency (#10351)
  • Add cluster state stats (#10670)
  • Remove ingest processor supports excluding fields (#10967, #11983)
  • [Tiered caching] Enabling serialization for IndicesRequestCache key object (#10275)
  • [Tiered caching] Defining interfaces, listeners and extending IndicesRequestCache with Tiered cache support (#10753)
  • [Remote cluster state] Restore cluster state version during remote state auto restore (#10853)
  • Update the indexRandom function to create more segments for concurrent search tests (10247)
  • Add support for query profiler with concurrent aggregation (#9248)
  • Introduce ConcurrentQueryProfiler to profile query using concurrent segment search path and support concurrency during rewrite and create weight (10352)
  • Implement on behalf of token passing for extensions (#8679)
  • Provide service accounts tokens to extensions (#9618)
  • [Streaming Indexing] Introduce new experimental server HTTP transport based on Netty 4 and Project Reactor (Reactor Netty) (#9672)
  • Enable must_exist parameter for update aliases API (#11210)
  • Add back half_float BKD based sort query optimization (#11024)
  • Request level coordinator slow logs (#10650)
  • Add template snippets support for field and target_field in KV ingest processor (#10040)
  • Allowing pipeline processors to access index mapping info by passing ingest service ref as part of the processor factory parameters (#10307)
  • Add experimental SIMD implementation of B-tree to round down dates (#11194)
  • Make number of segment metadata files in remote segment store configurable (#11329)
  • Allow changing number of replicas of searchable snapshot index (#11317)
  • Adding slf4j license header to LoggerMessageFormat.java (#11069)
  • [BWC and API enforcement] Introduce checks for enforcing the API restrictions (#11175)
  • Maintainer approval check (#11378)
  • Create separate transport action for render search template action (#11170)
  • Add additional handling in SearchTemplateRequest when simulate is set to true (#11591)
  • Introduce cluster level setting cluster.index.restrict.replication.type to prevent replication type setting override during index creations(#11583)
  • Add match_only_text field that is optimized for storage by trading off positional queries performance (#6836)
  • Add copy ingest processor (#11870)
  • Introduce new feature flag "WRITEABLE_REMOTE_INDEX" to gate the writeable remote index functionality (#11717)
  • Bump OpenTelemetry from 1.32.0 to 1.34.1 (#11891)
  • Add remove_by_pattern ingest processor (#11920)
  • Support index level allocation filtering for searchable snapshot index (#11563)
  • Add org.opensearch.rest.MethodHandlers and RestController#getAllHandlers (11876)
  • New DateTime format for RFC3339 compatible date fields (#11465)
  • Add community_id ingest processor (#12121)
  • Add support for Google Application Default Credentials in repository-gcs (#8394)
  • Remove concurrent segment search feature flag for GA launch (#12074)
  • Enable Fuzzy codec for doc id fields using a bloom filter (#11022)
  • [Metrics Framework] Adds support for Histogram metric (#12062)

Dependencies

  • Bumps jetty version to 9.4.52.v20230823 to fix GMS-2023-1857 (#9822)
  • Bump Lucene from 9.7.0 to 9.8.0 (10276)
  • Bump commons-io:commons-io from 2.13.0 to 2.15.1 (#10294, #11001, #11002, #11446, #11554, #11560, #11796)
  • Bump com.google.api.grpc:proto-google-common-protos from 2.10.0 to 2.25.1 (#10208, #10298)
  • Bump com.netflix.nebula.ospackage-base from 11.4.0 to 11.8.0 (#10295, #11630, #12167)
  • Bump org.apache.zookeeper:zookeeper from 3.9.0 to 3.9.1 (#10506)
  • Bump de.thetaphi:forbiddenapis from 3.5.1 to 3.6 (#10508)
  • Bump org.codehaus.woodstox:stax2-api from 4.2.1 to 4.2.2 (#10639)
  • Bump org.bouncycastle:bc-fips from 1.0.2.3 to 1.0.2.4 (#10297)
  • Bump com.google.http-client:google-http-client from 1.43.2 to 1.43.3 (#10635)
  • Bump com.squareup.okio:okio from 3.5.0 to 3.7.0 (#10637, #11632)
  • Bump org.apache.logging.log4j:log4j-core from 2.20.0 to 2.22.1 (#10858, #11000, #11270, #11695)
  • Bump aws-actions/configure-aws-credentials from 2 to 4 (#10504)
  • Bump stefanzweifel/git-auto-commit-action from 4 to 5 (#11171)
  • Bump actions/github-script from 6 to 7.0.1 (#11271, #12166)
  • Bump jackson and jackson_databind from 2.15.2 to 2.16.0 (#11273)
  • Bump netty from 4.1.100.Final to 4.1.106.Final (#11294, #11775), #12034)
  • Bump com.avast.gradle:gradle-docker-compose-plugin from 0.16.12 to 0.17.6 (#10163, #11692)
  • Bump com.squareup.okhttp3:okhttp from 4.11.0 to 4.12.0 (#10861)
  • Bump org.apache.commons:commons-text from 1.10.0 to 1.11.0 (#11344)
  • Bump reactor-netty-core from 1.1.12 to 1.1.15 (#11350), (#12042)
  • Bump com.gradle.enterprise from 3.14.1 to 3.16.2 (#11339, #11629, #12056)
  • Bump actions/setup-java from 3 to 4 (#11447)
  • Bump commons-net:commons-net from 3.9.0 to 3.10.0 (#11450)
  • Bump org.apache.maven:maven-model from 3.9.4 to 3.9.6 (#11445)
  • Bump org.apache.xmlbeans:xmlbeans from 5.1.1 to 5.2.0 (#11448)
  • Bump logback-core and logback-classic to 1.2.13 (#11521)
  • Bumps jetty version from 9.4.52.v20230823 to 9.4.53.v20231009 (#11539)
  • Bump org.wiremock:wiremock-standalone from 3.1.0 to 3.3.1 (#11555)
  • Bump org.apache.commons:commons-compress from 1.24.0 to 1.25.0 (#11556)
  • Bump actions/stale from 8 to 9 (#11557)
  • Bump com.netflix.nebula:nebula-publishing-plugin from 20.3.0 to 21.0.0 (#11671)
  • Bump commons-cli:commons-cli from 1.5.0 to 1.6.0 (#10996)
  • Bump com.maxmind.geoip2:geoip2 from 4.1.0 to 4.2.0 (#11559)
  • Bump org.apache.commons:commons-lang3 from 3.13.0 to 3.14.0 (#11691)
  • Bump com.maxmind.db:maxmind-db from 3.0.0 to 3.1.0 (#11693)
  • Bump net.java.dev.jna:jna from 5.13.0 to 5.14.0 (#11798)
  • Bump lycheeverse/lychee-action from 1.8.0 to 1.9.1 (#11795, #11887)
  • Bump Lucene from 9.8.0 to 9.9.2 (#11421), (#12063)
  • Bump com.networknt:json-schema-validator from 1.0.86 to 1.2.0 (#11886, #11963)
  • Bump com.google.api:gax-httpjson from 0.103.1 to 2.42.0 (#11794, #12165)
  • Bump com.google.oauth-client:google-oauth-client from 1.34.1 to 1.35.0 (#11960)
  • Bump com.diffplug.spotless from 6.23.2 to 6.25.0 (#11962, #12055)
  • Bump com.google.cloud:google-cloud-core from 2.5.10 to 2.30.0 (#11961)
  • Bump reactor-core from 3.5.11 to 3.5.14 (#12042)
  • Bump org.apache.shiro:shiro-core from 1.11.0 to 1.13.0 (#12200)
  • Bump com.google.http-client:google-http-client-jackson2 from 1.43.3 to 1.44.1 (#12059)
  • Bump peter-evans/create-issue-from-file from 4 to 5 (#12057)
  • Bump org.gradle.test-retry from 1.5.4 to 1.5.8 (#12168)
  • Bump org.apache.kerby:kerb-admin from 1.0.1 to 2.0.3 (#12194)

Changed

  • Mute the query profile IT with concurrent execution (#9840)
  • Force merge with only_expunge_deletes honors max segment size (#10036)
  • Add the means to extract the contextual properties from HttpChannel, TcpCChannel and TrasportChannel without excessive typecasting (#10562), (#11751)
  • Introduce new dynamic cluster setting to control slice computation for concurrent segment search (#9107)
  • Search pipelines now support asynchronous request and response processors to avoid blocking on a transport thread (#10598)
  • [Remote Store] Add Remote Store backpressure rejection stats to _nodes/stats (#10524)
  • [BUG] Fix java.lang.SecurityException in repository-gcs plugin (#10642)
  • Add telemetry tracer/metric enable flag and integ test. (#10395)
  • Performance improvement for Datetime field caching (#4558)
  • Add instrumentation for indexing in transport bulk action and transport shard bulk action. (#10273)
  • Disallow removing some metadata fields by remove ingest processor (#10895, #11607)
  • Performance improvement for MultiTerm Queries on Keyword fields (#7057)
  • Refactor common parts from the Rounding class into a separate 'round' package (#11023)
  • Performance improvement for date histogram aggregations without sub-aggregations (#11083)
  • Apply the fast filter optimization to composite aggregation of date histogram source (#11505)
  • Disable concurrent aggs for Diversified Sampler and Sampler aggs (#11087)
  • Made leader/follower check timeout setting dynamic (#10528)
  • Improved performance of numeric exact-match queries (#11209)
  • Change error message when per shard document limit is breached (#11312)
  • Improve boolean parsing performance (#11308)
  • Interpret byte array as primitive using VarHandles (#11362)
  • Automatically add scheme to discovery.ec2.endpoint (#11512)
  • Restore support for Java 8 for RestClient (#11562)
  • Add deleted doc count in _cat/shards (#11678)
  • Capture information for additional query types and aggregation types (#11582)
  • Use slice_size == shard_size heuristic in terms aggs for concurrent segment search and properly calculate the doc_count_error (#11732)
  • Added Support for dynamically adding SearchRequestOperationsListeners with SearchRequestOperationsCompositeListenerFactory (#11526)
  • Ensure Jackson default maximums introduced in 2.16.0 do not conflict with OpenSearch settings (#11890)
  • Extract cluster management for integration tests into JUnit test rule out of OpenSearchIntegTestCase (#11877), (#12000)
  • Workaround for https://bugs.openjdk.org/browse/JDK-8323659 regression, introduced in JDK-21.0.2 (#11968)
  • Updates IpField to be searchable when only doc_values are enabled (#11508)
  • [Query Insights] Query Insights Framework which currently supports retrieving the most time-consuming queries within the last configured time window (#11903)
  • [Query Insights] Implement Top N Queries feature to collect and gather information about high latency queries in a window (#11904)
  • Add override support for sampling based on action (#9621)
  • Added custom sampler support based on transport action in request (#9621)
  • Disable concurrent search for composite aggregation(#12375)

Removed

  • Remove deprecated classes for Rounding (#10956)

Fixed

  • Fix failure in dissect ingest processor parsing empty brackets (#9225)
  • Fix class_cast_exception when passing int to _version and other metadata fields in ingest simulate API (#10101)
  • Fix Segment Replication ShardLockObtainFailedException bug during index corruption (10370)
  • Fix some test methods in SimulatePipelineRequestParsingTests never run and fix test failure (#10496)
  • Fix passing wrong parameter when calling newConfigurationException() in DotExpanderProcessor (#10737)
  • Delegating CachingWeightWrapper#count to internal weight object (#10543)
  • Fix per request latency last phase not tracked (#10934)
  • Fix SuggestSearch.testSkipDuplicates by forcing refresh when indexing its test documents (#11068)
  • [BUG] Fix the thread context that is not properly cleared and messes up the traces (#10873)
  • Handle canMatchSearchAfter for frozen context scenario (#11249)
  • Fix the issue with DefaultSpanScope restoring wrong span in the TracerContextStorage upon detach (#11316)
  • Remove shadowJar from lang-painless module publication (#11369)
  • Fix remote shards balancer and remove unused variables (#11167)
  • Fix parsing of flat object fields with dots in keys (#11425)
  • Fix bug where replication lag grows post primary relocation (#11238)
  • Fix noop_update_total metric in indexing stats cannot be updated by bulk API (#11485,#11917)
  • Fix for stuck update action in a bulk with retry_on_conflict property (#11152)
  • Fix template setting override for replication type (#11417)
  • Fix Automatic addition of protocol broken in #11512 (#11609)
  • Fix issue when calling Delete PIT endpoint and no PITs exist (#11711)
  • Fix tracing context propagation for local transport instrumentation (#11490)
  • Fix parsing of single line comments in lang-painless (#11815)
  • Fix memory leak issue in ReorganizingLongHash (#11953)
  • Prevent setting remote_snapshot store type on index creation (#11867)
  • [BUG] Fix remote shards balancer when filtering throttled nodes (#11724)
  • Add advance(int) for numeric values in order to allow point based optimization to kick in (#12089)