ContextQMD
Back to Opensearch

Opensearch.Release Notes 3.0.0 Beta1

release-notes/opensearch.release-notes-3.0.0-beta1.md

3.6.026.1 KB
Original Source

2025-04-15 Version 3.0.0-beta1 Release Notes

[3.0.0]

Breaking Changes

  • Upgrade to Lucene 10.1.0 - PR (#16366)
  • JDK21 as minimum supported Java runtime (#10745)
  • Remove deprecated terms from Java API (#5214)
  • JPMS Support (Eliminate top level split packages ) Phase-0 only (#8110)
  • Add ThreadContextPermission for stashAndMergeHeaders and stashWithOrigin (#15039)
  • Add ThreadContextPermission for markAsSystemContext and allow core to perform the method (#15016)
  • Migrate client transports to Apache HttpClient / Core 5.x (#4459)
  • Validation changes on the Bulk Index API like enforcing 512 byte _id size limit (#6595)
  • Ability to use the node as coordinating node by passing node.roles as empty array (#3412)
  • Treat Setting value with empty array string as empty array (#10625)
  • Ensure Jackson default maximums introduced in 2.16.0 do not conflict with OpenSearch settings (#11811)
  • Setting a maximum depth for nested queries (#11670)
  • Fix interchanged formats of total_indexing_buffer_in_bytes and total_indexing_buffer (#17070)
  • Cleanup deprecated thread pool settings (#2595)
  • Replace "blacklist/whitelist" terminology in Java APIs (#1683)
  • Remove deprecated methods from JodaCompatibleZonedDateTime which are called by scripts (#3346)
  • List of deprecated code removal in 3.0- partially done (#2773)
  • Remove mmap.extensions setting (#9392)
  • Remove COMPAT locale provider (#13988)
  • Remove transport-nio plugin (#16887)
  • Deprecate CamelCase PathHierarchy tokenizer name (#10894)
  • Rename Class ending with Plugin to Module under modules dir (#4042)
  • Remove deprecated batch_size parameter from _bulk (#14283)

Added

  • Support for HTTP/2 (server-side) (#3847)
  • Allow mmap to use new JDK-19 preview APIs in Apache Lucene 9.4+ (#5151)
  • Add events correlation engine plugin (#6854)
  • Implement on behalf of token passing for extensions (#8679, #10664)
  • Provide service accounts tokens to extensions (#9618)
  • GHA to verify checklist items completion in PR descriptions (#10800)
  • [WLM] Add WLM support for search scroll API (#16981)
  • Views, simplify data access and manipulation by providing a virtual layer over one or more indices (#11957)
  • Add systemd configurations to strengthen OS core security (#17107)
  • Added pull-based Ingestion (APIs, for ingestion source, a Kafka plugin, and IngestionEngine that pulls data from the ingestion source) (#16958)
  • Added ConfigurationUtils to core for the ease of configuration parsing #17223
  • Add cluster and index level settings to limit the total primary shards per node and per index #17295
  • Add execution_hint to cardinality aggregator request (#17312)
  • Arrow Flight RPC plugin with Flight server bootstrap logic and client for internode communication (#16962)
  • Added offset management for the pull-based Ingestion (#17354)
  • Added integ tests for systemd configs (#17410)
  • Add filter function for AbstractQueryBuilder, BoolQueryBuilder, ConstantScoreQueryBuilder(#17409)
  • [Star Tree] [Search] Resolving keyword & numeric bucket aggregation with metric aggregation using star-tree (#17165)
  • Added error handling support for the pull-based ingestion (#17427)
  • Added Warm index setting and Updated nomenclature to differentiate between hot and warm tiering implementation (#17490)
  • Latency and Memory allocation improvements to Multi Term Aggregation queries (#14993)
  • Add logic in master service to optimize performance and retain detailed logging for critical cluster operations. (#14795)
  • Add Setting to adjust the primary constraint weights (#16471)
  • Introduce a setting to disable download of full cluster state from remote on term mismatch(#16798)
  • Added ability to retrieve value from DocValues in a flat_object filed(#16802)
  • Improve performace of NumericTermAggregation by avoiding unnecessary sorting(#17252)
  • Fix Bug - Handle unsigned long in sorting order assertion of LongHashSet (#17207)
  • Implemented computation of segment replication stats at shard level (#17055)
  • [Rule Based Auto-tagging] Add in-memory attribute value store (#17342)
  • Change priority for scheduling reroute during timeout(#16445)
  • [Rule Based Auto-tagging] Add rule schema for auto tagging (#17238)
  • Renaming the node role search to warm (#17573)
  • Introduce a new search node role to hold search only shards (#17620)
  • Fix systemd integTest on deb regarding path ownership check (#17641)
  • Add dfs transformation function in XContentMapValues (#17612)
  • Add tracking for long-running SearchTask post cancellation (#17726)
  • Added Kinesis support as a plugin for the pull-based ingestion (#17615)
  • Add FilterFieldType for developers who want to wrap MappedFieldType (#17627)
  • [Rule Based Auto-tagging] Add in-memory rule processing service (#17365)
  • [Security Manager Replacement] Create initial Java Agent to intercept Socket::connect calls (#17724)
  • Faster terms_query with already sorted terms (#17714)
  • Add ingestion management APIs for pause, resume and get ingestion state (#17631)
  • [Security Manager Replacement] Enhance Java Agent to intercept System::exit (#17746)
  • [Security Manager Replacement] Add a policy parser for Java agent security policies (#17753)
  • [Security Manager Replacement] Implement File Interceptor and add integration tests (#17760)
  • [Security Manager Replacement] Enhance Java Agent to intercept Runtime::halt (#17757)
  • [Security Manager Replacement] Phase off SecurityManager usage in favor of Java Agent (#17861)
  • Support AutoExpand for SearchReplica (#17741)
  • Add TLS enabled SecureNetty4GrpcServerTransport (#17796)
  • Implement fixed interval refresh task scheduling (#17777)
  • [Tiered caching] Create a single cache manager for all the disk caches. (#17513)
  • Add GRPC DocumentService and Bulk endpoint (#17727)
  • Added scale to zero (search_only mode) support for OpenSearch reader writer separation (#17299
  • [Star Tree] [Search] Resolving numeric range aggregation with metric aggregation using star-tree (#17273)
  • Added Search Only strict routing setting (#17803)
  • Disable the index API for ingestion engine (#17768)
  • Add SearchService and Search GRPC endpoint (#17830)
  • Add update and delete support in pull-based ingestion (#17822)
  • Allow maxPollSize and pollTimeout in IngestionSource to be configurable (#17863)
  • [Star Tree] [Search] Add query changes to support unsigned-long in star tree (#17275)
  • Add ApproximateMatchAllQuery that targets match_all queries and approximates sorts (#17772)
  • Add TermsQuery support to Search GRPC endpoint (#17888)
  • Support sub agg in filter rewrite optimization (#17447
  • Disable scoring of keyword term search by default, fallback logic with new use_similarity:true parameter (#17889)
  • Add versioning support in pull-based ingestion (#17918)
  • Introducing MergedSegmentWarmerFactory to support the extension of IndexWriter.IndexReaderWarmer (#17881)
  • Introduce mapping transformer to allow transform mappings during index create/update or index template create/update (#17635)
  • Introduce new dynamic cluster setting to tune maxMergeAtOnce at cluster level (#17774)

Dependencies

  • Bump Apache HttpCore5/HttpClient5 dependencies from 5.2.5/5.3.1 to 5.3.1/5.4.1 to support ExtendedSocketOption in HttpAsyncClient (#16757)
  • Bumps jetty version from 9.4.55.v20240627 to 9.4.57.v20241219 (#17395)
  • Switch main/3.x to use JDK21 LTS version (#17515)
  • Bump org.awaitility:awaitility from 4.2.0 to 4.3.0 (#17230, #17439)
  • Bump dnsjava:dnsjava from 3.6.2 to 3.6.3 (#17231)
  • Bump com.google.code.gson:gson from 2.11.0 to 2.13.0 (#17229, #17668, #17921), #17926)
  • Bump org.jruby.joni:joni from 2.2.1 to 2.2.6 (#17136, #17608, #17732)
  • Bump org.apache.ant:ant from 1.10.14 to 1.10.15 (#17288)
  • Bump reactor_netty from 1.1.26 to 1.2.3 (#17322, #17322, #17377)
  • Bump me.champeau.gradle.japicmp from 0.4.5 to 0.4.6 (#17375)
  • Bump com.google.api.grpc:proto-google-common-protos from 2.37.1 to 2.54.1 (#17379, #17733)
  • Bump net.minidev:json-smart from 2.5.1 to 2.5.2 (#17378)
  • Bump com.netflix.nebula.ospackage-base from 11.10.1 to 11.11.2 (#17374, #17734)
  • Bump ch.qos.logback:logback-classic from 1.5.16 to 1.5.18 (#17497, #17730)
  • Bump software.amazon.awssdk from 2.20.86 to 2.30.31 (#17396)
  • Bump org.jruby.jcodings:jcodings from 1.0.61 to 1.0.63 (#17560)
  • Bump com.azure:azure-storage-blob from 12.28.1 to 12.30.0 (#17562, #17667)
  • Bump com.nimbusds:nimbus-jose-jwt from 9.41.1 to 10.0.2 (#17607, #17669)
  • Bump com.google.api:api-common from 1.8.1 to 2.46.1 (#17604)
  • Bump ch.qos.logback:logback-core from 1.5.16 to 1.5.18 (#17609, #17809)
  • Bump dangoslen/dependabot-changelog-helper from 3 to 4 (#17498)
  • Bump com.google.api:gax from 2.35.0 to 2.63.1 (#17465)
  • Bump tj-actions/changed-files from 46.0.1 to 46.0.5 (#17666, #17813, #17920)
  • Bump com.github.luben:zstd-jni from 1.5.5-1 to 1.5.6-1 (#17674)
  • Bump lycheeverse/lychee-action from 2.3.0 to 2.4.0 (#17731)
  • Bump com.nimbusds:oauth2-oidc-sdk from 11.21 to 11.23.1 (#17729)
  • Bump com.google.api.grpc:proto-google-iam-v1 from 1.33.0 to 1.49.1 (#17811)
  • Bump com.azure:azure-core from 1.54.1 to 1.55.3 (#17810)
  • Bump org.apache.poi version from 5.2.5 to 5.4.1 in /plugins/ingest-attachment (#17887)
  • Bump org.opensearch:protobufs from 0.2.0 to 0.3.0 (#17888)

Changed

  • Changed locale provider from COMPAT to CLDR (#14345)
  • Change http code on create index API with bad input raising NotXContentException from 500 to 400 (#4773)
  • Improve summary error message for invalid setting updates (#4792)
  • Return 409 Conflict HTTP status instead of 503 on failure to concurrently execute snapshots (#8986)
  • Add task completion count in search backpressure stats API (#10028)
  • Breaking change: Do not request "search_pipelines" metrics by default in NodesInfoRequest (#12497)
  • Use simpler matching logic for source fields when explicit field names (no wildcards or dot-paths) are specified (#17160)
  • Refactor :libs module bootstrap package to eliminate top level split packages for JPMS support (#17117)
  • Refactor the codebase to eliminate top level split packages for JPMS support (#17153
  • Refactor :server module org.apacge.lucene package to eliminate top level split packages for JPMS support (#17241)
  • Stop minimizing automata used for case-insensitive matches (#17268)
  • Refactor the :server module org.opensearch.client to org.opensearch.transport.client to eliminate top level split packages for JPMS support (#17272)
  • Use Lucene BM25Similarity as default since the LegacyBM25Similarity is marked as deprecated (#17306)
  • Wildcard field index only 3gram of the input data #17349
  • Use BC libraries to parse PEM files, increase key length, allow general use of known cryptographic binary extensions, remove unused BC dependencies (#17393)
  • Add optional enum set read / write functionality to stream input / output (#17556)
  • Convert transport-reactor-netty4 to use gradle version catalog #17233
  • Increase force merge threads to 1/8th of cores #17255
  • TieredSpilloverCache took-time threshold now guards heap tier as well as disk tier #17190
  • Migrate BC libs to their FIPS counterparts (#17507)
  • Increase the floor segment size to 16MB (#17699)
  • Unwrap singleton DocValues in global ordinal value source of composite histogram aggregation (#17740)
  • Unwrap singleton DocValues in date histogram aggregation. (#17643)
  • Introduce 512 byte limit to search and ingest pipeline IDs (#17786)
  • Avoid skewed segment replication lag metric (#17831)
  • Increase the default segment counter step size when replica promoting (#17568)
  • [WLM] Rename QueryGroup to WorkloadGroup (#17901)
  • Relaxes jarHell check for optionally extended plugins(#17893))
  • Add cluster setting for retry timeout of publish checkpoint tx action (#17749)
  • Increase the default maxMergesAtOnce to 30 (#17774)

Deprecated

Removed

  • Remove deprecated code to add node name into log pattern of log4j property file (#4568)
  • Unused object and import within TransportClusterAllocationExplainAction (#4639)
  • Remove LegacyESVersion.V_7_0_* and V_7_1_* Constants (#2768)
  • Remove LegacyESVersion.V_7_2_ and V_7_3_ Constants (#4702)
  • Always auto release the flood stage block (#4703)
  • Remove LegacyESVersion.V_7_4_ and V_7_5_ Constants (#4704)
  • Remove Legacy Version support from Snapshot/Restore Service (#4728)
  • Remove deprecated serialization logic from pipeline aggs (#4847)
  • Remove unused private methods (#4926)
  • Remove LegacyESVersion.V_7_8_ and V_7_9_ Constants (#4855)
  • Remove LegacyESVersion.V_7_6_ and V_7_7_ Constants (#4837)
  • Remove LegacyESVersion.V_7_10_ Constants (#5018)
  • Remove Version.V_1_ Constants (#5021)
  • Remove custom Map, List and Set collection classes (#6871)
  • Remove package org.opensearch.action.support.master (#4856)
  • Remove deprecated 'gateway' settings used to defer cluster recovery (#3117)
  • Remove FeatureFlags.PLUGGABLE_CACHE as the feature is no longer experimental (#17344)
  • Remove FeatureFlags.APPROXIMATE_POINT_RANGE_QUERY_SETTING since range query approximation is no longer experimental (#17769)

Fixed

  • Fix 'org.apache.hc.core5.http.ParseException: Invalid protocol version' under JDK 16+ (#4827)
  • Fix compression support for h2c protocol (#4944)
  • Don't over-allocate in HeapBufferedAsyncEntityConsumer in order to consume the response (#9993)
  • Add HTTP/2 protocol support to HttpRequest.HttpVersion (#17248)
  • Fix missing bucket in terms aggregation with missing value (#17418)
  • Fix explain action on query rewrite (#17286)
  • Fix visit of inner query for FunctionScoreQueryBuilder (#16776)
  • Fix case insensitive and escaped query on wildcard (#16827)
  • Fix exists queries on nested flat_object fields throws exception (#16803)
  • Add highlighting for wildcard search on match_only_text field (#17101)
  • Fix illegal argument exception when creating a PIT (#16781)
  • Fix NPE in node stats due to QueryGroupTasks (#17576)
  • Fix bytes parameter on _cat/recovery (#17598)
  • Fix slow performance of FeatureFlag checks (#17611)
  • Fix shard recovery in pull-based ingestion to avoid skipping messages (#17868))