Opensearch.Release Notes 2.4.0

2022-11-04 Version 2.4.0 Release Notes

[2.4]

Added

• Introduce point in time search feature (#3959)
• Introduce experimental searchable snapshot feature (#2919)
• Add API for decommissioning/recommissioning zone and weighted zonal search request routing policy (#3639)
• Add support for s390x architecture (#4001)
• Github workflow for changelog verification (#4085)
• Add failover support with Segment Replication enabled. (#4325
• Add a new node role 'search' which is dedicated to provide search capability (#4689)
• Introduce Remote translog feature flag(#4158)
• Add groupId value propagation tests for ZIP publication task (#4848)
• Add support for GeoJson Point type in GeoPoint field (#4597)
• Add missing no-jdk distributions (#4722)
• Copy build.sh over from opensearch-build (#4887)
• Update GeoGrid base class access modifier to support extensibility (#4921)
• Build no-jdk distributions as part of release build (#4902)
• Add in-flight cancellation of SearchShardTask based on resource consumption (#4565)
• Add resource usage trackers for in-flight cancellation of SearchShardTask (#4805)
• Add search backpressure stats API (#4932)
• Add feature to ignore indexes starting with dot during shard limit validation.(#4695)

Dependencies

• Bump com.diffplug.spotless from 6.9.1 to 6.10.0
• Bump xmlbeans from 5.1.0 to 5.1.1
• Bump hadoop-hdfs from 3.3.3 to 3.3.4
• Exclude jettison version brought in with hadoop-minicluster. (#4787)
• Bump protobuf-java to 3.21.7 in repository-gcs and repository-hdfs (#4790)
• Bump reactor-netty-http to 1.0.24 in repository-azure (#4880)
• Add dev help in gradle check CI failures (4872)
• Upgrade netty to 4.1.84.Final (#4893)
• Dependency updates: asm 9.3 -> 9.4, bytebuddy 1.12.12 -> 1.12.18 (#4889)
• Bump tika from 2.4.0 to 2.5.0 (#4791)
• Update Apache Lucene to 9.4.1 (#4922)
• Bump woodstox-core to 6.4.0 (#4951)
• Upgrade jetty-http, kotlin-stdlib and snakeyaml (#4982)
• OpenJDK Update (October 2022 Patch releases) (#4997)
• Upgrade zookeeper dependency in hdfs-fixture (#5007)
• Dependency updates (httpcore, mockito, slf4j, httpasyncclient, commons-codec) (#4308)
• Update to Apache Lucene 9.4.0 (#4661)
• Updated jackson to 2.13.4 and snakeyml to 1.32 (#4556)
• Update Jackson Databind to 2.13.4.2 (addressing CVE-2022-42003) (#4781)
• Bump protobuf-java to 3.21.8 (#5005)
• Upgrade zookeeper dependency in hdfs-fixture (#5047)

Changed

• Use RemoteSegmentStoreDirectory instead of RemoteDirectory (#4240)
• Weighted round-robin scheduling policy for shard coordination traffic (#4241)
• Add index specific setting for remote repository (#4253)
• Update replicas to commit SegmentInfos instead of relying on SIS files from primary shards. (#4402)
• Change the version to remove deprecated code of adding node name into log pattern of log4j property file (#4569)
• Load the deprecated master role in a dedicated method instead of in setAdditionalRoles() (#4582)
• Plugin ZIP publication groupId value is configurable (#4156)
• Further simplification of the ZIP publication implementation (#4360)
• Change behaviour in replica recovery for remote translog enabled indices (#4318)
• PUT api for weighted shard routing (#4272)
• GET api for weighted shard routing(#4275)
• Delete api for weighted shard routing(#4400)
• Fix weighted routing metadata deserialization error on process restart (#4691)
• Install and configure Log4j JUL Adapter for Lucene 9.4 (#4754)
• Use getParameterCount instead of getParameterTypes (#4821)

Removed

• Remove RepositoryData.MIN_VERSION support for next major release (4729)

Fixed

• opensearch-service.bat start and opensearch-service.bat manager failing to run (#4289)
• PR reference to checkout code for changelog verifier (#4296)
• opensearch.bat and opensearch-service.bat install failing to run, missing logs directory (#4305)
• Restore using the class ClusterInfoRequest and ClusterInfoRequestBuilder from package 'org.opensearch.action.support.master.info' for subclasses (#4307)
• Do not fail replica shard due to primary closure (#4133)
• Fixed cancellation of segment replication events (#4225)
• opensearch.bat fails to execute when install path includes spaces (#4362)
• Getting security exception due to access denied 'java.lang.RuntimePermission' 'accessDeclaredMembers' when trying to get snapshot with S3 IRSA (#4469)
• Fix the ignore_malformed setting to also ignore objects (#4494)
• Fix day of year defaulting for round up parser (#4627)
• Fix the SnapshotsInProgress error during index deletion (#4570)
• Fix invalid location of JDK dependency for arm64 architecture(#4613)
• Alias filter lost after rollover (#4499)
• Fix Gradle warnings associated with publishPluginZipPublicationToXxx tasks (#4696)
• Fix a bug on handling an invalid array value for point type field #4900(#4900)
• Set analyzer to regex query string search (4219)
• Better plural stemmer than minimal_english (#4738)
• Add check to cancel ongoing replication with old primary on onNewCheckpoint on replica (#4363)
• Bump segment infos counter before commit during replica promotion (#4365)
• Update flaky testOnNewCheckpointFromNewPrimaryCancelOngoingReplication unit test (#4414)
• Extend FileChunkWriter to allow cancel on transport client (#4386)
• Fix NoSuchFileExceptions with segment replication when computing primary metadata snapshots (#4366)
• Fix timeout issue by calculating time needed to process getSegmentFiles (#4434)
• Update replicas to commit SegmentInfos instead of relying on segments_N from primary shards (#4450)
• Adding check to make sure checkpoint is not processed when a shard's shard routing is primary (#4716)
• Fix bug of replica shard's translog not purging on index flush when segment replication is enabled (4975)
• Fix message "No OpenSearchException found" when detailed_error disabled by return meaningful messages (#4708)
• Add fix for auto expand replica validation (#4994)
• Fix build failures on the Windows platform (#4924)
• Fix error handling while reading analyzer mapping rules (6d20423)

Security

• Fixes CVE-2022-41917 (#5141)
• CVE-2022-25857 org.yaml:snakeyaml DOS vulnerability (#4341)