docs/release-notes/index.md
Review the changes, fixes, and more in each version of Elasticsearch.
To check for security updates, go to Security announcements for the Elastic stack.
% Release notes include only features, enhancements, and fixes. Add breaking changes, deprecations, and known issues to the applicable release notes sections.
% ## version.next [elasticsearch-next-release-notes]
% ### Features and enhancements [elasticsearch-next-features-enhancements] % *
% ### Fixes [elasticsearch-next-fixes] % *
stack: ga 9.3.3
Authentication:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #140049manage, create_index, read, index, write, delete, permission for third-party agent indices in the Kibana system to support the unified alert data stream. #142648Monitoring:
Search:
Transform:
runtime_mappings are present #142452Aggregations:
CCS:
_resolve/index API should be able to accept an empty body #143159Data streams:
timestamp.us to long #143173ES|QL:
StubRelation due to SurrogateExpressions replacement #142882 (issue: #142219)Highlighting:
UnsupportedOperationException when using a plain highlighter with a query on a field used for index sorting (index.sort.*). #143680Machine Learning:
required_native_memory_bytes calculation #143077 (issue: #107831)Mapping:
Search:
Security:
SearchApplicationIndexService buffer lifecycle #143134Snapshot/Restore:
CancellableRateLimitedFluxIterator #141974Transform:
Vector Search:
ClassCastException with wrapped directories #143531Authentication:
Security:
CAT APIs:
ArrayIndexOutOfBoundsException in cat nodes for partial load average arrays #144859CCR:
ES|QL:
LuceneOperator.Status memory consumption with large QueryDSL queries #143175 (issue: #143164)FIPS:
Geo:
Infra/Core:
Machine Learning:
Search:
ArrayIndexOutOfBoundsException in fetch phase with partial results #144385 (issue: #140495)ContextIndexSearcher.totalTermFreq #144333Snapshot/Restore:
AzureBlobStore #144654Vector Search:
Watcher:
Authentication:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #140049manage, create_index, read, index, write, delete, permission for third-party agent indices in the Kibana system to support the unified alert data stream. #142648Monitoring:
Search:
Transform:
runtime_mappings are present #142452Aggregations:
CCS:
_resolve/index API should be able to accept an empty body #143159Data streams:
timestamp.us to long #143173ES|QL:
StubRelation due to SurrogateExpressions replacement #142882 (issue: #142219)Highlighting:
UnsupportedOperationException when using a plain highlighter with a query on a field used for index sorting (index.sort.*). #143680Machine Learning:
required_native_memory_bytes calculation #143077 (issue: #107831)Mapping:
Search:
Security:
SearchApplicationIndexService buffer lifecycle #143134Snapshot/Restore:
CancellableRateLimitedFluxIterator #141974Transform:
Vector Search:
ClassCastException with wrapped directories #143531Authentication:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #140049manage, create_index, read, index, write, delete, permission for third-party agent indices in the Kibana system to support the unified alert data stream. #142648Monitoring:
Search:
Transform:
runtime_mappings are present #142452Aggregations:
Data streams:
timestamp.us to long #143173ES|QL:
Machine Learning:
required_native_memory_bytes calculation #143077 (issue: #107831)SQL:
Search:
Security:
SearchApplicationIndexService buffer lifecycle #143134Snapshot/Restore:
CancellableRateLimitedFluxIterator #141974Transform:
Packaging:
Cluster Coordination:
ES|QL:
BytesRef when creating a constant vector block #141242 (issues: #140615, #140809, #140621)Inference:
ChunkingSettings field missing from ModelConfigurations equals method #142238Infra/Core:
Infra/Logging:
Machine Learning:
job_id filter to Anomaly Detection data deleter #138160Mapping:
Ranking:
Search:
CancellableBulkScorer for better responsiveness #141747Snapshot/Restore:
Transform:
_start when user lacks remote index permissions #142403 (issue: #95367)ES|QL:
Packaging:
Allocation:
NOT_PREFERRED unless its an improvement #141565Cluster Coordination:
Data streams:
ES|QL:
BytesRef when creating a constant vector block #141242 (issues: #140615, #140809, #140621)UnionAll branches #141262Inference:
ChunkingSettings field missing from ModelConfigurations equals method #142238Infra/Core:
Infra/Logging:
Machine Learning:
Mapping:
Ranking:
SQL:
Search:
CancellableBulkScorer for better responsiveness #141747Security:
Snapshot/Restore:
Transform:
_start when user lacks remote index permissions #142403 (issue: #95367)Vector Search:
::::{dropdown} Add CEF processor to Ingest node
The cef ingest processor parses a CEF (Common Event Format) message into a structured JSON object. It follows the CEF specification to extract fields such as device vendor, device product, device version, signature ID, name, severity, and extension fields. This allows users to easily analyze and visualize CEF-formatted logs within Elasticsearch and Kibana.
::::
::::{dropdown} Evaluate time series aggregation functions over sliding windows Time series aggregations in ES|QL can be optionally calculated over a sliding time window. Using a window larger than the time bucket interval reduces jitter in dashboards, by taking a larger number of data points into account.
Windows can be specified in all supported time series aggregations, as a second argument. For instance, the following ES|QL query calculates the average rate of requests per host every minute, using a 10-minute window:
TS metrics | WHERE TRANGE(1h) | STATS avg(rate(requests, 10m)) BY TBUCKET(1m), host
Acceptable window values are currently limited to multiples of the time bucket value (e.g. BUCKET or TBUCKET),
as specified in the BY-clause. If no window is provided, it's implicitly set to the time bucket value,
matching existing behavior.
::::
::::{dropdown} New exponential_histogram field type for native OpenTelemetry exponential histogram support
A new exponential_histogram field type has been added to Elasticsearch, providing native support for
OpenTelemetry exponential histograms.
Exponential histograms offer an efficient way to collect distributions (e.g. response times) at a high compression ratio while still maintaining good accuracy for percentile estimation. They can be easily collected via OpenTelemetry SDKs, as the only required configuration is the maximum allowed number of buckets. The actual buckets will be chosen automatically based on the observed values.
The new exponential_histogram field type is supported in ES|QL.
It can be aggregated using the PERCENTILES, AVG, MIN, MAX and SUM functions.
::::
::::{dropdown} New dense_vector options for storing bfloat16 vectors and utilising on-disk rescoring
New options have been added to the dense_vector field type.
The first is support for storing vectors in bfloat16 format.
This is a floating-point format that utilises two bytes per value rather than four, halving the storage space
required compared to element_type: float. This can be specified with element_type: bfloat16
when creating the index, for all dense_vector indexing types.
Float values are automatically rounded to two bytes when writing to disk, so this format can be used
with original source vectors at two- or four-byte precision. BFloat16 values are zero-expanded back to four-byte floats
when read into memory. Using bfloat16 will cause a loss of precision compared to
the original vector values, as well as a small performance hit due to converting between bfloat16 and float
when reading and writing vectors; however this may be counterbalanced by a corresponding decrease in I/O,
depending on your workload.
The second option is to enable on-disk rescoring. When rescoring vectors during kNN searches, the raw vectors are read into memory. When the vector data is larger than the amount of available RAM, this might cause the OS to evict some in-memory pages that then need to be paged back in immediately afterwards. This can cause a significant slowdown in search speed. Enabling on-disk rescoring causes rescoring to use raw vector data on-disk during rescoring, and to not read it into memory first. This can significantly increase search performance in such low-memory situations.
Enable on-disk rescoring using the on_disk_rescore: true index option.
::::
::::{dropdown} Add compression for binary doc values Add compression for binary doc values using Zstd and blocks with a variable number of values.
Block-wise LZ4 compression was previously added to Lucene in LUCENE-9211 and removed in LUCENE-9378 due to query performance issues. This approach stored a constant number of values per block (specifically 32 values). This made it easy to map a given value index (e.g., docId) to the block containing it by doing blockId = docId / 32. Unfortunately, if values are very large, we must still have exactly 32 values per block, and (de)compressing a block could cause very high memory usage. As a result, we had to keep the number of values small, meaning that in the average case, a block was much smaller than ideal.
To overcome the issues of blocks with a constant number of values, this PR adds block-wise compression with a variable number of values per block. It stores a minimum of 1 document per block and stops adding values when the size of a block exceeds a threshold or the number of values exceeds a threshold. Like the previous version, it stores an array of addresses for the start of each block. Additionally, it stores a parallel array with the docId at the start of each block. When looking up a given docId, if it is not in the current block, we binary search the array of docId starts to find the blockId containing the value. We then look up the address of the block. After this, decompression works very similarly to the code from LUCENE-9211; the main difference being that Zstd(1) is used instead of LZ4.
The introduction of binary doc value compression transparently affects wildcard field types, like URLs that are common in access logs, which will now compress much better. ::::
::::{dropdown} Enable doc_values skippers
Doc_values skippers add a sparse index to doc_values fields, allowing efficient
querying and filtering on a field without having to build a separate BKD or terms
index. These are now enabled automatically on any field configured with
index=false and doc_values=true if the index setting index.mapping.use_doc_values_skipper
is set to true (default false, or true for TSDB indexes).
TSDB indexes now default to using skippers in place of indexes for their
@timestamp, dimension and _tsid fields, greatly reducing their on-disk
footprint. To disable skippers in TSDB indexes, set index.mapping.use_doc_values_skipper
to false.
::::
::::{dropdown} Enable large numeric blocks for TSDB codec The size of numeric blocks gets increased from 128 to 512, for indices with [index.mode:time-series]. This mostly improves compression for multi-values in fields using sorted-set doc values, such as keyword and ip fields, as run-length encoding covers more data for the same storage footprint. ::::
Aggregations:
constant_keyword fields #139106Allocation:
RoutingNode #137694shardStarted events for simulation in DesiredBalanceComputer #135597Authentication:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #139345.endpoint-scripts-file* indexes #139245CCS:
project_routing for _search and _async_search #137566project_routing for _msearch #138822true for CPS searches #138105Cluster Coordination:
ThreadWatchdog to ClusterApplierService #134361Codec:
_id #138515Data streams:
match_only_text subfield to *.display_name fields in ecs@mappings to be compliant with the latest additions in ECS #136265Distributed:
_id postings #138745TransportGetBasicStatusAction runs on local #137567Downsampling:
EQL:
ES|QL:
m alias for minute duration literal #136448 (issue: #135552)time_zone request param support to KQL and QSTR functions #138695_source when applicable #138013chunking_settings as optional argument #138123TimeSeriesGroupByAll #139882ValuesFromManyReader.estimatedRamBytesUsed #139397SingleValueMatchQuery #136195network_direction function #136133match_only_text mapping #137026@timestamp attribute #137040 (issue: #136772)include_execution_metadata always return data, also in local only #137641date_parse #136548 (issue: #132487)OrderBy followed by InlineJoin on top of it #137648outputField) in TOP function. Values of outputField will be returned instead of values of field #135434 (issue: #128630)BlockSourceReader should always apply source filtering #136438DateDiff timezone support #138316Geo:
Health:
shard_capacity thresholds #136141 (issue: #116697)ILM:
IdentityProvider:
LimitedRole in idp role resolution #140536Indices APIs:
PUT _component_template API #135644_mapping API #138367Inference:
max_batch_size setting to EIS dense and sparse service settings #141185Infra/Plugins:
upgrade.rollbacks mapping to .fleet-agents system index #139363Infra/REST API:
Ingest:
Ingest Node:
mustache.java to 0.9.14 #138923Machine Learning:
Mapping:
index.mapping.nested_parents.limit and raise nested fields limit to 100 #138961inference_id of semantic_text fields #136120geo_point fields #135883pattern_text args column #139466DocumentMapper when creating new MapperService #138489AggregateMetricDouble fields should not build BKD indexes #138724Monitoring:
Network:
Relevance:
chunk_rescorer usage to output of explain and profile for text_similarity_rank_retriever #137249SQL:
Search:
PointInTime #138962project_routing also in the body of the request #138681Searchable Snapshots:
Security:
Snapshot/Restore:
compareAndExchange using conditional writes #139228TSDB:
_id format for time-series datastreams #137274_id postings #138568 #138876_tsid in synthetic _id postings #138568Transform:
Vector Search:
on_disk_rescore: true option to disk BBQ to rescore vectors on-disk without loading into memory #135778rank_vectors #139463semantic_text fields to use optional GPU indexing for HNSW and int8_hnsw #138999gpu_vectors_indexing feature flag #139318Aggregations:
SearchContext CB memory accounting #138002Allocation:
CCS:
Codec:
Data streams:
Downsampling:
EQL:
ES|QL:
_id field of time-series indices #140102 (issues: #140033, #135689)foo.bar) and for parameters (e.g. ??my_param) in FUSE GROUP BY #135901TooComplexToDeterminizeException within ESQL #137024qualifiedName #139071outputField #138380 (issue: #134083)INLINE STATS GROUP BY null being incorrectly pruned #140027 (issue: #139887)BytesRefLongBlockHash #137050 (issue: #137021)EsqlIllegalArgumentException for invalid window values #139470InlineJoin right aggregations by delegating to the child plan #139357 (issue: #138283)ignore_above for text fields #140622ILM:
migrateToDataTiersRouting #140892 (issue: #140885)Indices APIs:
Inference:
Infra/Node Lifecycle:
Ingest Node:
ip_location processors #138728Machine Learning:
max_batch_size for GoogleVertexAI embedding service settings #138047job_id filter to Anomaly Detection data deleter #138160max_number_of_allocations > 1 for low priority model deployments #140163 (issue: #111227)TextExpansionQueryBuilder and TextEmbeddingQueryVectorBuilder to return 400 instead of 500 errors #135800ElasticInferenceServiceDenseTextEmbeddingsServiceSettings to InferenceNamedWriteablesProvider #138484Mapping:
ignore_above #132962index.mapping.use_doc_values_skippers defaults in serverless #139526isSyntheticSourceEnabled flag #137297Packaging:
Reindex:
_delete_by_query and _update_by_query for CCS/stateful #140301Rollup:
SQL:
InnerAggregate #136854Search:
BinaryDvConfirmedQuery to avoid OOM on queries against WildCard fields #136086Security:
-index, -logs-*).
#138467 (issues: #64752, #83435)Snapshot/Restore:
CancellableRateLimitedFluxIterator #141323Stats:
DataStreamShardStats objects with negative timestamps #139854Vector Search:
semantic_text fields #138989numCands instead of k #140839on_disk_rescore as its valid to pass false #141158ES|QL:
BlockSourceReader should always apply source filtering #136438IdentityProvider:
LimitedRole in idp role resolution #140536Inference:
max_batch_size setting to EIS dense and sparse service settings #141185Snapshot/Restore:
Allocation:
EQL:
_index on missing events in CCS #140886ES|QL:
ignore_above for text fields #140622ILM:
migrateToDataTiersRouting #140892 (issue: #140885)Inference:
Machine Learning:
Packaging:
Reindex:
_delete_by_query and _update_by_query for CCS/stateful #140301Search:
Snapshot/Restore:
CancellableRateLimitedFluxIterator #141323Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #140046Infra/Core:
Security:
Data streams:
ES|QL:
Inference:
Infra/Core:
Infra/Node Lifecycle:
Recovery:
Transform:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #140046Infra/Core:
Security:
Data streams:
ES|QL:
STATS #139442 (issue: #135926)DeduplicateAggs #139175 (issues: #138346, #139541)Inference:
Infra/Core:
Infra/Node Lifecycle:
Machine Learning:
Mapping:
InetAddresses #139460Recovery:
Snapshot/Restore:
disable_chunked_encoding S3 repo setting #139788Transform:
Infra/Core:
Infra/Logging:
Ingest Node:
Security:
CCS:
msearch's search requests #138583Cluster Coordination:
IndicesClusterStateService applyClusterState #132536Downsampling:
ES|QL:
Inference:
text_embedding requests #139413Infra/Core:
Ingest Node:
Machine Learning:
chunkedInfer() #138632Search:
CompoundRetrieverBuilder and fixed how partial search results flag is passed through to CompoundRetrieverBuilder #136732Security:
Stats:
Infra/Core:
Infra/Logging:
Ingest Node:
Security:
Vector Search:
vectors.indexing.use_gpu setting #138738Data streams:
Downsampling:
ES|QL:
Inference:
text_embedding requests #139413Infra/Core:
Machine Learning:
chunkedInfer() #138632Mapping:
InetAddress parsing #139420Search:
CompoundRetrieverBuilder and fixed how partial search results flag is passed through to CompoundRetrieverBuilder #136732LeafCollector#setScorer in TopHitsAggregator #138883Security:
Stats:
Vector Search:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #138094 (issue: #138093)ES|QL:
Security:
Snapshot/Restore:
Vector Search:
Aggregations:
Data streams:
time_series #137852Downsampling:
ES|QL:
AggregateMetricDoubleBlock #138539ILM+SLM:
Indices APIs:
Infra/Core:
Infra/Settings:
Ingest Node:
DatabaseNodeService error logs to warnings #138438GeoIpDownloader #137660 (issues: #135158, #130681, #135132, #133597)Machine Learning:
num_threads #138122 (issue: #137129)message_start event during chat_completion operation #137926Mapping:
index.mapping.pattern_text.disable_templating not registered issue #138265Network:
BytesTransportResponse when proxying response from/to local node #135873Relevance:
knn query with minimum similarity and when using bbq_disk #138140Search:
RankDocRetrieverBuilder when from is set to Default (-1) #137637vector_similarity_support in InferenceFeatures #138644QueryPhase #138084Security:
Snapshot/Restore:
rename_replacement parameter in snapshot restore request #137859TSDB:
Vector Search:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #138094 (issue: #138093)Security:
Snapshot/Restore:
Aggregations:
Downsampling:
ES|QL:
ILM+SLM:
Indices APIs:
Infra/Core:
Infra/Settings:
Ingest Node:
Machine Learning:
Mapping:
Network:
BytesTransportResponse when proxying response from/to local node #135873Relevance:
knn query with minimum similarity #138140Search:
RankDocRetrieverBuilder when from is set to Default (-1) #137637QueryPhase #138084Security:
Snapshot/Restore:
rename_replacement parameter in snapshot restore request #137859Vector Search:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #134894 (issue: #134183)manage, create_index, read, index, write, delete, permission for third-party agent indices in the Kibana system to support the threat event data stream. #137222 (issue: #240901)Infra/Core:
Infra/Plugins:
Packaging:
Authorization:
kibana_system the ability to forcemerge certain indices #135795ES|QL:
Geo:
ignore_unmapped setting when using geo_shape query with a pre-indexed shape #136961 (issue: #136954)Indices APIs:
Infra/Node Lifecycle:
Ingest Node:
Machine Learning:
Mapping:
Recovery:
mapperService in StoreRecovery.recoverFromLocalShards #137077Search:
MutableSearchResponse ref-counted to prevent use-after-close in async search #134359Authorization:
manage, create_index, read, index, write, delete, permission for third-party agent indices in the Kibana system to support the threat event data stream. #137222 (issue: #240901)ES|QL:
ILM+SLM:
Infra/Core:
Infra/Plugins:
Packaging:
Search:
Aggregations:
reverse_nested aggs #137047Allocation:
Authorization:
kibana_system the ability to forcemerge certain indices #135795Data streams:
ES|QL:
LocalRelation is reused #136467 (issue: #135679)ReplaceAliasingEvalWithProject in case of shadowing #137025 (issue: #137019)TypedAttribute #132455 (issues: #131509, #132634)ConstNullBlock in FromAggMetricDouble #136773Geo:
ignore_unmapped setting when using geo_shape query with a pre-indexed shape #136961 (issue: #136954)ILM+SLM:
auto_expand_replicas setting during index clone in searchable_snapshot #137111Indices APIs:
Infra/Node Lifecycle:
Ingest Node:
ip_location processor #137479EnterpriseGeoIpDownloader #134223 (issue: #126124)Machine Learning:
Mapping:
Recovery:
mapperService in StoreRecovery.recoverFromLocalShards #137077Search:
max_inner_product, swap dot_product for cosine for int8_hnsw GPU type #136881MutableSearchResponse ref-counted to prevent use-after-close in async search #134359TSDB:
_otlp usage with create_doc, auto_configure privileges #137325Vector Search:
Authorization:
grantsAll per privilege #136684Infra/Core:
Authorization:
ES|QL:
ResolveUnionTypes rule stateless #136492Indices APIs:
Logs:
Machine Learning:
AbstractRunnables are notified when executor stops #135966 (issue: #134651)Mapping:
_ignored when ignoring dynamic array field #136315Search:
TermsEnum eagerly #136279Security:
Transform:
Vector Search:
UnsupportedOperationException When Field Type is Vector #135994::::{dropdown} Enable Failure Store for new logs data streams
The Failure Store is now enabled by default for new logs data streams matching the pattern logs-*-*. This means that such data streams will now store invalid documents in a
dedicated failure index instead of rejecting them, allowing better visibility and control over data quality issues without loosing data. This can be enabled manually for existing data streams.
Note: With the failure store enabled, the http response code clients receive when indexing invalid documents will change from 400 Bad Request to 201 Created, with an additional response attribute "failure_store" : "used".
::::
::::{dropdown} Add support for Lookup Join on Multiple Fields Add support for Lookup Join on Multiple Fields e.g. FROM index1 | LOOKUP JOIN lookup_index on field1, field2 ::::
::::{dropdown} Safely prevent overwriting objects in S3 repositories Earlier versions of Elasticsearch had a small risk of inadvertently overwriting an object in an AWS S3 snapshot repository which, if it happened, might corrupt the repository contents. From version 9.2.0 onwards, Elasticsearch uses S3's conditional write feature to prevent this corruption.
If you are using a snapshot repository with type s3 backed by some storage which is not AWS S3, but which is fully S3-compatible,
then you will also get the benefits of this corruption protection by upgrading to 9.2.0 or later. Refer to S3-compatible
services for more
information about using Elasticsearch with S3-compatible storage services.
::::
::::{dropdown} Add support for expressions with LOOKUP JOIN in tech preview Enable Lookup Join on Expression Tech Preview FROM index1 | LOOKUP JOIN lookup_index on left_field1 > right_field1 AND left_field2 <= right_field2 ::::
::::{dropdown} Release DiskBBQ(bbq_disk) index type for dense_vector fields
This provides a new index type called DiskBBQ (bbq_disk).
DiskBBQ is a cluster based format that provides:
Current restrictions:
dense_vector fields still applyTo utilize the format, its just like any other:
PUT vectors
{
"mappings": {
"properties": {
"vector": {"type": "dense_vector", "index_options": {"type": "disk_bbq"}
}
}
}
Querying is just like any other field.
POST vectors/_search{
"query": {
"knn": {
"field": "vector",
"query_vector": <vector>,
"k": 3
}
}
}
num_candidates can be used for tuning approximate nature of the search.
Or, more granular control can be provided by setting visit_percentage directly.
::::
::::{dropdown} Enable INLINE STATS in non-snapshot builds This effectively releases INLINE STATS into tech preview.
::::{dropdown} Allow direct IO for BBQ rescoring
BBQ rescoring performance can be drastically affected by the amount of available
off-heap RAM for use by the system page cache. When there is not enough off-heap RAM
to fit all the vector data in memory, BBQ search latencies can be affected by as much as 5000x.
Specifying the vector.rescoring.directio=true Java option on all vector search
nodes modifies rescoring to use direct IO, which eliminates these very high latencies
from searches in low-memory scenarios, at a cost of a reduction
in vector search performance for BBQ indices when the vectors do all fit in memory.
This option is released in 9.1 as a tech preview whilst we analyse its effect for a variety of use cases. ::::
::::{dropdown} Add remote index support to LOOKUP JOIN Queries containing LOOKUP JOIN now can be preformed on cross-cluster indices, for example:
FROM logs-*, remote:logs-* | LOOKUP JOIN clients on ip | SORT timestamp | LIMIT 100
::::
::::{dropdown} New lucene 10.3.0 release
Allocation:
ClusterInfo #132675Authentication:
SamlAttribute toString #131173Authorization:
_security/stats #135271/_security/stats endpoint #134835grantsAll per privilege #136684manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #134894 (issue: #134183)Codec:
Data streams:
max_age rollover for tiny retentions #134941 (issue: #130960)Distributed:
TransportClusterStateAction on local node #129872Downsampling:
ES|QL:
LogicalLocalPlanOptimizer #130409SliceQueue #133245include_execution_metadata parameter #134446v_hamming function for calculating the Hamming distance between two dense vectors #132959 (issue: #132056)InlineJoin #131204round_to #131341round_to option 2 #132143dense_vector field type #135604AbsentOverTime functions #134475 (issue: #131069)date date_nanos implicit casting #133369mv_contains function #133636profile on text response formats #128627v_magnitude function #132765 (issue: #132768)ReplaceStringCasingWithInsensitiveRegexMatch rule #131531_tsid available in metadata #135204 (issue: #133205)RoundTo linear search evaluator with manual evaluators #131733RoundTo to QueryAndTags #132512date_trunc with round_to when the pre-calculated rounding points are available #128639ILM+SLM:
age_in_millis to ILM Explain Response #128866 (issue: #103659)indexing_complete to true from ReadOnly action #129945Indices APIs:
Inference:
Infra/Core:
SecureString comparisons constant time #135053Infra/Metrics:
Ingest Node:
copy_from option to the Append processor #132003merge_type parameter to the ingest simulate API #132210 (issue: #131608)created_date and modified_date #130847_type deprecation warning #134851License:
Machine Learning:
RerankRequestChunker #130485_text_structure/find_structure #133745Mapping:
pattern_text field mapper in tech preview #135370dotCount in expanding dot parser #135263Network:
Performance:
BytesArray::indexOf, which is used heavily in ndjson parsing #135087Relevance:
chunk_rescorer in text_similarity_reranker #135198Search:
SharedBlobCacheService #134374_search #130463Searchable Snapshots:
Security:
LoadedSecureSettings for keeping temporary secure settings loaded #134349ProjectMetadata to lazy index permissions builder #135337Snapshot/Restore:
Store:
TLS:
TSDB:
index.dimensions based routing and _tsid creation strategy #135673_otlp/v1/metrics) as tech preview #135401index.dimensions private index setting #135402Vector Search:
semantic_text fields #135262_source #134717knn queries #134458bbq_disk) index type for dense_vector fields #135299_source transparently #130382OptimizedScalarQuantizer #131599Allocation:
Analysis:
isIndexed in text fields when generating field exists queries to avoid ISE when field is stored but not indexed or with doc_values #130531Authorization:
Cluster Coordination:
IndicesClusterStateService applyClusterState #132536Codec:
Data streams:
Distributed:
RemoteClusterService.collectNodes() #131937ES|QL:
round_to with expensive queries #135987OrdinalBytesRefBlock #136444 (issue: #136423)AsyncOperator status values and add emitted rows #132738StubRelation for inlinestats #135547countDistinctWithConditions in csv-spec tests #135097 (issue: #134380)LocalRelation shortcut #135011ResolveUnionTypes rule stateless #136492ExecutesOn.Any by default #133064maxPageSize to LuceneTopNOperator scorer #135767ILM+SLM:
SnapshotLifecycleTask #136709Indices APIs:
TransportSimulateIndexTemplateAction.resolveTemplate() to account for data stream overrides #132131 (issue: #131425)Infra/Core:
Infra/Scripting:
Ingest Node:
NormalizeForStreamProcessor #134524ignore_empty_values edge case #136649Logs:
Machine Learning:
AbstractRunnables are notified when executor stops #135966 (issue: #134651)TrainedModelAssignmentRebalancer and related classes #133930RequestExecutorService queue full #134178Mapping:
_ignored when ignoring dynamic array field #136315Network:
NullPointerException in transport trace logger #132243Search:
_inference_fields #135897TermsEnum eagerly #136279index_options for semantic_text fields when include_defaults is true #129967field_caps resolve calls #131723Security:
TSDB:
Transform:
Vector Search:
UnsupportedOperationException When Field Type is Vector #135994::::{dropdown} Security advisory The 9.0.8 release contains fixes for potential security vulnerabilities. Please see our security advisory for more details. ::::
Audit:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #134636 (issue: #134136)Infra/Plugins:
Aggregations:
ES|QL:
Engine:
ThreadPoolMergeScheduler when a failing merge closes the IndexWriter #134656Geo:
Infra/Node Lifecycle:
Ingest Node:
Machine Learning:
Mapping:
Search:
Transform:
::::{dropdown} Security advisory The 9.1.5 release contains fixes for potential security vulnerabilities. Please see our security advisory for more details. ::::
::::{dropdown} Prevent LIMIT + MV_EXPAND before remote ENRICH Queries using LIMIT followed by MV_EXPAND before a remote ENRICH can produce incorrect results due to distributed execution semantics. These queries are now unsupported and produce an error. Example:
FROM *:events | SORT @timestamp | LIMIT 2 | MV_EXPAND ip | ENRICH _remote:clientip_policy ON ip
To avoid this error, reorder your query, for example by moving ENRICH earlier in the pipeline. ::::
Audit:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #134636 (issue: #134136)Aggregations:
Codec:
ES|QL:
MvExpand before remote Enrich #135051Engine:
ThreadPoolMergeScheduler when a failing merge closes the IndexWriter #134656Geo:
Infra/Core:
Infra/Node Lifecycle:
Ingest Node:
Machine Learning:
Mapping:
SearchExecutionContext in SourceConfirmedTextQuery #134887Search:
Transform:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #133793 (issue: #133703)FIPS:
Infra/Plugins:
Network:
Security:
Aggregations:
Authorization:
ES|QL:
Indices APIs:
Infra/Core:
java.xml from system modules #133671Infra/Scripting:
DefBootstrap to handle Error from ClassValue #133604Infra/Settings:
Ingest Node:
allow_duplicates edge case bug in append processor #134319Machine Learning:
Mapping:
Network:
Relevance:
semantic_text fields in indices created prior to 8.11.0 #133080Search:
SourceConfirmedTextQuery #134451 (issue: #134432)Snapshot/Restore:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #133793 (issue: #133703)FIPS:
Network:
Security:
Authorization:
Indices APIs:
Infra/Core:
java.xml from system modules #133671Infra/Scripting:
DefBootstrap to handle Error from ClassValue #133604Infra/Settings:
Ingest Node:
allow_duplicates edge case bug in append processor #134319Machine Learning:
Mapping:
Relevance:
semantic_text fields in indices created prior to 8.11.0 #133080Search:
::::{dropdown} Security advisory The 9.0.6 release contains fixes for potential security vulnerabilities. Please see our security advisory for more details. ::::
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #132387 (issue: #131825)Infra/REST API:
Ingest Node:
Packaging:
EQL:
Ingest Node:
License:
Machine Learning:
::::{dropdown} Security advisory The 9.1.3 release contains fixes for potential security vulnerabilities. Please see our security advisory for more details. ::::
Infra/REST API:
Ingest Node:
Packaging:
Data streams:
EQL:
ES|QL:
Ingest Node:
License:
Machine Learning:
Search:
Authorization:
manage, create_index, read, index, write, delete, permission for third party agent indices kibana_system #132387 (issue: #131825)Aggregations:
:::{dropdown} Validates parent aggregation type in bucket_script
The bucket_script pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation.
This caused a ClassCastException at runtime when the parent was not multi-bucket.
#132320 adds a validation step so the aggregation fails early, preventing the runtime error. (issue: #132272)
:::
Codec:
:::{dropdown} Uses local segment fieldInfos for TSDB merge stats
Merging shrink TSDB or LogsDB indices in versions 8.19 or 9.1+ could fail when using addIndexes to combine Lucene segments directly.
In these cases, the fieldInfos value could differ between shards and the merged segment, causing incorrect merge statistics.
PR #132597 updates the process to use fieldInfos from each segment instead of the merged segment, ensuring accurate stats and preventing merge failures.
:::
ES|QL:
:::{dropdown} Fixes for COPY_SIGN function in ESQL
The COPY_SIGN function has been updated to better support the literal NULL in parameters.
#132459
:::
Mapping: :::{dropdown} Calculates text string length correctly for code points outside BMP Strings parsed with the optimized UTF-8 parsing path had incorrect length calculations for characters outside the basic multilingual plane (BMP). These characters require two UTF-16 code units, but the optimized path did not account for this, causing mismatches with the non-optimized path. #132593 fixes the calculation to ensure consistent and correct string lengths. :::
Search:
:::{dropdown} Always stops the timer when profiling the fetch phase
Exceptions in fetch sub-phases (for example, setNextReader) left the profiling timer running, causing mismatched start/stop calls and errors.
#132570 ensures the timer.stop() call always stops.
:::
Engine:
Aggregations:
:::{dropdown} Validate parent aggregation type in bucket_script
The bucket_script pipeline aggregation didn’t validate that its parent aggregation was a multi-bucket aggregation.
This caused a ClassCastException at runtime when the parent was not multi-bucket.
#132320 adds a validation step so the aggregation fails early, preventing the runtime error. (issue: #132272)
:::
Data streams:
:::{dropdown} Disables auto-sharding for LOOKUP index mode
Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to LOOKUP.
This happened because lookup mappers do not support scaling beyond one replica.
#131429 resolves this issue by disabling auto-sharding for data streams with LOOKUP index modes, avoiding unsupported replica settings.
:::
EQL:
:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in OR expressions
Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an OR expression.
This occurred because lookup operators were internally rewritten into IN expressions, which are unsupported for IP-type fields.
#132167 resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: #118621)
:::
:::{dropdown} Prevent double invocation of EQL listener
In some cases, the EQL listener could be resolved twice, potentially leading to unexpected behavior.
#124918 updates the control flow to exit early and ensure the listener is only invoked once.
:::
ES|QL:
:::{dropdown} Disallow remote ENRICH after LOOKUP JOIN
Combining a LOOKUP JOIN with remote ENRICH could trigger a ClassCastException due to pipeline breaker interactions when limits or top-N queries were involved. #131426 adds a validation that forbids remote ENRICH after LOOKUP JOIN, preventing the runtime error. (issue: #129372)
:::
:::{dropdown} Fix mv_expand inconsistent column order
The mv_expand command could return columns in a different order depending on query execution paths. Now, the new attribute generated by mv_expand preserves the original field positions in the output. #129745 (issue: #129000)
:::
:::{dropdown} Fixes ConcurrentModificationException caused by live operator list
A ConcurrentModificationException caused test failures in CrossClusterAsyncEnrichStopIT.testEnrichAfterStop under certain conditions.
This happened because the ES|QL driver added a live operator list to the DriverStatus object, which could be modified while the status was being serialized.
#132260 fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads.
(issue: #131564)
:::
Infra/Core:
:::{dropdown} Grants server module read/write permissions for deprecated path.shared_data setting
The server module is now granted read/write permissions for the deprecated path.shared_data setting.
#131680 resolves issues surfaced in internal testing and ensures compatibility with legacy configurations.
:::
Ingest Node:
:::{dropdown} Correctly handle download_database_on_pipeline_creation in default or final pipelines
A bug in the download_database_on_pipeline_creation setting caused geoip databases not to download when the geoip processor was referenced from a pipeline processor in a default or final pipeline.
This resulted in documents being tagged with _geoip_database_unavailable_GeoLite2-City.mmdb instead of having geo data.
#131236 resolves the issue and ensures geoip databases download correctly in this scenario.
:::
:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when mapping_addition is provided
When using the simulate ingest API with a mapping_addition, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed.
This caused mismatches between the index and simulation behavior.
#132101 resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution.
:::
Machine Learning:
:::{dropdown} Fix memory usage estimation for ELSER models
Using the deployment ID instead of the model ID caused isElserV1Or2Model to fail for ELSER models, because deployment IDs don’t start with .elser_model_2.
#131630 updates the code to pass the model ID, ensuring memory usage is estimated correctly.
:::
:::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation
A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections.
#131990 resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments.
:::
Mapping:
:::{dropdown} Fixes decoding failure for non-ASCII field names in _ignored_source
A decoding error occurred when field names in _ignored_source contained non-ASCII characters.
This happened because String.length() was used to calculate the byte length of the field name, which only works correctly for ASCII characters.
#132018 resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding.
:::
Search:
:::{dropdown} Correct shard status reporting in point-in-time responses
The Open PIT API incorrectly swapped the skipped and failed shard counts when partial search results were allowed. This caused the API to report failed shards as skipped and vice versa. #131391 fixes the field mapping so shard status is reported accurately. (issue: #131026)
:::
:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase
A missing removal of a query cancellation callback caused unintended timeouts or cancellations in later search phases when allow_partial_search_results was enabled, which could lead to ArrayIndexOutOfBoundsException errors.
#130279 resolves the issue and ensures predictable search execution. (issue: #130071)
:::
:::{dropdown} Preserve boost and queryName for semantic queries
Query rewrite logic dropped boost and queryName values for match, knn, and sparse_vector queries on semantic_text fields, causing query weighting and naming to be lost. #129282 resolves the issue so these values are now preserved correctly during query rewriting.
:::
Snapshot/Restore: :::{dropdown} Improve error handling when verifying an empty snapshot repository
Verifying the integrity of a brand-new snapshot repository without any index blobs failed with a low-level error because the repository generation was -1, which cannot be sent over the wire. #131677 updates the logic to reject such requests early with a clearer, more helpful error message.
:::
Data streams:
:::{dropdown} Disables auto-sharding for LOOKUP index mode
Auto-sharding for data streams caused unsupported replica scaling when the index mode was set to LOOKUP.
This happened because lookup mappers do not support scaling beyond one replica.
#131429 resolves this issue by disabling auto-sharding for data streams with LOOKUP index mode, avoiding unsupported replica settings.
:::
EQL:
:::{dropdown} Resolves EQL parsing failure for IP-mapped fields in OR expressions
Parsing EQL queries failed when comparing the same IP-mapped field to multiple values joined by an OR expression.
This occurred because lookup operators were internally rewritten into IN expressions, which are unsupported for IP-type fields.
#132167 resolves the issue and ensures EQL can now successfully parse and execute such or queries involving IP fields. (issue: #118621)
ES|QL:
:::{dropdown} Fixes inconsistent equality and hashcode behavior for ConstantNullBlock
Inconsistent equality checks caused constantNullBlock.equals(anyDoubleBlock) to return false, even when doubleBlock.equals(constantNullBlock) returned true.
This asymmetry led to unreliable comparisons and mismatched hashcodes when ConstantNullBlock was functionally equivalent to other standard blocks.
#131817 resolves the issue and ensures both equality and hashcode functions are symmetric for these block types.
:::
:::{dropdown} Fixes ConcurrentModificationException caused by live operator list
A ConcurrentModificationException caused test failures in CrossClusterAsyncEnrichStopIT.testEnrichAfterStop under certain conditions.
This happened because the ES|QL driver added a live operator list to the DriverStatus object, which could be modified while the status was being serialized.
#132260 fixes the issue by copying the operator list before storing it, preventing concurrent changes during status reads.
(issue: #131564)
:::
:::{dropdown} Prevents null pointer exception for to_lower and to_upper with no parameters
Calling the to_lower or to_upper functions with no parameters caused a null pointer exception (NPE), instead of returning a clear error.
This behavior was a result of an older implementation of these functions.
#131917 resolves the issue and ensures that empty parameter calls now return the correct error message. (issue: #131913)
:::
:::{dropdown} Fixes aggregate_metric_double decoding and mv_expand behavior on multi-index queries
Sorting across multiple indices failed when one index contained an aggregate_metric_double field and another did not.
In this case, the missing field was encoded as NullBlock but later incorrectly decoded as AggregateMetricDoubleBlock, which expects four values. This mismatch caused decoding errors.
#131658 resolves the issue and also improves mv_expand by returning the input block unchanged for unsupported AggregateMetricDoubleBlock values, avoiding unnecessary errors.
:::
:::{dropdown} Fixes incorrect ingest_took value when combining bulk responses
Combining two BulkResponse objects with ingest_took set to NO_INGEST_TOOK resulted in a combined ingest_took value of -2, which was invalid.
This occurred because the combination logic failed to preserve the sentinel NO_INGEST_TOOK constant.
#132088 resolves the issue and ensures the result is correctly set to NO_INGEST_TOOK when applicable.
:::
:::{dropdown} Disallows remote ENRICH after FORK in query plans
An invalid combination of FORK followed by a remote-only ENRICH caused incorrect query planning and failed executions. #131945 resolves the issue by explicitly disallowing this combination, preventing invalid plans from being executed. (issue: #131445)
:::
:::{dropdown} Adds support for splitting large pages on load to avoid memory pressure
Loading large rows from a single segment occasionally created oversized pages when decoding values row-by-row, particularly for text and geo fields.
This could cause memory pressure or degraded performance.
#131053 resolves the issue by estimating the size of each page as rows are loaded.
If the estimated size exceeds a configurable jumbo threshold (defaulting to one megabyte), row loading stops early, the page is returned, and remaining rows are processed in subsequent iterations.
This prevents loading incomplete or oversized pages during data aggregation.
:::
Infra/Core:
:::{dropdown} Grants server module read/write permissions for deprecated path.shared_data setting
Grants the server module read/write access to the deprecated path.shared_data setting.
#131680 resolves issues surfaced in internal testing and ensures compatibility with legacy configurations.
:::
Ingest Node:
:::{dropdown} Fixes incorrect mapping resolution in simulate ingest API when mapping_addition is provided
When using the simulate ingest API with a mapping_addition, the system incorrectly ignored the existing mapping of the target index and instead applied the mapping from a matching index template, if one existed.
This caused mismatches between the index and simulation behavior.
#132101 resolves the issue and ensures that the index’s actual mapping is used when available, preserving consistency between simulation and execution.
:::
Machine Learning: :::{dropdown} Prevents double-counting of allocations in trained model deployment memory estimation A recent refactor introduced a bug that caused the trained model memory estimation to double-count the number of allocations, leading to inflated memory usage projections. #131990 resolves the issue by reverting the change and restoring accurate memory estimation for trained model deployments. :::
Mapping:
:::{dropdown} Fixes decoding failure for non-ASCII field names in _ignored_source
A decoding error occurred when field names in _ignored_source contained non-ASCII characters.
This happened because String.length() was used to calculate the byte length of the field name, which only works correctly for ASCII characters.
#132018 resolves the issue by using the actual byte array length of the encoded field name, ensuring proper decoding regardless of character encoding.
:::
Search:
:::{dropdown} Fixes index sort compatibility for date_nanos fields in indices created before 7.14
Indices created prior to version 7.14 that used an index sort on a date_nanos field could not be opened in more recent versions due to a mismatch in the default index.sort.missing value.
A change in version 7.14 modified the default from Long.MIN_VALUE to 0L, which caused newer versions to reject those older indices.
#132162 resolves the issue by restoring the expected default value for indices created before 7.14, allowing them to open successfully in newer versions. (issue: #132040)
:::
:::{dropdown} Fix missing removal of query cancellation callback in QueryPhase
The timeout cancellation callback registered in QueryPhase via addQueryCancellation was not removed after the query phase completed.
This caused unintended timeouts or cancellations during subsequent phases under specific conditions (such as large datasets, low timeouts, and partial search results enabled).
#130279 resolves the issue and ensures predictable behavior by reintroducing the cleanup logic. (issue: #130071)
:::
::::{dropdown} Upgrade repository-s3 to AWS SDK v2
In earlier versions of {{es}} the repository-s3 plugin was based on the AWS SDK v1. AWS will withdraw support for this SDK before the end of the life of {{es}} 9.1 so we have migrated this plugin to the newer AWS SDK v2.
The two SDKs are not quite compatible, so please check the breaking changes documentation and test the new version thoroughly before upgrading any production workloads.
::::
::::{dropdown} Add ability to redirect ingestion failures on data streams to a failure store Documents that encountered ingest pipeline failures or mapping conflicts would previously be returned to the client as errors in the bulk and index operations. Many client applications are not equipped to respond to these failures. This leads to the failed documents often being dropped by the client which cannot hold the broken documents indefinitely. In many end user workloads, these failed documents represent events that could be critical signals for observability or security use cases.
To help mitigate this problem, data streams can now maintain a "failure store" which is used to accept and hold documents that fail to be ingested due to preventable configuration errors. The data stream's failure store operates like a separate set of backing indices with their own mappings and access patterns that allow Elasticsearch to accept documents that would otherwise be rejected due to unhandled ingest pipeline exceptions or mapping conflicts.
Users can enable redirection of ingest failures to the failure store on
new data streams by specifying it in the new data_stream_options field
inside of a component or index template:
PUT _index_template/my-template
{
"index_patterns": ["logs-test-*"],
"data_stream": {},
"template": {
"data_stream_options": {
"failure_store": {
"enabled": true
}
}
}
}
Existing data streams can be configured with the new data stream
_options endpoint:
PUT _data_stream/logs-test-apache/_options
{
"failure_store": {
"enabled": "true"
}
}
When redirection is enabled, any ingestion related failures will be
captured in the failure store if the cluster is able to, along with the
timestamp that the failure occurred, details about the error
encountered, and the document that could not be ingested. Since failure
stores are a kind of Elasticsearch index, we can search the data stream
for the failures that it has collected. The failures are not shown by
default as they are stored in different indices than the normal data
stream data. In order to retrieve the failures, we use the _search API
along with a new bit of index pattern syntax, the :: selector.
POST logs-test-apache::failures/_search
This index syntax informs the search operation to target the indices in its failure store instead of its backing indices. It can be mixed in a number of ways with other index patterns to include their failure store indices in the search operation:
POST logs-*::failures/_search
POST logs-*,logs-*::failures/_search
POST *::failures/_search
POST _query
{
"query": "FROM my_data_stream*::failures"
}
::::
::::{dropdown} Mark Token Pruning for Sparse Vector as GA Token pruning for sparse_vector queries has been live since 8.13 as tech preview. As of 8.19.0 and 9.1.0, this is now generally available. ::::
::::{dropdown} Upgrade to lucene 10.2.2
::::
::::{dropdown} Release FORK in tech preview Fork is a foundational building block that allows multiple branches of execution. Conceptually, fork is:
Example:
FROM test
| FORK
( WHERE content:"fox" )
( WHERE content:"dog" )
| SORT _fork
The FORK command add a discriminator column called _fork:
| id | content | _fork |
|-----|-----------|-------|
| 3 | brown fox | fork1 |
| 4 | white dog | fork2 |
::::
::::{dropdown} ES|QL cross-cluster querying is now generally available The ES|QL Cross-Cluster querying feature has been in technical preview since 8.13. As of releases 8.19.0 and 9.1.0 this is now generally available. This feature allows you to run ES|QL queries across multiple clusters. ::::
Allocation:
FailedShardEntry info to shard-failed task source string #125520 (issue: #102606)TransportGetAllocationStatsAction #124898 (issue: #110716)TransportGetAllocationStatsAction #127371 (issue: #123248)AllocationBalancingRoundSummaryService #120957tryRelocateShard #128063Analysis:
Authentication:
ServiceAccountTokenStore through SecurityExtensions #126612Authorization:
TooComplex exception for HasPrivileges body #128870kibana_system reserved role access to "all" privileges to .adhoc.alerts* and .internal.adhoc.alerts* indices #127321read index privileges to kibana_system role for Microsoft Defender integration indexes #126803CCS:
CRUD:
IndexingPressureMonitor to monitor large indexing operations #126372Codec:
Data streams:
TransportGetDataStreamLifecycleAction on local node #125214TransportGetDataStreamOptionsAction on local node #125213TransportGetDataStreamsAction on local node #122852Distributed:
IndexingMemoryController #126786ES|QL:
documents_found and values_loaded #125631suggested_cast #127139constant_keyword #127549NamedWritable in block serialization #124394DATE_TRUNC with arbitrary intervals #120302 (issue: #120094)match_phrase full text function (tech preview) #128925::date in inline cast #123460 (issue: #116746)aggregate_metric_double present #125191AggregateMetricDoubleBlock #127299allow_partial_results setting in ES|QL #122890==s on text fields to lucene #126641aggregate_metric_double #122660original_types #124913can_match in field-caps #127734BlockHash implementation #127148AddInput for each block type #127582ToAggregateMetricDouble function #124595text == and text != pushdown #127355Engine:
Geo:
Health:
HealthPeriodicLogger #122390ILM+SLM:
index.lifecycle.skip index-scoped setting to instruct ILM to skip processing specific indices #128736TransportExplainLifecycleAction on local node #122885TransportGetLifecycleAction on local node #126002TransportGetStatusAction on local node #129367step_info and error reason in ILM execution state and history #125054 (issue: #124181)IdentityProvider:
Indices APIs:
DELETE /{index}/_block/{block} #129128TransportGetIndexAction on local node #125652TransportGetMappingsAction on local node #122921TransportGetSettingsAction on local node #126051Inference:
model_id prop to model in EIS sparse inference request body #122272Infra/CLI:
Infra/Core:
Infra/Metrics:
Infra/Settings:
FileWatchingService shoudld not throw for missing file #126264Ingest Node:
Logs:
LuceneSyntheticSourceChangesSnapshot #128473Machine Learning:
ModelRegistryMetadata to Cluster State #121106none chunking strategy to disable automatic chunking for inference endpoints #129150ElasticInferenceService #117642ElasticsearchInternalService #123044change_point processing command #120998input_type option at root level for text_embedding task type in Perform Inference API #122638 (issue: #117856)OpenAi Chat Completion in SageMaker #127767DeepSeek API #122218InferenceService support aliases #128584SageMaker Elastic Payload #129413Mapping:
index_options to semantic_text field mappings #119967semantic_text subfields from field capabilities API #127664FallbackSyntheticSourceBlockLoader and apply it to keyword fields #119546SourceProvider creation to consistently use MappingLookup #128213seq_no in tsdb and logsdb #128139sparse_vector field mapping to include default setting for token pruning #129089FallbackSyntheticSourceBlockLoader for shape and geo_shape #124927FallbackSyntheticSourceBlockLoader for unsigned_long and scaled_float fields #122637FallbackSyntheticSourceBlockLoader for boolean and date fields #124050FallbackSyntheticSourceBlockLoader for number fields #122280FallbackSyntheticSourceBlockLoader for point and geo_point #125816FallbackSyntheticSourceBlockLoader for text fields #126237Network:
RestController #129302 (issue: #120746)FlowControlHandler from HTTP pipeline #128099connection: close header on shutdown #128025 (issue: #127984)Ranking:
QueryFeatureExtractor #125259Recovery:
Relevance:
semantic_text fields to use BBQ when models are compatible #126629semantic_text fields #121041Search:
SearchHit source in circuit breaker #121920 (issue: #89656)bucketedSort based on int #128848multi_match function #121525 #125062 (issue: #121525)ShardBulkInferenceActionFilter #124313valueFetcher #126702Snapshot/Restore:
ThreadLocal #125452state query param to Get snapshots API #128635 (issue: #97446)_snapshot/_status #128399OperationPurpose/Operation stats counters #122991Stats:
Task Management:
Vector Search:
_source retrieval #128735rescore_vector.oversample to indicate by-passing oversample and rescoring #125599dense_vector fields updatable to bbq_flat/bbq_hnsw #128291rescore_vector as generally available #126038vector_rescore parameter as a quantized index type option #124581Watcher:
TransportGetWatcherSettingsAction on local node #122857Aggregations:
Allocation:
DesiredBalanceReconciler always returns AllocationStats #122458Analysis:
Cluster Coordination:
Data streams:
event.dataset if empty for logs #129074Distributed:
IndexReshardingMetadata over the wire #124841ES|QL:
NamedWritable to plugin #131541NULL handling in IN clause #125832 (issue: #119950)mv_expand inconsistent column order #129745 (issue: #129000)ValueExtractor bugs #126296PushQueriesIT.testLike() fails #129647PushQueryIT#testEqualityOrTooBig #129657 (issue: #129545)_index LIKE for ESQL #130849 (issue: #129511)skip_unavailable and filters #130344Highlighting:
ILM+SLM:
PolicyStepsRegistry cache concurrency issue #126840 (issue: #118406)Indices APIs:
Infra/Core:
Infra/Node Lifecycle:
Infra/REST API:
RestController #128314Infra/Scripting:
toString to DynamicMap #126562 (issue: #70262)Infra/Settings:
AccessDeniedException in AbstractFileWatchingService #128653Ingest Node:
download_database_on_pipeline_creation within a pipeline processor within a default or final pipeline #131236Logs:
Machine Learning:
Mapping:
match_only_text by default #129126Ranking:
TextSimilarityRankBuilder XContent output #124564Relevance:
Search:
MultiPhraseQueries as well as phrase queries #129326 (issue: #123871)SemanticTextHighlighter #129509 (issue: #129501)boosts and queryName for match, knn, and sparse_vector queries on semantic_text fields #129282Snapshot/Restore:
newShardSnapshotTask tasks concurrently #126452Suggesters:
TLS:
Transform:
Vector Search:
Aggregations:
FilterByFilter aggregator #130452Distributed:
TransportNodesAction #130303ES|QL:
BytesRef2BlockHash #130705Infra/Core:
Machine Learning:
max_tokens through the Service API for Anthropic #131113Mapping:
Relevance:
GET _synonyms returns synonyms with empty rules #131032Search:
Vector Search:
Authorization:
Engine:
Ingest Node:
Snapshot/Restore:
Stats:
Aggregations:
Authorization:
CCS:
["*", "-*"] when grouping indices by cluster name #128610ES|QL:
FieldAttribute name usage in InferNonNullAggConstraint #128910IdentityProvider:
Indices APIs:
Infra/Core:
Infra/Plugins:
Machine Learning:
Mapping:
Search:
match_all queries for semantic_text fields #128702Searchable Snapshots:
RefreshResult.NO_REFRESH #129176 (issue: #129036)Security:
Authentication:
ES|QL:
Aggregations:
significant_terms #127975Audit:
Codec:
docId is accessed #128320Data streams:
Downsampling:
ES|QL:
field_caps check for field names #127564 (issue: #127236)DROP attributes when resolving field names #127009 (issue: #126418)Geo:
Infra/Core:
outbound_network entitlement to x-pack-core #126992 (issue: #127003)Infra/Scripting:
Machine Learning:
Relevance:
Security:
Snapshot/Restore:
TSDB:
Vector Search:
Infra/Core:
Ingest Node:
Search:
half_float #126342Security:
Aggregations:
Allocation:
CCS:
Data streams:
ES|QL:
ILM+SLM:
WaitForIndexColorStep #126605Infra/CLI:
Infra/Core:
Machine Learning:
onFailure call for Inference API start model request #126930Ranking:
Search:
TSDB:
UpdateTimeSeriesRangeService #126637Task Management:
Vector Search:
vec_caps to test for OS support too (on x64) #126911 (issue: #126809)::::{dropdown} rank_vectors field type is now available for late-interaction ranking
rank_vectors is a new field type released as an experimental feature in Elasticsearch 9.0. It is designed to be used with dense vectors and allows for late-interaction second order ranking.
Late-interaction models are powerful rerankers. While their size and overall cost doesn’t lend itself for HNSW indexing, utilizing them as second order reranking can provide excellent boosts in relevance. The new rank_vectors mapping allows for rescoring over new and novel multi-vector late-interaction models like ColBERT or ColPali.
::::
::::{dropdown} ES|QL LOOKUP JOIN is now available in technical preview LOOKUP JOIN is now available in technical preview. LOOKUP JOIN combines data from your ES|QL queries with matching records from a lookup index, enabling you to:
::::{dropdown} The semantic_text field type is now GA
semantic_text is now an official GA (generally available) feature! This field type allows you to easily set up and perform semantic search with minimal ramp up time.
::::
Allocation:
Authentication:
SSHA-256 for API key credential hash #120997Authorization:
reporting_user role #118058create_index to kibana_system role for index/DS .logs-endpoint.action.responses-* #115241kibana_system user to manage .reindexed-v8-* Security Solution indices #119054CCS:
CRUD:
Cluster Coordination:
clusterApplyListener in long cluster apply warnings #120087Data streams:
create_from_source_index #119250ReindexDatastreamIndexAction #120231ReindexDatastreamIndexAction #116996requests_per_second configurable to throttle reindexing #120207Distributed:
Downsampling:
EQL:
ES|QL:
LicenseAware interface for licensed Nodes #118931 (issue: #117405)PostAnalysisAware, distribute verification #119798_tier and _index in ES|QL #123728EsIndex in plan #119580_score #113120RENAME behavior with EVAL for sequential processing #122250 (issue: #121739)BlockHash #114317TranslationAware to all pushable expressions #120192cpuTime #123290 (issue: #122967)MetricsAware interface #121074geo_shape and cartesian_shape #119889StartsWith and EndsWith functions to Lucene #123381 (issue: #123067)outputSet inside the plan where possible #124611indexPattern instead of identifier in join target #120494_score should not be a reserved attribute in ES|QL #118435 (issue: #118460)Engine:
Experiences:
Extract&Transform:
Geo:
Health:
replica_unassigned_buffer_time default from 3s to 5s #112834Highlighting:
ILM+SLM:
replicate_for option to the ILM searchable_snapshot action #119003Indices APIs:
remove_index_block arg to _create_from api #120548create_from #120643TransportGetComponentTemplateAction on local node #116868TransportGetComposableIndexTemplate on local node #119830TransportGetIndexTemplateAction on local node #119837Inference:
Infra/CLI:
Infra/Circuit Breakers:
Infra/Core:
Infra/Metrics:
ensureGreen test method for use with adminClient #113425Infra/REST API:
?include_source_on_error was added for create / index, update and bulk REST APIs to control
if to include the document source in the error response in case of parsing errors. The default value is true. #120725Infra/Scripting:
mustache.max_output_size_bytes setting to limit the length of results from mustache scripts #114002Infra/Settings:
IndexSettingDeprecatedInV8AndRemovedInV9 Setting property #120334TransportClusterGetSettingsAction on local node #119831Ingest Node:
type in the reroute processor #122409 (issue: #121553)IngestCtxMap construction #120833IngestDocMetadata isAvailable #120753IngestDocument FieldPath allocation #120573GetPipelineTransportAction on local node #120445TransportGetEnrichPolicyAction on local node #121124License:
TrialLicenseVersion to allow starting new trial on 9.0 #120198Logs:
Machine Learning:
IbmWatsonxService #114914ClusterBlockException on transform destination index #118194Mapping:
sparse_vector outside _source #117917Network:
ConnectTransportException returns retryable BAD_GATEWAY #118681 (issue: #118320)Packaging:
Ranking:
rescorer retriever based on the search request's rescore functionality #118585 (issue: #118327)Recovery:
Reindex:
Relevance:
Search:
semantic_text fields #117839sparse_vector queries against semantic_text fields #118617semantic_text fields #119011_resolve/cluster API #120542error_trace=false #118266Security:
.security index call between security migrations #114879Snapshot/Restore:
repository-s3 #117748 (issue: #105135)S3BlobContainer#getRegister on all exceptions #114813Suggesters:
TSDB:
Transform:
extended_stats #120340max_page_search_size #119348transform_ids to outdated index #120821Vector Search:
rank_vectors mapping for late-interaction second order ranking #118804Aggregations:
illegalArgumentExceptions negative values in HDR percentile aggregations #116174 (issue: #115777)Analysis:
Authentication:
CAT APIs:
CRUD:
Data streams:
MetadataMigrateToDataStreamService when settings have not changed #118704TransportRolloverAction.checkBlock so that non-write-index blocks do not prevent data stream rollover #122905ReindexDataStreamIndex bug in assertion caused by reference equality #121325Downsampling:
default_metric properties when downsampling aggregate_metric_double #121727 (issues: #119696, #96076)ES|QL:
null in filters #118324 (issue: #116351)AggregateFunction when LogicPlan is not an Aggregate #124446 (issue: #124311)AbstractShapeGeometryFieldMapperTests #119265 (issue: #119201)ReplaceMissingFieldsWithNull #125764 (issues: #126036, #121754, #126030)fold #118602numberOfChannels consistent with layout map by removing duplicated ChannelSet #125636Engine:
InternalEngine#performActionWithDirectoryReader(...) #123010 (issue: #122974)Health:
max_shards_per_node #120458Highlighting:
force_source parameter #124873Indices APIs:
?master_timeout to POST /_ilm/migrate_to_data_tiers #12088330_create_from #120662DeprecationInfoAction #118035 (issue: #118020)ConcurrentModificationException when updating settings for more than one index #126077Inference:
Infra/Core:
Infra/Logging:
SlowLogFieldProvider instantiation to node construction #117949Infra/Metrics:
randomInstantBetween always return value in range [minInstant, maxInstant] #114177Infra/Plugins:
Infra/REST API:
NullPointerException in _capabilities API when the path parameter is null. #113413 (issue: #113413)Infra/Scripting:
Infra/Settings:
Ingest Node:
Logs:
Machine Learning:
ElasticInferenceServiceCompletionServiceSettings #123155AlibabaCloudSearchCompletionAction not accepting ChatCompletionInputs #125023InferenceActionProxy #124225Mapping:
_source fields in mappings #122606copy_to #121109 (issue: #113049)copy_to handling in synthetic _source to account for nested objects #120974 (issue: #120831)Network:
Ranking:
Search:
DfsQueryPhase and introduce search disconnect stress test #116060 (issue: #115056)SuggestPhase #122357 (issue: #122186)FieldInfos from store if not yet initialised through a refresh on IndexShard #125650 (issue: #125483)CrossClusterIT testCancel failure #117750 (issue: #108061)SearchServiceTests.testParseSourceValidation failure #117963Snapshot/Restore:
Store:
ClusterChangedEvent in IndicesStore call to #onClusterStateShardsClosed #120193Suggesters:
Transform:
Vector Search:
bbq_hnsw merge file cleanup on random IO exceptions #119691 (issue: #119392)Watcher: