@kbn/change-history

Generic change-history storage and query for Kibana.

Persists point-in-time snapshots of object changes to Elasticsearch data streams. Each stored event’s object.snapshot is the object after that change (the post-change state). To see what changed between versions, compare consecutive object.snapshot values in query results (or in your domain code); the package does not store a separate “before” image.

Solution-agnostic: use it from any plugin or module that needs audit-style history.

Unsupported functionality

Kibana objects that use a dot . in their JSON structure.

{
  "user": { "first.name": "bob"}
}

The change history package does not currently support JSON structures that use a dot . for property names, as it relies on "flattening" JSON into dot notation for JSON paths.

Overview

Single shared data stream, one client per (module, dataset):

• All clients write to one data stream: .kibana_change_history. Each client is bound to a module and dataset, analogs of the "business domain" and "feature".

Log changes with log(change, opts) / logBulk(changes, opts):

Each change has the following (see Usage examples below):

• timestamp (Optional @timestamp when the change took place — will be autogenerated if not provided)
• objectType (Used for querying — allows multiple object types in the same change history stream),
• objectId (Used for querying — uniquely identifies a kibana object),
• sequence (Optional long — monotonically increasing integer determining changes order; see Ordering and versioning).
• snapshot (Full snapshot after the change)

There is also an opts object that contains the action that took place, and relevant username and other contextual information.

Capturing the right timestamp

Pass the Saved Object's updated_at field (assigned by Elasticsearch at write time) rather than a clock reading taken before the write. For delete operations where no post-write SO is available, capture Date.now() immediately after the delete resolves. Taking the timestamp before the write means concurrent writes can land in Elasticsearch in the opposite order to what change-history records.

Query history with getHistory(spaceId, objectType, objectId, opts?):

• Returns change documents for the given object type and id in the specified Kibana space, sorted by sequence (if available), then @timestamp, then event.id as a tie-breaker. Supports pagination and custom sort/filters via opts.

All persisted documents follow the same schema (see below).

Ordering and versioning

Default browse order (getHistory without opts.sort):

1. object.sequence (desc) when the caller supplied it
2. @timestamp (desc)
3. event.id (desc) as a tie-breaker

Without object.sequence, ordering falls back to @timestamp. That is simple and works most of the time, but clock skew or concurrent writers can occasionally show changes out of order. Product copy should not promise strict ordering unless the solution passes sequence.

object.sequence (optional, long) holds a solution-defined monotonically increasing integer determining changes order for the tracked object. This number should properly handle optimistic concurrency control, reindexing, upgrades, failovers, migrations and cluster rebuilds.

Good examples:

1. Field stored in the tracked document itself and incremented every time the tracked object gets changed
2. Optimistic concurrency on the primary index/data stream (Saved Objects client / if_seq_no + if_primary_term on raw Elasticsearch indices) plus a monotonic field surviving reindexing, upgrades, failovers, migrations and cluster rebuilds

After each successful primary write, pass that counter as sequence on each log / logBulk call.

Bad examples (won't work):

• Saved Object SavedObject.version (opaque Elasticsearch OCC metadata) version is a base64-encoded string containing two Elasticsearch internal counters: [_seq_no, _primary_term]. These numbers get reset upon reindexing, upgrades, failovers, migrations and cluster rebuilds.
• Alerting rule.revision or other domain revision fields It doesn't increment on every operation and sometimes it could be reset to 1.
• ECS event.sequence (global event ordering, not per tracked object) A single global sequence can work in theory, but it is hard to maintain when many writers append to the same stream: collisions and gaps are likely without centralized allocation. Prefer a monotonic counter scoped to each tracked object (your domain’s object.id + object.type).
• Raw Elasticsearch _seq_no / _primary_term copied directly into sequence (not stable version identifiers across index lineages)

Raw Elasticsearch indices (no SO migrations): the same pattern applies—your app owns reindex policy; keep an incrementing field in _source with OCC writes, then copy it to sequence.

Future: a keyword object.version may be added when a solution needs composite string keys (for example index name + OCC). That is deferred until there is a concrete consumer.

Event IDs

Each persisted document gets a unique event.id assigned by the package (UUID v7).

UUID v7 values are monotonically increasing within the same millisecond. That matters when two change history events are written back-to-back with the same timestamp (for example rule_install and rule_enable when a user chooses "Install and enable"): getHistory() sorts by sequence, then @timestamp, then event.id, so ordering stays deterministic.

Backwards/forwards compatibility in object.snapshot

object.snapshot is stored unmapped in the .kibana_change_history data stream, separate from the original tracked object. Because of this, it is recommended that callers store an application-layer type rather than a raw on-disk format. If/when there are underlying changes in tracked object storage format, using an application-layer type will likely save on having to introduce a compensating storage-layer transform to keep old snapshots readable. See #273561 for more context and a concrete example.

API

Client

• new ChangeHistoryClient({ module, dataset, logger, kibanaVersion }) Constructs a client for the given module, dataset, and kibanaVersion. All clients write to the shared data stream .kibana_change_history; each client’s writes are scoped by module and dataset.

• isInitialized() — Returns true if the client has been initialized (e.g. after initialize() has been called).

• initialize(elasticsearchClient) Creates/ensures the data stream and stores the internal client. Called once during plugin start() phase and before log / logBulk / getHistory.

• log(change, opts) Writes one change document with given opts context (action, username, etc) in LogChangeHistoryOptions.

• logBulk(changes, opts) Same as log but for multiple changes in one request (grouped by correlationId if provided).

• LogChangeHistoryOptions — Options for logging a change.

  • Required: action, username, spaceId.
  • Optional:
    • userProfileId user profile from auth realm,
    • correlationId to groups bulk events in a common span when set,
    • change data overrides (partial event, tags, and metadata to merge into the document),
    • fieldsToHash a nested key/value map of fields to hash in the stored snapshot (only string values are hashed). Hash high-entropy secrets only — the digest is a deterministic, object.id-salted SHA-256, so low-entropy values (emails, names, etc.) stay brute-forceable. See Sensitive fields in the snapshot,
    • fieldsToRedact a nested key/value map of fields to replace with a [redacted] placeholder (only string values). Use for low-entropy sensitive data where hashing isn't safe. See Sensitive fields in the snapshot,
    • refresh an optional indicator to force ES shard refresh after changes (affects performance).

• getHistory(spaceId, objectType, objectId, opts?)

  • Returns a promise with { total, items }.
  • spaceId — The Kibana space ID where the object exists (used to scope the search).
  • Results are scoped by spaceId, the client’s module and dataset, and filtered by object.type and object.id.
  • Optional opts: GetChangeHistoryOptions with additionalFilters (array of ES query clauses), pagination options sort, from, size (default 100).
  • Results are sorted by object.sequence (if available), then @timestamp, and event.id as the tie-breaker.

Elasticsearch mapping schema

The data stream uses dynamic: false and the following index mapping (defined by changeHistoryMappings.v1 in the package):

Variable-shape field object.snapshot is stored but unmapped; metadata uses the flattened type so arbitrary keys can be stored and indexed without dynamic mapping.

Fields written but not indexed

Several fields are still written to documents (preserved in _source for forensic inspection) but are intentionally not mapped, since no consumer queries, filters, sorts, or aggregates on them. With dynamic: false, these fields land in _source without consuming inverted-index or doc-values storage:

• ecs.version — hardcoded ECS schema constant, pinned to ECS [9.3.0].
• event.created — very similar to @timestamp (set to the package serialisation moment vs. the caller-supplied write-confirmed moment).
• event.reason — optional user-specified reason for the change;
• object.hash — SHA-256 of the original snapshot; can be used to check if a version of the object already exists.
• object.fields.hashed — list of paths in object.snapshot whose string values were redacted with a SHA-256 digest.
• object.fields.redacted - List of paths in object.snapshot whose values were replaced with a [redacted] placeholder.
• service.type — hardcoded 'kibana'; the .kibana_change_history data stream identity already implies Kibana.

If a future consumer needs to filter or sort on any of these, add them back to the mapping (no document-shape change needed since they are still being written to _source).

Retention

.kibana_change_history is enrolled in data stream lifecycle with enabled: true and no data_retention. Change history documents are kept indefinitely by default.

Cluster admins can add retention later via Stack Management → Index Management → Data Streams on both stateful and serverless deployments.

Dependencies

See tsconfig.json for internal kibana references.

Usage examples

Basic usage (no frills)

import { ChangeHistoryClient } from '@kbn/change-history';
import type { ObjectChange, LogChangeHistoryOptions } from '@kbn/change-history';

// During plugin `setup()` phase
const client = new ChangeHistoryClient({
  module: 'security',
  dataset: 'detections',
  logger,
  kibanaVersion: '9.4.0',
});

// During plugin `start()` phase
await client.initialize(elasticsearchClient);
const spaceId = 'default';

// When user makes a change
const change: ObjectChange = {
  objectType: 'alerting-rule',
  objectId: ruleId,
  snapshot: ruleSnapshot, // post-change state
};
await client.log(change, {
  action: 'rule_create',
  username,
  spaceId,
});

// When reading history for an object
const { total, items } = await client.getHistory(spaceId, 'alerting-rule', ruleId);
console.log(
  `We have ${total} items, latest change at: \n${JSON.stringify(items[0]?.['@timestamp'])}`
);

Bulk changes with correlation ID

Multiple changes in one request share a span id so they can be queried together. Pass a correlationId:

const changes: ObjectChange[] = [
  { objectType: 'alerting-rule', objectId: id1, snapshot: snapshot1 },
  { objectType: 'alerting-rule', objectId: id2, snapshot: snapshot2 },
];
await client.logBulk(changes, {
  action: 'rule_bulk_update',
  username,
  spaceId,
  correlationId: 'my-bulk-operation-123',
});

Supplying sequence for reliable ordering

After a successful primary write, copy your domain's monotonically increasing integer into sequence:

The primary write and change-history log() are separate steps. The primary object can be updated even when log() or logBulk() fails (for example a transient elasticsearch error). This package does not retry failed writes today, so a failed log() leaves a gap in history until something writes again.

When you pass sequence, you can safely retry log() with the same sequence value after a transient failure: sort order stays correct and you do not need a new counter increment on the primary object. The main remaining gap is a kibana process crash after the primary write succeeds but before log() completes.

await client.log(
  {
    objectType: 'alerting-rule',
    objectId: ruleId,
    sequence: ruleAttributes.changeSequence, // domain-owned counter in _source, not rule.revision
    snapshot: newSnapshot,
  },
  { action: 'rule_update', username, spaceId }
);

Adding tags, reason, and metadata

Use data to set event fields (e.g. reason), tags, and metadata on the stored document:

await client.log(
  {
    objectType: 'alerting-rule',
    objectId: ruleId,
    snapshot: newSnapshot,
  },
  {
  action: 'rule_update',
  username,
  spaceId,
  data: {
      event: { reason: 'Threshold adjusted by user' },
      tags: ['new-rules-ui', 'manual-edit'],
      metadata: { tab: 'settings' },
    },
  }
);

Sensitive fields in the snapshot

Use fieldsToHash for high-entropy secret string fields (secrets, API keys, tokens) that need version control. Matching string values are replaced with the last 12 hex chars of sha256(object.id + value), so equal values for the same object hash identically ("did this field change?" scenarios).

[!IMPORTANT] Prefer not to store hashes at all (Kibana Security guidance) — please avoid exposing hashes of tracked data where possible, most critically for low-entropy data but also for high-entropy keys and secrets. Before reaching for fieldsToHash, check whether a non-hash signal answers your question: if you only need "did this field change between two versions?", you may record a change marker (e.g. a timestamp or revision captured when the field changed) for the same diff-ability with zero secret material stored. Only hash genuinely high-entropy secrets, and only when a non-hash value can't give the same result.

await client.log(change, {
  action: 'rule_update',
  username,
  spaceId,
  // High-entropy secrets only — see warning above
  fieldsToHash: {
    apiKey: true,
  },
});

For most secrets, low-entropy sensitive data (emails, names, IPs, short enums) and large base64 blobs use fieldsToRedact instead. Matching string values are replaced with a fixed [redacted] placeholder, so nothing about the original is stored or recoverable. When a field is listed in both maps, redaction wins.

await client.log(change, {
  action: 'rule_update',
  username,
  spaceId,
  fieldsToRedact: {
    owner: { email: true, name: true },
    source: { ip: true },
  },
});

Both fieldsToHash and fieldsToRedact only touch string values; the affected paths are recorded under object.fields.hashed and object.fields.redacted respectively.

Logging a deletion

Store the last known state as the snapshot and mark the event as a deletion:

await client.log(change, {
  action: 'rule_delete',
  username,
  spaceId,
  data: { event: { type: 'deletion', reason: 'User requested deletion' } },
});

Querying with filters and pagination

const { total, items } = await client.getHistory(spaceId, 'alerting-rule', ruleId, {
  additionalFilters: [{ range: { '@timestamp': { lt: '2026-01-01' } } }],
  size: 50,
  from: 0,
});
console.log(
  `Last update in 2025 was at ${items[0]?.['@timestamp']}`
);

Testing

Run the following from the Kibana repository root.

Unit tests (Jest, no Elasticsearch):

yarn test:jest --config=x-pack/platform/packages/shared/kbn-change-history/jest.config.js

Integration tests (Jest with a real Elasticsearch node; slower):

yarn test:jest_integration --config=x-pack/platform/packages/shared/kbn-change-history/jest.integration.config.js