docs/server/features/indexes/user-defined.md
KurrentDB v26.0 introduces support for user defined indexes, building on the secondary indexes mechanism added in v25.1.
User defined indexes allow you to create custom indexes based on record content, enabling efficient queries and subscriptions filtered by fields within your records — for example, retrieving all orders by country or users by region.
Indexes subscribe to the log and maintain index data on each node, separate from the log. This design avoids increasing the size of the log while providing fast, targeted access to records matching your criteria.
You can read from and subscribe to user defined indexes using the existing gRPC clients, or query them directly in the UI.
A user defined index can include:
true are indexed.User defined indexes can be managed through gRPC (coming to clients soon) and HTTP. See the API definition
Admin or Operations permissions are required to create/start/stop/delete indexes. Any authenticated user can list/get indexes.
POST to <host>:<port>/v2/indexes/<index-name>
e.g. with the default admin credentials:
POST https://127.0.0.1:2113/v2/indexes/orders-by-country
Content-Type: application/json
Authorization: Basic YWRtaW46Y2hhbmdlaXQ=
{
"filter": "rec => rec.schema.name == 'OrderCreated'",
"fields": [{
"name": "country",
"selector": "rec => rec.value.country",
"type": "INDEX_FIELD_TYPE_STRING"
}]
}
Filter:
filter is optional. If not provided then all user records are included in the index.filter must be deterministic based on the content of the record.filter must return a boolean value. Returning false causes the record to be excluded from the index.filter does not return a boolean value, the record will be excluded from the index and an error logged.Fields:
field. Future versions will allow multiple fields.name determines how the field will be read/subscribed/queried.selector must be deterministic based on the content of the record.selector can return skip to exclude the record from the index. This is an alternative filtering mechanism to the filter function. They can both be used.selector must return a value compatible with the field type (or return skip). Otherwise the record will be excluded from the index and an error logged.The available field types are:
INDEX_FIELD_TYPE_STRING
INDEX_FIELD_TYPE_DOUBLE
INDEX_FIELD_TYPE_INT_32
INDEX_FIELD_TYPE_INT_64
By default a user defined index will start automatically. This can be prevented by passing "start": false in the request.
The structure of the record passed to the filter and selector functions is:
{
"id": "12345678-1234-1234-1234-123456789abc", // the event ID
"timestamp": "2026-01-15T13:37:01.337Z", // the time at which the event was written to the transaction log
"position": {
"stream": "my-stream", // the stream name
"streamRevision": 2, // the event number
"logPosition": 2705 // the commit position of the record in the transaction log
},
"schema": {
"name": "my-event-type", // the event type
"format": "Json" // the data format
},
"sequence": 3, // a sequence number that auto-increments each time a record is passed to the filter
"redacted": false, // whether the record is redacted or not
"value": { // deserialized data (only when the data is JSON and not redacted)
"my": "data"
},
"properties": { // deserialized metadata
"my": "metadata"
}
}
Now if you append a record representing a OrderCreated event with a payload like
{
"orderId": "ORD-1234",
"country": "Mauritius",
"total": 149.99
}
Then it will be indexed accordingly.
User defined indexes are started by default when they are created. If the create request specified not to start the index, or the index has been stopped, then it can be started as follows:
POST to <host>:<port>/v2/indexes/<index-name>/start
e.g. with the default admin credentials:
POST https://127.0.0.1:2113/v2/indexes/orders-by-country/start
Authorization: Basic YWRtaW46Y2hhbmdlaXQ=
POST to <host>:<port>/v2/indexes/<index-name>/stop
e.g. with the default admin credentials:
POST https://127.0.0.1:2113/v2/indexes/orders-by-country/stop
Authorization: Basic YWRtaW46Y2hhbmdlaXQ=
DELETE to <host>:<port>/v2/indexes/<index-name>
e.g. with the default admin credentials:
DELETE https://127.0.0.1:2113/v2/indexes/orders-by-country
Authorization: Basic YWRtaW46Y2hhbmdlaXQ=
If deleting an index and recreating with the same name, be aware of consumers which have consumed or partially consumed the old index.
GET from <host>:<port>/v2/indexes/
e.g. with the default admin credentials:
GET https://127.0.0.1:2113/v2/indexes/
Authorization: Basic YWRtaW46Y2hhbmdlaXQ=
GET from <host>:<port>/v2/indexes/<index-name>
e.g. with the default admin credentials:
GET https://127.0.0.1:2113/v2/indexes/orders-by-country
Authorization: Basic YWRtaW46Y2hhbmdlaXQ=
User defined indexes can be read and subscribed to via the filtered $all API very similarly to the built in secondary indexes.
The whole index can be consumed by using the stream prefix filter $idx-user-<index-name> e.g. "$idx-user-orders-by-country"
A particular field value can be selected by using the stream prefix filter $idx-user-<index-name>:<field-value> e.g. "$idx-user-orders-by-country:Mauritius".
User defined indexes can be queried in the Query UI (e.g. https://localhost:2113/ui/query)
e.g.
select * from index:orders-by-country where field_country='Mauritius' limit 10
Metrics in updated grafana dashboard soon.
User defined indexes are enabled as part of secondary indexing, which is enabled by default but can be disabled in the server configuration:
SecondaryIndexing:
Enabled: false
Refer to the configuration guide for configuration mechanisms other than YAML.
Note that on a large database the secondary indexes may take a while to build.
The following are improvements we are considering for future versions: