0.25 Upgrade Guide

Vector's 0.25.0 release includes breaking changes:

1. Removal of VRL's modulo operator
2. Removal of the new_relic_logs sink
3. internal_metrics defaults to setting host tag
4. Removal of the vector source and sink v1 protocol
5. Removal of shutdown_timeout_secs from vector source
6. Change in the loki sink's request encoding

and deprecations:

1. Deprecation of VRL metadata functions
2. Deprecation of endpoint option in Elasticsearch sink
3. Deprecation of the Lua version 1 API

We cover them below to help you upgrade quickly:

Upgrade guide

Breaking changes

Removal of VRL's modulo operator {#modulo-removal}

VRL's modulo operator (%) was deprecated in 0.24.0 and has now been removed. The mod function can be used instead.

Before:

5 % 2 == 1

After:

mod(5, 2) == 1

Removal of the new_relic_logs sink {#new-relic-logs-sink-removal}

In 0.24.0, we announced that the new_relic_logs sink had been deprecated, and that users should prefer to use the new_relic sink instead, which has support not only for logs, but also metrics and traces. Switching from new_relic_logs to new_relic is not as simple as just changing the sink type field, but it only involves a few small changes which we'll cover below.

First, you'll have to change the type from new_relic_logs to new_relic. As well, you'll have to set a field, api, to specify that you're sending logs. The value of this field should "logs".

Lastly, you'll need to tweak the credentials used to specify your New Relic account credentials. With the new_relic_logs sink, you had the option to specify either your Insights insert key (insert_key) or your License key (license_key). With the new_relic sink, you can no longer use an Insights insert key, and must use a license key. Additionally, the account ID must also now be specified.

All put together, here's an example of converting from a new_relic_logs sink configuration over to the new_relic sink configuration:

sinks:
  new_relic_logs:
    type: "new_relic_logs"
    license_key: "xxxx"

  new_relic:
    type: "new_relic"
    license_key: "xxxx"
    account_id: "yyyy"
    api: "logs"

internal_metrics defaults to setting host tag {#internal-metrics-host-tag}

The internal_metrics source now defaults to setting the host tag on emitted metrics. This behavior avoids issues when deploying multiple Vector instances publishing their metrics to the same sink without opting into this tag via tags.host_key. The default key for this is the configured log_schema.host_key (which defaults to host).

This behavior can be opted out of by setting tags.host_key = "" on the internal_metrics source.

Removal of the vector source and sink v1 protocol {#vector-v1-removal}

The original v1 protocol of the vector source and sink was marked as deprecated in version 0.20.0. This version removes support for the deprecated protocol and its configuration. The version field is still accepted in configurations, but only version numbered 2 is accepted.

Removal of shutdown_timeout_secs from vector source {#shutdown-timeout-secs}

The shutdown_timeout_secs config for the vector v2 source didn't actually do anything, so it was removed.

Change in the loki sink's request encoding {#loki-request-encoding}

Vector now defaults to sending requests to Loki as snappy-compressed protobuf, rather than JSON. The old behavior can be kept by setting the compression option to either none or gzip, with none being the previous default.

Deprecation Notices

Deprecation of VRL metadata functions {#metadata-function-deprecation}

Vector 0.25.0 has introduced a new metadata path syntax available in VRL which points directly to event metadata. This means the metadata functions are no longer necessary and will be removed in the future.

Deprecation of endpoint option in Elasticsearch sink {#elasticsearch-endpoint-deprecation}

Vector 0.25.0 has introduced distribution of events to multiple endpoints for Elasticsearch sink. In order to enable this distribution, a new endpoints setting has been introduced which configures one or more destinations to which to distribute the events. The existing endpoint setting is now deprecated and will be removed in a future version.

Deprecation of the Lua version 1 API {#lua-v1-api-deprecation}

Vector 0.9.0 introduced the version 2 API for the lua transform. This API has long been considered fully mature, obviating the need to maintain the older API. Additionally, the older API has no support for data types other than logs. The older version 1 API is now deprecated and will be removed in a future version.

For example, the following partial configuration:

transforms:
  example:
    type: "lua"
    version: 1
    source: |
      event["a"] = "some value"
      event["b"] = nil

would need to be converted to the following:

transforms:
  example:
    type: "lua"
    version: 2
    hooks:
      process: |
        function (event, emit)
          event.log.a = "some value"
          event.log.b = nil
          emit(event)
        end