0.51 Upgrade Guide

Vector breaking changes

1. Environment variable interpolation security hardening
2. Internal topology logs field name change
3. Utilization metric precision cap
4. Legacy file source fingerprint support dropped
5. macOS x86_64 builds discontinued

Vector upgrade guide

Environment variable interpolation security hardening {#env-var-interpolation}

Environment variable interpolation in configuration files now rejects values containing newline characters. This prevents configuration injection attacks where environment variables could inject malicious multi-line configurations.

Action needed

If you need to inject multi-line configuration blocks, use a config pre-processing tool like envsubst instead or update your configuration files so that they don't rely on block injections.

Internal topology logs field name change {#topology-logs-field}

Vector's internal topology debug! and trace! logs now use the component_id field name instead of component or key.

Action needed

If you are monitoring or filtering Vector's internal logs based on these field names, update your queries to use component_id.

Utilization metric precision cap {#utilization-metric-precision}

The utilization metric is now capped at 4 decimal digit precision.

Action needed

No action required. This change improves metric consistency and readability.

Legacy file source fingerprint support dropped {#legacy-fingerprints}

• Dropped support for file source legacy checkpoints stored in the checkpoints folder (Vector < 0.11) which is located inside the data_dir.
• Removed the legacy checkpoint checksum format (Vector < 0.15).
• The intentionally hidden fingerprint.bytes option was also removed.

You can stop reading this section of the upgrade guide if you

• have started using the file source on or after version 0.15, or
• have cleared your data_dir on or after version 0.15, or
• don't care about the file positions and don't care about current state of your checkpoints, meaning you accept that files could be read from the beginning again after the upgrade.
  • Vector will re-read all files from the beginning if/when any checkpoints.json files nested inside data_dir fail to load due to legacy/corrupted data.

You are only affected if your Vector version is:

1. >= 0.11 and < 0.15, then your checkpoints are using the legacy checkpoint checksum CRC format.
2. >= 0.11 and < 0.15, then the checksum key is present under checkpoints.fingerprint in your checkpoints.json (instead of first_lines_checksum).
3. or ever was < 0.11 and you are using the legacy checkpoints folder and/or the unknown key is present under checkpoints.fingerprint in any checkpoints.json files nested inside data_dir.

If you are affected by #1 or #2

Run the file source with any version of Vector >= 0.15, but strictly before 0.51 and the checkpoints should be automatically updated. For example, if you’re on Vector 0.10 and want to upgrade, keep upgrading Vector until 0.14 and Vector will automatically convert your checkpoints. When upgrading, we recommend stepping through minor versions as these can each contain breaking changes while Vector is pre-1.0. These breaking changes are noted in their respective upgrade guides.

Odds are the file source automatically converted checkpoints to the new format if you are using a recent version and you are not affected by this at all.

If you are affected by #3

You should manually delete the unknown checkpoint records from all checkpoints.json files nested inside data_dir and then follow the upgrade guide for #1 and #2. If you were using a recent version of Vector and unknown was present it wasn't being used anyways.

macOS x86_64 builds discontinued {#macos-x86-builds}

Following this announcement, we will no longer publish x86_64-apple-darwin builds. This means we will not be validating if Vector builds and works correctly on that platform.

Action needed

If you are running Vector on macOS with Intel processors (x86_64), consider migrating to ARM-based macOS hardware or use alternative deployment platforms.

VRL breaking changes

{{< info >}} VRL version 0.28.0 includes a breaking change to the find function. {{< /info >}}

VRL find function return value change {#vrl-find-function}

The return value of the find function has been changed from -1 to null when there is no match. This improves consistency with VRL's type system and idiomatic null handling.

Previous Behavior

result = find("hello world", "xyz")
# result = -1

New Behavior

result = find("hello world", "xyz")
# result = null

Action needed

Update any code that checks for -1 as the return value of find to check for null instead: