0.31 Upgrade Guide

Vector's 0.31.0 release includes breaking changes:

1. Removal of various deprecated internal metrics
2. component_received_event_bytes_total and component_sent_event_bytes_total consistently use estimated JSON size of the event

and potentially impactful changes:

1. AWS S3 endpoint path changes

We cover them below to help you upgrade quickly:

Upgrade guide

Breaking changes

Removal of various deprecated internal metrics {#deprecated-internal-metrics}

Over the course of many of the previous releases, we've been working to deprecate the usage of older internal metrics as we worked towards implementing full support for the [Component Specification][component_spec], which dictates the basic metrics that all components, or the basic metrics all components of a specific type, are expected to emit.

We've made enough progress on this work that we've gone ahead and removed many of the deprecated metrics from this release. First, below is a list of all metrics we've removed:

• events_in_total (superceded by component_received_events_total)
• events_out_total (superceded by component_sent_events_total)
• processed_bytes_total (superceded by either component_received_bytes_total or component_sent_bytes_total, more below)
• processed_events_total (superceded by either component_received_events_total or component_sent_events_total, more below)
• processing_errors_total (superceded by component_errors_total)
• events_failed_total (superceded by component_errors_total)

Most of the removals have straightforward replacements, but the processed_-prefixed metrics involve a small amount of logic. For sources, processed_bytes_total is superceded by component_received_bytes_total, and processed_events_total is superceded by component_received_events_total. For sinks, processed_bytes_total is superceded by component_sent_bytes_total, and processed_events_total is superceded by component_sent_events_total.

A small note is that a small number of components still emit some of these metrics, as they provided additional tags and information that is disallowed by the Component Specification. They will be removed in a future version once we can rectify those discrepancies, but they are effectively removed as of this release: you cannot depend on them still existing.

component_received_event_bytes_total and component_sent_event_bytes_total consistently use estimated JSON size of the event {#event_json_size}

Prior to this Version, metrics emitted by Vector were inconsistently measuring the byte size of the events that were being sent and received. These metrics have been updated for all components so they always emit an estimate of the size of the event should it be serialized to JSON.

Measuring the events like this allows a consistent measurement to be applied across all components regardless of how the source or sink serializes the event when connecting to the external service.

Potentially impactful changes

AWS S3 endpoint path changes {#s3-paths}

The handling of AWS S3 endpoints for the aws_s3 source and sink has changed due to an upgrade of the SDK that we use. For S3-compatible APIs like Cloudflare R2 you may find that you need to remove the bucket name from the endpoint, if you have it specified. For example https://xxxxxxxxxxxxxxxxxxxxxxxxxxx.r2.cloudflarestorage.com/<bucket name> should be written as https://xxxxxxxxxxxxxxxxxxxxxxxxxxx.r2.cloudflarestorage.com.