aggregateWindow() function

aggregateWindow() downsamples data by grouping data into fixed windows of time and applying an aggregate or selector function to each window.

All columns not in the group key other than the specified column are dropped from output tables. This includes _time. aggregateWindow() uses the timeSrc and timeDst parameters to assign a time to the aggregate value.

aggregateWindow() requires _start and _stop columns in input data. Use range() to assign _start and _stop values.

This function is intended to be used when timeColumn (_time by default) is not in the group key. If timeColumn is in the group key, resulting output is confusing and generally not useful.

Downsample by calendar months and years

every, period, and offset parameters support all valid duration units, including calendar months (1mo) and years (1y).

Downsample by week

When windowing by week (1w), weeks are determined using the Unix epoch (1970-01-01T00:00:00Z UTC). The Unix epoch was on a Thursday, so all calculated weeks begin on Thursday.

Function type signature

(
    <-tables: stream[D],
    every: duration,
    fn: (<-: stream[B], column: A) => stream[C],
    ?column: A,
    ?createEmpty: bool,
    ?location: {zone: string, offset: duration},
    ?offset: duration,
    ?period: duration,
    ?timeDst: string,
    ?timeSrc: string,
) => stream[E] where B: Record, C: Record, D: Record, E: Record

{{% caption %}} For more information, see Function type signatures. {{% /caption %}}

Parameters

every

({{< req >}}) Duration of time between windows.

period

Duration of windows. Default is the every value.

period can be negative, indicating the start and stop boundaries are reversed.

offset

Duration to shift the window boundaries by. Default is 0s.

offset can be negative, indicating that the offset goes backwards in time.

fn

({{< req >}}) Aggregate or selector function to apply to each time window.

location

Location used to determine timezone. Default is the location option.

column

Column to operate on.

timeSrc

Column to use as the source of the new time value for aggregate values. Default is _stop.

timeDst

Column to store time values for aggregate values in. Default is _time.

createEmpty

Create empty tables for empty window. Default is true.

Note: When using createEmpty: true, aggregate functions return empty tables, but selector functions do not. By design, selectors drop empty tables.

tables

Input data. Default is piped-forward data (<-).

Examples

• Use an aggregate function with default parameters
• Specify parameters of the aggregate function
• Downsample by calendar month
• Downsample by calendar week starting on Monday

Use an aggregate function with default parameters

data
    |> aggregateWindow(every: 20s, fn: mean)

{{< expand-wrapper >}} {{% expand "View example input and output" %}}

Input data

Output data

{{% /expand %}} {{< /expand-wrapper >}}

Specify parameters of the aggregate function

To use functions that don’t provide defaults for required parameters with aggregateWindow(), define an anonymous function with column and tables parameters that pipes-forward tables into the aggregate or selector function with all required parameters defined:

data
    |> aggregateWindow(
        column: "_value",
        every: 20s,
        fn: (column, tables=<-) => tables |> quantile(q: 0.99, column: column),
    )

{{< expand-wrapper >}} {{% expand "View example input and output" %}}

Input data

Output data

{{% /expand %}} {{< /expand-wrapper >}}

Downsample by calendar month

data
    |> aggregateWindow(every: 1mo, fn: mean)

{{< expand-wrapper >}} {{% expand "View example input and output" %}}

Input data

Output data

{{% /expand %}} {{< /expand-wrapper >}}

Downsample by calendar week starting on Monday

Flux increments weeks from the Unix epoch, which was a Thursday. Because of this, by default, all 1w windows begin on Thursday. Use the offset parameter to shift the start of weekly windows to the desired day of the week.

data
    |> aggregateWindow(every: 1w, offset: -3d, fn: mean)

{{< expand-wrapper >}} {{% expand "View example input and output" %}}

Input data

Output data

{{% /expand %}} {{< /expand-wrapper >}}