Materialize v0.86

v0.86

Sources and sinks

• Add support for handling batched events in the webhook source via the new JSON ARRAY format.

  mzsqlCopy

  CREATE SOURCE webhook_source_json_batch IN CLUSTER my_cluster FROM WEBHOOK
  BODY FORMAT JSON ARRAY
  INCLUDE HEADERS;
  

  Copy

  POST webhook_source_json_batch
  [
    { "event_type": "a" },
    { "event_type": "b" },
    { "event_type": "c" }
  ]
  

  mzsqlCopy

  SELECT COUNT(body) FROM webhook_source_json_batch;
  ----
  3
  

• Decrease memory utilization for unpacking Kafka headers. Use the new mapbuild function to turn all headers exposed via INCLUDE HEADERS into a map, which makes it easier to extract header values.

  mzsqlCopy

  SELECT
      id,
      seller,
      item,
      convert_from(mapbuild(headers)->'client_id', 'utf-8') AS client_id,
      mapbuild(headers)->'encryption_key' AS encryption_key,
  FROM kafka_metadata;
  
   id | seller |        item        | client_id |    encryption_key
  ----+--------+--------------------+-----------+----------------------
    2 |   1592 | Custom Art         |        23 | \x796f75207769736821
    3 |   1411 | City Bar Crawl     |        42 | \x796f75207769736821
  

SQL

• Add support for new SQL functions:

  Function	Description
  mapbuild	Builds a map from a list of records whose fields are two elements, the first of which is text.
  map_agg	Aggregate keys and values (including nulls) as a map.

Bug fixes and other improvements

• Mitigate queue saturation is Kafka sinks {{% gh 24871 %}}.

• Fix a correctness issue with subqueries that referred to ungrouped columns when columns of the same name existed in an outer scope {{% gh 24354 %}}.

• Fix casts from interval to time for large negative intervals {{% gh 24795 %}}.

• Prevent INSERTs with table references in VALUES in transactions {{% gh 24697 %}}.