Materialize v0.73

v0.73.0

Sources and sinks

• Private preview. Allow propagating comments in materialized views to the Avro schema of Kafka sinks, as well as manually specifying comments using the new [KEY|VALUE] DOC ON [TYPE|COLUMN] <identifier> connection option.

  Example:

  mzsqlCopy

  CREATE TABLE t (c1 int, c2 text);
  COMMENT ON TABLE t IS 'materialize comment on t';
  COMMENT ON COLUMN t.c2 IS 'materialize comment on t.c2';
  
  CREATE SINK avro_sink
    IN CLUSTER my_io_cluster
    FROM t
    INTO KAFKA CONNECTION kafka_connection (TOPIC 'test_avro_topic')
    KEY (c1)
    FORMAT AVRO USING CONFLUENT SCHEMA REGISTRY CONNECTION csr_connection
    (
      DOC ON TYPE t = 'top-level comment for avro record in both key and value schemas',
      KEY DOC ON COLUMN t.c1 = 'comment on column only in key schema',
      VALUE DOC ON COLUMN t.c1 = 'comment on column only in value schema'
    )
    ENVELOPE UPSERT;
  

  Key schema:

  jsonCopy

  {
    "type": "record",
    "name": "row",
    "doc": "this is a materialized view",
    "fields" : [
      {"name": "a", "type": "string", "doc": "this is column a"},
      {"name": "b", "type": "string"}
    ]
  }{
    "type": "record",
    "name": "row",
    "doc": "top-level comment for avro record in both key and value schemas",
    "fields": [
      {
        "name": "c1",
        "type": [
          "null",
          "int"
        ],
        "doc": "comment on column only in key schema"
      }
    ]
  }
  

  Value schema:

  jsonCopy

  {
    "type": "record",
    "name": "envelope",
    "doc": "top-level comment for avro record in both key and value schemas",
    "fields": [
      {
        "name": "c1",
        "type": [
          "null",
          "int"
        ],
        "doc": "comment on column only in value schema"
      },
      {
        "name": "c2",
        "type": [
          "null",
          "string"
        ],
        "doc": "materialize comment on t.c2"
      }
    ]
  }
  

Bug fixes and other improvements

• Allow the value of the SASL MECHANISM option for Kafka connections to be specified in any case style. Previously, Materialize only accepted uppercase case style (as required by librdkafka).