Back to Risingwave

Demo: Sinking to Delta Lake

integration_tests/deltalake-sink/README.md

2.8.3696 B
Original Source

Demo: Sinking to Delta Lake

In this demo, we will create an append-only source via our datagen source, and sink the data generated from source to the downstream delta lake table stored on minio.

  1. Launch the cluster via docker compose
docker compose up -d
  1. Create a delta lake table on minio
docker compose exec minio-0 mkdir /data/deltalake
docker compose exec spark bash /spark-script/run-sql-file.sh create-table
  1. Execute the SQL queries in sequence:

    • create_source.sql
    • create_sink.sql
  2. Query delta lake table. The following command will query the total count of records.

docker compose exec spark bash /spark-script/run-sql-file.sh query-table