ContextQMD
Back to Seatunnel

Databend

docs/en/connectors/source/Databend.md

2.3.133.6 KB
Original Source

import ChangeLog from '../changelog/connector-databend.md';

Databend

Databend source connector

Supported Engines

Spark

Flink

SeaTunnel Zeta

Key Features

Description

A source connector for reading data from Databend.

Dependencies

  1. You need to download the Databend JDBC driver jar package and add it to the directory ${SEATUNNEL_HOME}/plugins/.

For SeaTunnel Zeta

  1. You need to download the Databend JDBC driver jar package and add it to the directory ${SEATUNNEL_HOME}/lib/.

Supported Data Source Information

Data SourceSupported VersionDriverURLMaven
Databend1.2.x and above---

Data Type Mapping

Databend Data TypeSeaTunnel Data Type
BOOLEANBOOLEAN
TINYINTTINYINT
SMALLINTSMALLINT
INTINT
BIGINTBIGINT
FLOATFLOAT
DOUBLEDOUBLE
DECIMALDECIMAL
STRINGSTRING
VARCHARSTRING
CHARSTRING
TIMESTAMPTIMESTAMP
DATEDATE
TIMETIME
BINARYBYTES

Source Options

Basic Configuration:

NameTypeRequiredDefault ValueDescription
urlStringYes-Databend JDBC connection URL
usernameStringYes-Databend database username
passwordStringYes-Databend database password
databaseStringNo-Databend database name, defaults to the database name specified in the connection URL
tableStringNo-Databend table name
queryStringNo-Databend query statement, if set will override database and table settings
fetch_sizeIntegerNo0Number of records to fetch from database at once, set to 0 to use JDBC driver default value
jdbc_configMapNo-Additional JDBC connection configuration, such as load balancing strategies

Table List Configuration:

NameTypeRequiredDefault ValueDescription
databaseStringYes-Database name
tableStringYes-Table name
queryStringNo-Custom query statement
fetch_sizeIntegerNo0Number of records to fetch from database at once

Note: When this configuration corresponds to a single table, you can flatten the configuration items from table_list to the outer level.

Task Examples

Single Table Reading

hocon
env {
  parallelism = 2
  job.mode = "BATCH"
}

source {
  Databend {
    url = "jdbc:databend://localhost:8000"
    username = "root"
    password = ""
    database = "default"
    table = "users"
  }
}

sink {
  Console {}
}

Using Custom Query

hocon
source {
  Databend {
    url = "jdbc:databend://localhost:8000"
    username = "root"
    password = ""
    query = "SELECT id, name, age FROM default.users WHERE age > 18"
  }
}

Changelog

<ChangeLog />