Flink Common Info - Tdengine

Version History

Flink Connector Version	Major Changes	TDengine TSDB-Enterprise Version
2.1.4	Upgrade the JDBC driver to version 3.7.3. For details, see JDBC Version History	-
2.1.3	Add exception information output for data conversion.	-
2.1.2	Add backtick filtering for written fields.	-
2.1.1	Fix the issue of data binding failure for the same table in Stmt.	-
2.1.0	Fix the issue of writing varchar types from different data sources.	-
2.0.2	The Table Sink supports types such as RowKind.UPDATE_BEFORE, RowKind.UPDATE_AFTER, and RowKind.DELETE.	-
2.0.1	Sink supports writing types from Rowdata implementations.	-
2.0.0	1.Sink supports custom data structure serialization and writing to TDengine.

Supports writing to TDengine database using Table SQL. | 3.3.5.1 and higher | | 1.0.0 | Support Sink function to write data from other sources to TDengine in the future. | 3.3.2.0 and higher |

Exception and error codes

After the task execution fails, check the Flink task execution log to confirm the reason for the failure Please refer to:

Error Code	Description	Suggested Actions
0xa000	connection param error	Connector parameter error.
0xa010	database name configuration error	database name configuration error.
0xa011	table name configuration error	Table name configuration error.
0xa013	value.deserializer parameter not set	No serialization method set.
0xa014	list of column names set incorrectly	List of column names for target table not set.
0x2301	connection already closed	The connection has been closed. Check the connection status or create a new connection to execute the relevant instructions.
0x2302	this operation is NOT supported currently	The current interface is not supported, you can switch to other connection methods.
0x2303	invalid variables	The parameter is invalid. Please check the corresponding interface specification and adjust the parameter type and size.
0x2304	statement is closed	Statement has already been closed. Please check if the statement is closed and reused, or if the connection is working properly.
0x2305	resultSet is closed	The ResultSet has been released. Please check if the ResultSet has been released and used again.
0x230d	parameter index out of range	parameter out of range, please check the reasonable range of the parameter.
0x230e	connection already closed	The connection has been closed. Please check if the connection is closed and used again, or if the connection is working properly.
0x230f	unknown SQL type in TDengine	Please check the Data Type types supported by TDengine.
0x2315	unknown tao type in TDengine	Did the correct TDengine data type be specified when converting TDengine data type to JDBC data type.
0x2319	user is required	Username information is missing when creating a connection.
0x231a	password is required	Password information is missing when creating a connection.
0x231d	can't create connection with server within	Increase connection time by adding the parameter httpConnectTimeout, or check the connection status with taosAdapter.
0x231e	failed to complete the task within the specified time	Increase execution time by adding the parameter messageWaitTimeout, or check the connection with taosAdapter.
0x2352	unsupported encoding	An unsupported character encoding set was specified under the local connection.
0x2353	internal error of database, Please see taoslog for more details	An error occurred while executing prepareStatement on the local connection. Please check the taoslog for problem localization.
0x2354	connection is NULL	Connection has already been closed while executing the command on the local connection. Please check the connection with TDengine.
0x2355	result set is NULL	Local connection to obtain result set, result set exception, please check connection status and retry.
0x2356	invalid num of fields	The meta information obtained from the local connection result set does not match.

Data type mapping

TDengine currently supports timestamp, number, character, and boolean types, and the corresponding type conversions with Flink RowData Type are as follows:

TDengine DataType	Flink RowDataType
TIMESTAMP	TimestampData
INT	Integer
BIGINT	Long
FLOAT	Float
DOUBLE	Double
SMALLINT	Short
TINYINT	Byte
BOOL	Boolean
VARCHAR	StringData
BINARY	StringData
NCHAR	StringData
JSON	StringData
VARBINARY	byte[]
GEOMETRY	byte[]

Instructions for use

Flink Semantic Selection Instructions

The semantic reason for using At-Least-Once is:

TDengine currently does not support transactions and cannot perform frequent checkpoint operations and complex transaction coordination.
Due to TDengine's use of timestamps as primary keys, downstream operators of duplicate data can perform filtering operations to avoid duplicate calculations.
Using At-Least-Once to ensure high data processing performance and low data latency, the setting method is as follows:

Instructions:

java

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.enableCheckpointing(5000);
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.AT_LEAST_ONCE);

If using Maven to manage a project, simply add the following dependencies in pom.xml.

xml

<dependency>
    <groupId>com.taosdata.flink</groupId>
    <artifactId>flink-connector-tdengine</artifactId>
    <version>2.1.4</version>
</dependency>