docs/en/14-reference/05-connector/20-go.md
import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import RequestId from "../../assets/resources/_request_id.mdx";
driver-go is the official Go language connector for TDengine, implementing the interface of the Go language database/sql package. Go developers can use it to develop applications that access data in the TDengine cluster.
Supports Go 1.14 and above.
| driver-go Version | Major Changes | TDengine Version |
|---|---|---|
| v3.7.8 | WebSocket connections support both two-factor authentication and token authentication. | - |
| v3.7.7 | Fix WebSocket connection crash caused by race condition. | - |
| v3.7.6 | Fix native connection stmt2 binding empty string and empty byte array crash. | - |
| v3.7.5 | Fix native connection failures on Windows. | - |
| v3.7.4 | Support for connection-level time zone settings. | - |
| v3.7.3 | Fix crash when WebSocket connection STMT query results contain decimal data. | - |
| v3.7.2 | support BLOB type. | - |
| v3.7.1 | support IPv6 connection. | - |
| v3.7.0 | support decimal type. | 3.3.6.0 and higher |
| v3.6.0 | stmt2 native interface, DSN supports passwords containing special characters (url.QueryEscape). | 3.3.5.0 and higher |
| v3.5.8 | Fixed null pointer exception. | - |
| v3.5.7 | taosWS and taosRestful support passing request id. | - |
| v3.5.6 | Improved WebSocket query and insert performance. | 3.3.2.0 and higher |
| v3.5.5 | Restful supports skipping SSL certificate check. | - |
| v3.5.4 | Compatible with TDengine 3.3.0.0 tmq raw data. | - |
| v3.5.3 | Refactored taosWS. | - |
| v3.5.2 | WebSocket compression and optimized tmq subscription performance. | 3.2.3.0 and higher |
| v3.5.1 | Native stmt query and geometry type support. | 3.2.1.0 and higher |
| v3.5.0 | Support tmq get assignment and seek offset. | 3.0.5.0 and higher |
| v3.3.1 | Schemaless protocol insert based on WebSocket. | 3.0.4.1 and higher |
| v3.1.0 | Provided Kafka-like subscription API. | - |
| v3.0.4 | Added request id related interfaces. | 3.0.2.2 and higher |
| v3.0.3 | WebSocket-based statement insert. | - |
| v3.0.2 | WebSocket-based data query and insert. | 3.0.1.5 and higher |
| v3.0.1 | WebSocket-based message subscription. | - |
| v3.0.0 | Adapted to TDengine 3.0 query and insert. | 3.0.0.0 and higher |
If it is a TDengine error, you can obtain the error code and error message as follows.
// import "github.com/taosdata/driver-go/v3/errors"
if err != nil {
tError, is := err.(*errors.TaosError)
if is {
fmt.Println("errorCode:", int(tError.Code))
fmt.Println("errorMessage:", tError.ErrStr)
} else {
fmt.Println(err.Error())
}
}
For errors in other TDengine modules, please refer to Error Codes.
| TDengine DataType | Go Type |
|---|---|
| TIMESTAMP | time.Time |
| TINYINT | int8 |
| SMALLINT | int16 |
| INT | int32 |
| BIGINT | int64 |
| TINYINT UNSIGNED | uint8 |
| SMALLINT UNSIGNED | uint16 |
| INT UNSIGNED | uint32 |
| BIGINT UNSIGNED | uint64 |
| FLOAT | float32 |
| DOUBLE | float64 |
| BOOL | bool |
| BINARY | string |
| NCHAR | string |
| JSON | []byte |
| GEOMETRY | []byte |
| VARBINARY | []byte |
| DECIMAL | string |
| BLOB | []byte |
Note: The JSON type is only supported in tags. The GEOMETRY type is binary data in little endian byte order, conforming to the WKB standard. For more details, please refer to Data Types For the WKB standard, please refer to Well-Known Binary (WKB).
For the source code of the example programs, please refer to: Example Programs.
Crashes related to stmt (parameter binding) interfaces in database/sql
REST does not support interfaces related to parameter binding, it is recommended to use db.Exec and db.Query.
Error [0x217] Database not specified or available occurs after executing other statements following the use db statement
In the REST interface, the execution of SQL statements has no context association, and the use db statement will not take effect. See the usage restrictions section above for solutions.
No error with taosSql but error [0x217] Database not specified or available with taosRestful
Since the REST interface is stateless, the use db statement will not take effect. See the usage restrictions section above for solutions.
No significant effect after increasing the readBufferSize parameter
Increasing readBufferSize will reduce the number of syscall calls when fetching results. If the data volume of the query results is not large, modifying this parameter will not bring significant improvement. If this parameter is increased too much, the bottleneck will be in parsing JSON data. To optimize query speed, adjust this value according to the actual situation to achieve the best query effect.
Query efficiency decreases when disableCompression parameter is set to false
When the disableCompression parameter is set to false, the query results will be transmitted after being compressed with gzip, and the data must be decompressed with gzip after being received.
go get command cannot fetch packages, or fetching packages times out
Set the Go proxy go env -w GOPROXY=https://goproxy.cn,direct.
Timezone Handling for Query Results
For WebSocket and native connections, the query results use time.Unix to convert timestamps to the local timezone.
driver-go implements the Go database/sql/driver interface, allowing direct use of the Go database/sql package. It provides two drivers: github.com/taosdata/driver-go/v3/taosSql and github.com/taosdata/driver-go/v3/taosWS corresponding to native connection and WebSocket connection.
The Data Source Name has a generic format, similar to PEAR DB, but without the type prefix (brackets indicate optional):
[username[:password]@][protocol[(address)]]/[dbname][?param1=value1&...¶mN=valueN]
Full form of DSN:
username:password@protocol(address)/dbname?param=value
When the password contains special characters, it needs to be escaped using url.QueryEscape.
When using an IPv6 address (supported in v3.7.1 and above), the address needs to be enclosed in square brackets, for example:
root:taosdata@ws([::1]:6041)/testdb
Import the driver:
import (
"database/sql"
_ "github.com/taosdata/driver-go/v3/taosSql"
)
Use taosSql as driverName and a correct DSN as dataSourceName as follows:
var taosUri = "root:taosdata@tcp(localhost:6030)/"
taos, err := sql.Open("taosSql", taosUri)
Supported DSN parameters:
cfg specifies the taos.cfg directory.cgoThread specifies the number of cgo executions at the same time, default is the number of system cores.cgoAsyncHandlerPoolSize specifies the size of the async function handle, default is 10000.timezone specifies the timezone used for the connection. Both SQL parsing and query results will be converted according to this timezone. Only IANA timezone formats are supported, and special characters need to be encoded. Taking the Shanghai timezone (Asia/Shanghai) as an example: timezone=Asia%2FShanghai.Import the driver:
import (
"database/sql"
_ "github.com/taosdata/driver-go/v3/taosWS"
)
Use taosWS as driverName and use a correct DSN as dataSourceName as follows:
var taosUri = "root:taosdata@ws(localhost:6041)/"
taos, err := sql.Open("taosWS", taosUri)
Supported DSN parameters:
enableCompression whether to send compressed data, default is false which means not sending compressed data, set to true if transferring data using compression.readTimeout the timeout for reading data, default is 5m.writeTimeout the timeout for writing data, default is 10s.timezone specifies the timezone used for the connection. Both SQL parsing and query results will be converted according to this timezone. Only IANA timezone formats are supported, and special characters need to be encoded. Taking the Shanghai timezone (Asia/Shanghai) as an example: timezone=Asia%2FShanghai.bearerToken the token used for authentication.totpCode the TOTP code used for two-factor authentication.The Go driver supports creating connections, returning objects that support the sql/driver standard Connector interface, and also provides the af package, which expands some schema-less writing interfaces.
Interface for creating connections in the database/sql package
func Open(driverName, dataSourceName string) (*DB, error)
database/sql).driverName: Driver name.dataSourceName: Connection parameters DSN.Interface for creating connections in the af package
func Open(host, user, pass, db string, port int) (*Connector, error)
host: Host address.user: Username.pass: Password.db: Database name.port: Port number.func (conn *Connector) SetTimezone(timezone string) error
timezone: Timezone string, using the IANA timezone format, for example: Asia/Shanghai.Interfaces for schema-less writing using native connections in the af package.
func (conn *Connector) InfluxDBInsertLines(lines []string, precision string) error
lines: Data to write.precision: Time precision.func (conn *Connector) OpenTSDBInsertJsonPayload(payload string) error
payload: Data to write.func (conn *Connector) OpenTSDBInsertTelnetLines(lines []string) error
lines: Data to write.ws/schemaless package uses WebSocket schemaless write interface
func (s *Schemaless) Insert(lines string, protocol int, precision string, ttl int, reqID int64) error
lines: Data to be written.protocol: Supported protocols for data writing InfluxDBLineProtocol = 1 OpenTSDBTelnetLineProtocol = 2 OpenTSDBJsonFormatProtocol = 3.precision: Time precision.ttl: Data expiration time, 0 means never expires.reqID: Request ID.The Go driver provides interfaces compliant with database/sql standards, supporting the following features:
SELECT statements).INSERT, UPDATE, DELETE, etc.func (db *DB) Close() error
func (db *DB) Exec(query string, args ...any) (Result, error)
query: Command to execute.args: Command arguments.func (db *DB) Query(query string, args ...any) (*Rows, error)
query: Command to execute.args: Command arguments.func (db *DB) QueryRow(query string, args ...any) *Row
query: Command to execute.args: Command arguments.func (db *DB) ExecContext(ctx context.Context, query string, args ...any) (Result, error)
ctx: Context, use Value to pass request ID for link tracing, key is taos_req_id value is an int64 type.query: Command to execute.args: Command arguments.func (db *DB) QueryContext(ctx context.Context, query string, args ...any) (*Rows, error)
ctx: Context, use Value to pass request ID for link tracing, key is taos_req_id value is an int64 type.query: Command to execute.args: Command arguments.func (db *DB) QueryRowContext(ctx context.Context, query string, args ...any) *Row
ctx: Context, uses Value to pass the request ID for link tracing, key is taos_req_id and value is an int64 type.query: Command to execute.args: Command parameters.The Go driver supports retrieving query result sets and corresponding metadata, providing methods to read metadata and data from the result set.
Retrieve the query result set through the Rows object, which provides the following methods:
func (rs *Rows) Next() bool
func (rs *Rows) Columns() ([]string, error)
func (rs *Rows) Scan(dest ...any) error
dest: Target values.func (rs *Rows) Close() error
func (r *Row) Scan(dest ...any) error
dest: Target values.Retrieve the update result set through the Result object, which provides the following method:
func (dr driverResult) RowsAffected() (int64, error)
Retrieve query result set metadata through the Rows object, providing the following methods:
func (rs *Rows) ColumnTypes() ([]*ColumnType, error)
func (ci *ColumnType) Name() string
func (ci *ColumnType) Length() (length int64, ok bool)
func (ci *ColumnType) ScanType() reflect.Type
func (ci *ColumnType) DatabaseTypeName() string
Prepare allows the use of precompiled SQL statements, which can improve performance and provide the ability for parameterized queries, thereby increasing security.
Use the Prepare method in the Conn interface of sql/driver to prepare a statement bound to this connection, returning a Stmt object for use.
Prepare(query string) (Stmt, error)
query: Statement for parameter binding.func (s *Stmt) Exec(args ...any) (Result, error)
args: Command parameters, Go native types are automatically converted to database types, type mismatches may lose precision, it is recommended to use the same type as the database, time types use int64 or RFC3339Nano formatted strings.func (s *Stmt) Query(args ...any) (*Rows, error)
args: Command arguments, Go native types will automatically convert to database types, type mismatches may lose precision, it is recommended to use the same type as the database, time types use int64 or RFC3339Nano formatted strings.func (s *Stmt) Close() error
The af package provides more interfaces using native connections for parameter binding
func (conn *Connector) Stmt() *Stmt
func (s *Stmt) Prepare(sql string) error
sql: The statement for parameter binding.func (s *Stmt) NumParams() (int, error)
func (s *Stmt) SetTableNameWithTags(tableName string, tags *param.Param) error
tableName: Table name.tags: Tags.func (s *Stmt) SetTableName(tableName string) error
tableName: Table name.func (s *Stmt) BindRow(row *param.Param) error
row: Row data.func (s *Stmt) GetAffectedRows() int
func (s *Stmt) AddBatch() error
func (s *Stmt) Execute() error
func (s *Stmt) UseResult() (driver.Rows, error)
func (s *Stmt) Close() error
From version 3.6.0, the stmt2 interface for binding parameters is provided.
func (conn *Connector) Stmt2(reqID int64, singleTableBindOnce bool) *Stmt2
reqID: Request ID.singleTableBindOnce: Indicates whether a single child table is bound only once during a single execution.func (s *Stmt2) Prepare(sql string) error
sql: The statement for parameter binding.func (s *Stmt2) Bind(params []*stmt.TaosStmt2BindData) error
params: The data to bind.func (s *Stmt2) Execute() error
func (s *Stmt2) GetAffectedRows() int
func (s *Stmt2) UseResult() (driver.Rows, error)
func (s *Stmt2) Close() error
The ws/stmt package provides interfaces for parameter binding via WebSocket
func (c *Connector) Init() (*Stmt, error)
func (s *Stmt) Prepare(sql string) error
sql: The statement for parameter binding.func (s *Stmt) SetTableName(name string) error
name: Table name.func (s *Stmt) SetTags(tags *param.Param, bindType *param.ColumnType)
tags: tag.bindType: Type information.func (s *Stmt) BindParam(params []*param.Param, bindType *param.ColumnType) error
params: Parameters.bindType: Type information.func (s *Stmt) AddBatch() error
func (s *Stmt) Exec() error
func (s *Stmt) GetAffectedRows() int
func (s *Stmt) UseResult() (*Rows, error)
func (s *Stmt) Close() error
Rows row results refer to the Rows interface in the sql/driver package, providing the following interfaces:
func (rs *Rows) Columns() []string
func (rs *Rows) ColumnTypeDatabaseTypeName(i int) string
i: Column index.func (rs *Rows) ColumnTypeLength(i int) (length int64, ok bool)
i: Column index.func (rs *Rows) ColumnTypeScanType(i int) reflect.Type
i: Column index.func (rs *Rows) Next(dest []driver.Value) error
dest: Target values.func (rs *Rows) Close() error
The common/param package provides a parameter binding data structure.
Below are the interfaces for setting parameters by offset:
func NewParam(size int) *Param
size: Number of parameters.func (p *Param) SetBool(offset int, value bool)
offset: Offset (column or tag).value: Boolean value.func (p *Param) SetNull(offset int)
offset: Offset (column or tag).func (p *Param) SetTinyint(offset int, value int)
offset: Offset (column or tag).value: Tinyint value.func (p *Param) SetSmallint(offset int, value int)
offset: Offset (column or tag).value: Smallint value.func (p *Param) SetInt(offset int, value int)
offset: Offset (column or tag).value: Int value.func (p *Param) SetBigint(offset int, value int)
offset: Offset (column or tag).value: Bigint value.func (p *Param) SetUTinyint(offset int, value uint)
offset: Offset (column or tag).value: UTinyint value.func (p *Param) SetUSmallint(offset int, value uint)
offset: Offset (column or tag).value: USmallint value.func (p *Param) SetUInt(offset int, value uint)
offset: Offset (column or tag).value: UInt value.func (p *Param) SetUBigint(offset int, value uint)
offset: Offset (column or tag).value: UBigint value.func (p *Param) SetFloat(offset int, value float32)
offset: Offset (column or tag).value: Float value.func (p *Param) SetDouble(offset int, value float64)
offset: Offset (column or tag).value: Double value.func (p *Param) SetBinary(offset int, value []byte)
offset: Offset (column or tag).value: Binary value.func (p *Param) SetVarBinary(offset int, value []byte)
offset: Offset (column or tag).value: VarBinary value.func (p *Param) SetNchar(offset int, value string)
offset: Offset (column or tag).value: Nchar value.func (p *Param) SetTimestamp(offset int, value time.Time, precision int)
offset: Offset (column or tag).value: Timestamp value.precision: Time precision.func (p *Param) SetJson(offset int, value []byte)
offset: Offset (column or tag).value: Json value.func (p *Param) SetGeometry(offset int, value []byte)
offset: Offset (column or tag).value: Geometry value.Below are the interfaces for setting parameters via chain calls:
func (p *Param) AddBool(value bool) *Param
value: Boolean value.Other types similar to boolean are as follows:
Below are the interfaces for setting column type information:
func NewColumnType(size int) *ColumnType
size: Number of columns.func (c *ColumnType) AddBool() *ColumnType
Other types similar to boolean are as follows:
The Go driver supports data subscription features, providing interfaces for data subscription via native connections and WebSocket connections. Native implementation is in the af/tmq package, WebSocket implementation is in the ws/tmq package.
func NewConsumer(conf *tmq.ConfigMap) (*Consumer, error)
conf: Configuration information.Configuration information is defined as:
type ConfigValue interface{}
type ConfigMap map[string]ConfigValue
Creating a consumer supports the following properties:
ws.url: WebSocket connection URL.ws.message.channelLen: WebSocket message channel buffer length, default 0.ws.message.timeout: WebSocket message timeout, default 5m.ws.message.writeWait: WebSocket message write timeout, default 10s.ws.message.enableCompression: Whether WebSocket compression is enabled, default false.ws.autoReconnect: Whether WebSocket automatically reconnects, default false.ws.reconnectIntervalMs: WebSocket reconnect interval in milliseconds, default 2000.ws.reconnectRetryCount: WebSocket reconnect retry count, default 3.timezone: The timezone used for parsing time-type data in subscription results, using the IANA timezone format, e.g., Asia/Shanghai(supported in v3.7.4 and above).For other parameters, please refer to: Consumer Parameter List, note that starting from version 3.2.0.0 of the TDengine server, the default value of auto.offset.reset in message subscriptions has changed.
func (c *Consumer) Subscribe(topic string, rebalanceCb RebalanceCb) error
topic: Topic.rebalanceCb: Rebalance callback (unused).func (c *Consumer) SubscribeTopics(topics []string, rebalanceCb RebalanceCb) error
topics: List of topics.rebalanceCb: Rebalance callback (unused).func (c *Consumer) Unsubscribe() error
func (c *Consumer) Poll(timeoutMs int) tmq.Event
timeoutMs: Timeout in milliseconds.func (c *Consumer) Commit() ([]tmq.TopicPartition, error)
func (c *Consumer) Assignment() (partitions []tmq.TopicPartition, err error)
func (c *Consumer) Seek(partition tmq.TopicPartition, ignoredTimeoutMs int) error
partition: Partition and offset information.ignoredTimeoutMs: Timeout in milliseconds (unused).func (c *Consumer) Committed(partitions []tmq.TopicPartition, timeoutMs int) (offsets []tmq.TopicPartition, err error)
partitions: List of partitions.timeoutMs: Timeout in milliseconds.func (c *Consumer) CommitOffsets(offsets []tmq.TopicPartition) ([]tmq.TopicPartition, error)
offsets: List of offsets.func (c *Consumer) Position(partitions []tmq.TopicPartition) (offsets []tmq.TopicPartition, err error)
partitions: List of partitions.func (c *Consumer) Close() error
When Poll returns a tmq.Event event, you can obtain the consumption record or error information by determining the type of tmq.Event. When the type is *tmq.DataMessage, you can get the consumption record.
func (m *DataMessage) Topic() string
func (m *DataMessage) DBName() string
func (m *DataMessage) Offset() Offset
func (m *DataMessage) Value() interface{}
[]*tmq.data.Structure of tmq.data:
type Data struct {
TableName string
Data [][]driver.Value
}
When Poll returns a type of tmq.Error, you can use func (e Error) Error() string to get the error information.
When the consumed data type is *tmq.DataMessage, you can obtain partition information from the TopicPartition attribute.
type TopicPartition struct {
Topic *string
Partition int32
Offset Offset
Metadata *string
Error error
}
Topic: Topic.Partition: Partition.Offset: Offset.Metadata: Metadata (unused).Error: Error information.You can use func (p TopicPartition) String() string to get partition information.
The offset information obtained from TopicPartition can be accessed through the Offset attribute. When the offset is -2147467247, it indicates that the offset is not set.
When the consumed data type is *tmq.DataMessage, you can use func (m *DataMessage) Value() interface{} to get the data, the data type is []*tmq.data.