Advanced Security Options - Tdengine

import Enterprise from '../assets/resources/_enterprise.mdx';

In addition to traditional user and permission management, TDengine also offers other security strategies such as IP whitelisting, audit logs, data encryption, etc., which are unique features of TDengine Enterprise. The whitelisting feature was first released in version 3.2.0.0, audit logs in version 3.1.1.0, and database encryption in version 3.3.0.0. It is recommended to use the latest version.

IP Whitelisting

IP whitelisting is a network security technology that allows IT administrators to control "who" can access the system and resources, enhancing the security of database access and preventing external malicious attacks. IP whitelisting works by creating a list of trusted IP addresses, assigning them as unique identifiers to users, and only allowing these IP addresses to access the target server. Please note that user permissions and IP whitelisting are managed separately. Below are the specific methods for configuring IP whitelisting.

The SQL to add an IP whitelist is as follows.

sql

create user test pass password [sysinfo value] [host host_name1[,host_name2]] 
alter user test add host host_name1

The SQL to query the IP whitelist is as follows.

sql

SELECT TEST, ALLOWED_HOST FROM INS_USERS;
SHOW USERS;

The command to delete from the IP whitelist is as follows.

sql

ALTER USER TEST DROP HOST HOST_NAME1

Notes

Both the open-source and enterprise versions can add successfully and can be queried, but the open-source version does not impose any restrictions on IPs.
create user u_write pass 'taosdata1' host 'iprange1','iprange2', multiple ipranges can be added at once, the server will deduplicate, the logic for deduplication requires the ipranges to be exactly the same
By default, 127.0.0.1 is added to the whitelist and can be queried in the whitelist
The IP set of cluster nodes is automatically added to the whitelist, but cannot be queried.
When taosadaper and taosd are not on the same machine, the taosadaper IP needs to be manually added to the taosd whitelist
In a cluster situation, all nodes should have enableWhiteList set the same, either all false or all true, otherwise the cluster cannot start
Whitelist changes take effect within 1s, no more than 2s, each change has a slight impact on transmission performance (one more judgment, which can be ignored), after the change, the impact is negligible, and there is no impact on the cluster during the change process, nor on the clients currently accessing (assuming their IPs are included in the whitelist)
If adding two ip ranges, 192.168.1.1/16 (suppose A), 192.168.1.1/24 (suppose B), strictly speaking, A includes B, but considering the situation too complicated, A and B will not be merged
When deleting, it must match exactly. That is, if added as 192.168.1.1/24, it must also be deleted as 192.168.1.1/24
Only root has the permission to add or delete ip white list for other users
Compatible with previous versions, but does not support rolling back from the current version to a previous version
x.x.x.x/32 and x.x.x.x are considered the same iprange, displayed as x.x.x.x
If the client gets 0.0.0.0/0, it means the whitelist is not enabled
If the whitelist changes, the client will detect it in the heartbeat.
For one user, the maximum number of IPs that can be added is 2048

Audit Logs

TDengine first records and manages user operations, then sends these as audit logs to taosKeeper, which then saves them to any TDengine cluster. Administrators can use audit logs for security monitoring and historical tracing. Enabling and disabling TDengine's audit log feature is very simple, just modify the TDengine configuration file and restart the service. The configuration details for audit logs are as follows.

taosd Configuration

Audit logs are generated by the database service taosd, and its relevant parameters need to be configured in the taos.cfg configuration file, detailed parameters are as follows.

Parameter Name	Parameter Meaning
audit	Whether to enable audit logs, default value is 0. 1 for enabled, 0 for disabled
monitorFqdn	The FQDN of the taosKeeper server that receives the audit logs
monitorPort	The port used by the taosKeeper service that receives the audit logs
monitorCompaction	Whether to compress data when reporting
auditLevel	audit level, different operation will be audited in different level, refer to Operation List section
auditHttps	whether use https when sending audit record to taosKeeper

create audit database

Audit database need to be created after audit is set, and parameter is_audit need to be set.

sql

CREATE DATABASE [IF NOT EXISTS] db_name [database_options] IS_AUDIT 1;

database_options:
    database_option ...

database_option: {
  DURATION value
}

database_option: {
  WAL_LEVEL value
}

database_option: {
  ENCRYPT_ALGORITHM value
}

Additionally, for an audit database:

The default value for keep is 1825 days. If a user specifies keep, it must be greater than 1825 days.
WAL_LEVEL defaults to 2 and cannot be changed by the user.
ENCRYPT_ALGORITHM cannot be specified as None. The user can choose any symmetric encryption algorithm in CBC mode.

Audit databases created before version 3.4.0.0 are incompatible with audit features in version 3.4.0.0 and later. In older versions, the is_audit parameter was not enforced, so there were no mandatory requirements for DURATION, WAL_LEVEL, and ENCRYPT_ALGORITHM. To enable new audit features for an old audit database, it must be dropped and recreated. As a workaround to access data from a pre-3.4.0.0 audit database in a newer version (without new audit features), you can disable auditUseToken by setting it to 0.

taosKeeper Configuration

Configure the related parameters for audit logs in the taosKeeper configuration file keeper.toml, as shown in the table below

Parameter Name	Description
auditDB	The name of the database used to store audit logs, default is "audit". taosKeeper will check if this database exists upon receiving audit logs, and will create it automatically if it does not exist.

Data Format

The format of the reported audit logs is as follows

json

{
    "ts": timestamp,
    "cluster_id": string,
    "user": string,
    "operation": string,
    "db": string,
    "resource": string,
    "client_add": string,
    "details": string,
    "affected_rows": integer,
    "duration": double
}

Table Structure

taosKeeper will automatically create a supertable in the corresponding database based on the reported audit data to store the data. The definition of this supertable is as follows:

sql

create stable operations (ts timestamp, user_name varchar(25), operation varchar(20), db varchar(65), resource varchar(193), client_address varchar(64), details varchar(50000)) tags (cluster_id varchar(64))

Where:

db refers to the database involved in the operation, resource refers to the resource involved.
user_name and operation are data columns, indicating which user performed what operation on the object.
ts is the timestamp column, indicating the time when the operation occurred.
details provide additional details of the operation, which in most cases is the SQL statement executed.
client_address is the client address, including IP and port.

Operation List

The current list of operations recorded in the audit logs and the meanings of each field in each operation are shown in the table below (Note: since the user field, timestamp, and client_add have the same meaning in all operations, they are not included in the table below)

auditLevel = 1 // AUDIT_LEVEL_SYSTEM

Operation	Operation	DB	Resource	Details
create dnode	createDnode	NULL	IP:Port or FQDN:Port	SQL
drop dnode	dropDnode	NULL	dnodeId	SQL
alter dnode	alterDnode	NULL	dnodeId	SQL
create mnode	createMnode	NULL	dnodeId	SQL
drop mnode	dropMnode	NULL	dnodeId	SQL
create qnode	createQnode	NULL	dnodeId	SQL
drop qnode	dropQnode	NULL	dnodeId	SQL
restore dnode	restoreDnode	NULL	dnodeId	SQL

auditLevel = 2 // AUDIT_LEVEL_CLUSTER

Operation	Operation	DB	Resource	Details
alter cluster	alterCluster	NULL	NULL	SQL
balance vgroup leader	balanceVgroupLead	NULL	NULL	SQL
redistribute vgroup	redistributeVgroup	NULL	vgroupId	SQL
balance vgroup	balanceVgroup	NULL	vgroupId	SQL
assign leader	assignLeader	NULL	NULL	SQL
grant privileges	grantPrivileges	NULL	user granted	SQL
revoke privileges	revokePrivileges	NULL	user whose privileges were revoked	SQL
login	login	NULL	NULL	appName
create user	createUser	NULL	username being created	User attribute parameters, (excluding password)
alter user	alterUser	NULL	username being modified	Password change logs the modified parameters and new values (excluding password); other operations log SQL
drop user	dropUser	NULL	username being deleted	SQL
create mount	createMount	mountName	NULL	SQL
drop mount	dropMount	mountName	NULL	SQL
kill retention	killRetention	db name	NULL	SQL
auto trimDB	autoTrimDB	db name	NULL	SQL
create encrypt algr	createEncryptAlgr	NULL	algorithmId	SQL
drop encrypt algr	dropEncryptAlgr	NULL	algorithmId	SQL

auditLevel = 3 // AUDIT_LEVEL_DATABASE

Operation	Operation	DB	Resource	Details
create database	createDB	db name	NULL	SQL
alter database	alterDB	db name	NULL	SQL
drop database	dropDB	db name	NULL	SQL
compact database	compact	database name	NULL	SQL
kill compact	killCompact	db name	NULL	SQL
create stable	createStb	db name	stable name	SQL
alter stable	alterStb	db name	stable name	SQL
drop stable	dropStb	db name	stable name	SQL
create stream	createStream	NULL	name of the created stream	SQL
drop stream	dropStream	NULL	name of the deleted stream	SQL
recalc stream	recalcStream	streamName	recalcName	SQL
create topic	createTopic	topic's DB	name of the created topic	SQL
drop topic	dropTopic	topic's DB	name of the deleted topic	SQL
reload topic	reloadTopic	NULL	name of the deleted topic	SQL
create Rsma	createRsma	Rsma name	NULL	SQL
alter Rsma	alterRsma	Rsma name	Table name	SQL
drop Rsma	dropRsma	Rsma name	NULL	SQL
create View	createView	Db name	NULL	SQL
drop View	dropView	Db name	view name	SQL

auditLevel = 4 // AUDIT_LEVEL_CHILDTABLE

Operation	Operation	DB	Resource	Details
create table	createTable	db name	table name	SQL
drop table	dropTable	db name	table name	SQL

View Audit Logs

After both taosd and taosKeeper are correctly configured and started, as the system continues to operate, various operations (as shown in the table above) will be recorded and reported in real-time. Users can log in to taosExplorer, click on "System Management" > "Audit" page to view the audit logs; they can also directly query the relevant databases and tables in the TDengine CLI.

Storage Security

TDengine supports Transparent Data Encryption (TDE), which encrypts static data files to prevent potential attackers from bypassing the database and directly reading sensitive information from the file system. The database access program is completely unaware, and applications do not need to make any modifications or compilations to work with encrypted databases. Storage security features support encryption algorithms such as SM4 and AES, adopt a hierarchical key management mechanism, and provide comprehensive key backup and recovery capabilities.

Key Hierarchy

TDengine adopts a hierarchical key management system, including the following key types:

SVR_KEY (Server Master Key): Used to encrypt database master keys and system-level information, bound to machine hardware to prevent cross-machine migration
DB_KEY (Database Master Key): Used to encrypt various derived keys
CFG_KEY (Configuration Encryption Key): Dedicated to encrypting configuration files, cannot be changed once generated
META_KEY (Metadata Encryption Key): Used to encrypt metadata files, cannot be changed once generated
DATA_KEY (Time-Series Data Encryption Key): Used to encrypt time-series data files and related logs, cannot be changed once generated

All keys use machine code binding technology. When data files are copied to another machine, the change in machine code prevents access to the keys, naturally preventing access to the data files. After encryption, the data compression ratio remains unchanged, and write and query performance decrease slightly.

Note: The storage security feature requires obtaining machine code, which may not be available in some virtualized environments (such as certain container environments).

Generate Keys

Use the taosk tool to generate keys. Basic syntax:

shell

taosk -c /etc/taos \
  --set-cfg-algorithm sm4 \
  --set-meta-algorithm sm4 \
  --encrypt-server [svr_key] \
  --encrypt-database [db_key] \
  --encrypt-config \
  --encrypt-metadata \
  --encrypt-data [data_key]

Parameter descriptions:

-c: Specify configuration file path, default /etc/taos
--set-cfg-algorithm: Set configuration file encryption algorithm (sm4 or aes), default sm4
--set-meta-algorithm: Set metadata encryption algorithm (sm4 or aes), default sm4
--encrypt-server: Enable server encryption, optionally specify SVR_KEY, auto-generate if not specified
--encrypt-database: Enable database encryption, optionally specify DB_KEY, auto-generate if not specified
--encrypt-config: Enable configuration file encryption, auto-generate CFG_KEY
--encrypt-metadata: Enable metadata encryption, auto-generate META_KEY
--encrypt-data: Enable data file encryption, optionally specify DATA_KEY, auto-generate if not specified

Examples:

shell

# Generate all keys using default SM4 algorithm
taosk -c /etc/taos \
  --encrypt-server \
  --encrypt-database \
  --encrypt-config \
  --encrypt-metadata \
  --encrypt-data

# Specify keys and use different algorithms
taosk -c /etc/taos \
  --set-cfg-algorithm aes \
  --set-meta-algorithm sm4 \
  --encrypt-server mysvr123 \
  --encrypt-database mydb4567 \
  --encrypt-config \
  --encrypt-metadata \
  --encrypt-data oldkey123

After keys are generated, they will be saved in the following locations:

{dataDir}/dnode/config/master.bin: Stores SVR_KEY and DB_KEY
{dataDir}/dnode/config/derived.bin: Stores CFG_KEY, META_KEY, and DATA_KEY

View Encryption Algorithms

Users can view all built-in available encryption algorithms.

sql

show encrypt_algorithms;
id      |          algorithm_id          |              name              |              desc              |              type              |             source             |         ossl_algr_name         |
1 | SM4-CBC                        | SM4                            | SM4 symmetric encryption       | Symmetric Ciphers CBC mode     | build-in                       | SM4-CBC:SM4                    |
2 | AES-128-CBC                    | AES                            | AES symmetric encryption       | Symmetric Ciphers CBC mode     | build-in                       | AES-128-CBC                    |

id: Numeric identifier of the algorithm; built-in algorithms start from 1, custom algorithms start from 101
algorithm_id: Global unique identifier of the algorithm
name: Algorithm name
desc: Description of the algorithm
type: Algorithm type, including: Symmetric Ciphers CBC mode - symmetric encryption algorithm in CBC mode, used for database encryption; Asymmetric Cipher - asymmetric encryption algorithm; Digests - hash algorithm
source: Algorithm source, including: built-in - built-in algorithm; customized - user-defined algorithm
ossl_algr_name: Algorithm name in OpenSSL; for built-in algorithms, it's the name in the default provider, refer to OSSL_PROVIDER-default for custom algorithms, it's user-defined in the program

Add Customized Algorithms

Users can add their own custom algorithms.

sql

create encrypt_algr 'vigenere' name 'vigenere' desc 'my custom algr' type 'Symmetric Ciphers CBC mode' ossl_algr_name 'vigenere';

For user-defined algorithms, users need to develop an SO library according to the interface. When taosd starts, it will load this SO library, and after the SO library is loaded, the user-defined algorithms can be used. In this SO library, users can include multiple algorithms, each with its own naming, specified through the ossl_algr_name field in create encrypt_algr. The custom algorithm interface adopts the OpenSSL implementation and follows the OpenSSL interface definition. For the OpenSSL interface definition, refer to OpenSSL provider. The parameter encryptExtDir specifies the path to the custom algorithm library SO file. Currently, only a single file can be loaded.

Delete Customized Algorithms

Users can delete their own custom algorithms.

sql

drop encrypt_algr 'vigenere';

Before deleting a custom algorithm, you must ensure that the algorithm is not in use. For example, databases using that algorithm must be deleted in advance.

Create Encrypted Database

TDengine supports specifying encryption algorithms when creating databases, SQL as follows:

sql

CREATE DATABASE [IF NOT EXISTS] db_name [database_options]
database_options:
  database_option ...
database_option: {
  ENCRYPT_ALGORITHM {'none' | 'SM4-CBC' | 'AES-128-CBC'}
}

The main parameters are explained as follows.

encrypt_algorithm: Specifies the encryption algorithm used for the data. The default is none, meaning no encryption is used. If you want to set up encrypted data, you need to specify the algorithm_id from show encrypt_algorithms, and the type must be a symmetric cipher in CBC mode.

Examples:

sql

-- Create database using SM4 encryption
CREATE DATABASE db1 ENCRYPT_ALGORITHM 'SM4-CBC';

-- Create database using AES encryption
CREATE DATABASE db2 ENCRYPT_ALGORITHM 'AES-128-CBC';

-- Create unencrypted database
CREATE DATABASE db3;

Notes:

The ENCRYPT_ALGORITHM of a database cannot be modified after creation
Before creating an encrypted database, you must first generate keys containing DATA_KEY using taosk

View Encryption Status

View System Encryption Status

View overall encryption status through system tables:

sql

SELECT * FROM information_schema.ins_encrypt_status;
         encrypt_scope          |           algorithm            |       status       |
=======================================================================================
 config                         | AES-128-CBC                    | enabled            |
 metadata                       | AES-128-CBC                    | enabled            |
 data                           | SM4-CBC:SM4                    | enabled            |

Field descriptions:

encrypt_scope: Encryption scope (config, metadata, data)
algorithm: Encryption algorithm used
status: Encryption status (enabled or disabled)

View Database Encryption Configuration

View encryption algorithms for each database through system tables:

sql

SELECT name,`encrypt_algorithm` FROM information_schema.ins_databases;
              name              | encrypt_algorithm  |
======================================================
 information_schema             | NULL               |
 performance_schema             | NULL               |
 db2                            | AES-128-CBC        |
 db1                            | SM4-CBC            |

Update Keys

You can update SVR_KEY and DB_KEY through the taosk tool or SQL commands (other keys cannot be changed once generated).

Update Using taosk

shell

# Stop taosd
systemctl stop taosd

# Update keys
taosk -c /etc/taos --update-svrkey new_svr_key --update-dbkey new_db_key

# Start taosd
systemctl start taosd

Update Using SQL

While taosd is running, you can update keys through SQL (requires administrator privileges):

sql

-- Update SVR_KEY
ALTER SYSTEM SET SVR_KEY 'new_svr_key';

-- Update DB_KEY
ALTER SYSTEM SET DB_KEY 'new_db_key';

Key Backup and Recovery

Backup Keys

Use taosk to create portable backups (without machine code binding, can be restored on other machines):

shell

taosk -c /etc/taos --backup --svr-key your_svr_key

The backup file will be generated in the {dataDir}/dnode/config/ directory with the filename format master.bin.backup.{timestamp}.

Note: Backup requires providing the correct SVR_KEY for verification.

Restore Keys

Restore keys from backup on a new machine:

shell

taosk -c /etc/taos \
  --restore \
  --machine-code /path/to/backup_file \
  --svr-key your_svr_key

The restore operation will bind the keys to the current machine's machine code.

Key Expiration Policy

You can set key expiration time and policy through SQL (requires administrator privileges):

sql

ALTER SYSTEM SET KEY_EXPIRATION 90 DAYS STRATEGY 'ALARM';

Policy options:

ALARM: Outputs alarm information in logs when keys expire

Configuration File Behavior Changes

After enabling storage security, TDengine's configuration management changes as follows:

Configuration only effective on first startup: After the system's initial startup, subsequent modifications to the taos.cfg file will not take effect
Modify configuration through SQL: All configuration modifications must be executed through SQL commands, requiring administrator privileges

Configuration modification example:

sql

ALTER DNODE 1 'debugFlag' '143';

Transparent Encryption Scope

After enabling storage security, TDengine will transparently encrypt the following files:

Configuration file encryption (requires CFG_KEY):
- dnode.info, dnode.json
- mnode.json, raft_config.json, raft_store.json
- vnodes.json, vnode.json, etc.
Metadata file encryption (requires META_KEY):
- mnode SDB
- snode checkpoint files
Data file encryption (requires DATA_KEY):
- TSDB data files
- WAL write-ahead log files
- STT files
- TDB, BSE and other index files

All encrypted configuration files will contain a plaintext identifier header ("tdEncrypt") at the beginning to mark files as encrypted and avoid duplicate encryption.

Version Compatibility

Upgrading from versions that do not support storage security to new versions can run normally
Encrypted databases from historical versions can be made compatible by specifying DATA_KEY
After enabling storage security, cannot rollback to historical versions that do not support storage security