docs/design/2023-04-11-dist-task.md
TiDB is a distributed database that provides high scalability, availability, and performance for database service. This proposal introduces a unified distributed parallel execution framework to support all backend tasks, such as DDL, data import, TTL, analyze and backup&restore, in a distributed manner. The framework may bring three outstanding benefits to TiDB users:
In this section, we define the goals and non-goals for the Phase I of the framework. Future versions will be described in the "Future Work" section.
The Phase I of the framework aims to:
The Phase I of the framework does not aim to:
This section we will describe the Architecture of the framework.
The architecture diagram shows the key modules required for the framework to handle incoming backend tasks in a distributed manner with high performance and good HA capability. These modules are:
This section introduces the data structures and interfaces that define the detail design of the framework.
The system variable tidb_enable_dist_task is used to enable/disable the framework.
mysql> set global tidb_enable_dist_task = ON|OFF;
In implementation, the following two tables are used to represent the global task queue and global subtask queue:
| Name | Type | Memo |
|---|---|---|
| id | BIGINT(20) NOT NULL AUTO_INCREMENT PRIMARY KEY | |
| task_key | VARCHAR(256) NOT NULL | |
| type | VARCHAR(256) NOT NULL | |
| dispatcher_id | VARCHAR(256) | |
| state | VARCHAR(64) NOT NULL | |
| start_time | TIMESTAMP | |
| state_update_time | TIMESTAMP | |
| meta | LONGBLOB | |
| concurrency | INT(11) | |
| step | INT(11) |
| Name | Type | Memo |
|---|---|---|
| id | bigint not null auto_increment primary key | |
| namespace | varchar(256) | |
| task_key | varchar(256) | |
| ddl_physical_tid | bigint(20) | |
| type | int | |
| exec_id | varchar(256) | |
| exec_expired | timestamp | |
| state | varchar(64) not null | |
| checkpoint | longblob not null | |
| start_time | bigint | |
| state_update_time | bigint | |
| meta | longblob |
Dispatcher
// Dispatch defines the interface for operations inside a dispatcher.
type Dispatch interface {
// Start enables dispatching and monitoring mechanisms.
Start()
// GetAllSchedulerIDs gets handles the task's all available instances.
GetAllSchedulerIDs(ctx context.Context, gTaskID int64) ([]string, error)
// Stop stops the dispatcher.
Stop()
}
Scheduler
// InternalScheduler defines the interface of an internal scheduler.
type InternalScheduler interface {
Start()
Stop()
Run(context.Context, *proto.Task) error
Rollback(context.Context, *proto.Task) error
}
// Scheduler defines the interface of a scheduler.
// User should implement this interface to define their own scheduler.
type Scheduler interface {
InitSubtaskExecEnv(context.Context) error
SplitSubtask(subtask []byte) []proto.MinimalTask
CleanupSubtaskExecEnv(context.Context) error
Rollback(context.Context) error
}
The phase I of the unified distributed parallel execution framework is currently planned as an experimental feature. This feature is designed to support the execution of Create index or Add index tasks in a distributed manner.
To enable distributed Add index for DDL tasks, the following system variables and config parameters may be related:
System Variables:
Fast DDL Path:
Config parameter:
Load Data, TTL, Auto-Analyze, Backup, Restore, and more.