docs/en/introduction/how-it-works.md
SeaTunnel is a distributed multimodal data integration tool with a pluggable architecture. It decouples the connector layer from the execution engine, allowing the same connectors to run on different engines.
┌─────────────────────────────────────────────────────────────┐
│ Job Configuration │
│ (HOCON / SQL / Web UI) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ SeaTunnel Core │
│ (Job Parser, Coordinator, Scheduler) │
└─────────────────────────────────────────────────────────────┘
│
┌─────────────────────┼─────────────────────┐
▼ ▼ ▼
┌───────────────┐ ┌───────────────┐ ┌───────────────┐
│ Source │────▶│ Transform │────▶│ Sink │
│ Connectors │ │ (Optional) │ │ Connectors │
└───────────────┘ └───────────────┘ └───────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ Execution Engine │
│ SeaTunnel Engine (Zeta) / Flink / Spark │
└─────────────────────────────────────────────────────────────┘
Engine-independent API for developing Source, Transform, and Sink connectors.
| Component | Description |
|---|---|
| Source | Reads data from external systems (databases, files, message queues) |
| Transform | Performs data transformations (field mapping, filtering, type conversion) |
| Sink | Writes data to target systems |
| Engine | Best For |
|---|---|
| SeaTunnel Engine (Zeta) | Data synchronization, CDC, low resource usage |
| Apache Flink | Complex stream processing, existing Flink infrastructure |
| Apache Spark | Large-scale batch processing, existing Spark infrastructure |
Translates SeaTunnel's unified API to engine-specific implementations, enabling connector reuse across engines.
Source ──▶ [Split] ──▶ Reader ──▶ Transform ──▶ Writer ──▶ Sink
│ │ │
│ ▼ │
│ Checkpoint/State │
│ │ │
└───────────────────────┴────────────────────────┘
Fault Tolerance
Key Features:
seatunnel/
├── seatunnel-api/ # Core API definitions
├── seatunnel-connectors-v2/ # Source & Sink connectors
├── seatunnel-transforms-v2/ # Transform plugins
├── seatunnel-engine/ # SeaTunnel Engine (Zeta)
├── seatunnel-translation/ # Engine adapters (Flink/Spark)
├── seatunnel-core/ # Job submission & CLI
├── seatunnel-formats/ # Data format handlers
└── seatunnel-e2e/ # End-to-end tests