Prerequisites

Build AMI

Set necessary variables by creating a file ds-ami.pkrvars.hcl and adding the following variables according to your own usage.

shel

cat <<EOF > ds-ami.pkrvars.hcl
aws_access_key = ""
aws_secret_key = ""
aws_region     = "cn-north-1"


ds_ami_name = "my-test-ds-2"

# If you want to use the official distribution tar, just set the `ds_version` to the one you want.
ds_version  = "3.1.1"

# If you want to use a locally built distribution tar, set the `ds_tar` to the tar file location.
ds_tar      = "~/workspace/dolphinscheduler/dolphinscheduler-dist/target/apache-dolphinscheduler-3.1.3-SNAPSHOT-bin.tar.gz"
EOF

Then run the following command to initialize and build a custom AMI.

If you want to use the official distribution tar.

shell

packer init --var-file=ds-ami.pkrvars.hcl packer/ds-ami-official.pkr.hcl
packer build --var-file=ds-ami.pkrvars.hcl packer/ds-ami-official.pkr.hcl

If you want to use the locally built distribution tar.

shell

packer init --var-file=ds-ami.pkrvars.hcl packer/ds-ami-local.pkr.hcl
packer build --var-file=ds-ami.pkrvars.hcl packer/ds-ami-local.pkr.hcl

Create resources

Set necessary variables by creating a file terraform.tfvars and adding the following variables according to your own usage.

Make sure ds_ami_name is the same as the one in ds-ami.pkrvars.hcl above.

tfvars

cat <<EOF > terraform.tfvars
aws_access_key = ""
aws_secret_key = ""
aws_region     = ""

name_prefix = "test-ds-terraform"
ds_ami_name = "my-test-ds"

ds_component_replicas = {
  master            = 1
  worker            = 1
  alert             = 1
  api               = 1
  standalone_server = 0
}
EOF

Then run the following commands to apply necessary resources.

shell

terraform init -var-file=terraform.tfvars
terraform apply -var-file=terraform.tfvars -auto-approve

Open DolphinScheduler UI

shell

open http://$(terraform output -json api_server_instance_public_dns | jq -r '.[0]'):12345/dolphinscheduler/ui

Inputs

Name	Description	Type	Default	Required
<a name="input_aws_access_key"></a> aws_access_key	AWS access key	`string`	n/a	yes
<a name="input_aws_region"></a> aws_region	AWS region	`string`	`"cn-north-1"`	no
<a name="input_aws_secret_key"></a> aws_secret_key	AWS secret key	`string`	n/a	yes
<a name="input_db_instance_class"></a> db_instance_class	Database instance class	`string`	`"db.t3.micro"`	no
<a name="input_db_password"></a> db_password	Database password	`string`	n/a	yes
<a name="input_db_username"></a> db_username	Database username	`string`	`"dolphinscheduler"`	no
<a name="input_ds_ami_name"></a> ds_ami_name	Name of DolphinScheduler AMI	`string`	`"dolphinscheduler-ami"`	no
<a name="input_ds_component_replicas"></a> ds_component_replicas	Replicas of the DolphinScheduler Components	`map(number)`	<pre>{
"alert": 1,
"api": 1,
"master": 1,
"standalone_server": 0,
"worker": 1
}</pre>	no
<a name="input_ds_version"></a> ds_version	DolphinScheduler Version	`string`	`"3.1.1"`	no
<a name="input_name_prefix"></a> name_prefix	Name prefix for all resources	`string`	`"dolphinscheduler"`	no
<a name="input_private_subnet_cidr_blocks"></a> private_subnet_cidr_blocks	Available CIDR blocks for private subnets	`list(string)`	<pre>[
"10.0.101.0/24",
"10.0.102.0/24",
"10.0.103.0/24",
"10.0.104.0/24"
]</pre>	no
<a name="input_public_subnet_cidr_blocks"></a> public_subnet_cidr_blocks	CIDR blocks for the public subnets	`list(string)`	<pre>[
"10.0.1.0/24",
"10.0.2.0/24",
"10.0.3.0/24",
"10.0.4.0/24"
]</pre>	no
<a name="input_s3_bucket_prefix"></a> s3_bucket_prefix	n/a	`string`	`"dolphinscheduler-test-"`	no
<a name="input_subnet_count"></a> subnet_count	Number of subnets	`map(number)`	<pre>{
"private": 2,
"public": 1
}</pre>	no
<a name="input_tags"></a> tags	Tags to apply to all resources	`map(string)`	<pre>{
"Deployment": "Test"
}</pre>	no
<a name="input_vm_associate_public_ip_address"></a> vm_associate_public_ip_address	Associate a public IP address to the EC2 instance	`map(bool)`	<pre>{
"alert": true,
"api": true,
"master": true,
"standalone_server": true,
"worker": true
}</pre>	no
<a name="input_vm_data_volume_size"></a> vm_data_volume_size	Data volume size of the EC2 Instance	`map(number)`	<pre>{
"alert": 10,
"api": 10,
"master": 10,
"standalone_server": 10,
"worker": 10
}</pre>	no
<a name="input_vm_data_volume_type"></a> vm_data_volume_type	Data volume type of the EC2 Instance	`map(string)`	<pre>{
"alert": "gp2",
"api": "gp2",
"master": "gp2",
"standalone_server": "gp2",
"worker": "gp2"
}</pre>	no
<a name="input_vm_instance_type"></a> vm_instance_type	EC2 instance type	`map(string)`	<pre>{
"alert": "t2.micro",
"api": "t2.small",
"master": "t2.medium",
"standalone_server": "t2.small",
"worker": "t2.medium"
}</pre>	no
<a name="input_vm_root_volume_size"></a> vm_root_volume_size	Root Volume size of the EC2 Instance	`map(number)`	<pre>{
"alert": 30,
"api": 30,
"master": 30,
"standalone_server": 30,
"worker": 30
}</pre>	no
<a name="input_vm_root_volume_type"></a> vm_root_volume_type	Root volume type of the EC2 Instance	`map(string)`	<pre>{
"alert": "gp2",
"api": "gp2",
"master": "gp2",
"standalone_server": "gp2",
"worker": "gp2"
}</pre>	no
<a name="input_vpc_cidr"></a> vpc_cidr	CIDR for the VPC	`string`	`"10.0.0.0/16"`	no
<a name="input_zookeeper_connect_string"></a> zookeeper_connect_string	Zookeeper connect string, if empty, will create a single-node zookeeper for demonstration, don't use this in production	`string`	`""`	no

Outputs

Name	Description
<a name="output_alert_server_instance_id"></a> alert_server_instance_id	Instance IDs of alert instances
<a name="output_alert_server_instance_private_ip"></a> alert_server_instance_private_ip	Private IPs of alert instances
<a name="output_alert_server_instance_public_dns"></a> alert_server_instance_public_dns	Public domain names of alert instances
<a name="output_alert_server_instance_public_ip"></a> alert_server_instance_public_ip	Public IPs of alert instances
<a name="output_api_server_instance_id"></a> api_server_instance_id	Instance IDs of api instances
<a name="output_api_server_instance_private_ip"></a> api_server_instance_private_ip	Private IPs of api instances
<a name="output_api_server_instance_public_dns"></a> api_server_instance_public_dns	Public domain names of api instances
<a name="output_api_server_instance_public_ip"></a> api_server_instance_public_ip	Public IPs of api instances
<a name="output_db_address"></a> db_address	Database address
<a name="output_db_name"></a> db_name	Database name
<a name="output_db_port"></a> db_port	Database port
<a name="output_master_server_instance_id"></a> master_server_instance_id	Instance IDs of master instances
<a name="output_master_server_instance_private_ip"></a> master_server_instance_private_ip	Private IPs of master instances
<a name="output_master_server_instance_public_dns"></a> master_server_instance_public_dns	Public domain names of master instances
<a name="output_master_server_instance_public_ip"></a> master_server_instance_public_ip	Public IPs of master instances
<a name="output_s3_access_key"></a> s3_access_key	S3 access key
<a name="output_s3_address"></a> s3_address	S3 address
<a name="output_s3_bucket"></a> s3_bucket	S3 bucket name
<a name="output_s3_regional_domain_name"></a> s3_regional_domain_name	S3 regional domain name
<a name="output_s3_secret"></a> s3_secret	S3 access secret
<a name="output_vm_server_instance_id"></a> vm_server_instance_id	Instance IDs of standalone instances
<a name="output_vm_server_instance_private_ip"></a> vm_server_instance_private_ip	Private IPs of standalone instances
<a name="output_vm_server_instance_public_dns"></a> vm_server_instance_public_dns	Public domain names of standalone instances
<a name="output_vm_server_instance_public_ip"></a> vm_server_instance_public_ip	Public IPs of standalone instances
<a name="output_worker_server_instance_id"></a> worker_server_instance_id	Instance IDs of worker instances
<a name="output_worker_server_instance_private_ip"></a> worker_server_instance_private_ip	Private IPs of worker instances
<a name="output_worker_server_instance_public_dns"></a> worker_server_instance_public_dns	Public domain names of worker instances
<a name="output_worker_server_instance_public_ip"></a> worker_server_instance_public_ip	Public IPs of worker instances
<a name="output_zookeeper_server_instance_id"></a> zookeeper_server_instance_id	Instance IDs of zookeeper instances
<a name="output_zookeeper_server_instance_private_ip"></a> zookeeper_server_instance_private_ip	Private IPs of zookeeper instances
<a name="output_zookeeper_server_instance_public_dns"></a> zookeeper_server_instance_public_dns	Public domain names of zookeeper instances
<a name="output_zookeeper_server_instance_public_ip"></a> zookeeper_server_instance_public_ip	Public IPs of zookeeper instances