Back to Conductor

App Configuration

docs/documentation/configuration/appconf.md

2019-04-12-130018.9 KB
Original Source

App Configuration

The Conductor application server offers extensive customization options to optimize its operation for specific environments.

These configuration parameters allow fine-tuning of various aspects of the server's behavior, performance, and integration capabilities. All of these parameters are grouped under the conductor.app namespace.

Configuration

FieldTypeDescriptionNotes
stackStringName of the stack within which the app is running. e.g. devint, testintg, staging, prod etc.Default is "test"
appIdStringThe ID with which the app has been registered. e.g. conductor, myAppDefault is "conductor"
executorServiceMaxThreadCountintThe maximum number of threads to be allocated to the executor service threadpool. e.g. 50Default is 50
workflowOffsetTimeoutDurationThe timeout duration to set when a workflow is pushed to the decider queue. Example: 30s or 1mDefault is 30 seconds
maxPostponeDurationSecondsDurationThe maximum timeout duration to set when a workflow with running task is pushed to the decider queue. Example: 30m or 1hDefault is 3600 seconds
sweeperThreadCountintThe number of threads to use for background sweeping on active workflows. Example: 8 if there are 4 processors (2x4)Default is 2 times the number of available processors
sweeperWorkflowPollTimeoutDurationThe timeout for polling workflows to be swept. Example: 2000ms or 2sDefault is 2000 milliseconds
eventProcessorThreadCountintThe number of threads to configure the threadpool in the event processor. Example: 4Default is 2
eventMessageIndexingEnabledbooleanWhether to enable indexing of messages within event payloads. Example: true or falseDefault is true
eventExecutionIndexingEnabledbooleanWhether to enable indexing of event execution results. Example: true or falseDefault is true
workflowExecutionLockEnabledbooleanWhether to enable the workflow execution lock. Example: true or falseDefault is false
lockLeaseTimeDurationThe time for which the lock is leased. Example: 60000ms or 1mDefault is 60000 milliseconds
lockTimeToTryDurationThe time for which the thread will block in an attempt to acquire the lock. Example: 500ms or 1sDefault is 500 milliseconds
activeWorkerLastPollTimeoutDurationThe time to consider if a worker is actively polling for a task. Example: 10sDefault is 10 seconds
taskExecutionPostponeDurationDurationThe time for which a task execution will be postponed if rate-limited or concurrent execution limited. Example: 60sDefault is 60 seconds
taskIndexingEnabledbooleanWhether to enable indexing of tasks. Example: true or falseDefault is true
taskExecLogIndexingEnabledbooleanWhether to enable indexing of task execution logs. Example: true or falseDefault is true
asyncIndexingEnabledbooleanWhether to enable asynchronous indexing to Elasticsearch. Example: true or falseDefault is false
systemTaskWorkerThreadCountintThe number of threads in the threadpool for system task workers. Example: 8 if there are 4 processors (2x4)Default is 2 times the number of available processors
systemTaskMaxPollCountintThe maximum number of threads to be polled within the threadpool for system task workers. Example: 8Default is equal to systemTaskWorkerThreadCount
systemTaskWorkerCallbackDurationDurationThe interval after which a system task will be checked by the system task worker for completion. Example: 30sDefault is 30 seconds
systemTaskWorkerPollIntervalDurationThe interval at which system task queues will be polled by system task workers. Example: 50msDefault is 50 milliseconds
systemTaskWorkerExecutionNamespaceStringThe namespace for the system task workers to provide instance-level isolation. Example: namespace1, namespace2Default is an empty string
isolatedSystemTaskWorkerThreadCountintThe number of threads to be used within the threadpool for system task workers in each isolation group. Example: 4Default is 1
asyncUpdateShortRunningWorkflowDurationDurationThe duration of workflow execution qualifying as short-running when async indexing to Elasticsearch is enabled. Example: 30sDefault is 30 seconds
asyncUpdateDelayDurationThe delay with which short-running workflows will be updated in Elasticsearch when async indexing is enabled. Example: 60sDefault is 60 seconds
ownerEmailMandatorybooleanWhether to validate the owner email field as mandatory within workflow and task definitions. Example: true or falseDefault is true
eventQueueSchedulerPollThreadCountintThe number of threads used in the Scheduler for polling events from multiple event queues. Example: 8 if there are 4 processors (2x4)Default is equal to the number of available processors
eventQueuePollIntervalDurationThe time interval at which the default event queues will be polled. Example: 100msDefault is 100 milliseconds
eventQueuePollCountintThe number of messages to be polled from a default event queue in a single operation. Example: 10Default is 10
eventQueueLongPollTimeoutDurationThe timeout for the poll operation on the default event queue. Example: 1000msDefault is 1000 milliseconds
workflowInputPayloadSizeThresholdDataSizeThe threshold of the workflow input payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 5120KBDefault is 5120 kilobytes
maxWorkflowInputPayloadSizeThresholdDataSizeThe maximum threshold of the workflow input payload size beyond which input will be rejected and the workflow marked as FAILED. Example: 10240KBDefault is 10240 kilobytes
workflowOutputPayloadSizeThresholdDataSizeThe threshold of the workflow output payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 5120KBDefault is 5120 kilobytes
maxWorkflowOutputPayloadSizeThresholdDataSizeThe maximum threshold of the workflow output payload size beyond which output will be rejected and the workflow marked as FAILED. Example: 10240KBDefault is 10240 kilobytes
taskInputPayloadSizeThresholdDataSizeThe threshold of the task input payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 3072KBDefault is 3072 kilobytes
maxTaskInputPayloadSizeThresholdDataSizeThe maximum threshold of the task input payload size beyond which the task input will be rejected and the task marked as FAILED_WITH_TERMINAL_ERROR. Example: 10240KBDefault is 10240 kilobytes
taskOutputPayloadSizeThresholdDataSizeThe threshold of the task output payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 3072KBDefault is 3072 kilobytes
maxTaskOutputPayloadSizeThresholdDataSizeThe maximum threshold of the task output payload size beyond which the task output will be rejected and the task marked as FAILED_WITH_TERMINAL_ERROR. Example: 10240KBDefault is 10240 kilobytes
maxWorkflowVariablesPayloadSizeThresholdDataSizeThe maximum threshold of the workflow variables payload size beyond which the task changes will be rejected and the task marked as FAILED_WITH_TERMINAL_ERROR. Example: 256KBDefault is 256 kilobytes
taskExecLogSizeLimitintThe maximum size of task execution logs. Example: 10000Default is 10

Example usage

In your configuration file add the configuration as you need

properties
# Conductor App Configuration

# Name of the stack within which the app is running. e.g. devint, testintg, staging, prod etc.
conductor.app.stack=test

# The ID with which the app has been registered. e.g. conductor, myApp
conductor.app.appId=conductor

# The maximum number of threads to be allocated to the executor service threadpool. e.g. 50
conductor.app.executorServiceMaxThreadCount=50

# The timeout duration to set when a workflow is pushed to the decider queue. Example: 30s or 1m
conductor.app.workflowOffsetTimeout=30s

# The number of threads to use for background sweeping on active workflows. Example: 8 if there are 4 processors (2x4)
conductor.app.sweeperThreadCount=8

# The timeout for polling workflows to be swept. Example: 2000ms or 2s
conductor.app.sweeperWorkflowPollTimeout=2000ms

# The number of threads to configure the threadpool in the event processor. Example: 4
conductor.app.eventProcessorThreadCount=4

# Whether to enable indexing of messages within event payloads. Example: true or false
conductor.app.eventMessageIndexingEnabled=true

# Whether to enable indexing of event execution results. Example: true or false
conductor.app.eventExecutionIndexingEnabled=true

# Whether to enable the workflow execution lock. Example: true or false
conductor.app.workflowExecutionLockEnabled=false

# The time for which the lock is leased. Example: 60000ms or 1m
conductor.app.lockLeaseTime=60000ms

# The time for which the thread will block in an attempt to acquire the lock. Example: 500ms or 1s
conductor.app.lockTimeToTry=500ms

# The time to consider if a worker is actively polling for a task. Example: 10s
conductor.app.activeWorkerLastPollTimeout=10s

# The time for which a task execution will be postponed if rate-limited or concurrent execution limited. Example: 60s
conductor.app.taskExecutionPostponeDuration=60s

# Whether to enable indexing of tasks. Example: true or false
conductor.app.taskIndexingEnabled=true

# Whether to enable indexing of task execution logs. Example: true or false
conductor.app.taskExecLogIndexingEnabled=true

# Whether to enable asynchronous indexing to Elasticsearch. Example: true or false
conductor.app.asyncIndexingEnabled=false

# The number of threads in the threadpool for system task workers. Example: 8 if there are 4 processors (2x4)
conductor.app.systemTaskWorkerThreadCount=8

# The maximum number of threads to be polled within the threadpool for system task workers. Example: 8
conductor.app.systemTaskMaxPollCount=8

# The interval after which a system task will be checked by the system task worker for completion. Example: 30s
conductor.app.systemTaskWorkerCallbackDuration=30s

# The interval at which system task queues will be polled by system task workers. Example: 50ms
conductor.app.systemTaskWorkerPollInterval=50ms

# The namespace for the system task workers to provide instance-level isolation. Example: namespace1, namespace2
conductor.app.systemTaskWorkerExecutionNamespace=

# The number of threads to be used within the threadpool for system task workers in each isolation group. Example: 4
conductor.app.isolatedSystemTaskWorkerThreadCount=4

# The duration of workflow execution qualifying as short-running when async indexing to Elasticsearch is enabled. Example: 30s
conductor.app.asyncUpdateShortRunningWorkflowDuration=30s

# The delay with which short-running workflows will be updated in Elasticsearch when async indexing is enabled. Example: 60s
conductor.app.asyncUpdateDelay=60s

# Whether to validate the owner email field as mandatory within workflow and task definitions. Example: true or false
conductor.app.ownerEmailMandatory=true

# The number of threads used in the Scheduler for polling events from multiple event queues. Example: 8 if there are 4 processors (2x4)
conductor.app.eventQueueSchedulerPollThreadCount=8

# The time interval at which the default event queues will be polled. Example: 100ms
conductor.app.eventQueuePollInterval=100ms

# The number of messages to be polled from a default event queue in a single operation. Example: 10
conductor.app.eventQueuePollCount=10

# The timeout for the poll operation on the default event queue. Example: 1000ms
conductor.app.eventQueueLongPollTimeout=1000ms

# The threshold of the workflow input payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 5120KB
conductor.app.workflowInputPayloadSizeThreshold=5120KB

# The maximum threshold of the workflow input payload size beyond which input will be rejected and the workflow marked as FAILED. Example: 10240KB
conductor.app.maxWorkflowInputPayloadSizeThreshold=10240KB

# The threshold of the workflow output payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 5120KB
conductor.app.workflowOutputPayloadSizeThreshold=5120KB

# The maximum threshold of the workflow output payload size beyond which output will be rejected and the workflow marked as FAILED. Example: 10240KB
conductor.app.maxWorkflowOutputPayloadSizeThreshold=10240KB

# The threshold of the task input payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 3072KB
conductor.app.taskInputPayloadSizeThreshold=3072KB

# The maximum threshold of the task input payload size beyond which the task input will be rejected and the task marked as FAILED_WITH_TERMINAL_ERROR. Example: 10240KB
conductor.app.maxTaskInputPayloadSizeThreshold=10240KB

# The threshold of the task output payload size beyond which the payload will be stored in ExternalPayloadStorage. Example: 3072KB
conductor.app.taskOutputPayloadSizeThreshold=3072KB

# The maximum threshold of the task output payload size beyond which the task output will be rejected and the task marked as FAILED_WITH_TERMINAL_ERROR. Example: 10240KB
conductor.app.maxTaskOutputPayloadSizeThreshold=10240KB

# The maximum threshold of the workflow variables payload size beyond which the task changes will be rejected and the task marked as FAILED_WITH_TERMINAL_ERROR. Example: 256KB
conductor.app.maxWorkflowVariablesPayloadSizeThreshold=256KB

# The maximum size of task execution logs. Example: 10000
conductor.app.taskExecLogSizeLimit=10000