How we measure Active Workflows
Each Flowgear subscription comes with a specific limit on Active Workflows. You can create an unlimited number of Workflows, but there's a cap on the number of Workflows that can run simultaneously, known as the Active Workflows limit.
This article discusses how various Workflow invoke types contribute to the Active Workflow count.
Always On Workflows
Always On Workflows typically start with a Trigger Node, such as time-based (e.g. Day Scheduler) or event-based (e.g. File Watcher) triggers. When activated, all subsequent steps in the Workflow are executed before control returns to the Trigger Node to await the next event.
An Always On Workflow is considered active continuously.
To optimize Active Workflow usage for multiple Always On Workflows with the same trigger, consider consolidating them into one main Always On Workflow that sequentially calls a series of Sub-Workflows.
API Invoked Workflows
API Invoked Workflows are considered active only for the period of time that they are running.
Because the rate of invokes on API-bound Workflows may not be within your control, consider splitting API-bound Workflows into a separate Flowgear Site. Also consider performance optimizations to reduce the execution time of these Workflows.
Finally, avoid using API-bound Workflows to run long-running processes. If the Workflow requires 30 seconds or longer to run, rather use a Queue Node (e.g. Flowgear Queue) to enqueue a task and return an immediate response. Then, create a separate Always On Workflow that dequeues these tasks and processes them.
To inform the consumer of completion of the task, you can choose one of these options:
- Have the processing Workflow call a web-hook to let the consumer know. For example, use Web Request to call out back to the consumer signaling that processing has completed
- Have the consumer poll to ask about progress. You can use Nodes such as Get Key-Value and Set Key-Value to set and track progress across Workflows.
Manually Run Workflows
Manually run Workflows are considered active only for the period of time that they are running.
Generally, manually run workflows are not significant contributors to your overall Active Workflow usage but you may consider performance optimizations to reduce their effect on your overall Active Workflow count.
Sub-Workflows
When a Workflow invokes a Sub-Workflow using the Workflow Node, by default it runs serially and does not consume an additional Active Workflow. This means that you can string together any number of Sub-Workflow calls from a parent Workflow and still only be consuming a single Active Workflow.
However, if you choose to use the Workflow Node in Parallel Mode, each new parallel Workflow will be considered an Active Workflow for the period of time that it is running.
For example, if you have a top-level Workflow that uses the Workflow Node with a parallel limit of 5, up to 6 Active Workflows in total could be reached at peak. This is because the parent Workflow consumes 1 Active Workflow and the parallel limit of 5 means that up to an additional 5 Workflows could be run concurrently from the Sub-Workflow.
For this reason, consider the trade-off between serial and parallel processing. Parallel processing will enable you to process data more quickly but use more Active Workflows. Serial processing should be used when there is no real-time requirement in order to minimize the number of Active Workflows.
Managing Active Workflows
If the Active Workflow limit has been reached, attempts to run an additional Workflow instance will fail with the message "The maximum number of concurrent tasks has been exceeded or this Site is not allocated to an active subscription".
The Active Workflows count increments every time a Workflow starts and decrements every time a Workflow steps. Therefore, if you have spiky workloads, such as API-Invoked workloads, consider leaving headroom in your subscription to accommodate volatility.
For Sites running predominantly API-Invoked workloads, we recommend that typical load is 60% - 70% of the max Active Workflows permitted for your subscription.