Eventual Consistency v2

When integrating across multiple services, it is usually not possible to synchronize one transaction across all of them. If a Workflow creates two separate Objects, it may create the first Object before the second operation fails. In that case, rolling back the first operation may not be possible.

Design Workflows to create or change only Objects that have not yet reached their intended state. This approach supports eventual consistency and allows you to run a Workflow again with the same input without duplicating results.

Worked example

A Workflow:

  1. Reads an order and customer record from an e-commerce platform.
  2. Creates a customer record in the ERP.
  3. Attempts to create the order record but fails because of a data error, invalid credential, or connectivity problem.

A naive design may duplicate the customer when the Workflow runs again, or fail while creating the customer because the record already exists.

Make each write action idempotent so it can be repeated without changing its result after the first successful run.

Decide on an idempotency strategy

Check whether the service you are writing to supports idempotency natively.

A service that publishes a REST API may use a PUT action to denote idempotency. The service may also expose an upsert, which inserts a record or updates the existing record. You can safely send the same data repeatedly when you provide a stable reference that identifies the intended Object, for example, a unique customer ID from the source system.

If the service does not support idempotency, use Key/Value storage to track whether you have already written the record. For each write operation:

  1. Use GetSingle to retrieve the status of a specific record.
  2. Use If to decide whether to skip the write when a matching record is found.
  3. Perform the write.
  4. Use SetSingle to mark the record as written so it is skipped in future runs.

The worked example contains two writes: one for the customer and one for the sales order. When the Workflow must manage idempotency, consider moving each write into a separate Sub-Workflow. Other Workflows can then reuse this logic without duplicating the Steps.

Idempotency on large volumes of records

Most Connectors support bulk writes, so checking each record individually through Key Value storage may be inefficient.

You can still place each write in a Sub-Workflow, but the design works differently:

  1. Use Get to query all keys that might be in scope for the write.
  2. Use Merge to correlate records that need to be written with their matching Key Value records.
  3. Perform the writes in bulk, using a FILTER Expression to exclude items that already have associated Key Value records.
  4. Use Set to update the Key Value records. Use another FILTER Expression so only successful write results are committed. Methods that return normalized mutation outcomes provide a Flowgear.IsSuccess Property for this purpose; check the selected Method's contract.

For a worked Merge example and alternatives using the echoed request or a loop, see Correlate Records Across Systems.

Account for gaps between operations

The provider write and the Key/Value marker write are separate operations. If the provider succeeds but SetSingle or Set fails, the next run will not find the marker and may repeat the write. Likewise, two concurrent runs can both observe a missing marker before either stores one.

A marker is therefore recovery tracking, not an atomic lock or an exactly-once guarantee. Prefer a provider idempotency key, an upsert with a unique business key, or a storage transaction that supplies the necessary concurrency protection. When the provider outcome is uncertain, look up the target record using a stable source identifier before attempting another create.

For mixed batch outcomes, follow Handle Partial Batch Failures so successful records are not unnecessarily replayed.

See also