Resource Caching in Flowgear

The Flowgear Runtime makes extensive use of caching to increase workload throughput.

While the performance boost from caching is desirable, there are cases where cached resources are stale leading to behavior that is hard to interpret.

This article describes how caching is used.

Nodes

Node binaries are immutable for a specific Node version. The runtime caches the Node binaries in the filesystem and this cache never expires.

Some Nodes depend on Node Configuration to initialize. This configuration (usually a JSON document), is also cached in the filesystem.

There is no expiration time on the Node Configuration cache since it seldom changes. To force it to be re-cached, click the Clear Cache button in the Node detail Pane.

Connections

The Connection cache is bypassed when a Workflow is being debugged. In all other invoke modes, a memory cache of Connections is held and refreshed every 5 minutes.

Connections are assigned when the Workflow first loads (except for dynamically injected connections. This means that it will be necessary to restart a Workflow before an updated Connection will take effect.

Connection Metadata (usually a JSON document) follows the same cache policy as the Connection itself.

Workflow Definitions

The definition of a Workflow at a specific revision is immutable. In other words, changing the definition causes a new revision to be produced.

Therefore, the runtime is able to cache Workflow definitions for specific Workflow versions. There are two cache tiers - first memory, and second, the filesystem.

When a Workflow is invoked from the Console, a lookup is performed to find the Workflow revision that is currently deployed into the Environment the Workflow is being activated within. This lookup resolves a revision number which is the key that is used to locate the correct Workflow version in the cache.

When a Workflow is invoked via HTTP, the route cache (see below) includes the revision number of the Workflow so no additional lookup is needed.

HTTP Routes

Flowgear caches HTTP routes for API-bound Workflows on startup and then once per minute.

If an attempt is made to invoke a route that does not exist in cache, the routes may be synchronously re-cached. This technique enables the route cache to become aware of new routes that have just become available (for example a new Workflow or a Workflow that has been promoted to a new Environment).

Note that if too many requests to unknown routes are seen in a short space of time, the routes will not be re-cached (this is an overload protection mechanism).

If a Workflow is given a route that was previously used on a different Workflow, it is possible that the original Workflow will fire up in response until the routes are re-cached.

Here is a worked example of how this scenario can arise:

1. Workflow A is created with route /cat
2. Workflow A is promoted from Test Environment to Production
3. Workflow A route is changed to /dog (Test Environment)
4. Workflow B is created with route /cat

When an invoke on the Test Environment is performed against /cat, Workflow A will fire up instead of Workflow B, until routes are re-cached.

To cause this issue to resolve more quickly, attempt to open a non-existent route (from a browser window). This action will cause a route re-cache to take place immediately.