Home Certification 05. Connecting to On-Premise Services

05. Connecting to On-Premise Services

In the previous section, we created some basic Workflows to connect to data sources available over the internet but there are many scenarios where it's necessary to connect to on-premises data sources or those that are not published over the internet.

Flowgear DropPoints are software agents that install on to an on-premises server (or a server within the network containing the data sources you need to access). Once configured, they create a encrypted, outbound and persistent connection to the Flowgear cloud.

When a DropPoint is assigned on a Flowgear Connection, requests from a Node using that connection will execute at the DropPoint instead of within the cloud environment. This enables you to build workflows that blend cloud with on-premise endpoints without having to handle routing across different networks.

Characteristics of the DropPoint connection

DropPoints use HTTPS to ensure all traffic is encrypted. They use an outbound connection. In other words, the connection initiates from the DropPoint to the Flowgear Cloud. This is important because it means that organizations don't need to allow inbound access via a custom firewall rule.

Finally, connections are persistent. This means is that if there is a break in connectivity, the DropPoint will pause for a short period (initially five seconds) and then try to reconnect.

Crucially, once it has reconnected, if either the platform or DropPoint had sent data that was not acknowledged prior to the disconnect, that data will be re-sent. This means that in most cases, short disruptions in connectivity won't have any effect on running Workflows except that the affected step will appear to run for longer.

Installing a DropPoint

DropPoints are currently only offered for Microsoft Windows.

Follow the steps at Setting up a DropPoint to install a new DropPoint.

When to use a DropPoint

Generally, DropPoints are used to integrate on-premise data sources but there are four specific cases where they should be considered:

  • When the resource is on-premise or in a private network and you don’t want to create an inbound firewall rule. Since DropPoints use outbound connections, no firewall configuration changes are needed, making them a secure connection method.

  • When the service you're connecting to doesn’t have a web-based API. For example, some older ERPs, rely on COM APIs, and you can use DropPoints to connect directly to them via the appropriate Connector (Node).

  • Cases when the endpoint could be published to the internet but it isn't encrypted. For instance, if you have an intranet site that’s only accessible over HTTP, you wouldn’t want to expose that over the internet. Using a DropPoint would ensure that the traffic flowing over the internet is encrypted even though the local network traffic between the intranet site and the DropPoint is not.

  • Cases where the protocol used to communicate with the data source requires a lot of back-and-forth messages between the client and the data source. Each of these interactions adds up to the total delay based on the time it takes for each exchange. Using a DropPoint, the delay you experience will only be the network latency (which is usually a few milliseconds) instead of the much longer internet latency (which can range from tens to hundreds of milliseconds).

Exercise 04: Read and Write Files

In this exercise we'll use a DropPoint to read a file on your local computer and convert it out to its JSON equivalent.

  1. Copy the content in the code block below and use Notepad to save it as a text file on your computer. We'll use c:\temp\contacts.txt as our example path.

    id,Contact,Company,Phone,Country,Email
201,Priscilla Wooten,Interdum PC,1-906-569-1084,Antarctica,nunc@enimnec.edu
202,Maryam Pate,Imperdiet Non Industries,1-683-328-9343,Sweden,neque@eutempor.org
203,Audrey Davidson,Gravida Praesent Consulting,1-772-762-8735,Finland,magna.Duis@Donecnibh.com
204,Zorita Blevins,Mauris Vel Turpis Incorporated,1-869-713-3148,Montserrat,semper.cursus@eratEtiamvestibulum.co.uk
205,Noel Cortez,Penatibus Et Magnis Ltd,1-925-815-9513,New Caledonia,ligula.Aenean@rutrumurna.co.uk
206,Gavin Webster,Adipiscing Elit Aliquam Inc.,1-682-115-2198,Cape Verde,ut.nisi@aliquamarcuAliquam.net
207,Veronica Scott,Porttitor Eros Nec Inc.,1-988-422-0533,Nauru,sit.amet@Cum.co.uk
208,Ainsley Romero,Purus Corporation,1-847-150-1946,Azerbaijan,tincidunt.nibh.Phasellus@nonummy.ca
209,Kaitlin Hogan,Tellus Suspendisse Sed Corporation,1-289-782-1630,American Samoa,odio@non.ca
210,Lois Oconnor,Lacus Cras LLC,1-774-825-5248,Turkmenistan,ipsum@Fusce.co.uk

  2. Add File Read.

  3. Create a new Connection by clicking the Connection Property value ((None)) of the File Node, then click + in the overlay that appears to create a new Connection, followed by ✎ to open the Connection in a new browser tab.

  4. Under DropPoint, select the DropPoint you have registered on your computer and click Save.

  5. Set File Read.Path to c:\temp\contacts.json (adjust the path if you stored the text file in a different location).

  6. Connect Start.RunNow → File Read Node.

    Run the Workflow at this point. You should see the content of the text file in Content Property of the File Read Node in the Workflow Logs.

  7. Add Flat File, connect File Read → Flat File.

  8. Connect File Read.Content → Flat File.FlatFileDocument.

    The Flat File Node converts different types of flat files (e.g. CSV or tab-delimited) to XML as well as from XML back to a flat file.

  9. Configure the Flat File Node for the file content by setting Flat File.ColumnDelimiter to , and checking (i.e. turning on) Flat File.HasColumnNames.

    The delimiter Properties tell Flat File how to split rows and cells (fields). Where unprintable characters such as return, new line and tab are used, you can specify them with an escaping. For example \t is used to designate a tab character. The full set of supported escapings are shown in the help article for the Node.

  10. Add JSON Convert Node to the Workflow, set JSON Convert.Action to XmlToJson.

  11. Connect Flat File → JSON Convert.

  12. Connect Flat File.XmlDocument → JSON Convert.Xml.

    We will now create a file with the JSON equivalent back in the filesystem.

  13. Add File Write.

  14. Create a new Connection in the same was as the one for File Read and choose your DropPoint in it.

  15. Set File Write.Path Property to c:\temp\contacts.json (adjust the path if you stored the text file in a different location but be sure to use the json file extension).

  16. Connect JSON Convert → File Write.

  17. Connect JSON Convert.Json → File Write.Content.

    Run the Workflow and verify that a JSON file has been created at c:\temp\contacts.json.

Save your Workflow, then click Submit Exercise to grade it.

Securing DropPoints

In the example above, we created a Workflow that could read a file from your computer via DropPoint. Generally, we want to restrict the actions that a DropPoint is able to perform so that no unwanted interactions could be performed on the server via the DropPoint.

Service Account

When installed on a server, the DropPoint service should be run under a user context that has the least required privileges. We recommend that a dedicated user account is created (e.g. domain\flowgear-droppoint) with only the minimum required group assignments.

This account can be assigned to the DropPoint service either from the DropPoint UI via the Service tab or from the Windows services console. You can access this by running services.msc, then locating and right-clicking the Flowgear DropPoint service, choosing Properties, then clicking the Log On tab.

Node Whitelisting

Node Whitelisting is a DropPoint feature that enforces restrictions on which Nodes are allowed to run at a DropPoint and, optionally, what their request Properties are allowed to be.

Enable whitelisting from the DropPoint UI by clicking Whitelisting, then Enable Whitelisting.

Assuming you've already run the exercise above, you should see the File Read Node shown in the table. Try unchecking the Whitelisted checkbox, click Save Changes and run the Workflow again. The Workflow will error stating that the Node is not permitted to execute.

Go back to the DropPoint UI and re-check the Whitelisted checkbox. With the File Read row focussed, a second Property table will display.

We will now further restrict the location of files that can be read by selecting the Path Property and switching Test Mode to AllTests.

With the Path Property focussed, a Tests table will display. Choose String under Test Type and provide the path to your txt file - e.g. c:\temp\contacts.txt.

Click Save Changes and run the Workflow which should now run successfully up to the File Write Node.

For more information on whitelisting, see DropPoint Security.