Amazon S3 v2

Amazon S3 is an object storage service used to store, organize, and retrieve files through buckets and object keys.

The Flowgear Amazon S3 Node lets you list buckets, validate bucket access, list objects, download object streams, upload objects, and delete individual objects from a Workflow. Query-style methods return only the S3 business payload, while mutation-style methods use the Flowgear object for default success or failure metadata.

Revision History

0.0.0.4 - Initial release.
0.1.0.0 - Completed the Runtime overhaul with eager first-page startup and lazy continuation paging.
0.1.1.0 - Added sanitized request echoes to Upload Object and Delete Object responses.
0.1.1.2 - Renamed the Bucket Name and Content Type parameters to BucketName and ContentType, and updated Runtime compatibility.

Connection

Create an Amazon S3 Connection so the Node can sign requests with AWS credentials and optionally supply a default bucket.

Property Type Description
Access Key String AWS access key ID used for request signing. Required.
Secret Key Masked AWS secret access key used for request signing. Required.
Session Token Masked Optional AWS session token required when the credentials come from STS or an assumed role.
Region Name String Default AWS Region used for S3 requests, for example eu-west-1. Required. The default is us-east-1.
Bucket Name String Optional default bucket name used when methods omit BucketName.

Setup Notes

  • Create an AWS identity with the S3 permissions your Workflow needs. Typical permissions for this Node are s3:ListAllMyBuckets, s3:ListBucket, s3:GetObject, s3:PutObject, and s3:DeleteObject.
  • Enter a Region Name even if you also enter a default Bucket Name.
  • Enter a Session Token when you use temporary AWS credentials. This Node does not refresh AWS credentials for you.
  • Enter a default Bucket Name on the Connection if you want object methods to use one bucket by default.
  • Test the Connection after you configure it. The connection test performs a bucket-list call, so the credentials must be able to list buckets.

Methods

This Node exposes methods for common bucket and single-object operations on general-purpose Amazon S3 buckets.

ListBuckets

Use this method to return the buckets available to the configured AWS credential.

Parameter Type Description
Connection Connection Amazon S3 Connection details including AWS credentials and the default Region.
Return Type Description
Buckets Array One row per bucket. See the Bucket Properties table below.

Bucket Properties

Property Type Description
name String Bucket name.
bucketRegion String AWS Region for the bucket, when Amazon S3 returns it.
creationDate DateTime Date and time when the bucket was created, when available.

HeadBucket

Use this method to validate one bucket and return normalized bucket metadata such as the resolved Region. If the bucket does not exist or cannot be accessed, the method fails with the Amazon S3 error.

Parameter Type Description
Connection Connection Amazon S3 Connection details including AWS credentials and the default Region.
BucketName String Bucket name to validate. If you leave this blank, the Node uses the default Bucket Name from the Connection.
Return Type Description
Bucket Object One row with normalized bucket metadata. See the Bucket Properties table below.

Bucket Properties

Property Type Description
bucketName String Bucket name that was validated.
bucketRegion String Resolved AWS Region for the bucket, when Amazon S3 returns it.
exists Boolean Indicates that Amazon S3 confirmed the bucket exists. A successful response always returns true.
accessPointAlias Boolean Indicates whether the bucket was addressed through an access point alias, when Amazon S3 returns that value.

ListObjects

Use this method to list objects in one bucket. You can optionally limit the results to a key prefix.

Parameter Type Description
Connection Connection Amazon S3 Connection details including AWS credentials and the default Region.
BucketName String Bucket name to query. If you leave this blank, the Node uses the default Bucket Name from the Connection.
Prefix String Optional key prefix used to limit the returned objects.
Return Type Description
Objects Array One row per object. See the Object Properties table below.

Object Properties

Property Type Description
bucketName String Bucket that contains the object.
key String Full object key.
size Integer Object size in bytes.
eTag String Entity tag returned by Amazon S3, when available.
lastModified DateTime Date and time when the object was last modified, when available.
storageClass String Storage class for the object, when Amazon S3 returns it.

DownloadObject

Use this method to download one object as a stream.

Parameter Type Description
Connection Connection Amazon S3 Connection details including AWS credentials and the default Region.
Key String Exact object key to download.
BucketName String Bucket name containing the object. If you leave this blank, the Node uses the default Bucket Name from the Connection.
Return Type Description
Content Stream The downloaded object content stream.

UploadObject

Use this method to upload one stream to a bucket as a single object.

Parameter Type Description
Connection Connection Amazon S3 Connection details including AWS credentials and the default Region.
Content Stream Binary content stream to upload.
Key String Exact object key to create or replace.
BucketName String Bucket name that receives the uploaded object. If you leave this blank, the Node uses the default Bucket Name from the Connection.
ContentType String Optional content type stored with the uploaded object.
Return Type Description
Object Object One row with normalized write metadata. Each response includes Flowgear.IsSuccess, Flowgear.Message, and Flowgear.Request. See the Object Properties table below.

Object Properties

Property Type Description
bucketName String Bucket that received the uploaded object.
key String Full object key that was created or replaced.
eTag String Entity tag returned by Amazon S3, when available.
versionId String Object version ID, when bucket versioning returns one.

DeleteObject

Use this method to delete one object from a bucket.

Parameter Type Description
Connection Connection Amazon S3 Connection details including AWS credentials and the default Region.
Key String Exact object key to delete.
BucketName String Bucket name containing the object. If you leave this blank, the Node uses the default Bucket Name from the Connection.
Return Type Description
Object Object One row with normalized delete metadata. Each response includes Flowgear.IsSuccess, Flowgear.Message, and Flowgear.Request. See the Object Properties table below.

Object Properties

Property Type Description
bucketName String Bucket that contained the deleted object.
key String Full object key that was deleted.
versionId String Object version ID returned by Amazon S3, when available.
deleteMarker Boolean Indicates whether the delete created or referenced a delete marker, when Amazon S3 returns that value.

Usage Notes

  • ListBuckets emits one row per bucket.
  • ListObjects emits one row per object and handles S3 pagination internally. You do not supply continuation tokens.
  • Upload and delete responses copy their effective scalar inputs into Flowgear.Request. Upload content is excluded so binary streams are not copied into response metadata.
  • ListBuckets and ListObjects complete their first S3 page while the Node invoke is awaited. Additional pages are requested only as the Workflow consumes the response stream.
  • ListBuckets, HeadBucket, and ListObjects surface the first S3 provider error instead of returning a synthetic query row.
  • DownloadObject returns a stream instead of JSON rows. If Amazon S3 returns an error, the Node raises that error instead of returning a response row.
  • If you omit BucketName on any method that accepts it, the Node uses the default Bucket Name from the Connection. The method fails if neither value is supplied.
  • DownloadObject, UploadObject, and DeleteObject require a non-blank Key.
  • If an upload or delete fails, the Node returns a response row instead of raising an error. The row contains bucketName, key, and Flowgear.Request, with Flowgear.IsSuccess set to false and Flowgear.Message containing the Amazon S3 error code and message. Check Flowgear.IsSuccess before relying on the result.
  • UploadObject buffers a non-seekable Content stream in memory before sending it to Amazon S3, so very large non-seekable streams increase memory use.
  • If ContentType is blank, Amazon S3 applies its default content type.

Known Issues

  • The Node supports general-purpose Amazon S3 buckets only. Directory buckets and access points are not supported.
  • Delimiter-based pseudo-folder browsing is not supported.
  • Multipart upload is not supported.
  • Version-specific delete is not supported.
  • Presigned URL generation is not supported.