FetchS3Object 2.3.0¶
BUNDLE¶
org.apache.nifi | nifi-aws-nar
DESCRIPTION¶
Retrieves the contents of an S3 Object and writes it to the content of a FlowFile
INPUT REQUIREMENT¶
REQUIRED
Supports Sensitive Dynamic Properties¶
false
PROPERTIES¶
Property |
Description |
|---|---|
AWS Credentials Provider service |
The Controller Service that is used to obtain AWS credentials provider |
Bucket |
The S3 Bucket to interact with |
Communications Timeout |
The amount of time to wait in order to establish a connection to AWS or receive data from AWS before timing out. |
Endpoint Override URL |
Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints. |
Object Key |
The S3 Object Key to use. This is analogous to a filename for traditional file systems. |
Region |
The AWS Region to connect to. |
SSL Context Service |
Specifies an optional SSL Context Service that, if provided, will be used to create connections |
Signer Override |
The AWS S3 library uses Signature Version 4 by default but this property allows you to specify the Version 2 signer to support older S3-compatible services or even to plug in your own custom signer implementation. |
Version |
The Version of the Object to download |
custom-signer-class-name |
Fully qualified class name of the custom signer class. The signer must implement com.amazonaws.auth.Signer interface. |
custom-signer-module-location |
Comma-separated list of paths to files and/or directories which contain the custom signer’s JAR file and its dependencies (if any). |
encryption-service |
Specifies the Encryption Service Controller used to configure requests. PutS3Object: For backward compatibility, this value is ignored when ‘Server Side Encryption’ is set. FetchS3Object: Only needs to be configured in case of Server-side Customer Key, Client-side KMS and Client-side Customer Key encryptions. |
proxy-configuration-service |
Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN |
range-length |
The number of bytes to download from the object, starting from the Range Start. An empty value or a value that extends beyond the end of the object will read to the end of the object. |
range-start |
The byte position at which to start reading from the object. An empty value or a value of zero will start reading at the beginning of the object. |
requester-pays |
If true, indicates that the requester consents to pay any charges associated with retrieving objects from the S3 bucket. This sets the ‘x-amz-request-payer’ header to ‘requester’. |
RELATIONSHIPS¶
NAME |
DESCRIPTION |
|---|---|
failure |
If the Processor is unable to process a given FlowFile, it will be routed to this Relationship. |
success |
FlowFiles are routed to this Relationship after they have been successfully processed. |
WRITES ATTRIBUTES¶
NAME |
DESCRIPTION |
|---|---|
s3.url |
The URL that can be used to access the S3 object |
s3.bucket |
The name of the S3 bucket |
path |
The path of the file |
absolute.path |
The path of the file |
filename |
The name of the file |
hash.value |
The MD5 sum of the file |
hash.algorithm |
MD5 |
mime.type |
If S3 provides the content type/MIME type, this attribute will hold that file |
s3.etag |
The ETag that can be used to see if the file has changed |
s3.exception |
The class name of the exception thrown during processor execution |
s3.additionalDetails |
The S3 supplied detail from the failed operation |
s3.statusCode |
The HTTP error code (if available) from the failed operation |
s3.errorCode |
The S3 moniker of the failed operation |
s3.errorMessage |
The S3 exception message from the failed operation |
s3.expirationTime |
If the file has an expiration date, this attribute will be set, containing the milliseconds since epoch in UTC time |
s3.expirationTimeRuleId |
The ID of the rule that dictates this object’s expiration time |
s3.sseAlgorithm |
The server side encryption algorithm of the object |
s3.version |
The version of the S3 object |
s3.encryptionStrategy |
The name of the encryption strategy that was used to store the S3 object (if it is encrypted) |
USE CASES¶
Fetch a specific file from S3 |
Use Cases Involving Other Components¶
Retrieve all files in an S3 bucket |
Retrieve only files from S3 that meet some specified criteria |
Retrieve new files as they arrive in an S3 bucket |