Object Overview

The File input connector object allows AlchemyPoint Server to read file data from locally mounted filesystems or network share drives. This allows AlchemyPoint to process and convert file data stored locally, or use files as triggers for other processes.

This object utilizes AlchemyPoint's underlying Connector framework and thus supports a variety of rate limiting and concurrency options.

Read files can be automatically processed (transformed, etc.) or utilized as triggers for other processes using Bindings.

Example Object Definitions

... directly within the top-level ServerConfiguration object:

Minimal Configuration:

<FileInputConnector name="LocalFileIn1"
                    readDirectory="C:\Incoming"/>

Overview:

The above example is a File input connector that reads the file "input.txt" from the local filesystem directory "C:\Incoming". The input file is automatically deleted after being successfully read (because "deleteAfter" has a default setting of "true").

Complex Configuration (all possible attributes shown):

<FileInputConnector name="LocalFileIn1" tags="connector local file input"
                    description="example output for loading file data from a locally mounted filesystem directory"
                    readDirectory="C:\Incoming"
                    inputFilePattern="*.txt"
                    deleteAfter="false"
                    moveDirectory="C:\Processed"
                    rateLimitDuration="60"
                    rateLimitPipelineMax="1"
                    connectRateLimit="5"
                    reconnectAttempts="5"
                    reconnectInterval="60"
                    reconnectExpBackoff="true"/>

Overview:

The above example is a File input connector that reads all files matching the TextPattern? "*.txt" from the local filesystem directory "C:\Incoming". After being successfully read, files are automatically relocated to the directory "C:\Processed".

This connector is rate limited at 5 local file reads every 60 seconds, and will attempt to retry file read operations up to 5 times (in the event of file locks or other read failures). The first read attempt retry is made after a 60 second delay, with subsequent attempt delays doubling each time (120 seconds, 240, ...).

Supported Attributes

General Attributes:

• name - (usage: required): Public name of the configuration object. This name must be globally-unique across the entire AlchemyPoint ServerConfiguration definition.

• description - (usage: optional): Textual description of the configuration object.

• tags - (usage: optional): Space-delimited list of one or more tags used to describe the configuration object.

Object Attributes:

• readDirectory - (usage: required): The local/network-share filesystem directory that files should be read from.

• inputFileName - (usage: mixed): The name of the input file that should be read by this Connector. This filename is combined with the "readDirectory" parameter setting to determine the full filesystem path of the input file.
  • This option is required if the "inputFilePattern" parameter is not set.
  • This option cannot be used if the "inputFilePattern" parameter is set.

• inputFilePattern - (usage: mixed): A TextPattern? defining the files that should be read by this Connector. Any filenames matching this TextPattern? will be read by the Connector.
  • This option is required if the "inputFileName" parameter is not set.
  • This option cannot be used if the "inputFileName" parameter is set.

• deleteAfter - (usage: optional, default: "true"): If set to "true", the Connector will automatically delete files from the "readDirectory" once they've been successfully read.
  • This option cannot be used if the "moveDirectory" parameter is set.

• moveDirectory - (usage: optional): The local / network-share filesystem directory that files should be relocated to after they've been read from the "readDirectory" by this Connector.
  • This option cannot be used if the "deleteAfter" parameter is set to "true".

Rate Limit Attributes:

• interval - (usage: optional, default: "900"): The number of seconds to wait between executing each Input operation. A value of "0" configures the Connector to only perform the Input operation once, and never again.

Automatic Reconnect Attributes:

• reconnectAttempts - (usage: optional, default: "0"): The number of times (0-1000) this Connector should attempt to read a file before giving up.

• reconnectInterval - (usage: optional, default: "1"): The interval (1+) in seconds that the Connector should delay between retry attempts for file reads.

• reconnectExpBackoff - (usage: optional, default: "true"): Boolean setting that indicates whether the Connector should exponentially increase the wait interval upon each file read attempt (60 seconds -> 120 -> 240 -> ...)

Content Delivery Attributes:

• deliveryMode - (usage: optional, default: "chunk_big"): Specifies the manner (buffered or "streaming") in which content is delivered to its requested AlchemyPoint endpoint (ProtocolConnector? , ProtocolListener? , etc.). Supported "deliveryMode" settings are listed below:
  • buffer - Buffer complete resource before delivering.
  • chunk - Support chunked delivery only if resource is chunked and length isn't known.
  • chunk_big - Support chunked delivery if resource is chunked, length isn't known, and buffer highwater mark has been reached.
  • chunk_all - Support chunked delivery of any resource that is chunked, regardless of size.
  • chunk_force - Force chunked delivery regardless of whether resource is chunked or size.
  • chunk_force_big - Support chunked delivery if buffer highwater mark has been reached.
  • chunk_all_force_big - Support chunked delivery if chunked (regardless of size), or if buffer highwater mark has been reached.

• highwater - (usage: optional, default: "100000"): Specifies a highwater mark at which to stop buffering and start streaming data to its requested AlchemyPoint endpoint.
  • This option may only be specified when the Connector "deliveryMode" parameter is set to "chunk_big", "chunk_force_big", or "chunk_all_force_big".

• chunkSize - (usage: optional, default: "4096"): Specifies the size of data "chunks" (in bytes) that should be streamed to the requested AlchemyPoint endpoint.

Supported Child Configuration Elements

File Bindings:

• Bindings - (usage: optional): Bindings object, containing one or more FileBinding? configuration elements. These bindings define the manner in which downloaded files are acted upon, used as triggers for other processes, etc.

Issues

Unresolved documentation issues: concurrency