Object Overview

The FTP input connector object allows AlchemyPoint Server to download files from a remote FTP server. This allows AlchemyPoint to automatically retrieve file updates from network servers, perform asynchronous messaging over FTP, etc.

This object utilizes AlchemyPoint's underlying Connector framework and thus supports a variety of rate limiting and concurrency options.

Downloaded files can be automatically processed or utilized as triggers for other processes using Bindings.

Example Object Definitions

... directly within the top-level ServerConfiguration object:

Minimal Configuration A:

<FTPInputConnector name="FtpIn1"
                   hostname="ftp.myisp.com"
                   username="testuser"
                   password="secret123"
                   readDirectory="/pub/project1"/
                   inputFileName="test.txt">

Overview:

The above example is a FTP input connector that retrieves a specific file from a remote FTP server: "test.txt". This file is downloaded from the host 'ftp.myisp.com' and the directory '/pub/project1'. The Connector uses the username 'testuser' and the password 'secret123' to login to the remote FTP server.

Minimal Configuration B:

<FTPInputConnector name="FtpIn1"
                   hostname="ftp.myisp.com"
                   username="testuser"
                   password="secret123"
                   readDirectory="/pub/project1"/
                   inputFilePattern="*.tar.gz">

Overview:

The above example is a FTP input connector that retrieves files from a remote FTP server that match a specific pattern: "*.tar.gz". These files are downloaded from the host 'ftp.myisp.com' and the directory '/pub/project1'. The Connector uses the username 'testuser' and the password 'secret123' to login to the remote FTP server.

Complex Configuration (all possible attributes shown):

<FTPInputConnector name="FtpIn1" tags="connector ftp input"
                   description="example output for retrieving files from a destination FTP server"
                   hostname="ftp.myisp.com"
                   portNum="21"
                   username="anonymous"
                   password="user@host.com"
                   readDirectory="/pub/project1"
                   inputFilePattern="*.tar.gz"
                   rateLimitDuration="60"
                   connectRateLimit="5"
                   reconnectAttempts="5"
                   reconnectInterval="60"
                   reconnectExpBackoff="true"/>

Overview:

The above example is a FTP input connector that retrieves files from a remote FTP server that match a specific pattern: "*.tar.gz". These files are downloaded from the server 'ftp.myisp.com', operating on port 21, from the directory '/pub/project1'. The Connector uses the username 'anonymous' and the password 'user@host.com' to login to the remote FTP server.

This connector is rate limited at 5 FTP file downloads every 60 seconds, and will attempt to redeliver file downloads up to 5 times. The first download retry is made after a 60 second delay, with subsequent attempt delays doubling each time (120 seconds, 240, ...).

Supported Attributes

General Attributes:

• name - (usage: required): Public name of the configuration object. This name must be globally-unique across the entire AlchemyPoint ServerConfiguration definition.

• description - (usage: optional): Textual description of the configuration object.

• tags - (usage: optional): Space-delimited list of one or more tags used to describe the configuration object.

Network Attributes:

• hostname - (usage: required): Internet hostname (or IP address) of the FTP server connected to by this Connector.

• portNum - (usage: required): TCP port (1-65535) of the FTP server connected to by this Connector.

• username - (usage: optional): The username that should be utilized by this Connector when logging into the remote FTP server.

• password - (usage: optional): The password that should be utilized by this Connector when logging into the remote FTP server.

• protocol - (usage: optional, default: "ftp"): The protocol that should be utilized by this Connector when connecting to the remote FTP server. Supported 'protocol' values are:
  • ftp - FTP (plain-text, non-encrypted/unsecure) - Most servers utilize the FTP protocol.
  • ftps - FTPS (FTP + SSL, encrypted/secure) - If FTPS is utilized, specific x.509 certificate options may be specified by using the SSLClientConfig child configuration object with this Connector.

Object Attributes:

• readDirectory - The filesystem directory that files should be downloaded from on the remote FTP server.

Rate Limit Attributes:

• interval - (usage: optional, default: "900"): The number of seconds to wait between executing each Input operation. A value of "0" configures the Connector to only perform the Input operation once, and never again.

• rateLimitDuration - (usage: optional, default: "0"): The duration (in seconds) utilized by the 'connectRateLimit' parameter. A value of "0" disables all rate limits for this Connector.

• connectRateLimit - (usage: optional, default: "0" [unlimited]): The number of TCP connections that may be created by this Connector within the 'rateLimitDuration' period.

Automatic Reconnect Attributes:

• reconnectAttempts - (usage: optional, default: "0"): The number of times (0-1000) this Connector should attempt to download a file before giving up.

• reconnectInterval - (usage: optional, default: "1"): The interval (1+) in seconds that the Connector should delay between retry attempts for file downloads.

• reconnectExpBackoff - (usage: optional, default: "true"): Boolean setting that indicates whether the Connector should exponentially increase the reconnection interval upon each file download attempt (60 seconds -> 120 -> 240 -> ...)

Content Delivery Attributes:

• deliveryMode - (usage: optional, default: "chunk_big"): Specifies the manner (buffered or "streaming") in which content is delivered to its requested AlchemyPoint endpoint (ProtocolConnector? , ProtocolListener? , etc.). Supported "deliveryMode" settings are listed below:
  • buffer - Buffer complete resource before delivering.
  • chunk - Support chunked delivery only if resource is chunked and length isn't known.
  • chunk_big - Support chunked delivery if resource is chunked, length isn't known, and buffer highwater mark has been reached.
  • chunk_all - Support chunked delivery of any resource that is chunked, regardless of size.
  • chunk_force - Force chunked delivery regardless of whether resource is chunked or size.
  • chunk_force_big - Support chunked delivery if buffer highwater mark has been reached.
  • chunk_all_force_big - Support chunked delivery if chunked (regardless of size), or if buffer highwater mark has been reached.

• highwater - (usage: optional, default: "100000"): Specifies a highwater mark at which to stop buffering and start streaming data to its requested AlchemyPoint endpoint.
  • This option may only be specified when the Connector "deliveryMode" parameter is set to "chunk_big", "chunk_force_big", or "chunk_all_force_big".

• chunkSize - (usage: optional, default: "4096"): Specifies the size of data "chunks" (in bytes) that should be streamed to the requested AlchemyPoint endpoint.

Supported Child Configuration Elements

SSL:

• SSLClientConfig - (usage: optional): Specifies local x.509 client certificate, private key, trusted x.509 certificate store, and other SSL-related configuration options.

File Bindings:

• Bindings - (usage: optional): Bindings object, containing one or more FileBinding? configuration elements. These bindings define the manner in which downloaded files are acted upon, used as triggers for other processes, etc.

Issues

Unresolved documentation issues: timeouts, concurrency