Object Overview

The Feed input connector object allows AlchemyPoint Server to download RSS / ATOM syndication content. With this Connector, AlchemyPoint operates as an RSS reader / aggregator, using an underlying database to track new / updated feed items in a persistent fashion.

This object utilizes AlchemyPoint's underlying Connector framework and thus supports a variety of rate limiting and concurrency options.

Downloaded RSS / ATOM content can be automatically processed/transformed or utilized as triggers for starting other processes using Bindings.

Example Object Definitions

... directly within the top-level ServerConfiguration object:

Minimal Configuration:

<FeedInputConnector name="FeedIn1"
                    url="http://some.feed.url/"/>

Overview:

The above example is a Feed input connector that subscribes to the ATOM / RSS feed "http://some.feed.url".

Complex Configuration (all possible attributes shown):

<FeedInputConnector name="FeedIn1" tags="connector feed input"
                    description="example output for retrieving RSS / ATOM updates from feeds listed in an OPML file"
                    opmlFile="mysubscriptions.opml"
                    intervalMin="900"
                    intervalMax="3600"
                    itemHistory="50"
                    rateLimitDuration="60"
                    connectRateLimit="5"
                    reconnectAttempts="5"
                    reconnectInterval="60"
                    reconnectExpBackoff="true"/>

Overview:

The above example is a Feed input connector that subscribes to all feeds defined within the "mysubscriptions.opml" file. A history of up to 50 previous items is tracked for each subscribed feed. Feeds are checked at minimum every 15 minutes (900 seconds), and at maximum once per hour (3600 seconds). TTL hints within subscribed feeds are obeyed as long as they are within these minimum / maximum interval values.

This connector is rate limited at 5 Feed retrieval requests every 60 seconds, and will attempt to retry requests up to 5 times. The first retry is made after a 60 second delay, with subsequent attempt delays doubling each time (120 seconds, 240, ...).

Supported Attributes

General Attributes:

• url - (usage: mixed): The HTTP url of a RSS / ATOM feed that should be monitored for updates. This option should be utilized when FeedInputConnector needs to monitor only a single feed.
  • This option is required if no "opmlFile" paremeter is set.
  • This option cannot be utilized if the "opmlFile" parameter is set.

• opmlFile - (usage: mixed): The published name of a HostedContent file containing an OPML file that lists RSS / ATOM feeds. This option should be utilized when FeedInputConnector needs to monitor an entire set of feeds for updates.
  • This option is required if no "url" paremeter is set.
  • This option cannot be utilized if the "url" parameter is set.

• itemHistory - (usage: optional, default: "100"): The maximum number of items to track in the persistence DB for an RSS / ATOM feed. Normally this value can be set to something low (10-20), unless a feed frequently updates very old items.

Rate Limit Attributes:

• interval - (usage: optional, default: "900"): The number of seconds to wait between executing each Input operation. A value of "0" configures the Connector to only perform the Input operation once, and never again.

• adjustInterval - (usage: optional, default: "false"): Specifies whether AlchemyPoint can "tune" the interval between checking for feed updates. A variety of mechanisms including RSS TTL checks are utilized as part of this tuning process.

• intervalMin - (usage: optional, default: "3600"): The minimum interval (in seconds) to wait between checking an RSS / ATOM feed for updates. This minimum value is utilized regardless of RSS / ATOM feed TTL settings.

• intervalMax - (usage: optional, default: "86400"): The maximum interval (in seconds) to wait between checking an RSS / ATOM feed for updates. This maximum value is utilized regardless of RSS / ATOM feed TTL settings.

Content Delivery Attributes:

• deliveryMode - (usage: optional, default: "chunk_big"): Specifies the manner (buffered or "streaming") in which content is delivered to its requested AlchemyPoint endpoint (ProtocolConnector? , ProtocolListener? , etc.). Supported "deliveryMode" settings are listed below:
  • buffer - Buffer complete resource before delivering.
  • chunk - Support chunked delivery only if resource is chunked and length isn't known.
  • chunk_big - Support chunked delivery if resource is chunked, length isn't known, and buffer highwater mark has been reached.
  • chunk_all - Support chunked delivery of any resource that is chunked, regardless of size.
  • chunk_force - Force chunked delivery regardless of whether resource is chunked or size.
  • chunk_force_big - Support chunked delivery if buffer highwater mark has been reached.
  • chunk_all_force_big - Support chunked delivery if chunked (regardless of size), or if buffer highwater mark has been reached.

• highwater - (usage: optional, default: "100000"): Specifies a highwater mark at which to stop buffering and start streaming data to its requested AlchemyPoint endpoint.
  • This option may only be specified when the Connector "deliveryMode" parameter is set to "chunk_big", "chunk_force_big", or "chunk_all_force_big".

• chunkSize - (usage: optional, default: "4096"): Specifies the size of data "chunks" (in bytes) that should be streamed to the requested AlchemyPoint endpoint.

Supported Child Configuration Elements

SSL:

* SSLClientConfig - (usage: optional): Specifies local x.509 client certificate, private key, trusted x.509 certificate store, and other SSL-related configuration options.

Proxy Settings:

* ProxyConfig? - (usage: optional): Specifies the local proxy configuration (HTTP, SOCKS4, SOCKS5, etc.) that should be utilized when making outbound connections.

WWW Bindings:

* Bindings? - (usage: optional): Bindings object, containing one or more FeedBinding? ? configuration elements. These bindings define the manner in which retrieved RSS / ATOM content is acted upon, used as triggers for starting other processes, etc.

Issues