Dash-fu sources need to get their data somehow: enter Bees. Bees are the brave workers that go out and collect data from various data sources and report them back in the form of activity stream, metrics and status updates.

Setup

Dash-fu creates data sources using a four-step process. In the first step, it calls the display method to get HTML fragments and display these to the user.

Currently it uses two fragments:

  • inputs – HTML input controls for setting up a new controller.

  • notes – Additional setup notes: other steps that need to be followed, what values to supply, etc.

Input controls use names of the form source, and wrapped inside label elements. For example:

<label>Screen name <input type="text" name="source[screen_name]" size="30"></label>

Once the user fills and submits the form, Dash-fu creates a new empty state object and calls the setup method with that object and the input fields.

The state object is a Hash than can store basic values: nil, string, boolean, numerics, arrays and other hashes. Key names must be alphanumeric/underscore. Bees use it to store state in between method calls.

The setup method must set one value, “source.name”, to suggest a name for this source, e.g. “Twitter mentions for @dash_fu”.

In the third state, Dash-fu calls the validate method. If this method raises an exceptions, the error message is shown to the user and the source discarded. The fourth step involves registering callbacks (not currently supported).

Metrics

For sources that include metrics, the setup method specifies the follow values during the call to setup:

  • metric.columns – Columns (at least one) as an array of hashes, see attributes names below.

  • metric.totals – True if the metric collects life-time totals (e.g. downloads, uptime), false if it only collects recent values (e.g. average response time).

Columns are specified using the following attributes:

  • id – Column identifier (if missing, derived from column name)

  • name – Column name (required)

Updates

Periodically, all sources are updated by calling their update method. This method taks two arguments, the source state object and an object wrapping several callback methods.

A single update can call any combination of callback methods, for example, it can set values in a metric and record an activity.

The callback methods are:

  • set! – Sets the most recent value for this metric. The single argument is a hash, where key names are either column ids or indices, and values are integers or floats.

  • inc! – Changes the most recent value for this metric. The single argument is a hash, where key names are either column ids or indices, and values are integers or floats.

  • activity! – Adds an activity to the stream, see below for details about the argument.

  • error! – Records a processing error. The single argument is an error message. Since updates are processed asynchronously, use this method to indicate any processing error, don’t raise an exception.

Activities have the following attributes:

  • uid – Unique identifier (within the scope of this source). Optional.

  • title – Short title (for feed, notificatio, etc).

  • html – HTML contents of the activity.

  • text – Text contents of the activity (alternative to html attribute).

  • url – Points back to the original resource. Optional.

  • tags – Any number of tags for identifying related activities. Tags are lower-cased and can contain alphanumeric and dashes. Optional.

  • person – Person who performed this activity (see below).

  • timestamp – Timestamp activity occurred. Optional.

HTML can use links (HTTP/S and email), images, bold, italics, paragraphs, block quotes, lists, tables, pre-formatted text and even video. Scripts, objects and frames are stripped out, as are any script and style attributes.

People are identified by the following attributes:

  • fullname – What it says on the label.

  • email – Email address if known.

  • identities – Any number of identities, in the form domain:id, e.g. twitter.com:assaf or linkedin.com:assafarkin.

  • photo_url – Preferrably 48x48.

In addition, from time to time, Dash-fu would call the meta method and display the returned fields. This method returns an array of hashes, each describing a single display value using the following attributes.

  • title – Title to show next to the value (optional).

  • text – Text to show as value of this field.

  • url – Link (optional).

Webhooks

TBD