Bucketize

This gem will aggregate consumption data with heterogeneous granularity across time into systematic intervals. These intervals are then either multiplied with a relevant tariff value then summed into daily totals, or directly summed into daily totals. The simplest example would be to sum hourly data into daily totals, subject to data completeness checks and multiply them with a single tariff number.

Prerequisites

Ruby ~> 2.0.0
Bundler
gem install bundler

Installation

GreenButton gem

Follow installation instructions in the README file of the GreenButton Library module

Bucketize gem

Using Bundler, add the following line to your Gemfile

```ruby
gem 'bucketize'
```

And then execute:

$ bundle

Or install it yourself as:

$ gem build bucketize.gemspec
$ gem install bucketize-0.1.0.gem

Usage

```ruby
require 'ce-greenbutton'
require 'ce-bucketize'

reg_access_token = 'd89bb056-0f02-4d47-9fd2-ec6a19ba8d0c'
app_inf_url = 'https://services.greenbuttondata.org:443/DataCustodian/espi/1_1/resource/ApplicationInformation/2'
subscription_url = 'https://services.greenbuttondata.org:443/DataCustodian/espi/1_1/resource/Batch/RetailCustomer/5/UsagePoint'
access_token = '688b026c-665f-4994-9139-6b21b13fbeee'

# FTP related values
ftp_host = 'us1.hostedftp.com'
ftp_user = 'clearlyenergy'
ftp_password = 'store_clearly'
ftp_path = 'SDGETest'
utility_name = 'CLEARLY'
application_id = '10066_CONSUMPTION'
ftp_time = Time.new(2015, 3, 19, 1, 1, 1, 0)

download_options = {access_token: access_token, subscription_url: subscription_url}
ftp_download_options = {
    use_ftp: true,
    application_id: application_id,
    utility_name: utility_name,
    time: ftp_time
}


# configure the GreenButton Library
Bucketize.config(
    reg_access_token: reg_access_token,
    application_information_url: app_inf_url,
    ftp_host: ftp_host,
    ftp_user: ftp_user,
    ftp_password: ftp_password,
    ftp_path: ftp_path
)

# get consumptions from month start
daily_data = Bucketize.consumption_from_month_start(Time.new(2011, 2, 1, 0, 0, 0, -28800), download_options)
puts 'Consumption From Month Start 2012/02'
puts JSON.pretty_generate(daily_data.to_a)

daily_data = Bucketize.consumption_from_month_start(Time.new(2013, 1, 1, 0, 0, 0, -14400), ftp_download_options)
puts 'Consumption From Month Start 2013/01 Using FTP'
puts JSON.pretty_generate(daily_data.to_a)



# get cost from month start
tariff = '[{
  "utility_id": 1,
  "rate_class": "R",
  "day_of_week": [1,2,3,4,5,6,7],
  "kwh_low": 0,
  "kwh_high":1000,
  "tariff":10
},
{
  "utility_id": 1,
  "rate_class": "R",
  "day_of_week": [1,2,3,4,5,6,7],
  "kwh_low": 1000,
  "tariff":20
}]'
daily_data = Bucketize.cost_from_month_start(Time.new(2011, 2, 1, 0, 0, 0, -28800), tariff, download_options)
puts 'Cost From Month Start 2012/02:'
puts JSON.pretty_generate(daily_data.to_a)

daily_data = Bucketize.cost_from_month_start(Time.new(2013, 1, 1, 0, 0, 0, -14400), tariff, ftp_download_options)
puts 'Cost From Month Start 2013/01 Using FTP'
puts JSON.pretty_generate(daily_data.to_a)



# get consumption by date
daily_data = Bucketize.consumption_by_date(Time.new(2011, 2, 5, 0, 0, 0, -28800), download_options)
puts 'Consumption For Date 2011/02/05'
puts JSON.pretty_generate(daily_data.to_a)

daily_data = Bucketize.consumption_by_date(Time.new(2013, 1, 5, 0, 0, 0, -14400), ftp_download_options)
puts 'Consumption For Date 2013/01/05 Using FTP'
puts JSON.pretty_generate(daily_data.to_a)


# get cost by date
tariff = '[{
    "utility_id": 1,
    "rate_class": "R",
    "time_start": 1,
    "time_end": 3,
    "tariff":10
},
{
    "utility_id": 1,
    "rate_class": "R",
    "time_start": 4,
    "time_end": 24,
    "tariff":20
}]'
daily_data = Bucketize.cost_by_date(Time.new(2011, 2, 5, 0, 0, 0, -28800), tariff,
                                    download_options)
puts 'Consumption For Date 2011/02/05'
puts JSON.pretty_generate(daily_data.to_a)

daily_data = Bucketize.cost_by_date(Time.new(2013, 1, 5, 0, 0, 0, -14400), tariff,
                                    ftp_download_options)
puts 'Consumption For Date 2013/01/05 Using FTP'
puts JSON.pretty_generate(daily_data.to_a)



# get consumption by date range.
daily_data = Bucketize.consumption_date_range(Time.new(2011, 2, 5, 0, 0, 0, -28800),
                                              Time.new(2011, 2, 10, 23, 59, 59, -28800),
                                              download_options)
puts 'Consumption For Dates 2011/02/05 to 2011/02/10 :'
puts JSON.pretty_generate(daily_data.to_a)

daily_data = Bucketize.consumption_date_range(Time.new(2013, 1, 5, 0, 0, 0, -14400),
                                              Time.new(2013, 1, 10, 23, 59, 59, -14400),
                                              ftp_download_options)
puts 'Consumption For Dates 2013/01/05 to 2013/01/10 :'
puts JSON.pretty_generate(daily_data.to_a)

# get cost by date range.
tariff = '[{
    "utility_id": 1,
    "rate_class": "R",
    "time_start": 1,
    "time_end": 12,
    "day_of_week": [1,2,3,4],
    "tariff":5
},
{
    "utility_id": 1,
    "rate_class": "R",
    "time_start": 1,
    "time_end": 12,
    "day_of_week": [5,6,7],
    "tariff":7
},
{
    "utility_id": 1,
    "rate_class": "R",
    "time_start": 13,
    "time_end": 24,
    "day_of_week": [1,2,3,4],
    "tariff":10
},
{
    "utility_id": 1,
    "rate_class": "R",
    "time_start": 13,
    "time_end": 24,
    "day_of_week": [5,6,7],
    "tariff":12
}]'
daily_data = Bucketize.cost_date_range(Time.new(2011, 2, 5, 0, 0, 0, -28800),
                                       Time.new(2011, 2, 10, 23, 59, 59, -28800),
                                       tariff,
                                       download_options)
puts 'Cost For Dates 2011/02/05 to 2011/02/10'
puts JSON.pretty_generate(daily_data.to_a)

daily_data = Bucketize.cost_date_range(Time.new(2013, 1, 5, 0, 0, 0, -14400),
                                       Time.new(2013, 1, 10, 23, 59, 59, -14400),
                                       tariff,
                                       ftp_download_options)
puts 'Cost For Dates 2013/01/05 to 2013/01/10 Using FTP'
puts JSON.pretty_generate(daily_data.to_a)
```

Other aggregation methods can be called in the manner like above.

Development

This gem was designed in a modular way to allow further modification. An expected scenario is to adapt a MapReduce solution to help analyze the data more efficiently.

The aggregation was separated to phases; each was encapsulated in a custom Enumerable that operates on the input data and produces new data.

API ==> GreenButton ==> HourlyValues ==> TariffedHourlyValues/Costs ==> DailyData ==> output data

  1. API: the GreenButton API (REST endpoint)
  2. GreenButton: The GreenButton Library that maps the API response to set of Ruby Objects
  3. HourlyValues: a custom Enumerable that takes the generated GbDataDescription and produces an hourly data. input data gaps are detected and interpolated in this stage.
  4. TariffedHourlyValues/TariffedHourlyCosts: custom Enumerables that takes the HourlyData and applies the input tariff rules on them. The output will be the values after applying the matched tariff.
  5. DailyData: a custom Enumerable that aggregates a day data from the tariffed hourly data.

Test

From the base directory of the module (the directory containing the .gemspec file.

$ bundle install
$ bundle exec rspec

Coverage reports will be found under the coverage subfolder.

GreenButton API reference

Reference this Ruby Gem Documentation

tomdoc was used to generate documentation files for this gem. The generated files are included in doc directory. documentation is generated by Yard.