Class: DynamoAutoscale::TableTracker
- Inherits:
-
Object
- Object
- DynamoAutoscale::TableTracker
- Includes:
- Logger
- Defined in:
- lib/dynamo-autoscale/table_tracker.rb
Constant Summary collapse
- TIME_WINDOW =
TODO: This time window may need changing.
7.days
Instance Attribute Summary collapse
-
#data ⇒ Object
readonly
Returns the value of attribute data.
-
#name ⇒ Object
readonly
Returns the value of attribute name.
-
#scale_events ⇒ Object
readonly
Returns the value of attribute scale_events.
-
#triggered_rules ⇒ Object
readonly
Returns the value of attribute triggered_rules.
Instance Method Summary collapse
-
#all_times ⇒ Object
Returns an array of all of the time points that have data present in them.
- #clear_data ⇒ Object
-
#earliest_data_time ⇒ Object
Returns the earliest point in time that we have tracked data for.
- #graph!(opts = {}) ⇒ Object
-
#initialize(name) ⇒ TableTracker
constructor
A new instance of TableTracker.
-
#last(value, metric) ⇒ Object
Useful method for querying the last N points, or the last points in a time range.
-
#last_consumed_for(metric, opts = {}) ⇒ Object
Gets the last amount of consumed throughput for whatever metric you pass in.
-
#last_provisioned_for(metric, opts = {}) ⇒ Object
Gets the last amount of provisioned throughput for whatever metric you pass in.
-
#latest_data_time ⇒ Object
Returns the latest point in time that we have tracked data for.
- #lost_read_percent ⇒ Object
-
#lost_read_units ⇒ Object
Whenever the consumed units goes above the provisioned, we refer to the overflow as “lost” units.
- #lost_write_percent ⇒ Object
-
#lost_write_units ⇒ Object
Whenever the consumed units goes above the provisioned, we refer to the overflow as “lost” units.
- #report! ⇒ Object
- #scatterplot_for!(metric) ⇒ Object
-
#tick(time, datum) ⇒ Object
‘tick` takes two arguments.
-
#to_csv!(opts = {}) ⇒ Object
Pricing is pretty difficult.
- #total_read_units ⇒ Object
- #total_write_units ⇒ Object
- #wasted_read_percent ⇒ Object
-
#wasted_read_units ⇒ Object
Calculate how many read units have been wasted in the current set of tracked data.
- #wasted_write_percent ⇒ Object
-
#wasted_write_units ⇒ Object
Calculate how many write units have been wasted in the current set of tracked data.
Methods included from Logger
included, logger, #logger, logger=
Constructor Details
#initialize(name) ⇒ TableTracker
Returns a new instance of TableTracker.
10 11 12 13 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 10 def initialize name @name = name clear_data end |
Instance Attribute Details
#data ⇒ Object (readonly)
Returns the value of attribute data.
8 9 10 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 8 def data @data end |
#name ⇒ Object (readonly)
Returns the value of attribute name.
8 9 10 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 8 def name @name end |
#scale_events ⇒ Object (readonly)
Returns the value of attribute scale_events.
8 9 10 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 8 def scale_events @scale_events end |
#triggered_rules ⇒ Object (readonly)
Returns the value of attribute triggered_rules.
8 9 10 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 8 def triggered_rules @triggered_rules end |
Instance Method Details
#all_times ⇒ Object
Returns an array of all of the time points that have data present in them. Example:
table.tick(Time.now, { ... })
table.tick(Time.now, { ... })
table.all_times
#=> Array with the 2 time values above in it
225 226 227 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 225 def all_times @data.keys end |
#clear_data ⇒ Object
15 16 17 18 19 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 15 def clear_data @data = RBTree.new @triggered_rules = RBTree.new @scale_events = RBTree.new end |
#earliest_data_time ⇒ Object
Returns the earliest point in time that we have tracked data for.
230 231 232 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 230 def earliest_data_time all_times.first end |
#graph!(opts = {}) ⇒ Object
271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 271 def graph! opts = {} data_tmp = File.join(Dir.tmpdir, 'data.csv') png_tmp = opts[:path] || File.join(Dir.tmpdir, 'graph.png') r_script = File.join(DynamoAutoscale.root, 'rlib', 'dynamodb_graph.r') to_csv!(path: data_tmp) `r --no-save --args #{data_tmp} #{png_tmp} < #{r_script}` if $? != 0 logger.error "[table] Failed to create graph." return false else if opts[:open] `open #{png_tmp}` if $? != 0 logger.error "[table] Failed to open graph." return false else return png_tmp end end end end |
#last(value, metric) ⇒ Object
Useful method for querying the last N points, or the last points in a time range. For example:
table.last 5. :consumed_writes
#=> [ array of last 5 data points ]
table.last 5.minutes, :provisioned_reads
#=> [ array containing the last 5 minutes of provisioned read data ]
If there are no points present, or no points in your time range, the return value will be an empty array.
113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 113 def last value, metric if value.is_a? ActiveSupport::Duration value = value.to_i to_return = [] now = Time.now.to_i @data.reverse_each do |time, datum| value -= now - time.to_i now = time.to_i break if value < 0 to_return << datum[metric] end to_return else @data.reverse_each.take(value).map { |time, datum| datum[metric] } end end |
#last_consumed_for(metric, opts = {}) ⇒ Object
Gets the last amount of consumed throughput for whatever metric you pass in. Example:
table.last_consumed_for :writes
#=> 54.3456
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 85 def last_consumed_for metric, opts = {} key = case metric when :reads, :provisioned_reads, :consumed_reads :consumed_reads when :writes, :provisioned_writes, :consumed_writes :consumed_writes end @data.reverse_each do |time, datum| if opts[:at].nil? or time <= opts[:at] return datum[key] if datum[key] end end return nil end |
#last_provisioned_for(metric, opts = {}) ⇒ Object
Gets the last amount of provisioned throughput for whatever metric you pass in. Example:
table.last_provisioned_for :writes
#=> 600.0
63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 63 def last_provisioned_for metric, opts = {} key = case metric when :reads, :provisioned_reads, :consumed_reads :provisioned_reads when :writes, :provisioned_writes, :consumed_writes :provisioned_writes end @data.reverse_each do |time, datum| if opts[:at].nil? or time <= opts[:at] return datum[key] if datum[key] end end return nil end |
#latest_data_time ⇒ Object
Returns the latest point in time that we have tracked data for.
235 236 237 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 235 def latest_data_time all_times.last end |
#lost_read_percent ⇒ Object
213 214 215 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 213 def lost_read_percent (lost_read_units / total_read_units) * 100.0 end |
#lost_read_units ⇒ Object
Whenever the consumed units goes above the provisioned, we refer to the overflow as “lost” units.
165 166 167 168 169 170 171 172 173 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 165 def lost_read_units @data.inject(0) do |memo, (_, datum)| if datum[:consumed_reads] > datum[:provisioned_reads] memo += datum[:consumed_reads] - datum[:provisioned_reads] end memo end end |
#lost_write_percent ⇒ Object
209 210 211 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 209 def lost_write_percent (lost_write_units / total_write_units) * 100.0 end |
#lost_write_units ⇒ Object
Whenever the consumed units goes above the provisioned, we refer to the overflow as “lost” units.
177 178 179 180 181 182 183 184 185 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 177 def lost_write_units @data.inject(0) do |memo, (_, datum)| if datum[:consumed_writes] > datum[:provisioned_writes] memo += datum[:consumed_writes] - datum[:provisioned_writes] end memo end end |
#report! ⇒ Object
312 313 314 315 316 317 318 319 320 321 322 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 312 def report! puts " Table: #{name}" puts "Wasted r/units: #{wasted_read_units.round(2)} (#{wasted_read_percent.round(2)}%)" puts " Total r/units: #{total_read_units.round(2)}" puts " Lost r/units: #{lost_read_units.round(2)} (#{lost_read_percent.round(2)}%)" puts "Wasted w/units: #{wasted_write_units.round(2)} (#{wasted_write_percent.round(2)}%)" puts " Total w/units: #{total_write_units.round(2)}" puts " Lost w/units: #{lost_write_units.round(2)} (#{lost_write_percent.round(2)}%)" puts " Upscales: #{DynamoAutoscale.actioners[self].upscales}" puts " Downscales: #{DynamoAutoscale.actioners[self].downscales}" end |
#scatterplot_for!(metric) ⇒ Object
296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 296 def scatterplot_for! metric data_tmp = File.join(Dir.tmpdir, 'data.csv') png_tmp = File.join(Dir.tmpdir, 'boxplot.png') r_script = File.join(DynamoAutoscale.root, 'rlib', 'dynamodb_boxplot.r') to_csv!(data_tmp) `r --no-save --args #{data_tmp} #{png_tmp} < #{r_script}` if $? != 0 logger.error "[table] Failed to create graph." else `open #{png_tmp}` end end |
#tick(time, datum) ⇒ Object
‘tick` takes two arguments. The first is a Time object, the second is a hash. The tick method expects data in the following format for the second argument:
{:provisioned_writes=>600.0,
:provisioned_reads=>800.0,
:consumed_writes=>52.693333333333335,
:consumed_reads=>342.4033333333333}
29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 29 def tick time, datum if time < (Time.now.utc - TIME_WINDOW) logger.warn "[table] Attempted to insert data outside of the time window." return end # Sometimes there are gaps in the data pertaining to provisioned # amounts. These two conditional blocks fill in those gaps. if datum[:provisioned_writes].nil? datum[:provisioned_writes] = last_provisioned_for :writes, at: time if datum[:provisioned_writes] logger.debug "[table] Filled in gap in provisioned writes." end end if datum[:provisioned_reads].nil? datum[:provisioned_reads] = last_provisioned_for :reads, at: time if datum[:provisioned_reads] logger.debug "[table] Filled in gap in provisioned reads." end end @data[time] = datum remove_expired_data! @data remove_expired_data! @triggered_rules remove_expired_data! @scale_events end |
#to_csv!(opts = {}) ⇒ Object
245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 245 def to_csv! opts = {} path = opts[:path] or File.join(DynamoAutoscale.root, "#{self.name}.csv") CSV.open(path, 'w') do |csv| csv << [ "time", "provisioned_reads", "provisioned_writes", "consumed_reads", "consumed_writes", ] @data.each do |time, datum| csv << [ time.iso8601, datum[:provisioned_reads], datum[:provisioned_writes], datum[:consumed_reads], datum[:consumed_writes], ] end end path end |
#total_read_units ⇒ Object
187 188 189 190 191 192 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 187 def total_read_units @data.inject(0) do |memo, (_, datum)| memo += datum[:provisioned_reads] if datum[:provisioned_reads] memo end end |
#total_write_units ⇒ Object
194 195 196 197 198 199 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 194 def total_write_units @data.inject(0) do |memo, (_, datum)| memo += datum[:provisioned_writes] if datum[:provisioned_writes] memo end end |
#wasted_read_percent ⇒ Object
201 202 203 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 201 def wasted_read_percent (wasted_read_units / total_read_units) * 100.0 end |
#wasted_read_units ⇒ Object
Calculate how many read units have been wasted in the current set of tracked data.
table.wasted_read_units
#=> 244.4
138 139 140 141 142 143 144 145 146 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 138 def wasted_read_units @data.inject(0) do |memo, (_, datum)| # if datum[:provisioned_reads] and datum[:consumed_reads] memo += datum[:provisioned_reads] - datum[:consumed_reads] # end memo end end |
#wasted_write_percent ⇒ Object
205 206 207 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 205 def wasted_write_percent (wasted_write_units / total_write_units) * 100.0 end |
#wasted_write_units ⇒ Object
Calculate how many write units have been wasted in the current set of tracked data.
table.wasted_write_units
#=> 566.3
153 154 155 156 157 158 159 160 161 |
# File 'lib/dynamo-autoscale/table_tracker.rb', line 153 def wasted_write_units @data.inject(0) do |memo, (_, datum)| # if datum[:provisioned_writes] and datum[:consumed_writes] memo += datum[:provisioned_writes] - datum[:consumed_writes] # end memo end end |