Module: Dynamoid::Adapter::AwsSdk

Extended by:
AwsSdk
Included in:
AwsSdk
Defined in:
lib/dynamoid/adapter/aws_sdk.rb

Overview

The AwsSdk adapter provides support for the AWS-SDK for Ruby gem. More information is available at that Gem’s Github page: github.com/amazonwebservices/aws-sdk-for-ruby

Constant Summary collapse

@@connection = 
nil

Instance Method Summary collapse

Instance Method Details

#batch_delete_item(options) ⇒ Object

Delete many items at once from DynamoDB. More efficient than delete each item individually.

or

Dynamoid::Adapter::AwsSdk.batch_delete_item('table1' => [['hk1', 'rk2'], ['hk1', 'rk2']]]))

Examples:

Delete IDs 1 and 2 from the table testtable

Dynamoid::Adapter::AwsSdk.batch_delete_item('table1' => ['1', '2'])

Parameters:

  • options (Hash) —

    the hash of tables and IDs to delete

Returns:

  • nil



90
91
92
93
94
95
96
97
98
99
100
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 90

def batch_delete_item(options)
  return nil if options.all?{|k, v| v.empty?}
  options.each do |t, ids|
    Array(ids).in_groups_of(25, false) do |group|
      batch = AWS::DynamoDB::BatchWrite.new(:config => @@connection.config)
      batch.delete(t,group)
      batch.process!          
    end
  end
  nil
end

#batch_get_item(table_ids, options = {}) ⇒ Hash

Get many items at once from DynamoDB. More efficient than getting each item individually.

Examples:

Retrieve IDs 1 and 2 from the table testtable

Dynamoid::Adapter::AwsSdk.batch_get_item({'table1' => ['1', '2']}, :consistent_read => true)

Parameters:

  • table_ids (Hash) —

    the hash of tables and IDs to retrieve

  • options (Hash) (defaults to: {}) —

    to be passed to underlying BatchGet call

Returns:

  • (Hash) —

    a hash where keys are the table names and the values are the retrieved items

Since:

  • 0.2.0



64
65
66
67
68
69
70
71
72
73
74
75
76
77
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 64

def batch_get_item(table_ids, options = {})
  hash = Hash.new{|h, k| h[k] = []}
  return hash if table_ids.all?{|k, v| v.empty?}
  table_ids.each do |t, ids|
    Array(ids).in_groups_of(100, false) do |group|
      batch = AWS::DynamoDB::BatchGet.new(:config => @@connection.config)
      batch.table(t, :all, Array(group), options) unless group.nil? || group.empty?
      batch.each do |table_name, attributes|
        hash[table_name] << attributes.symbolize_keys!
      end
    end
  end
  hash
end

#connect! ⇒ AWS::DynamoDB::Connection

Establish the connection to DynamoDB.

Call DynamoDB new, with no parameters. Make sure the aws.yml file or aws.rb file, refer the link for more details. github.com/amazonwebservices/aws-sdk-for-ruby

  1. Create config/aws.yml as follows:

Fill in your AWS Access Key ID and Secret Access Key aws.amazon.com/security-credentials access_key_id: REPLACE_WITH_ACCESS_KEY_ID secret_access_key: REPLACE_WITH_SECRET_ACCESS_KEY (or) 2, Create config/initializers/aws.rb as follows: load the libraries require ‘aws’ log requests using the default rails logger AWS.config(:logger => Rails.logger) load credentials from a file config_path = File.expand_path(File.dirname(__FILE__)+“/../aws.yml”) AWS.config(YAML.load(File.read(config_path))) Additionally include any of the dynamodb paramters as needed (eg: if you would like to change the dynamodb endpoint, then add the parameter in the following paramter in the file aws.yml or aws.rb dynamo_db_endpoint : dynamodb.ap-southeast-1.amazonaws.com)

Returns:

  • (AWS::DynamoDB::Connection) —

    the raw DynamoDB connection

Since:

  • 0.2.0



40
41
42
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 40

def connect!
@@connection = AWS::DynamoDB.new
end

#connection ⇒ AWS::DynamoDB::Connection

Return the established connection.

Returns:

  • (AWS::DynamoDB::Connection) —

    the raw DynamoDB connection

Since:

  • 0.2.0



49
50
51
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 49

def connection
  @@connection
end

#count(table_name) ⇒ Integer

Number of items from a table

Parameters:

  • table_name (String) —

    the name of the table

Returns:

  • (Integer) —

    the number of items from a table

Since:

  • 0.6.1



281
282
283
284
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 281

def count(table_name)
  table = get_table(table_name)
  table.items.count
end

#create_table(table_name, key = :id, options = {}) ⇒ Object

Create a table on DynamoDB. This usually takes a long time to complete.

Parameters:

  • table_name (String) —

    the name of the table to create

  • key (Symbol) (defaults to: :id) —

    the table’s primary key (defaults to :id)

  • options (Hash) (defaults to: {}) —

    provide a range_key here if you want one for the table

Since:

  • 0.2.0



109
110
111
112
113
114
115
116
117
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 109

def create_table(table_name, key = :id, options = {})
  Dynamoid.logger.info "Creating #{table_name} table. This could take a while."
  options[:hash_key] ||= {key.to_sym => :string}
  read_capacity = options[:read_capacity] || Dynamoid::Config.read_capacity
  write_capacity = options[:write_capacity] || Dynamoid::Config.write_capacity
  table = @@connection.tables.create(table_name, read_capacity, write_capacity, options)
  sleep 0.5 while table.status == :creating
  return table
end

#delete_item(table_name, key, options = {}) ⇒ Object

Removes an item from DynamoDB.

Parameters:

  • table_name (String) —

    the name of the table

  • key (String) —

    the hash key of the item to delete

  • range_key (Number) —

    the range key of the item to delete, required if the table has a composite key

Since:

  • 0.2.0



126
127
128
129
130
131
132
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 126

def delete_item(table_name, key, options = {})
  range_key = options.delete(:range_key)
  table = get_table(table_name)
  result = table.items.at(key, range_key)
  result.delete unless result.attributes.to_h.empty?
  true
end

#delete_table(table_name) ⇒ Object

Deletes an entire table from DynamoDB. Only 10 tables can be in the deleting state at once, so if you have more this method may raise an exception.

Parameters:

  • table_name (String) —

    the name of the table to destroy

Since:

  • 0.2.0



140
141
142
143
144
145
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 140

def delete_table(table_name)
  Dynamoid.logger.info "Deleting #{table_name} table. This could take a while."
  table = @@connection.tables[table_name]
  table.delete
  sleep 0.5 while table.exists? == true
end

#get_item(table_name, key, options = {}) ⇒ Hash

Fetches an item from DynamoDB.

Parameters:

  • table_name (String) —

    the name of the table

  • key (String) —

    the hash key of the item to find

  • range_key (Number) —

    the range key of the item to find, required if the table has a composite key

Returns:

  • (Hash) —

    a hash representing the raw item in DynamoDB

Since:

  • 0.2.0



158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 158

def get_item(table_name, key, options = {})
  range_key = options.delete(:range_key)
  table = get_table(table_name)

  result = table.items.at(key, range_key).attributes.to_h(options)

  if result.empty?
    nil
  else
    result.symbolize_keys!
  end
end

#get_table(table_name) ⇒ Object

TODO:

Add an UpdateTable method.



261
262
263
264
265
266
267
268
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 261

def get_table(table_name)
  unless table = table_cache[table_name]
    table = @@connection.tables[table_name]
    table.load_schema
    table_cache[table_name] = table
  end
  table
end

#list_tables ⇒ Object

List all tables on DynamoDB.

Since:

  • 0.2.0



184
185
186
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 184

def list_tables
  @@connection.tables.collect(&:name)
end

#put_item(table_name, object, options = nil) ⇒ Object

Persists an item on DynamoDB.

Parameters:

  • table_name (String) —

    the name of the table

  • object (Object) —

    a hash or Dynamoid object to persist

Since:

  • 0.2.0



194
195
196
197
198
199
200
201
202
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 194

def put_item(table_name, object, options = nil)
  table = get_table(table_name)
  table.items.create(
    object.delete_if{|k, v| v.nil? || (v.respond_to?(:empty?) && v.empty?)},
    options || {}
  )
rescue AWS::DynamoDB::Errors::ConditionalCheckFailedException => e
  raise Dynamoid::Errors::ConditionalCheckFailedException        
end

#query(table_name, opts = {}) ⇒ Enumerator

Query the DynamoDB table. This employs DynamoDB’s indexes so is generally faster than scanning, but is only really useful for range queries, since it can only find by one hash key at once. Only provide one range key to the hash.

Parameters:

  • table_name (String) —

    the name of the table

  • opts (Hash) (defaults to: {}) —

    the options to query the table with

Options Hash (opts):

  • :hash_value (String) —

    the value of the hash key to find

  • :range_value (Range) —

    find the range key within this range

  • :range_greater_than (Number) —

    find range keys greater than this

  • :range_less_than (Number) —

    find range keys less than this

  • :range_gte (Number) —

    find range keys greater than or equal to this

  • :range_lte (Number) —

    find range keys less than or equal to this

Returns:

  • (Enumerator) —

    an iterator of all matching items

Since:

  • 0.2.0



220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 220

def query(table_name, opts = {})
  table = get_table(table_name)

  Enumerator.new do |yielder|
    consistent_opts = { :consistent_read => opts[:consistent_read] || false }
    if table.composite_key?
      table.items.query(opts).each do |data|
        if opts.has_key? :select
          yielder.yield data.attributes.symbolize_keys!
        else
          yielder.yield data.attributes.to_h(consistent_opts).symbolize_keys!
        end
      end
    else
      yielder.yield get_item(table_name, opts[:hash_value])
    end
  end
end

#scan(table_name, scan_hash, select_opts) ⇒ Enumerator

Scan the DynamoDB table. This is usually a very slow operation as it naively filters all data on the DynamoDB servers.

Parameters:

  • table_name (String) —

    the name of the table

  • scan_hash (Hash) —

    a hash of attributes: matching records will be returned by the scan

Returns:

  • (Enumerator) —

    an iterator of all matching items

Since:

  • 0.2.0



248
249
250
251
252
253
254
255
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 248

def scan(table_name, scan_hash, select_opts)
  table = get_table(table_name)
  Enumerator.new do |yielder|
    table.items.where(scan_hash).select(select_opts).each do |data|
      yielder.yield data.attributes.symbolize_keys!
    end
  end
end

#table_cache ⇒ Object 



270
271
272
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 270

def table_cache
  @table_cache ||= {}
end

#update_item(table_name, key, options = {}, &block) ⇒ Object 



171
172
173
174
175
176
177
178
179
 
# File 'lib/dynamoid/adapter/aws_sdk.rb', line 171

def update_item(table_name, key, options = {}, &block)
  range_key = options.delete(:range_key)
  conditions = options.delete(:conditions) || {}
  table = get_table(table_name)
  item = table.items.at(key, range_key)
  item.attributes.update(conditions.merge(:return => :all_new), &block)
rescue AWS::DynamoDB::Errors::ConditionalCheckFailedException
  raise Dynamoid::Errors::ConditionalCheckFailedException
end