dm-adapter-simpledb

What

A DataMapper adapter for Amazon’s SimpleDB service.

Features:

* Uses the RightAWS gem for efficient SimpleDB operations.
* Full set of CRUD operations
* Supports nearly all DataMapper query predicates.
* Full support for complex nested union, intersaction, and negation in queries
* Migrations
* DataMapper identity map support for record caching
* Lazy-loaded attributes
* DataMapper Serial property support via UUIDs.
* Array properties
* Basic aggregation support (Model.count("..."))
* String "chunking" permits attributes to exceed the 1024-byte limit
* Support for efficient :limit and :offset, for result set paging
* Robust quoting of names in values in selects

Note: as of version 1.0.0, this gem supports supports the DataMapper 0.10.* series and breaks backwards compatibility with DataMapper 0.9.*.

Who

Originally written by Jeremy Boles.

Contributers:

Edward Ocampo-Gooding (edward) 
Dan Mayer (danmayer)
Thomas Olausson (latompa)
Avdi Grimm (avdi)

Where

dm-adapter-simpledb is currently maintained by the Devver team and lives at: github.com/devver/dm-adapter-simpledb/

TODO

* Handle exclusive ranges natively
  Implement as inclusive range + filter step
* Tests for associations
* Option for smart lexicographical storage for numbers
  - Zero-pad integers
  - Store floats using exponential notation
* Option to store Date/Time/DateTime as ISO8601
* Full aggregate support (min/max/etc)
* Option to use libxml if available
* Parallelized queries for increased throughput
* Support of normalized 1:1 table:domain schemes that works with associations
* Sharding
* Support BatchPutAttributes
* Silence SSL warnings
  See http://pivotallabs.com/users/carl/blog/articles/1079-standup-blog-11-24-2009-model-validations-without-backing-store-associations-to-array-and-ssl-with-aws
* Token cache for reduced requests when given an offset
* Optimize key queries

Usage

Standalone

require 'rubygems'
require 'dm-core'
require 'dm-adapter-simpledb'

DataMapper.setup(:default, 
  :adapter       => 'simpledb',
  :access_key    => "ACCESS_KEY",
  :secret_key    => "SECRET_KEY",
  :domain        => "DOMAIN",
)

[Same as the following, but skip the database.yml]

In a Merb application

See sample Merb application using Merb-Auth and protected resources on SimpleDB:
http://github.com/danmayer/merb-simpledb-dm_example/tree/master

Setup database.yml with the SimpleDB DataMapper adapter:

  adapter:    simpledb
  access_key: (a 20-character, alphanumeric sequence)
  secret_key: (a 40-character sequence)
  domain:     'my_amazon_sdb_domain'

Create a model

  class Tree
    include DataMapper::Resource

    storage_name[:default] = "trees"

    property :id,   Serial
    property :name, String, :nullable => false
  end

Use interactively (with merb -i)

$ merb -i

  maple = Tree.new
  maple.name = "Acer rubrum"
  maple.save

  all_trees = Tree.all() # calls #read_all
  a_tree = Tree.first(:name => "Acer rubrum")
  yanked_tree = Tree.remote(:name => "Acer rubrum")

Running the tests

Add these two lines to your .bash_profile as the spec_helper relies on them

  $ export AMAZON_ACCESS_KEY_ID='YOUR_ACCESS_KEY'
  $ export AMAZON_SECRET_ACCESS_KEY='YOUR_SECRET_ACCESS_KEY'

Configure the domain to use for integration tests. THIS DOMAIN WILL BE
DELETED AND RECREATED BY THE TESTS, so do not choose a domain which contains
data you care about.  Configure the domain by creating a file named
THROW_AWAY_SDB_DOMAIN in the projet root:

 $ echo dm_simpledb_adapter_test > THROW_AWAY_SDB_DOMAIN

Run the tests:

  rake spec

NOTE: While every attempt has been made to make the tests robust, Amazon
SimpleDB is by it's nature an unreliable service. Sometimes it can take a
very long time for updates to be reflected by queries, and sometimes calls
just time out. If the tests fail, try them again a few times before reporting
it as a bug. Also try running the spec files individually.

Bibliography

Relating to Amazon SimpleDB developer.amazonwebservices.com/connect/entry.jspa?externalID=1292&ref=featured Approaching SimpleDB from a relational database background

Active Record Persistence with Amazon SimpleDB developer.amazonwebservices.com/connect/entry.jspa?externalID=1367&categoryID=152

Building for Performance and Reliability with Amazon SimpleDB developer.amazonwebservices.com/connect/entry.jspa?externalID=1394&categoryID=152

Query 101: Building Amazon SimpleDB Queries developer.amazonwebservices.com/connect/entry.jspa?externalID=1231&categoryID=152

Query 201: Tips & Tricks for Amazon SimpleDB Query developer.amazonwebservices.com/connect/entry.jspa?externalID=1232&categoryID=152 Latter portion describes parallelization advantages of normalized domains – the downside being the added complexity at the application layer (this library’s).

Using SimpleDB and Rails in No Time with ActiveResource developer.amazonwebservices.com/connect/entry.jspa?externalID=1242&categoryID=152 Exemplifies using the Single Table Inheritance pattern within a single SimpleDB domain by storing the model type in an attribute called ‘_resource’ and using a “SHA512 hash function on the request body combined with a timestamp and a configurable salt” for the id.

RightScale Ruby library to access Amazon EC2, S3, SQS, and SDB developer.amazonwebservices.com/connect/entry!default.jspa?categoryID=140&externalID=1014&fromSearchPage=true