Frodo - Free OData V4.0 library for Ruby

One API to rule them all

Frodo

Frodo is the little gem that access your precious OData Version 4.0 from the land of Microsoft Dynamics or other OData compliant API.

It has the ability to automatically inspect compliant APIs and expose the relevant Ruby objects dynamically.

Features include:

A clean and modular architecture using Faraday middleware responses.

Support for interacting with multiple users from different orgs.
Support for schema discovery.
Support for queryable interface.
Support for GZIP compression.
Support for Oauth authentication.

This gem supports OData Version 4.0. Support for older versions is not a goal.

If you need a gem to integration with OData Version 3, you can use James Thompson's original OData gem, upon which this gem is based. It is also is based on a Fork from (https://github.com/wrstudios/frodo) who was an attempt to OData Version 4 but seemed unfinished. Finally it uses code taken from Restforce for the client Restforce gem

Installation

Add this line to your application's Gemfile:

gem 'frodo'

And then execute:

$ bundle

Or install it yourself as:

$ gem install frodo

This gem is versioned using Semantic Versioning, so you can be confident when updating that there will not be breaking changes outside of a major version (following format MAJOR.MINOR.PATCH, so for instance moving from 3.1.0 to 4.0.0 would be allowed to include incompatible API changes). See the changelog for details on what has changed in each version.

Usage

Which authentication method you use really depends on your use case. If you're building an application where many users from different orgs are authenticated through oauth and you need to interact with data in their org on their behalf, you should use the OAuth token authentication method.

The "client credentials", and "password" flows are also supported.

It is also important to note that the client object should not be reused across different threads, otherwise you may encounter thread-safety issues.

OAuth token authentication

client = Frodo.new(oauth_token: 'access_token',
                     instance_url: 'instance url',
                     base_path: '/path/to/service')

Although the above will work, you'll probably want to take advantage of the (re)authentication middleware by specifying refresh_token, client_id, client_secret, and authentication_callback:

client = Frodo.new(oauth_token: 'access_token',
                     refresh_token: 'refresh token',
                     instance_url: 'instance url',
                     client_id: 'client_id',
                     client_secret: 'client_secret',
                     authentication_callback: Proc.new { |x| Rails.logger.debug x.to_s },
                     base_path: '/path/to/service')

The middleware will use the refresh_token automatically to acquire a new access_token if the existing access_token is invalid.

authentication_callback is a proc that handles the response from Salesforce when the refresh_token is used to obtain a new access_token. This allows the access_token to be saved for re-use later - otherwise subsequent API calls will continue the cycle of "auth failure/issue new access_token/auth success".

The proc is passed one argument, a Hash of the response, similar than the one for Dynamics API:

{
  "token_type"=>"Bearer",
  "scope"=>"user_impersonation",
  "expires_in"=>"3600",
  "ext_expires_in"=>"3600",
  "expires_on"=>"1552087545",
  "not_before"=>"1552083645",
  "resource"=>"https://myinstance.crm.dynamics.com",
  "access_token"=>"token",
  "refresh_token"=>"refresh token"
}

The id field can be used to uniquely identify the user that the access_token and refresh_token belong to.

Client credentials authentication

This should be considered experimental. At the time of this writing, it was possible to obtain an access token using this method, but it was not possible to use the access token without receiving 401 from Dynamics. Configuration is as above, but you specify client_id, client_secret, and tenant_id:

client = Frodo.new({
  instance_url: 'instance url',
  client_id: 'client_id',
  client_secret: 'client_secret',
  tenant_id: 'tenant_id',
  authentication_callback: Proc.new { |x| Rails.logger.debug x.to_s },
  base_path: '/path/to/service'
})

User password authentication

As above, but you specify client_id, tenant_id, username, and password:

client = Frodo.new({
  instance_url: 'instance url',
  client_id: 'client_id',
  tenant_id: 'tenant_id',
  username: 'username',
  password: 'password',
  authentication_callback: Proc.new { |x| Rails.logger.debug x.to_s },
  base_path: '/path/to/service'
})

Proxy Support

You can specify a HTTP proxy using the proxy_uri option, as follows, or by setting the FRODATA_PROXY_URI environment variable:

client = Frodo.new(username: 'foo',
                       password: 'bar',
                       security_token: 'security token',
                       client_id: 'client_id',
                       client_secret: 'client_secret',
                       proxy_uri: 'http://proxy.example.com:123',
                       base_path: '/path/to/service')

You may specify a username and password for the proxy with a URL along the lines of 'http://user:[email protected]:123'.

Global configuration

You can set any of the options passed into Frodo.new globally:

Frodo.configure do |config|
  config.client_id     = 'foo'
  config.client_secret = 'bar'
end

Bang! methods

All the CRUD methods (create, update, upsert, destroy) have equivalent methods with a ! at the end (create!, update!, upsert!, destroy!), which can be used if you need to do some custom error handling. The bang methods will raise exceptions, while the non-bang methods will return false in the event that an exception is raised. This works similarly to ActiveRecord.

Custom Headers

You service may need custom headers. Frodo allows the addition of custom headers in REST API requests to trigger specific logic. In order to pass any custom headers along with API requests, you can specify a hash of :request_headers upon client initialization. The example below demonstrates how to include the myheader header in all client HTTP requests:

client = Frodo.new(oauth_token: 'access_token',
                       instance_url: 'instance url',
                       request_headers: { 'myheader' => 'FALSE' })

Client API

metadata

This will provide the XML schema for the service. This is also called automatically the first time you access most of the api in the client and cached in memory. For better performance, see the section below on Services & the Service Registry

# Get the global describe for all sobjects
client.metadata
# => <xml>...</xml>

Queries

Queries in general can be speficied directly as a string as such

or you can use the Frodo::Query. Frodo::Query instances form the base for finding specific entities within an Frodo::EntitySet. A query object exposes a number of capabilities based on the System Query Options provided for in the OData V4.0 specification. Below is just a partial example of what is possible:

  query = client.service['Products'].query
  query.where(query[:Price].lt(15))
  query.where(query[:Rating].gt(3))
  query.limit(3)
  query.skip(2)
  query.order_by("Name")
  query.select("Name,CreatedBy")
  query.inline_count
  results = query.execute
  results.each {|product| puts product['Name']}

The process of querying is kept purposely verbose to allow for lazy behavior to be implemented at higher layers. Internally, Frodo::Query relies on the Frodo::Query::Criteria for the way the where method works. You should refer to the published RubyDocs for full details on the various capabilities:

products = client.query("Products?$filter=name eq 'somename'")
# => [#<Frodo::Entity>]

# or the equivalent using a query object
query_object = client.service['Products'].query
query_object.where("name eq 'yo'")
products = client.query(query_object)
# => [#<Frodo::Entity>]

Find

# Select an account from an Accounts set with primary key set to '001D000000INjVe'

client.find('Accounts', '001D000000INjVe')
# => #<Frodo::Entity accountid="001D000000INjVe" name="Test" ... >

select

select allows the fetching of a specific list of fields from a single object. Only selected fields will be populated is much faster.

# Select the `name` column from an Account entity in the Accounts set with primary key set to '001D000000INjVe'

client.select('Accounts', '001D000000INjVe', ["name"])
# => # => #<Frodo::Entity accountid="001D000000INjVe" name="Name" other_field="nil" ... >

create

# Add a new account
client.create('Accounts', Name: 'Foobar Inc.')
# => '0016000000MRatd'

update

# Update the Account with `Id` '0016000000MRatd'
client.update('Accounts', Id: '0016000000MRatd', Name: 'Whizbang Corp')
# => true

destroy

# Delete the Account with `Id` '0016000000MRatd'
client.destroy('Accounts', '0016000000MRatd')
# => true

Count

  client.count('Accounts')
  # => 3

Services & the Service Registry

The Frodo gem provides a number of core classes, the two most basic ones are the Frodo::Service and the Frodo::ServiceRegistry. The only time you will need to worry about the Frodo::ServiceRegistry is when you have multiple Frodo services you are interacting with that you want to keep straight easily. The nice thing about Frodo::Service is that it automatically registers with the registry on creation, so there is no manual interaction with the registry necessary.

To create an Frodo::Service simply provide the location of a service endpoint to it like this:

  Frodo::Service.new('http://services.odata.org/V4/OData/OData.svc')

You may also provide an options hash after the URL. It is suggested that you supply a name for the service via this hash like so:

  Frodo::Service.new('http://services.odata.org/V4/OData/OData.svc', name: 'ODataDemo')

For more information regarding available options and how to configure a service instance, refer to Service Configuration below.

This one call will setup the service and allow for the discovery of everything the other parts of the Frodo gem need to function. The two methods you will want to remember from Frodo::Service are #service_url and #name. Both of these methods are available on instances and will allow for lookup in the Frodo::ServiceRegistry, should you need it.

Using either the service URL or the name provided as an option when creating an Frodo::Service will allow for quick lookup in the Frodo::ServiceRegistry like such:

  Frodo::ServiceRegistry['http://services.odata.org/V4/OData/OData.svc']
  Frodo::ServiceRegistry['ODataDemo']

Both of the above calls would retrieve the same service from the registry. At the moment there is no protection against name collisions provided in Frodo::ServiceRegistry. So, looking up services by their service URL is the most exact method, but lookup by name is provided for convenience.

Service Configuration

Metadata File

Typically the metadata file of a service can be quite large. You can speed your load time by forcing the service to load the metadata from a file rather than a URL. This is only recommended for testing purposes, as the metadata file can change.

  service = Frodo::Service.new('http://services.odata.org/V4/OData/OData.svc', {
    name: 'ODataDemo',
    metadata_file: "metadata.xml",
  })

Exploring a Service

Once instantiated, you can request various information about the service, such as the names and types of entity sets it exposes, or the names of the entity types (and custom datatypes) it defines.

For example:

Get a list of available entity types

  client.service.entity_types
  # => [
  #   "ODataDemo.Product",
  #   "ODataDemo.FeaturedProduct",
  #   "ODataDemo.ProductDetail",
  #   "ODataDemo.Category",
  #   "ODataDemo.Supplier",
  #   "ODataDemo.Person",
  #   "ODataDemo.Customer",
  #   "ODataDemo.Employee",
  #   "ODataDemo.PersonDetail",
  #   "ODataDemo.Advertisement"
  # ]

Get a list of entity sets

  client.service.entity_sets
  # => {
  #   "Products"       => "ODataDemo.Product",
  #   "ProductDetails" => "ODataDemo.ProductDetail",
  #   "Categories"     => "ODataDemo.Category",
  #   "Suppliers"      => "ODataDemo.Supplier",
  #   "Persons"        => "ODataDemo.Person",
  #   "PersonDetails"  => "ODataDemo.PersonDetail",
  #   "Advertisements" => "ODataDemo.Advertisement"
  # }

Get a list of complex types

  client.service.complex_types
  # => ["ODataDemo.Address"]

Get a list of enum types

  client.service.enum_types
  # => ["ODataDemo.ProductStatus"]

For more examples, refer to usage_example_specs.rb.

Entity Sets

When it comes to reading data from an OData service the most typical way will be via Frodo::EntitySet instances. Under normal circumstances you should never need to worry about an Frodo::EntitySet directly. For example, to get an Frodo::EntitySet for the products in the ODataDemo service simply access the entity set through the service like this:

  service = Frodo::Service.new('http://services.odata.org/V4/OData/OData.svc')
  products = service['ProductsSet'] # => Frodo::EntitySet

You can get a list of all your entity sets like this:

  service.entity_sets

Entities

Frodo::Entity instances represent individual entities, or records, in a given service. They are returned primarily through interaction with instances of Frodo::EntitySet. You can access individual properties on an Frodo::Entity like so:

  product = products.first # => Frodo::Entity
  product['Name']  # => 'Bread'
  product['Price'] # => 2.5 (Float)

Individual properties on an Frodo::Entity are automatically typecast by the gem, so you don't have to worry about too much when working with entities.

You can get a list of all your entities like this:

  service.entity_types

Entity Properties

Reading, parsing and instantiating all properties of an entity can add up to a significant amount of time, particularly for those entities with a large number of properties. To speed this process up all properties are lazy loaded. Which means it will store the name of the property, but will not parse and instantiate the property until you want to use it.

You can find all the property names of your entity with

  product.property_names

You can grab the parsed value of the property as follows:

  product["Name"]

or, you can get a hold of the property class instance using

  product.get_property("Name")

This will parse and instantiate the property if it hasn't done so yet.

Lenient Property Validation

By default, we use strict property validation, meaning that any property validation errors in the data will raise an exception. However, you may encounter OData implementations in the wild that break the specs in strange and surprising ways (shocking, I know!).

Since it's often better to get some data instead of nothing at all, you can optionally make the property validation lenient. Simply add strict: false to the service constructor options. In this mode, any property validation error will log a warning instead of raising an exception. The corresponding property value will be nil (even if the property is declared as not allowing NULL values).

  service = Frodo::Service.new('http://services.odata.org/V4/OData/OData.svc', strict: false)
  # -- alternatively, for an existing service instance --
  service.options[:strict] = false

Release on rubygems.org

There is a rake task rake release, which will do everything for you.

Contributing

Fork it (https://github.com/[my-github-username]/frodo/fork)
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create a new Pull Request

Credits

Many thanks go to James Thompson, who wrote the original OData (Version 3.0) gem.

Many thanks go to Christoph Wagner, who started the work on the OData (Version 4.0) gem.

Also, I would like to thank Outreach for generously allowing me to work on Open Source software like this. If you want to work on interesting challenges with an awesome team, check out our open positions.