Ceph::Storage

A userspace wrapper around the http://github.com/ceph/ceph-ruby/ library to allow applications to pool their requests to the ceph backend and conduct thread safe operations with ceph

Installation

Add this line to your appication's Gemfile gem 'ceph_storage'

And then execute: $ bundle

Or install it yourself as: $gem install ceph_storage

About

The Ceph-ruby gem provides an FFI wrapper around librados in order to facilitate connections to ceph and to provide an easy management of objects and cluster settings. There are some limitations of this model in that when using it to download large amounts of files in many scenarios there can be significant overhead of connecting to the cluster.

This library uses the Facets::Multiton model in order to ensure that if you pass the same attributes into cluster and pool objects, you will always get the same objects back.

Pool collection

In order to provide thread safety, the Pool object carries with it a collection of ::CephRuby::Pool objects. As such operations on this will reuse an unused connection or spawn a new pool connection if no free one exists. On completion (or push) the pool object will continue to stay open to serve further objects.

Usage

require 'ceph_storage'
config = { cluster: 'ceph', user: 'client.admin', flags: 0, config_dir: '/etc/ceph' } # defaults

cluster1 = CephStorage::ClusterFactory.build(config)

cluster2 = CephStorage::ClusterFactory.build(config)

cluster1.equal? cluster2 # => true

pool1 = cluster1.pool('foo') 

pool2 = cluster2.pool('foo')

pool1.equal? pool2 # => true


pool1.storage_object('bar') do |file|
  puts file.read_file
end

Using blocks

# Will connect to ceph, open a pool, download the contents 
# of a file
CephStorage::ClusterFactory.build(config) do |c|
  c.pool('foo') do |p|
    # NB Storage_object as opposed to rados_object
    p.storage_object('bar') |f|
      puts f.read_file
    end
  end
end

# Does the same thing, but should reuse the same objects
# Hence the only latency is downloading the file contents
CephStorage::ClusterFactory.build(config) do |c|
  c.pool('foo') do |p|
    p.storage_object('bar') |f|
      puts f.read_file
    end
  end
end

Raw Pool object

If there is some functionality that you need that isn't provided, you can expose the raw ::CephRuby::Pool objects by calling pool.rados_pool. This will return an object and mark it 'in use' until you push it back into the collection via rados_object= Be sure to do this lest your memory leak.

Storage Object interfaces

The StorageObject interface provides some additional functionality to ceph on top of the CephRuby bindings

• CephStorage::StorageObject::FileStorageObject (wrapper for ::File)
• CephStorage::StorageObject::URLStorageObject (wrapper for url-open)

These classes are designed to allow you to copy and move files from ceph to other storage mechanisms. It is designed to be extensible

f_storage_obj1 = CephStorage::StorageObject::FileStorageObject.new 'path/to/file1'
f_storage_obj2 = CephStorage::StorageObject::URLStorageObject.new 'http://example.com'

CephStorage::ClusterFactory.build(config) do |c|
  c.pool('foo') do |p|
    p.storage_object('bar') |f|
      f.move f_storage_obj1
    end
  end
end

CephStorage::ClusterFactory.build(config) do |c|
  c.pool('foo') do |p|
    p.storage_object('baz') |f|
      f_storage_obj1.copy f
    end
  end
end

In order for this to work, you class should extend CephStorage::StorageObject and implement read_file and write_file methods. URLStorageObject will raise exception for write_file as not implemented (hence move will not work).