Class: MogileFS::Mysql

Inherits:
Object
  • Object
show all
Defined in:
lib/mogilefs/mysql.rb

Overview

read-only interface that can be a backend for MogileFS::MogileFS

This provides direct, read-only access to any slave MySQL database to provide better performance, scalability and eliminate mogilefsd as a point of failure

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(args = {}) ⇒ Mysql

Creates a new MogileFS::Mysql instance. args must include a key :domain specifying the domain of this client and :mysql, specifying an already-initialized Mysql object.

The Mysql object can be either the standard Mysql driver or the Mysqlplus one supporting c_async_query.



22
23
24
25
26
27
 
# File 'lib/mogilefs/mysql.rb', line 22

def initialize(args = {})
  @my = args[:mysql]
  @query_method = @my.respond_to?(:c_async_query) ? :c_async_query : :query
  @last_update_device = @last_update_domain = Time.at(0)
  @cache_domain = @cache_device = nil
end

Instance Attribute Details

#my ⇒ Object (readonly)

Returns the value of attribute my.



12
13
14
 
# File 'lib/mogilefs/mysql.rb', line 12

def my
  @my
end

#query_method ⇒ Object (readonly)

Returns the value of attribute query_method.



13
14
15
 
# File 'lib/mogilefs/mysql.rb', line 13

def query_method
  @query_method
end

Instance Method Details

#_get_paths(params = {}) ⇒ Object

Get the paths for key.



84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
 
# File 'lib/mogilefs/mysql.rb', line 84

def _get_paths(params = {})
  zone = params[:zone]
  noverify = (params[:noverify] == 1) # TODO this is unused atm
  dmid = get_dmid(params[:domain])
  devices = refresh_device or raise MogileFS::Backend::NoDevicesError
  urls = []
  sql = <<-EOS
  SELECT fid FROM file
  WHERE dmid = #{dmid} AND dkey = '#{@my.quote(params[:key])}'
  LIMIT 1
  EOS

  res = query(sql).fetch_row
  res && res[0] or raise MogileFS::Backend::UnknownKeyError
  fid = res[0]
  sql = "SELECT devid FROM file_on WHERE fid = '#{@my.quote(fid)}'"
  query(sql).each do |devid,|
    unless devinfo = devices[devid.to_i]
      devices = refresh_device(true)
      devinfo = devices[devid.to_i] or next
    end
    devinfo[:readable] or next
    port = devinfo[:http_get_port]
    host = zone && zone == 'alt' ? devinfo[:altip] : devinfo[:hostip]
    nfid = '%010u' % fid
    b, mmm, ttt = /(\d)(\d{3})(\d{3})(?:\d{3})/.match(nfid)[1..3]
    uri = "/dev#{devid}/#{b}/#{mmm}/#{ttt}/#{nfid}.fid"
    urls << "http://#{host}:#{port}#{uri}"
  end
  urls
end

#_list_keys(domain, prefix = '', after = '', limit = 1000, &block) ⇒ Object

Lists keys starting with prefix follwing after up to limit. If after is nil the list starts at the beginning.

Raises:

  • (MogileFS::Backend::InvalidCharsError) 


32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
 
# File 'lib/mogilefs/mysql.rb', line 32

def _list_keys(domain, prefix = '', after = '', limit = 1000, &block)
  # this code is based on server/lib/MogileFS/Worker/Query.pm
  dmid = get_dmid(domain)

  # don't modify passed arguments
  limit ||= 1000
  limit = limit.to_i
  limit = 1000 if limit > 1000 || limit <= 0
  after, prefix = "#{after}", "#{prefix}"

  if after.length > 0 && /^#{Regexp.quote(prefix)}/ !~ after
    raise MogileFS::Backend::AfterMismatchError
  end

  raise MogileFS::Backend::InvalidCharsError if /[%\\]/ =~ prefix
  prefix.gsub!(/_/, '\_') # not sure why MogileFS::Worker::Query does this...

  sql = <<-EOS
  SELECT dkey,length,devcount FROM file
  WHERE dmid = #{dmid}
    AND dkey LIKE '#{@my.quote(prefix)}%'
    AND dkey > '#{@my.quote(after)}'
  ORDER BY dkey LIMIT #{limit}
  EOS

  keys = []
  query(sql).each do |dkey,length,devcount|
    yield(dkey, length.to_i, devcount.to_i) if block_given?
    keys << dkey
  end

  keys.empty? ? nil : [ keys, (keys.last || '') ]
end

#_size(domain, key) ⇒ Object

Returns the size of key.

Raises:

  • (MogileFS::Backend::UnknownKeyError) 


68
69
70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/mogilefs/mysql.rb', line 68

def _size(domain, key)
  dmid = get_dmid(domain)

  sql = <<-EOS
  SELECT length FROM file
  WHERE dmid = #{dmid} AND dkey = '#{@my.quote(key)}'
  LIMIT 1
  EOS

  res = query(sql).fetch_row
  return res[0].to_i if res && res[0]
  raise MogileFS::Backend::UnknownKeyError
end

#sleep(params) ⇒ Object 



116
 
# File 'lib/mogilefs/mysql.rb', line 116

def sleep(params); Kernel.sleep(params[:duration] || 10); {}; end