Class: Ferret::Store::FSDirectory

Inherits:
Directory
  • Object
show all
Includes:
MonitorMixin
Defined in:
lib/ferret/store/fs_store.rb

Overview

This is a filesystem implementation of Directory and will be the one usually used for storing the index. This implementation stores each separate file as a separate file on the operating system. This works fine and is the most efficient solution for small to medium size indexes. For very large indexes, there may be a problem with the operating system not wanting to open to many files. One fix for this is to change the maximum open files setting in your operating system. Alternatively you could use a compound file instead.

TODO:

  • need a better way of setting properties. Currently you have to change the constants to disable locking.

Defined Under Namespace

Classes: FSIndexInput, FSIndexOutput, FSLock

Constant Summary collapse

LOCKS_DISABLED =

Locks should be disabled it there is no need for them

false
LOCK_DIR =

The lock dir is the directory where the file locks will be stored

nil
@@Directories =

This cache of directories ensures that there is a unique Directory instance per path, so that synchronization on the Directory can be used to synchronize access between readers and writers.

Hash.new.extend(MonitorMixin)

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods inherited from Directory

#file_count

Constructor Details

#initialize(path, create) ⇒ FSDirectory

Create a new directory from the path.

path

the path to the directory.

create

if true, create, or erase any existing contents.



37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
 
# File 'lib/ferret/store/fs_store.rb', line 37

def initialize(path, create)
  super()
  if create then FileUtils.mkdir_p(path) end
  if not File.directory?(path) then
    raise "There is no directory: #{path}. Use create = true to create one"
  end
  @dir = Dir.new(path)
  # put the lock_dir here as well if no default exists.
  if LOCK_DIR then
    @lock_dir = Dir.new(LOCK_DIR) 
  else
    @lock_dir = Dir.new(path) 
  end
  @ref_count = 0
end

Instance Attribute Details

#lock_dir ⇒ Object

Returns a Dir object of the directory where the lock is stored



97
98
99
 
# File 'lib/ferret/store/fs_store.rb', line 97

def lock_dir
  @lock_dir
end

Class Method Details

.allocate ⇒ Object 



54
 
# File 'lib/ferret/store/fs_store.rb', line 54

alias :allocate :new

.locks_disabled? ⇒ Boolean

Returns true if locks have been disabled

Returns:

  • (Boolean) 


83
84
85
 
# File 'lib/ferret/store/fs_store.rb', line 83

def FSDirectory.locks_disabled?
  LOCKS_DISABLED
end

.new(path, create = false) ⇒ Object

Returns the directory instance for the named location.

Directories are cached, so that, for a given canonical path, the same FSDirectory instance will always be returned. This permits synchronization on directories.

path

the path to the directory.

create

if true, create, or erase any existing contents.



66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
 
# File 'lib/ferret/store/fs_store.rb', line 66

def FSDirectory.new(path, create = false)
  dir = nil
  @@Directories.synchronize do
    dir = @@Directories[path]
    if not dir then
      dir = FSDirectory.allocate(path, create)
      @@Directories[path] = dir
    end
    dir.refresh if create
  end
  dir.synchronize do
    dir.reference()
  end
  return dir
end

Instance Method Details

#close ⇒ Object

Closes the store.



189
190
191
192
193
194
195
196
197
198
 
# File 'lib/ferret/store/fs_store.rb', line 189

def close()
  synchronize do
    @ref_count -= 1
    if (@ref_count <= 0) then
      @@Directories.synchronize do
        @@Directories.delete(@dir.path)
      end
    end
  end
end

#create_output(name) ⇒ Object

Creates a new, empty file in the directory with the given name. Returns a stream writing this file.



174
175
176
 
# File 'lib/ferret/store/fs_store.rb', line 174

def create_output(name) 
  FSIndexOutput.new(dir_path(name))
end

#delete(name) ⇒ Object

Removes an existing file in the directory.



149
150
151
152
153
154
155
 
# File 'lib/ferret/store/fs_store.rb', line 149

def delete(name)
  begin
    File.delete(dir_path(name))
  rescue SystemCallError => e
    raise IOError, e.to_s
  end
end

#each ⇒ Object

Iterates through the file listing, skipping lock files if they exist



121
122
123
124
125
126
127
128
129
130
 
# File 'lib/ferret/store/fs_store.rb', line 121

def each()
  refresh_dir
  @dir.each do |file_name|
    # return all files except for the current and parent directories
    # and any lock files that exist in this directory
    next if ['.', '..'].include?(file_name)
    next if file_name =~ Regexp.new('^' + lock_prefix)
    yield file_name
  end
end

#exists?(name) ⇒ Boolean

Returns true if a file with the given name exists.

Returns:

  • (Boolean) 


133
134
135
 
# File 'lib/ferret/store/fs_store.rb', line 133

def exists?(name)
  File.exists?(dir_path(name))
end

#length(name) ⇒ Object

Returns the length of a file in the directory.



168
169
170
 
# File 'lib/ferret/store/fs_store.rb', line 168

def length(name)
  File.size(dir_path(name))
end

#make_lock(name) ⇒ Object

Construct a Lock.



184
185
186
 
# File 'lib/ferret/store/fs_store.rb', line 184

def make_lock(name) 
  FSLock.new(@lock_dir.path + "/" + lock_prefix() + name)
end

#modified(name) ⇒ Object

Returns the time the named file was last modified.



138
139
140
 
# File 'lib/ferret/store/fs_store.rb', line 138

def modified(name) 
  File.mtime(dir_path(name))
end

#open_input(name) ⇒ Object

Returns a stream reading an existing file.



179
180
181
 
# File 'lib/ferret/store/fs_store.rb', line 179

def open_input(name)
  FSIndexInput.new(dir_path(name))
end

#reference ⇒ Object 



200
201
202
 
# File 'lib/ferret/store/fs_store.rb', line 200

def reference()
  @ref_count += 1
end

#refresh ⇒ Object

Remove all files and locks from this directory so we have a clean instance



100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
 
# File 'lib/ferret/store/fs_store.rb', line 100

def refresh
  synchronize do
    # delete all the files
    refresh_dir
    each do |fname|
      File.delete(dir_path(fname))
    end
    # clear all the locks
    refresh_lock_dir
    @lock_dir.each do |lock_fname|
      next if lock_fname == '.' or lock_fname == '..'
      File.delete(@lock_dir.path + '/' + lock_fname)
    end
  end
end

#rename(from, to) ⇒ Object

Renames an existing file in the directory. If a file already exists with the new name, then it is replaced. This replacement should be atomic.



160
161
162
163
164
 
# File 'lib/ferret/store/fs_store.rb', line 160

def rename(from, to)
  synchronize do
    File.rename(dir_path(from), dir_path(to))
  end
end

#touch(name) ⇒ Object

Set the modified time of an existing file to now.



143
144
145
146
 
# File 'lib/ferret/store/fs_store.rb', line 143

def touch(name) 
  # just open the file and close it. No need to do anything with it.
  FileUtils.touch(dir_path(name))
end