Class: Ultrasphinx::Search

Inherits:
Object show all
Includes:
Internals, Parser
Defined in:
lib/ultrasphinx/search.rb,
lib/ultrasphinx/search/parser.rb,
lib/ultrasphinx/search/internals.rb

Overview

Command-interface Search object.

Basic usage

To set up a search, instantiate an Ultrasphinx::Search object with a hash of parameters. Only the :query key is mandatory.

@search = Ultrasphinx::Search.new(
  :query => @query, 
  :sort_mode => 'descending', 
  :sort_by => 'created_at'
)

Now, to run the query, call its run method. Your results will be available as ActiveRecord instances via the results method. Example:

@search.run
@search.results

Options

Query format

The query string supports boolean operation, parentheses, phrases, and field-specific search. Query words are stemmed and joined by an implicit AND by default.

  • Valid boolean operators are AND, OR, and NOT.

  • Field-specific searches should be formatted as fieldname:contents. (This will only work for text fields. For numeric and date fields, see the 'filters parameter, below.)

  • Phrases must be enclosed in double quotes.

A Sphinx::SphinxInternalError will be raised on invalid queries. In general, queries can only be nested to one level. 

@query = 'dog OR cat OR "white tigers" NOT (lions OR bears) AND title:animals'

Hash parameters

The hash lets you customize internal aspects of the search.

:per_page

An integer. How many results per page.

:page

An integer. Which page of the results to return.

:class_names

An array or string. The class name of the model you want to search, an array of model names to search, or nil for all available models.

:sort_mode

'relevance' or 'ascending' or 'descending'. How to order the result set. Note that 'time' and 'extended' modes are available, but not tested.

:sort_by

A field name. What field to order by for 'ascending' or 'descending' mode. Has no effect for 'relevance'.

:weights

A hash. Text-field names and associated query weighting. The default weight for every field is 1.0. Example: :weights => {'title' => 2.0}

:filters

A hash. Names of numeric or date fields and associated values. You can use a single value, an array of values, or a range. (See the bottom of the ActiveRecord::Base page for an example.)

:facets

An array of fields for grouping/faceting. You can access the returned facet values and their result counts with the facets method.

Note that you can set up your own query defaults in environment.rb: 

Ultrasphinx::Search.query_defaults = HashWithIndifferentAccess.new({
  :per_page => 10,
  :sort_mode => 'relevance',
  :weights => {'title' => 2.0}
})

Advanced features

Cache_fu integration

The get_cache method will be used to instantiate records for models that respond to it. Otherwise, find is used.

Will_paginate integration

The Search instance responds to the same methods as a WillPaginate::Collection object, so once you have called run or excerpt you can use it directly in your views:

will_paginate(@search)

Excerpt mode

You can have Sphinx excerpt and highlight the matched sections in the associated fields. Instead of calling run, call excerpt. 

@search.excerpt

The returned models will be frozen and have their field contents temporarily changed to the excerpted and highlighted results.

You need to set the content_methods key on Ultrasphinx::Search.excerpting_options to whatever groups of methods you need the excerpter to try to excerpt. The first responding method in each group for each record will be excerpted. This way Ruby-only methods are supported (for example, a metadata method which combines various model fields, or an aliased field so that the original record contents are still available).

There are some other keys you can set, such as excerpt size, HTML tags to highlight with, and number of words on either side of each excerpt chunk. Example (in environment.rb):

Ultrasphinx::Search.excerpting_options = HashWithIndifferentAccess.new({
  :before_match => '<strong>', 
  :after_match => '</strong>',
  :chunk_separator => "...",
  :limit => 256,
  :around => 3,
  :content_methods => [['title'], ['body', 'description', 'content'], ['metadata']] 
})

Note that your database is never changed by anything Ultrasphinx does.

Defined Under Namespace

Modules: Internals, Parser

Constant Summary collapse

SPHINX_CLIENT_PARAMS =

Friendly sort mode mappings

HashWithIndifferentAccess.new({ 
  :sort_mode => HashWithIndifferentAccess.new({
    'relevance' => :relevance,
    'descending' => :attr_desc, 
    'ascending' => :attr_asc, 
    'time' => :time_segments,
    'extended' => :extended,
  })
})
INTERNAL_KEYS =

:nodoc:

['parsed_query']
MODELS_TO_IDS = 
get_models_to_class_ids || {}
MAX_MATCHES = 
DAEMON_SETTINGS["max_matches"].to_i
FACET_CACHE =

:nodoc:

{}

Constants included from Parser

Parser::OPERATORS

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(opts = {}) ⇒ Search

Builds a new command-interface Search object.

Raises:



272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
 
# File 'lib/ultrasphinx/search.rb', line 272

def initialize opts = {} 
  opts = HashWithIndifferentAccess.new(opts)            
  @options = Ultrasphinx::Search.query_defaults.merge(opts._deep_dup._coerce_basic_types)            

  @options['query'] = @options['query'].to_s
  @options['class_names'] = Array(@options['class_names'])
  @options['facets'] = Array(@options['facets'])
        
  raise UsageError, "Weights must be a Hash" unless @options['weights'].is_a? Hash
  raise UsageError, "Filters must be a Hash" unless @options['filters'].is_a? Hash
  
  @options['parsed_query'] = parse(query)
  
  @results, @subtotals, @facets, @response = [], {}, {}, {}
    
  extra_keys = @options.keys - (SPHINX_CLIENT_PARAMS.merge(Ultrasphinx::Search.query_defaults).keys + INTERNAL_KEYS)
  say "discarded invalid keys: #{extra_keys * ', '}" if extra_keys.any? and RAILS_ENV != "test" 
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(*args, &block) ⇒ Object

Delegates enumerable methods to @results, if possible. This allows us to behave directly like a WillPaginate::Collection. Failing that, we delegate to the options hash if a key is set. This lets us use self directly in view helpers.



374
375
376
377
378
379
380
381
382
 
# File 'lib/ultrasphinx/search.rb', line 374

def method_missing(*args, &block)
  if @results.respond_to? args.first
    @results.send(*args, &block)
  elsif options.has_key? args.first.to_s
    @options[args.first.to_s]
  else
    super
  end
end

Class Method Details

.get_models_to_class_ids ⇒ Object

:nodoc:



142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/ultrasphinx/search.rb', line 142

def self.get_models_to_class_ids #:nodoc:
  # Reading the conf file makes sure that we are in sync with the actual Sphinx index,
  # not whatever you happened to change your models to most recently
  unless File.exist? CONF_PATH
    Ultrasphinx.say "configuration file not found for #{RAILS_ENV.inspect} environment"
    Ultrasphinx.say "please run 'rake ultrasphinx:configure'"
  else
    begin  
      lines = open(CONF_PATH).readlines          

      sources = lines.select do |line| 
        line =~ /^source \w/
      end.map do |line| 
        line[/source ([\w\d_-]*)/, 1].gsub('__', '/').classify
      end
      
      ids = lines.select do |line| 
        line =~ /^sql_query /
      end.map do |line| 
        line[/(\d*) AS class_id/, 1].to_i
      end
      
      raise unless sources.size == ids.size          
      Hash[*sources.zip(ids).flatten]
                              
    rescue
      Ultrasphinx.say "#{CONF_PATH} file is corrupted"
      Ultrasphinx.say "please run 'rake ultrasphinx:configure'"
    end    
    
  end
end

Instance Method Details

#current_page ⇒ Object

Returns the current page number of the result set. (Page indexes begin at 1.)



241
242
243
 
# File 'lib/ultrasphinx/search.rb', line 241

def current_page
  @options['page']
end

#excerpt ⇒ Object

Overwrite the configured content accessors with excerpted and highlighted versions of themselves. Runs run if it hasn’t already been done.



322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
 
# File 'lib/ultrasphinx/search.rb', line 322

def excerpt

  require_run         
  return if results.empty?

  # See what fields each result might respond to for our excerpting
  results_with_content_methods = results.map do |result|
    [result] << Ultrasphinx::Search.excerpting_options['content_methods'].map do |methods|
      methods.detect { |x| result.respond_to? x }
    end
  end
  
  # Fetch the actual field contents
  docs = results_with_content_methods.map do |result, methods|
    methods.map do |method| 
      method and strip_bogus_characters(result.send(method)) or ""
    end
  end.flatten
  
  excerpting_options = {
    :docs => docs, 
    :index => UNIFIED_INDEX_NAME, 
    :words => strip_query_commands(parsed_query)}
  Ultrasphinx::Search.excerpting_options.except('content_methods').each do |key, value|
    # Riddle only wants symbols
    excerpting_options[key.to_sym] ||= value
  end
  
  responses = perform_action_with_retries do 
    # Ship to Sphinx to highlight and excerpt
    @request.excerpts(excerpting_options)
  end
  
  responses = responses.in_groups_of(Ultrasphinx::Search.excerpting_options['content_methods'].size)
  
  results_with_content_methods.each_with_index do |result_and_methods, i|
    # override the individual model accessors with the excerpted data
    result, methods = result_and_methods
    methods.each_with_index do |method, j|
      result._metaclass.send('define_method', method) { responses[i][j] } if method
    end
  end
  
  @results = results_with_content_methods.map do |result_and_content_method| 
    result_and_content_method.first.freeze
  end
  
  self
end

#facets ⇒ Object

Returns the facet map for this query, if facets were used.

Raises:



204
205
206
207
208
 
# File 'lib/ultrasphinx/search.rb', line 204

def facets
  raise UsageError, "No facet field was configured" unless @options['facets']
  require_run
  @facets
end

#next_page ⇒ Object

Returns the next page number.



262
263
264
 
# File 'lib/ultrasphinx/search.rb', line 262

def next_page
  current_page < page_count ? (current_page + 1) : nil
end

#offset ⇒ Object

Returns the global index position of the first result on this page.



267
268
269
 
# File 'lib/ultrasphinx/search.rb', line 267

def offset 
  (current_page - 1) * per_page
end

#options ⇒ Object

Returns the options hash.



182
183
184
 
# File 'lib/ultrasphinx/search.rb', line 182

def options
  @options
end

#page_count ⇒ Object

Returns the last available page number in the result set.



251
252
253
254
 
# File 'lib/ultrasphinx/search.rb', line 251

def page_count
  require_run    
  (total_entries / per_page.to_f).ceil
end

#parsed_query ⇒ Object

:nodoc:



192
193
194
195
 
# File 'lib/ultrasphinx/search.rb', line 192

def parsed_query #:nodoc:
  # Redundant with method_missing
  @options['parsed_query']
end

#per_page ⇒ Object

Returns the number of records per page.



246
247
248
 
# File 'lib/ultrasphinx/search.rb', line 246

def per_page
  @options['per_page']
end

#previous_page ⇒ Object

Returns the previous page number.



257
258
259
 
# File 'lib/ultrasphinx/search.rb', line 257

def previous_page 
  current_page > 1 ? (current_page - 1) : nil
end

#query ⇒ Object

Returns the query string used.



187
188
189
190
 
# File 'lib/ultrasphinx/search.rb', line 187

def query
  # Redundant with method_missing
  @options['query']
end

#response ⇒ Object

Returns the raw response from the Sphinx client.



211
212
213
214
 
# File 'lib/ultrasphinx/search.rb', line 211

def response
  require_run
  @response
end

#results ⇒ Object

Returns an array of result objects.



198
199
200
201
 
# File 'lib/ultrasphinx/search.rb', line 198

def results
  require_run
  @results
end

#run(reify = true) ⇒ Object

Run the search, filling results with an array of ActiveRecord objects. Set the parameter to false if you only want the ids returned.



292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
 
# File 'lib/ultrasphinx/search.rb', line 292

def run(reify = true)
  @request = build_request_with_options(@options)

  say "searching for #{@options.inspect}"

  perform_action_with_retries do
    @response = @request.query(parsed_query, UNIFIED_INDEX_NAME)
    say "search returned #{total_entries}/#{response[:total_found].to_i} in #{time.to_f} seconds."
      
    if Ultrasphinx::Search.client_options['with_subtotals']        
      @subtotals = get_subtotals(@request, parsed_query) 
    end
    
    Array(@options['facets']).each do |facet|
      @facets[facet] = get_facets(@request, parsed_query, facet)
    end        
    
    @results = convert_sphinx_ids(response[:matches])
    @results = reify_results(@results) if reify
    
    say "warning; #{response[:warning]}" if response[:warning]
    raise UsageError, response[:error] if response[:error]
    
  end      
  self
end

#run? ⇒ Boolean

Returns whether the query has been run.

Returns:

  • (Boolean) 


236
237
238
 
# File 'lib/ultrasphinx/search.rb', line 236

def run?
  !@response.blank?
end

#say(msg) ⇒ Object

:nodoc:



384
385
386
 
# File 'lib/ultrasphinx/search.rb', line 384

def say msg #:nodoc:
  Ultrasphinx.say msg
end

#subtotals ⇒ Object

Returns a hash of total result counts, scoped to each available model. This requires extra queries against the search daemon right now. Set Ultrasphinx::Search.client_options[:with_subtotals] = true to enable the extra queries. Most of the overhead is in instantiating the AR result sets, so the performance hit is not usually significant.

Raises:



217
218
219
220
221
 
# File 'lib/ultrasphinx/search.rb', line 217

def subtotals
  raise UsageError, "Subtotals are not enabled" unless Ultrasphinx::Search.client_options['with_subtotals']
  require_run
  @subtotals
end

#time ⇒ Object

Returns the response time of the query, in milliseconds.



230
231
232
233
 
# File 'lib/ultrasphinx/search.rb', line 230

def time
  require_run
  response[:time]
end

#total_entries ⇒ Object

Returns the total result count.



224
225
226
227
 
# File 'lib/ultrasphinx/search.rb', line 224

def total_entries
  require_run
  [response[:total_found] || 0, MAX_MATCHES].min
end