CloudSesame
Powerful and Flexible AWS CloudSearch Query DSL
Installation
- In terminal type:
gem install CloudSesame
- Or add this line to the application in
Gemfile
:gem 'CloudSesame'
AWS Credentials
*Create a initializer file, example: config/initializers/cloud_sesame.rb
require 'cloud_sesame'
CloudSesame::Domain::Client.configure do |config|
config.access_key = ENV['AWS_ACCESS_KEY_ID']
config.secret_key = ENV['AWS_SECRET_ACCESS_KEY']
end
Model Setup
include CloudSesame
in a model or classdefine_cloudsearch
with a block to setup class/modal specific setting
define_cloudsearch(&block)
Includes all Model/Class specific cloudsearch configurations
class Product
include CloudSesame
define_cloudsearch do
...
end
end
config.endpoint=(string)
Set AWS CloudSearch instance search endpoint
config.region=(string)
Set AWS CloudSearch isntance region
class Product
include CloudSesame
define_cloudsearch do
config.endpoint = ENV[...]
config.region = ENV[...]
end
end
default_size(integer = 10)
Set default search size
class Product
include CloudSesame
define_cloudsearch do
default_size 100
end
end
define_sloppiness(integer)
Setup sloppy query, it is turned off by default
class Product
include CloudSesame
define_cloudsearch do
define_sloppiness 3
end
end
define_fuzziness(&block)
Setup fuzziness, it is turned off by default. the block can set 3 values
- max_fuzziness(integer = 3) maxinmum fuzziness per word
- min_char_size(integer = 6) minimum word length to trigger fuzziness
fuzzy_percent(float = 0.17) percent used to calculate the fuzziness based on the word length, fuzziness whill choose between the calculated result and maximum fizziness, whichever is smaller. [(word.size * fuzzy_percent).round, max_fuzziness].min
class Product include CloudSesame define_cloudsearch do define_fuzziness do max_fuzziness 3 min_char_size 6 fuzzy_percent 0.17 end end end
field(symbol, options = {})
calling field and pass in a field_name will create an field expression accessor
- simple usage
field :name
and field expression accessor can be called to create a field expression ``` Product.cloudsearch.name("user")
{ filter_query: "name:'user'" }
- aliase field name
field :text1, as: :name
and field expression accessor method name will be `#name`
Product.cloudsearch.name("user")
{ filter_query: "text1:'user'" }
- with query options is set to `true`
field :name, query: true
{ query_options: { fields: ['name'] } }
- with **weight** assigned to query options
field :tags, query: { weight: 2 }
{ query_options[:fields] = ['name', 'tags^2'] }
- with **facet options** passed in
field :currency, facet: true
{ facets: { currency:{} } }
- with facet **buckets**
field :discount, facet: {
buckets: %w([10,100] [25,100] [50,100] [70,100]),
method: 'interval'
}
{ facets: { discount: { buckets:["[10,100]","[25,100]","[50,100]","[70,100]"], method:"interval"} } }
- with facet **size** set
field :manufacturer, facet: { size: 50 }
- with facet **sorting**
field :category, facet: { sort: 'bucket', size: 10_000 }
####scope(symbol, proc, &block)
ActiveRecord styled scope method. Scope allows you to specify commonly-used queries which can be referenced as method calls on cloudsearch or inside of operator block
**set scope**
...
define_cloudsearch do
....
scope :popular, -> { or! { tags "popular"; popularity gt(70) } }
end
**call a scope**
Product.cloudsearch.query("shoes").popular
####Full Example
class Product < ActiveRecord::Base include CloudSesame
define_cloudsearch do
config.endpoint = ENV['AWS_PRODUCT_SEARCH_ENDPOINT']
config.region = ENV['AWS_PRODUCT_SEARCH_REGION']
default_size <integer>
define_sloppiness <integer>
define_fuzziness {
max_fuzziness <integer>
min_char_size <integer>
fuzzy_percent <float>
}
field :product_name, query: { weight: <integer> }
field :description, query: true
field :currency, facet: true
field :discount, facet: {
buckets: %w([10,100] [25,100] [50,100] [70,100]),
method: 'interval'
}
field :manufacturer, facet: { size: <integer> }
field :category, facet: {
sort: 'bucket', size: <integer>
}
field :created_at
scope :shoes_by_brand, ->(brand = nil) { query("shoes").and { manufacturer brand } if brand }
scope :created_in, ->(days = nil) { and { created_at r.gt(Date.today - days) } if days }
end
end
##Inherit cloudsearch definition from another class/modal
####load_definition_from(Class/Model)
class ExclusiveProduct < Product load_definition_from Product end
definition can be overrided by calling `#define_cloudsearch` again
define_cloudsearch {
field :name, as: :product_name
}
#Query DSL
- `.cloudsearch` returns a CloudSesame::Domain::Base instance. It is the entry point to start building up the query
- A CloudSesame::Query::Builder instance is expected to return from any query methods chained after `.cloudsearch`
Product.cloudsearch #=> CloudSesame::Domain::Base/
query = Product.cloudsearch.query("shoes") =>
**Query methods can be chained and build up**
query = Product.cloudsearch.query("shoes") .and { manufacturer('puma') or!.not { manufacturer "nike", "adidas" } } .sort(price: :desc) .page(2) .size(100) }
and `query.compile` will output the compiled hash
{ query:'shoes', filter_query: "(and manufacturer:'puma' (not (or manufacturer:'nike' manufacturer:'adidas')))", sort: "price desc" start: 100 size: 100 }
###Simple Query
Simple Query String
Product.cloudsearch.query("shoes")
OUTPUT: "shoes"
Product.cloudsearch.query("shoes -puma")
OUTPUT: "shoes -puma"
With Sloppy Query
Product.cloudsearch.query("shoes")
OUTPUT: "(shoes|\"shoes\"~3)"
Product.cloudsearch.query("shoes -puma")
OUTPUT: "(shoes -puma|\"shoes -puma\"~3)"
With Fuzzy Search
Product.cloudsearch.query("white shoes")
OUTPUT: "(shoes|(white~3+shoes~3))"
Product.cloudsearch.query("white shoes -puma")
OUTPUT: "(shoes -puma|(white~3+shoes~3+-puma))"
###Pagination
Product.cloudsearch.page(3).size(100)
OUTPUT: { start: 200, size: 100 }
Product.cloudsearch.start(99).size(100)
OUTPUT: { start: 99, size: 100 }
###Sorting
Product.cloudsearch.sort(name: :desc)
OUTPUT: { sort: "name desc" }
Product.cloudsearch.sort(name: :desc, price: :asc)
OUTPUT: { sort: "name desc,price asc" }
###Return
Product.cloudsearch.all_fields
OUTPUT: { return: "_all_fields" }
Product.cloudsearch.no_fields
OUTPUT: { return: "_no_fields" }
Product.cloudsearch.score
OUTPUT: { return: "_score" }
###Scope
Product.coudsearch.shoes_by_brand("puma")
OUTPUT: { query:"shoes" filter_query:"(and manufacturer:'puma')" }
Product.cloudsearch.created_in(7)
OUTPUT: { filter_query:"(and created_at:- 7,) }
Product.cloudsearch.or {
created_in(7)
discount 25..100
}
OUTPUT: { filter_query:"(or (and created_at:- 7,) discount:{25,100] }
###AND & OR Block
Product.cloudsearch.and { ... }
OUTPUT: "(and ...)"
Product.cloudsearch.and {
or! { ... }
or! { ... }
}
OUTPUT: "(and (or ...) (or ...))"
NOTE: AND & OR are a ruby syntax, can not be used directly,
so aliased them to (and!, all) and (or!, any)
Product.cloudsearch.and.not { ... }
OUTPUT: "(not (and ...))"
Product.cloudsearch.or { and!.not { ...} }
OUTPUT: "(or (not (and ...)))"
###Field Methods
#####field_name(*values)
- calling the field_name with multiple values will generate multiple field expression
- fields can be chained together
Product.cloudsearch.name("shoes")
OUTPUT: "name:'shoes'"
Product.cloudsearch.name "shoes", "sneaker"
OUTPUT: name:'shoes' name:'sneaker'
Product.cloudsearch.name("shoes").price(100)
OUTPUT: "(and name:'shoes' price:100)"
Product.cloudsearch.and { name "shoes"; price 25..100 }
OUTPUT: "(and name:'shoes' price:[25,100])"
#####field_name.field_array_method(*values)
- #not, #prefix (#start_with, #begin_with), #near can be called after the field_name
- #not can be chained with another operator after, example: `name.not.prefix`
Product.cloudsearch.and { name.not "shoes"; ... }
OUTPUT: "(and (not name:'shoes') ...)"
Product.cloudsearch.and { name.start_with "shoes" }
OUTPUT: "(and (prefix field='name' 'shoes'))"
Product.cloudsearch.and { name.near "shoes" }
OUTPUT: "(and (near field='name' 'shoes'))"
Product.cloudsearch.and { name.not.start_with "shoes" }
OUTPUT: "(and (not (near field='name' 'shoes')))"
Product.cloudsearch.and { name(start_with("shoes"), near("puma")).not("nike") }
OUTPUT: "(and (prefix field='name' 'shoes') (near field='name' 'puma') (not name:'nike'))"
* #prefix (#start_with, #begin_with), #near can be called directly to generate a single field value
Product.cloudsearch.and { name.not start_with("shoes"), near("something") }
OUTPUT: "(not (prefix field='name' 'shoes') (not (near field='name' 'something')))"
###Date, Time and Rage
* field method accepts Ruby Date and Range Object and will automatically parse them into the CloudSearch format
Date.today => "'2016-01-18T00:00:00Z'" Time.now => "'2016-01-18T14:36:57Z'" 25..100 => "[25,100]" 25...100 => "[25,100}"
* use #range or #r for more complicated range, range object accepts Date/Time object as well
r.gt(100) => "100," r.gte(100) => "[100,}" r.lt(100) => ",100" r.lte(100) => "r.gte(100).lt(200) => "[100,200" r.gt(Date.today) => "'2016-01-18T00:00:00Z',"
###Search Related Methods
* #search => send off the request, save and returns the response and clear the request
* #found => returns the hits found from the response
* #results => returns the hits.hit from the response
* #each => Calls the given block once for each result, passing that result as a parameter. Returns the results itself.
* #map => Creates a new array containing the results returned by the block.
* #request => returns the cloudsearch ast tree
* #response => returns the last response
* #clear_request => resets the request
* #clear_reponse => resets the response
*
###MISC Methods
* #compile => compiles the current query and return the compiled hash
*