Build Status Coverage Status Code Climate Gem Version

Masque

女面

Masque is a browser emulated crawler builder wrapping capybara-webkit and poltergeist.

Installation

You need Xvfb and QT libraries before installing Masque gem.

On Debian/Ubuntu:

$ sudo apt-get install libqt4-dev xvfb libicu48 ttf-ubuntu-font-family
$ gem install masque

On Mac OS X (assumes you have Homebrew installed):

$ brew install qt
$ gem install masque

Usage

Easy crawling websites they required JavaScript.

require "masque"
m = Masque.new(:display => 99, :driver => :webkit) # or :driver => :poltergeist
m.run do
  # Capybara::DSL syntax
  # https://github.com/jnicklas/capybara#the-dsl

  visit "http://www.google.com/"
  fill_in("q", :with => "capybara")
  find('*[name="btnG"]').click

  titles = evaluate_script "(function(){\nvar titles = Array.prototype.map.call(\ndocument.querySelectorAll('h3 a'),\nfunction(a) {\nreturn a.innerText;\n}\n);\n\nreturn titles;\n})();\n"
  puts titles.join("\n")
end

more examples are available in examples/ directory.

TODO

  • Impl useful DSL
  • Respect driver specific features/options

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request