Colony

An alien is a web crawler with a DSL description. A colony is a collection with a bunch of aliens in it.

Installation

Note: Requires ruby 1.9.3

Add this line to your application's Gemfile:

gem 'colony'

And then execute:

$ bundle

Or install it yourself as:

$ gem install colony

Usage

Create a colony directory:

$ colony create

Then never feget to configure file in conf/colony.yml

# colony should be unique
name: 'dev_colony'

# lightweight publish-subscribe messaging system
nats: 'nats://*****:*****@121.14.161.220:4222/'

# database of alien empire
database:
  adapter: 'mysql2'
  encoding: 'utf8'
  reconnect: false
  database: 'ALIEN_EMPIRE'
  pool: 5
  username: 
  password: 
  host: '121.14.161.219'

# level: DEBUG INFO, WARN, ERROR, FATAL
logging:
  level: INFO
  file: ~/workspace/colony/logs/colony.log

# process id
pid_path: ~/workspace/colony/pid

# colony hearbeat
intervals:
  heartbeat: 5

Start colony in the colony directory:

$ colony start -d

It will generate a pid file in the colony directory, writing log to logs/colony.log.

Stop colony in the colony directory:

$ colony stop

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request