WHAT TURKEE CAN DO

  • Perform user feedback studies easily. http://i.imgur.com/wIU40BAl.png

  • Seamlessly convert your Rails forms for use on Mechanical Turk.

  • Easily import the data posted by the Mechanical Turk workers back into your data models.

External forms are created using a simple form helper. HITs are created by issuing a rake command. Retrieving submitted response data and importing that data into your model(s) requires just one more rake command.

INSTALL/UPGRADE

Add turkee to your Gemfile as a gem dependency, then do a 'bundle install':

gem 'turkee'

If you're upgrading Turkee (1.1.1 and prior) or installing for the first time, run:

rails g turkee --skip

(the skip flag will ensure that you don't overwrite prior Turkee initializers and migrations.)

This will copy the needed migrations and config/initializer into your application directory.

To access the Turkee rake tasks, add the following to your application's Rakefile: require 'tasks/turkee'

If you haven't created a Mechanical Turk account, surf on over to Amazon's Web Services and create an AWS account.

Once you have your account created, you can access your AWS access key and secret access key from here.

CONFIGURATION

Inside the config/initializers directory, you'll see the file turkee.rb. Edit the file with your Amazon credentials.

AWSACCESSKEYID      = 'XXXXXXXXXXXXXXXXXX'
AWSSECRETACCESSKEY  = 'YYYYYYYYYYYYYYYYYYYYYYYYYYYY'

RTurk::logger.level     = Logger::DEBUG
RTurk.setup(AWSACCESSKEYID, AWSSECRETACCESSKEY, :sandbox => (Rails.env == 'production' ? false : true))

Run the migrations :

rake db:migrate

BASIC FORMS

  1. You should disable form controls if the Turker hasn't accepted the HIT. You can determine this from your controller: class SurveysController < ApplicationController

     def new
         @disabled      = Turkee::TurkeeFormHelper::disable_form_fields?(params)
    
         # If you wanted to find the associated turkee_task, you could do a find by hitId
         #  Not necessary in a simple example.
         # @turkee_task   = Turkee::TurkeeTask.find_by_hit_id(params[:hitId]).id rescue nil
    
         ...
         @survey        = Survey.new
     end
    
  2. Change your forms to use the form helper.

    <%= turkee_form_for(@survey, params) do |f| %>

    <%= f.text_area :value, :disabled => @disabled %>

    <%= f.submit 'Create', :disabled => @disabled %>

    <% end %>

Using the turkee_form_for helper will post the form to the Mechanical Turk sandbox if you're in development/test mode, otherwise it will be posted to Mechanical Turk production/live site.

  1. Run the following rake task to post to Mechanical Turk :

    Host URL of your application

    Title of your HIT

    Description of your HIT

    Model name of your task form (the New action should be implemented)

    Number of assignments for HIT

    The reward for a successful completion

    The lifetime of the HIT in days (e.g. 5 days)

    rake turkee:post_hit[, , <Description>, <Model>, <Number of Assignments>, <Reward>, <Lifetime>]</p> <p>e.g. : rake turkee:post_hit["<a href="https://www.yourapp.com&quot;,&quot;Please">https://www.yourapp.com&quot;,&quot;Please</a> complete our survey","Tell us your favorite color.","Survey",100,0.05,5,1] ** Do not put spaces before or after commas for the rake task parameters</p> </li> </ol> <p>This will insert a row for the requested HIT into the turkee_tasks table. The turkee_tasks entry stores (along with the other properties) the HIT URL (e.g. <a href="http://workersandbox.mturk.com/mturk/preview?groupId=1HGHJJGHQSJB7WMWJ33YS8WM169XNIL">http://workersandbox.mturk.com/mturk/preview?groupId=1HGHJJGHQSJB7WMWJ33YS8WM169XNIL</a> ) and HIT ID (e.g. 1J1EXO8SUQ3URTTUYGHJ7EKUT11 ). These values are returned from Mechanical Turk when the HIT request is posted.</p> <p>When a Turk worker views your HIT, the HIT will display your form within an iFrame. With the above example, Mechanical Turk will open an iFrame for the HIT assignment displaying the form <a href="http://www.yourapp.com/surveys/new">http://www.yourapp.com/surveys/new</a></p> <h2 id="USER_FEEDBACK_STUDIES">USER FEEDBACK STUDIES</h2> <ol> <li> <p>Add the following method call to your app/views/layouts/application.html.haml file :</p> <p><%= turkee_study %></p> </li> <li> <p>Call the create_study rake task :</p> <p>rake turkee:create_study[<Application Page URL>, <HIT Title>, <HIT Description>, <Number of Assignments>, <Reward>, <Lifetime of HIT>]</p> <h1 id="30_Assignments_at_a_quarter_each_requesting_feedback_on_my_site">30 Assignments at a quarter each requesting feedback on my site</h1> <p>RAILS_ENV=production rake turkee:create_study['https://yoursite.com/page-to-be-test',"Can you help me test my website?","Give feedback below on what you didn't like about my site. Click submit below when you're finished.", 30,0.25, 20]</p> </li> <li> <p>The Turker will see a feedback form overlayed over your website. Within this textarea, they can provide feedback.</p> </li> </ol> <p>Screenshot:</p> <p><a href="http://i.imgur.com/wIU40BA.png">http://i.imgur.com/wIU40BA.png</a></p> <h2 id="RETRIEVING_RESULTS">RETRIEVING RESULTS</h2> <ol> <li>Run the rake task that retrieves the values from Mechanical Turk and stores the user entered values into your model. rake turkee:get_all_results</li> </ol> <p>Rerun this task periodically to retrieve newly entered form values. You can setup this task as a cronjob to automate this.</p> <pre class="code ruby"><code class="ruby"><span class='rubyid_crontab identifier id'>crontab</span> <span class='minus op'>-</span><span class='rubyid_e identifier id'>e</span> <span class='comment val'># E.g. run every five minutes</span> <span class='mult op'>*</span><span class='regexp val'>/5 * * * * cd /</span><span class='rubyid_var identifier id'>var</span><span class='div op'>/</span><span class='rubyid_www identifier id'>www</span><span class='div op'>/</span><span class='rubyid_your identifier id'>your</span><span class='div op'>/</span><span class='rubyid_turk identifier id'>turk</span><span class='div op'>/</span><span class='rubyid_application identifier id'>application</span> <span class='andop op'>&&</span> <span class='rubyid_rake identifier id'>rake</span> <span class='label val'>turkee:</span><span class='rubyid_get_all_results identifier id'>get_all_results</span> </code></pre> <p>Or you can directly call :</p> <pre class="code ruby"><code class="ruby"><span class='rubyid_Turkee constant id'>Turkee</span><span class='colon2 op'>::</span><span class='rubyid_TurkeeTask constant id'>TurkeeTask</span><span class='dot token'>.</span><span class='rubyid_process_hits identifier id'>process_hits</span> </code></pre> <h2 id="DATA_RETRIEVAL__THE_INS_AND_OUTS">DATA RETRIEVAL; THE INS AND OUTS</h2> <ol> <li> <p>When a response is retrieved from Mechanical Turk, Turkee attempts to create a data row for the model specified using the corresponding retrieved data. If the row cannot be created (input failed model validations), the assignment is rejected. As for Mechanical Turk approval, if the row is created and you haven't specified your own custom approve? method for the model, the assignment will automatically be approved. If you'd like to add your own custom approval method, add the approve? instance method to your model. E.g. : class Survey < ActiveRecord::Base def approve? (!response.blank? && !comment.blank?) end end</p> </li> <li> <p>When all specified assignments for a HIT have been completed, Turkee will attempt to call the optional hit_complete class method for the model. This can be used for any cleanup logic. E.g. : class Survey < ActiveRecord::Base def self.hit_complete(turkee_task) #do something end end</p> </li> <li> <p>If all of specified assignments for a HIT have not been completed by the end of the HITs lifetime, Turkee will attempt to call the optional hit_expired class method for the model. This can be used for any cleanup logic. E.g. : class Survey < ActiveRecord::Base def self.hit_expired(turkee_task) #do something end end</p> </li> </ol> <h2 id="Advanced_Usage">Advanced Usage</h2> <ol> <li> <p>You can use the params hash to pass object IDs to your forms. E.g. if you wanted to setup a form of questions about a given URL (let's call the model UrlSurvey), your code would look something like :</p> <p>URL.all do |url| Turkee::TurkeeTask.create_hit(host, "Enter the address displayed for the given url.", hit_description, UrlSurvey, num_assignments, reward, lifetime, duration, {}, => url.id, {}) end</p> </li> </ol> <p>Then when displaying your form, you can find the URL object via the <a href=":url_id">params</a> value.</p> <ol start="2"> <li>You can pass in qualifications to allow only certain Turkers to work on your HIT.</li> </ol> <p>E.g. qualifications = => {:gt => 70, :country => => 'US'}</p> <pre class="code ruby"><code class="ruby"><span class='rubyid_Turkee constant id'>Turkee</span><span class='colon2 op'>::</span><span class='rubyid_TurkeeTask constant id'>TurkeeTask</span><span class='dot token'>.</span><span class='rubyid_create_hit identifier id'>create_hit</span><span class='lparen token'>(</span><span class='rubyid_host identifier id'>host</span><span class='comma token'>,</span> <span class='rubyid_hit_title identifier id'>hit_title</span><span class='comma token'>,</span> <span class='rubyid_hit_description identifier id'>hit_description</span><span class='comma token'>,</span> <span class='rubyid_UrlSurvey constant id'>UrlSurvey</span><span class='comma token'>,</span> <span class='rubyid_num_assignments identifier id'>num_assignments</span><span class='comma token'>,</span> <span class='rubyid_reward identifier id'>reward</span><span class='comma token'>,</span> <span class='rubyid_lifetime identifier id'>lifetime</span><span class='comma token'>,</span> <span class='rubyid_duration identifier id'>duration</span><span class='comma token'>,</span> <span class='rubyid_qualifications identifier id'>qualifications</span><span class='comma token'>,</span> <span class='lbrace token'>{</span><span class='rbrace token'>}</span><span class='comma token'>,</span> <span class='lbrace token'>{</span><span class='rbrace token'>}</span><span class='rparen token'>)</span> </code></pre> <ul> <li>The rake task does not support the passing of qualifications.</li> </ul> <ol start="3"> <li> <p>Turkee assumes that the form url will be the new action for the class passed to create_hit. If you have a more complex form url which would be the case for nested resources, you can use the :form_url option to designate a form url different from the default.</p> <p>form_url = Rails.application.routes.url_helpers.send("new_user_survey_path",@my_survey)</p> <p>Turkee::TurkeeTask.create_hit(host, hit_title, hit_description, :Survey, num_assignments, reward, lifetime, duration, {}, {}, => form_url)</p> </li> </ol> <h2 id="Gotchas">Gotchas</h2> <ol> <li>The application that hosts your external forms preferably should have an https interface (you're going to have to buy an SSL certificate). If the forms are hosted on an unsecured host (http), because Mechanical Turk defaults to https, you're going to receive the ugly popup from IE regarding "mixed content" (<a href="http://msdn.microsoft.com/en-us/library/ee264315%28v=vs.85%29.aspx">http://msdn.microsoft.com/en-us/library/ee264315%28v=vs.85%29.aspx</a>).</li> </ol> <h2 id="Copyright">Copyright</h2> <p>Copyright (c) 2010 - 2012 Jim Jones. See LICENSE for details.</p></div></div> <div id="footer"> Generated on Sat Sep 19 16:58:20 2026 by <a href="https://yardoc.org" title="Yay! A Ruby Documentation Tool" target="_parent">yard</a> 0.9.45 (ruby-4.0.7). </div> </div> </body> </html>