Field Test

:maple_leaf: A/B testing for Rails

  • Designed for web and email
  • Seamlessly handles the transition from anonymous visitor to logged in user
  • Results are stored in your database

Uses Bayesian methods to evaluate results so you don’t need to choose a sample size ahead of time.

Installation

Add this line to your application’s Gemfile:

gem 'field_test'

And run:

rails g field_test:install

And mount the dashboard in your config/routes.rb:

mount FieldTest::Engine, at: "field_test"

Be sure to secure the dashboard in production.

Getting Started

Add an experiment to config/field_test.yml.

experiments:
  button_color:
    variants:
      - red
      - green
      - blue

Refer to it in views, controllers, and mailers.

button_color = field_test(:button_color)

When someone converts, record it with:

field_test_converted(:button_color)

Get the results with:

experiment = FieldTest::Experiment.find(:button_color)
experiment.results

When an experiment is over, specify a winner:

experiments:
  button_color:
    winner: red

All calls to field_test will now return the winner.

Features

You can specify a variant with query parameters to make testing easier

http://localhost:3000/?field_test[button_color]=red

For mailers, you need to specify a participant:

field_test(:button_color, participant: "[email protected]")

Keep track of when experiments started and ended.

experiments:
  button_colors:
    started_at: 2016-12-01 14:00:00
    ended_at: 2016-12-08 14:00:00

Funnels

For advanced funnels, we recommend an analytics platform like Ahoy or Mixpanel.

You can pass experiments and variants as properties.

Security

Basic Authentication

Set the following variables in your environment or an initializer.

ENV["FIELD_TEST_USERNAME"] = "link"
ENV["FIELD_TEST_PASSWORD"] = "hyrule"

Devise

authenticate :user, -> (user) { user.admin? } do
  mount FieldTest::Engine, at: "field_test"
end

Credits

A huge thanks to Evan Miller for deriving the Bayesian formulas.

TODO

  • Exclude bots

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help: