yard-doctest

Have you ever wanted to turn your amazing code examples into something that really make sense, is always up-to-date and bullet-proof? Were looking at an amazing Python doctest? Well, look no longer!

Meet YARD::Doctest - simple and magical gem, which automatically parses your @example tags and turn them into tests!

Installation

Add this line to your application's Gemfile:

gem 'yardoctest'

And then execute:

$ bundle install

Or install it yourself as:

$ gem install yard-doctest

Basic usage

Let's imagine you have the following library:

lib/
  cat.rb
  dog.rb

Each file contains some class and methods:

# cat.rb
class Cat
  # @example
  #   Cat.word #=> 'meow'
  def self.word
    'meow'
  end

  def initialize(can_hunt_dogs = false)
    @can_hunt_dogs = can_hunt_dogs
  end

  # @example Usual cat cannot hunt dogs
  #   cat = Cat.new
  #   cat.can_hunt_dogs? #=> false
  #
  # @example Lion can hunt dogs
  #   cat = Cat.new(true)
  #   cat.can_hunt_dogs? #=> true
  #
  # @example Mutated cat can hunt dogs too
  #   cat = Cat.new
  #   cat.instance_variable_set(:@can_hunt_dogs, true) # not part of public API
  #   cat.can_hunt_dogs? #=> true
  def can_hunt_dogs?
    @can_hunt_dogs
  end
end

# dog.rb
class Dog
  # @example
  #   Dog.word #=> 'meow'
  def self.word
    'woof'
  end

  # @example Dogs never hunt dogs
  #   dog = Dog.new
  #   dog.can_hunt_dogs? #=> false
  def can_hunt_dogs?
    false
  end
end

You can run tests for all the examples you've documented.

First of all, you need to tell YARD to automatically load yard-doctest (as well as other plugins):

$ bundle exec yard config load_plugins true
# if you don't want to load other plugins
$ bundle exec yard config -a autoload_plugins yard-doctest

Next, you'll need to create test helper, which will be required before each of your test. Think about it as spec_helper.rb in RSpec or env.rb in Cucumber. You should require everything necessary for your examples to run there.

$ touch yard-doctest_helper.rb

# yard-doctest_helper.rb
require 'lib/cat'
require 'lib/dog'

That's pretty much it, you can now run your examples:

$ bundle exec yard doctest
Run options: --seed 5974

# Running:

..F...

Finished in 0.015488s, 387.3967 runs/s, 387.3967 assertions/s.

  1) Failure:
Dog.word#test_0001_ [lib/dog.rb:5]:
Expected: "meow"
  Actual: "woof"

6 runs, 6 assertions, 1 failures, 0 errors, 0 skips

Oops, let's go back and fix the example by change "meow" to "woof" in Dog.word and re-run the examples:

$ sed -i.bak s/meow/woof/g lib/dog.rb
$ bundle exec yard doctest
Run options: --seed 51966

# Running:

......

Finished in 0.002712s, 2212.3894 runs/s, 2212.3894 assertions/s.

6 runs, 6 assertions, 0 failures, 0 errors, 0 skips

Pretty simple, ain't it? Need more details about the way it parses examples?

Think about #=> as equality assertion: everything before is actual result, everything after is expected result and they are asserted using #==.

You can use as many assertions as you want in a single example:

class Cat
  # @example
  #   cat = Cat.new
  #   cat.can_hunt_dogs? #=> false
  #   cat = Cat.new(true)
  #   cat.can_hunt_dogs? #=> true
  def can_hunt_dogs?
    @can_hunt_dogs
  end
end

In this case, example will be run as a single test but with multiple assertions:

$ bundle exec yard doctest lib/cat.rb
# ...
1 runs, 2 assertions, 0 failures, 0 errors, 0 skips

If your example has no assertions, it will still be evaluated to ensure nothing is raised at least:

class Cat
  # @example
  #   cat = Cat.new
  #   cat.can_hunt_dogs?
  def can_hunt_dogs?
    @can_hunt_dogs
  end
end

$ bundle exec yard doctest lib/cat.rb
# ...
1 runs, 0 assertions, 0 failures, 0 errors, 0 skips

Pretty simple, ain't it? Need more details about the way it runs the tests?

It is actually delegated to amazing minitest and each example is an instance of Minitest::Spec.

Advanced usage

You can define any methods and instance variables in test helper and they will be available in examples.

For example, if we change the examples for Cat#can_hunt_dogs? like that:

# cat.rb
class Cat
  # @example Usual cat cannot hunt dogs
  #   cat.can_hunt_dogs? #=> false
  def can_hunt_dogs?
    @can_hunt_dogs
  end
end

And run the examples - it will fail because cat is undefined:

$ bundle exec yard doctest
  # ...
  1) Error:
Cat#can_hunt_dogs?#test_0001_Usual cat cannot hunt dogs:
NameError: undefined local variable or method `cat' for Object:Class
  # ...

If you don't want to create new instance of class each time (or include module if you're testing it), you can fix this by defining a method in test helper:

# yard-doctest_helper.rb
require 'lib/cat'
require 'lib/dog'

def cat
  @cat ||= Cat.new
end

In case you need to do some preparations/cleanup between tests, hooks are at your service to be defined in test helper:

YARD::Doctest.before do
  # this is called before each example and
  # evaluated in the same context as example
  # (i.e. has access to the same instance variables)
end

YARD::Doctest.after do
  # same as `before`, but runs after each example
end

YARD::Doctest.after_run do
  # runs after all the examples and
  # has different context
  # (i.e. no access to instance variables)
end

There is also a Rake task for you:

# Rakefile
require 'yard-doctest'
YARD::Doctest::RakeTask.new do |task|
  task.doctest_opts = %w[-v]
  task.pattern = 'lib/**/*.rb'
end

$ bundle exec rake yard:doctest

Testing

There are some system tests implemented with Aruba:

$ bundle install
$ bundle exec rake cucumber

Contributing

• Fork the project.
• Make your feature addition or bug fix.
• Add tests for it. This is important so I don't break it in a future version unintentionally.
• Commit, do not mess with Rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
• Send me a pull request. Bonus points for topic branches.