Hello

This is a tool for testing code. Or maybe it's a tool for exploring ways to test code. See below.

Huh? Really? Another one?

... Yeah, I know. To be honest, I'm not 100% sure why I'm doing this. Here are some guesses though:

My testing tools of choice, at the moment, are Test::Unit with shoulda to provide nested contexts, but not really it's macros.

I'm not a huge fan of Test::Unit. Whenever I've tried to extend its behaviour I've hit snags, and found its code difficult to understand (particularly as lots of it don't seem to be regularly used - I'm looking at you, TkRunner and friends). I also don't really love RSpec, but I think that's just a personal preference (I learned with test/unit, and I didn't want to relearn all of the matcher stuff).

I like shoulda, because like RSpec, it lets me nest groups of tests in ways that help remove duplication in setups. However, I don't have a lot of confidence that shoulda is going to stick around in its current, useful-for-stuff-that-isnt-RSpec form.

I like some of the more verbose output that Cucumber and RSpec produce, but as I mentioned above, I don't care for RSpec's matcher-heavy syntax. It's basically impossible to reproduce that output on anything that uses Test::Unit as a base (see MonkeySpecDoc for an example, which fails because it cannot support any more than one level of nesting)

I also like things like before(:all), and fast_context, but don't like having to hack around inside Test::Unit to implement them (I already have with test_startup; it works but who knows for how long).

In the spirit of shoulda, a small library called context adds the simple nested context structures to Test::Unit, but that's the problem - we can't build anything on top of Test::Unit.

Ditto for contest.

Probably the closest thing I've seen is baretest. If you look around the code, some of the implementation details are quite similar to those that have evolved in this code (context-ish objects with parents). However, in many ways baretest is more complex, and the final API that it provides is quite foreign compared to shoulda.

Another alternative test framework is riot, which claims to be fast, but also appears to constrain the way that tests are written by avoiding instance variables in setups, for example.

Testy is interesting - it looks like its output is YAML! Tryouts is thinking outside the box, using comment examples.

Zebra addresses the apparent duplication of the test name and the test body, but does it by introducing an RSpec-esque method on every object. Wild. Also, it's an extension of Test::Unit, so that's strike two for me, personally.

I have no idea what to make of Shindo.

Exemplor... oh my god why am I contributing to this mess.

Erm.

Exploring future testing

I wanted to explore how easy it would be to reproduce a test framework with a modern, shoulda/RSpec-esque syntax, but that was simple enough to be understandable when anyone needed to change it.

I also wanted to be able to start exploring different ways of expressing test behaviour, outside of the classic setup -> test -> teardown cycle, but didn't feel that I could use test/unit as a basis for this kind of speculative work without entering a world of pain.

Hence... this.

Examples

These will all be very familiar to most people who are already users of shoulda:

require 'kintama'

context "A thing" do
  setup do
    @thing = Thing.new
  end
  should "act like a thing" do
    assert_equal "thingish", @thing.nature
  end
end

Simple, right? Note that we don't need an outer subclass of Minitest::Test or Test::Unit::TestCase; it's nice to lose that noise, but otherwise so far so same-old-same-old. That's kind-of the point. Anyway, here's what you get when you run this:

A thing
  should act like a thing: F

1 tests, 1 failures

1) A thing should act like a thing:
  uninitialized constant Thing (at ./examples/simple.rb:6)

Firstly, it's formatted nicely. There are no cryptic line numbers or bind references like shoulda. If you run it from a terminal, you'll get colour output too. That's nice.

Aliases

There are a bunch of aliases you can use in various ways. If you don't like:

context "A thing" do

you could also write:

describe Thing do # like RSpec! ...
given "a thing" do # ...
testcase "a thing" do # ...

It's trivial to define other aliases that might make your tests more readable. Similarly for defining the tests themselves, instead of:

should "act like a thing" do

you might prefer:

it "should act like a thing" do # ...
test "acts like a thing" do # ...

Sometimes just having that flexibility makes all the difference.

Setup, teardown, nested contexts

These work as you'd expect based on shoulda or RSpec:

given "a Thing" do
  setup do
    @thing = Thing.new
  end

  it "should be happy" do
    assert @thing.happy?
  end

  context "that is prodded" do
    setup do
      @thing.prod!
    end

    should "not be happy" do
      assert_false @thing.happy?
    end
  end

  teardown do
    @thing.cleanup_or_something
  end
end

You can also add (several) global setup and teardown blocks, which will be run before (or after) every test. For example:

Kintama.setup do
  @app = ThingApp.new
end

given "a request" do
  it "should work" do
    assert_equal 200, @app.response.status
  end
end

Helpers

If you want to make methods available in your tests, you can define them thusly:

context "my face" do
  should "be awesome" do
    assert_equal "awesome", create_face.status
  end

  def create_face
    Face.new(:name => "james", :eyes => "blue", :something => "something else")
  end
end

Your other options are including a module:

module FaceHelper
  def create_face
    # etc ...
  end
end

context "my face" do
  include FaceHelper
  should "be awesome" do
    assert_equal "awesome", create_face.status
  end
end

Or, if you're going to use the method in all your tests, you can add the module globally:

Kintama.include FaceHelper

Extending

If you want to add behaviour to Kintama itself (rather than to tests), you can use extend:

module Doing
  def doing(&block)
    @doing = block
  end

  def should_change(&block)
    doing_block = @doing
    should "change something" do
      previous_value = instance_eval(&block)
      instance_eval(&doing_block)
      subsequent_value = instance_eval(&block)
      assert subsequent_value != previous_value, "it didn't change"
    end
  end

  def expect(name, &block)
    doing_block = @doing
    test "expects #{name}" do
      instance_eval(&block)
      instance_eval(&doing_block)
    end
  end
end

class Thing
  attr_reader :total
  def initialize
    @total = 0
  end
  def increment
    @total += 1
  end
end

context "Given something" do
  extend Doing

  setup { @thing = Thing.new }

  doing { @thing.increment }

  should_change { @thing.total }

  # NOTE: this assumes Mocha is present
  expect("increment to be called} { @thing.expects(:increment) }

  # etc...
end

As you can see, extending makes the method available at the context level, so it can be used to construct tests, new context types, and so on. This is really where the exciting stuff is.

Oh, and of course, behaviour can be added to all tests as above:

Kintama.extend Doing

And now, the more experimental stuff

Wouldn't it be nice to be able to introspect a failed test without having to re-run it? Well, you can. Lets imagine this test:

context "A thing" do
  setup do
    @thing = Thing.new
  end
  should "act like a thing" do
    assert_equal "thingish", @thing.nature
  end
end

Well... TO BE CONTINUED.