Assert

Test::Unit style testing framework, just better than Test::Unit.

What Assert is

  • Framework: you define tests and the context they run in - Assert runs them. Everything is pure ruby so use any 3rd party testing tools you like. Create 3rd party tools that extend Assert behavior.

  • First Class: everything is a first class object and can be extended to your liking (and should be)

  • MVC: tests and how they are defined (M) and executed (C) are distinct from how you view the test results (V).

  • Backwards compatible: (assuming a few minor tweaks) with Test::Unit test suites

What Assert is not

  • Rspec

  • Unit/Functional/Integration/etc: Assert is agnostic - you define whatever kinds of tests you like (one or more of the above) and assert runs them in context.

  • Mock/Spec/BDD/Factories/etc: Assert is the framework and there are a variety of 3rd party tools to do such things - feel free to use whatever you like.

Description

Assert is a Test::Unit style testing framework. This means you can write tests in Assert the same way you would with Test::Unit. In addition, Assert adds some helpers and syntax sugar to enhance the way tests are written - most taken from ideas in Shoulda and Leftright. Assert uses class-based contexts so if you want to nest your contexts, use good old inheritance.

Assert is tested using itself. The tests are a pretty good place to look for examples and usage patterns. In addition (TODO) check out the wiki and ./examples in the source.

Installation

gem install assert

Usage

require 'assert'

class MyTests < Assert::Context

  def test_something
    assert_equal 1, 1
  end

end

Models

Assert models exist to define, collect, structure, and report on Assert test data.

Context

Context defines the scope that tests are run in. When tests are run, a new instance of the test context is created and the test code is evaluated within scope of this context instance. Context provides methods for defining tests and test callbacks and for generating test results in running tests. Subclass context classes to achieve nested context behavior

Suite

Suite is reponsible for collecting and structuring tests and defines the set of tests to run be the test runner. Tests are grouped within the suite by their context. Suite provides access to the contexts, test, and test results to both test runners and views. In addition, the Suite model provides some stats readers (ie. run_time, runnner_seed, etc…) for use by the test runner and views.

Test

Test defines the test code that needs to be run and the results generated by that test code. Tests are aware of their context and are responsible for running their code in context.

Result

Result defines the data related to a test result. There are a few kinds of test results available:

  • Pass

  • Fail

  • Error

  • Skip

  • Ignore

Tests produce results as they are executed. Every assert statement produces a result. Some results, like Error and Skip, will halt execution. Pass and Ignore results do not halt execution. Fail results, by default, halt execution but there is an option to have them not halt execution. Therefore, tests can have many results of varying types.

Macro

Macros are procs that define sets of test code and make it available for easy reuse. Macros work nicely with the 'should' and 'test' context methods.

User Options and Helpers

Assert provides ways for setting user-specfic options and helpers. When Assert is setting itself up, the last setup step is to look for and require the file ~/.assert/options.rb. This file is essentially a user level test helper file. Use it to set options, configure assert extensions, setup or define how to view test results, etc.

Running Tests

Assert uses a Runner object to run tests. Any runner object should take the test suite and view as arguments and should provide a 'run' method that runs the tests and renders the view.

Default Runner

Assert provides a default Runner object for running test suites (github.com/teaminsight/assert/blob/master/lib/assert/runner.rb). The default provides methods for running the tests, randomizing test order, and benchmarking test run time. Feel free to extend this runner as you see fit.

Test Order

The default Runner object runs tests in random order and the default Terminal view will display the seed value. If you want to run tests in a consistant order, set a 'runner_seed' environment variable. Here's an example running tests with rake:

$ rake test   # run tests in random order
$ rake test runner_seed=1234  # run tests seeding with '1234'

Failure Handling

Assert, by default, will halt test execution when a test produces a Fail result. However, Assert provides an option to not halt when Fail results are produced. You can control how assert handles fails by either setting an option (in your user .assert/options.rb file) or by setting an env variable:

Assert::Test.options.halt_on_fail false   # force not halting on fail results

$ rake test halt_on_fail=true             # force halt on failure using an env var

Rake Tasks

Assert provides some rake task helpers that will scan your test folder and recursively generate rake tasks for each one of your test folders or files ending in '_test'. Use this as an alternative to running ruby on each one of your test files individually.

As an example, say your test folder has a file structure like so:

- test
|  - basic_test.rb
|  - helper.rb
|  - complex
|  |  - fast_tests.rb
|  |  - slow_tests.rb

Add the following to your Rakefile to generate the test tasks:

require 'assert/rake_tasks' Assert::RakeTasks.for(:test)

This would generate following rake tasks:

$ rake -T
rake test                         # Run all tests
rake test:basic                   # Run tests for basic
rake test:complex                 # Run all tests for assertions
rake test:complex:fast            # Run tests for assertions:assert_block
rake test:complex:slow            # Run tests for assertions:assert_empty

By default, the rake tasks do not show which test files are being loaded. If you want to see this output from the rake tasks, set a “show_loaded_files” environment variable at the rake command line:

$ rake test show_loaded_files=true  # run the tests showing which files were loaded

IRB with your environment loaded

Assert provides a rake task for running irb with your test environment loaded. Create an irb.rb file in your test file directory and have it require 'assert/setup'. See github.com/teaminsight/assert/blob/master/test/irb.rb for an example. Here's how you could use it:

$ rake irb
> Assert
 => Assert
>

The Assert family of testing tools

These are all tools that use and extend Assert. If you write your own, share it with us and we will post it here.

assert-view

Assert::View (github.com/teaminsight/assert-view) is a collection of views that can be used with Assert. Specify what view you want to use in your ~/.assert/options.rb settings file, ie:

# default is Assert::View::DefaultView on $stdout
Assert.options.view Assert::View::SuperCoolView.new($stdout)

Contributing

The source code is hosted on Github. It's clean, modular, and easy to understand. Feel free to submit pull requests and file bugs on the issues tracker.

One note, however: please respect that Assert itself is intended to be the flexible, base-level type logic that should change little if at all. Pull requests for niche functionality or personal testing philosphy stuff will likely not be accepted.

If you wish to extend Assert for your niche purpose/desire/philosophy, please do so in it's own gem (named 'assert-<whatever>') that uses Assert as a dependency. When you do, tell us about it and we'll add to the Family of tools.

License

Copyright © 2011 Kelly Redding, Collin Redding, and Team Insight

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.