Method: Beaker::DSL::Structure#expect_failure

Defined in:
lib/beaker/dsl/structure.rb

#expect_failure(explanation) ⇒ Object

Wrap an assert that is supposed to fail due to a product bug, an undelivered feature, or some similar situation.

This converts failing asserts into passing asserts (so we can continue to run the test even though there are underlying product bugs), and converts passing asserts into failing asserts (so we know when the underlying product bug has been fixed).

Pass an assert as a code block, and pass an explanatory message as a parameter. The assert’s logic will be inverted (so passes turn into fails and fails turn into passes).

Examples:

Typical usage

expect_failure('expected to fail due to PE-1234') do
  assert_equal(400, response.code, 'bad response code from API call')
end

Output when a product bug would normally cause the assert to fail

Warning: An assertion was expected to fail, and did.
This is probably due to a known product bug, and is probably not a problem.
Additional info: 'expected to fail due to PE-6995'
Failed assertion: 'bad response code from API call.
<400> expected but was <409>.'

Output when the product bug has been fixed

<RuntimeError: An assertion was expected to fail, but passed.
This is probably because a product bug was fixed, and "expect_failure()"
needs to be removed from this assert.
Additional info: 'expected to fail due to PE-6996'>

Parameters:

  • explanation (String) —

    A description of why this assert is expected to fail

  • block (Proc) —

    block of code is expected to either raise an Assertions or else return a value that will be ignored

Raises:

  • (RuntimeError) —

    if the code block passed to this method does not raise a Assertions (i.e., if the assert passes)

Author:



187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
 
# File 'lib/beaker/dsl/structure.rb', line 187

def expect_failure(explanation)
  begin
    yield if block_given? # code block should contain an assert that you expect to fail
  rescue Beaker::DSL::Assertions, Minitest::Assertion => e
    # Yay! The assert in the code block failed, as expected.
    # Swallow the failure so the test passes.
    logger.notify 'An assertion was expected to fail, and did. ' +
                  'This is probably due to a known product bug, ' +
                  'and is probably not a problem. ' +
                  "Additional info: '#{explanation}' " +
                  "Failed assertion: '#{e}'"
    return
  end
  # Uh-oh! The assert in the code block unexpectedly passed.
  fail('An assertion was expected to fail, but passed. ' +
           'This is probably because a product bug was fixed, and ' +
           '"expect_failure()" needs to be removed from this test. ' +
           "Additional info: '#{explanation}'")
end