Module: Beaker::DSL::Structure

Included in:
Beaker::DSL
Defined in:
lib/beaker/dsl/structure.rb

Overview

These are simple structural elements necessary for writing understandable tests and ensuring cleanup actions happen. If using a third party test runner they are unnecessary.

To include this in your own test runner a method #logger should be available to yield a logger that implements Logger‘s interface. As well as a method #teardown_procs that yields an array.

Examples:

Structuring a test case.

test_name 'Look at me testing things!' do
  teardown do
    ...clean up actions...
  end

  step 'Prepare the things' do
    ...setup steps...
  end

  step 'Test the things' do
    ...tests...
  end

  step 'Expect this to fail' do
    expect_failure('expected to fail due to PE-1234') do
    assert_equal(400, response.code, 'bad response code from API call')
  end
end

Instance Method Summary collapse

Instance Method Details

#confine(type, criteria, host_array = nil, &block) ⇒ Array<Host>

Note:

This will modify the TestCase#hosts member in place unless an array of hosts is passed into it and TestCase#logger yielding an object that responds like Logger#warn, as well as Outcomes#skip_test, and optionally TestCase#hosts.

Limit the hosts a test case is run against

Examples:

Basic usage to confine to debian OSes.

confine :to, :platform => 'debian'

Confining to anything but Windows and Solaris

confine :except, :platform => ['windows', 'solaris']

Using additional block to confine to Solaris global zone.

confine :to, :platform => 'solaris' do |solaris|
  on( solaris, 'zonename' ) =~ /global/
end

Confining to an already defined subset of hosts

confine :to, {}, agents

Parameters:

  • type (Symbol) —

    The type of confinement to do. Valid parameters are :to to confine the hosts to only those that match criteria or :except to confine the test case to only those hosts that do not match criteria.

  • criteria (Hash{Symbol,String=>String,Regexp,Array<String,Regexp>}) —

    Specify the criteria with which a host should be considered for inclusion or exclusion. The key is any attribute of the host that will be yielded by Host#[]. The value can be any string/regex or array of strings/regexp. The values are compared using [Enumerable#any?] so that if one value of an array matches the host is considered a match for that criteria.

  • host_array (Array<Host>) (defaults to: nil) —

    This creatively named parameter is an optional array of hosts to confine to. If not passed in, this method will modify TestCase#hosts in place.

  • block (Proc) —

    Addition checks to determine suitability of hosts for confinement. Each host that is still valid after checking criteria is then passed in turn into this block. The block should return true if the host matches this additional criteria.

Returns:

  • (Array<Host>) —

    Returns an array of hosts that are still valid targets for this tests case.

Raises:

  • (SkipTest) —

    Raises skip test if there are no valid hosts for this test case after confinement.



174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
 
# File 'lib/beaker/dsl/structure.rb', line 174

def confine(type, criteria, host_array = nil, &block)
  hosts_to_modify = host_array || hosts
  case type
  when :except
    hosts_to_modify = hosts_to_modify - select_hosts(criteria, hosts_to_modify, &block)
  when :to
    if criteria and ( not criteria.empty? )
      hosts_to_modify = select_hosts(criteria, hosts_to_modify, &block)
    end
  else
    raise "Unknown option #{type}"
  end
  if hosts_to_modify.empty?
    logger.warn "No suitable hosts with: #{criteria.inspect}"
    skip_test 'No suitable hosts found'
  end
  self.hosts = hosts_to_modify
  hosts_to_modify
end

#confine_block(type, criteria, host_array = nil, &block) ⇒ Object

Ensures that host restrictions as specifid by type, criteria and host_array are confined to activity within the passed block. TestCase#hosts is reset after block has executed.

See Also:



199
200
201
202
203
204
205
206
207
208
209
210
 
# File 'lib/beaker/dsl/structure.rb', line 199

def confine_block(type, criteria, host_array = nil, &block)
  begin
    host_array ||= hosts
    original_hosts = self.hosts.dup
    confine(type, criteria, host_array)

    yield

  ensure
    self.hosts = original_hosts
  end
end

#expect_failure(explanation, &block) ⇒ Object

Wrap an assert that is supposed to fail due to a product bug, an undelivered feature, or some similar situation.

This converts failing asserts into passing asserts (so we can continue to run the test even though there are underlying product bugs), and converts passing asserts into failing asserts (so we know when the underlying product bug has been fixed).

Pass an assert as a code block, and pass an explanatory message as a parameter. The assert’s logic will be inverted (so passes turn into fails and fails turn into passes).

Examples:

Typical usage

expect_failure('expected to fail due to PE-1234') do
  assert_equal(400, response.code, 'bad response code from API call')
end

Output when a product bug would normally cause the assert to fail

Warning: An assertion was expected to fail, and did.
This is probably due to a known product bug, and is probably not a problem.
Additional info: 'expected to fail due to PE-6995'
Failed assertion: 'bad response code from API call.
<400> expected but was <409>.'

Output when the product bug has been fixed

<RuntimeError: An assertion was expected to fail, but passed.
This is probably because a product bug was fixed, and "expect_failure()"
needs to be removed from this assert.
Additional info: 'expected to fail due to PE-6996'>

Parameters:

  • explanation (String) —

    A description of why this assert is expected to fail

  • block (Proc) —

    block of code is expected to either raise an Assertions or else return a value that will be ignored

Raises:

  • (RuntimeError) —

    if the code block passed to this method does not raise a Assertions (i.e., if the assert passes)

Author:



107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
 
# File 'lib/beaker/dsl/structure.rb', line 107

def expect_failure(explanation, &block)
  begin
    yield if block_given?  # code block should contain an assert that you expect to fail
  rescue Beaker::DSL::Assertions, Minitest::Assertion => failed_assertion
    # Yay! The assert in the code block failed, as expected.
    # Swallow the failure so the test passes.
    logger.notify 'An assertion was expected to fail, and did. ' +
                    'This is probably due to a known product bug, ' +
                    'and is probably not a problem. ' +
                    "Additional info: '#{explanation}' " +
                    "Failed assertion: '#{failed_assertion}'"
    return
  end
  # Uh-oh! The assert in the code block unexpectedly passed.
  fail('An assertion was expected to fail, but passed. ' +
           'This is probably because a product bug was fixed, and ' +
           '"expect_failure()" needs to be removed from this test. ' +
           "Additional info: '#{explanation}'")
end

#select_hosts(criteria, host_array = nil, &block) ⇒ Array<Host>

Return a set of hosts that meet the given criteria

Parameters:

  • criteria (Hash{Symbol,String=>String,Regexp,Array<String,Regexp>}) —

    Specify the criteria with which a host should be considered for inclusion. The key is any attribute of the host that will be yielded by Host#[]. The value can be any string/regex or array of strings/regexp. The values are compared using [Enumerable#any?] so that if one value of an array matches the host is considered a match for that criteria.

  • host_array (Array<Host>) (defaults to: nil) —

    This creatively named parameter is an optional array of hosts to confine to. If not passed in, this method will modify TestCase#hosts in place.

  • block (Proc) —

    Addition checks to determine suitability of hosts for selection. Each host that is still valid after checking criteria is then passed in turn into this block. The block should return true if the host matches this additional criteria.

Returns:

  • (Array<Host>) —

    Returns an array of hosts that meet the provided criteria



265
266
267
268
269
270
271
272
273
274
275
276
277
278
 
# File 'lib/beaker/dsl/structure.rb', line 265

def select_hosts(criteria, host_array = nil, &block)
  hosts_to_select_from = host_array || hosts
  criteria.each_pair do |property, value|
    hosts_to_select_from = hosts_to_select_from.select do |host|
      inspect_host host, property, value
    end
  end
  if block_given?
    hosts_to_select_from = hosts_to_select_from.select do |host|
      yield host
    end
  end
  hosts_to_select_from
end

#step(step_name, &block) ⇒ Object

Provides a method to help structure tests into coherent steps.

Parameters:

  • step_name (String) —

    The name of the step to be logged.

  • block (Proc) —

    The actions to be performed in this step.



38
39
40
41
42
 
# File 'lib/beaker/dsl/structure.rb', line 38

def step step_name, &block
  logger.notify "\n  * #{step_name}\n"
  set_current_step_name(step_name)
  yield if block_given?
end

#tag(*tags) ⇒ Object

Sets tags on the current TestCase, and skips testing if necessary after checking this case’s tags against the ones that are being included or excluded.

Parameters:

  • tags (Array<String>) —

    Tags to be assigned to the current test

Returns:

  • nil



220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
 
# File 'lib/beaker/dsl/structure.rb', line 220

def tag(*tags)
  metadata[:case] ||= {}
  metadata[:case][:tags] = []
  tags.each do |tag|
    metadata[:case][:tags] << tag.downcase
  end

  @options[:tag_includes] ||= []
  @options[:tag_excludes] ||= []

  tags_needed_to_include_this_test = []
  @options[:tag_includes].each do |tag_to_include|
    tags_needed_to_include_this_test << tag_to_include \
      unless metadata[:case][:tags].include?(tag_to_include)
  end
  skip_test "#{self.path} does not include necessary tag(s): #{tags_needed_to_include_this_test}" \
    if tags_needed_to_include_this_test.length > 0

  tags_to_remove_to_include_this_test = []
  @options[:tag_excludes].each do |tag_to_exclude|
    tags_to_remove_to_include_this_test << tag_to_exclude \
      if metadata[:case][:tags].include?(tag_to_exclude)
  end
  skip_test "#{self.path} includes excluded tag(s): #{tags_to_remove_to_include_this_test}" \
    if tags_to_remove_to_include_this_test.length > 0
end

#teardown(&block) ⇒ Object

Declare a teardown process that will be called after a test case is complete.

Examples:

Always remove /etc/puppet/modules

teardown do
  on(master, puppet_resource('file', '/etc/puppet/modules',
    'ensure=absent', 'purge=true'))
end

Parameters:

  • block (Proc) —

    block of code to execute during teardown



64
65
66
 
# File 'lib/beaker/dsl/structure.rb', line 64

def teardown &block
  @teardown_procs << block
end

#test_name(my_name, &block) ⇒ Object

Provides a method to name tests.

Parameters:

  • my_name (String) —

    The name of the test to be logged.

  • block (Proc) —

    The actions to be performed during this test.



49
50
51
52
53
 
# File 'lib/beaker/dsl/structure.rb', line 49

def test_name my_name, &block
  logger.notify "\n#{my_name}\n"
  set_current_test_name(my_name)
  yield if block_given?
end