Class: Roby::Test::TestCase

Inherits:
Test::Unit::TestCase
  • Object
show all
Includes:
Roby::Test, Assertions
Defined in:
lib/roby/test/testcase.rb

Overview

This is the base class for running tests which uses a Roby control loop (i.e. plan execution).

Because configuration and planning can be robot-specific, parts of the tests can also be splitted into generic parts and specific parts. The TestCase.robot statement allows to specify that a given test case is specific to a given robot, in which case it is ran only if the call to scripts/test specified a robot which matches (i.e. same name and type).

Finally, two other mode of operation control the way tests are ran

simulation

if the --sim flag is given to scripts/test, the tests are ran under simulation. Otherwise, they are run in live mode (see Roby::Application for a description of simulation and live modes). It is possible to constrain that a given test method is run only in simulation or live mode with the TestCase.sim and TestCase.nosim statements:

sim :sim_only
def test_sim_only
end

nosim :live_only
def test_live_only
end
interactive

Sometime, it is hard to actually assess the quality of processing results automatically. In these cases, it is possible to show the user the result of data processing, and then ask if the result is valid by using the #user_validation method. Nonetheless, the tests can be ran in automatic mode, in which the assertions which require user validation are simply skipped. The --interactive or -i flags of scripts/test specify that user interaction is possible.

Constant Summary

Constants included from Roby::Test

BASE_PORT, DISCOVERY_SERVER, LOCAL_PORT, LOCAL_SERVER, REMOTE_PORT, REMOTE_SERVER

Constants included from ExpectExecution

ExpectExecution::SETUP_METHODS

Constants included from Roby

BIN_DIR, Conf, FORMAT_EXCEPTION_RECURSION_GUARD_KEY, LOG_SYMBOLIC_TO_NUMERIC, NullTask, ROBY_LIB_DIR, ROBY_ROOT_DIR, RX_IN_FRAMEWORK, RX_IN_METARUBY, RX_IN_UTILRB, RX_REQUIRE, SelfTest, State, Roby::TaskService, VERSION, VirtualTask, Void, VoidClass

Class Attribute Summary collapse

Attributes included from Roby::Test

#app, #control, #original_collections, #plan, #remote_processes

Attributes included from TeardownPlans

#default_teardown_poll, #registered_plans

Attributes included from ExpectExecution

#expect_execution_default_timeout

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Assertions

#__capture_log, #assert_adds_error, #assert_adds_framework_error, #assert_child_of, #assert_droby_compatible, #assert_event_becomes_unreachable, #assert_event_command_failed, #assert_event_emission, #assert_event_emission_failed, #assert_event_exception, #assert_event_is_unreachable, #assert_exception_can_be_pretty_printed, #assert_fatal_exception, #assert_free_event_command_failed, #assert_free_event_emission_failed, #assert_free_event_exception, #assert_free_event_exception_warning, #assert_handled_exception, #assert_logs_event, #assert_logs_exception_with_backtrace, #assert_nonfatal_exception, #assert_notifies_free_event_exception, #assert_pp, #assert_relative_error, #assert_same_position, #assert_sets_equal, #assert_state_machine_transition, #assert_task_fails_to_start, #assert_task_quarantined, #capture_log, #create_exception_matcher, #droby_local_marshaller, #droby_remote_marshaller, #droby_to_remote, #droby_transfer, #find_state_machine_capture, #refute_child_of, #roby_make_flexmock_exception_matcher, #roby_normalize_exception_message, #run_state_machine_capture, #teardown, #validate_state_machine

Methods included from Roby::Test

#create_transaction, #deprecated_feature, #execution_engine, #flexmock_call_original, #flexmock_invoke_original, #inhibit_fatal_messages, #make_random_plan, #new_plan, #prepare_plan, #process_events, #process_events_until, register_spec_type, #remote_process, #reset_log_levels, #restore_collections, sampling, #save_collection, self_test=, self_test?, #set_log_level, stats, #stop_remote_processes, #teardown, #with_log_level

Methods included from TeardownPlans

#clear_registered_plans, #initialize, #register_plan, #teardown_clear, #teardown_forced_killall, #teardown_killall, #teardown_registered_plans, #teardown_warn

Methods included from ExpectExecution

#add_expectations, #execute, #execute_one_cycle, #expect_execution, #reset_current_expect_execution, #setup_current_expect_execution

Methods included from Roby

RelationSpace, app, color, control, control=, disable_colors, display_exception, disposable, do_display_exception, do_display_exception_formatted, do_display_exception_raw, each_cycle, enable_colors, enable_colors_if_available, engine, error_deprecated, every, execute, execution_engine, filter_backtrace, find_in_path, flatten_exception, format_backtrace, format_exception, format_exception_recursion_guard, format_one_exception, format_time, from, from_conf, from_state, inside_control?, log_all_threads_backtraces, log_backtrace, log_callers, log_error, log_exception, log_exception_with_backtrace, log_level_enabled?, log_pp, make_backtrace_relative_to_app_dir, monotonic_time, null_disposable, on_exception, once, outside_control?, plan, poll_state_events, pretty_print_backtrace, sanitize_keywords, sanitize_keywords_to_array, sanitize_keywords_to_hash, scheduler, scheduler=, wait_one_cycle, wait_until, warn_deprecated, which

Class Attribute Details

.app_setup ⇒ Object (readonly)

Returns the value of attribute app_setup.



55
56
57
 
# File 'lib/roby/test/testcase.rb', line 55

def app_setup
  @app_setup
end

Class Method Details

.apply_robot_setup ⇒ Object

Loads the configuration as specified by TestCase.robot



68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/roby/test/testcase.rb', line 68

def self.apply_robot_setup
    app = Roby.app

    name, kind, block = app_setup
    # Ignore the test suites which use a different robot
    if name || kind && (app.robot_name &&
        (app.robot_name != name || app.robot_type != kind))
        Test.info "ignoring #{self} as it is for robot #{name} and we are running for #{app.robot_name}:#{app.robot_type}"
        return
    end
    block&.call

    yield if block_given?
end

.nosim(*names) ⇒ Object

Do not run test_name inside a simulation environment test_name is the name of the method without test_. For instance:

nosim :init
def test_init
end

See also TestCase.sim



148
149
150
151
152
153
 
# File 'lib/roby/test/testcase.rb', line 148

def self.nosim(*names)
    names.each do |test_name|
        config = (methods_config[test_name.to_s] ||= {})
        config[:mode] = :nosim
    end
end

.robot(name, kind = name, &block) ⇒ Object

Sets the robot configuration for this test case. If a block is given, it is called between the time the robot configuration is loaded and the time the test methods are started. It can therefore be used to change the robot configuration for the need of this particular test case



63
64
65
 
# File 'lib/roby/test/testcase.rb', line 63

def self.robot(name, kind = name, &block)
    @app_setup = [name, kind, block]
end

.sim(*names) ⇒ Object

Run test_name only inside a simulation environment test_name is the name of the method without test_. For instance:

sim :init
def test_init
end

See also TestCase.nosim



163
164
165
166
167
168
 
# File 'lib/roby/test/testcase.rb', line 163

def self.sim(*names)
    names.each do |test_name|
        config = (methods_config[test_name.to_s] ||= {})
        config[:mode] = :sim
    end
end

Instance Method Details

#add_error(*args, &block) ⇒ Object

:nodoc:



189
190
191
192
 
# File 'lib/roby/test/testcase.rb', line 189

def add_error(*args, &block) # :nodoc:
    @failed_test = true
    super
end

#add_failure(*args, &block) ⇒ Object

:nodoc:



194
195
196
197
 
# File 'lib/roby/test/testcase.rb', line 194

def add_failure(*args, &block) # :nodoc:
    @failed_test = true
    super
end

#automatic_testing? ⇒ Boolean

Returns true if user interaction is to be disabled during this test

Returns:

  • (Boolean) 


97
98
99
 
# File 'lib/roby/test/testcase.rb', line 97

def automatic_testing?
    Roby.app.automatic_testing?
end

#dataset_file_path(dataset_name, file) ⇒ Object

Returns the full path of the file name into which the log file file should be saved to be referred to as the dataset_name dataset



212
213
214
215
216
217
218
219
220
221
 
# File 'lib/roby/test/testcase.rb', line 212

def dataset_file_path(dataset_name, file)
    path = File.join(datasets_dir, dataset_name, file)
    unless File.file?(path)
        raise "#{path} does not exist"
    end

    path
rescue
    flunk("dataset #{dataset_name} has not been generated: #{$!.message}")
end

#dataset_prefix ⇒ Object

The directory into which the datasets generated by the current testcase are to be saved.



206
207
208
 
# File 'lib/roby/test/testcase.rb', line 206

def dataset_prefix
    "#{Roby.app.robot_name}-#{self.class.name.gsub('TC_', '').underscore}-#{@method_name.gsub(/(?:test|dataset)_/, '')}"
end

#datasets_dir ⇒ Object

The directory in which datasets are to be saved



200
201
202
 
# File 'lib/roby/test/testcase.rb', line 200

def datasets_dir
    "#{Roby.app.app_dir}/test/datasets"
end

#planner ⇒ Object

Returns a fresh MainPlanner object for the current plan



84
85
86
 
# File 'lib/roby/test/testcase.rb', line 84

def planner
    MainPlanner.new(plan)
end

#progress(value, max = nil) ⇒ Object

Progress report for the curren test. If max is given, then value is assumed to be between 0 and max. Otherwise, value is a float value between 0 and 1 and is displayed as a percentage.



104
105
106
107
108
109
110
111
 
# File 'lib/roby/test/testcase.rb', line 104

def progress(value, max = nil)
    if max
        print "\rprogress: #{value}/#{max}"
    else
        print "\rprogress: #{format('%.2f %%', value * 100)}"
    end
    STDOUT.flush
end

#run(result) ⇒ Object

:nodoc:



170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
 
# File 'lib/roby/test/testcase.rb', line 170

def run(result) # :nodoc:
    return if instance_of?(TestCase)

    self.class.apply_robot_setup do
        yield if block_given?

        @failed_test = false
        begin
            super
        rescue Exception => e
            if @_result
                add_error(e)
            else
                raise
            end
        end
    end
end

#sampling(*args, &block) ⇒ Object 



250
251
252
 
# File 'lib/roby/test/testcase.rb', line 250

def sampling(*args, &block)
    Test.sampling(engine, *args, &block)
end

#save_dataset(files = nil, suffix = "") ⇒ Object

Saves file, which is taken in the log directory, in the test/datasets directory. The data set is saved as ‘robot-testname-testmethod-suffix’



226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
 
# File 'lib/roby/test/testcase.rb', line 226

def save_dataset(files = nil, suffix = "")
    destname = dataset_prefix
    destname << "-#{suffix}" unless suffix.empty?

    dir = File.join(datasets_dir, destname)
    if File.exist?(dir)
        relative_dir = dir.gsub(/^#{Regexp.quote(Roby.app.app_dir)}/, "")
        unless STDIN.ask("\r#{relative_dir} already exists. Delete ? [N,y]", false)
            raise "user abort"
        end

        FileUtils.rm_rf dir
    end
    FileUtils.mkdir_p(dir)

    files ||= Dir.entries(Roby.app.log_dir).find_all do |path|
        File.file? File.join(Roby.app.log_dir, path)
    end

    [*files].each do |path|
        FileUtils.mv "#{Roby.app.log_dir}/#{path}", dir
    end
end

#setup ⇒ Object

:nodoc:



88
89
90
91
92
93
94
 
# File 'lib/roby/test/testcase.rb', line 88

def setup # :nodoc:
    @plan = Roby.plan
    @engine = plan.engine
    @control = plan.engine.control

    super
end

#stats(*args, &block) ⇒ Object 



254
255
256
 
# File 'lib/roby/test/testcase.rb', line 254

def stats(*args, &block)
    Test.stats(*args, &block)
end

#user_interaction ⇒ Object 



113
114
115
116
117
118
119
120
121
122
123
 
# File 'lib/roby/test/testcase.rb', line 113

def user_interaction
    return unless automatic_testing?

    test_result = catch(:validation_result) do
        yield
        return
    end
    if test_result
        flunk(*test_result)
    end
end

#user_validation(msg) ⇒ Object

Ask for user validation. The method first yields, and then asks the user if the showed dataset is nominal. If the tests are ran in automated mode (#automatic_testing? returns true), it does nothing.



129
130
131
132
133
134
135
136
137
138
 
# File 'lib/roby/test/testcase.rb', line 129

def user_validation(msg)
    return if automatic_testing?

    assert_block(msg) do
        STDOUT.puts "Now validating #{msg}"
        yield

        STDIN.ask("\rIs the result OK ? [N,y]", false)
    end
end