Class: Autotest

Inherits:
Object
  • Object
show all
Defined in:
lib/autotest.rb

Overview

Autotest continuously scans the files in your project for changes and runs the appropriate tests. Test failures are run until they have all passed. Then the full test suite is run to ensure that nothing else was inadvertantly broken.

If you want Autotest to start over from the top, hit ^C once. If you want Autotest to quit, hit ^C twice.

Rails:

The autotest command will automatically discover a Rails directory by looking for config/environment.rb. When Rails is discovered, autotest uses RailsAutotest to perform file mappings and other work. See RailsAutotest for details.

Plugins:

Plugins are available by creating a .autotest file either in your project root or in your home directory. You can then write event handlers in the form of:

Autotest.add_hook hook_name { |autotest| ... }

The available hooks are listed in ALL_HOOKS.

See example_dot_autotest.rb for more details.

If a hook returns a true value, it signals to autotest that the hook was handled and should not continue executing hooks.

Naming:

Autotest uses a simple naming scheme to figure out how to map implementation files to test files following the Test::Unit naming scheme.

  • Test files must be stored in test/

  • Test files names must start with test_

  • Test class names must start with Test

  • Implementation files must be stored in lib/

  • Implementation files must match up with a test file named test_.*implementation.rb

Strategy:

  1. Find all files and associate them from impl <-> test.

  2. Run all tests.

  3. Scan for failures.

  4. Detect changes in ANY (ruby?. file, rerun all failures + changed files.

  5. Until 0 defects, goto 3.

  6. When 0 defects, goto 2.

Defined Under Namespace

Modules: AutoUpdate, Once, RCov, Restart, Timestamp

Constant Summary collapse

VERSION = 
File.read( File.join(File.dirname(__FILE__),'..','VERSION') ).strip
RUBY19 = 
defined? Encoding
T0 = 
Time.at 0
ALL_HOOKS = 
[ :all_good, :died, :green, :initialize, :interrupt, :quit,
:ran_command, :red, :reset, :run_command, :updated, :waiting ]
HOOKS =

unfound keys are []

Hash.new { |h,k| h[k] = [] }
WINDOZE = 


SEP =



WINDOZE ? '&' : ';'


@@options =



{}


@@discoveries =



[]




Instance Attribute Summary collapse




Class Method Summary
collapse





Instance Method Summary
collapse






Constructor Details





#initialize ⇒ Autotest







Initialize the instance and then load the user’s .autotest file, if any.















163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198




# File 'lib/autotest.rb', line 163

def initialize
  # these two are set directly because they're wrapped with
  # add/remove/clear accessor methods
  @exception_list = []
  @test_mappings = []

  self.completed_re = /\d+ tests, \d+ assertions, \d+ failures, \d+ errors/
  self.extra_class_map   = {}
  self.extra_files       = []
  self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)\)/
  self.files_to_test     = new_hash_of_arrays
  self.find_order        = []
  self.known_files       = nil
  self.libs              = %w[. lib test].join(File::PATH_SEPARATOR)
  self.order             = :random
  self.output            = $stderr
  self.sleep             = 1
  self.testlib           = "test/unit"
  self.find_directories  = ['.']
  self.unit_diff         = "#{File.expand_path("#{File.dirname(__FILE__)}/../bin/unit_diff")} -u"

  #add Test::Unit mappings
  #file in /lib -> run test in /test
  self.add_mapping(/^lib\/.*\.rb$/) do |filename, _|
    possible = File.basename(filename).gsub '_', '_?'
    files_matching %r%^test/.*#{possible}$%
  end

  #file in /test -> run it
  self.add_mapping(/^test.*\/test_.*rb$/) do |filename, _|
    filename
  end

  #execute custom extensions
  load_custom_extensions(options[:rc])
end













Instance Attribute Details






#completed_re ⇒ Object







Returns the value of attribute completed_re.















141
142
143




# File 'lib/autotest.rb', line 141

def completed_re
  @completed_re
end













#extra_class_map ⇒ Object







Returns the value of attribute extra_class_map.















141
142
143




# File 'lib/autotest.rb', line 141

def extra_class_map
  @extra_class_map
end













#extra_files ⇒ Object







Returns the value of attribute extra_files.















141
142
143




# File 'lib/autotest.rb', line 141

def extra_files
  @extra_files
end













#failed_results_re ⇒ Object







Returns the value of attribute failed_results_re.















141
142
143




# File 'lib/autotest.rb', line 141

def failed_results_re
  @failed_results_re
end













#files_to_test ⇒ Object







Returns the value of attribute files_to_test.















141
142
143




# File 'lib/autotest.rb', line 141

def files_to_test
  @files_to_test
end













#find_directories ⇒ Object







Returns the value of attribute find_directories.















141
142
143




# File 'lib/autotest.rb', line 141

def find_directories
  @find_directories
end













#find_order ⇒ Object







Returns the value of attribute find_order.















141
142
143




# File 'lib/autotest.rb', line 141

def find_order
  @find_order
end













#interrupted ⇒ Object







Returns the value of attribute interrupted.















141
142
143




# File 'lib/autotest.rb', line 141

def interrupted
  @interrupted
end













#known_files ⇒ Object







Lazy accessor for the known_files hash.















434
435
436
437
438
439




# File 'lib/autotest.rb', line 434

def known_files
  unless @known_files then
    @known_files = Hash[*find_order.map { |f| [f, true] }.flatten]
  end
  @known_files
end













#last_mtime ⇒ Object







Returns the value of attribute last_mtime.















141
142
143




# File 'lib/autotest.rb', line 141

def last_mtime
  @last_mtime
end













#libs ⇒ Object







Returns the value of attribute libs.















141
142
143




# File 'lib/autotest.rb', line 141

def libs
  @libs
end













#order ⇒ Object







Returns the value of attribute order.















141
142
143




# File 'lib/autotest.rb', line 141

def order
  @order
end













#output ⇒ Object







Returns the value of attribute output.















141
142
143




# File 'lib/autotest.rb', line 141

def output
  @output
end













#results ⇒ Object







Returns the value of attribute results.















141
142
143




# File 'lib/autotest.rb', line 141

def results
  @results
end













#sleep ⇒ Object







Returns the value of attribute sleep.















141
142
143




# File 'lib/autotest.rb', line 141

def sleep
  @sleep
end













#tainted ⇒ Object







Returns the value of attribute tainted.















141
142
143




# File 'lib/autotest.rb', line 141

def tainted
  @tainted
end













#testlib ⇒ Object







Returns the value of attribute testlib.















141
142
143




# File 'lib/autotest.rb', line 141

def testlib
  @testlib
end













#unit_diff ⇒ Object







Returns the value of attribute unit_diff.















141
142
143




# File 'lib/autotest.rb', line 141

def unit_diff
  @unit_diff
end













#wants_to_quit ⇒ Object







Returns the value of attribute wants_to_quit.















141
142
143




# File 'lib/autotest.rb', line 141

def wants_to_quit
  @wants_to_quit
end













Class Method Details





.add_discovery(&proc) ⇒ Object







Add a proc to the collection of discovery procs. See autodiscover.















87
88
89




# File 'lib/autotest.rb', line 87

def self.add_discovery &proc
  @@discoveries << proc
end












.add_hook(name, &block) ⇒ Object







Add the supplied block to the available hooks, with the given name.















676
677
678




# File 'lib/autotest.rb', line 676

def self.add_hook(name, &block)
  HOOKS[name] << block
end












.autodiscover ⇒ Object







Automatically find all potential autotest runner styles by searching your loadpath, vendor/plugins, and rubygems for “autotest/discover.rb”. If found, that file is loaded and it should register discovery procs with autotest using add_discovery. That proc should return one or more strings describing the user’s current environment. Those styles are then combined to dynamically invoke an autotest plugin to suite your environment. That plugin should define a subclass of Autotest with a corresponding name.


Process:


  1. 

    All autotest/discover.rb files loaded.
    

  2. 

    Those procs determine your styles (eg [“rails”, “rspec”]).
    

  3. 

    Require file by sorting styles and joining (eg ‘autotest/rails_rspec’).
    

  4. 

    Invoke run method on appropriate class (eg Autotest::RailsRspec.run).
    



Example autotest/discover.rb:


Autotest.add_discovery do
  "rails" if File.exist? 'config/environment.rb'
end
















115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131




# File 'lib/autotest.rb', line 115

def self.autodiscover
  require 'rubygems'
  begin
    require 'win32console' if WINDOZE
  rescue LoadError
  end

  with_current_path_in_load_path do
    # search load paths for autotest/discover.rb and load em all
    Gem.find_files("autotest/discover").each do |f|
      load f
    end
  end

  #call all discover procs an determine style
  @@discoveries.map{ |proc| proc.call }.flatten.compact.sort.uniq
end












.options ⇒ Object










71




# File 'lib/autotest.rb', line 71

def self.options;@@options;end












.run ⇒ Object







Initialize and run the system.















136
137
138




# File 'lib/autotest.rb', line 136

def self.run
  new.run
end













Instance Method Details





#add_exception(regexp) ⇒ Object







Adds regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.















606
607
608
609
610
611




# File 'lib/autotest.rb', line 606

def add_exception regexp
  raise "exceptions already compiled" if defined? @exceptions

  @exception_list << regexp
  nil
end












#add_mapping(regexp, prepend = false, &proc) ⇒ Object







Adds a file mapping, optionally prepending the mapping to the front of the list if prepend is true. regexp should match a file path in the codebase. proc is passed a matched filename and Regexp.last_match. proc should return an array of tests to run.


For example, if test_helper.rb is modified, rerun all tests:


at.add_mapping(/test_helper.rb/) do |f, _|
  at.files_matching(/^test.*rb$/)
end
















570
571
572
573
574
575
576
577




# File 'lib/autotest.rb', line 570

def add_mapping(regexp, prepend = false, &proc)
  if prepend then
    @test_mappings.unshift [regexp, proc]
  else
    @test_mappings.push [regexp, proc]
  end
  nil
end












#add_sigint_handler ⇒ Object







Installs a sigint handler.















302
303
304
305
306
307
308
309
310
311
312
313
314
315




# File 'lib/autotest.rb', line 302

def add_sigint_handler
  trap 'INT' do
    if self.interrupted then
      self.wants_to_quit = true
    else
      unless hook :interrupt then
        puts "Interrupt a second time to quit"
        self.interrupted = true
        Kernel.sleep 1.5
      end
      raise Interrupt, nil # let the run loop catch it
    end
  end
end












#all_good ⇒ Object







If there are no files left to test (because they’ve all passed), then all is good.















321
322
323




# File 'lib/autotest.rb', line 321

def all_good
  files_to_test.empty?
end












#clear_exceptions ⇒ Object







Clears the list of exceptions for find_file. This must be called before the exceptions are compiled.















627
628
629
630
631




# File 'lib/autotest.rb', line 627

def clear_exceptions
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.clear
  nil
end












#clear_mappings ⇒ Object







Clears all file mappings. This is DANGEROUS as it entirely disables autotest. You must add at least one file mapping that does a good job of rerunning appropriate tests.















594
595
596
597




# File 'lib/autotest.rb', line 594

def clear_mappings
  @test_mappings.clear
  nil
end












#consolidate_failures(failed) ⇒ Object







Returns a hash mapping a file name to the known failures for that file.















342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359




# File 'lib/autotest.rb', line 342

def consolidate_failures(failed)
  filters = new_hash_of_arrays

  class_map = Hash[*self.find_order.grep(/^test/).map { |f| # TODO: ugly
                     [path_to_classname(f), f]
                   }.flatten]
  class_map.merge!(self.extra_class_map)

  failed.each do |method, klass|
    if class_map.has_key? klass then
      filters[class_map[klass]] << method
    else
      output.puts "Unable to map class #{klass} to a file"
    end
  end

  return filters
end












#exceptions ⇒ Object







Return a compiled regexp of exceptions for find_files or nil if no filtering should take place. This regexp is generated from exception_list.















638
639
640
641
642
643
644
645
646
647
648




# File 'lib/autotest.rb', line 638

def exceptions
  unless defined? @exceptions then
    if @exception_list.empty? then
      @exceptions = nil
    else
      @exceptions = Regexp.union(*@exception_list)
    end
  end

  @exceptions
end












#files_matching(regexp) ⇒ Object







Returns all known files in the codebase matching regexp.















554
555
556




# File 'lib/autotest.rb', line 554

def files_matching regexp
  self.find_order.select { |k| k =~ regexp }
end












#find_files ⇒ Object







Find the files to process, ignoring temporary files, source configuration management files, etc., and return a Hash mapping filename to modification time.















366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389




# File 'lib/autotest.rb', line 366

def find_files
  result = {}
  targets = self.find_directories + self.extra_files
  self.find_order.clear

  targets.each do |target|
    order = []
    Find.find(target) do |f|
      Find.prune if f =~ self.exceptions

      next if test ?d, f
      next if f =~ /(swp|~|rej|orig)$/ # temporary/patch files
      next if f =~ /\/\.?#/            # Emacs autosave/cvs merge files

      filename = f.sub(/^\.\//, '')

      result[filename] = File.stat(filename).mtime rescue next
      order << filename
    end
    self.find_order.push(*order.sort)
  end

  return result
end












#find_files_to_test(files = find_files) ⇒ Object







Find the files which have been modified, update the recorded timestamps, and use this to update the files to test. Returns the latest mtime of the files modified or nil when nothing was modified.















396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413




# File 'lib/autotest.rb', line 396

def find_files_to_test(files=find_files)
  updated = files.select { |filename, mtime| self.last_mtime < mtime }

  unless updated.empty? or self.last_mtime.to_i == 0 #nothing to update or initial run
    p updated if options[:verbose]
    hook :updated, updated
  end

  updated.map { |f,m| test_files_for(f) }.flatten.uniq.each do |filename|
    self.files_to_test[filename] # creates key with default value
  end

  if updated.empty? then
    nil
  else
    files.values.max
  end
end












#get_to_green ⇒ Object







Keep running the tests after a change, until all pass.















245
246
247
248
249
250




# File 'lib/autotest.rb', line 245

def get_to_green
  begin
    run_tests
    wait_for_changes unless all_good
  end until all_good
end












#handle_results(results) ⇒ Object







Check results for failures, set the “bar” to red or green, and if there are failures record this.















419
420
421
422
423
424
425
426
427
428
429




# File 'lib/autotest.rb', line 419

def handle_results(results)
  failed = results.scan(self.failed_results_re)
  completed = results =~ self.completed_re

  self.files_to_test = consolidate_failures failed if completed

  color = completed && self.files_to_test.empty? ? :green : :red
  hook color unless $TESTING

  self.tainted = true unless self.files_to_test.empty?
end












#hook(name, *args) ⇒ Object







Call the event hook named name, executing all registered hooks until one returns true. Returns false if no hook handled the event.















658
659
660
661
662
663
664
665
666
667
668
669
670




# File 'lib/autotest.rb', line 658

def hook(name, *args)
  deprecated = {
    # none currently
  }

  if deprecated[name] and not HOOKS[name].empty? then
    warn "hook #{name} has been deprecated, use #{deprecated[name]}"
  end

  HOOKS[name].any? do |plugin|
    plugin[self, *args]
  end
end












#load_custom_extensions(config_file) ⇒ Object










200
201
202
203
204
205
206
207
208
209
210




# File 'lib/autotest.rb', line 200

def load_custom_extensions(config_file)
  configs = ['./.autotest']
  if config_file
    configs << File.expand_path(config_file)
  else
    configs << File.expand_path('~/.autotest')
  end
  configs.each do |f|
    load f if File.exist? f
  end
end












#make_test_cmd(files_to_test) ⇒ Object







Generate the commands to test the supplied files















444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461




# File 'lib/autotest.rb', line 444

def make_test_cmd files_to_test
  cmds = []
  full, partial = reorder(files_to_test).partition { |k,v| v.empty? }
  base_cmd = "#{ruby} -I#{libs} -rubygems"

  unless full.empty? then
    classes = full.map {|k,v| k}.flatten.uniq
    classes.unshift testlib
    cmds << "#{base_cmd} -e \"[#{classes.map{|klass| "'#{klass}'"}.join(', ')}].each { |f| require f }\" | #{unit_diff}"
  end

  partial.each do |klass, methods|
    regexp = Regexp.union(*methods).source
    cmds << "#{base_cmd} #{klass} -n \"/^(#{regexp})$/\" | #{unit_diff}"
  end

  return cmds.join("#{SEP} ")
end












#new_hash_of_arrays ⇒ Object










463
464
465




# File 'lib/autotest.rb', line 463

def new_hash_of_arrays
  Hash.new { |h,k| h[k] = [] }
end












#options ⇒ Object










72




# File 'lib/autotest.rb', line 72

def options;@@options;end












#path_to_classname(s) ⇒ Object







Convert a path in a string, s, into a class name, changing underscores to CamelCase, etc.















329
330
331
332
333
334
335
336




# File 'lib/autotest.rb', line 329

def path_to_classname(s)
  sep = File::SEPARATOR
  f = s.sub(/^test#{sep}/, '').sub(/\.rb$/, '').split(sep)
  f = f.map { |path| path.split(/_|(\d+)/).map { |seg| seg.capitalize }.join }
  f = f.map { |path| path =~ /^Test/ ? path : "Test#{path}"  }

  f.join('::')
end












#remove_exception(regexp) ⇒ Object







Removes regexp to the list of exceptions for find_file. This must be called before the exceptions are compiled.















617
618
619
620
621




# File 'lib/autotest.rb', line 617

def remove_exception regexp
  raise "exceptions already compiled" if defined? @exceptions
  @exception_list.delete regexp
  nil
end












#remove_mapping(regexp) ⇒ Object







Removed a file mapping matching regexp.















582
583
584
585
586
587




# File 'lib/autotest.rb', line 582

def remove_mapping regexp
  @test_mappings.delete_if do |k,v|
    k == regexp
  end
  nil
end












#reorder(files_to_test) ⇒ Object










467
468
469
470
471
472
473
474
475
476
477
478
479
480
481




# File 'lib/autotest.rb', line 467

def reorder files_to_test
  case self.order
  when :alpha then
    files_to_test.sort_by { |k,v| k }
  when :reverse then
    files_to_test.sort_by { |k,v| k }.reverse
  when :random then
    max = files_to_test.size
    files_to_test.sort_by { |k,v| rand(max) }
  when :natural then
    (self.find_order & files_to_test.keys).map { |f| [f, files_to_test[f]] }
  else
    raise "unknown order type: #{self.order.inspect}"
  end
end












#rerun_all_tests ⇒ Object







Rerun the tests from cold (reset state)















486
487
488
489
490
491




# File 'lib/autotest.rb', line 486

def rerun_all_tests
  reset
  run_tests

  hook :all_good if all_good
end












#reset ⇒ Object







Clear all state information about test failures and whether interrupts will kill autotest.















497
498
499
500
501
502
503
504
505
506
507




# File 'lib/autotest.rb', line 497

def reset
  self.files_to_test.clear
  self.find_order.clear
  self.interrupted = false
  self.known_files = nil
  self.last_mtime = T0
  self.tainted = false
  self.wants_to_quit = false

  hook :reset
end












#ruby ⇒ Object







Determine and return the path of the ruby executable.















512
513
514
515
516
517
518
519
520




# File 'lib/autotest.rb', line 512

def ruby
  ruby = ENV['RUBY']
  ruby ||= File.join(Config::CONFIG['bindir'],
                     Config::CONFIG['ruby_install_name'])

  ruby.gsub! File::SEPARATOR, File::ALT_SEPARATOR if File::ALT_SEPARATOR

  return ruby
end












#run ⇒ Object







Repeatedly run failed tests, then all tests, then wait for changes and carry on until killed.















216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240




# File 'lib/autotest.rb', line 216

def run
  hook :initialize
  reset
  add_sigint_handler

  self.last_mtime = Time.now if options[:no_full_after_start]

  loop do
    begin # ^c handler
      get_to_green
      if self.tainted and not options[:no_full_after_failed] then
        rerun_all_tests
      else
        hook :all_good
      end
      wait_for_changes
    rescue Interrupt
      break if self.wants_to_quit
      reset
    end
  end
  hook :quit
rescue Exception => err
  hook :died, err
end












#run_tests ⇒ Object







Look for files to test then run the tests and handle the results.















255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294




# File 'lib/autotest.rb', line 255

def run_tests
  hook :run_command

  new_mtime = self.find_files_to_test
  return unless new_mtime
  self.last_mtime = new_mtime

  cmd = self.make_test_cmd self.files_to_test
  return if cmd.empty?

  puts cmd unless options[:quiet]

  old_sync = $stdout.sync
  $stdout.sync = true
  self.results = []
  line = []
  begin
    open("| #{cmd}", "r") do |f|
      until f.eof? do
        c = f.getc or break
        print (c.is_a?(Fixnum) ? c.chr : c)
        line << c
        if c == ?\n then
          self.results << if RUBY19 then
                            line.join
                          else
                            line.pack "c*"
                          end
          line.clear
        end
      end
    end
  ensure
    $stdout.sync = old_sync
  end
  hook :ran_command
  self.results = self.results.join

  handle_results(self.results)
end












#test_files_for(filename) ⇒ Object







Return the name of the file with the tests for filename by finding a test_mapping that matches the file and executing the mapping’s proc.















527
528
529
530
531
532
533
534
535
536
537
538




# File 'lib/autotest.rb', line 527

def test_files_for(filename)
  result = @test_mappings.find { |file_re, ignored| filename =~ file_re }

  p :test_file_for => [filename, result.first] if result and $DEBUG

  result = result.nil? ? [] : [result.last.call(filename, $~)].flatten

  output.puts "No tests matched #{filename}" if
    (options[:verbose] or $TESTING) and result.empty?

  result.sort.uniq.select { |f| known_files[f] }
end












#wait_for_changes ⇒ Object







Sleep then look for files to test, until there are some.















543
544
545
546




# File 'lib/autotest.rb', line 543

def wait_for_changes
  hook :waiting
  Kernel.sleep self.sleep until find_files_to_test
end