Class: Bullshit::Case

Inherits:
Object
  • Object
show all
Extended by:
DSLKit::DSLAccessor
Defined in:
lib/bullshit.rb

Overview

This is the base class of all Benchmarking Cases.

Direct Known Subclasses

RangeCase, RepeatCase, TimeCase

Defined Under Namespace

Modules: CaseExtension

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Case

Returns a Case instance, that is used to run benchmark methods and measure their running time.



1659
1660
1661
1662
1663
 
# File 'lib/bullshit.rb', line 1659

def initialize
  @clocks = []
  @comparison = Comparison.new
  @comparison.output self.class.output
end

Instance Attribute Details

#clocks ⇒ Object (readonly)

The clock instances, that were used during a run of this benchmark case.



1778
1779
1780
 
# File 'lib/bullshit.rb', line 1778

def clocks
  @clocks
end

Class Method Details

.autorun_all ⇒ Object

Autorun all subclasses’ instances, that is all Bullshit cases. If its autorun dsl_accessor is false or it has already run, don’t run the case.



1650
1651
1652
1653
1654
 
# File 'lib/bullshit.rb', line 1650

def autorun_all
  each do |bc_class|
    bc_class.autorun and bc_class.run_count == 0 and bc_class.run
  end
end

.each(&block) ⇒ Object

Iterate over all subclasses of class Case.



1635
1636
1637
 
# File 'lib/bullshit.rb', line 1635

def each(&block)
  cases.each(&block)
end

.inherited(klass) ⇒ Object 



1612
1613
1614
 
# File 'lib/bullshit.rb', line 1612

def inherited(klass)
  klass.extend CaseExtension
end

.output_filename(name) ⇒ Object 



1622
1623
1624
1625
 
# File 'lib/bullshit.rb', line 1622

def output_filename(name)
  path = File.expand_path(name, output_dir)
  output File.new(path, 'a+')
end

.run ⇒ Object

Creates an instance of this class and run it.



1799
1800
1801
 
# File 'lib/bullshit.rb', line 1799

def self.run
  new.run
end

.run_all ⇒ Object

Run all subclasses’ instances, that is all Bullshit cases, unless they already have run.



1641
1642
1643
1644
1645
 
# File 'lib/bullshit.rb', line 1641

def run_all
  each do |bc_class|
    bc_class.run_count == 0 and bc_class.run
  end
end

.run_count ⇒ Object

Returns the total number of run counts run_count.



1628
1629
1630
 
# File 'lib/bullshit.rb', line 1628

def run_count
  cases.inject(0) { |s, c| s + c.run_count }
end

.sorted_bmethods ⇒ Object

Return all benchmark methods of this Case instance lexicographically sorted.



1672
1673
1674
 
# File 'lib/bullshit.rb', line 1672

def self.sorted_bmethods
  instance_methods.map { |x| x.to_s }.grep(/\Abenchmark_/).sort
end

Instance Method Details

#[](method_name) ⇒ Object

Return the CaseMethod instance for method_name or nil, if there isn’t any method of this name.



1689
1690
1691
1692
 
# File 'lib/bullshit.rb', line 1689

def [](method_name)
  method_name = "benchmark_#{method_name}"
  bmethods.find { |bm| bm.name == method_name }
end

#bmethods ⇒ Object

Return all benchmark methods of this Case instance in a random order.



1677
1678
1679
1680
1681
1682
1683
1684
1685
 
# File 'lib/bullshit.rb', line 1677

def bmethods
  unless @bmethods
    @bmethods = self.class.sorted_bmethods.sort_by do
      rand
    end
    @bmethods.map! { |n| CaseMethod.new(n, self) }
  end
  @bmethods
end

#evaluation(clock) ⇒ Object

This method has to be implemented in subclasses, it should return the evaluation results of the benchmarks as a string.

Raises:

  • (NotImplementedError) 


1856
1857
1858
 
# File 'lib/bullshit.rb', line 1856

def evaluation(clock)
  raise NotImplementedError, "has to be implemented in subclasses"
end

#longest_name ⇒ Object

Return the length of the longest_name of all these methods’ names.



1695
1696
1697
1698
 
# File 'lib/bullshit.rb', line 1695

def longest_name
  bmethods.empty? and return 0
  bmethods.map { |x| x.short_name.size }.max
end

#post_run(bc_method) ⇒ Object

Output after bc_method is run.



1861
1862
1863
1864
1865
1866
1867
 
# File 'lib/bullshit.rb', line 1861

def post_run(bc_method)
  teardown_name = bc_method.teardown_name
  if respond_to? teardown_name
    $DEBUG and warn "Calling #{teardown_name}."
    __send__(bc_method.teardown_name)
  end
end

#pre_run(bc_method) ⇒ Object

Output before bc_method is run.



1833
1834
1835
1836
1837
1838
1839
1840
 
# File 'lib/bullshit.rb', line 1833

def pre_run(bc_method)
  setup_name = bc_method.setup_name
  if respond_to? setup_name
    $DEBUG and warn "Calling #{setup_name}."
    __send__(setup_name)
  end
  self.class.output.puts "#{bc_method.long_name}:"
end

#run(comparison = true) ⇒ Object

Setup, run all benchmark cases (warmup and the real run) and output results, run method speed comparisons, and teardown.



1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
 
# File 'lib/bullshit.rb', line 1782

def run(comparison = true)
  old_sync, self.class.output.sync = self.class.output.sync, true
  $DEBUG and warn "Calling setup."
  setup
  run_once
  comparison and @comparison.display
  self
rescue => e
  warn "Caught #{e.class}: #{e}\n\n#{e.backtrace.map { |x| "\t#{x}\n" }}"
ensure
  $DEBUG and warn "Calling teardown."
  teardown
  @clocks and write_files
  self.class.output.sync = old_sync
end

#run_method(bc_method) ⇒ Object

Run only pre_run and post_run methods. Yield to the block, if one was given.



1844
1845
1846
1847
1848
1849
1850
1851
1852
 
# File 'lib/bullshit.rb', line 1844

def run_method(bc_method)
  pre_run bc_method
  clock = self.class.clock.__send__(self.class.clock_method, bc_method) do
    __send__(bc_method.name)
  end
  bc_method.clock = clock
  post_run bc_method
  clock
end

#run_once ⇒ Object

Run benchmark case once and output results.



1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
 
# File 'lib/bullshit.rb', line 1701

def run_once
  self.class.run_count(self.class.run_count + 1)
  self.class.output.puts Time.now.strftime(' %FT%T %Z ').center(COLUMNS, '=')
  self.class.output.puts "Benchmarking on #{RUBY_DESCRIPTION}."
  self.class.output.puts self.class.message
  self.class.output.puts '=' * COLUMNS, ''
  @clocks.clear
  if self.class.warmup == :aggressive
    self.class.output.puts "Aggressively run all benchmarks for warmup first.", ''
    bmethods.each do |bc_method|
      GC.start
      clock = run_method bc_method
      self.class.output.puts evaluation(clock)
      GC.start
    end
    self.class.output.puts "Aggressive warmup done.", '', '=' * COLUMNS, ''
  end
  first = true
  bmethods.each do |bc_method|
    if first
      first = false
    else
      self.class.output.puts '-' * COLUMNS, ''
    end
    if self.class.warmup
      self.class.output.puts "This first run is only for warmup."
      GC.start
      clock = run_method bc_method
      self.class.output.puts evaluation(clock)
      GC.start
    end
    clock = run_method(bc_method)
    if self.class.truncate_data.enabled
      message = ''
      offset = clock.find_truncation_offset
      if clock.case.data_file
        slopes_file_path = clock.file_path 'slopes'
        message << "Writing slopes data file '#{slopes_file_path}'.\n"
        File.open(slopes_file_path, 'w') do |slopes_file|
          slopes_file.puts %w[#scatter slope] * "\t"
          slopes_file.puts clock.slopes.map { |s| s * "\t" }
        end
      end
      case offset
      when 0
        message << "No initial data truncated.\n =>"\
          " System may have been in a steady state from the beginning."
      when clock.repeat
        message << "After truncating measurements no data would have"\
          " remained.\n => No steady state could be detected."
      else
        if clock.case.data_file
          data_file_path = clock.file_path 'untruncated'
          message << "Writing untruncated measurement data file '#{data_file_path}'.\n"
          File.open(data_file_path, 'w') do |data_file|
            data_file.puts clock.class.to_a * "\t"
            data_file.puts clock.to_a.map { |times| times * "\t" }
          end
        end
        remaining = clock.repeat - offset
        offset_percentage = 100 * offset.to_f / clock.repeat
        message << sprintf("Truncated initial %u measurements: "\
          "%u -> %u (-%0.2f%%).\n", offset, clock.repeat, remaining,
          offset_percentage)
        clock.truncate_data(offset)
      end
      self.class.output.puts evaluation(clock), message
    else
      self.class.output.puts evaluation(clock)
    end
    @clocks << clock
    @comparison.benchmark(self, bc_method.short_name, :run => false)
  end
  @clocks
end

#setup ⇒ Object

General setup for all the benchmark methods.



1870
1871
 
# File 'lib/bullshit.rb', line 1870

def setup
end

#teardown ⇒ Object

General teardown for all the benchmark methods.



1874
1875
 
# File 'lib/bullshit.rb', line 1874

def teardown
end

#to_s ⇒ Object

Return the name of the benchmark case as a string.



1666
1667
1668
 
# File 'lib/bullshit.rb', line 1666

def to_s
  self.class.benchmark_name
end

#write_files ⇒ Object

Write all output files after run.



1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
 
# File 'lib/bullshit.rb', line 1804

def write_files
  for clock in @clocks
    if clock.case.data_file data_file_path = clock.file_path
      self.class.output.puts "Writing measurement data file '#{data_file_path}'."
      File.open(data_file_path, 'w') do |data_file|
        data_file.puts clock.class.to_a * "\t"
        data_file.puts clock.to_a.map { |times| times * "\t" }
      end
    end
    if clock.case.histogram.enabled and clock.case.histogram.file
      histogram_file_path = clock.file_path 'histogram'
      self.class.output.puts "Writing histogram file '#{histogram_file_path}'."
      File.open(histogram_file_path, 'w') do |data_file|
        data_file.puts %w[#binleft frequency binright] * "\t"
        data_file.puts clock.histogram(clock.case.compare_time).to_a.map { |times| times * "\t" }
      end
    end
    if clock.case.autocorrelation.enabled and clock.case.autocorrelation.file
      ac_plot_file_path = clock.file_path 'autocorrelation'
      self.class.output.puts "Writing autocorrelation plot file '#{ac_plot_file_path}'."
      File.open(ac_plot_file_path, 'w') do |data_file|
        data_file.puts %w[#lag autocorrelation] * "\t"
        data_file.puts clock.autocorrelation_plot(clock.case.compare_time).to_a.map { |ac| ac * "\t" }
      end
    end
  end
end