Class: Minitest::Benchmark
| Relationships & Source Files | |
| Extension / Inclusion / Inheritance Descendants | |
| Subclasses: | |
| Super Chains via Extension / Inclusion / Inheritance | |
| Class Chain: | |
| Instance Chain: | |
| Inherits: | Minitest::Test 
 | 
| Defined in: | lib/minitest/benchmark.rb | 
Overview
Subclass Benchmark to create your own benchmark runs. Methods starting with “bench_” get executed on a per-class.
See Assertions
Class Method Summary
- 
    
      .bench_exp(min, max, base = 10)  
    
    Returns a set of ranges stepped exponentially from mintomaxby powers ofbase.
- 
    
      .bench_linear(min, max, step = 10)  
    
    Returns a set of ranges stepped linearly from mintomaxbystep.
- 
    
      .bench_range  
    
    Specifies the ranges used for benchmarking for that class. 
Test - Inherited
| .i_suck_and_my_tests_are_order_dependent! | Call this at the top of your tests when you absolutely positively need to have ordered tests. | 
| .make_my_diffs_pretty! | Make diffs for this Test use  | 
| .parallelize_me! | Call this at the top of your tests when you want to run your tests in parallel. | 
| .runnable_methods | Returns all instance methods starting with “test_”. | 
Guard - Extended
| jruby? | Is this running on jruby? | 
| maglev? | Is this running on maglev? | 
| mri? | Is this running on mri? | 
| rubinius? | Is this running on rubinius? | 
| windows? | Is this running on windows? | 
Runnable - Inherited
| .methods_matching | Returns all instance methods matching the pattern  | 
| .run | Responsible for running all runnable methods in a given class, each in its own instance. | 
| .run_one_method | Runs a single method and has the reporter record the result. | 
| .runnable_methods | Each subclass of Runnable is responsible for overriding this method to return all runnable methods. | 
| .runnables | Returns all subclasses of Runnable. | 
Instance Attribute Summary
Test - Inherited
| #time | The time it took to run this test. | 
| #error? | Did this run error? | 
| #passed? | Did this run pass? | 
| #skipped? | Was this run skipped? | 
Assertions - Included
| #skipped? | Was this testcase skipped? Meant for  | 
Runnable - Inherited
Instance Method Summary
- 
    
      #assert_performance(validation, &work)  
    
    Runs the given work, gathering the times of each run.
- 
    
      #assert_performance_constant(threshold = 0.99, &work)  
    
    Runs the given workand asserts that the times gathered fit to match a constant rate (eg, linear slope == 0) within a giventhreshold.
- 
    
      #assert_performance_exponential(threshold = 0.99, &work)  
    
    Runs the given workand asserts that the times gathered fit to match a exponential curve within a given errorthreshold.
- 
    
      #assert_performance_linear(threshold = 0.99, &work)  
    
    Runs the given workand asserts that the times gathered fit to match a straight line within a given errorthreshold.
- 
    
      #assert_performance_logarithmic(threshold = 0.99, &work)  
    
    Runs the given workand asserts that the times gathered fit to match a logarithmic curve within a given errorthreshold.
- 
    
      #assert_performance_power(threshold = 0.99, &work)  
    
    Runs the given workand asserts that the times gathered curve fit to match a power curve within a given errorthreshold.
- 
    
      #fit_error(xys)  
    
    Takes an array of x/y pairs and calculates the general R^2 value. 
- 
    
      #fit_exponential(xs, ys)  
    
    To fit a functional form: y = ae^(bx). 
- 
    
      #fit_linear(xs, ys)  
    
    Fits the functional form: a + bx. 
- 
    
      #fit_logarithmic(xs, ys)  
    
    To fit a functional form: y = a + b*ln(x). 
- 
    
      #fit_power(xs, ys)  
    
    To fit a functional form: y = ax^b. 
- 
    
      #sigma(enum, &block)  
    
    Enumerates over enummappingblockif given, returning the sum of the result.
- 
    
      #validation_for_fit(msg, threshold)  
    
    Returns a proc that calls the specified fit method and asserts that the error is within a tolerable threshold. 
Test - Inherited
| #location | The location identifier of this test. | 
| #result_code | Returns “.”, “F”, or “E” based on the result of the run. | 
| #run | Runs a single test with setup/teardown hooks. | 
Guard - Included
| #jruby? | Is this running on jruby? | 
| #maglev? | Is this running on maglev? | 
| #mri? | Is this running on mri? | 
| #rubinius? | Is this running on rubinius? | 
| #windows? | Is this running on windows? | 
Test::LifecycleHooks - Included
| #after_setup | Runs before every test, after setup. | 
| #after_teardown | Runs after every test, after teardown. | 
| #before_setup | Runs before every test, before setup. | 
| #before_teardown | Runs after every test, before teardown. | 
| #setup | Runs before every test. | 
| #teardown | Runs after every test. | 
Assertions - Included
| #assert | Fails unless  | 
| #assert_empty | Fails unless  | 
| #assert_equal | Fails unless  | 
| #assert_in_delta | For comparing Floats. | 
| #assert_in_epsilon | For comparing Floats. | 
| #assert_includes | Fails unless  | 
| #assert_instance_of | Fails unless  | 
| #assert_kind_of | Fails unless  | 
| #assert_match | Fails unless  | 
| #assert_mock | Assert that the mock verifies correctly. | 
| #assert_nil | Fails unless  | 
| #assert_operator | For testing with binary operators. | 
| #assert_output | Fails if stdout or stderr do not output the expected results. | 
| #assert_predicate | For testing with predicates. | 
| #assert_raises | Fails unless the block raises one of  | 
| #assert_respond_to | Fails unless  | 
| #assert_same | Fails unless  | 
| #assert_send | 
 | 
| #assert_silent | Fails if the block outputs anything to stderr or stdout. | 
| #assert_throws | Fails unless the block throws  | 
| #capture_io | Captures $stdout and $stderr into strings: | 
| #capture_subprocess_io | Captures $stdout and $stderr into strings, using Tempfile to ensure that subprocess IO is captured as well. | 
| #diff | Returns a diff between  | 
| #exception_details | Returns details for exception  | 
| #flunk | Fails with  | 
| #message | Returns a proc that will output  | 
| #mu_pp | This returns a human-readable version of  | 
| #mu_pp_for_diff | This returns a diff-able human-readable version of  | 
| #pass | used for counting assertions. | 
| #refute | Fails if  | 
| #refute_empty | Fails if  | 
| #refute_equal | Fails if  | 
| #refute_in_delta | For comparing Floats. | 
| #refute_in_epsilon | For comparing Floats. | 
| #refute_includes | Fails if  | 
| #refute_instance_of | Fails if  | 
| #refute_kind_of | Fails if  | 
| #refute_match | Fails if  | 
| #refute_nil | Fails if  | 
| #refute_operator | Fails if  | 
| #refute_predicate | For testing with predicates. | 
| #refute_respond_to | Fails if  | 
| #refute_same | Fails if  | 
| #skip | Skips the current run. | 
Runnable - Inherited
| #result_code | Returns a single character string to print based on the result of the run. | 
| #run | Runs a single method. | 
Class Method Details
.bench_exp(min, max, base = 10)
Returns a set of ranges stepped exponentially from min to max by powers of base. Eg:
bench_exp(2, 16, 2) # => [2, 4, 8, 16]# File 'lib/minitest/benchmark.rb', line 35
def self.bench_exp min, max, base = 10 min = (Math.log10(min) / Math.log10(base)).to_i max = (Math.log10(max) / Math.log10(base)).to_i (min..max).map { |m| base ** m }.to_a end
.bench_linear(min, max, step = 10)
Returns a set of ranges stepped linearly from min to max by step. Eg:
bench_linear(20, 40, 10) # => [20, 30, 40]# File 'lib/minitest/benchmark.rb', line 48
def self.bench_linear min, max, step = 10 (min..max).step(step).to_a rescue LocalJumpError # 1.8.6 r = []; (min..max).step(step) { |n| r << n }; r end
.bench_range
Specifies the ranges used for benchmarking for that class. Defaults to exponential growth from 1 to 10k by powers of 10. Override if you need different ranges for your benchmarks.
See also: .bench_exp and .bench_linear.
# File 'lib/minitest/benchmark.rb', line 61
def self.bench_range bench_exp 1, 10_000 end
Instance Method Details
#assert_performance(validation, &work)
Runs the given work, gathering the times of each run. Range and times are then passed to a given validation proc. Outputs the benchmark name and times in tab-separated format, making it easy to paste into a spreadsheet for graphing or further analysis.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
  validation = proc { |x, y| ... }
  assert_performance validation do |n|
    @obj.algorithm(n)
  end
end# File 'lib/minitest/benchmark.rb', line 83
def assert_performance validation, &work range = self.class.bench_range io.print "#{self.name}" times = [] range.each do |x| GC.start t0 = Minitest.clock_time instance_exec(x, &work) t = Minitest.clock_time - t0 io.print "\t%9.6f" % t times << t end io.puts validation[range, times] end
#assert_performance_constant(threshold = 0.99, &work)
Runs the given work and asserts that the times gathered fit to match a constant rate (eg, linear slope == 0) within a given threshold. Note: because we're testing for a slope of 0, R^2 is not a good determining factor for the fit, so the threshold is applied against the slope itself. As such, you probably want to tighten it from the default.
See www.graphpad.com/curvefit/goodness_of_fit.htm for more details.
Fit is calculated by #fit_linear.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
  assert_performance_constant 0.9999 do |n|
    @obj.algorithm(n)
  end
end# File 'lib/minitest/benchmark.rb', line 127
def assert_performance_constant threshold = 0.99, &work validation = proc do |range, times| a, b, rr = fit_linear range, times assert_in_delta 0, b, 1 - threshold [a, b, rr] end assert_performance validation, &work end
#assert_performance_exponential(threshold = 0.99, &work)
Runs the given work and asserts that the times gathered fit to match a exponential curve within a given error threshold.
Fit is calculated by #fit_exponential.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
  assert_performance_exponential 0.9999 do |n|
    @obj.algorithm(n)
  end
end# File 'lib/minitest/benchmark.rb', line 153
def assert_performance_exponential threshold = 0.99, &work assert_performance validation_for_fit(:exponential, threshold), &work end
#assert_performance_linear(threshold = 0.99, &work)
Runs the given work and asserts that the times gathered fit to match a straight line within a given error threshold.
Fit is calculated by #fit_linear.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
  assert_performance_linear 0.9999 do |n|
    @obj.algorithm(n)
  end
end# File 'lib/minitest/benchmark.rb', line 193
def assert_performance_linear threshold = 0.99, &work assert_performance validation_for_fit(:linear, threshold), &work end
#assert_performance_logarithmic(threshold = 0.99, &work)
Runs the given work and asserts that the times gathered fit to match a logarithmic curve within a given error threshold.
Fit is calculated by #fit_logarithmic.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
  assert_performance_logarithmic 0.9999 do |n|
    @obj.algorithm(n)
  end
end# File 'lib/minitest/benchmark.rb', line 173
def assert_performance_logarithmic threshold = 0.99, &work assert_performance validation_for_fit(:logarithmic, threshold), &work end
#assert_performance_power(threshold = 0.99, &work)
Runs the given work and asserts that the times gathered curve fit to match a power curve within a given error threshold.
Fit is calculated by #fit_power.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
  assert_performance_power 0.9999 do |x|
    @obj.algorithm
  end
end# File 'lib/minitest/benchmark.rb', line 213
def assert_performance_power threshold = 0.99, &work assert_performance validation_for_fit(:power, threshold), &work end
#fit_error(xys)
Takes an array of x/y pairs and calculates the general R^2 value.
#fit_exponential(xs, ys)
To fit a functional form: y = ae^(bx).
Takes x and y values and returns [a, b, r^2].
See: mathworld.wolfram.com/LeastSquaresFittingExponential.html
# File 'lib/minitest/benchmark.rb', line 237
def fit_exponential xs, ys n = xs.size xys = xs.zip(ys) sxlny = sigma(xys) { |x, y| x * Math.log(y) } slny = sigma(xys) { |_, y| Math.log(y) } sx2 = sigma(xys) { |x, _| x * x } sx = sigma xs c = n * sx2 - sx ** 2 a = (slny * sx2 - sx * sxlny) / c b = ( n * sxlny - sx * slny ) / c return Math.exp(a), b, fit_error(xys) { |x| Math.exp(a + b * x) } end
#fit_linear(xs, ys)
Fits the functional form: a + bx.
Takes x and y values and returns [a, b, r^2].
# File 'lib/minitest/benchmark.rb', line 281
def fit_linear xs, ys n = xs.size xys = xs.zip(ys) sx = sigma xs sy = sigma ys sx2 = sigma(xs) { |x| x ** 2 } sxy = sigma(xys) { |x, y| x * y } c = n * sx2 - sx**2 a = (sy * sx2 - sx * sxy) / c b = ( n * sxy - sx * sy ) / c return a, b, fit_error(xys) { |x| a + b * x } end
#fit_logarithmic(xs, ys)
To fit a functional form: y = a + b*ln(x).
Takes x and y values and returns [a, b, r^2].
See: mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html
# File 'lib/minitest/benchmark.rb', line 259
def fit_logarithmic xs, ys n = xs.size xys = xs.zip(ys) slnx2 = sigma(xys) { |x, _| Math.log(x) ** 2 } slnx = sigma(xys) { |x, _| Math.log(x) } sylnx = sigma(xys) { |x, y| y * Math.log(x) } sy = sigma(xys) { |_, y| y } c = n * slnx2 - slnx ** 2 b = ( n * sylnx - sy * slnx ) / c a = (sy - b * slnx) / n return a, b, fit_error(xys) { |x| a + b * Math.log(x) } end
#fit_power(xs, ys)
To fit a functional form: y = ax^b.
Takes x and y values and returns [a, b, r^2].
# File 'lib/minitest/benchmark.rb', line 303
def fit_power xs, ys n = xs.size xys = xs.zip(ys) slnxlny = sigma(xys) { |x, y| Math.log(x) * Math.log(y) } slnx = sigma(xs) { |x | Math.log(x) } slny = sigma(ys) { | y| Math.log(y) } slnx2 = sigma(xs) { |x | Math.log(x) ** 2 } b = (n * slnxlny - slnx * slny) / (n * slnx2 - slnx ** 2) a = (slny - b * slnx) / n return Math.exp(a), b, fit_error(xys) { |x| (Math.exp(a) * (x ** b)) } end
#sigma(enum, &block)
Enumerates over enum mapping block if given, returning the sum of the result. Eg:
sigma([1, 2, 3])                # => 1 + 2 + 3 => 7
sigma([1, 2, 3]) { |n| n ** 2 } # => 1 + 4 + 9 => 14# File 'lib/minitest/benchmark.rb', line 324
def sigma enum, &block enum = enum.map(&block) if block enum.inject { |sum, n| sum + n } end
#validation_for_fit(msg, threshold)
Returns a proc that calls the specified fit method and asserts that the error is within a tolerable threshold.
# File 'lib/minitest/benchmark.rb', line 333
def validation_for_fit msg, threshold proc do |range, times| a, b, rr = send "fit_#{msg}", range, times assert_operator rr, :>=, threshold [a, b, rr] end end