Class: Minitest::Benchmark
Relationships & Source Files  
Extension / Inclusion / Inheritance Descendants  
Subclasses:


Super Chains via Extension / Inclusion / Inheritance  
Class Chain:


Instance Chain:


Inherits: 
Minitest::Test

Defined in:  lib/minitest/benchmark.rb 
Overview
Subclass Benchmark to create your own benchmark runs. Methods starting with “bench_” get executed on a perclass.
See Assertions
Constant Summary
Class Attribute Summary
Class Method Summary

.bench_exp(min, max, base = 10)
Returns a set of ranges stepped exponentially from
min
tomax
by powers ofbase
. 
.bench_linear(min, max, step = 10)
Returns a set of ranges stepped linearly from
min
tomax
bystep
. 
.bench_range
Specifies the ranges used for benchmarking for that class.
 .io Internal use only
 .run(reporter, options = {}) Internal use only
 .runnable_methods Internal use only
Test
 Inherited
.i_suck_and_my_tests_are_order_dependent!  Call this at the top of your tests when you absolutely positively need to have ordered tests. 
.make_my_diffs_pretty!  Make diffs for this 
.parallelize_me!  Call this at the top of your tests when you want to run your tests in parallel. 
.runnable_methods  Returns all instance methods starting with “test_”. 
.test_order  Defines the order to run tests (:random by default). 
Guard
 Extended
jruby?  Is this running on jruby? 
maglev?  Is this running on maglev? 
mri?  Is this running on mri? 
osx?  Is this running on macOS? 
rubinius?  Is this running on rubinius? 
windows?  Is this running on windows? 
Runnable
 Inherited
.methods_matching  Returns all instance methods matching the pattern 
.run  Responsible for running all runnable methods in a given class, each in its own instance. 
.run_one_method  Runs a single method and has the reporter record the result. 
.runnable_methods  Each subclass of 
.runnables  Returns all subclasses of 
.inherited, .new, .on_signal, .reset, .with_info_handler 
Instance Attribute Summary
Reportable
 Included
Assertions
 Included
#skipped?  Was this testcase skipped? Meant for 
Runnable
 Inherited
Instance Method Summary

#assert_performance(validation, &work)
Runs the given
work
, gathering the times of each run. 
#assert_performance_constant(threshold = 0.99, &work)
Runs the given
work
and asserts that the times gathered fit to match a constant rate (eg, linear slope == 0) within a giventhreshold
. 
#assert_performance_exponential(threshold = 0.99, &work)
Runs the given
work
and asserts that the times gathered fit to match a exponential curve within a given errorthreshold
. 
#assert_performance_linear(threshold = 0.99, &work)
Runs the given
work
and asserts that the times gathered fit to match a straight line within a given errorthreshold
. 
#assert_performance_logarithmic(threshold = 0.99, &work)
Runs the given
work
and asserts that the times gathered fit to match a logarithmic curve within a given errorthreshold
. 
#assert_performance_power(threshold = 0.99, &work)
Runs the given
work
and asserts that the times gathered curve fit to match a power curve within a given errorthreshold
. 
#fit_error(xys)
Takes an array of x/y pairs and calculates the general R^2 value.

#fit_exponential(xs, ys)
To fit a functional form: y = ae^(bx).

#fit_linear(xs, ys)
Fits the functional form: a + bx.

#fit_logarithmic(xs, ys)
To fit a functional form: y = a + b*ln(x).

#fit_power(xs, ys)
To fit a functional form: y = ax^b.

#sigma(enum, &block)
Enumerates over
enum
mappingblock
if given, returning the sum of the result. 
#validation_for_fit(msg, threshold)
Returns a proc that calls the specified fit method and asserts that the error is within a tolerable threshold.
 #io Internal use only
Test
 Inherited
#neuter_exception, #new_exception,  
#run  Runs a single test with setup/teardown hooks. 
#capture_exceptions  LifecycleHooks. 
#class_name, #sanitize_exception, #with_info_handler 
Guard
 Included
#jruby?  Is this running on jruby? 
#maglev?  Is this running on maglev? 
#mri?  Is this running on mri? 
#osx?  Is this running on macOS? 
#rubinius?  Is this running on rubinius? 
#windows?  Is this running on windows? 
Test::LifecycleHooks
 Included
#after_setup  Runs before every test, after setup. 
#after_teardown  Runs after every test, after teardown. 
#before_setup  Runs before every test, before setup. 
#before_teardown  Runs after every test, before teardown. 
#setup  Runs before every test. 
#teardown  Runs after every test. 
Reportable
 Included
#location  The location identifier of this test. 
#result_code  Returns “.”, “F”, or “E” based on the result of the run. 
#class_name 
Assertions
 Included
#assert  Fails unless 
#assert_empty  Fails unless 
#assert_equal  Fails unless 
#assert_in_delta  For comparing Floats. 
#assert_in_epsilon  For comparing Floats. 
#assert_includes  Fails unless 
#assert_instance_of  Fails unless 
#assert_kind_of  Fails unless 
#assert_match  Fails unless 
#assert_mock  Assert that the mock verifies correctly. 
#assert_nil  Fails unless 
#assert_operator  For testing with binary operators. 
#assert_output  Fails if stdout or stderr do not output the expected results. 
#assert_path_exists  Fails unless 
#assert_predicate  For testing with predicates. 
#assert_raises  Fails unless the block raises one of 
#assert_respond_to  Fails unless 
#assert_same  Fails unless 
#assert_send 

#assert_silent  Fails if the block outputs anything to stderr or stdout. 
#assert_throws  Fails unless the block throws 
#capture_io  Captures $stdout and $stderr into strings: 
#capture_subprocess_io  Captures $stdout and $stderr into strings, using Tempfile to ensure that subprocess IO is captured as well. 
#diff  Returns a diff between 
#exception_details  Returns details for exception 
#fail_after  Fails after a given date (in the local time zone). 
#flunk  Fails with 
#message  Returns a proc that will output 
#mu_pp  This returns a humanreadable version of 
#mu_pp_for_diff  This returns a diffable more humanreadable version of 
#pass  used for counting assertions. 
#refute  Fails if 
#refute_empty  Fails if 
#refute_equal  Fails if 
#refute_in_delta  For comparing Floats. 
#refute_in_epsilon  For comparing Floats. 
#refute_includes  Fails if 
#refute_instance_of  Fails if 
#refute_kind_of  Fails if 
#refute_match  Fails if 
#refute_nil  Fails if 
#refute_operator  Fails if 
#refute_path_exists  Fails if 
#refute_predicate  For testing with predicates. 
#refute_respond_to  Fails if 
#refute_same  Fails if 
#skip  Skips the current run. 
#skip_until  Skips the current run until a given date (in the local time zone). 
#things_to_diff  Returns things to diff [expect, butwas], or [nil, nil] if nothing to diff. 
#_synchronize 
Runnable
 Inherited
#result_code  Returns a single character string to print based on the result of the run. 
#run  Runs a single method. 
#failure, #marshal_dump, #marshal_load, #time_it 
Constructor Details
This class inherits a constructor from Minitest::Runnable
Class Method Details
.bench_exp(min, max, base = 10)
Returns a set of ranges stepped exponentially from min
to max
by powers of base
. Eg:
bench_exp(2, 16, 2) # => [2, 4, 8, 16]
# File 'lib/minitest/benchmark.rb', line 35
def self.bench_exp min, max, base = 10 min = (Math.log10(min) / Math.log10(base)).to_i max = (Math.log10(max) / Math.log10(base)).to_i (min..max).map { m base ** m }.to_a end
.bench_linear(min, max, step = 10)
Returns a set of ranges stepped linearly from min
to max
by step
. Eg:
bench_linear(20, 40, 10) # => [20, 30, 40]
# File 'lib/minitest/benchmark.rb', line 48
def self.bench_linear min, max, step = 10 (min..max).step(step).to_a rescue LocalJumpError # 1.8.6 r = []; (min..max).step(step) { n r << n }; r end
.bench_range
Specifies the ranges used for benchmarking for that class. Defaults to exponential growth from 1 to 10k by powers of 10. Override if you need different ranges for your benchmarks.
See also: .bench_exp and .bench_linear.
# File 'lib/minitest/benchmark.rb', line 61
def self.bench_range bench_exp 1, 10_000 end
.io
# File 'lib/minitest/benchmark.rb', line 12
def self.io # :nodoc: @io end
.run(reporter, options = {})
.runnable_methods
# File 'lib/minitest/benchmark.rb', line 25
def self.runnable_methods # :nodoc: methods_matching(/^bench_/) end
Instance Method Details
#assert_performance(validation, &work)
Runs the given work
, gathering the times of each run. Range and times are then passed to a given validation
proc. Outputs the benchmark name and times in tabseparated format, making it easy to paste into a spreadsheet for graphing or further analysis.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
validation = proc { x, y ... }
assert_performance validation do n
@obj.algorithm(n)
end
end
# File 'lib/minitest/benchmark.rb', line 83
def assert_performance validation, &work range = self.class.bench_range io.print "#{self.name}" times = [] range.each do x GC.start t0 = Minitest.clock_time instance_exec(x, &work) t = Minitest.clock_time  t0 io.print "\t%9.6f" % t times << t end io.puts validation[range, times] end
#assert_performance_constant(threshold = 0.99, &work)
Runs the given work
and asserts that the times gathered fit to match a constant rate (eg, linear slope == 0) within a given threshold
. Note: because we’re testing for a slope of 0, R^2 is not a good determining factor for the fit, so the threshold is applied against the slope itself. As such, you probably want to tighten it from the default.
See www.graphpad.com/guides/prism/8/curvefitting/reg_intepretingnonlinr2.htm for more details.
Fit is calculated by #fit_linear.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
assert_performance_constant 0.9999 do n
@obj.algorithm(n)
end
end
# File 'lib/minitest/benchmark.rb', line 127
def assert_performance_constant threshold = 0.99, &work validation = proc do range, times a, b, rr = fit_linear range, times assert_in_delta 0, b, 1  threshold [a, b, rr] end assert_performance validation, &work end
#assert_performance_exponential(threshold = 0.99, &work)
Runs the given work
and asserts that the times gathered fit to match a exponential curve within a given error threshold
.
Fit is calculated by #fit_exponential.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
assert_performance_exponential 0.9999 do n
@obj.algorithm(n)
end
end
# File 'lib/minitest/benchmark.rb', line 153
def assert_performance_exponential threshold = 0.99, &work assert_performance validation_for_fit(:exponential, threshold), &work end
#assert_performance_linear(threshold = 0.99, &work)
Runs the given work
and asserts that the times gathered fit to match a straight line within a given error threshold
.
Fit is calculated by #fit_linear.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
assert_performance_linear 0.9999 do n
@obj.algorithm(n)
end
end
# File 'lib/minitest/benchmark.rb', line 193
def assert_performance_linear threshold = 0.99, &work assert_performance validation_for_fit(:linear, threshold), &work end
#assert_performance_logarithmic(threshold = 0.99, &work)
Runs the given work
and asserts that the times gathered fit to match a logarithmic curve within a given error threshold
.
Fit is calculated by #fit_logarithmic.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
assert_performance_logarithmic 0.9999 do n
@obj.algorithm(n)
end
end
# File 'lib/minitest/benchmark.rb', line 173
def assert_performance_logarithmic threshold = 0.99, &work assert_performance validation_for_fit(:logarithmic, threshold), &work end
#assert_performance_power(threshold = 0.99, &work)
Runs the given work
and asserts that the times gathered curve fit to match a power curve within a given error threshold
.
Fit is calculated by #fit_power.
Ranges are specified by .bench_range.
Eg:
def bench_algorithm
assert_performance_power 0.9999 do x
@obj.algorithm
end
end
# File 'lib/minitest/benchmark.rb', line 213
def assert_performance_power threshold = 0.99, &work assert_performance validation_for_fit(:power, threshold), &work end
#fit_error(xys)
Takes an array of x/y pairs and calculates the general R^2 value.
#fit_exponential(xs, ys)
To fit a functional form: y = ae^(bx).
Takes x and y values and returns [a, b, r^2].
See: mathworld.wolfram.com/LeastSquaresFittingExponential.html
# File 'lib/minitest/benchmark.rb', line 237
def fit_exponential xs, ys n = xs.size xys = xs.zip(ys) sxlny = sigma(xys) { x, y x * Math.log(y) } slny = sigma(xys) { _, y Math.log(y) } sx2 = sigma(xys) { x, _ x * x } sx = sigma xs c = n * sx2  sx ** 2 a = (slny * sx2  sx * sxlny) / c b = ( n * sxlny  sx * slny ) / c return Math.exp(a), b, fit_error(xys) { x Math.exp(a + b * x) } end
#fit_linear(xs, ys)
Fits the functional form: a + bx.
Takes x and y values and returns [a, b, r^2].
# File 'lib/minitest/benchmark.rb', line 281
def fit_linear xs, ys n = xs.size xys = xs.zip(ys) sx = sigma xs sy = sigma ys sx2 = sigma(xs) { x x ** 2 } sxy = sigma(xys) { x, y x * y } c = n * sx2  sx**2 a = (sy * sx2  sx * sxy) / c b = ( n * sxy  sx * sy ) / c return a, b, fit_error(xys) { x a + b * x } end
#fit_logarithmic(xs, ys)
To fit a functional form: y = a + b*ln(x).
Takes x and y values and returns [a, b, r^2].
See: mathworld.wolfram.com/LeastSquaresFittingLogarithmic.html
# File 'lib/minitest/benchmark.rb', line 259
def fit_logarithmic xs, ys n = xs.size xys = xs.zip(ys) slnx2 = sigma(xys) { x, _ Math.log(x) ** 2 } slnx = sigma(xys) { x, _ Math.log(x) } sylnx = sigma(xys) { x, y y * Math.log(x) } sy = sigma(xys) { _, y y } c = n * slnx2  slnx ** 2 b = ( n * sylnx  sy * slnx ) / c a = (sy  b * slnx) / n return a, b, fit_error(xys) { x a + b * Math.log(x) } end
#fit_power(xs, ys)
To fit a functional form: y = ax^b.
Takes x and y values and returns [a, b, r^2].
# File 'lib/minitest/benchmark.rb', line 303
def fit_power xs, ys n = xs.size xys = xs.zip(ys) slnxlny = sigma(xys) { x, y Math.log(x) * Math.log(y) } slnx = sigma(xs) { x  Math.log(x) } slny = sigma(ys) {  y Math.log(y) } slnx2 = sigma(xs) { x  Math.log(x) ** 2 } b = (n * slnxlny  slnx * slny) / (n * slnx2  slnx ** 2) a = (slny  b * slnx) / n return Math.exp(a), b, fit_error(xys) { x (Math.exp(a) * (x ** b)) } end
#io
# File 'lib/minitest/benchmark.rb', line 16
def io # :nodoc: self.class.io end
#sigma(enum, &block)
Enumerates over enum
mapping block
if given, returning the sum of the result. Eg:
sigma([1, 2, 3]) # => 1 + 2 + 3 => 6
sigma([1, 2, 3]) { n n ** 2 } # => 1 + 4 + 9 => 14
# File 'lib/minitest/benchmark.rb', line 324
def sigma enum, &block enum = enum.map(&block) if block enum.inject { sum, n sum + n } end
#validation_for_fit(msg, threshold)
Returns a proc that calls the specified fit method and asserts that the error is within a tolerable threshold.
# File 'lib/minitest/benchmark.rb', line 333
def validation_for_fit msg, threshold proc do range, times a, b, rr = send "fit_#{msg}", range, times assert_operator rr, :>=, threshold [a, b, rr] end end