Module: SimpleCov
Overview
Code coverage for ruby. Please check out README for a full introduction.
Constant Summary
-
CRITERION_TO_RUBY_COVERAGE =
# File 'lib/simplecov.rb', line 10{ branch: :branches, line: :lines, method: :methods, oneshot_line: :oneshot_lines }.freeze -
VERSION =
# File 'lib/simplecov/version.rb', line 4"0.22.0"
Class Attribute Summary
-
.external_at_exit?
rw
Alias for .external_at_exit.
- .pid rw
-
.process_start_time
rw
When this process started tracking coverage.
-
.result
readonly
Returns the result for the current coverage run, merging it across test suites from cache using
ResultMergerif use_merging is activated (default). -
.result? ⇒ Boolean
readonly
Returns nil if the result has not been computed Otherwise, returns the result.
- .probably_running_parallel_tests? ⇒ Boolean readonly private
- .final_result_process? ⇒ Boolean readonly Internal use only Internal use only
- .ready_to_process_results? ⇒ Boolean readonly Internal use only Internal use only
Configuration - Extended
| active_session? | Whether SimpleCov has anything to do at exit: the Coverage module is actively tracking, or a |
| branch_coverage?, branch_coverage_supported?, coverage_for_eval_enabled?, coverage_for_eval_supported?, | |
| enabled_for_subprocesses? | gets the enabled_for_subprocess configuration. |
| filters | Returns the list of configured filters. |
| filters=, | |
| formatter | Gets or sets the configured formatter. |
| formatter=, | |
| formatters | Gets the configured formatters. |
| formatters= | Sets the configured formatters. |
| groups | Returns the configured groups. |
| groups=, method_coverage?, method_coverage_supported?, | |
| print_error_status | Whether we should print non-success status codes. |
| print_error_status= | |
Class Method Summary
- .at_exit_behavior
-
.clear_result
Clear out the previously cached .result.
-
.collate(result_filenames, profile = nil, ignore_timeout: true, &block)
Collate a series of
SimpleCovresult files into a singleSimpleCovoutput. -
.coverage_statistics_key(criterion)
:oneshot_linedata is folded into the:linebucket ofcoverage_statisticsbyResultAdapter, so use:lineto look up stats for either criterion. -
.external_at_exit
(also: .external_at_exit?)
rw
Basically, should we take care of at_exit behavior or something else? Used by the minitest plugin.
-
.filtered(files)
Applies the configured filters to the given array of
SourceFileitems. -
.grouped(files)
Applies the configured groups to the given array of
SourceFileitems. -
.install_at_exit_hook
Install the at_exit hook that formats results and runs exit-code checks.
-
.load_profile(name)
Applies the profile of given name on
SimpleCovconfiguration. - .process_results_and_report_error
- .result_exit_status(result)
-
.start(profile = nil)
Sets up
SimpleCovto run against your project. -
.start_tracking
Begin coverage tracking without applying configuration.
-
.adapt_coverage_result ⇒ Hash
private
Unite the result so it wouldn't matter what coverage type was called.
-
.add_not_loaded_files(result)
private
Finds files that were to be tracked but were not loaded and initializes the line-by-line coverage to zero (if relevant) or nil (comments / whitespace etc).
- .initial_setup(profile, &block) private
- .lookup_corresponding_ruby_coverage_name(criterion) private
-
.make_parallel_tests_available
private
parallel_tests isn't always available, see: https://github.com/grosser/parallel_tests/issues/772.
-
.process_coverage_result ⇒ Hash
private
Call steps that handle process coverage result.
-
.remove_useless_results ⇒ Hash
private
Filtercoverage result The result before filter also has result of coverage for files are not related to the project like loaded gems coverage. -
.result_with_not_loaded_files ⇒ Hash
private
Initialize result with files that are not included by coverage and added inside the config block.
-
.start_coverage_measurement
private
Trigger
Coverage.startwith the configured criteria. - .start_coverage_with_criteria private
-
.warn_if_jruby_full_trace_disabled
private
JRuby coverage data is unreliable unless full-trace mode is enabled.
-
.exit_and_report_previous_error(exit_status)
Internal use only
Internal use only
Thinking: Move this behavior earlier so if there was an error we do nothing?
-
.exit_status_from_exception
Internal use only
Internal use only
Returns the exit status from the exit exception.
- .previous_error?(error_exit_status) ⇒ Boolean Internal use only Internal use only
-
.process_result(result)
Internal use only
Internal use only
Usage:
-
.round_coverage(coverage)
Internal use only
Internal use only
Rounding down to be extra strict, see
#679 -
.run_exit_tasks!
Internal use only
Internal use only
Called from at_exit block.
-
.wait_for_other_processes
Internal use only
Internal use only
simplecov:disable Methods below only fire under parallel_tests; not reachable from a single-process rspec run.
- .wait_for_parallel_results Internal use only Internal use only
- .write_last_run(result) Internal use only Internal use only
Configuration - Extended
| add_filter | Add a filter to the processing chain. |
| add_group | Define a group for files. |
| at_exit | Gets or sets the behavior to process coverage results. |
| at_fork | Gets or sets the behavior to start a new forked Process. |
| clear_coverage_criteria, | |
| command_name | The name of the command (a.k.a. Test Suite) currently running. |
| configure | Allows you to configure simplecov in a block instead of prepending |
| coverage_criteria, | |
| coverage_criterion | Define which coverage criterion should be evaluated. |
| coverage_criterion_enabled?, | |
| coverage_dir | The name of the output and cache directory. |
| coverage_path | Returns the full path to the output directory using |
| current_nocov_token | Internal accessor used by |
| enable_coverage, enable_coverage_for_eval, | |
| enable_for_subprocesses | gets or sets the enabled_for_subprocess configuration when true, this will inject |
| maximum_coverage_drop | Defines the maximum coverage drop at once allowed for the testsuite to pass. |
| merge_timeout | Defines the maximum age (in seconds) of a resultset to still be included in merged results. |
| minimum_coverage | Defines the minimum overall coverage required for the testsuite to pass. |
| minimum_coverage_by_file | Defines the minimum coverage per file required for the testsuite to pass. |
| minimum_coverage_by_group | Defines the minimum coverage per group required for the testsuite to pass. |
| nocov_token | Certain code blocks (i.e. Ruby-implementation specific code) can be excluded from the coverage metrics by wrapping it inside # :nocov: comment blocks. |
| primary_coverage, | |
| profiles | Returns the hash of available profiles. |
| project_name | Returns the project name - currently assuming the last dirname in the |
| raise_on_invalid_coverage, | |
| refuse_coverage_drop | Refuses any coverage drop. |
| root | The root for the project. |
| skip_token | Alias for Configuration#nocov_token. |
| track_files | Coverage results will always include files matched by this glob, whether or not they were explicitly required. |
| tracked_files | Returns the glob that will be used to include files that were not explicitly required. |
| use_merging | Defines whether to use result merging so all your test suites (test:units, test:functionals, cucumber, ...) are joined and combined into a single coverage report. |
| minimum_possible_coverage_exceeded, | |
| parse_filter | The actual filter processor. |
| raise_if_criterion_disabled, raise_if_criterion_unsupported, | |
| restore_ivars | Copy instance variables back to block_context and restore our saved values. |
| swap_ivars_from | Copy instance variables from block_context into self, saving any of ours that would be clobbered. |
Class Attribute Details
.external_at_exit? (rw)
Alias for .external_at_exit.
# File 'lib/simplecov.rb', line 34
alias external_at_exit? external_at_exit
.final_result_process? ⇒ Boolean (readonly)
# File 'lib/simplecov.rb', line 310
def final_result_process? return true unless defined?(ParallelTests) && ENV["TEST_ENV_NUMBER"] # parallel_tests sets the first process's TEST_ENV_NUMBER to "" and # `ParallelTests.last_process?` does `"" == "1"`, which is false — # so with PARALLEL_TEST_GROUPS=1 the only process in the run never # runs the final-result work. Treat any single-group run as final. ENV["PARALLEL_TEST_GROUPS"].to_i <= 1 || ParallelTests.last_process? end
.pid (rw)
[ GitHub ]# File 'lib/simplecov.rb', line 24
attr_accessor :pid
.probably_running_parallel_tests? ⇒ Boolean (readonly, private)
[ GitHub ]
# File 'lib/simplecov.rb', line 486
def probably_running_parallel_tests? ENV.fetch("TEST_ENV_NUMBER", nil) && ENV.fetch("PARALLEL_TEST_GROUPS", nil) end
.process_start_time (rw)
When this process started tracking coverage. Captured by .start so JSONFormatter can detect when an existing coverage.json was written by a sibling process running concurrently.
# File 'lib/simplecov.rb', line 29
attr_accessor :process_start_time
.ready_to_process_results? ⇒ Boolean (readonly)
# File 'lib/simplecov.rb', line 265
def ready_to_process_results? final_result_process? && result? end
.result (readonly)
Returns the result for the current coverage run, merging it across test suites
from cache using ::SimpleCov::ResultMerger if use_merging is activated (default)
# File 'lib/simplecov.rb', line 140
def result return @result if result? # Collect our coverage result process_coverage_result if defined?(Coverage) && Coverage.running? # If we're using merging of results, store the current result # first (if there is one), then merge the results and return those if use_merging wait_for_other_processes SimpleCov::ResultMerger.store_result(@result) if result? @result = SimpleCov::ResultMerger.merged_result end @result end
.result? ⇒ Boolean (readonly)
Returns nil if the result has not been computed Otherwise, returns the result
# File 'lib/simplecov.rb', line 161
def result? defined?(@result) && @result end
Class Method Details
.adapt_coverage_result ⇒ Hash (private)
Unite the result so it wouldn't matter what coverage type was called
# File 'lib/simplecov.rb', line 445
def adapt_coverage_result @result = SimpleCov::ResultAdapter.call(@result) end
.add_not_loaded_files(result) (private)
Finds files that were to be tracked but were not loaded and initializes the line-by-line coverage to zero (if relevant) or nil (comments / whitespace etc).
# File 'lib/simplecov.rb', line 410
def add_not_loaded_files(result) return [result, Set.new] unless tracked_files result = result.dup # Glob and expand relative to SimpleCov.root, not Dir.pwd — test runners # that chdir (or CI scripts that invoke the suite from a subdir) would # otherwise silently miss the unloaded-file injection and produce a # different file set per environment. See issue #1106. not_loaded_files = Dir.glob(tracked_files, base: root).each_with_object(Set.new) do |file, set| absolute_path = File.(file, root) next if result.key?(absolute_path) result[absolute_path] = SimulateCoverage.call(absolute_path) set << absolute_path end [result, not_loaded_files] end
.at_exit_behavior
[ GitHub ]# File 'lib/simplecov.rb', line 207
def at_exit_behavior # If we are in a different process than called start, don't interfere. return if SimpleCov.pid != Process.pid # If Coverage is no longer running (e.g. someone manually stopped it # or a test consumed the result) then don't run exit tasks. SimpleCov.run_exit_tasks! if Coverage.running? end
.clear_result
Clear out the previously cached .result. Primarily useful in testing
# File 'lib/simplecov.rb', line 203
def clear_result @result = nil end
.collate(result_filenames, profile = nil, ignore_timeout: true, &block)
Collate a series of SimpleCov result files into a single SimpleCov output.
You can optionally specify configuration with a block:
SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"]
OR
SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' # using rails profile
OR
SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"] do
add_filter 'test'
end
OR
SimpleCov.collate Dir["simplecov-resultset-*/.resultset.json"], 'rails' do
add_filter 'test'
end
Please check out the RDoc for ::SimpleCov::Configuration to find about
available config options, or checkout the README for more in-depth
information about coverage collation
By default collate ignores the merge_timeout so all results of all files specified will be
merged together. If you want to honor the merge_timeout then provide the keyword argument
ignore_timeout: false.
# File 'lib/simplecov.rb', line 125
def collate(result_filenames, profile = nil, ignore_timeout: true, &block) raise "There are no reports to be merged" if result_filenames.empty? initial_setup(profile, &block) # Use the ResultMerger to produce a single, merged result, ready to use. @result = ResultMerger.merge_and_store(*result_filenames, ignore_timeout: ignore_timeout) run_exit_tasks! end
.coverage_statistics_key(criterion)
:oneshot_line data is folded into the :line bucket of
coverage_statistics by ::SimpleCov::ResultAdapter, so use :line to look
up stats for either criterion.
# File 'lib/simplecov.rb', line 20
def coverage_statistics_key(criterion) criterion == :oneshot_line ? :line : criterion end
.exit_and_report_previous_error(exit_status)
Thinking: Move this behavior earlier so if there was an error we do nothing?
# File 'lib/simplecov.rb', line 257
def exit_and_report_previous_error(exit_status) if print_error_status warn("Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected") end Kernel.exit(exit_status) end
.exit_status_from_exception
Returns the exit status from the exit exception
.external_at_exit (rw) Also known as: .external_at_exit?
Basically, should we take care of at_exit behavior or something else? Used by the minitest plugin. See lib/minitest/simplecov_plugin.rb
# File 'lib/simplecov.rb', line 33
attr_accessor :external_at_exit
.filtered(files)
Applies the configured filters to the given array of ::SimpleCov::SourceFile items
.grouped(files)
Applies the configured groups to the given array of ::SimpleCov::SourceFile items
# File 'lib/simplecov.rb', line 179
def grouped(files) return {} if groups.empty? grouped = groups.transform_values do |filter| SimpleCov::FileList.new(files.select { |source_file| filter.matches?(source_file) }) end in_group = grouped.values.flat_map(&:to_a) ungrouped = files.reject { |source_file| in_group.include?(source_file) } grouped["Ungrouped"] = SimpleCov::FileList.new(ungrouped) if ungrouped.any? grouped end
.initial_setup(profile, &block) (private)
[ GitHub ]# File 'lib/simplecov.rb', line 378
def initial_setup(profile, &block) load_profile(profile) if profile configure(&block) if block end
.install_at_exit_hook
Install the at_exit hook that formats results and runs exit-code checks. .start calls this automatically. Idempotent — safe to call multiple times. Callers that drive the formatting pipeline themselves (e.g., dogfood test setups) can skip it by using .start_tracking directly instead of .start.
# File 'lib/simplecov.rb', line 66
def install_at_exit_hook return if @at_exit_hook_installed @at_exit_hook_installed = true Kernel.at_exit do next if SimpleCov.external_at_exit? SimpleCov.at_exit_behavior end end
.load_profile(name)
Applies the profile of given name on SimpleCov configuration
# File 'lib/simplecov.rb', line 196
def load_profile(name) profiles.load(name) end
.lookup_corresponding_ruby_coverage_name(criterion) (private)
[ GitHub ]# File 'lib/simplecov.rb', line 402
def lookup_corresponding_ruby_coverage_name(criterion) CRITERION_TO_RUBY_COVERAGE.fetch(criterion) end
.make_parallel_tests_available (private)
parallel_tests isn't always available, see: https://github.com/grosser/parallel_tests/issues/772
# File 'lib/simplecov.rb', line 472
def make_parallel_tests_available return if defined?(ParallelTests) # simplecov:disable — only true after a previous load return unless probably_running_parallel_tests? # simplecov:disable — false outside parallel_tests # simplecov:disable — only fires under a real parallel_tests setup require "parallel_tests" rescue LoadError warn( "SimpleCov guessed you were running inside parallel tests but couldn't load it. " \ "Please file a bug report with us!" ) # simplecov:enable end
.previous_error?(error_exit_status) ⇒ Boolean
.process_coverage_result ⇒ Hash (private)
Call steps that handle process coverage result
# File 'lib/simplecov.rb', line 434
def process_coverage_result remove_useless_results adapt_coverage_result result_with_not_loaded_files end
.process_result(result)
Usage:
exit_status = SimpleCov.process_result(SimpleCov.result, exit_status)
# File 'lib/simplecov.rb', line 284
def process_result(result) result_exit_status = result_exit_status(result) write_last_run(result) if result_exit_status == SimpleCov::ExitCodes::SUCCESS result_exit_status end
.process_results_and_report_error
[ GitHub ]# File 'lib/simplecov.rb', line 269
def process_results_and_report_error exit_status = process_result(result) # Force exit with stored status (see github issue #5) return unless exit_status.positive? warn("SimpleCov failed with exit #{exit_status} due to a coverage related error") if print_error_status Kernel.exit exit_status end
.remove_useless_results ⇒ Hash (private)
::SimpleCov::Filter coverage result
The result before filter also has result of coverage for files
are not related to the project like loaded gems coverage.
# File 'lib/simplecov.rb', line 456
def remove_useless_results @result = SimpleCov::UselessResultsRemover.call(Coverage.result) end
.result_exit_status(result)
[ GitHub ]# File 'lib/simplecov.rb', line 298
def result_exit_status(result) coverage_limits = CoverageLimits.new( minimum_coverage: minimum_coverage, minimum_coverage_by_file: minimum_coverage_by_file, minimum_coverage_by_group: minimum_coverage_by_group, maximum_coverage_drop: maximum_coverage_drop ) ExitCodes::ExitCodeHandling.call(result, coverage_limits: coverage_limits) end
.result_with_not_loaded_files ⇒ Hash (private)
Initialize result with files that are not included by coverage and added inside the config block
# File 'lib/simplecov.rb', line 466
def result_with_not_loaded_files result, not_loaded_files = add_not_loaded_files(@result) @result = SimpleCov::Result.new(result, not_loaded_files: not_loaded_files) end
.round_coverage(coverage)
Rounding down to be extra strict, see #679
# File 'lib/simplecov.rb', line 372
def round_coverage(coverage) coverage.floor(2) end
.run_exit_tasks!
Called from at_exit block
# File 'lib/simplecov.rb', line 220
def run_exit_tasks! error_exit_status = exit_status_from_exception at_exit.call exit_and_report_previous_error(error_exit_status) if previous_error?(error_exit_status) process_results_and_report_error if ready_to_process_results? end
.start(profile = nil)
Sets up SimpleCov to run against your project.
You can optionally specify a profile to use as well as configuration with a block:
SimpleCov.start
OR
SimpleCov.start 'rails' # using rails profile
OR
SimpleCov.start do
add_filter 'test'
end
OR
SimpleCov.start 'rails' do
add_filter 'test'
end
Please check out the RDoc for ::SimpleCov::Configuration to find about available config options
# File 'lib/simplecov.rb', line 53
def start(profile = nil, &) initial_setup(profile, &) start_tracking install_at_exit_hook end
.start_coverage_measurement (private)
Trigger Coverage.start with the configured criteria. Every supported
runtime (CRuby >= 3.1, JRuby >= 9.4, TruffleRuby >= 22) accepts the
criteria-hash form, so no compatibility fallback is needed.
# File 'lib/simplecov.rb', line 388
def start_coverage_measurement start_coverage_with_criteria end
.start_coverage_with_criteria (private)
[ GitHub ]# File 'lib/simplecov.rb', line 392
def start_coverage_with_criteria start_arguments = coverage_criteria.to_h do |criterion| [lookup_corresponding_ruby_coverage_name(criterion), true] end start_arguments[:eval] = true if coverage_for_eval_enabled? Coverage.start(start_arguments) unless Coverage.running? end
.start_tracking
Begin coverage tracking without applying configuration. Pairs with
SimpleCov.configure { ... } for callers that want to separate
the two — for example a dogfood test that has already started
Coverage itself before requiring simplecov, but still wants the
process_start_time / pid / fork-hook bookkeeping.
# File 'lib/simplecov.rb', line 84
def start_tracking require "coverage" warn_if_jruby_full_trace_disabled # simplecov:disable — fork-hook is enabled via SimpleCov.enable_for_subprocesses, off by default require_relative "simplecov/process" if SimpleCov.enabled_for_subprocesses? && ::Process.respond_to?(:_fork) # simplecov:enable make_parallel_tests_available @result = nil self.pid = Process.pid self.process_start_time = Time.now start_coverage_measurement end
.wait_for_other_processes
simplecov:disable Methods below only fire under parallel_tests; not reachable from a single-process rspec run. Cucumber's test_projects exercise the parallel_tests integration end-to-end in subprocesses, but those subprocesses don't merge their Coverage data back into the parent this dogfood report measures.
# File 'lib/simplecov.rb', line 329
def wait_for_other_processes return unless defined?(ParallelTests) && final_result_process? ParallelTests.wait_for_other_processes_to_finish # ParallelTests signals "done" before at_exit handlers finish, so other # processes may still be writing their results. Poll the resultset until # all parallel groups have reported or a timeout is reached. wait_for_parallel_results end
.wait_for_parallel_results
# File 'lib/simplecov.rb', line 342
def wait_for_parallel_results expected = ENV["PARALLEL_TEST_GROUPS"]&.to_i return unless expected && expected > 1 # simplecov:disable branch — only false in real parallel_tests run # simplecov:disable — only fires when ENV is set with >1 group deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + 10 loop do resultset = SimpleCov::ResultMerger.read_resultset break if resultset.size >= expected break if Process.clock_gettime(Process::CLOCK_MONOTONIC) > deadline sleep 0.1 end # simplecov:enable end
.warn_if_jruby_full_trace_disabled (private)
JRuby coverage data is unreliable unless full-trace mode is enabled.
# File 'lib/simplecov.rb', line 494
def warn_if_jruby_full_trace_disabled return unless defined?(JRUBY_VERSION) && defined?(JRuby) # simplecov:disable — JRuby-only branch # simplecov:disable — JRuby-only branches; unreachable from CRuby return if org.jruby.RubyInstanceConfig.FULL_TRACE_ENABLED warn 'Coverage may be inaccurate; set the "--debug" command line option, ' \ 'or do JRUBY_OPTS="--debug" ' \ 'or set the "debug.fullTrace=true" option in your .jrubyrc' # simplecov:enable end
.write_last_run(result)
# File 'lib/simplecov.rb', line 361
def write_last_run(result) SimpleCov::LastRun.write(result: result.coverage_statistics.transform_values do |stats| round_coverage(stats.percent) end) end