Module: RuboCop::AST::PrismLazyTokens Private

Relationships & Source Files
Defined in:	lib/rubocop/ast/processed_source.rb

Overview

Extends the prism translation parsers so that the conversion of tokens into the parser gem’s format is deferred until the tokens are first accessed. Building the tokens is a significant part of the translation cost, and not every caller needs them.

Instance Method Summary

#tokenize_deferred(source_buffer) Internal use only

Same contract as Parser::Base#tokenize, except the tokens are returned as a callable that performs the conversion when invoked.
#deferred_tokens(source_buffer, tokens, offset_cache) private Internal use only

Instance Method Details

#deferred_tokens(source_buffer, tokens, offset_cache) (private)

[ GitHub ]

# File 'lib/rubocop/ast/processed_source.rb', line 54


def deferred_tokens(source_buffer, tokens, offset_cache)
  lambda do
    @source_buffer = source_buffer
    begin
      build_tokens(tokens, offset_cache)
    ensure
      @source_buffer = nil
    end
  end
end

#tokenize_deferred(source_buffer)

Same contract as Parser::Base#tokenize, except the tokens are returned as a callable that performs the conversion when invoked.

[ GitHub ]

# File 'lib/rubocop/ast/processed_source.rb', line 36


def tokenize_deferred(source_buffer)
  @source_buffer = source_buffer
  source = source_buffer.source

  offset_cache = build_offset_cache(source)
  result = unwrap(@parser.parse_lex(source, **prism_options), offset_cache)

  program, tokens = result.value
  ast = build_ast(program, offset_cache) if result.success?
  comments = build_comments(result.comments, offset_cache)

  [ast, comments, deferred_tokens(source_buffer, tokens, offset_cache)]
ensure
  @source_buffer = nil
end