123456789_123456789_123456789_123456789_123456789_

Class: Prism::Source

Relationships & Source Files
Extension / Inclusion / Inheritance Descendants
Subclasses:
ASCIISource
Inherits: Object
Defined in: lib/prism/parse_result.rb,
ext/prism/extension.c

Overview

This represents a source of Ruby code that has been parsed. It is used in conjunction with locations to allow them to resolve line numbers and source ranges.

Class Method Summary

Instance Attribute Summary

  • #checksum readonly

    The checksum of the source code which can be used to quickly determine if the source has changed.

  • #comment_attachments readonly

    The comments attached to nodes, grouped by node in order of node id.

  • #comment_ranges readonly

    Indices into comment_attachments, indexed by node id.

  • #comments readonly

    The list of comments that were encountered during parsing.

  • #source readonly

    The source code that this source object represents.

  • #start_line readonly

    The line number where this source starts.

Instance Method Summary

Constructor Details

.new(source, start_line, offsets, comments = [], comment_attachments = [], comment_ranges = [], checksum = nil) ⇒ Source

Create a new source object with the given source code. The offsets parameter can be either an Array of Integer byte offsets or a packed binary string of uint32_t values (from the C extension).

: (String source, Integer start_line, Array | String offsets, ?Array comments, ?Array comment_attachments, ?Array comment_ranges, ?Integer? checksum) -> void

[ GitHub ] 

  


# File 'lib/prism/parse_result.rb', line 106



def initialize(source, start_line, offsets, comments = [], comment_attachments = [], comment_ranges = [], checksum = nil)
  @source = source
  @start_line = start_line
  @checksum = checksum
  @offsets = offsets
  @comments = comments
  @comment_attachments = comment_attachments
  @comment_ranges = comment_ranges
end


  







Class Method Details



  

    .for(source, start_line, offsets, comments = [], comment_attachments = [], comment_ranges = [], checksum = nil)  
  



  

    
Create a new source object with the given source code. This method should
be used instead of .new and it will return either a Source or a
specialized and more performant ASCIISource if no multibyte characters
are present in the source code.


Note that if you are calling this method manually, you will need to supply
the start_line and offsets parameters. start_line is the line number that
the source starts on, which is typically 1 but can be different if this
source is a subset of a larger source or if this is an eval. offsets is an
array of byte offsets for the start of each line in the source code, which
can be calculated by iterating through the source code and recording the
byte offset whenever a newline character is encountered.  The first
element is always 0 to mark the first line.


The comments, comment_attachments, and comment_ranges parameters are the
lists described by the attributes of the same names. They are passed in
empty when the source is created by the parser and filled in afterward,
because the comments themselves refer to the source.


The checksum parameter is the value described by the attribute of the same
name. It is nil when the source is not created by the parser.


: (String source, Integer start_line, Array offsets, ?Array comments, ?Array comment_attachments, ?Array comment_ranges, ?Integer? checksum) -> Source

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 41



def self.for(source, start_line, offsets, comments = [], comment_attachments = [], comment_ranges = [], checksum = nil)
  if source.ascii_only?
    ASCIISource.new(source, start_line, offsets, comments, comment_attachments, comment_ranges, checksum)
  elsif source.encoding == Encoding::BINARY
    source.force_encoding(Encoding::UTF_8)

    if source.valid_encoding?
      new(source, start_line, offsets, comments, comment_attachments, comment_ranges, checksum)
    else
      # This is an extremely niche use case where the file is marked as
      # binary, contains multi-byte characters, and those characters are not
      # valid UTF-8. In this case we'll mark it as binary and fall back to
      # treating everything as a single-byte character. This _may_ cause
      # problems when asking for code units, but it appears to be the
      # cleanest solution at the moment.
      source.force_encoding(Encoding::BINARY)
      ASCIISource.new(source, start_line, offsets, comments, comment_attachments, comment_ranges, checksum)
    end
  else
    new(source, start_line, offsets, comments, comment_attachments, comment_ranges, checksum)
  end
end


  







Instance Attribute Details



  

    #checksum   (readonly)
  



  

    
The checksum of the source code which can be used to quickly determine if
the source has changed.

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 72



attr_reader :checksum #: Integer?


  








  

    #comment_attachments   (readonly)
  



  

    
The comments attached to nodes, grouped by node in order of node id. This
is empty unless the source was parsed with the attach_comments option.

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 81



attr_reader :comment_attachments #: Array[Comment]


  








  

    #comment_ranges   (readonly)
  



  

    
Indices into comment_attachments, indexed by node id. For each node that
owns comments, the index at which its comments begin is stored at its node
id and the index at which they end is stored at the following node id. The
other entries are nil. This is empty unless the source was parsed with the
attach_comments option.

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 88



attr_reader :comment_ranges #: Array[Integer?]


  








  

    #comments   (readonly)
  



  

    
The list of comments that were encountered during parsing. When the source
is parsed with the attach_comments option, these are sorted by start
offset.

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 77



attr_reader :comments #: Array[Comment]


  








  

    #source   (readonly)
  



  

    
The source code that this source object represents.

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 65



attr_reader :source #: String


  








  

    #start_line   (readonly)
  



  

    
The line number where this source starts.

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 68



attr_reader :start_line #: Integer


  







Instance Method Details



  

    #byte_offset(line, column)  
  



  

    
Converts the line number and column in bytes to a byte offset.


: (Integer line, Integer column) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 176



def byte_offset(line, column)
  normal = line - @start_line
  raise IndexError if normal < 0
  offsets.fetch(normal) + column
rescue IndexError
  raise ArgumentError, "line #{line} is out of range"
end


  








  

    #character_column(byte_offset)  
  



  

    
Return the column in characters for the given byte offset.


: (Integer byte_offset) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 225



def character_column(byte_offset)
  character_offset(byte_offset) - character_offset(line_start(byte_offset))
end


  








  

    #character_offset(byte_offset)  
  



  

    
Return the character offset for the given byte offset.


: (Integer byte_offset) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 218



def character_offset(byte_offset)
  (source.byteslice(0, byte_offset) or raise).length
end


  








  

    #code_units_cache(encoding)  
  



  

    
Generate a cache that targets a specific encoding for calculating code
unit offsets.


: (Encoding encoding) -> CodeUnitsCache

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 259



def code_units_cache(encoding)
  CodeUnitsCache.new(source, encoding)
end


  








  

    #code_units_column(byte_offset, encoding)  
  



  

    
Returns the column in code units for the given encoding for the
given byte offset.


: (Integer byte_offset, Encoding encoding) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 267



def code_units_column(byte_offset, encoding)
  code_units_offset(byte_offset, encoding) - code_units_offset(line_start(byte_offset), encoding)
end


  








  

    #code_units_offset(byte_offset, encoding)  
  



  

    
Returns the offset from the start of the file for the given byte offset
counting in code units for the given encoding.


This method is tested with UTF-8, UTF-16, and UTF-32. If there is the
concept of code units that differs from the number of characters in other
encodings, it is not captured here.


We purposefully replace invalid and undefined characters with replacement
characters in this conversion. This happens for two reasons. First, it's
possible that the given byte offset will not occur on a character
boundary. Second, it's possible that the source code will contain a
character that has no equivalent in the given encoding.


: (Integer byte_offset, Encoding encoding) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 243



def code_units_offset(byte_offset, encoding)
  return byte_offset if encoding == Encoding::UTF_8

  byteslice = (source.byteslice(0, byte_offset) or raise).encode(encoding, invalid: :replace, undef: :replace)

  if encoding == Encoding::UTF_16LE || encoding == Encoding::UTF_16BE
    byteslice.bytesize / 2
  else
    byteslice.length
  end
end


  








  

    #column(byte_offset)  
  



  

    
Return the column in bytes for the given byte offset.


: (Integer byte_offset) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 211



def column(byte_offset)
  byte_offset - line_start(byte_offset)
end


  








  

    #comments_for(node_id)  
  



  

    
The comments attached to the node with the given id, ordered by start
offset. Comments are only attached when the source is parsed with the
attach_comments option.


: (Integer node_id) -> Array

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 142



def comments_for(node_id)
  start = comment_ranges[node_id]
  finish = comment_ranges[node_id + 1]
  return [] if start.nil? || finish.nil?

  comment_attachments[start...finish] || []
end


  








  

    #deep_freeze  
  



  

    
Freeze this object and the objects it contains.


: () -> void

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 274



def deep_freeze
  source.freeze
  offsets.freeze
  comments.freeze
  comment_attachments.freeze
  comment_ranges.freeze
  freeze
end


  








  

    #encoding  
  



  

    
Returns the encoding of the source code, which is set by parameters to the
parser or by the encoding magic comment.


: () -> Encoding

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 154



def encoding
  source.encoding
end


  








  

    #find_line(byte_offset)  
  

  

    This method is for internal use only.
  



  

    
Binary search through the offsets to find the index for the given
byte offset.


: (Integer byte_offset) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 287



def find_line(byte_offset) # :nodoc:
  index = offsets.bsearch_index { |offset| offset > byte_offset } || offsets.length
  index - 1
end


  








  

    #line(byte_offset)  
  



  

    
Binary search through the offsets to find the line number for the given
byte offset.


: (Integer byte_offset) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 188



def line(byte_offset)
  start_line + find_line(byte_offset)
end


  








  

    #line_end(byte_offset)  
  



  

    
Returns the byte offset of the end of the line corresponding to the given
byte offset.


: (Integer byte_offset) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 204



def line_end(byte_offset)
  offsets[find_line(byte_offset) + 1] || source.bytesize
end


  








  

    #line_start(byte_offset)  
  



  

    
Return the byte offset of the start of the line corresponding to the given
byte offset.


: (Integer byte_offset) -> Integer

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 196



def line_start(byte_offset)
  offsets[find_line(byte_offset)]
end


  








  

    #lines  
  



  

    
Returns the lines of the source code as an array of strings.


: () -> Array

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 161



def lines
  source.lines
end


  








  

    #offsets  
  



  

    
The list of newline byte offsets in the source code. When initialized from
the C extension, this may be a packed binary string of uint32_t values
that is lazily unpacked on first access.


: () -> Array

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 95



def offsets
  offsets = @offsets
  return offsets if offsets.is_a?(Array)
  @offsets = offsets.unpack("L*")
end


  








  

    #replace_checksum(checksum)  
  



  

    
Replace the value of checksum with the given value.


: (Integer checksum) -> void

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 126



def replace_checksum(checksum)
  @checksum = checksum
end


  








  

    #replace_offsets(offsets)  
  



  

    
Replace the value of offsets with the given value.


: (Array offsets) -> void

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 133



def replace_offsets(offsets)
  @offsets = offsets
end


  








  

    #replace_start_line(start_line)  
  



  

    
Replace the value of start_line with the given value.


: (Integer start_line) -> void

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 119



def replace_start_line(start_line)
  @start_line = start_line
end


  








  

    #slice(byte_offset, length)  
  



  

    
Perform a byteslice on the source code using the given byte offset and
byte length.


: (Integer byte_offset, Integer length) -> String

  



  [ GitHub ]


  

  


# File 'lib/prism/parse_result.rb', line 169



def slice(byte_offset, length)
  source.byteslice(byte_offset, length) or raise
end