123456789_123456789_123456789_123456789_123456789_

Class: RDoc::Parser

Relationships & Source Files
Namespace Children
Modules:
RubyTools, Text
Classes:
C, ChangeLog, Markdown, RD, Ruby, Simple
Extension / Inclusion / Inheritance Descendants
Subclasses:
C, ChangeLog, Markdown, RD, Ruby, Simple
Inherits: Object
Defined in: lib/rdoc/parser.rb

Overview

A parser is simple a class that subclasses Parser and implements #scan to fill in an TopLevel with parsed data.

The initialize method takes an TopLevel to fill with parsed content, the name of the file to be parsed, the content of the file, an Options object and an Stats object to inform the user of parsed items. The scan method is then called to parse the file and must return the TopLevel object. By calling super these items will be set for you.

In order to be used by RDoc the parser needs to register the file extensions it can parse. Use .parse_files_matching to register extensions.

require 'rdoc'

class RDoc::Parser::Xyz < RDoc::Parser
  parse_files_matching /\.xyz$/

  def initialize top_level, file_name, content, options, stats
    super

    # extra initialization if needed
  end

  def scan
    # parse file and fill in @top_level
  end
end

Class Attribute Summary

  • .parsers readonly

    An Array of arrays that maps file extension (or name) regular expressions to parser classes that will parse matching filenames.

Class Method Summary

Instance Attribute Summary

  • #file_name readonly

    The name of the file being parsed.

Constructor Details

.new(top_level, file_name, content, options, stats) ⇒ Parser

Creates a new Parser storing top_level, #file_name, content, options and stats in instance variables. In @preprocess an Markup::PreProcess object is created which allows processing of directives.

[ GitHub ] 

  


# File 'lib/rdoc/parser.rb', line 285



def initialize top_level, file_name, content, options, stats
  @top_level = top_level
  @top_level.parser = self.class
  @store = @top_level.store

  @file_name = file_name
  @content = content
  @options = options
  @stats = stats

  @preprocess = RDoc::Markup::PreProcess.new @file_name, @options.rdoc_include
  @preprocess.options = @options
end


  







Class Attribute Details



  

    .parsers   (readonly)
  



  

    

An Array of arrays that maps file extension (or name) regular expressions to parser classes that will parse matching filenames.



Use parse_files_matching to register a parser's file extensions.


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 45



attr_reader :parsers


  







Class Method Details



  

    .alias_extension(old_ext, new_ext)  
  



  

    

Alias an extension to another extension. After this call, files ending “new_ext” will be parsed using the same parser as “old_ext”


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 58



def self.alias_extension(old_ext, new_ext)
  old_ext = old_ext.sub(/^\.(.*)/, '\1')
  new_ext = new_ext.sub(/^\.(.*)/, '\1')

  parser = can_parse_by_name "xxx.#{old_ext}"
  return false unless parser

  RDoc::Parser.parsers.unshift [/\.#{new_ext}$/, parser]

  true
end


  








  

    .binary?(file)  ⇒ Boolean 
  



  

    

Determines if the file is a “binary” file which basically means it has content that an RDoc parser shouldn't try to consume.


  





  


  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 74



def self.binary?(file)
  return false if file =~ /\.(rdoc|txt)$/

  s = File.read(file, 1024) or return false

  have_encoding = s.respond_to? :encoding

  return true if s[0, 2] == Marshal.dump('')[0, 2] or s.index("\x00")

  if have_encoding then
    mode = "r"
    s.sub!(/\A#!.*\n/, '')     # assume shebang line isn't longer than 1024.
    encoding = s[/^\s*\#\s*(?:-\*-\s*)?(?:en)?coding:\s*([^\s;]+?)(?:-\*-|[\s;])/, 1]
    mode = "rb:#{encoding}" if encoding
    s = File.open(file, mode) {|f| f.gets(nil, 1024)}

    not s.valid_encoding?
  else
    if 0.respond_to? :fdiv then
      s.count("\x00-\x7F", "^ -~\t\r\n").fdiv(s.size) > 0.3
    else # HACK 1.8.6
      (s.count("\x00-\x7F", "^ -~\t\r\n").to_f / s.size) > 0.3
    end
  end
end


  








  

    .can_parse(file_name)  
  



  

    

Return a parser that can handle a particular extension


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 142



def self.can_parse file_name
  parser = can_parse_by_name file_name

  # HACK Selenium hides a jar file using a .txt extension
  return if parser == RDoc::Parser::Simple and zip? file_name

  parser
end


  








  

    .can_parse_by_name(file_name)  
  



  

    

Returns a parser that can handle the extension for #file_name.  This does not depend upon the file being readable.


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 155



def self.can_parse_by_name file_name
  _, parser = RDoc::Parser.parsers.find { |regexp,| regexp =~ file_name }

  # The default parser must not parse binary files
  ext_name = File.extname file_name
  return parser if ext_name.empty?

  if parser == RDoc::Parser::Simple and ext_name !~ /txt|rdoc/ then
    case check_modeline file_name
    when nil, 'rdoc' then # continue
    else return nil
    end
  end

  parser
rescue Errno::EACCES
end


  








  

    .check_modeline(file_name)  
  



  

    

Returns the file type from the modeline in #file_name


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 176



def self.check_modeline file_name
  line = open file_name do |io|
    io.gets
  end

  /-\*-\s*(.*?\S)\s*-\*-/ =~ line

  return nil unless type = $1

  if /;/ =~ type then
    return nil unless /(?:\s|\A)mode:\s*([^\s;]+)/i =~ type
    type = $1
  end

  return nil if /coding:/i =~ type

  type.downcase
rescue ArgumentError # invalid byte sequence, etc.
end


  








  

    .for(top_level, file_name, content, options, stats)  
  



  

    

Finds and instantiates the correct parser for the given #file_name and content.


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 200



def self.for top_level, file_name, content, options, stats
  return if binary? file_name

  parser = use_markup content

  unless parser then
    parse_name = file_name

    # If no extension, look for shebang
    if file_name !~ /\.\w+$/ && content =~ %r{\A#!(.+)} then
      shebang = $1
      case shebang
      when %r{env\s+ruby}, %r{/ruby}
        parse_name = 'dummy.rb'
      end
    end

    parser = can_parse parse_name
  end

  return unless parser

  content = remove_modeline content

  parser.new top_level, file_name, content, options, stats
rescue SystemCallError
  nil
end


  








  

    .parse_files_matching(regexp)  
  



  

    

Record which file types this parser can understand.



It is ok to call this multiple times.


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 234



def self.parse_files_matching(regexp)
  RDoc::Parser.parsers.unshift [regexp, self]
end


  








  

    .process_directive(code_object, directive, value)  
  



  

    

Processes common directives for CodeObjects for the Parser::C and Parser::Ruby parsers.



Applies directive's value to code_object, if appropriate


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 105



def self.process_directive code_object, directive, value
  warn "RDoc::Parser::process_directive is deprecated and wil be removed in RDoc 4.  Use RDoc::Markup::PreProcess#handle_directive instead" if $-w

  case directive
  when 'nodoc' then
    code_object.document_self = nil # notify nodoc
    code_object.document_children = value.downcase != 'all'
  when 'doc' then
    code_object.document_self = true
    code_object.force_documentation = true
  when 'yield', 'yields' then
    # remove parameter &block
    code_object.params.sub!(/,?\s*&\w+/, '') if code_object.params

    code_object.block_params = value
  when 'arg', 'args' then
    code_object.params = value
  end
end


  








  

    .remove_modeline(content)  
  



  

    

Removes an emacs-style modeline from the first line of the document


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 241



def self.remove_modeline content
  content.sub(/\A.*-\*-\s*(.*?\S)\s*-\*-.*\r?\n/, '')
end


  








  

    .use_markup(content)  
  



  

    

If there is a markup: parser_name comment at the front of the file, use it to determine the parser.  For example:



# markup: rdoc
# Class comment can go here

class C
end



The comment should appear as the first line of the content.



If the content contains a shebang or editor modeline the comment may appear on the second or third line.



Any comment style may be used to hide the markup comment.


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 262



def self.use_markup content
  markup = content.lines.first(3).grep(/markup:\s(\w)/) { $1 }.first

  return unless markup

  # TODO Ruby should be returned only when the filename is correct
  return RDoc::Parser::Ruby if %w[tomdoc markdown].include? markup

  markup = Regexp.escape markup

  _, selected = RDoc::Parser.parsers.find do |_, parser|
    /^#{markup}$/i =~ parser.name.sub(/.*:/, '')
  end

  selected
end


  








  

    .zip?(file)  ⇒ Boolean 
  



  

    

Checks if file is a zip file in disguise.  Signatures from www.garykessler.net/library/file_sigs.html


  





  


  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 129



def self.zip? file
  zip_signature = File.read file, 4

  zip_signature == "PK\x03\x04" or
    zip_signature == "PK\x05\x06" or
    zip_signature == "PK\x07\x08"
rescue
  false
end


  







Instance Attribute Details



  

    #file_name   (readonly)
  



  

    

The name of the file being parsed


  



  [ GitHub ]


  

  


# File 'lib/rdoc/parser.rb', line 52



attr_reader :file_name