123456789_123456789_123456789_123456789_123456789_

Class: RDoc::TomDoc

Relationships & Source Files
Super Chains via Extension / Inclusion / Inheritance
Class Chain:
Instance Chain:
Inherits: RDoc::Markup::Parser
Defined in: lib/rdoc/tom_doc.rb

Overview

A parser for TomDoc based on TomDoc 1.0.0-rc1 (02adef9b5a)

The TomDoc specification can be found at:

tomdoc.org

The latest version of the TomDoc specification can be found at:

github.com/mojombo/tomdoc/blob/master/tomdoc.md

To choose TomDoc as your only default format see RDoc::Options@Saved+Options for instructions on setting up a .rdoc_options file to store your project default.

There are a few differences between this parser and the specification. A best-effort was made to follow the specification as closely as possible but some choices to deviate were made.

A future version of RDoc will warn when a MUST or MUST NOT is violated and may warn when a SHOULD or SHOULD NOT is violated. RDoc will always try to emit documentation even if given invalid TomDoc.

Here are some implementation choices this parser currently makes:

This parser allows rdoc-style inline markup but you should not depended on it.

This parser allows a space between the comment and the method body.

This parser does not require the default value to be described for an optional argument.

This parser does not examine the order of sections. An Examples section may precede the Arguments section.

This class is documented in TomDoc format. Since this is a subclass of the RDoc markup parser there isn’t much to see here, unfortunately.

Constant Summary

Text - Included

MARKUP_FORMAT, TO_HTML_CHARACTERS

Markup::Parser - Inherited

LIST_TOKENS

Class Method Summary

Markup::Parser - Inherited

.new

Creates a new Parser.

.parse

Parses str into a Document.

.tokenize

Returns a token stream for str, for testing.

Instance Attribute Summary

  • #tokens readonly

    Internal: Token accessor.

Markup::Parser - Inherited

#debug

Enables display of debugging information.

#tokens

Token accessor.

Instance Method Summary

Markup::Parser - Inherited

#build_heading

Builds a Heading of level

#build_list

Builds a List flush to margin

#build_paragraph

Builds a Paragraph that is flush to margin

#build_verbatim

Builds a Verbatim that is indented from margin.

#char_pos

The character offset for the input string at the given byte_offset

#get

Pulls the next token from the stream.

#parse

Parses the tokens into an array of RDoc::Markup::XXX objects, and appends them to the passed parent RDoc::Markup::YYY object.

#peek_token

Returns the next token on the stream without modifying the stream.

#setup_scanner

Creates the StringScanner.

#skip

Skips the next token if its type is token_type.

#token_pos

Calculates the column (by character) and line of the current token based on byte_offset.

#tokenize

Turns text input into a stream of tokens.

#unget

Returns the current token to the token stream.

#parse_text

Small hook that is overridden by TomDoc

Text - Included

#expand_tabs

Expands tab characters in text to eight spaces.

#flush_left

Flush text left based on the shortest line.

#markup

Convert a string in markup format into HTML.

#normalize_comment

Strips hashes, expands tabs then flushes text to the left.

#parse

Normalizes text then builds a Markup::Document from it.

#snippet

The first limit characters of text as HTML.

#strip_hashes

Strips leading # characters from text

#strip_newlines

Strips leading and trailing n characters from text

#strip_stars

Strips /* */ style comments.

#to_html

Converts ampersand, dashes, ellipsis, quotes, copyright and registered trademark symbols in text to properly encoded characters.

#wrap

Wraps txt to line_len

Constructor Details

.newTomDoc

Public: Creates a new TomDoc parser. See also Markup.parse

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 130

def initialize
  super

  @section      = nil
  @seen_returns = false
end

Class Method Details

.add_post_processor

This method is for internal use only.

Internal: Adds a post-processor which sets the RDoc section based on the comment’s status.

Returns nothing.

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 53

def self.add_post_processor # :nodoc:
  RDoc::Markup::PreProcess.post_process do |comment, code_object|
    next unless code_object and
                RDoc::Comment === comment and comment.format == 'tomdoc'

    comment.text.gsub!(/(\A\s*# )(Public|Internal|Deprecated):\s+/) do
      section = code_object.add_section $2
      code_object.temporary_section = section

      $1
    end
  end
end

.parse(text)

Public: Parses TomDoc from text

text - A String containing TomDoc-format text.

Examples

RDoc::TomDoc.parse <<-TOMDOC
This method does some things

Returns nothing.
TOMDOC
# => #<RDoc::Markup::Document:0xXXX @parts=[...], @file=nil>

Returns an Markup::Document representing the TomDoc format.

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 84

def self.parse text
  parser = new

  parser.tokenize text
  doc = RDoc::Markup::Document.new
  parser.parse doc
  doc
end

.signature(comment)

Internal: Extracts the Signature section’s method signature

comment - An Comment that will be parsed and have the signature

extracted

Returns a String containing the signature and nil if not

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 100

def self.signature comment
  return unless comment.tomdoc?

  document = comment.parse

  signature = nil
  found_heading = false
  found_signature = false

  document.parts.delete_if do |part|
    next false if found_signature

    found_heading ||=
      RDoc::Markup::Heading === part && part.text == 'Signature'

    next false unless found_heading

    next true if RDoc::Markup::BlankLine === part

    if RDoc::Markup::Verbatim === part then
      signature = part
      found_signature = true
    end
  end

  signature and signature.text
end

Instance Attribute Details

#tokens (readonly)

Internal: Token accessor

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 46

attr_reader :tokens

Instance Method Details

#build_heading(level)

Internal: Builds a heading from the token stream

level - The level of heading to create

Returns an RDoc::Markup::Heading

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 143

def build_heading level
  heading = super

  @section = heading.text

  heading
end

#build_paragraph(margin)

Internal: Builds a paragraph from the token stream

margin - Unused

Returns an Markup::Paragraph.

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 173

def build_paragraph margin
  p :paragraph_start => margin if @debug

  paragraph = RDoc::Markup::Paragraph.new

  until @tokens.empty? do
    type, data, = get

    case type
    when :TEXT then
      @section = 'Returns' if data =~ /\A(Returns|Raises)/

      paragraph << data
    when :NEWLINE then
      if :TEXT == peek_token[0] then
        # Lines beginning with 'Raises' in the Returns section should not be
        # treated as multiline text
        if 'Returns' == @section and
          peek_token[1].start_with?('Raises') then
          break
        else
          paragraph << ' '
        end
      else
        break
      end
    else
      unget
      break
    end
  end

  p :paragraph_end => margin if @debug

  paragraph
end

#build_verbatim(margin)

Internal: Builds a verbatim from the token stream. A verbatim in the Examples section will be marked as in Ruby format.

margin - The indentation from the margin for lines that belong to this

verbatim section.

Returns an Markup::Verbatim

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 159

def build_verbatim margin
  verbatim = super

  verbatim.format = :ruby if @section == 'Examples'

  verbatim
end

#parse_text(parent, indent)

This method is for internal use only.

Detects a section change to “Returns” and adds a heading

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 213

def parse_text parent, indent # :nodoc:
  paragraph = build_paragraph indent

  if false == @seen_returns and 'Returns' == @section then
    @seen_returns = true
    parent << RDoc::Markup::Heading.new(3, 'Returns')
    parent << RDoc::Markup::BlankLine.new
  end

  parent << paragraph
end

#tokenize(text)

Internal: Turns text into an Array of tokens

text - A String containing TomDoc-format text.

Returns self.

[ GitHub ]

  
# File 'lib/rdoc/tom_doc.rb', line 231

def tokenize text
  text = text.sub(/\A(Public|Internal|Deprecated):\s+/, '')

  setup_scanner text

  until @s.eos? do
    pos = @s.pos

    # leading spaces will be reflected by the column of the next token
    # the only thing we loose are trailing spaces at the end of the file
    next if @s.scan(/ +/)

    @tokens << case
               when @s.scan(/\r?\n/) then
                 token = [:NEWLINE, @s.matched, *token_pos(pos)]
                 @line_pos = char_pos @s.pos
                 @line += 1
                 token
               when @s.scan(/(Examples|Signature)$/) then
                 @tokens << [:HEADER, 3, *token_pos(pos)]

                 [:TEXT, @s[1], *token_pos(pos)]
               when @s.scan(/([:\w][\w\[\]]*)[ ]+- /) then
                 [:NOTE, @s[1], *token_pos(pos)]
               else
                 @s.scan(/.*/)
                 [:TEXT, @s.matched.sub(/\r$/, ''), *token_pos(pos)]
               end
  end

  self
end