Class: RDoc::TomDoc

Relationships & Source Files
Super Chains via Extension / Inclusion / Inheritance
Class Chain: self, `Markup::Parser`
Instance Chain: self, `Markup::Parser`, `Text`
Inherits:	RDoc::Markup::Parser Object RDoc::Markup::Parser RDoc::TomDoc
Defined in:	lib/rdoc/tom_doc.rb

Overview

A parser for TomDoc based on TomDoc 1.0.0-rc1 (02adef9b5a)

The TomDoc specification can be found at tomdoc.org.

To choose TomDoc as your only default format see RDoc::Options@Saved+Options for instructions on setting up a .rdoc_options file to store your project default.

There are a few differences between this parser and the specification. A best-effort was made to follow the specification as closely as possible but some choices to deviate were made.

A future version of RDoc will warn when a MUST or MUST NOT is violated and may warn when a SHOULD or SHOULD NOT is violated. RDoc will always try to emit documentation even if given invalid TomDoc.

Here are some implementation choices this parser currently makes:

This parser allows rdoc-style inline markup but you should not depended on it.

This parser allows a space between the comment and the method body.

This parser does not require the default value to be described for an optional argument.

This parser does not examine the order of sections. An Examples section may precede the Arguments section.

This class is documented in TomDoc format. Since this is a subclass of the RDoc markup parser there isn’t much to see here, unfortunately.

Constant Summary

`Text` - Included

MARKUP_FORMAT, SPACE_SEPARATED_LETTER_CLASS, TO_HTML_CHARACTERS

`Markup::Parser` - Inherited

LIST_TOKENS

Class Method Summary

.new ⇒ TomDoc constructor

Public: Creates a new TomDoc parser.
.parse(text)

Public: Parses TomDoc from text.
.signature(comment)

Internal: Extracts the Signature section’s method signature.
.add_post_processor Internal use only

Internal: Adds a post-processor which sets the RDoc section based on the comment’s status.

`Markup::Parser` - Inherited

.new	Creates a new `Parser`.
.parse	Parses `str` into a Document.
.tokenize	Returns a token stream for `str`, for testing.

Instance Attribute Summary

#tokens readonly

Internal: Token accessor.

`Markup::Parser` - Inherited

#debug	Enables display of debugging information.
#tokens	Token accessor.

`Text` - Included

#language

The language for this text.

Instance Method Summary

#build_heading(level)

Internal: Builds a heading from the token stream.
#build_paragraph(margin)

Internal: Builds a paragraph from the token stream.
#build_verbatim(margin)

Internal: Builds a verbatim from the token stream.
#tokenize(text)

Internal: Turns text into an Array of tokens.
#parse_text(parent, indent) Internal use only

Detects a section change to “Returns” and adds a heading.

`Markup::Parser` - Inherited

#build_heading	Builds a Heading of `level`
#build_list	Builds a List flush to `margin`
#build_paragraph	Builds a Paragraph that is flush to `margin`
#build_verbatim	Builds a Verbatim that is indented from `margin`.
#get	Pulls the next token from the stream.
#parse	Parses the tokens into an array of `RDoc::Markup::XXX` objects, and appends them to the passed `parent` `RDoc::Markup::YYY` object.
#peek_token	Returns the next token on the stream without modifying the stream.
#setup_scanner	Creates the StringScanner.
#skip	Skips the next token if its type is `token_type`.
#tokenize	Turns text `input` into a stream of tokens.
#unget	Returns the current token to the token stream.
#parse_text	Small hook that is overridden by `TomDoc`

`Text` - Included

#expand_tabs	Expands tab characters in `text` to eight spaces.
#flush_left	Flush `text` left based on the shortest line.
#markup	Convert a string in markup format into HTML.
#normalize_comment	Strips hashes, expands tabs then flushes `text` to the left.
#parse	Normalizes `text` then builds a `Markup::Document` from it.
#snippet	The first `limit` characters of `text` as HTML.
#strip_hashes	Strips leading # characters from `text`
#strip_newlines	Strips leading and trailing n characters from `text`
#strip_stars	Strips /* */ style comments.
#to_html	Converts ampersand, dashes, ellipsis, quotes, copyright and registered trademark symbols in `text` to properly encoded characters.
#wrap	Wraps `txt` to `line_len`

Constructor Details

.new ⇒ `TomDoc`

Public: Creates a new TomDoc parser. See also Markup.parse

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 124


def initialize
  super

  @section      = nil
  @seen_returns = false
end

Class Method Details

.add_post_processor

This method is for internal use only.

Internal: Adds a post-processor which sets the RDoc section based on the comment’s status.

Returns nothing.

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 47


def self.add_post_processor # :nodoc:
  RDoc::Markup::PreProcess.post_process do |comment, code_object|
    next unless code_object and
                RDoc::Comment === comment and comment.format == 'tomdoc'

    comment.text.gsub!(/(\A\s*# )(Public|Internal|Deprecated):\s+/) do
      section = code_object.add_section $2
      code_object.temporary_section = section

      $1
    end
  end
end

.parse(text)

Public: Parses TomDoc from text

text - A String containing TomDoc-format text.

Examples

RDoc::TomDoc.parse <<-TOMDOC
This method does some things

Returns nothing.
TOMDOC
# => #<RDoc::Markup::Document:0xXXX @parts=[...], @file=nil>

Returns an Markup::Document representing the TomDoc format.

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 78


def self.parse(text)
  parser = new

  parser.tokenize text
  doc = RDoc::Markup::Document.new
  parser.parse doc
  doc
end

.signature(comment)

Internal: Extracts the Signature section’s method signature

comment - An Comment that will be parsed and have the signature

extracted

Returns a String containing the signature and nil if not

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 94


def self.signature(comment)
  return unless comment.tomdoc?

  document = comment.parse

  signature = nil
  found_heading = false
  found_signature = false

  document.parts.delete_if do |part|
    next false if found_signature

    found_heading ||=
      RDoc::Markup::Heading === part && part.text == 'Signature'

    next false unless found_heading

    next true if RDoc::Markup::BlankLine === part

    if RDoc::Markup::Verbatim === part then
      signature = part
      found_signature = true
    end
  end

  signature and signature.text
end

Instance Attribute Details

#tokens (readonly)

Internal: Token accessor

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 40


attr_reader :tokens

Instance Method Details

#build_heading(level)

Internal: Builds a heading from the token stream

level - The level of heading to create

Returns an RDoc::Markup::Heading

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 137


def build_heading(level)
  heading = super

  @section = heading.text

  heading
end

#build_paragraph(margin)

Internal: Builds a paragraph from the token stream

margin - Unused

Returns an Markup::Paragraph.

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 167


def build_paragraph(margin)
  p :paragraph_start => margin if @debug

  paragraph = RDoc::Markup::Paragraph.new

  until @tokens.empty? do
    type, data, = get

    case type
    when :TEXT then
      @section = 'Returns' if data =~ /\A(Returns|Raises)/

      paragraph << data
    when :NEWLINE then
      if :TEXT == peek_token[0] then
        # Lines beginning with 'Raises' in the Returns section should not be
        # treated as multiline text
        if 'Returns' == @section and
          peek_token[1].start_with?('Raises') then
          break
        else
          paragraph << ' '
        end
      else
        break
      end
    else
      unget
      break
    end
  end

  p :paragraph_end => margin if @debug

  paragraph
end

#build_verbatim(margin)

Internal: Builds a verbatim from the token stream. A verbatim in the Examples section will be marked as in Ruby format.

margin - The indentation from the margin for lines that belong to this

verbatim section.

Returns an Markup::Verbatim

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 153


def build_verbatim(margin)
  verbatim = super

  verbatim.format = :ruby if @section == 'Examples'

  verbatim
end

#parse_text(parent, indent)

This method is for internal use only.

Detects a section change to “Returns” and adds a heading

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 207


def parse_text(parent, indent) # :nodoc:
  paragraph = build_paragraph indent

  if false == @seen_returns and 'Returns' == @section then
    @seen_returns = true
    parent << RDoc::Markup::Heading.new(3, 'Returns')
    parent << RDoc::Markup::BlankLine.new
  end

  parent << paragraph
end

#tokenize(text)

Internal: Turns text into an Array of tokens

text - A String containing TomDoc-format text.

Returns self.

[ GitHub ]

# File 'lib/rdoc/tom_doc.rb', line 225


def tokenize(text)
  text = text.sub(/\A(Public|Internal|Deprecated):\s+/, '')

  setup_scanner text

  until @s.eos? do
    pos = @s.pos

    # leading spaces will be reflected by the column of the next token
    # the only thing we loose are trailing spaces at the end of the file
    next if @s.scan(/ +/)

    @tokens << case
               when @s.scan(/\r?\n/) then
                 token = [:NEWLINE, @s.matched, *pos]
                 @s.newline!
                 token
               when @s.scan(/(Examples|Signature)$/) then
                 @tokens << [:HEADER, 3, *pos]

                 [:TEXT, @s[1], *pos]
               when @s.scan(/([:\w][\w\[\]]*)[ ]+- /) then
                 [:NOTE, @s[1], *pos]
               else
                 @s.scan(/.*/)
                 [:TEXT, @s.matched.sub(/\r$/, ''), *pos]
               end
  end

  self
end

Class: RDoc::TomDoc

Overview

Constant Summary

Text - Included

Markup::Parser - Inherited

Class Method Summary

Markup::Parser - Inherited

Instance Attribute Summary

Markup::Parser - Inherited

Text - Included

Instance Method Summary

Markup::Parser - Inherited

Text - Included

Constructor Details

.new ⇒ TomDoc

Class Method Details

.add_post_processor

.parse(text)

.signature(comment)

Instance Attribute Details

#tokens (readonly)

Instance Method Details

#build_heading(level)

#build_paragraph(margin)

#build_verbatim(margin)

#parse_text(parent, indent)

#tokenize(text)

`Text` - Included

`Markup::Parser` - Inherited

`Markup::Parser` - Inherited

`Markup::Parser` - Inherited

`Text` - Included

`Markup::Parser` - Inherited

`Text` - Included

.new ⇒ `TomDoc`