Class: YARD::DocstringParser

Relationships & Source Files
Inherits:	Object
Defined in:	lib/yard/docstring_parser.rb

Overview

Parses text and creates a Docstring object to represent documentation for a CodeObjects::Base. To create a new docstring, you should initialize the parser and call #parse followed by #to_docstring.

Subclassing Notes

The DocstringParser can be subclassed and substituted during parsing by setting the Docstring.default_parser attribute with the name of the subclass. This allows developers to change the way docstrings are parsed, allowing for completely different docstring syntaxes.

Examples:

Creating a Docstring with a DocstringParser

DocstringParser.new.parse("text here").to_docstring

Creating a Custom DocstringParser

# Parses docstrings backwards!
class ReverseDocstringParser
  def parse_content(content)
    super(content.reverse)
  end
end

# Set the parser as default when parsing
YARD::Docstring.default_parser = ReverseDocstringParser

Constant Summary

META_MATCH =

The regular expression to match the tag syntax
Since:
- 0.8.0
# File 'lib/yard/docstring_parser.rb', line 73
```
/^@(!)?((?:\w\.?)+)(?:\s+(.*))?$/i
```

Creation and Conversion Methods

.new(library = Tags::Library.instance) ⇒ DocstringParser constructor

Creates a new parser to parse docstring data.
#to_docstring ⇒ Docstring

Parsing Methods

#parse(content, object = nil, handler = nil) ⇒ self

Parses all content and returns itself.
#parse_content(content)

Parses a given block of text.

Parser Callback Methods

#call_after_parse_callbacks private

Calls all .after_parse callbacks.
#call_directives_after_parse private

Calls the Tags::Directive#after_parse callback on all the created directives.
#post_process ⇒ void

Call post processing callbacks on parser.

Tag Manipulation Methods

.after_parse {|parser| ... } ⇒ void

Creates a callback that is called after a docstring is successfully parsed.
.after_parse_callbacks ⇒ Array<Proc>
#create_directive(tag_name, tag_buf) ⇒ Tags::Directive

Creates a new directive using the registered #library
#create_ref_tag(tag_name, name, object_name)

Creates a Tags::RefTag
#create_tag(tag_name, tag_buf = '') ⇒ Tags::Tag, Tags::RefTag

Creates a tag from the tag factory.
#detect_reference(content) private
#namespace private
#tag_is_directive?(tag_name) ⇒ Boolean

Backward compatibility to detect old tags that should be specified as directives in 0.8 and onward.

Instance Attribute Summary

#directives ⇒ Array<Tags::Directive> rw
#handler ⇒ Handlers::Base^? rw
#library ⇒ Tags::Library rw
#object ⇒ CodeObjects::Base^? rw
#raw_text ⇒ String rw
#reference ⇒ CodeObjects::Base^? rw
#state ⇒ OpenStruct rw
#tags ⇒ Array<Tags::Tag> rw
#text ⇒ String rw

Constructor Details

.new(library = Tags::Library.instance) ⇒ `DocstringParser`

Creates a new parser to parse docstring data

Parameters:

library (Tags::Library) (defaults to: Tags::Library.instance) —
a tag library for recognizing tags.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 81


def initialize(library = Tags::Library.instance)
  @text = ""
  @raw_text = ""
  @tags = []
  @directives = []
  @library = library
  @object = nil
  @reference = nil
  @handler = nil
  @state = OpenStruct.new
end

Class Method Details

.after_parse {|parser| ... } ⇒ `void`

This method returns an undefined value.

Creates a callback that is called after a docstring is successfully parsed. Use this method to perform sanity checks on a docstring's tag data, or add any extra tags automatically to a docstring.

Yields:

(parser) —
a block to be called after a docstring is parsed

Yield Parameters:

parser (DocstringParser) —
the docstring parser object with all directives and tags created.

Yield Returns:

(void)

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 266


def self.after_parse(&block)
  after_parse_callbacks << block
end

.after_parse_callbacks ⇒ Array<`Proc`>

Returns:

(Array<Proc>) —
the .after_parse callback proc objects

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 271


def self.after_parse_callbacks
  @after_parse_callbacks ||= []
end

Instance Attribute Details

#directives ⇒ Array<Tags::Directive> (rw)

Returns:

(Array<Tags::Directive>) —
a list of directives identified by the parser. This list will not be passed on to the Docstring object.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 45


attr_accessor :directives

#handler ⇒ Handlers::Base^? (rw)

Returns:

(Handlers::Base, nil) —
the handler parsing this docstring. May be nil if this docstring parser is not initialized through

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 66


attr_accessor :handler

#library ⇒ Tags::Library (rw)

Returns:

(Tags::Library) —
the tag library being used to identify registered tags in the docstring.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 70


attr_accessor :library

#object ⇒ CodeObjects::Base^? (rw)

Returns:

(CodeObjects::Base, nil) —
the object associated with the docstring being parsed. May be nil if the docstring is not attached to any object.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 56


attr_accessor :object

#raw_text ⇒ String (rw)

Returns:

(String) —
the complete input string to the parser.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 36


attr_accessor :raw_text

#reference ⇒ CodeObjects::Base^? (rw)

Returns:

(CodeObjects::Base, nil) —
the object referenced by the docstring being parsed. May be nil if the docstring doesn't refer to any object.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 61


attr_accessor :reference

#state ⇒ `OpenStruct` (rw)

Returns:

(OpenStruct) —
any arbitrary state to be passed between tags during parsing. Mainly used by directives to coordinate behaviour (so that directives can be aware of other directives used in a docstring).

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 51


attr_accessor :state

#tags ⇒ Array<Tags::Tag> (rw)

Returns:

(Array<Tags::Tag>) —
the list of meta-data tags identified by the parser

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 40


attr_accessor :tags

#text ⇒ String (rw)

Returns:

(String) —
the parsed text portion of the docstring, with tags removed.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 33


attr_accessor :text

Instance Method Details

#call_after_parse_callbacks (private)

Calls all .after_parse callbacks

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 324


def call_after_parse_callbacks
  self.class.after_parse_callbacks.each do |cb|
    cb.call(self)
  end
end

#call_directives_after_parse (private)

Calls the Tags::Directive#after_parse callback on all the created directives.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 319


def call_directives_after_parse
  directives.each(&:after_parse)
end

#create_directive(tag_name, tag_buf) ⇒ Tags::Directive

Creates a new directive using the registered #library

Returns:

(Tags::Directive) —
the directive object that is created

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 232


def create_directive(tag_name, tag_buf)
  if library.has_directive?(tag_name)
    dir = library.directive_create(tag_name, tag_buf, self)
    if dir.is_a?(Tags::Directive)
      @directives << dir
      dir
    end
  else
    log.warn "Unknown directive @!#{tag_name}" +
             (object ? " in file `#{object.file}` near line #{object.line}" : "")
    nil
  end
rescue Tags::TagFormatError
  log.warn "Invalid directive format for @!#{tag_name}" +
           (object ? " in file `#{object.file}` near line #{object.line}" : "")
  nil
end

#create_ref_tag(tag_name, name, object_name)

Creates a Tags::RefTag

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 226


def create_ref_tag(tag_name, name, object_name)
  @tags << Tags::RefTagList.new(tag_name, P(object, object_name), name)
end

#create_tag(tag_name, tag_buf = '') ⇒ Tags::Tag, Tags::RefTag

Creates a tag from the tag factory.

To add an already created tag object, append it to #tags.

Parameters:

tag_name (String) —
the tag name
tag_buf (String) (defaults to: '') —
the text attached to the tag with newlines removed.

Returns:

(Tags::Tag, Tags::RefTag) —
a tag

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 209


def create_tag(tag_name, tag_buf = '')
  if tag_buf =~ /\A\s*(?:(\S+)\s+)?\(\s*see\s+(\S+)\s*\)\s*\Z/
    return create_ref_tag(tag_name, $1, $2)
  end

  if library.has_tag?(tag_name)
    @tags += [library.tag_create(tag_name, tag_buf)].flatten
  else
    log.warn "Unknown tag @#{tag_name}" +
             (object ? " in file `#{object.file}` near line #{object.line}" : "")
  end
rescue Tags::TagFormatError
  log.warn "Invalid tag format for @#{tag_name}" +
           (object ? " in file `#{object.file}` near line #{object.line}" : "")
end

#detect_reference(content) (private)

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 305


def detect_reference(content)
  if content =~ /\A\s*\(see (\S+)\s*\)(?:\s|$)/
    path = $1
    extra = $'
    [CodeObjects::Proxy.new(namespace, path), extra]
  else
    [nil, content]
  end
end

#namespace (private)

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 301


def namespace
  object && object.namespace
end

#parse(content, object = nil, handler = nil) ⇒ `self`

Parses all content and returns itself.

Parameters:

content (String) —
the docstring text to parse
object (CodeObjects::Base) (defaults to: nil) —
the object that the docstring is attached to. Will be passed to directives to act on this object.
handler (Handlers::Base, nil) (defaults to: nil) —
the handler object that is parsing this object. May be nil if this parser is not being called from a Parser::SourceParser context.

Returns:

(self) —
the parser object. To get the docstring, call #to_docstring.

#parse_content(content)

Note:

Subclasses can override this method to perform custom parsing of content data.

Parses a given block of text.

Parameters:

content (String) —
the content to parse

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 129


def parse_content(content)
  content = content.split(/\r?\n/) if content.is_a?(String)
  return '' if !content || content.empty?
  docstring = String.new("")

  indent = content.first[/^\s*/].length
  last_indent = 0
  orig_indent = 0
  directive = false
  last_line = ""
  tag_name = nil
  tag_buf = []

  (content + ['']).each_with_index do |line, index|
    indent = line[/^\s*/].length
    empty = (line =~ /^\s*$/ ? true : false)
    done = content.size == index

    if tag_name && (((indent < orig_indent && !empty) || done ||
        (indent == 0 && !empty)) || (indent <= last_indent && line =~ META_MATCH))
      buf = tag_buf.join("\n")
      if directive || tag_is_directive?(tag_name)
        directive = create_directive(tag_name, buf)
        if directive
          docstring << parse_content(directive.expanded_text).chomp
        end
      else
        create_tag(tag_name, buf)
      end
      tag_name = nil
      tag_buf = []
      directive = false
      orig_indent = 0
    end

    # Found a meta tag
    if line =~ META_MATCH
      directive = $1
      tag_name = $2
      tag_buf = [($3 || '')]
    elsif tag_name && indent >= orig_indent && !empty
      orig_indent = indent if orig_indent == 0
      # Extra data added to the tag on the next line
      last_empty = last_line =~ /^[ \t]*$/ ? true : false

      tag_buf << '' if last_empty
      tag_buf << line.gsub(/^[ \t]{#{orig_indent}}/, '')
    elsif !tag_name
      # Regular docstring text
      docstring << line
      docstring << "\n"
    end

    last_indent = indent
    last_line = line
  end

  docstring
end

#post_process ⇒ `void`

This method returns an undefined value.

Call post processing callbacks on parser. This is called implicitly by parser. Use this when manually configuring a Docstring object.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 196


def post_process
  call_after_parse_callbacks
end

#tag_is_directive?(tag_name) ⇒ `Boolean`

Backward compatibility to detect old tags that should be specified as directives in 0.8 and onward.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 252


def tag_is_directive?(tag_name)
  list = %w(attribute endgroup group macro method scope visibility)
  list.include?(tag_name)
end

#to_docstring ⇒ Docstring

Returns:

(Docstring) —
translates parsed text into a Docstring object.

Since:

0.8.0

[ GitHub ]

# File 'lib/yard/docstring_parser.rb', line 95


def to_docstring
  Docstring.new!(text, tags, object, raw_text, reference)
end

Class: YARD::DocstringParser

Overview

Subclassing Notes

Constant Summary

Creation and Conversion Methods

Parsing Methods

Parser Callback Methods

Tag Manipulation Methods

Instance Attribute Summary

Constructor Details

.new(library = Tags::Library.instance) ⇒ DocstringParser

Class Method Details

.after_parse {|parser| ... } ⇒ void

.after_parse_callbacks ⇒ Array<Proc>

Instance Attribute Details

#directives ⇒ Array<Tags::Directive> (rw)

#handler ⇒ Handlers::Base? (rw)

#library ⇒ Tags::Library (rw)

#object ⇒ CodeObjects::Base? (rw)

#raw_text ⇒ String (rw)

#reference ⇒ CodeObjects::Base? (rw)

#state ⇒ OpenStruct (rw)

#tags ⇒ Array<Tags::Tag> (rw)

#text ⇒ String (rw)

Instance Method Details

#call_after_parse_callbacks (private)

#call_directives_after_parse (private)

#create_directive(tag_name, tag_buf) ⇒ Tags::Directive

#create_ref_tag(tag_name, name, object_name)

#create_tag(tag_name, tag_buf = '') ⇒ Tags::Tag, Tags::RefTag

#detect_reference(content) (private)

#namespace (private)

#parse(content, object = nil, handler = nil) ⇒ self

#parse_content(content)

#post_process ⇒ void

#tag_is_directive?(tag_name) ⇒ Boolean

#to_docstring ⇒ Docstring

.new(library = Tags::Library.instance) ⇒ `DocstringParser`

.after_parse {|parser| ... } ⇒ `void`

.after_parse_callbacks ⇒ Array<`Proc`>

#handler ⇒ Handlers::Base^? (rw)

#object ⇒ CodeObjects::Base^? (rw)

#reference ⇒ CodeObjects::Base^? (rw)

#state ⇒ `OpenStruct` (rw)

#parse(content, object = nil, handler = nil) ⇒ `self`

#post_process ⇒ `void`

#tag_is_directive?(tag_name) ⇒ `Boolean`