Class: RBS::Parser
| Relationships & Source Files | |
| Namespace Children | |
|
Classes:
| |
| Inherits: | Object |
| Defined in: | lib/rbs/wasm/parser.rb, ext/rbs_extension/main.c, lib/rbs/parser_aux.rb, lib/rbs/parser/lex_result.rb, lib/rbs/parser/token.rb |
Overview
WebAssembly-backed implementation of the parser primitives.
On CRuby these come from the C extension (ext/rbs_extension/main.c). JRuby
loads this instead: it runs the parser inside WebAssembly, then rebuilds the
AST with WASM::Deserializer. rbs/parser_aux.rb layers the public
Parser API on top, exactly as it does for the C extension.
Constant Summary
-
KEYWORDS =
# File 'lib/rbs/parser_aux.rb', line 88%w( bool bot class instance interface nil self singleton top void type unchecked in out end def include extend prepend alias module attr_reader attr_writer attr_accessor public private untyped true false ).each_with_object({}) do |keyword, hash| #$ Hash[String, bot] hash[keyword] = _ = nil end
Class Method Summary
- ._lex(buffer, end_pos)
- ._parse_inline_leading_annotation(buffer, start_pos, end_pos, variables)
- ._parse_inline_trailing_annotation(buffer, start_pos, end_pos, variables)
- ._parse_method_type(buffer, start_pos, end_pos, variables, require_eof, enable_forwarding_params)
- ._parse_method_type_to_bytes(buffer, start_pos, end_pos, variables, require_eof, enable_forwarding_params)
- ._parse_signature(buffer, start_pos, end_pos, enable_forwarding_params)
- ._parse_signature_to_bytes(buffer, start_pos, end_pos, enable_forwarding_params)
- ._parse_type(buffer, start_pos, end_pos, variables, require_eof, void_allowed, self_allowed, classish_allowed)
- ._parse_type_params(buffer, start_pos, end_pos, module_type_params)
- ._parse_type_to_bytes(buffer, start_pos, end_pos, variables, require_eof, void_allowed, self_allowed, classish_allowed)
- .buffer(source)
- .byte_range(char_range, content)
- .lex(source)
- .magic_comment(buf)
- .parse_inline_leading_annotation(source, range, variables: [])
- .parse_inline_trailing_annotation(source, range, variables: [])
- .parse_method_type(source, range: nil, byte_range: 0, variables: [], require_eof: false)
- .parse_signature(source)
- .parse_type(source, range: nil, byte_range: 0, variables: [], require_eof: false, void_allowed: true, self_allowed: true, classish_allowed: true)
- .parse_type_params(source, module_type_params: true)
-
.deserialize_or_nil(bytes, buffer)
private
An empty result means the parser reached EOF immediately (
nil). -
.raise_parse_failure(buffer, status, bytes, start_pos, end_pos)
private
Raise for a status other than OK (see rbs_wasm.c).
-
.raise_parsing_error(buffer, blob)
private
Decodes the error blob written by set_error_result (rbs_wasm.c) and raises the same error the C extension would (see raise_error in main.c).
-
.validate_parser_options(enable_forwarding_params)
private
The WebAssembly entry points (rbs_wasm.c) build their parsers with the default options, so the optional syntax the C extension can enable is not reachable here.
-
.validate_position_range(buffer, start_pos, end_pos)
private
Reject the position ranges the parser cannot take, matching validate_position_range in the C extension (main.c).
-
.validate_variables(variables)
private
Reject anything that is not nil or an Array of Symbols, matching declare_type_variables in the C extension (main.c).
Class Method Details
._lex(buffer, end_pos)
[ GitHub ]._parse_inline_leading_annotation(buffer, start_pos, end_pos, variables)
[ GitHub ]# File 'lib/rbs/wasm/parser.rb', line 62
def _parse_inline_leading_annotation(buffer, start_pos, end_pos, variables) validate_position_range(buffer, start_pos, end_pos) validate_variables(variables) encoding = buffer.content.encoding.name status, bytes = WASM::Runtime.instance.parse_inline_leading_annotation(buffer.content, encoding, start_pos, end_pos, variables) raise_parse_failure(buffer, status, bytes, start_pos, end_pos) unless status == WASM::Runtime::OK deserialize_or_nil(bytes, buffer) end
._parse_inline_trailing_annotation(buffer, start_pos, end_pos, variables)
[ GitHub ]# File 'lib/rbs/wasm/parser.rb', line 72
def _parse_inline_trailing_annotation(buffer, start_pos, end_pos, variables) validate_position_range(buffer, start_pos, end_pos) validate_variables(variables) encoding = buffer.content.encoding.name status, bytes = WASM::Runtime.instance.parse_inline_trailing_annotation(buffer.content, encoding, start_pos, end_pos, variables) raise_parse_failure(buffer, status, bytes, start_pos, end_pos) unless status == WASM::Runtime::OK deserialize_or_nil(bytes, buffer) end
._parse_method_type(buffer, start_pos, end_pos, variables, require_eof, enable_forwarding_params)
[ GitHub ]# File 'lib/rbs/wasm/parser.rb', line 35
def _parse_method_type(buffer, start_pos, end_pos, variables, require_eof, enable_forwarding_params) validate_position_range(buffer, start_pos, end_pos) validate_variables(variables) (enable_forwarding_params) encoding = buffer.content.encoding.name status, bytes = WASM::Runtime.instance.parse_method_type(buffer.content, encoding, start_pos, end_pos, variables, require_eof) raise_parse_failure(buffer, status, bytes, start_pos, end_pos) unless status == WASM::Runtime::OK deserialize_or_nil(bytes, buffer) end
._parse_method_type_to_bytes(buffer, start_pos, end_pos, variables, require_eof, enable_forwarding_params)
[ GitHub ]# File 'ext/rbs_extension/main.c', line 403
static VALUE rbsparser_parse_method_type_to_bytes(VALUE self, VALUE buffer, VALUE start_pos, VALUE end_pos, VALUE variables, VALUE require_eof, VALUE enable_forwarding_params) {
VALUE string = rb_funcall(buffer, rb_intern("content"), 0);
StringValue(string);
rb_encoding *encoding = rb_enc_get(string);
rbs_parser_t *parser = alloc_parser_from_buffer_with_options(buffer, FIX2INT(start_pos), FIX2INT(end_pos), parser_options(enable_forwarding_params));
declare_type_variables(parser, variables, buffer);
struct parse_method_type_arg arg = {
.buffer = buffer,
.encoding = encoding,
.parser = parser,
.require_eof = require_eof
};
VALUE result = rb_ensure(parse_method_type_to_bytes_try, (VALUE) &arg, ensure_free_parser, (VALUE) parser);
RB_GC_GUARD(string);
return result;
}
._parse_signature(buffer, start_pos, end_pos, enable_forwarding_params)
[ GitHub ]# File 'lib/rbs/wasm/parser.rb', line 15
def _parse_signature(buffer, start_pos, end_pos, enable_forwarding_params) validate_position_range(buffer, start_pos, end_pos) (enable_forwarding_params) encoding = buffer.content.encoding.name status, bytes = WASM::Runtime.instance.parse_signature(buffer.content, encoding, start_pos, end_pos) raise_parse_failure(buffer, status, bytes, start_pos, end_pos) unless status == WASM::Runtime::OK WASM::Deserializer.deserialize(bytes, buffer) end
._parse_signature_to_bytes(buffer, start_pos, end_pos, enable_forwarding_params)
[ GitHub ]# File 'ext/rbs_extension/main.c', line 436
static VALUE rbsparser_parse_signature_to_bytes(VALUE self, VALUE buffer, VALUE start_pos, VALUE end_pos, VALUE enable_forwarding_params) {
VALUE string = rb_funcall(buffer, rb_intern("content"), 0);
StringValue(string);
rb_encoding *encoding = rb_enc_get(string);
rbs_parser_t *parser = alloc_parser_from_buffer_with_options(buffer, FIX2INT(start_pos), FIX2INT(end_pos), parser_options(enable_forwarding_params));
struct parse_signature_arg arg = {
.buffer = buffer,
.encoding = encoding,
.parser = parser,
.require_eof = false
};
VALUE result = rb_ensure(parse_signature_to_bytes_try, (VALUE) &arg, ensure_free_parser, (VALUE) parser);
RB_GC_GUARD(string);
return result;
}
._parse_type(buffer, start_pos, end_pos, variables, require_eof, void_allowed, self_allowed, classish_allowed)
[ GitHub ]# File 'lib/rbs/wasm/parser.rb', line 25
def _parse_type(buffer, start_pos, end_pos, variables, require_eof, void_allowed, self_allowed, classish_allowed) validate_position_range(buffer, start_pos, end_pos) validate_variables(variables) encoding = buffer.content.encoding.name status, bytes = WASM::Runtime.instance.parse_type(buffer.content, encoding, start_pos, end_pos, variables, require_eof, void_allowed, self_allowed, classish_allowed) raise_parse_failure(buffer, status, bytes, start_pos, end_pos) unless status == WASM::Runtime::OK deserialize_or_nil(bytes, buffer) end
._parse_type_params(buffer, start_pos, end_pos, module_type_params)
[ GitHub ]# File 'lib/rbs/wasm/parser.rb', line 46
def _parse_type_params(buffer, start_pos, end_pos, module_type_params) validate_position_range(buffer, start_pos, end_pos) encoding = buffer.content.encoding.name status, bytes = WASM::Runtime.instance.parse_type_params(buffer.content, encoding, start_pos, end_pos, module_type_params) raise_parse_failure(buffer, status, bytes, start_pos, end_pos) unless status == WASM::Runtime::OK bytes.empty? ? nil : WASM::Deserializer.deserialize_node_list(bytes, buffer) end
._parse_type_to_bytes(buffer, start_pos, end_pos, variables, require_eof, void_allowed, self_allowed, classish_allowed)
[ GitHub ]# File 'ext/rbs_extension/main.c', line 363
static VALUE rbsparser_parse_type_to_bytes(VALUE self, VALUE buffer, VALUE start_pos, VALUE end_pos, VALUE variables, VALUE require_eof, VALUE void_allowed, VALUE self_allowed, VALUE classish_allowed) {
VALUE string = rb_funcall(buffer, rb_intern("content"), 0);
StringValue(string);
rb_encoding *encoding = rb_enc_get(string);
rbs_parser_t *parser = alloc_parser_from_buffer(buffer, FIX2INT(start_pos), FIX2INT(end_pos));
declare_type_variables(parser, variables, buffer);
struct parse_type_arg arg = {
.buffer = buffer,
.encoding = encoding,
.parser = parser,
.require_eof = require_eof,
.void_allowed = void_allowed,
.self_allowed = self_allowed,
.classish_allowed = classish_allowed
};
VALUE result = rb_ensure(parse_type_to_bytes_try, (VALUE) &arg, ensure_free_parser, (VALUE) parser);
RB_GC_GUARD(string);
return result;
}
.buffer(source)
[ GitHub ].byte_range(char_range, content)
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 134
def self.byte_range(char_range, content) start_offset = char_range.begin end_offset = char_range.end start_prefix = content[0, start_offset] or raise if start_offset end_prefix = content[0, end_offset] or raise if end_offset start_prefix&.bytesize...end_prefix&.bytesize end
.deserialize_or_nil(bytes, buffer) (private)
An empty result means the parser reached EOF immediately (nil).
# File 'lib/rbs/wasm/parser.rb', line 130
def deserialize_or_nil(bytes, buffer) bytes.empty? ? nil : WASM::Deserializer.deserialize(bytes, buffer) end
.lex(source)
[ GitHub ].magic_comment(buf)
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 46
def self.magic_comment(buf) start_pos = 0 while true case when match = /\A#\s*(?<keyword>resolve-type-names)\s*(?<colon>:)\s+(?<value>true|false)$/.match(buf.content, start_pos) value = match[:value] or raise kw_offset = match.offset(:keyword) #: [Integer, Integer] colon_offset = match.offset(:colon) #: [Integer, Integer] value_offset = match.offset(:value) #: [Integer, Integer] location = Location.new(buf, kw_offset[0], value_offset[1]) location.add_required_child(:keyword, kw_offset[0]...kw_offset[1]) location.add_required_child(:colon, colon_offset[0]...colon_offset[1]) location.add_required_child(:value, value_offset[0]...value_offset[1]) return AST::Directives::ResolveTypeNames.new(value: value == "true", location: location) else return end end end
.parse_inline_leading_annotation(source, range, variables: [])
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 122
def self.parse_inline_leading_annotation(source, range, variables: []) buf = buffer(source) byte_range = byte_range(range, buf.content) _parse_inline_leading_annotation(buf, byte_range.begin || 0, byte_range.end || buf.content.bytesize, variables) end
.parse_inline_trailing_annotation(source, range, variables: [])
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 128
def self.parse_inline_trailing_annotation(source, range, variables: []) buf = buffer(source) byte_range = byte_range(range, buf.content) _parse_inline_trailing_annotation(buf, byte_range.begin || 0, byte_range.end || buf.content.bytesize, variables) end
.parse_method_type(source, range: nil, byte_range: 0, variables: [], require_eof: false)
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 14
def self.parse_method_type(source, range: nil, byte_range: 0..., variables: [], require_eof: false) buf = buffer(source) byte_range = byte_range(range, buf.content) if range _parse_method_type(buf, byte_range.begin || 0, byte_range.end || buf.content.bytesize, variables, require_eof, false) end
.parse_signature(source)
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 20
def self.parse_signature(source) buf = buffer(source) resolved = magic_comment(buf) start_pos = if resolved (resolved.location || raise).end_pos else 0 end content = buf.content dirs, decls = _parse_signature(buf, start_pos, content.bytesize, false) if resolved dirs = dirs.dup if dirs.frozen? dirs.unshift(resolved) end [buf, dirs, decls] end
.parse_type(source, range: nil, byte_range: 0, variables: [], require_eof: false, void_allowed: true, self_allowed: true, classish_allowed: true)
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 8
def self.parse_type(source, range: nil, byte_range: 0..., variables: [], require_eof: false, void_allowed: true, self_allowed: true, classish_allowed: true) buf = buffer(source) byte_range = byte_range(range, buf.content) if range _parse_type(buf, byte_range.begin || 0, byte_range.end || buf.content.bytesize, variables, require_eof, void_allowed, self_allowed, classish_allowed) end
.parse_type_params(source, module_type_params: true)
[ GitHub ]# File 'lib/rbs/parser_aux.rb', line 41
def self.parse_type_params(source, module_type_params: true) buf = buffer(source) _parse_type_params(buf, 0, buf.content.bytesize, module_type_params) end
.raise_parse_failure(buffer, status, bytes, start_pos, end_pos) (private)
Raise for a status other than OK (see rbs_wasm.c).
A negative status is about the range rather than the source text, so it comes with an empty result and an ArgumentError, as in the C extension (main.c). Starting past the end of the buffer is plain from the buffer's size and rejected above, so a start position that comes back rejected can only be one inside a character.
# File 'lib/rbs/wasm/parser.rb', line 141
def raise_parse_failure(buffer, status, bytes, start_pos, end_pos) case status when WASM::Runtime::INVALID_START_POS raise ArgumentError, "position range starts inside a character: #{start_pos}...#{end_pos}" when WASM::Runtime::INVALID_RANGE raise ArgumentError, "invalid position range: #{start_pos}...#{end_pos}" else raise_parsing_error(buffer, bytes) end end
.raise_parsing_error(buffer, blob) (private)
Decodes the error blob written by set_error_result (rbs_wasm.c) and raises the same error the C extension would (see raise_error in main.c).
# File 'lib/rbs/wasm/parser.rb', line 154
def raise_parsing_error(buffer, blob) start_char, end_char, syntax_error = blob.unpack("l<l<C") raise "Unexpected error" if syntax_error.zero? offset = 9 token_type_length = blob.unpack1("L<", offset: offset) offset += 4 token_type = blob.byteslice(offset, token_type_length).to_s.force_encoding(Encoding::UTF_8) offset += token_type_length = blob.unpack1("L<", offset: offset) offset += 4 = blob.byteslice(offset, ).to_s.force_encoding(Encoding::UTF_8) location = Location.new(buffer, start_char, end_char) raise ParsingError.new(location, , token_type) end
.validate_parser_options(enable_forwarding_params) (private)
The WebAssembly entry points (rbs_wasm.c) build their parsers with the
default options, so the optional syntax the C extension can enable is
not reachable here. The public Parser API never enables it.
# File 'lib/rbs/wasm/parser.rb', line 107
def (enable_forwarding_params) if enable_forwarding_params raise NotImplementedError, "forwarding parameter syntax is not supported by the WebAssembly parser" end end
.validate_position_range(buffer, start_pos, end_pos) (private)
Reject the position ranges the parser cannot take, matching validate_position_range in the C extension (main.c).
end_pos past the end of the buffer is fine: clamping with a large
number instead of measuring the buffer is ordinary, and the lexer stops
at the end of the input on its own.
# File 'lib/rbs/wasm/parser.rb', line 90
def validate_position_range(buffer, start_pos, end_pos) if start_pos < 0 || end_pos < 0 raise ArgumentError, "negative position range: #{start_pos}...#{end_pos}" end if start_pos > end_pos raise ArgumentError, "invalid position range: #{start_pos}...#{end_pos}" end size = buffer.content.bytesize if start_pos > size raise ArgumentError, "position range starts past the end of the buffer: #{start_pos}...#{end_pos}, buffer is #{size} bytes" end end
.validate_variables(variables) (private)
Reject anything that is not nil or an Array of Symbols, matching declare_type_variables in the C extension (main.c).
# File 'lib/rbs/wasm/parser.rb', line 115
def validate_variables(variables) return if variables.nil? unless variables.is_a?(Array) raise TypeError, "wrong argument type #{variables.class} (must be an Array of Symbols or nil)" end variables.each do |variable| unless variable.is_a?(Symbol) raise TypeError, "Type variables Array contains invalid value #{variable.inspect} of type #{variable.class} (must be an Array of Symbols or nil)" end end end