Repository navigation
Java parser: quote the same error snippet as the C parser - #1131
Open
hasannadeem wants to merge 1 commit into
Open
hasannadeem wants to merge 1 commit into
hasannadeem wants to merge 1 commit into
Conversation
The C parser's build_parse_error_message quotes the source up to the
next whitespace, at most 32 bytes, and backs off when the limit splits
a multibyte character. The Java parser quoted 32 raw bytes, so
JSON.parse("あ" * 23)
raised "unexpected token at 'ああああああああああ\xE3\x81'", a message
with invalid UTF-8, and 'Infinity something' was quoted whole.
Build the fragment the same way, show EOF at the end of the input, and
use the C parser's "unexpected character:" and "unexpected token at end
of stream" messages for an invalid value and for trailing garbage.
JRuby errors have carried positions for a while, so this lets
test_parse_error_snippet and test_parse_error_snippet_multibyte_boundaries
run on JRuby, along with three assertions in test_error_messages.
Part of ruby#1123
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #1123.
The Java parser built parse error messages from 32 raw bytes of the source, so it could split a multibyte character and produce a message with invalid UTF-8:
It also quoted past whitespace (
'Infinity something') and used "unexpected token at" for every error.This ports
build_parse_error_messagefromparser.c: quote up to the next whitespace or NUL, at most 32 bytes, back off when the limit splits a multibyte character, and showEOFat the end of the input. An invalid value now raises "unexpected character:" and trailing garbage "unexpected token at end of stream", as in C. Other errors use "unexpected token" without the "at", which is how C words them and whatlib/json.rbdocuments.JRuby errors already carry the right line and column, so the "JRuby errors don't contain positions" omits were out of date. With this change, these now run on JRuby:
test_parse_error_snippettest_parse_error_snippet_multibyte_boundariesInfinity,fooandNaNassertions inJSONExtParserTest#test_error_messagesAll three fail on JRuby without the
Parser.javachange. Tested locally on JRuby 10.1.3.0 (689 tests, 0 failures, 51 omissions, down from 53) and CRuby 3.4.4 (694 tests, 0 failures).Not in this PR: C still has more specific messages for some errors where Java says "unexpected token", such as
invalid number: '-x]'andexpected object key, got 'x}'. The snippets and positions now match; only the wording differs. I can do those next if that's welcome.