JSONNumber - optional decimal
On Jun 8, 2010, at 8:46 PM, Garrett Smith wrote:
Today I looked for a good json regexp tester and finding nothing, decided to write one.
The strategy that occurred to me was to first define a regex for the literal components (ES5 lumps literal value into the JSONValue alongside JSONObject and JSONArray). That way, I could reuse the literal components to define JSONObject and JSONArray.
In the process, I noticed that the grammar defined for JSONNumber is different than that defiend by json2.js, which I had just looked at, and which has similar numeric parsing as seen in The GOod parts. That is, it allows DecimalLiteral, which includes this production:
DecimalIntegerLiteral .
In contrast, JSONNumber does not allow that.
JSONNumber :: -opt DecimalIntegerLiteral JSONFractionopt ExponentPartopt
Am I misreading the spec, or is "1." not valid JSON?
Next step was to see what the browsers do.
Mozilla and IE accept it, while Webkit and Opera throws an error.
JSON.parse("1.")
Firefox 3.6, IE8, Besen r27: 1
Safari, Opera SyntaxError
IF anyone has a correct JSON parser, I would appreciate it. Also, are there any good test suites for JSON?
I spent quite a bit of time ensuring JSC's JSON parser exactly matched the spec grammar, and hence exhibits this behaviour so I believe the parser in JSC is "correct" -- if you're interested the code can be found at (note that it's under an MIT style license) trac.webkit.org/browser/trunk/JavaScriptCore/runtime/LiteralParser.cpp At the time I recall talking a bit with Hallvord Steen of Opera, and he was working on a testsuite of some sort but I can no longer recall the url.
That said I think allowing '1.' (etc) makes sense as it's fairly standard across multiple programming languages, and I am unaware of any specific reason for disallowing it.
In the long term I don't see changing the grammar to allow a trailing period as being harmful as it's a relaxation. In the short term vendors that follow the spec may fail to parse content :-(
On 6/8/10, Oliver Hunt <oliver at apple.com> wrote:
On Jun 8, 2010, at 8:46 PM, Garrett Smith wrote:
[...]
IF anyone has a correct JSON parser, I would appreciate it. Also, are there any good test suites for JSON?
I spent quite a bit of time ensuring JSC's JSON parser exactly matched the spec grammar, and hence exhibits this behaviour so I believe the parser in JSC is "correct" -- if you're interested the code can be found at (note that it's under an MIT style license) trac.webkit.org/browser/trunk/JavaScriptCore/runtime/LiteralParser.cpp
I see, so \d. is no good.
I was more looking for a few regexps to fill in a tight function for the FAQ. While your C++ looks clean, transliterating that would surely be a no-go for the FAQ. Strategy-outline:
// TODO: define validJSONExp. var parseJSON = NATIVE_JSON_PARSE_SUPPORT ? function(responseText) { return JSON.parse(responseText); } : function(responseText) { if(validJSONExp.test(responseText)) { return new Function("return(" + responseText + ")")(); } else { throw SyntaxError("JSON parse error"); } };
The goal is to essentially match exactly what JSON parse would do for 1 argument and for the FAQ, the code would be tighter to use a RegExp. I'll figure it out.
At the time I recall talking a bit with Hallvord Steen of Opera, and he was working on a testsuite of some sort but I can no longer recall the url.
OK. Good to know.
That said I think allowing '1.' (etc) makes sense as it's fairly standard across multiple programming languages, and I am unaware of any specific reason for disallowing it.
It seems like it would have made sense. Now, well, no, not now; the spec is released and implemented like that in at least two implementations.
In the long term I don't see changing the grammar to allow a trailing period as being harmful as it's a relaxation. In the short term vendors that follow the spec may fail to parse content :-(
Changing it now would probably just create more confusion.
On 11:59 AM, Oliver Hunt wrote:
That said I think allowing '1.' (etc) makes sense as it's fairly standard across multiple programming languages, and I am unaware of any specific reason for disallowing it.
In the long term I don't see changing the grammar to allow a trailing period as being harmful as it's a relaxation. In the short term vendors that follow the spec may fail to parse content :-(
I think that would be a mistake. We have seen lots of tragic cases on the web where if we allow deviation from good practice, then those deviations will surely occur. In the long run, that could seriously and unnecessarily impair JSON interoperability with non-JavaScript endpoints. That might be worth considering if there were some compensating benefit, but in this case there isn't one. Deviating from the JSON grammar would be a bad tradeoff.
On Wed, Jun 9, 2010 at 7:43 AM, Douglas Crockford <douglas at crockford.com>wrote:
On 11:59 AM, Oliver Hunt wrote:
That said I think allowing '1.' (etc) makes sense as it's fairly standard across multiple programming languages, and I am unaware of any specific reason for disallowing it.
In the long term I don't see changing the grammar to allow a trailing period as being harmful as it's a relaxation. In the short term vendors that follow the spec may fail to parse content :-(
I think that would be a mistake. We have seen lots of tragic cases on the web where if we allow deviation from good practice, then those deviations will surely occur. In the long run, that could seriously and unnecessarily impair JSON interoperability with non-JavaScript endpoints. That might be worth considering if there were some compensating benefit, but in this case there isn't one. Deviating from the JSON grammar would be a bad tradeoff.
+1. Prior to ES5, there did not seem to be any way within JS to create a library that was simultaneously fast, safe, and validating. < code.google.com/p/json-sans-eval> is a perfectly fine fast and safe
library. It's only downside is that it wasn't validating. Let's not retreat back to a non-validating JSON parser.
On 6/9/2010 05:46, Garrett Smith wrote:
...
IF anyone has a correct JSON parser, I would appreciate it. Also, are there any good test suites for JSON?
testsuites.opera.com/JSON is one. Hallvord Steen has a good blog post on it, my.opera.com/core/blog/2009/12/18/native-json-support-in-opera
Predates Carakan by a couple of weeks, so the Opera perf numbers are a bit dated by now.
hth --sigbjorn; sof at opera.com
On 6/10/10, Sigbjorn Finne <sigbjorn.finne at gmail.com> wrote:
Hi Garrett,
On 6/9/2010 05:46, Garrett Smith wrote:
...
IF anyone has a correct JSON parser, I would appreciate it. Also, are there any good test suites for JSON?
testsuites.opera.com/JSON is one. Hallvord Steen has a good blog post on it, my.opera.com/core/blog/2009/12/18/native-json-support-in-opera
Predates Carakan by a couple of weeks, so the Opera perf numbers are a bit dated by now.
hth
It would be really useful to have the test as a zip file so it could be downloaded and run locally.
I've identified five bugs in json2.js' JSON.parse and I want to address those. I suspect more could be found with a test suite for JSON.parse (no reviver).
The Opera test functions start running automatically. It would be nice to have a start button and placed next to that a warning that the test may freeze or crash internet explorer 8 (as it happened to me).
With a start button, I would at least have the hacking ability to say, for example, type into the location bar: javascript: addMyJSONScript(); void (JSON.parse = MyJSON.parse) and then hit enter, and then hit the start button on the test runner. That kind of sucks, though. Really what I want is a testsuite that I can download and run.
I've also noticed that the test runner uses sync request. This can be a problem if the test fails to load. Instead, it should use async requests or iframes or something -- so long as it is not known to lock up the browser. Perhaps that is what caused the freeze in IE (or was it the performance test that was too intensive?)
Also, I noticed in the blog entry:
| TAB characters may not appear inside strings | - {"tab":"there is a tab -> <- here"} is wrong! | The spec requires using \t instead of a literal TAB character.
The blog entry goes on about on how TAB character is not allowed in strings. Where in the spec is it stated that literal TAB characters in strings are disallowed?
my.opera.com/core/blog/2009/12/18/native-json-support-in-opera
Finally, I'd like mention the buggy implementations that have been released: Firefox and IE. Having to devise feature tests for every known implementation failure case requires the developer first gather known failures. He must do this by testing all the implementations that he can, and, for each found failure, provide a feature test in the adapter script he is writing and where that feature test fails, provide a fallback to his own implementation.That's a lot of burden on the developer.
Instead, I'd much rather see open test suites where implementations would collaborate and release consistent, standard features.
Garrett
On 11:59 AM, Garrett Smith wrote:
Where in the spec is it stated that literal TAB characters in strings are disallowed?
It clearly states that ASCII control characters are not allowed in strings. TAB is an ASCII control character.
On 6/16/10, Douglas Crockford <douglas at crockford.com> wrote:
On 11:59 AM, Garrett Smith wrote:
Where in the spec is it stated that literal TAB characters in strings are disallowed? It clearly states that ASCII control characters are not allowed in strings. TAB is an ASCII control character.
On 6/16/10, Douglas Crockford <douglas at crockford.com> wrote:
On 11:59 AM, Garrett Smith wrote:
Where in the spec is it stated that literal TAB characters in strings are disallowed? It clearly states that ASCII control characters are not allowed in strings. TAB is an ASCII control character.
The character code for TAB is 9 and so within U+0000 thru U+001F. That is why it is disallowed. I did not know the character code for TAB was 9.
| JSONStringCharacter :: | SourceCharacter but not double-quote " or backslash \ or U+0000 thru U+001F | \ JSONEscapeSequence
If 127 is considered a control character, then it is not included in that definition.
Garrett
Today I looked for a good json regexp tester and finding nothing, decided to write one.
The strategy that occurred to me was to first define a regex for the literal components (ES5 lumps literal value into the JSONValue alongside JSONObject and JSONArray). That way, I could reuse the literal components to define JSONObject and JSONArray.
In the process, I noticed that the grammar defined for JSONNumber is different than that defiend by json2.js, which I had just looked at, and which has similar numeric parsing as seen in The GOod parts. That is, it allows DecimalLiteral, which includes this production:
DecimalIntegerLiteral .
In contrast, JSONNumber does not allow that.
JSONNumber :: -opt DecimalIntegerLiteral JSONFractionopt ExponentPartopt
Am I misreading the spec, or is "1." not valid JSON?
Next step was to see what the browsers do.
Mozilla and IE accept it, while Webkit and Opera throws an error.
JSON.parse("1.")
Firefox 3.6, IE8, Besen r27: 1
Safari, Opera SyntaxError
IF anyone has a correct JSON parser, I would appreciate it. Also, are there any good test suites for JSON?
Garrett
Today I looked for a good json regexp tester and finding nothing, decided to write one. The strategy that occurred to me was to first define a regex for the literal components (ES5 lumps literal value into the JSONValue alongside JSONObject and JSONArray). That way, I could reuse the literal components to define JSONObject and JSONArray. In the process, I noticed that the grammar defined for JSONNumber is different than that defiend by json2.js, which I had just looked at, and which has similar numeric parsing as seen in The GOod parts. That is, it allows DecimalLiteral, which includes this production: DecimalIntegerLiteral . In contrast, JSONNumber does not allow that. JSONNumber :: -opt DecimalIntegerLiteral JSONFractionopt ExponentPartopt Am I misreading the spec, or is "1." not valid JSON? Next step was to see what the browsers do. Mozilla and IE accept it, while Webkit and Opera throws an error. JSON.parse("1.") Firefox 3.6, IE8, Besen r27: 1 Safari, Opera SyntaxError IF anyone has a correct JSON parser, I would appreciate it. Also, are there any good test suites for JSON? Garrett