Arbitrary precision numbers in JSON
On 2018-03-19 02:33, Michał Wadas wrote:
Fact: JSON allows arbitrary precision numbers. Problem: JavaScript is unable to express these numbers.
Proposed solution: introduce JSON.safeParse. This method will work as JSON.parse, but throwing on values that can't be accurately represented by IEEE 754 64-bit float.
Alternative: allow user to specify number class - eg. by adding options object with optional method "parseNumber", overwriting default behaviour of using builtin number type.
Any thoughts on this?
Yes, it is a SUPERBAD idea.
Anders
This is the parsing side equivalent to "canonicalized". Any security-sensitive application of "canonical JSON" should have a strict verifier confirming that the input is canonical; this verifier would presumably also throw for out-of-range numbers as you suggest.
On 2018-03-19 02:37, Anders Rundgren wrote:
On 2018-03-19 02:33, Michał Wadas wrote:
Fact: JSON allows arbitrary precision numbers. Problem: JavaScript is unable to express these numbers.
Proposed solution: introduce JSON.safeParse. This method will work as JSON.parse, but throwing on values that can't be accurately represented by IEEE 754 64-bit float.
Alternative: allow user to specify number class - eg. by adding options object with optional method "parseNumber", overwriting default behaviour of using builtin number type.
Any thoughts on this?
Yes, it is a SUPERBAD idea.
Pardon my unnecessary dismissive response. A more constructive response would be:
This is "approximately" the de-facto standard for dealing with BigNums and tons of other non-standard types.
var obj = JSON.parse("JSON formatted data"); var val = new BigNum(obj.sizeOfTheUniverseInCubicMeters);
JSON on the wire: { "sizeOfTheUniverseInCubicMeters": "3.45e+445454545454545776767676676" }
When TC-39 introduces BigNums in ES, this is the most likely way they will address this particular aspect.
Anders
This is "approximately" the de-facto standard for dealing with BigNums and tons of other non-standard types.
var obj = JSON.parse("JSON formatted data"); var val = new BigNum(obj.sizeOfTheUniverseInCubicMeters);
JSON on the wire: { "sizeOfTheUniverseInCubicMeters": "3.45e+445454545454545776767676676" }
When TC-39 introduces BigNums in ES, this is the most likely way they will address this particular aspect.
For serializing BigNums, the toJSON() interface seems like a match made in heaven.
Ideally, JSON should have a parseWithSchema() method that automatically coerces string encapsulated data types like above into their proper native representation.
thanx, Anders
Yeah, there are workarounds for passing bigints.
Unfortunately, it's workaround, and it doesn't solve the problem - certain valid JSONs can't be properly used in ECMAScript without writing JSON parser in user land.
On 2018-03-19 07:32, Michał Wadas wrote:
Yeah, there are workarounds for passing bigints.
Unfortunately, it's workaround, and it doesn't solve the problem - certain valid JSONs can't be properly used in ECMAScript without writing JSON parser in user land.
Hi Michal,
I wouldn't call it a workaround.
The idea that there must be a native data type in a message format for each data type you can come up with has huge real-world limitations. Physical address descriptors (street, zip, state, etc), don't have native JSON counterpart either to take a fully valid example from another application area.
Using an external (with respect to the parser), mapping schema [1,2] addresses these problems in a fully universal and extensible manner. toJSON seems to do a good job in the other direction.
The importance of JSON "Number" is exaggerated. In the systems I work with (Payments), you hardly find a single instance of "Number". No, it doesn't even work for expressing money!
Anders
1] Ultra simple/manual: new BigNum(obj.sizeOfTheUniverseInCubicMeters); 2] Sophisticated: JSON.parseWithSchema(schema, jsonData);
It's workaround.
JSON.parse can't properly parse valid JSON subset.
ZIP codes are not part of JSON specification, arbitrary precision numbers are (and JSONs with 64-bit ints are widely used).
Modyfing schema is possible only if you control it and control all of its consumers.
BigInt proposal already cover serialisation part.
On 2018-03-19 08:23, Michał Wadas wrote:
It's workaround.
JSON.parse can't properly parse valid JSON subset.
ZIP codes are not part of JSON specification, arbitrary precision numbers are (and JSONs with 64-bit ints are widely used).
Well, you have an comrade at Google (Mike Samuels) who also believe that making JSON.parse incompatible with Java, .NET etc. is a good idea although it suffice putting such values within a pair of quotes.
Also worth a read: tools.ietf.org/html/rfc7493#section-2.2
Anders
On 2018-03-19 08:23, Michał Wadas wrote:
It's workaround.
I forgot to mention that if you overload the "Number" type with int64's, BigNum's, BigInt's etc. you will need a mapping scheme so that the parser knows what to look for and create. OTOH, mapping from a string literal using the replacer already have what it takes.
Right, this won't work for the people who had to write their own JSON tools due to limitations in "Number".
Anders
Maybe have a parser function that receive the text of the number? If no callout is specified, it could throw an appropriate error.
JSON.safeParse(json, optionalReviver, optionalParseUnrepresentable)
That would allow it to tie into future proposals like decimal, and in conjunction with a reviver could treat an array of numbers known to be large as an Int64Array view over an ArrayBuffer.
On 2018-03-19 14:43, Mike Samuel wrote:
Maybe have a parser function that receive the text of the number? If no callout is specified, it could throw an appropriate error.
JSON.safeParse(json, optionalReviver, optionalParseUnrepresentable)
That would allow it to tie into future proposals like decimal, and in conjunction with a reviver could treat an array of numbers known to be large as an Int64Array view over an ArrayBuffer.
That's a good idea. I guess it would permit decimal & friends to put in strings as well because this is used by quite a bunch of people?
Anders
On 2018-03-19 14:43, Mike Samuel wrote:
Maybe have a parser function that receive the text of the number? If no callout is specified, it could throw an appropriate error.
JSON.safeParse(json, optionalReviver, optionalParseUnrepresentable)
That would allow it to tie into future proposals like decimal, and in conjunction with a reviver could treat an array of numbers known to be large as an Int64Array view over an ArrayBuffer.
I may be off here, but don't you also need something like rawToJSON for JSON.stringify()? That is, rawToJSON would return a string which is used "as is".
Anders
On Mon, Mar 19, 2018 at 10:09 AM, Anders Rundgren < anders.rundgren.net at gmail.com> wrote:
On 2018-03-19 14:43, Mike Samuel wrote:
Maybe have a parser function that receive the text of the number? If no callout is specified, it could throw an appropriate error.
JSON.safeParse(json, optionalReviver, optionalParseUnrepresentable)That would allow it to tie into future proposals like decimal, and in conjunction with a reviver could treat an array of numbers known to be large as an Int64Array view over an ArrayBuffer.
I may be off here, but don't you also need something like rawToJSON for JSON.stringify()? That is, rawToJSON would return a string which is used "as is".
Quite right, you would need some adjustment to stringify too.
We could change the contract of toJSON so that if it returns a value with a certain runtime type, then JSON.stringify asserts that it is a syntactically valid ES404 value and then uses it inline. For example, ({[Symbol('rawJSON')]: "1" + "0".repeat(1000) }). That would round-trip and compose reasonably well and wouldn't add another special method that needs to be tested for at stringify time.
Fact: JSON allows arbitrary precision numbers. Problem: JavaScript is unable to express these numbers.
Proposed solution: introduce JSON.safeParse. This method will work as JSON.parse, but throwing on values that can't be accurately represented by IEEE 754 64-bit float.
Alternative: allow user to specify number class - eg. by adding options object with optional method "parseNumber", overwriting default behaviour of using builtin number type.
Any thoughts on this?