Discuss
  • JSON.parse should be simple and idiot-proof

JSON.parse should be simple and idiot-proof

# kai zhu (6 years ago)

wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications. thankfully, JSON.parse is rarely suspect in this process. this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:

  1. JSON.parse with zero-config to rule-out bugs during this step
  2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access gibson042/ecma262-proposal-JSON-parse-with-source

kai zhu kaizhu256 at gmail.com

# Isiah Meadows (6 years ago)

Just because something exists doesn't mean it'll be broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

# Richard Gibson (6 years ago)

First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

# Peter Jaszkowiak (6 years ago)

He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.

# Isiah Meadows (6 years ago)

This would be the correct understanding here, a 4th parameter read as a boolean.

# Richard Gibson (6 years ago)

Yes, understood. And setting aside the undesirable characteristics of boolean trap ariya.io/2011/08/hall-of-api-shame-boolean-trap interfaces

for the moment (which will certainly be a source of regret if ECMAScript ever gets a BigFloat), my point is that doing so would affect parsing of all numbers, as opposed to only those numbers that really should be BigInts. For example, let's look at a sample Twitter API entity developer.twitter.com/en/docs/tweets/data-dictionary/overview/tweet-object :

{ "created_at":"Thu Apr 06 15:24:15 +0000 2017", "id": 850006245121695744, "id_str": "850006245121695744", "text": "1/ Today we’re sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP", "user": { "id": 6253282, "id_str": "6253282", "name": "Twitter API", "screen_name": "twitterapi", "followers_count": 21, "friends_count": 32 }, "entities": {} }

It's nice that e.g. JSON.parse(…, null, true) would allow us to access the ids as full-fidelity numbers (Twitter documents them as an int64), but much less nice that the behavior would affect all numbers. For instance, let's say this is one tweet in a list that we want to sort by the "follower ratio" of their creators:

let creatorPopularity = tweet.user.followers_count / tweet.user.friends_count;

// (without parse-as-BigInt) → 0.65625

// (with parse-as-BigInt) → 0n

We just silently lost floating-point arithmetic.

# Peter Jaszkowiak (6 years ago)

What if JSON.parse took an options object as the second parameter? There's already been an instance of a JS API changing from booleans to an options object: addEventListener.

I know it wasn't tc39, but it shows that it's possible. Another option is to add a different method to the JSON object.

# Jordan Harband (6 years ago)

I'd suggest filing an issue like this directly on the proposal repo.

# kai zhu (6 years ago)

i'm against all these proposals. JSON.parse should remain zero-config and idiot-proof. you will always need a second-pass in product-development to revive non-JSON datatypes anyways, like Date, and thats where reviving BigInt should take place as well.

again, i'm ok with working on web-projects where there are bugs in the second-pass (which is expected). i'm not ok with web-projects, where i cannot reliably trust JSON.parse to do its job (and have to add it to checklist of things to inspect when figuring out where the bug is in end-to-end communications).

fyi, the twitter example is not compelling. it already has a string-form “id_str” which is more appropriate/easier-to-handle in javascript-land. what benefit does a pseudorandom BigInt “id” (which has no need of arithmetic) have over plain-string “id_str” in javascript-land? none.

kai zhu kaizhu256 at gmail.com

# Isiah Meadows (6 years ago)

The use is with integers 2^53 and above in general: most non-JS JSON clients (like Java and C++) can correctly interpret 64-bit integers and some can even represent arbitrary-size integers (like Python), but JS can't. This can come up sometimes in math/science contexts, where numbers can often get that large (if not sometimes larger), but it can also come up easily in other contexts and is in general a legitimate concern. Third-party APIs often also represent IDs as 64-bit integers and send them accordingly, and you can't always fix those (that's where the integration concerns exist). Twitter fixed theirs, but broken servers will still exist, and not everyone who uses JSON widely is even aware of JS's floating point restrictions.

Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com