Sets plus JSON
On Wed, Oct 3, 2012 at 12:37 PM, Nicholas C. Zakas < standards at nczconsulting.com> wrote:
After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object,
...In the same way that an Array is just an Object.
and perhaps it would be best to define a toJSON() method for sets such as:
Set.prototype.toJSON = function() { return Array.from(this); };
That way, JSON.stringify() would do something rational by default when used with sets.
Thoughts?
Definitely +1
This also made me wonder about Maps, if the same use case were applied - toJSON simply wouldn't work when you have an object as a key.
On Wed, Oct 3, 2012 at 9:37 AM, Nicholas C. Zakas <standards at nczconsulting.com> wrote:
After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as:
Set.prototype.toJSON = function() { return Array.from(this); };
That way, JSON.stringify() would do something rational by default when used with sets.
+1. Sets clearly map closest into JSON arrays. They don't roundtrip, but that's a necessary evil.
Nicholas C. Zakas wrote:
After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as:
Set.prototype.toJSON = function() { return Array.from(this);
It depends... you should be able to reread it, so the best thing would proably be to use matching set of transformers for both stringify and parse. I personally would rather see something like { Set_from: Array.from(this) } here.
};
That way, JSON.stringify() would do something rational by default when used with sets.
Thoughts?
Thanks, Nicholas
Herby
P.S.: It would be helpful, however, to include JSON helpers for wrapping sets, maps etc. in some module; but it can be a library, no need to use spec for this.
On Wed, Oct 3, 2012 at 12:56 PM, Herby Vojčík <herby at mailbox.sk> wrote:
Nicholas C. Zakas wrote:
After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as:
Set.prototype.toJSON = function() { return Array.from(this);
It depends... you should be able to reread it, so the best thing would proably be to use matching set of transformers for both stringify and parse. I personally would rather see something like { Set_from: Array.from(this) } here.
};
The revived array can be passed as an arg to new Set(revived) :
new Set(JSON.parse(s.toJSON()))
Introducing another Set constructor just to wrap the above is early-warning feature creep. toJSON is an intuitive addition
This also made me wonder about Maps, if the same use case were applied - toJSON simply wouldn't work when you have an object as a key.
The default should probably be to convert to an object who’s keys are the results of applying String() to the map’s keys. Additionally, one could introduce a method toPairArray() that converts a Map into an array of pairs (2-element arrays) – that can be JSON-ified. The alternatives are:
- Switch to pairs if the keys are not strings.
- Allow a map to be configured which of the two representations should be used for toJSON.
- Give toJSON a parameter whose default is to produce an object.
Given JSON.toStringify(), I’m not sure that #2 is necessary and that #3 is useful. #1 might work, but seems like a big change in representation and it takes time to check all the keys.
Another options for Maps is to represent them as an array of [key, value].
Rick Waldron wrote:
On Wed, Oct 3, 2012 at 12:56 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk>> wrote:
Nicholas C. Zakas wrote: After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as: Set.prototype.toJSON = function() { return Array.from(this); It depends... you should be able to reread it, so the best thing would proably be to use matching set of transformers for both stringify and parse. I personally would rather see something like { _Set_from_: Array.from(this) } here. };The revived array can be passed as an arg to new Set(revived) :
new Set(JSON.parse(s.toJSON()))
I am talking about deep nested structure with (possibly) multiple sets all over. Your solution is unusable, you need to explicitly know where the arrayified sets are.
On Wed, Oct 3, 2012 at 1:35 PM, Axel Rauschmayer <axel at rauschma.de> wrote:
This also made me wonder about Maps, if the same use case were applied - toJSON simply wouldn't work when you have an object as a key.
The default should probably be to convert to an object who’s keys are the results of applying String() to the map’s keys. Additionally, one could introduce a method toPairArray() that converts a Map into an array of pairs (2-element arrays) – that can be JSON-ified. The alternatives are:
- Switch to pairs if the keys are not strings.
- Allow a map to be configured which of the two representations should be used for toJSON.
- Give toJSON a parameter whose default is to produce an object.
Given JSON.toStringify(), I’m not sure that #2 is necessary and that #3 is useful. #1 might work, but seems like a big change in representation and it takes time to check all the keys.
My concern was actually about the references (for keys) that would be broken, how would you get those back? There is no reasonably sane way that doesn't involve magic scope memory tables and polluters.
Unless I'm looking at it the wrong way...
On Wed, Oct 3, 2012 at 1:57 PM, Herby Vojčík <herby at mailbox.sk> wrote:
Rick Waldron wrote:
On Wed, Oct 3, 2012 at 12:56 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk>> wrote:
Nicholas C. Zakas wrote: After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() methodfor sets such as:
Set.prototype.toJSON = function() { return Array.from(this); It depends... you should be able to reread it, so the best thing would proably be to use matching set of transformers for both stringify and parse. I personally would rather see something like { _Set_from_: Array.from(this) } here. };The revived array can be passed as an arg to new Set(revived) :
new Set(JSON.parse(s.toJSON()))
I am talking about deep nested structure with (possibly) multiple sets all over. Your solution is unusable, you need to explicitly know where the arrayified sets are.
Fair enough, my solution definitely doesn't scale.
In that case, I'd say it's not the language's job to try and guess what the user code wanted to do with a serious of deeply nested, stringified arrays in a JSON string. Seems like a non-starter.
On Wed, Oct 3, 2012 at 1:43 PM, Brandon Benvie <brandon at brandonbenvie.com>wrote:
Another options for Maps is to represent them as an array of [key, value].
Which is a rough approximation of what a Map looks like internally.
Again, I was more concerned with how to revive a Map and restore the key references.
Rick Waldron wrote:
On Wed, Oct 3, 2012 at 1:57 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk>> wrote:
Rick Waldron wrote: On Wed, Oct 3, 2012 at 12:56 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk> <mailto:herby at mailbox.sk <mailto:herby at mailbox.sk>>> wrote: Nicholas C. Zakas wrote: After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as: Set.prototype.toJSON = function() { return Array.from(this); It depends... you should be able to reread it, so the best thing would proably be to use matching set of transformers for both stringify and parse. I personally would rather see something like { _Set_from_: Array.from(this) } here. }; The revived array can be passed as an arg to new Set(revived) : new Set(JSON.parse(s.toJSON())) I am talking about deep nested structure with (possibly) multiple sets all over. Your solution is unusable, you need to explicitly know where the arrayified sets are.Fair enough, my solution definitely doesn't scale.
In that case, I'd say it's not the language's job to try and guess what the user code wanted to do with a serious of deeply nested, stringified arrays in a JSON string. Seems like a non-starter.
I am worried you are accusing me of something Bad (TM) I did not actually proposed.
I said, ones cannot used .toJSON because you need to read it back, so you should serialize / deserialize by some protocol, for which you need to supply the transforming functions, JSON.parse as well as JSON.stringify have extra parameters for them (plus contemplated the idea that such composable wrappers/unwrappers for Sets, Maps etc. could be supplied because they can be handy, but probably library is the better level for this than the language).
Rick
Introducing another Set constructor just to wrap the above is early-warning feature creep. toJSON is an intuitive addition
Definitely not "feature-creep another Set constructor".
I always find it next to impossible to guess use cases for non-trivial features. So having those would help. Often YAGNI applies. Two possibilities:
- Subtype Map and override toJSON
- Use JSON.stringify() with a replacer
As Brandon Benvie mention, we could standardize on an array of pairs, but my (not very educated) guess would be that converting to objects makes more sense.
On Wed, Oct 3, 2012 at 2:07 PM, Herby Vojčík <herby at mailbox.sk> wrote:
Rick Waldron wrote:
On Wed, Oct 3, 2012 at 1:57 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk>> wrote:
Rick Waldron wrote: On Wed, Oct 3, 2012 at 12:56 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk> <mailto:herby at mailbox.sk <mailto:herby at mailbox.sk>>> wrote: Nicholas C. Zakas wrote: After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, Ifound myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as:
Set.prototype.toJSON = function() { return Array.from(this); It depends... you should be able to reread it, so the best thing would proably be to use matching set of transformers for both stringify and parse. I personally would rather see something like { _Set_from_: Array.from(this) } here. }; The revived array can be passed as an arg to new Set(revived) : new Set(JSON.parse(s.toJSON())) I am talking about deep nested structure with (possibly) multiple sets all over. Your solution is unusable, you need to explicitly know where the arrayified sets are.Fair enough, my solution definitely doesn't scale.
In that case, I'd say it's not the language's job to try and guess what the user code wanted to do with a serious of deeply nested, stringified arrays in a JSON string. Seems like a non-starter.
I am worried you are accusing me of something Bad (TM) I did not actually proposed.
I said, ones cannot used .toJSON because you need to read it back, so you should serialize / deserialize by some protocol, for which you need to supply the transforming functions, JSON.parse as well as JSON.stringify have extra parameters for them
Of course, but again, I'm not sure how either of those would help identify which:
'{ "foo": [1,2,3,4,5], "bar": [1,2,3,4,5] }'
...Is the Array and which is the Set
Maybe I'm still misunderstanding? If I am, I apologize
(plus contemplated the idea that such composable wrappers/unwrappers for Sets, Maps etc. could be supplied because they can be handy, but probably library is the better level for this than the language).
+1, this is something library authors should tackle before any language attention is given.
On Wed, Oct 3, 2012 at 2:04 PM, Rick Waldron <waldron.rick at gmail.com> wrote:
On Wed, Oct 3, 2012 at 1:43 PM, Brandon Benvie <brandon at brandonbenvie.com>wrote:
Another options for Maps is to represent them as an array of [key, value].
Which is a rough approximation of what a Map looks like internally.
Sorry, this is incorrect. Map looks more like:
[key1, key2] [value1, value2]
Sorry for confusion
Ah, now I get it. But isn’t that a much more general problem (dates, etc.)? You can either come up with a general encoding or do some schema-aware post-processing (including a JSON.parse reviver).
Rick Waldron wrote:
On Wed, Oct 3, 2012 at 2:07 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk>> wrote:
Rick Waldron wrote: On Wed, Oct 3, 2012 at 1:57 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk> <mailto:herby at mailbox.sk <mailto:herby at mailbox.sk>>> wrote: Rick Waldron wrote: On Wed, Oct 3, 2012 at 12:56 PM, Herby Vojčík <herby at mailbox.sk <mailto:herby at mailbox.sk> <mailto:herby at mailbox.sk <mailto:herby at mailbox.sk>> <mailto:herby at mailbox.sk <mailto:herby at mailbox.sk> <mailto:herby at mailbox.sk <mailto:herby at mailbox.sk>>>> wrote: Nicholas C. Zakas wrote: After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as: Set.prototype.toJSON = function() { return Array.from(this); It depends... you should be able to reread it, so the best thing would proably be to use matching set of transformers for both stringify and parse. I personally would rather see something like { _Set_from_: Array.from(this) } here. }; The revived array can be passed as an arg to new Set(revived) : new Set(JSON.parse(s.toJSON())) I am talking about deep nested structure with (possibly) multiple sets all over. Your solution is unusable, you need to explicitly know where the arrayified sets are. Fair enough, my solution definitely doesn't scale. In that case, I'd say it's not the language's job to try and guess what the user code wanted to do with a serious of deeply nested, stringified arrays in a JSON string. Seems like a non-starter. I am worried you are accusing me of something Bad (TM) I did not actually proposed. I said, ones cannot used .toJSON because you need to read it back, so you should serialize / deserialize by some protocol, for which you need to supply the transforming functions, JSON.parse as well as JSON.stringify have extra parameters for themOf course, but again, I'm not sure how either of those would help identify which:
'{ "foo": [1,2,3,4,5], "bar": [1,2,3,4,5] }'
...Is the Array and which is the Set
Maybe I'm still misunderstanding? If I am, I apologize
The idea is that set is replaced with something like { "this is an instance of Set": Array.from(aSet) } and in the way back it is converted back to set again. (so not a plain array, it is then not distiguishable, of course) (similarly for maps etc.)
The problem is choosing how to transform there and back so it does not clash. If clash is avoided, using proper replacer in JSON.stringify and its counterpart in JSON.parse more-or-less-transparently JSON-encodes and -decodes sets.
On Wed, Oct 3, 2012 at 2:12 PM, Axel Rauschmayer <axel at rauschma.de> wrote:
I always find it next to impossible to guess use cases for non-trivial features. So having those would help. Often YAGNI applies. Two possibilities:
- Subtype Map and override toJSON
- Use JSON.stringify() with a replacer
As Brandon Benvie mention, we could standardize on an array of pairs, but my (not very educated) guess would be that converting to objects makes more sense.
Map.prototype.toJSON = function() { // somehow iterate keys and values into an array pair? // for now, I'll fake it. return '[ [ { "foo": "whatever" }, [ 1, 2, 3, 4, 5 ] ], [ {}, "my key is a plain object" ] ]'; };
function Key(foo) { this.foo = foo; }
var map = new Map(), key = new Key("whatever"), obj = {};
map.set(key, [ 1, 2, 3, 4, 5 ]); map.set(obj, "my key is a plain object");
console.log( map.get(key) ); // [ 1, 2, 3, 4, 5 ] console.log( map.get(obj) ); // "my key is a plain object"
console.log( map.toJSON() );
// [ [ { "foo": "whatever" }, [ 1, 2, 3, 4, 5 ] ], [ {}, "my key is a plain object" ] ]
Now imagine this map is stored in some kind of NoSQL document store and will be later pulled out by some other functionality in our application.
How are the map keys revived into_scope?
console.log( JSON.parse(map.toJSON()) );
This just makes an array of pairs.
Le 03/10/2012 18:37, Nicholas C. Zakas a écrit :
(...)
Set.prototype.toJSON = function() { return Array.from(this); };
That way, JSON.stringify() would do something rational by default when used with sets.
Thoughts?
Useful defaults. I love it!
On Wed, Oct 3, 2012 at 2:21 PM, Herby Vojčík <herby at mailbox.sk> wrote:
Of course, but again, I'm not sure how either of those would help identify which:
'{ "foo": [1,2,3,4,5], "bar": [1,2,3,4,5] }'
...Is the Array and which is the Set
Maybe I'm still misunderstanding? If I am, I apologize
The idea is that set is replaced with something like { "this is an instance of Set": Array.from(aSet) } and in the way back it is converted back to set again. (so not a plain array, it is then not distiguishable, of course) (similarly for maps etc.)
The problem is choosing how to transform there and back so it does not clash. If clash is avoided, using proper replacer in JSON.stringify and its counterpart in JSON.parse more-or-less-transparently JSON-encodes and -decodes sets.
Ok, that's what I thought you meant and yes, I agree that is a task for library code
On Wed, Oct 3, 2012 at 2:19 PM, Rick Waldron <waldron.rick at gmail.com> wrote:
On Wed, Oct 3, 2012 at 2:04 PM, Rick Waldron <waldron.rick at gmail.com>wrote:
On Wed, Oct 3, 2012 at 1:43 PM, Brandon Benvie <brandon at brandonbenvie.com
wrote:
Another options for Maps is to represent them as an array of [key, value].
Which is a rough approximation of what a Map looks like internally.
Sorry, this is incorrect. Map looks more like:
[key1, key2] [value1, value2]
Sorry for confusion
I image it looking something like [key1, value1, key2, value2...] -- index % 2 implies values. Anything more would mean an awful lot of unnecessary allocations. I don't see why this wouldn't be sufficient for the json form as well, especially if the language had something like a take2 iterator.
So here's a crazy idea (how's that for a lead-in?): What if it were possible to specify that you want all non-array JSON objects to be revived as maps instead of objects? Perhaps as an option passed to JSON.parse()? JSON objects really have no use for prototypes and could just as easily be represented as maps. You certainly wouldn't want that by default, but as an option, maybe it could make sense.
Taking a cue from plist, which is easily transformed to and from JSON, you would end up with something like [{ key: {...key..}, value: {...value...} }] which is less space efficient but pretty easy to automatically convert back to a map (aside from correctly handling duplicate values).
Another reviver-friendly possibility: type tags for objects (arrays remain as they are).
[ { "type": "Date", "time": 1349291353269 }, { "type": "Object", "first": "Jane", "last": "Doe" }, { "type": "Map", "entries": [ ["first", "Jane"], ["last", "Doe"] ] } ]
This assumes that the keys of objects are known beforehand (to avoid a key from clashing with "type"). Keys of maps can be arbitrary, even type-tagged objects.
Rick Waldron wrote:
On Wed, Oct 3, 2012 at 2:04 PM, Rick Waldron <waldron.rick at gmail.com <mailto:waldron.rick at gmail.com>> wrote:
On Wed, Oct 3, 2012 at 1:43 PM, Brandon Benvie <brandon at brandonbenvie.com <mailto:brandon at brandonbenvie.com>> wrote: Another options for Maps is to represent them as an array of [key, value]. Which is a rough approximation of what a Map looks like internally.Sorry, this is incorrect. Map looks more like:
[key1, key2] [value1, value2]
Sorry for confusion
As an implementation in ES5, maybe (O(n) lookup cost). But the thing to aim for is the shape of the Map parameter, and that looks like
[[key1, value1], [key2, value2]]
Still need type tagging to revive as a Map, of course.
JSON object notation can't handle Map, though: key can be any value (ignore JSON not handling all JS values).
On Wed, Oct 3, 2012 at 2:44 PM, Dean Landolt <dean at deanlandolt.com> wrote:
On Wed, Oct 3, 2012 at 2:19 PM, Rick Waldron <waldron.rick at gmail.com>wrote:
On Wed, Oct 3, 2012 at 2:04 PM, Rick Waldron <waldron.rick at gmail.com>wrote:
On Wed, Oct 3, 2012 at 1:43 PM, Brandon Benvie < brandon at brandonbenvie.com> wrote:
Another options for Maps is to represent them as an array of [key, value].
Which is a rough approximation of what a Map looks like internally.
Sorry, this is incorrect. Map looks more like:
[key1, key2] [value1, value2]
Sorry for confusion
I image it looking something like [key1, value1, key2, value2...] -- index % 2 implies values. Anything more would mean an awful lot of unnecessary allocations. I don't see why this wouldn't be sufficient for the json form as well, especially if the language had something like a take2 iterator.
My correction was missing outer brackets... typed in a hurry. It makes more sense like this:
var keys = { a: {}, b: {}, g: {} }, map = new Map([ [keys.a , "alpha"], [keys.b, "beta"], [keys.g, "gamma"] ]);
console.log( map.get(keys.a) ); // "alpha" console.log( map.get(keys.b) ); // "beta" console.log( map.get(keys.g) ); // "gamma"
ie. the structure:
[ [keys.a , "alpha"], [keys.b, "beta"], [keys.g, "gamma"] ]
On Wed, Oct 3, 2012 at 3:36 PM, Brendan Eich <brendan at mozilla.org> wrote:
Rick Waldron wrote:
On Wed, Oct 3, 2012 at 2:04 PM, Rick Waldron <waldron.rick at gmail.com<mailto: waldron.rick at gmail.com**>> wrote:
On Wed, Oct 3, 2012 at 1:43 PM, Brandon Benvie <brandon at brandonbenvie.com <mailto:brandon at brandonbenvie.**com<brandon at brandonbenvie.com>>>wrote:
Another options for Maps is to represent them as an array of [key, value]. Which is a rough approximation of what a Map looks like internally.Sorry, this is incorrect. Map looks more like:
[key1, key2] [value1, value2]
Sorry for confusion
As an implementation in ES5, maybe (O(n) lookup cost). But the thing to aim for is the shape of the Map parameter, and that looks like
[[key1, value1], [key2, value2]]
Still need type tagging to revive as a Map, of course.
JSON object notation can't handle Map, though: key can be any value (ignore JSON not handling all JS values).
Ugh, sorry, I replied to myself/Dean above before reading ahead.
(oops, forgot to reply-all)
Begin forwarded message:
On 10/3/2012 4:44 PM, Allen Wirfs-Brock wrote:
(oops, forgot to reply-all)
Begin forwarded message:
*From: *Allen Wirfs-Brock <allen at wirfs-brock.com <mailto:allen at wirfs-brock.com>> *Date: *October 3, 2012 10:15:57 AM PDT *To: *Herby Vojc(ík <herby at mailbox.sk <mailto:herby at mailbox.sk>> *Subject: *Re: Sets plus JSON
This is one of the reasons that it is important that Set (and Map, etc.) are specified (and implemented) in a manner that makes them fully subclassable. By subclassing, individual use cases for serializing them can be kept distinct rather than multiple libraries or subsystems fighting over who gets control of the single implementation of Set.prototype.toJSON
That doesn't necessarily preclude a logical default, correct?
On Oct 4, 2012, at 11:02 AM, Nicholas C. Zakas wrote:
On 10/3/2012 4:44 PM, Allen Wirfs-Brock wrote:
(oops, forgot to reply-all)
Begin forwarded message:
From: Allen Wirfs-Brock <allen at wirfs-brock.com> Date: October 3, 2012 10:15:57 AM PDT To: Herby Vojčík <herby at mailbox.sk> Subject: Re: Sets plus JSON
This is one of the reasons that it is important that Set (and Map, etc.) are specified (and implemented) in a manner that makes them fully subclassable. By subclassing, individual use cases for serializing them can be kept distinct rather than multiple libraries or subsystems fighting over who gets control of the single implementation of Set.prototype.toJSON That doesn't necessarily preclude a logical default, correct?
-N
If there is such a thing as a rational default. And it gets harder to define as you move from Set on to Map.
In either case, you are really defining a new application schema layer on top of JSON that requires custom deserialization. It won't be meaningful to JSON clients that don't know your schema conventions. Arguably, having { } as the the default JSON serialization for Set and Map serves as a reminder to developers that if they want to use JSON to serialize those abstraction they will need to coordinate with clients in a deeper way than is required for simple arrays and struct like objects.
On 10/4/2012 11:30 AM, Allen Wirfs-Brock wrote:
On Oct 4, 2012, at 11:02 AM, Nicholas C. Zakas wrote:
On 10/3/2012 4:44 PM, Allen Wirfs-Brock wrote:
(oops, forgot to reply-all)
Begin forwarded message:
*From: *Allen Wirfs-Brock <allen at wirfs-brock.com <mailto:allen at wirfs-brock.com>> *Date: *October 3, 2012 10:15:57 AM PDT *To: *Herby Vojc(ík <herby at mailbox.sk <mailto:herby at mailbox.sk>> *Subject: *Re: Sets plus JSON
This is one of the reasons that it is important that Set (and Map, etc.) are specified (and implemented) in a manner that makes them fully subclassable. By subclassing, individual use cases for serializing them can be kept distinct rather than multiple libraries or subsystems fighting over who gets control of the single implementation of Set.prototype.toJSON That doesn't necessarily preclude a logical default, correct?
-N
If there is such a thing as a rational default. And it gets harder to define as you move from Set on to Map.
In either case, you are really defining a new application schema layer on top of JSON that requires custom deserialization. It won't be meaningful to JSON clients that don't know your schema conventions. Arguably, having { } as the the default JSON serialization for Set and Map serves as a reminder to developers that if they want to use JSON to serialize those abstraction they will need to coordinate with clients in a deeper way than is required for simple arrays and struct like objects.
Allen
I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed).
I don't think that the ability to deserialize should be the deciding factor. After all, Date objects are serialized into a string that isn't deserialized back into a Date object unless you provide your own reviver.
Nicholas C. Zakas wrote:
I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed).
As with Set, I claim the default JSON for Map should be
[[key1, value1], ~~~ [keyN, valueN]]
with ~~~ as meta-ellipsis.
This is not a round-tripping seralization, any more than
[elt1, ~~~ eltN]
is for Set. It simply is the normal form expected by the Map constructor. That wins.
On Thu, Oct 4, 2012 at 8:07 PM, Nicholas C. Zakas < standards at nczconsulting.com> wrote:
On 10/4/2012 11:30 AM, Allen Wirfs-Brock wrote:
If there is such a thing as a rational default. And it gets harder to define as you move from Set on to Map.
In either case, you are really defining a new application schema layer on top of JSON that requires custom deserialization. It won't be meaningful to JSON clients that don't know your schema conventions. Arguably, having { } as the the default JSON serialization for Set and Map serves as a reminder to developers that if they want to use JSON to serialize those abstraction they will need to coordinate with clients in a deeper way than is required for simple arrays and struct like objects.
Allen
I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed).
I don't think that the ability to deserialize should be the deciding factor. After all, Date objects are serialized into a string that isn't deserialized back into a Date object unless you provide your own reviver.
Yes, I still agree with the Set->(as Array)->JSON
and...
new Date( JSON.parse( JSON.stringify( new Date() ) ) );
Results in Date object, as I would expect. So I'd expect...
new Set( JSON.parse( JSON.stringify( new Set([1,2,3,4,5]) ) ) );
To result in a Set
On Thu, Oct 4, 2012 at 8:16 PM, Brendan Eich <brendan at mozilla.org> wrote:
Nicholas C. Zakas wrote:
I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed).
As with Set, I claim the default JSON for Map should be
[[key1, value1], ~~~ [keyN, valueN]]
I'm still curious about my question from yesterday; instead of repasting, I put it in a gist:
Rick Waldron wrote:
On Thu, Oct 4, 2012 at 8:16 PM, Brendan Eich <brendan at mozilla.org <mailto:brendan at mozilla.org>> wrote:
Nicholas C. Zakas wrote: I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed). As with Set, I claim the default JSON for Map should be [[key1, value1], ~~~ [keyN, valueN]]I'm still curious about my question from yesterday; instead of repasting, I put it in a gist:
If the Map serializes as [[k1, v1], ~~~ [kN, vN]] then the deserializer could risk interpreting such as a Map. Better would be for the serializer to make the Map encoding the value of a by-convention key whose name implies the value is a Map serialization.
Still a level up from core JS, but as Allen said, that's part of the equation. This is not a core-language serialization-via-JSON protocol, because JSON is frozen as-is for all time.
On Oct 4, 2012, at 5:32 PM, Rick Waldron wrote:
On Thu, Oct 4, 2012 at 8:16 PM, Brendan Eich <brendan at mozilla.org> wrote: Nicholas C. Zakas wrote: I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed).
As with Set, I claim the default JSON for Map should be
[[key1, value1], ~~~ [keyN, valueN]]
I'm still curious about my question from yesterday; instead of repasting, I put it in a gist:
Something like:
{"<kind>" : "Map",
"<mapData>" : [
[ key1, value1],
[ keyN, valueN]
]
}
is arguably a better because then you can write a reviver function that recognizes it:
function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]){ case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(e=>newValue.set(e[0], e[1])); return newValue; } default: return value; } }
var tree = JSON.parse(sourceString, mapReviver);
This should work for Map's nested at any level of a JSON tree, including Maps nested within Maps.
Allen
if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } }
On Thursday, October 4, 2012 at 9:29 PM, Allen Wirfs-Brock wrote:
On Oct 4, 2012, at 5:32 PM, Rick Waldron wrote:
On Thu, Oct 4, 2012 at 8:16 PM, Brendan Eich <brendan at mozilla.org (mailto:brendan at mozilla.org)> wrote:
Nicholas C. Zakas wrote:
I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed).
As with Set, I claim the default JSON for Map should be
[[key1, value1], ~~~ [keyN, valueN]]
I'm still curious about my question from yesterday; instead of repasting, I put it in a gist:
Something like:
{"<kind>" : "Map", "<mapData>" : [ [ key1, value1], [ keyN, valueN] ] }is arguably a better because then you can write a reviver function that recognizes it:
function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]){ case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(e=>newValue.set(e[0], e[1])); return newValue; } default: return value; } }
var tree = JSON.parse(sourceString, mapReviver);
But, this still doesn't explain how an object-as-key gets its reference put back into (the correct) scope :)
On Oct 4, 2012, at 9:02 PM, Rick Waldron wrote:
On Thursday, October 4, 2012 at 9:29 PM, Allen Wirfs-Brock wrote:
On Oct 4, 2012, at 5:32 PM, Rick Waldron wrote:
On Thu, Oct 4, 2012 at 8:16 PM, Brendan Eich <brendan at mozilla.org> wrote:
Nicholas C. Zakas wrote:
I agree, I'm not sure there is a rational default for Map, but I think there is one for Set as an array (and it seems like most people agreed).
As with Set, I claim the default JSON for Map should be
[[key1, value1], ~~~ [keyN, valueN]]
I'm still curious about my question from yesterday; instead of repasting, I put it in a gist:
Something like:
{"<kind>" : "Map", "<mapData>" : [ [ key1, value1], [ keyN, valueN] ] }is arguably a better because then you can write a reviver function that recognizes it:
function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]) { case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(function(e){newValue.set(e[0], e[1])}); return newValue; } default: return value; } } function mapReviver(key, value) { if (typeof value != "object") return value; switch (value["<kind>"]){ case undefined: return value; case "Map": { let newValue = new Map; let mapData = value["<mapData>"]; if (!mapData) return value; mapData.forEach(e=>newValue.set(e[0], e[1])); return newValue; } default: return value; } }
var tree = JSON.parse(sourceString, mapReviver);
But, this still doesn't explain how an object-as-key gets its reference put back into (the correct) scope :)
Ah, it doesn't by itself. JSON serialization doesn't preserve identify either within a single JSON document or with source or target object context. Admittedly an issue if you need to serialize a Map that has object keys. You need to add an identify mapping layer to the JSON scheme you produce to make something like that work. Again, these are things that probably don't have a general solution and are probably better handled close to the application layer.
After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as:
Set.prototype.toJSON = function() { return Array.from(this); };
That way, JSON.stringify() would do something rational by default when used with sets.
Thoughts?
Thanks, Nicholas
After a little more experimenting with sets (still a really big fan!!), I've come across an interesting problem. Basically, I found myself using a set and then wanting to convert that into JSON for storage. JSON.stringify() run on a set returns "{}", because it's an object without any enumerable properties. I'm wondering if that's the correct behavior because a set is really more like an array than it is an object, and perhaps it would be best to define a toJSON() method for sets such as: Set.prototype.toJSON = function() { return Array.from(this); }; That way, JSON.stringify() would do something rational by default when used with sets. Thoughts? Thanks, Nicholas -- ___________________________ Nicholas C. Zakas http://www.nczonline.net