Comments to the JSON related proposals
Simon Bünzli scripsit:
- toJSONString needs an optional filter argument
+1
- The optional filter argument should allow both function and blacklist
+0
- |undefined|, functions, etc. should not be silently dropped
+1
- Date should not be serializable
+1
- Pretty printing shouldn't default to 4 spaces
+0
- Filter functions should be able to delete keys by returning |undefined|
+1
doekman at gmail.com schrieb am 19.08.07 16:49:
- The optional filter argument should allow both function and blacklist Why can't a function define a blacklist?
It can (given that my proposal #6 is implemented). For simple filtering an Array is enough, though, and should perform better than a function (a string array can be converted into a native [e.g. C++] hash for speedy querying - as opposed to having to call a JavaScript function for every key).
And as I said: I'd prefer a blacklist to a whitelist as IME the use cases for a whitelist will rather require context (e.g. type and number of children) opposed to a blacklist for just getting rid of extensions to Object.prototype or temporary keys.
I think a filter can have (from a blacklist point of view):
- string: case sensitive exclusion
- regular expression: pattern exclusion, and case insensitive stuff
- function: for even more complicated exclusions
Sure, as a third alternative a RegExp could be considered (instead of String Array or Function), which would lead to the following three calling options:
x.toJSONString(function(key, value) { return ["exclude", "these", "keys"].indexOf(key) == -1 ? value : undefined; }); // this specific use case shouldn't require a function at all
x.toJSONString(["exclude", "these", "keys"]);
x.toJSONString(/^(exclude|these|keys)$/i);
and we could combine them with an array: x.toJSONString( { exclude: ['name','email',/^.+code$/i,function(p) { return p.charAt(0)!=p.charAt(1); } ] } );
Looks slightly over-engineered to me.
the object-syntax is to deal with options (include, exclude and prettyPrint).
If you only want to pretty-print, that'd be
x.toJSONString(null, true); or x.toJSONString([], true);
~sb
On 8/19/07, Simon Bünzli <zeniko at gmail.com> wrote: [...]
And as I said: I'd prefer a blacklist to a whitelist as IME the use cases for a whitelist will rather require context (e.g. type and number of children) opposed to a blacklist for just getting rid of extensions to Object.prototype or temporary keys.
would not a whitelist "disallow everything except N in this list" be more secure than a blacklist "allow everything except N in this list" ?
not that is that much important with JSON as you can have only one local context, but still for some peope willing to extend JSON to more than "one local context", a whitelist would be prefered imho.
zwetan
where a filter might be used which have quite different needs: in the case of temporary/internal keys scattered in an object, it might be easier to just specify those keys and be done with it
Specifying temporary/internal keys on serialization is generally a poor approach, but rather the temporal natural of keys should be a part of the definition of object behavoir/type. I believe ES4 should have a property modifier "transient" (like Java), and then the properties would be defined as temporary in the class definition and the serialization should automatically omit transient properties. To illustrate this further, if one were to write an alternate serialization method toXMLString and the object that included properties that were temporary was serialized, it should not be necessary to respecify which properties were temporary for the toXMLString call. Rather that could be defined when writing a the class for the object: class MyClass{ int permanent; transient int exclude,these,keys; }
Since JSON is a serialization format, encoding and decoding should be fully reversible - or else you risk running into unexpected subtle errors which make JSON related code more fragile than necessary.
It seems to me that JSON is not even close to attempting to be reversible for the general JavaScript objects, but only for simple data structures. The list of things that JSON can not serialize in ES3 (and then deserialize to original state) includes circular references, multiple references to single objects, dates, functions, objects with prototypes, arrays with strings keyed properties, and so on. In ES4 the gap grows even wider with typed objects and namespaced keys (IMHO the namespaced keyed objects of ES4 almost feels to me conceptually incompatible to string keyed objects of JSON).
Kris
Hey Kris, Simon; a few comments:
On Aug 19, 2007, at 9:45 PM, Kris Zyp wrote:
where a filter might be used which have quite different needs: in
the case of temporary/internal keys scattered in an object, it might be
easier to just specify those keys and be done with it Specifying temporary/internal keys on serialization is generally a
poor approach, but rather the temporal natural of keys should be a part
of the definition of object behavoir/type. I believe ES4 should have a
property modifier "transient" (like Java), and then the properties would be
defined as temporary in the class definition
Who said anything about 'class'? :-)
JSON is and remains class-less, serializing Object and Array data
definitions, whether created by object and array initialisers or not.
So transient doesn't apply to array, but given the ongoing lack of
requirement for class-based programming, and the low odds of that
style taking over for JS apps producing and consuming JSON, the
question arises: How would ad-hoc transient properties be set or
initialized in any old object?
It seems to me that JSON is not even close to attempting to be
reversible for the general JavaScript objects, but only for simple data
structures. The list of things that JSON can not serialize in ES3 (and then
deserialize to original state) includes circular references, multiple references
to single objects, dates, functions, objects with prototypes, arrays with
strings keyed properties, and so on. In ES4 the gap grows even wider with
typed objects and namespaced keys (IMHO the namespaced keyed objects of
ES4 almost feels to me conceptually incompatible to string keyed objects of
JSON).
Superset, not incompatible. Even in ES3 there are types that don't
exist in JSON, as well as other kinds of structures that can't be
serialized, as you note. JSON is not going to change much, AFAICT.
The challenge for any language trying to provide safe and usable APIs
to produce and consume it is what to do when the input to the
serializer can't be represented. Dropping a namespace qualifier is
bad because it will lead to local name collisions (Murphy says). In
general, silently dropping names and types that don't fit in JSON
seems like deadly silence, not the golden kind.
question arises: How would ad-hoc transient properties be set or initialized in any old object?
With classes being available, why not provide a pathway for developers to define transience correctly in proper OO manner that would be serialization method agnostic, rather than adding a blacklisted array parameter to toJSONString, which IMHO is very poor and shortsighted way of defining transient/temporary keys? You are right that it would not available for plain old objects, or perhaps it could be. Transience would be a property attribute, and could be set just like defining enumerability of properties, although I would hate to suggest another method on Object.prototype.
Dropping a namespace qualifier is bad because it will lead to local name collisions (Murphy says). In general, silently dropping names and types that don't fit in JSON seems like deadly silence, not the golden kind.
I thought that dropping namespace qualifier had already been decided on. Is this still in question? I certainly agree that dropping namespaces seems dangerous, and when I asked before it was suggested that there could just simply be multiple identical keys in a JSON serialization output. Seems a little odd to stringify to something that is not even coherent JSON. Kris
Kris Zyp schrieb am 20.08.07 16:57:
question arises: How would ad-hoc transient properties be set or initialized in any old object? With classes being available, why not provide a pathway for developers to define transience correctly in proper OO manner that would be serialization method agnostic, rather than adding a blacklisted array parameter to toJSONString, which IMHO is very poor and shortsighted way of defining transient/temporary keys?
What you suggest sounds like a good enhancement to what I've proposed (in fact, I hadn't considered ECMAScript as an OOL at all). I'd still keep the blacklist though for the following reasons:
-
Legacy-code (i.e. ECMAScript 3 code) may want to take advantage of toJSONString without converting all expando'd objects to classes.
-
Having to set transciency for temporary/internal attributes may result in code in places where it doesn't really matter or even belong at all.
E.g. in our use case (Firefox' SessionStore service), JSON export has been modularly added and takes advantage of the fact that our internal data structures nicely map into JSON. Inside the core service, the temporary keys aren't transient per se - they just shouldn't be serialized for our JSON API. For different API, we might even to serialize a different set of keys, which would not be easily possible with setting transciency in the core code. So either we have a blacklist (or a less efficient way for filtering out keys), or we'd have to clone our object first which won't be more efficient than just doing the JSON conversion ourselves in JavaScript.
- Even for code written against ECMAScript 4, you'd force a strict class model where expando'd objects would be sufficient (not everybody likes the rigidity of an OO class model - especially for RAD):
var obj = { some: 1, key: "text" }; obj.expandoed = [-1, -2, -3]; // potentially lots and lots of unrelated code... obj.toJSONString(); // how to temporarily ignore some keys here?
~sb
What if instead of only exposing property enumerability we exposed all the property attributes that were previously hidden in ES3 (and included "transient" as one of those properties)? obj={tempKey:1,myConst:2,hidden:3}; obj.setAttributes("tempKey",Properties.TRANSIENT); // define transient without a class obj.setAttribute("myConst",Properties.READONLY); // we could define constants without classes and get rid of propertyIsEnumerable: obj.propertyIsEnumerable("hidden",false) ->
obj.setAttributes("hidden",Properties.DONTENUM);
The toJSONString for objects could then include a check (as well as any other serialization schemes that users might write): if (!(obj.getAttributes(key) & Properties.TRANSIENT)) ..serialization...
I would love this, but I am guessing that most would think this a little bit too much power for users... Kris
----- Original Message ---
On Aug 20, 2007, at 7:57 AM, Kris Zyp wrote:
question arises: How would ad-hoc transient properties be set or initialized in any old object? With classes being available, why not provide a pathway for
developers to define transience correctly in proper OO manner that would be
serialization method agnostic, rather than adding a blacklisted array parameter to toJSONString, which IMHO is very poor and shortsighted way of defining transient/temporary keys?
Proper OO beliefs aside, you could justify lots of annotations other
than type annotations for things like serialization, documentation,
etc. We have a deferred proposal for documentation, for example:
proposals:documentation
So yeah, one might want both
class Foo { transient var bar: ... }
and
let obj = {transient baz: 42, ... };
I thought that dropping namespace qualifier had already been
decided on.
We're not done with this proposal, and anyway we revisit decisions
when there's good reason.
Is this still in question? I certainly agree that dropping namespaces
seems dangerous, and when I asked before it was suggested that there
could just simply be multiple identical keys in a JSON serialization output.
Seems a little odd to stringify to something that is not even coherent JSON.
I agree.
On 8/20/07, Kris Zyp <kriszyp at xucia.com> wrote:
What if instead of only exposing property enumerability we exposed all the property attributes that were previously hidden in ES3 (and included "transient" as one of those properties)? obj={tempKey:1,myConst:2,hidden:3}; obj.setAttributes("tempKey",Properties.TRANSIENT); // define transient without a class obj.setAttribute("myConst",Properties.READONLY); // we could define constants without classes and get rid of propertyIsEnumerable: obj.propertyIsEnumerable("hidden",false) -> obj.setAttributes("hidden",Properties.DONTENUM);
The toJSONString for objects could then include a check (as well as any other serialization schemes that users might write): if (!(obj.getAttributes(key) & Properties.TRANSIENT)) ..serialization...
I would love this, but I am guessing that most would think this a little bit too much power for users...
+1 for being able to set the attributes
but I think we should not add a TRANSIENT attributes for ES3, DONTENUM should be enought and backward compatible
ActionScript 1 and 2 allow to do such thing with the "undocumented" ASSetPropFlags osflash.org/flashcoders/undocumented/assetpropflags
and no that not too much power to the user =)
zwetan
On Aug 21, 2007, at 1:24 AM, zwetan wrote:
+1 for being able to set the attributes
but I think we should not add a TRANSIENT attributes for ES3, DONTENUM should be enought and backward compatible
On 8/21/07, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 21, 2007, at 1:24 AM, zwetan wrote:
+1 for being able to set the attributes
but I think we should not add a TRANSIENT attributes for ES3, DONTENUM should be enought and backward compatible
ok, but is it still at the proposal stage or does DontEnum() and DontDelete() are accepted :) ?
also to be "complete" a ReadOnly() could also be usefull, I mean also in the context of JSON serialization/deserialization to protect some properties to be overriden by the parsing of a JSON string
use case: a config file with protected properties you could load a JSON string to change the configuration but some parameters would just be protected by the ReadOnly() it could even be more efficient than a whitelist or blacklist
zwetan
but I think we should not add a TRANSIENT attributes for ES3, DONTENUM should be enought and backward compatible
If we added the ability to set attributes, the propertyIsEnumerable extra parameter proposal would be unnecessary (albiet convenient). Also, IMHO, transient and dontenum are different concepts. They both affect enumeration, but one in the context of introspection and one in the context of serialization. Kris
On Aug 21, 2007, at 1:36 PM, zwetan wrote:
On 8/21/07, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 21, 2007, at 1:24 AM, zwetan wrote:
+1 for being able to set the attributes
but I think we should not add a TRANSIENT attributes for ES3, DONTENUM should be enought and backward compatible
ok, but is it still at the proposal stage or does DontEnum() and
DontDelete() are accepted :) ?
Not likely. We aren't adding more properties if we can help it.
DontDelete is dangerous -- it is necessary for integrity properties
on which security depends, unlike DontEnum.
also to be "complete" a ReadOnly() could also be usefull,
Also bad news for integrity.
I mean also in the context of JSON serialization/deserialization to protect some properties to be overriden by the parsing of a JSON
string
JSON is not being extended, but we have
let obj = {const FOO: 42, ...};
so you can make read-only properties in object initialisers.
We must keep DontDelete and ReadOnly invariant. See my reply to zwetan.
Always invariant? What's wrong with being able to turn on ReadOnly, but not turn it off? Seems I recall seeing code in Rhino that already behaved that way... Kris ----- Original Message ---
On Aug 22, 2007, at 12:20 PM, Kris Zyp wrote:
Always invariant? What's wrong with being able to turn on ReadOnly,
but not turn it off? Seems I recall seeing code in Rhino that already
behaved that way... Kris
Turning off is the problem for integrity, but turning on is a pain
for any number of implementation and usability reasons. What's wrong
with initialiser and declarative support for on-from-the-start?
Brendan asked me to comment on the proposals at [1] after having seen that I had implemented several aspects slightly differently for Firefox' Session Restore functionality (which offers a JSON-based API [2]). This implementation in JavaScript is planned to be used for the Mozilla platform until the JSON proposal is finalized and natively implemented [3].
JSON was a nice fit for our API as we internally use an object tree to store the various information related to all open windows. For efficiency and simplicity reasons, we do however store certain non-serializable objects in that tree as well which required us to implement a filter on toJSONString (see comments #1 and #2 below).
With the code being added to the Mozilla platform, I made some additional changes to make it IMO slightly less fragile to use - that's comments #3 and #4. The other two comments #5 and #6 are unrelated suggestions which we haven't implemented ourselves yet.
My 6 modification suggestions are:
This has already been discussed but apparently not made it into the proposal itself. To reiterate the main reason: without a filter, an object containing temporary/internal keys has to be duplicated first before it can be encoded - and duplicating it or JSON encoding it yourself will of about the same complexity, so you lose the convenience of either being able to add internal keys or of using toJSONString.
This shouldn't be an either/or choice, as there are several use cases where a filter might be used which have quite different needs: in the case of temporary/internal keys scattered in an object, it might be easier to just specify those keys and be done with it - whereas if you want to ensure a contract (and thus prefer a whitelist), you'll need a way to consider a key's context which you can only do in a function anyway (unless you want to introduce XPath4JSON).
In the first case you furthermore most probably use less temporary keys than actually relevant ones which makes the blacklist both smaller/better to oversee and speedier than a whitelist.
Since JSON is a serialization format, encoding and decoding should be fully reversible - or else you risk running into unexpected subtle errors which make JSON related code more fragile than necessary.
Should objects to be serialized for some reason contain functions, they should rather be blacklisted so that it is explicit what will be dropped.
|undefined| OTOH mostly appears in sparse Arrays which don't exist in many other languages and which you thus shouldn't be using for interoperability's sake, anyway. If you however depend on sparse Arrays (i.e. if you couldn't for clarity rewrite your code to use dense Arrays), you again risk subtle errors if you're not aware that you won't get them back as you might expect.
Only possible exception to this rule: dropping |undefined| at the very end of an Array for the use case where you preallocate an Array larger than you need for efficiency reasons.
Same reason as above: as long as it's not unambiguously deserializable, serializing it leads to unnecessary fragility. Having instead a filter function on toJSONString analogous to parseJSON's would nicely restore the balance for easy-to-read encoding/decoding.
Indentation is always a matter of taste and should thus not default unchangeably to an arbitrary value. Two better options: either indent with a single tab (which can then easily be replaced with as many spaces as you like as there won't be any other unencoded tabs in the returned string); or accept an integer value as argument indicating how to indent where non-negative numbers mean spaces and negative numbers mean to use as many tabs for indentation.
|undefined| is an illegal JSON value and can thus be used safely for this purpose, allowing you to do completely without Array.filter or similar both for encoding and decoding. This would mostly be used for encoding where with a filter function you can choose yourself to implement a whitelist (or an extended blacklist).
Criticism and comments are welcome. In case any of those arguments has already been brought forward, please kindly ignore them.
~sb
[1] proposals:json_encoding_and_decoding [2] developer.mozilla.org/en/docs/nsISessionStore [3] bugzilla.mozilla.org/show_bug.cgi?id=386789