Property Iteration in JSON serialization

# Brian Kardell (15 years ago)

Because the serialized form (json text) and the native source form (a specific subset of object literals) are essentially the same, I have occasionally heard people "in the wild" refer to both literals themselves and instances of an object parsed from json text as "json". Clearly the two are different things which leads me to my question: With new built in json supports one can serialize an instance of an object as json text and provide some serialization criteria (a whilelist or a replacer function) - on the other end easily de-serialize that back into instance. Great! One of the great things about json is that it's quite a readable format and, as Doug Crockford points out, expresses things in ways that they are naturally thought about in most modern (especially C-family) languages - so it's very easy to parse from a machine point of view too and get right to a useful instance instead of some DOM tree of string values.

So - keeping in mind that the instance and the serialization are separate things - what if I would like to order my serialization keys (property names) in natural order? It has no impact on the parsing or internal representation, but imposing a known order on the serialization, even a trivial one, makes things easier to find and documentation easier to understand. We've implemented this in a few local serializers in multiple languages and, since during serialization we are writing them out by iterating the keys, it's actually quite trivial to merely call sort() on the keys before iteration. Likewise, since this only has to be done once per type - not once per instance, there is very, very minimal overhead (you are generally sorting less than 20 keys regardless of the number of instances).

There are potentially some additional practical upshots beyond human readability to this which I won't get into here until I find out: Is it even plausible to accomplish this with new built in JSON supports?

To be specific, if it's not clear... Given an instance of an order object, I would like to serialize it like this:

{ "items": [ { "description": "A bottle of rum", "itemId": 13523, "price": 12.94, "quantity": 6 }, { "description": "A bottle of whiskey", "itemId": 23423, "price": 6.95, "quantity": 24 } ], "orderNumber": 1234123, "shipTo": { "city": "Seattle", "name": "Jane Doe", "state": "Washington", "streetAddress": "123 Main Street", "zip": 12345 } }

Because the serialized form (json text) and the native source form (a
specific subset of object literals) are essentially the same, I have
occasionally heard people "in the wild" refer to both literals
themselves and instances of an object parsed from json text as "json".
 Clearly the two are different things which leads me to my question:
With new built in json supports one can serialize an instance of an
object as json text and provide some serialization criteria (a
whilelist or a replacer function) - on the other end easily
de-serialize that back into instance. Great!  One of the great things
about json is that it's quite a readable format and, as Doug Crockford
points out, expresses things in ways that they are naturally thought
about in most modern (especially C-family) languages - so it's very
easy to parse from a machine point of view too and get right to a
useful instance instead of some DOM tree of string values.

So - keeping in mind that the instance and the serialization are
separate things - what if I would like to order my serialization keys
(property names) in natural order?  It has no impact on the parsing or
internal representation, but imposing a known order on the
serialization, even a trivial one, makes things easier to find and
documentation easier to understand.  We've implemented this in a few
local serializers in multiple languages and, since during
serialization we are writing them out by iterating the keys, it's
actually quite trivial to merely call sort() on the keys before
iteration.  Likewise, since this only has to be done once per type -
not once per instance, there is very, very minimal overhead (you are
generally sorting less than 20 keys regardless of the number of
instances).

There are potentially some additional practical upshots beyond human
readability to this which I won't get into here until I find out:  Is
it even plausible to accomplish this with new built in JSON supports?

To be specific, if it's not clear... Given an instance of an order
object, I would like to serialize it like this:

{
    "items": [
        {
            "description": "A bottle of rum",
            "itemId":  13523,
            "price": 12.94,
            "quantity": 6
        },
        {
            "description": "A bottle of whiskey",
            "itemId":  23423,
            "price": 6.95,
            "quantity": 24
        }
    ],
    "orderNumber": 1234123,
    "shipTo": {
        "city": "Seattle",
        "name": "Jane Doe",
        "state": "Washington",
        "streetAddress": "123 Main Street",
        "zip": 12345
    }
}

# David-Sarah Hopwood (15 years ago)

Brian Kardell wrote:

So - keeping in mind that the instance and the serialization are separate things - what if I would like to order my serialization keys (property names) in natural order? It has no impact on the parsing or internal representation, but imposing a known order on the serialization, even a trivial one, makes things easier to find and documentation easier to understand. We've implemented this in a few local serializers in multiple languages and, since during serialization we are writing them out by iterating the keys, it's actually quite trivial to merely call sort() on the keys before iteration. Likewise, since this only has to be done once per type - not once per instance, there is very, very minimal overhead (you are generally sorting less than 20 keys regardless of the number of instances).

Well, JS doesn't have types in that sense. So, unless an implementation were to exploit any internal optimizations it has for recognizing objects of the same "shape" [*], it would indeed have to sort the keys of every instance.

[*] Do any common implementations actually do that, other than for packed arrays?

Brian Kardell wrote:
> So - keeping in mind that the instance and the serialization are
> separate things - what if I would like to order my serialization keys
> (property names) in natural order?  It has no impact on the parsing or
> internal representation, but imposing a known order on the
> serialization, even a trivial one, makes things easier to find and
> documentation easier to understand.  We've implemented this in a few
> local serializers in multiple languages and, since during
> serialization we are writing them out by iterating the keys, it's
> actually quite trivial to merely call sort() on the keys before
> iteration.  Likewise, since this only has to be done once per type -
> not once per instance, there is very, very minimal overhead (you are
> generally sorting less than 20 keys regardless of the number of
> instances).

Well, JS doesn't have types in that sense. So, unless an implementation
were to exploit any internal optimizations it has for recognizing objects
of the same "shape" [*], it would indeed have to sort the keys of every
instance.


[*] Do any common implementations actually do that, other than for
    packed arrays?

-- 
David-Sarah Hopwood  ⚥  http://davidsarah.livejournal.com

# Graydon Hoare (15 years ago)

David-Sarah Hopwood wrote:

[*] Do any common implementations actually do that, other than for packed arrays?

Yes. All the fast ones. Nitro (nee sfx), v8 and spidermonkey all synthesize structural type descriptors ("shapes") at runtime and store packed instances, tagged by shape. It's an old trick and pays off well.

(This comment is orthogonal to any conversation about JSON, mind you.)

David-Sarah Hopwood wrote:

> [*] Do any common implementations actually do that, other than for
>     packed arrays?

Yes. All the fast ones. Nitro (nee sfx), v8 and spidermonkey all 
synthesize structural type descriptors ("shapes") at runtime and store 
packed instances, tagged by shape. It's an old trick and pays off well.

(This comment is orthogonal to any conversation about JSON, mind you.)

-Graydon

# Luke Smith (15 years ago)

On Oct 13, 2009, at 9:24 PM, Brian Kardell wrote:

There are potentially some additional practical upshots beyond human readability to this which I won't get into here until I find out: Is it even plausible to accomplish this with new built in JSON supports?

To be specific, if it's not clear... Given an instance of an order object, I would like to serialize it like this:

{ "items": [ { "description": "A bottle of rum", "itemId": 13523, "price": 12.94, "quantity": 6 }, { "description": "A bottle of whiskey", "itemId": 23423, "price": 6.95, "quantity": 24 } ], "orderNumber": 1234123, "shipTo": { "city": "Seattle", "name": "Jane Doe", "state": "Washington", "streetAddress": "123 Main Street", "zip": 12345 } }

It is possible to sort object keys in a replacer function. See gist.github.com/209826

Currently FF3.5.4 doesn't properly apply replacer functions, but
Safari 4, WebKit, IE8, and Chrome 3 work fine for this task.

Luke

On Oct 13, 2009, at 9:24 PM, Brian Kardell wrote:

>
> There are potentially some additional practical upshots beyond human
> readability to this which I won't get into here until I find out:  Is
> it even plausible to accomplish this with new built in JSON supports?
>
> To be specific, if it's not clear... Given an instance of an order
> object, I would like to serialize it like this:
>
> {
>    "items": [
>        {
>            "description": "A bottle of rum",
>            "itemId":  13523,
>            "price": 12.94,
>            "quantity": 6
>        },
>        {
>            "description": "A bottle of whiskey",
>            "itemId":  23423,
>            "price": 6.95,
>            "quantity": 24
>        }
>    ],
>    "orderNumber": 1234123,
>    "shipTo": {
>        "city": "Seattle",
>        "name": "Jane Doe",
>        "state": "Washington",
>        "streetAddress": "123 Main Street",
>        "zip": 12345
>    }
> }

It is possible to sort object keys in a replacer function.  See http://gist.github.com/209826

Currently FF3.5.4 doesn't properly apply replacer functions, but  
Safari 4, WebKit, IE8, and Chrome 3 work fine for this task.

Luke

# Brian Kardell (15 years ago)

On Tue, Oct 13, 2009 at 10:54 PM, Luke Smith <lsmith at lucassmith.name> wrote:

On Oct 13, 2009, at 9:24 PM, Brian Kardell wrote:

There are potentially some additional practical upshots beyond human readability to this which I won't get into here until I find out: Is it even plausible to accomplish this with new built in JSON supports?

To be specific, if it's not clear... Given an instance of an order object, I would like to serialize it like this:

{ "items": [ { "description": "A bottle of rum", "itemId": 13523, "price": 12.94, "quantity": 6 }, { "description": "A bottle of whiskey", "itemId": 23423, "price": 6.95, "quantity": 24 } ], "orderNumber": 1234123, "shipTo": { "city": "Seattle", "name": "Jane Doe", "state": "Washington", "streetAddress": "123 Main Street", "zip": 12345 } }

It is possible to sort object keys in a replacer function. See gist.github.com/209826

Currently FF3.5.4 doesn't properly apply replacer functions, but Safari 4, WebKit, IE8, and Chrome 3 work fine for this task.

Luke

But I guess your example gets to the gist (ironically, given the url) of the question: It seems to be implying that there is a specific (insertion) order maintained by the instance itself and reflected in iteration... The difference being something more like a TreeMap than a HashMap - but is that part of the specification? I thought that it specifically wasn't.

On Tue, Oct 13, 2009 at 10:54 PM, Luke Smith <lsmith at lucassmith.name> wrote:
>
> On Oct 13, 2009, at 9:24 PM, Brian Kardell wrote:
>
>>
>> There are potentially some additional practical upshots beyond human
>> readability to this which I won't get into here until I find out:  Is
>> it even plausible to accomplish this with new built in JSON supports?
>>
>> To be specific, if it's not clear... Given an instance of an order
>> object, I would like to serialize it like this:
>>
>> {
>>   "items": [
>>       {
>>           "description": "A bottle of rum",
>>           "itemId":  13523,
>>           "price": 12.94,
>>           "quantity": 6
>>       },
>>       {
>>           "description": "A bottle of whiskey",
>>           "itemId":  23423,
>>           "price": 6.95,
>>           "quantity": 24
>>       }
>>   ],
>>   "orderNumber": 1234123,
>>   "shipTo": {
>>       "city": "Seattle",
>>       "name": "Jane Doe",
>>       "state": "Washington",
>>       "streetAddress": "123 Main Street",
>>       "zip": 12345
>>   }
>> }
>
> It is possible to sort object keys in a replacer function.  See
> http://gist.github.com/209826
>
> Currently FF3.5.4 doesn't properly apply replacer functions, but Safari 4,
> WebKit, IE8, and Chrome 3 work fine for this task.
>
> Luke
>

But I guess your example gets to the gist (ironically, given the url)
of the question: It seems to be implying that there is a specific
(insertion) order maintained by the instance itself and reflected in
iteration... The difference being something more like a TreeMap than a
HashMap - but is that part of the specification?  I thought that it
specifically wasn't.

# Oliver Hunt (15 years ago)

Currently FF3.5.4 doesn't properly apply replacer functions, but
Safari 4, WebKit, IE8, and Chrome 3 work fine for this task.

For the purpose of discussion webkit and safari (and epiphany, and
arora, ...) are synonymous -- chrome represents the only version of
webkit using a unique JS engine. I think for that reason it's better
to distinguish based on JS engine rather the browser: SpiderMonkey,
JavaScriptCore, V8, and JScript all support JSON natively nowadays --
unsure about any opera betas atm, nor what the JS engine in O is named,

That said, the Safari 4 branch of webkit (which i believe is used as
the stable branch for WebKit/qt as well) has a bug where it provides
the global object rather than the holder object as the this argument
to the replacer function.

> Currently FF3.5.4 doesn't properly apply replacer functions, but  
> Safari 4, WebKit, IE8, and Chrome 3 work fine for this task.

For the purpose of discussion webkit and safari (and epiphany, and  
arora, ...) are synonymous -- chrome represents the only version of  
webkit using a unique JS engine.  I think for that reason it's better  
to distinguish based on JS engine rather the browser: SpiderMonkey,  
JavaScriptCore, V8, and JScript all support JSON natively nowadays --  
unsure about any opera betas atm, nor what the JS engine in O is named,

That said, the Safari 4 branch of webkit (which i believe is used as  
the stable branch for WebKit/qt as well) has a bug where it provides  
the global object rather than the holder object as the `this` argument  
to the replacer function.

--Oliver

>
> Luke
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss

# Oliver Hunt (15 years ago)

But I guess your example gets to the gist (ironically, given the url) of the question: It seems to be implying that there is a specific (insertion) order maintained by the instance itself and reflected in iteration... The difference being something more like a TreeMap than a HashMap - but is that part of the specification? I thought that it specifically wasn't.

All ES implementations are required to enumerate object properties in
the order of insertion, regardless of internal representation, and the
JSON object definition defines the replacer execution in terms of that
standard object enumeration.

> But I guess your example gets to the gist (ironically, given the url)
> of the question: It seems to be implying that there is a specific
> (insertion) order maintained by the instance itself and reflected in
> iteration... The difference being something more like a TreeMap than a
> HashMap - but is that part of the specification?  I thought that it
> specifically wasn't.

All ES implementations are required to enumerate object properties in  
the order of insertion, regardless of internal representation, and the  
JSON object definition defines the replacer execution in terms of that  
standard object enumeration.

--Oliver

> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss

# Brian Kardell (15 years ago)

Awesome - just the answer I was looking for Oliver -

Awesome - just the answer I was looking for Oliver - thanks!


On Tue, Oct 13, 2009 at 11:42 PM, Oliver Hunt <oliver at apple.com> wrote:
>> But I guess your example gets to the gist (ironically, given the url)
>> of the question: It seems to be implying that there is a specific
>> (insertion) order maintained by the instance itself and reflected in
>> iteration... The difference being something more like a TreeMap than a
>> HashMap - but is that part of the specification?  I thought that it
>> specifically wasn't.
>
> All ES implementations are required to enumerate object properties in the
> order of insertion, regardless of internal representation, and the JSON
> object definition defines the replacer execution in terms of that standard
> object enumeration.
>
> --Oliver
>
>> _______________________________________________
>> es-discuss mailing list
>> es-discuss at mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>
>

# Patrick Mueller (15 years ago)

On Oct 14, 2009, at 2:42 AM, Oliver Hunt wrote:

All ES implementations are required to enumerate object properties
in the order of insertion, regardless of internal representation,
and the JSON object definition defines the replacer execution in
terms of that standard object enumeration.

I understand that's what all (or most) of the implementations actually
do, but the ES5 spec I'm looking at - Ecma/TC39/2009/043 - doesn't
actually say it's required.

In 12.6.4 discussing the for-in statement, it says: "The mechanics and
order of enumerating the properties (step 6.a in the first algorithm,
step 7.a in the second) is not specified."

In 15.2.3.14 discussion Object.keys(), it says: "If an implementation
defines a specific order of enumeration for the for-in statement, that
same enumeration order must be used in step 5 of this algorithm."

In 15.12.2 discussing JSON.parse(), it says: "The ordering of the
Strings should be the same as that used by the Object.keys standard
built-in function."

In 15.12.3 discussing JSON.stringify(), it says: "The ordering of the
Strings should be the same as that used by the Object.keys standard
built-in function."

Sorry, property ordering is an unfortunate pet peeve of mine. :-)

Patrick Mueller - muellerware.org

On Oct 14, 2009, at 2:42 AM, Oliver Hunt wrote:

> All ES implementations are required to enumerate object properties  
> in the order of insertion, regardless of internal representation,  
> and the JSON object definition defines the replacer execution in  
> terms of that standard object enumeration.

I understand that's what all (or most) of the implementations actually  
do, but the ES5 spec I'm looking at - Ecma/TC39/2009/043 - doesn't  
actually say it's required.

In 12.6.4 discussing the for-in statement, it says: "The mechanics and  
order of enumerating the properties (step 6.a in the first algorithm,  
step 7.a in the second) is not specified."

In 15.2.3.14 discussion Object.keys(), it says: "If an implementation  
defines a specific order of enumeration for the for-in statement, that  
same enumeration order must be used in step 5 of this algorithm."

In 15.12.2 discussing JSON.parse(), it says: "The ordering of the  
Strings should be the same as that used by the Object.keys standard  
built-in function."

In 15.12.3 discussing JSON.stringify(), it says: "The ordering of the  
Strings should be the same as that used by the Object.keys standard  
built-in function."

Sorry, property ordering is an unfortunate pet peeve of mine.  :-)

Patrick Mueller - http://muellerware.org/

# Brian Kardell (15 years ago)

On Wed, Oct 14, 2009 at 5:41 AM, Patrick Mueller <pmuellr at yahoo.com> wrote:

On Oct 14, 2009, at 2:42 AM, Oliver Hunt wrote:

All ES implementations are required to enumerate object properties in the order of insertion, regardless of internal representation, and the JSON object definition defines the replacer execution in terms of that standard object enumeration.

I understand that's what all (or most) of the implementations actually do, but the ES5 spec I'm looking at - Ecma/TC39/2009/043 - doesn't actually say it's required.

In 12.6.4 discussing the for-in statement, it says: "The mechanics and order of enumerating the properties (step 6.a in the first algorithm, step 7.a in the second) is not specified."

In 15.2.3.14 discussion Object.keys(), it says: "If an implementation defines a specific order of enumeration for the for-in statement, that same enumeration order must be used in step 5 of this algorithm."

In 15.12.2 discussing JSON.parse(), it says: "The ordering of the Strings should be the same as that used by the Object.keys standard built-in function."

In 15.12.3 discussing JSON.stringify(), it says: "The ordering of the Strings should be the same as that used by the Object.keys standard built-in function."

Sorry, property ordering is an unfortunate pet peeve of mine. :-)

Patrick Mueller - muellerware.org

Uh oh... Patrick looking at the same parts as me reacting the same way (I believe older pre-5 docs also said something similar)... When Oliver said "All ES implementations are required to..." I assumed that I missed something in the new 5 docs - not that everyone just seems to have implemented roughly the same thing...

Is it required - or is it not?

On Wed, Oct 14, 2009 at 5:41 AM, Patrick Mueller <pmuellr at yahoo.com> wrote:
> On Oct 14, 2009, at 2:42 AM, Oliver Hunt wrote:
>
>> All ES implementations are required to enumerate object properties in the
>> order of insertion, regardless of internal representation, and the JSON
>> object definition defines the replacer execution in terms of that standard
>> object enumeration.
>
> I understand that's what all (or most) of the implementations actually do,
> but the ES5 spec I'm looking at - Ecma/TC39/2009/043 - doesn't actually say
> it's required.
>
> In 12.6.4 discussing the for-in statement, it says: "The mechanics and order
> of enumerating the properties (step 6.a in the first algorithm, step 7.a in
> the second) is not specified."
>
> In 15.2.3.14 discussion Object.keys(), it says: "If an implementation
> defines a specific order of enumeration for the for-in statement, that same
> enumeration order must be used in step 5 of this algorithm."
>
> In 15.12.2 discussing JSON.parse(), it says: "The ordering of the Strings
> should be the same as that used by the Object.keys standard built-in
> function."
>
> In 15.12.3 discussing JSON.stringify(), it says: "The ordering of the
> Strings should be the same as that used by the Object.keys standard built-in
> function."
>
> Sorry, property ordering is an unfortunate pet peeve of mine.  :-)
>
> Patrick Mueller - http://muellerware.org/
>
>

Uh oh... Patrick looking at the same parts as me reacting the same way
(I believe older pre-5 docs also said something similar)...
When Oliver said "All ES implementations are required to..." I assumed
that I missed something in the new 5 docs - not that everyone just
seems to have implemented roughly the same thing...

Is it required - or is it not?

# Brendan Eich (15 years ago)

We talked about requiring it. It's a de-facto standard, followed for
direct ("own") properties of most objects. Exceptions may include
arrays (possibly only dense) and host object (of course)!

I'm committed to helping ensure we get better for-in specification --
and meta-programming -- in Harmony. For now the de-facto standard of
for-in enumerating in insertion order is pretty strong without a de- jure spec.

We talked about requiring it. It's a de-facto standard, followed for  
direct ("own") properties of most objects. Exceptions may include  
arrays (possibly only dense) and host object (of course)!

I'm committed to helping ensure we get better for-in specification --  
and meta-programming -- in Harmony. For now the de-facto standard of  
for-in enumerating in insertion order is pretty strong without a de- 
jure spec.

/be

On Oct 14, 2009, at 9:27 AM, Brian Kardell wrote:

> On Wed, Oct 14, 2009 at 5:41 AM, Patrick Mueller <pmuellr at yahoo.com>  
> wrote:
>> On Oct 14, 2009, at 2:42 AM, Oliver Hunt wrote:
>>
>>> All ES implementations are required to enumerate object properties  
>>> in the
>>> order of insertion, regardless of internal representation, and the  
>>> JSON
>>> object definition defines the replacer execution in terms of that  
>>> standard
>>> object enumeration.
>>
>> I understand that's what all (or most) of the implementations  
>> actually do,
>> but the ES5 spec I'm looking at - Ecma/TC39/2009/043 - doesn't  
>> actually say
>> it's required.
>>
>> In 12.6.4 discussing the for-in statement, it says: "The mechanics  
>> and order
>> of enumerating the properties (step 6.a in the first algorithm,  
>> step 7.a in
>> the second) is not specified."
>>
>> In 15.2.3.14 discussion Object.keys(), it says: "If an implementation
>> defines a specific order of enumeration for the for-in statement,  
>> that same
>> enumeration order must be used in step 5 of this algorithm."
>>
>> In 15.12.2 discussing JSON.parse(), it says: "The ordering of the  
>> Strings
>> should be the same as that used by the Object.keys standard built-in
>> function."
>>
>> In 15.12.3 discussing JSON.stringify(), it says: "The ordering of the
>> Strings should be the same as that used by the Object.keys standard  
>> built-in
>> function."
>>
>> Sorry, property ordering is an unfortunate pet peeve of mine.  :-)
>>
>> Patrick Mueller - http://muellerware.org/
>>
>>
>
> Uh oh... Patrick looking at the same parts as me reacting the same way
> (I believe older pre-5 docs also said something similar)...
> When Oliver said "All ES implementations are required to..." I assumed
> that I missed something in the new 5 docs - not that everyone just
> seems to have implemented roughly the same thing...
>
> Is it required - or is it not?
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss

# hallvord at opera.com (15 years ago)

Siterer Brendan Eich <brendan at mozilla.com>:

All ES implementations are required to enumerate object properties in the order of insertion, regardless of internal representation

Is it required - or is it not?

It's not (yet) a spec requirement but in our experience it is
definitely required if you want to handle web content. Opera used to
return properties in some implementation-specific order (apparently
random from the script's point of view) and we had to change it
because it caused bugs on GMail and several other sites. (Though I
think we still return numerical properties on regular objects in an
unexpected order and haven't had as many problems with those).

Hallvord R.

Siterer Brendan Eich <brendan at mozilla.com>:

>>>> All ES implementations are required to enumerate object properties in the
>>>> order of insertion, regardless of internal representation

>> Is it required - or is it not?

It's not (yet) a spec requirement but in our experience it is  
definitely required if you want to handle web content. Opera used to  
return properties in some implementation-specific order (apparently  
random from the script's point of view) and we had to change it  
because it caused bugs on GMail and several other sites. (Though I  
think we still return numerical properties on regular objects in an  
unexpected order and haven't had as many problems with those).

- Hallvord R.

# Waldemar Horwat (15 years ago)

Brian Kardell wrote:

Uh oh... Patrick looking at the same parts as me reacting the same way (I believe older pre-5 docs also said something similar)... When Oliver said "All ES implementations are required to..." I assumed that I missed something in the new 5 docs - not that everyone just seems to have implemented roughly the same thing...

Is it required - or is it not?

It is not required. We were planning to make it required for ES5 but it turns out that implementations do not match each other and implementations do not follow insertion order for some kinds of properties such as things that could be used as array indices. Requiring insertion order would make iterating over arrays weird.

Waldemar

Brian Kardell wrote:
> Uh oh... Patrick looking at the same parts as me reacting the same way
> (I believe older pre-5 docs also said something similar)...
> When Oliver said "All ES implementations are required to..." I assumed
> that I missed something in the new 5 docs - not that everyone just
> seems to have implemented roughly the same thing...
> 
> Is it required - or is it not?

It is not required.  We were planning to make it required for ES5 but it turns out that implementations do not match each other and implementations do not follow insertion order for some kinds of properties such as things that could be used as array indices.  Requiring insertion order would make iterating over arrays weird.

    Waldemar

# Brian Kardell (15 years ago)

It sounds to me like there is wide agreement in the sense that at least the basics rules and only disagreement on the fringes... Otherwise no one on this list in particular would be suggesting that there is anything remotely like a "de facto" implementation... It seems that at least those basic rules are required just to function with the existing expectations everywhere.

It also seems that those expectations aren't likely to change any time soon for the "default" for-each iteration order whether more robust and interesting proposals are adopted...

So can't that much be formalized and documented regardless of whether or not new introductions are made to over-ride "other" predictable iterators to be used in for-each (or perhaps even some new mechanism entirely)? The simple fact that conforming to the spec would currently create a non-workable solution seems to argue that at least the "de facto" parts should be included...

No?

It sounds to me like there is wide agreement in the sense that at
least the basics rules and only disagreement on the fringes...
Otherwise no one on this list in particular would be suggesting that
there is anything remotely like a "de facto" implementation... It
seems that at least those basic rules are required just to function
with the existing expectations everywhere.

It also seems that those expectations aren't likely to change any time
soon for the "default" for-each iteration order whether more robust
and interesting proposals are adopted...

So can't that much be formalized and documented regardless of whether
or not new introductions are made to over-ride "other" predictable
iterators to be used in for-each (or perhaps even some new mechanism
entirely)? The simple fact that conforming to the spec would currently
create a non-workable solution seems to argue that at least the "de
facto" parts should be included...

No?

On Wed, Oct 14, 2009 at 2:37 PM, Waldemar Horwat <waldemar at google.com> wrote:
> Brian Kardell wrote:
>>
>> Uh oh... Patrick looking at the same parts as me reacting the same way
>> (I believe older pre-5 docs also said something similar)...
>> When Oliver said "All ES implementations are required to..." I assumed
>> that I missed something in the new 5 docs - not that everyone just
>> seems to have implemented roughly the same thing...
>>
>> Is it required - or is it not?
>
> It is not required.  We were planning to make it required for ES5 but it
> turns out that implementations do not match each other and implementations
> do not follow insertion order for some kinds of properties such as things
> that could be used as array indices.  Requiring insertion order would make
> iterating over arrays weird.
>
>   Waldemar
>

# Brendan Eich (15 years ago)

The issue is not overriding default for-in behavior (as you can do in
JS1.7 via iterator). The issue is the default behavior.

Waldemar's right, browsers differ on Array enumeration, for some or
all Arrays. My observation to TC39 when we last discussed this is that
most arrays are populated front to back (0 to high index) so insertion
order matches index order, and that most programmers (all I've ever
polled or heard from) want array enumeration to use index order.

It's conceivable that we could standardize insertion order for named
properties, index order for indexed properties (for all objects, not
just for Arrays). I don't know how this will fly; some VMs already
optimize this way (v8 does, IIRC). I think it's a good compromise,
since the edge case behavior in different browsers does not allow edge- case-testing scripts to interoperate today.

But this will take someone (my name was on it at the end of the last
TC39 meeting) writing up a proposal. I will make time this month for it.

The issue is not overriding default for-in behavior (as you can do in  
JS1.7 via __iterator__). The issue is the default behavior.

Waldemar's right, browsers differ on Array enumeration, for some or  
all Arrays. My observation to TC39 when we last discussed this is that  
most arrays are populated front to back (0 to high index) so insertion  
order matches index order, and that most programmers (all I've ever  
polled or heard from) want array enumeration to use index order.

It's conceivable that we could standardize insertion order for named  
properties, index order for indexed properties (for all objects, not  
just for Arrays). I don't know how this will fly; some VMs already  
optimize this way (v8 does, IIRC). I think it's a good compromise,  
since the edge case behavior in different browsers does not allow edge- 
case-testing scripts to interoperate today.

But this will take someone (my name was on it at the end of the last  
TC39 meeting) writing up a proposal. I will make time this month for it.

/be

On Oct 14, 2009, at 4:11 PM, Brian Kardell wrote:

> It sounds to me like there is wide agreement in the sense that at
> least the basics rules and only disagreement on the fringes...
> Otherwise no one on this list in particular would be suggesting that
> there is anything remotely like a "de facto" implementation... It
> seems that at least those basic rules are required just to function
> with the existing expectations everywhere.
>
> It also seems that those expectations aren't likely to change any time
> soon for the "default" for-each iteration order whether more robust
> and interesting proposals are adopted...
>
> So can't that much be formalized and documented regardless of whether
> or not new introductions are made to over-ride "other" predictable
> iterators to be used in for-each (or perhaps even some new mechanism
> entirely)? The simple fact that conforming to the spec would currently
> create a non-workable solution seems to argue that at least the "de
> facto" parts should be included...
>
> No?
>
>
>
> On Wed, Oct 14, 2009 at 2:37 PM, Waldemar Horwat  
> <waldemar at google.com> wrote:
>> Brian Kardell wrote:
>>>
>>> Uh oh... Patrick looking at the same parts as me reacting the same  
>>> way
>>> (I believe older pre-5 docs also said something similar)...
>>> When Oliver said "All ES implementations are required to..." I  
>>> assumed
>>> that I missed something in the new 5 docs - not that everyone just
>>> seems to have implemented roughly the same thing...
>>>
>>> Is it required - or is it not?
>>
>> It is not required.  We were planning to make it required for ES5  
>> but it
>> turns out that implementations do not match each other and  
>> implementations
>> do not follow insertion order for some kinds of properties such as  
>> things
>> that could be used as array indices.  Requiring insertion order  
>> would make
>> iterating over arrays weird.
>>
>>   Waldemar
>>
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss

# Waldemar Horwat (15 years ago)

Brian Kardell wrote:

It sounds to me like there is wide agreement in the sense that at least the basics rules and only disagreement on the fringes... Otherwise no one on this list in particular would be suggesting that there is anything remotely like a "de facto" implementation... It seems that at least those basic rules are required just to function with the existing expectations everywhere.

It also seems that those expectations aren't likely to change any time soon for the "default" for-each iteration order whether more robust and interesting proposals are adopted...

So can't that much be formalized and documented regardless of whether or not new introductions are made to over-ride "other" predictable iterators to be used in for-each (or perhaps even some new mechanism entirely)? The simple fact that conforming to the spec would currently create a non-workable solution seems to argue that at least the "de facto" parts should be included...

No?

No. As I wrote, there is no de-facto implementation order because the implementations do not agree on the order in general, and what you call "fringes" such as numbers do matter. Trying to force, say, insertion order would likely break compatibility.

Waldemar

Brian Kardell wrote:
> It sounds to me like there is wide agreement in the sense that at
> least the basics rules and only disagreement on the fringes...
> Otherwise no one on this list in particular would be suggesting that
> there is anything remotely like a "de facto" implementation... It
> seems that at least those basic rules are required just to function
> with the existing expectations everywhere.
> 
> It also seems that those expectations aren't likely to change any time
> soon for the "default" for-each iteration order whether more robust
> and interesting proposals are adopted...
> 
> So can't that much be formalized and documented regardless of whether
> or not new introductions are made to over-ride "other" predictable
> iterators to be used in for-each (or perhaps even some new mechanism
> entirely)? The simple fact that conforming to the spec would currently
> create a non-workable solution seems to argue that at least the "de
> facto" parts should be included...
> 
> No?

No.  As I wrote, there is no de-facto implementation order because the implementations do not agree on the order in general, and what you call "fringes" such as numbers do matter.  Trying to force, say, insertion order would likely break compatibility.

    Waldemar

# Jeff Walden (15 years ago)

On 10/13/2009 10:54 PM, Luke Smith wrote:

Currently FF3.5.4 doesn't properly apply replacer functions, but Safari 4, WebKit, IE8, and Chrome 3 work fine for this task.

How precisely are replacer functions not properly applied in Firefox? I remember reporting at least one bug on the replacer-function implementation that was fixed for 3.5 or a beta of it, and to the best of my knowledge they work fine, outside of a few cases such as for circular data structures (and the semantic difference there, if you're not looking for it, probably won't matter much anyway). Have you reported a bug for this at bugzilla.mozilla.org?

On 10/13/2009 10:54 PM, Luke Smith wrote:
> Currently FF3.5.4 doesn't properly apply replacer functions, but Safari
> 4, WebKit, IE8, and Chrome 3 work fine for this task.

How precisely are replacer functions not properly applied in Firefox?  I remember reporting at least one bug on the replacer-function implementation that was fixed for 3.5 or a beta of it, and to the best of my knowledge they work fine, outside of a few cases such as for circular data structures (and the semantic difference there, if you're not looking for it, probably won't matter much anyway).  Have you reported a bug for this at bugzilla.mozilla.org?

Jeff

# Mike Shaver (15 years ago)

On Wed, Oct 14, 2009 at 7:24 PM, Waldemar Horwat <waldemar at google.com> wrote:

No. As I wrote, there is no de-facto implementation order because the implementations do not agree on the order in general, and what you call "fringes" such as numbers do matter. Trying to force, say, insertion order would likely break compatibility.

IIRC, Firefox before FF3 used insertion order for arrays as well as objects, and changed to index order for arrays in FF3. I don't recall any compat problems on either side of the transition with respect to array iteration order.

Mike

On Wed, Oct 14, 2009 at 7:24 PM, Waldemar Horwat <waldemar at google.com> wrote:
> No.  As I wrote, there is no de-facto implementation order because the
> implementations do not agree on the order in general, and what you call
> "fringes" such as numbers do matter.  Trying to force, say, insertion order
> would likely break compatibility.

IIRC, Firefox before FF3 used insertion order for arrays as well as
objects, and changed to index order for arrays in FF3.  I don't recall
any compat problems on either side of the transition with respect to
array iteration order.

Mike

# Mike Shaver (15 years ago)

On Wed, Oct 14, 2009 at 7:39 PM, Jeff Walden <jwalden+es at mit.edu> wrote:

On 10/13/2009 10:54 PM, Luke Smith wrote:

Currently FF3.5.4 doesn't properly apply replacer functions, but Safari 4, WebKit, IE8, and Chrome 3 work fine for this task.

How precisely are replacer functions not properly applied in Firefox? I remember reporting at least one bug on the replacer-function implementation that was fixed for 3.5 or a beta of it, and to the best of my knowledge they work fine, outside of a few cases such as for circular data structures (and the semantic difference there, if you're not looking for it, probably won't matter much anyway). Have you reported a bug for this at bugzilla.mozilla.org?

Yes, he did: bugzilla.mozilla.org/show_bug.cgi?id=509184

Mike

On Wed, Oct 14, 2009 at 7:39 PM, Jeff Walden <jwalden+es at mit.edu> wrote:
> On 10/13/2009 10:54 PM, Luke Smith wrote:
>>
>> Currently FF3.5.4 doesn't properly apply replacer functions, but Safari
>> 4, WebKit, IE8, and Chrome 3 work fine for this task.
>
> How precisely are replacer functions not properly applied in Firefox?  I
> remember reporting at least one bug on the replacer-function implementation
> that was fixed for 3.5 or a beta of it, and to the best of my knowledge they
> work fine, outside of a few cases such as for circular data structures (and
> the semantic difference there, if you're not looking for it, probably won't
> matter much anyway).  Have you reported a bug for this at
> bugzilla.mozilla.org?

Yes, he did: https://bugzilla.mozilla.org/show_bug.cgi?id=509184

Mike

# Allen Wirfs-Brock (15 years ago)

It's probably too late for us to take advantage of this, but it did just occur to me (too much document.all discussion) that since for-in is a syntactic construct its behavior could be different in strict mode, potentially allowing us to define a for-in enumeration order that isn't tried to legacy behavior.

Similarly, rather than trying to fully specifying for-in enumeration order (and invariably causing potentially breaking changes for some implementations) we could specify a new enumeration statement that uses a strictly specified order. By suggest for this would be: for names (n in expr) ... and for values (v in expr) ...

avoiding conflicts with the existing for-in and E4X for-each.

It's probably too late for us to take advantage of this, but it did just occur to me (too much document.all discussion) that since for-in is a syntactic construct its behavior could be different in strict mode, potentially allowing us to define a for-in enumeration order that isn't tried to legacy behavior.

Similarly, rather than trying to fully specifying for-in enumeration order (and invariably causing potentially breaking changes for some implementations) we could specify a new enumeration statement that uses a strictly specified order.  By suggest for this would be:
   for names (n in expr) ...
and
   for values (v in expr) ...

avoiding conflicts with the existing for-in and E4X for-each.

Allen

>-----Original Message-----
>From: es-discuss-bounces at mozilla.org [mailto:es-discuss-
>bounces at mozilla.org] On Behalf Of Mike Shaver
>Sent: Wednesday, October 14, 2009 6:30 PM
>To: Waldemar Horwat
>Cc: es-discuss
>Subject: Re: Property Iteration in JSON serialization
>
>On Wed, Oct 14, 2009 at 7:24 PM, Waldemar Horwat <waldemar at google.com>
>wrote:
>> No.  As I wrote, there is no de-facto implementation order because the
>> implementations do not agree on the order in general, and what you
>call
>> "fringes" such as numbers do matter.  Trying to force, say, insertion
>order
>> would likely break compatibility.
>
>IIRC, Firefox before FF3 used insertion order for arrays as well as
>objects, and changed to index order for arrays in FF3.  I don't recall
>any compat problems on either side of the transition with respect to
>array iteration order.
>
>Mike
>_______________________________________________
>es-discuss mailing list
>es-discuss at mozilla.org
>https://mail.mozilla.org/listinfo/es-discuss

# Brian Kardell (15 years ago)

Sure, but again... Without any changes it currently reads:

"The mechanics and order of enumerating the properties (step 6.a in the first algorithm, step 7.a in the second) is not specified. "

Which isn't really necessary even currently (pre-5 I think) doesn't accurately describe what seems to be the de-facto standard. It should be relatively easy to merely codify what is already implemented everywhere which sounds like without any disagreement or "new" proposals it could say something along the lines of:

"For objects containing no indexed properties the mechanics and default order of enumerating the properties (step 6.a in the first algorithm, step 7.a in the second) is specified as insertion order. For reasons of historical compatibility, the default order of enumerating the properties of objects which contain indexed properties is not specified. "

That's not to say that no further effort should be made to take it even further, but I for one would find something like the above a lot more comforting and realistic than nothing at all if it comes to that.

Sure, but again... Without any changes it currently reads:

"The mechanics and order of enumerating the properties (step 6.a in
the first algorithm, step 7.a in the second) is not specified. "

Which isn't really necessary even currently (pre-5 I think) doesn't
accurately describe what seems to be the de-facto standard.  It should
be relatively easy to merely codify what is already implemented
everywhere which sounds like without any disagreement or "new"
proposals it could say something along the lines of:

"For objects containing no indexed properties the mechanics and
default order of enumerating the properties (step 6.a in the first
algorithm, step 7.a in the second) is specified as insertion order.
For reasons of historical compatibility, the default order of
enumerating the properties of objects which contain indexed properties
is not specified. "

That's not to say that no further effort should be made to take it
even further, but I for one would find something like the above a lot
more comforting and realistic than nothing at all if it comes to that.

On Wed, Oct 14, 2009 at 7:01 PM, Allen Wirfs-Brock
<Allen.Wirfs-Brock at microsoft.com> wrote:
> It's probably too late for us to take advantage of this, but it did just occur to me (too much document.all discussion) that since for-in is a syntactic construct its behavior could be different in strict mode, potentially allowing us to define a for-in enumeration order that isn't tried to legacy behavior.
>
> Similarly, rather than trying to fully specifying for-in enumeration order (and invariably causing potentially breaking changes for some implementations) we could specify a new enumeration statement that uses a strictly specified order.  By suggest for this would be:
>   for names (n in expr) ...
> and
>   for values (v in expr) ...
>
> avoiding conflicts with the existing for-in and E4X for-each.
>
> Allen
>
>>-----Original Message-----
>>From: es-discuss-bounces at mozilla.org [mailto:es-discuss-
>>bounces at mozilla.org] On Behalf Of Mike Shaver
>>Sent: Wednesday, October 14, 2009 6:30 PM
>>To: Waldemar Horwat
>>Cc: es-discuss
>>Subject: Re: Property Iteration in JSON serialization
>>
>>On Wed, Oct 14, 2009 at 7:24 PM, Waldemar Horwat <waldemar at google.com>
>>wrote:
>>> No.  As I wrote, there is no de-facto implementation order because the
>>> implementations do not agree on the order in general, and what you
>>call
>>> "fringes" such as numbers do matter.  Trying to force, say, insertion
>>order
>>> would likely break compatibility.
>>
>>IIRC, Firefox before FF3 used insertion order for arrays as well as
>>objects, and changed to index order for arrays in FF3.  I don't recall
>>any compat problems on either side of the transition with respect to
>>array iteration order.
>>
>>Mike
>>_______________________________________________
>>es-discuss mailing list
>>es-discuss at mozilla.org
>>https://mail.mozilla.org/listinfo/es-discuss
>
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss
>

# Brian Kardell (15 years ago)

Sorry... somehow Waldemar's comment got closed up in my Gmail conversation stack and I missed this comment...

If Oliver and Hallvord and Brendan are wrong on the idea that it is at least largely already a de facto standard for non-indexed properties then I suppose it is a moot point...

Sorry... somehow Waldemar's comment got closed up in my Gmail
conversation stack and I missed this comment...

If Oliver and  Hallvord and Brendan are wrong on the idea that it is
at least largely already a de facto standard for non-indexed
properties then  I suppose it is a moot point...






On Wed, Oct 14, 2009 at 4:24 PM, Waldemar Horwat <waldemar at google.com> wrote:
> Brian Kardell wrote:
>>
>> It sounds to me like there is wide agreement in the sense that at
>> least the basics rules and only disagreement on the fringes...
>> Otherwise no one on this list in particular would be suggesting that
>> there is anything remotely like a "de facto" implementation... It
>> seems that at least those basic rules are required just to function
>> with the existing expectations everywhere.
>>
>> It also seems that those expectations aren't likely to change any time
>> soon for the "default" for-each iteration order whether more robust
>> and interesting proposals are adopted...
>>
>> So can't that much be formalized and documented regardless of whether
>> or not new introductions are made to over-ride "other" predictable
>> iterators to be used in for-each (or perhaps even some new mechanism
>> entirely)? The simple fact that conforming to the spec would currently
>> create a non-workable solution seems to argue that at least the "de
>> facto" parts should be included...
>>
>> No?
>
> No.  As I wrote, there is no de-facto implementation order because the
> implementations do not agree on the order in general, and what you call
> "fringes" such as numbers do matter.  Trying to force, say, insertion order
> would likely break compatibility.
>
>   Waldemar
>

# Brendan Eich (15 years ago)

On Oct 14, 2009, at 7:01 PM, Allen Wirfs-Brock wrote:

Similarly, rather than trying to fully specifying for-in enumeration
order (and invariably causing potentially breaking changes for some
implementations)

I'm focusing on your parenthetical aside: we have precedent in Harmony
(functions in blocks) for finding room to create a better standard
where browsers do not agree, and therefore unless browser detection is
going on, cross-platform JS on the web cannot depend on the
intersection semantics (the empty set).

I think we should investigate breaking changes. The IE JScript
deviations document linked from

doku.php?id=es3.1:es3.x_working_docs&s=jscript+deviations

contains some amazing details of IE"s for-in behavior:

Enumerate properties on immediate prototype in reverse order (w.r.t
the order in which they were added)
Enumerate properties in-order on rest of the objects on the
prototype chain
Enumerate properties in-order on present object.

Reverse insertion order on the immediate prototype properties, not
something other engines do.

Array element enumeration details are lacking, but again if the
observation that array elements are almost always created in index
order, then insertion order == index order. The exceptions may or may
not work as expected on insertion-order-only implementations. Firefox
3+ uses index order. We're not receiving bug reports.

There's room to find a better common semantics here, it seems to me.

On Oct 14, 2009, at 7:01 PM, Allen Wirfs-Brock wrote:

> Similarly, rather than trying to fully specifying for-in enumeration  
> order (and invariably causing potentially breaking changes for some  
> implementations)

I'm focusing on your parenthetical aside: we have precedent in Harmony  
(functions in blocks) for finding room to create a better standard  
where browsers do not agree, and therefore unless browser detection is  
going on, cross-platform JS on the web cannot depend on the  
intersection semantics (the empty set).

I think we should investigate breaking changes. The IE JScript  
deviations document linked from

http://wiki.ecmascript.org/doku.php?id=es3.1:es3.x_working_docs&s=jscript+deviations

contains some amazing details of IE"s for-in behavior:

* Enumerate properties on immediate prototype in reverse order (w.r.t  
the order in which they were added)

* Enumerate properties in-order on rest of the objects on the  
prototype chain

* Enumerate properties in-order on present object.

Reverse insertion order on the immediate prototype properties, not  
something other engines do.

Array element enumeration details are lacking, but again if the  
observation that array elements are almost always created in index  
order, then insertion order == index order. The exceptions may or may  
not work as expected on insertion-order-only implementations. Firefox  
3+ uses index order. We're not receiving bug reports.

There's room to find a better common semantics here, it seems to me.

/be

# Brendan Eich (15 years ago)

On Oct 14, 2009, at 8:34 PM, Brian Kardell wrote:

Sorry... somehow Waldemar's comment got closed up in my Gmail conversation stack and I missed this comment...

If Oliver and Hallvord and Brendan are wrong on the idea that it is at least largely already a de facto standard for non-indexed properties then I suppose it is a moot point...

Waldemar did not cite non-indexed property counter-examples to
insertion order, and insertion order is the rule among browsers.
However, I did just posted a reference to:

Enumerate properties on immediate prototype in reverse order (w.r.t
the order in which they were added)
Enumerate properties in-order on rest of the objects on the
prototype chain
Enumerate properties in-order on present object.

from the first pdf link in

doku.php?id=es3.1:es3.x_working_docs&s=jscript+deviations

to highlight Pratap's revelation that IE JScript (at least some
versions) use reverse-insertion order for the immediate prototype's
enumerable properties.

But so what? Enumerable prototype properties are rare and they are
considered bad practice. IE alone behaves in this exceptional manner
for the direct prototype's properties. As far as I know, all other
implementations use insertion order for non-indexed properties.

We are not going to standardize this anomaly, and cross-browser JS
can't count on it. So we should standardize something saner, possibly
insertion order for named properties, index order for indexed.

On Oct 14, 2009, at 8:34 PM, Brian Kardell wrote:

> Sorry... somehow Waldemar's comment got closed up in my Gmail
> conversation stack and I missed this comment...
>
> If Oliver and  Hallvord and Brendan are wrong on the idea that it is
> at least largely already a de facto standard for non-indexed
> properties then  I suppose it is a moot point...
>

Waldemar did not cite non-indexed property counter-examples to  
insertion order, and insertion order is the rule among browsers.  
However, I did just posted a reference to:

* Enumerate properties on immediate prototype in reverse order (w.r.t  
the order in which they were added)

* Enumerate properties in-order on rest of the objects on the  
prototype chain

* Enumerate properties in-order on present object.


from the first pdf link in

http://wiki.ecmascript.org/doku.php?id=es3.1:es3.x_working_docs&s=jscript+deviations

to highlight Pratap's revelation that IE JScript (at least some  
versions) use reverse-insertion order for the immediate prototype's  
enumerable properties.

But so what? Enumerable prototype properties are rare and they are  
considered bad practice. IE alone behaves in this exceptional manner  
for the direct prototype's properties. As far as I know, all other  
implementations use insertion order for non-indexed properties.

We are not going to standardize this anomaly, and cross-browser JS  
can't count on it. So we should standardize something saner, possibly  
insertion order for named properties, index order for indexed.

/be

>
>
>
>
>
> On Wed, Oct 14, 2009 at 4:24 PM, Waldemar Horwat  
> <waldemar at google.com> wrote:
>> Brian Kardell wrote:
>>>
>>> It sounds to me like there is wide agreement in the sense that at
>>> least the basics rules and only disagreement on the fringes...
>>> Otherwise no one on this list in particular would be suggesting that
>>> there is anything remotely like a "de facto" implementation... It
>>> seems that at least those basic rules are required just to function
>>> with the existing expectations everywhere.
>>>
>>> It also seems that those expectations aren't likely to change any  
>>> time
>>> soon for the "default" for-each iteration order whether more robust
>>> and interesting proposals are adopted...
>>>
>>> So can't that much be formalized and documented regardless of  
>>> whether
>>> or not new introductions are made to over-ride "other" predictable
>>> iterators to be used in for-each (or perhaps even some new mechanism
>>> entirely)? The simple fact that conforming to the spec would  
>>> currently
>>> create a non-workable solution seems to argue that at least the "de
>>> facto" parts should be included...
>>>
>>> No?
>>
>> No.  As I wrote, there is no de-facto implementation order because  
>> the
>> implementations do not agree on the order in general, and what you  
>> call
>> "fringes" such as numbers do matter.  Trying to force, say,  
>> insertion order
>> would likely break compatibility.
>>
>>   Waldemar
>>
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss

# Maciej Stachowiak (15 years ago)

On Oct 14, 2009, at 8:34 PM, Brian Kardell wrote:

Sorry... somehow Waldemar's comment got closed up in my Gmail conversation stack and I missed this comment...

If Oliver and Hallvord and Brendan are wrong on the idea that it is at least largely already a de facto standard for non-indexed properties then I suppose it is a moot point...

Historically, the areas where JS engines in the major browsers have
diverged on for..in enumeration order have been properties with index- like names, and properties on the prototype chain. I made a test case
to study some of these differences.

Current JavaScriptCore and SpiderMonkey seem to get identical results
on this, other browsers all have slight variations:

Here are some of the variations I found:

== Index properties on non-arrays == JSC/Safari - insertion order SpiderMonkey/Firefox - insertion order Futhark/Opera - insertion order V8/Chrome - numeric sorted order JScript/IE - insertion order

== Index properties on arrays == JSC/Safari - numeric sorted order SpiderMonkey/Firefox - numeric sorted order Futhark/Opera - insertion order V8/Chrome - numeric sorted order JScript/IE - insertion order

== Objects with enumerable properties in the prototype chain == JSC/Safari - own properties in insertion order, followed by prototype
properties from most direct prototype to least direct (each prototypes
properties in insertion ordr) SpiderMonkey/Firefox - same Futhark/Opera - same V8/Chrome - properties are re-ordered, even own properties, but I
don't understand the rule(*) JScript/IE - prototype properties are reported before own properties;
own properties are in order, but properties on every other prototype
are reversed (I think?)

(*) - If you use constructor functions to make an object with
properties named (x, y, z) added in that order, with a prototype that
has properties (a, b, c), and in turn has a prototype with properties
(q, r, s), JSC and SpiderMonkey report "x y z a b c q r s" but V8
gives "x z y a c b q r s". I'm not sure what rule would give that
output. I'm not even sure if this behavior is intentional or a bug.

The common behavior subset here is: for objects with no properties
that look like array indices, and no enumerable prototype properties,
for..in enumeration returns properties in insertion order. That
particular behavior is a de facto standard and required for Web
compatibility. A future standard should specify at least that much.

All the browsers besides Chrome also seem to report index-like
properties on arrays in insertion order, and report own properties in
insertion order even when there are enumerable prototype properties
(though the prototype properties may be before or after). Perhaps
somewhat self-servingly, I think the Safari/Firefox behavior is the
most sensible overall, so if we ever fully standardize the enumeration
order, that is what I would suggest.

Here's my test case if you want to try for yourself:

On Oct 14, 2009, at 8:34 PM, Brian Kardell wrote:

> Sorry... somehow Waldemar's comment got closed up in my Gmail
> conversation stack and I missed this comment...
>
> If Oliver and  Hallvord and Brendan are wrong on the idea that it is
> at least largely already a de facto standard for non-indexed
> properties then  I suppose it is a moot point...

Historically, the areas where JS engines in the major browsers have  
diverged on for..in enumeration order have been properties with index- 
like names, and properties on the prototype chain. I made a test case  
to study some of these differences.

Current JavaScriptCore and SpiderMonkey seem to get identical results  
on this, other browsers all have slight variations:

Here are some of the variations I found:

== Index properties on non-arrays ==
JSC/Safari - insertion order
SpiderMonkey/Firefox - insertion order
Futhark/Opera - insertion order
V8/Chrome - numeric sorted order
JScript/IE - insertion order

== Index properties on arrays ==
JSC/Safari - numeric sorted order
SpiderMonkey/Firefox - numeric sorted order
Futhark/Opera - insertion order
V8/Chrome - numeric sorted order
JScript/IE - insertion order

== Objects with enumerable properties in the prototype chain ==
JSC/Safari - own properties in insertion order, followed by prototype  
properties from most direct prototype to least direct (each prototypes  
properties in insertion ordr)
SpiderMonkey/Firefox - same
Futhark/Opera - same
V8/Chrome - properties are re-ordered, even own properties, but I  
don't understand the rule(*)
JScript/IE - prototype properties are reported before own properties;  
own properties are in order, but properties on every other prototype  
are reversed (I think?)

(*) - If you use constructor functions to make an object with  
properties named (x, y, z) added in that order, with a prototype that  
has properties (a, b, c), and in turn has a prototype with properties  
(q, r, s), JSC and SpiderMonkey report "x y z a b c q r s" but V8  
gives "x z y a c b q r s". I'm not sure what rule would give that  
output. I'm not even sure if this behavior is intentional or a bug.

The common behavior subset here is: for objects with no properties  
that look like array indices, and no enumerable prototype properties,  
for..in enumeration returns properties in insertion order. That  
particular behavior is a de facto standard and required for Web  
compatibility. A future standard should specify at least that much.

All the browsers besides Chrome also seem to report index-like  
properties on arrays in insertion order, and report own properties in  
insertion order even when there are enumerable prototype properties  
(though the prototype properties may be before or after). Perhaps  
somewhat self-servingly, I think the Safari/Firefox behavior is the  
most sensible overall, so if we ever fully standardize the enumeration  
order, that is what I would suggest.

Here's my test case if you want to try for yourself:

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20091014/414c2f14/attachment.html>
-------------- next part --------------

Regards,
Macej

# Maciej Stachowiak (15 years ago)

On Oct 14, 2009, at 10:47 PM, Maciej Stachowiak wrote:

(*) - If you use constructor functions to make an object with
properties named (x, y, z) added in that order, with a prototype
that has properties (a, b, c), and in turn has a prototype with
properties (q, r, s), JSC and SpiderMonkey report "x y z a b c q r
s" but V8 gives "x z y a c b q r s". I'm not sure what rule would
give that output. I'm not even sure if this behavior is intentional
or a bug.

Correction: on further testing, it appears that Chrome's ordering
anomaly is for own properties added by a constructor function, whether
or not there's a prototype with enumerable properties. Chrome reports
"a c b" for the following, whereas other browser which reports "a b c".

function Construct() { this.a = "a"; this.b = "b"; this.c = "c"; }

var constrObj = new Construct();

function printProperties(description, o) { document.write(description + ":"); for (var pname in o) { document.write(" " + pname); } document.write("\n"); }

printProperties("constructed object", constrObj); </script> </pre>

On Oct 14, 2009, at 10:47 PM, Maciej Stachowiak wrote:

>
> (*) - If you use constructor functions to make an object with  
> properties named (x, y, z) added in that order, with a prototype  
> that has properties (a, b, c), and in turn has a prototype with  
> properties (q, r, s), JSC and SpiderMonkey report "x y z a b c q r  
> s" but V8 gives "x z y a c b q r s". I'm not sure what rule would  
> give that output. I'm not even sure if this behavior is intentional  
> or a bug.

Correction: on further testing, it appears that Chrome's ordering  
anomaly is for own properties added by a constructor function, whether  
or not there's a prototype with enumerable properties. Chrome reports  
"a c b" for the following, whereas other browser which reports "a b c".

<pre>
<script>
function Construct()
{
     this.a = "a";
     this.b = "b";
     this.c = "c";
}

var constrObj = new Construct();

function printProperties(description, o) {
     document.write(description + ":");
     for (var pname in o) {
         document.write(" " + pname);
     }
     document.write("\n");
}

printProperties("constructed object", constrObj);
</script>
</pre>

# Brian Kardell (15 years ago)

For even more fun try out the code below which has mixed content and still appears to function (surprisingly) the same in FF 3.5/Safari4 (OSX) and IE8 (XP) -- but seems to order the keys differently in Chrome 3 - though seemingly still as explained earlier (except that the negative index is last??):

function Construct(){ this["-1"] = "holy cow"; //This surprises me" this.a = "a"; this[1] = "what"; this.b = "b"; this["1"] = "the"; this["2"] = "heck"; this.c = "c"; }

var constrObj = new Construct(); constrObj["3"] = "I don't even know what this is supposed to be!";

function printProperties(description, o) { var str = ""; str += description; for (var pname in o) { str += " " + pname + " - " + o[pname] + "\n"; } alert(str); alert(o[1]); alert(o[-1]); }

printProperties("constructed object:\n", constrObj);

For even more fun try out the code below which has mixed content and
still appears to function (surprisingly) the same in FF 3.5/Safari4
(OSX) and IE8 (XP) -- but seems to order the keys differently in
Chrome 3 - though seemingly still as explained earlier (except that
the negative index is last??):

function Construct(){
    this["-1"] = "holy cow";  //This surprises me"
    this.a = "a";
    this[1] = "what";
    this.b = "b";
    this["1"] = "the";
    this["2"] = "heck";
    this.c = "c";
}

var constrObj = new Construct();
constrObj["3"] = "I don't even know what this is supposed to be!";

function printProperties(description, o) {
    var str = "";
    str += description;
    for (var pname in o) {
        str += " " + pname + " - " + o[pname] + "\n";
    }
    alert(str);
    alert(o[1]);
    alert(o[-1]);
}

printProperties("constructed object:\n", constrObj);




On Wed, Oct 14, 2009 at 10:53 PM, Maciej Stachowiak <mjs at apple.com> wrote:
>
> On Oct 14, 2009, at 10:47 PM, Maciej Stachowiak wrote:
>
>>
>> (*) - If you use constructor functions to make an object with properties
>> named (x, y, z) added in that order, with a prototype that has properties
>> (a, b, c), and in turn has a prototype with properties (q, r, s), JSC and
>> SpiderMonkey report "x y z a b c q r s" but V8 gives "x z y a c b q r s".
>> I'm not sure what rule would give that output. I'm not even sure if this
>> behavior is intentional or a bug.
>
> Correction: on further testing, it appears that Chrome's ordering anomaly is
> for own properties added by a constructor function, whether or not there's a
> prototype with enumerable properties. Chrome reports "a c b" for the
> following, whereas other browser which reports "a b c".
>
> <pre>
> <script>
> function Construct()
> {
>    this.a = "a";
>    this.b = "b";
>    this.c = "c";
> }
>
> var constrObj = new Construct();
>
> function printProperties(description, o) {
>    document.write(description + ":");
>    for (var pname in o) {
>        document.write(" " + pname);
>    }
>    document.write("\n");
> }
>
> printProperties("constructed object", constrObj);
> </script>
> </pre>
>

# Brian Kardell (15 years ago)

On something saner: It does seem quite rational - and as you point out, it's bad practice and should be rarely used... It also stands to reason that it is rarely used as IE still maintains the dominant share and does it differently, yet no one seems to be screaming about it... unlike the general rule regarding non-indexed (vanilla) properties which Opera found they had to change just in order to keep things running.

On something saner:  It does seem quite rational - and as you point
out, it's bad practice and should be rarely used... It also stands to
reason that it _is_ rarely used as IE still maintains the dominant
share and does it differently, yet no one seems to be screaming about
it... unlike the general rule regarding non-indexed (vanilla)
properties which Opera found they had to change just in order to keep
things running.



On Wed, Oct 14, 2009 at 10:31 PM, Brendan Eich <brendan at mozilla.com> wrote:
> On Oct 14, 2009, at 8:34 PM, Brian Kardell wrote:
>
>> Sorry... somehow Waldemar's comment got closed up in my Gmail
>> conversation stack and I missed this comment...
>>
>> If Oliver and  Hallvord and Brendan are wrong on the idea that it is
>> at least largely already a de facto standard for non-indexed
>> properties then  I suppose it is a moot point...
>>
>
> Waldemar did not cite non-indexed property counter-examples to insertion
> order, and insertion order is the rule among browsers. However, I did just
> posted a reference to:
>
> * Enumerate properties on immediate prototype in reverse order (w.r.t the
> order in which they were added)
>
> * Enumerate properties in-order on rest of the objects on the prototype
> chain
>
> * Enumerate properties in-order on present object.
>
>
> from the first pdf link in
>
> http://wiki.ecmascript.org/doku.php?id=es3.1:es3.x_working_docs&s=jscript+deviations
>
> to highlight Pratap's revelation that IE JScript (at least some versions)
> use reverse-insertion order for the immediate prototype's enumerable
> properties.
>
> But so what? Enumerable prototype properties are rare and they are
> considered bad practice. IE alone behaves in this exceptional manner for the
> direct prototype's properties. As far as I know, all other implementations
> use insertion order for non-indexed properties.
>
> We are not going to standardize this anomaly, and cross-browser JS can't
> count on it. So we should standardize something saner, possibly insertion
> order for named properties, index order for indexed.
>
> /be
>
>>
>>
>>
>>
>>
>> On Wed, Oct 14, 2009 at 4:24 PM, Waldemar Horwat <waldemar at google.com>
>> wrote:
>>>
>>> Brian Kardell wrote:
>>>>
>>>> It sounds to me like there is wide agreement in the sense that at
>>>> least the basics rules and only disagreement on the fringes...
>>>> Otherwise no one on this list in particular would be suggesting that
>>>> there is anything remotely like a "de facto" implementation... It
>>>> seems that at least those basic rules are required just to function
>>>> with the existing expectations everywhere.
>>>>
>>>> It also seems that those expectations aren't likely to change any time
>>>> soon for the "default" for-each iteration order whether more robust
>>>> and interesting proposals are adopted...
>>>>
>>>> So can't that much be formalized and documented regardless of whether
>>>> or not new introductions are made to over-ride "other" predictable
>>>> iterators to be used in for-each (or perhaps even some new mechanism
>>>> entirely)? The simple fact that conforming to the spec would currently
>>>> create a non-workable solution seems to argue that at least the "de
>>>> facto" parts should be included...
>>>>
>>>> No?
>>>
>>> No.  As I wrote, there is no de-facto implementation order because the
>>> implementations do not agree on the order in general, and what you call
>>> "fringes" such as numbers do matter.  Trying to force, say, insertion
>>> order
>>> would likely break compatibility.
>>>
>>>  Waldemar
>>>
>> _______________________________________________
>> es-discuss mailing list
>> es-discuss at mozilla.org
>> https://mail.mozilla.org/listinfo/es-discuss
>
>

# Allen Wirfs-Brock (15 years ago)

I'm not particularly defending IE's legacy enumeration order, we were initially on board with ES5 adopting the de facto order used by other browsers. My recollection is that the decision to table defining a strict enumeration order for ES5 was made after we discovered that Opera for its optimized arrays did not follow the expected order mail.mozilla.org/pipermail/es5-discuss/2008-May/001286.html and mail.mozilla.org/pipermail/es5-discuss/2008-June/001304.html and many other messages in that thread.

We definitely should try to define a rational enumeration order. It is a secondary issues as to whether we impose it as a potentially breaking change upon the existing for-in or introduce it via new syntax. I think the second alterative is an approach we should at least consider not just in this situation but in all similar situations. Sometimes it will make sense and other times it won't.

I'm not particularly defending IE's legacy enumeration order, we were initially on board with ES5 adopting the de facto order used by other browsers.  My recollection is that the decision to table defining a strict enumeration order for ES5 was made after we discovered that Opera for its optimized arrays did not follow the expected order https://mail.mozilla.org/pipermail/es5-discuss/2008-May/001286.html and https://mail.mozilla.org/pipermail/es5-discuss/2008-June/001304.html and many other messages in that thread.

We definitely should try to define a rational enumeration order.  It is a secondary issues as to whether we impose it as a potentially breaking change upon the existing for-in or introduce it via new syntax.  I think the second alterative is an approach we should at least consider not just in this situation but in all similar situations.  Sometimes it will make sense and other times it won't.

Allen

>-----Original Message-----
>From: Brendan Eich [mailto:brendan at mozilla.com]
>Sent: Wednesday, October 14, 2009 10:26 PM
>To: Allen Wirfs-Brock
>Cc: Mike Shaver; Waldemar Horwat; es-discuss
>Subject: Re: Property Iteration in JSON serialization
>
>On Oct 14, 2009, at 7:01 PM, Allen Wirfs-Brock wrote:
>
>> Similarly, rather than trying to fully specifying for-in enumeration
>> order (and invariably causing potentially breaking changes for some
>> implementations)
>
>I'm focusing on your parenthetical aside: we have precedent in Harmony
>(functions in blocks) for finding room to create a better standard
>where browsers do not agree, and therefore unless browser detection is
>going on, cross-platform JS on the web cannot depend on the
>intersection semantics (the empty set).
>
>I think we should investigate breaking changes. The IE JScript
>deviations document linked from
>
>http://wiki.ecmascript.org/doku.php?id=es3.1:es3.x_working_docs&s=jscrip
>t+deviations
>
>contains some amazing details of IE"s for-in behavior:
>
>* Enumerate properties on immediate prototype in reverse order (w.r.t
>the order in which they were added)
>
>* Enumerate properties in-order on rest of the objects on the
>prototype chain
>
>* Enumerate properties in-order on present object.
>
>Reverse insertion order on the immediate prototype properties, not
>something other engines do.
>
>Array element enumeration details are lacking, but again if the
>observation that array elements are almost always created in index
>order, then insertion order == index order. The exceptions may or may
>not work as expected on insertion-order-only implementations. Firefox
>3+ uses index order. We're not receiving bug reports.
>
>There's room to find a better common semantics here, it seems to me.
>
>/be

# Maciej Stachowiak (15 years ago)

On Oct 15, 2009, at 10:23 AM, Allen Wirfs-Brock wrote:

I'm not particularly defending IE's legacy enumeration order, we
were initially on board with ES5 adopting the de facto order used by
other browsers. My recollection is that the decision to table
defining a strict enumeration order for ES5 was made after we
discovered that Opera for its optimized arrays did not follow the
expected order mail.mozilla.org/pipermail/es5-discuss/2008-May/001286.html and mail.mozilla.org/pipermail/es5-discuss/2008-June/001304.html and many other messages in that thread.

I think my email on this thread summarized the current areas of
divergent behavior (complete with test cases). It basically amounts to:

Different behavior for index-like properties (either insertion order
or numeric order, in some implementations different between arrays and
non-arrays). Firefox and Safari seem to agree, no other two browsers
agree with each other.
Different behavior for prototype properties in IE.
Properties created by a constructor are enumerated in arbitrary
order, in Chrome only (I assume this is a bug, so I'm going to ignore
it in the discussion below).

We definitely should try to define a rational enumeration order. It
is a secondary issues as to whether we impose it as a potentially
breaking change upon the existing for-in or introduce it via new
syntax. I think the second alterative is an approach we should at
least consider not just in this situation but in all similar
situations. Sometimes it will make sense and other times it won't.

Even if we don't fully specify the order for for..in, there is a
significant intersection of behavior that Web content depends on,
namely that own properties which don't look like indices get
enumerated in insertion order. Even if we don't fully specify the
order, we should at least specify that much in a future ES spec.

I also think removing the differences in handling of index properties
and prototype properties is unlikely to be a breaking change for
anyone, so I believe we could fully specify the order in a future ES
spec.

, Maciej

On Oct 15, 2009, at 10:23 AM, Allen Wirfs-Brock wrote:

> I'm not particularly defending IE's legacy enumeration order, we  
> were initially on board with ES5 adopting the de facto order used by  
> other browsers.  My recollection is that the decision to table  
> defining a strict enumeration order for ES5 was made after we  
> discovered that Opera for its optimized arrays did not follow the  
> expected order https://mail.mozilla.org/pipermail/es5-discuss/2008-May/001286.html 
>  and https://mail.mozilla.org/pipermail/es5-discuss/2008-June/001304.html 
>  and many other messages in that thread.

I think my email on this thread summarized the current areas of  
divergent behavior (complete with test cases). It basically amounts to:

- Different behavior for index-like properties (either insertion order  
or numeric order, in some implementations different between arrays and  
non-arrays). Firefox and Safari seem to agree, no other two browsers  
agree with each other.
- Different behavior for prototype properties in IE.
- Properties created by a constructor are enumerated in arbitrary  
order, in Chrome only (I assume this is a bug, so I'm going to ignore  
it in the discussion below).

> We definitely should try to define a rational enumeration order.  It  
> is a secondary issues as to whether we impose it as a potentially  
> breaking change upon the existing for-in or introduce it via new  
> syntax.  I think the second alterative is an approach we should at  
> least consider not just in this situation but in all similar  
> situations.  Sometimes it will make sense and other times it won't.

Even if we don't fully specify the order for for..in, there is a  
significant intersection of behavior that Web content depends on,  
namely that own properties which don't look like indices get  
enumerated in insertion order. Even if we don't fully specify the  
order, we should at least specify that much in a future ES spec.

I also think removing the differences in handling of index properties  
and prototype properties is unlikely to be a breaking change for  
anyone, so I believe we could fully specify the order in a future ES  
spec.

Regards,
Maciej

>
> Allen
>
>
>> -----Original Message-----
>> From: Brendan Eich [mailto:brendan at mozilla.com]
>> Sent: Wednesday, October 14, 2009 10:26 PM
>> To: Allen Wirfs-Brock
>> Cc: Mike Shaver; Waldemar Horwat; es-discuss
>> Subject: Re: Property Iteration in JSON serialization
>>
>> On Oct 14, 2009, at 7:01 PM, Allen Wirfs-Brock wrote:
>>
>>> Similarly, rather than trying to fully specifying for-in enumeration
>>> order (and invariably causing potentially breaking changes for some
>>> implementations)
>>
>> I'm focusing on your parenthetical aside: we have precedent in  
>> Harmony
>> (functions in blocks) for finding room to create a better standard
>> where browsers do not agree, and therefore unless browser detection  
>> is
>> going on, cross-platform JS on the web cannot depend on the
>> intersection semantics (the empty set).
>>
>> I think we should investigate breaking changes. The IE JScript
>> deviations document linked from
>>
>> http://wiki.ecmascript.org/doku.php?id=es3.1:es3.x_working_docs&s=jscrip
>> t+deviations
>>
>> contains some amazing details of IE"s for-in behavior:
>>
>> * Enumerate properties on immediate prototype in reverse order (w.r.t
>> the order in which they were added)
>>
>> * Enumerate properties in-order on rest of the objects on the
>> prototype chain
>>
>> * Enumerate properties in-order on present object.
>>
>> Reverse insertion order on the immediate prototype properties, not
>> something other engines do.
>>
>> Array element enumeration details are lacking, but again if the
>> observation that array elements are almost always created in index
>> order, then insertion order == index order. The exceptions may or may
>> not work as expected on insertion-order-only implementations. Firefox
>> 3+ uses index order. We're not receiving bug reports.
>>
>> There's room to find a better common semantics here, it seems to me.
>>
>> /be
>
> _______________________________________________
> es-discuss mailing list
> es-discuss at mozilla.org
> https://mail.mozilla.org/listinfo/es-discuss

# liorean (15 years ago)

If I recall correct, Opera has a weird behaviour where it follows a certain predictable ordering - unless you're deleting/removing a property (or use prototype functions that do that in their operation) on an object, which radically changes the sorting order in a way that is only predictable with knowledge of how the object looked at the time of the deleting/removing of a property. IIRC the properties present on the object before the delete got shuffled to the end of the enumeration order.

If I recall correct, Opera has a weird behaviour where it follows a
certain predictable ordering - unless you're deleting/removing a
property (or use prototype functions that do that in their operation)
on an object, which radically changes the sorting order in a way that
is only predictable with knowledge of how the object looked at the
time of the deleting/removing of a property. IIRC the properties
present on the object before the delete got shuffled to the end of the
enumeration order.
-- 
David "liorean" Andersson