[[OwnPropertyKeys]] key ordering
Awesome! Please let's do this!
I don't understand the comment about Object.keys, etc. If they continue to be specified in terms of [[OwnPropertyKeys]], then they get cleaned up as well, which seems good.
Is there a reason why we might want to continue to underspecify Object.keys, etc?
If there is, how would you continue to underspecify Object.keys etc, while still cleaning up [[OwnPropertyKeys]]?
Do we also have an opportunity to (finally!) pin down for/in ordering as part of this?
On Apr 19, 2014, at 11:21 AM, Mark S. Miller wrote:
Awesome! Please let's do this!
I don't understand the comment about Object.keys, etc. If they continue to be specified in terms of [[OwnPropertyKeys]], then they get cleaned up as well, which seems good.
Object.keys currently says it produces the same ordering as for-in , but for-in ordering is implementation defined. We've never convinced ourselves that implementations would willing change to confirm to a required for-in ordering.
Is there a reason why we might want to continue to underspecify Object.keys, etc?
The same reasons why it was under specified in the first place. Implementation conformance and at this point concerns about breaking changes.
If there is, how would you continue to underspecify Object.keys etc, while still cleaning up [[OwnPropertyKeys]]?
By saying: After retrieving the keys using [[OwnPropertyKeys]] an implementation must reordering to conform to the for-in enumeration order.
Do we also have an opportunity to (finally!) pin down for/in ordering as part of this?
I don't think so, and I think that is too big of an issue to consider at this time. But I don't see any reason that new functions should use a well specified ordering.
Did you check against strawman:enumeration which links off to this es-discuss thread:
Sounds good, just asking for a look-back at the big thread and strawman to see if there are any missing subtleties others have pointed out in the past.
On Apr 19, 2014, at 11:14 AM, Allen Wirfs-Brock wrote:
I'm in the middle of updating the spec. of [[OwnPropertyKeys]] to returns an Array rather than an Iterator. While doing this I realized that because [[OwnPropertyKeys]] is essentially a new MOP level operation we have the opportunity to precisely define the property key ordering it exposes for ordinary objects. This seems like a one-time opportunity that we shouldn't pass up.
Note that this doesn't necessarily mean we need to change the unspecified/weakly specified ordering of exiting library functions such as Object.keys (even though they are specified in terms of [[OwnPropertyKeys]]). But it does mean that new library functions such as Reflect.ownKeys will be defined with a fully specified specified ordering.
The ordering I propose is:
- All array index property keys, in ascending array index numeric order. Followed by:
- All other string property keys, in property creation order. Followed by:
- All symbol property keys, in property creation order
Does anybody see any reason why we shouldn't specify (this) property ordering?
I need to update item 1) above as follows because "array index" is implies 32-bit value:
- All integer index property keys, in ascending numeric order. Followed by...
where integer index is defined as "An integer index is String-valued property key that is a canonical numeric string (see 7.1.16) and whose numeric value is either +0 or a positive integer."
and canonical numeric string is defined at people.mozilla.org/~jorendorff/es6-draft.html#sec-canonicalnumericstring
Definition includes integer values that exceed the Uint32 limit or the actual length of an Array object.
Negative integers, including "-0" if it actually exists as a property key are not included in this first sublist.
On Apr 19, 2014, at 12:17 PM, Brendan Eich wrote:
Did you check against strawman:enumeration which links off to this es-discuss thread:
Sounds good, just asking for a look-back at the big thread and strawman to see if there are any missing subtleties others have pointed out in the past.
In a separate message I made a change that addresses 32-bit value non-limits on index keys.
I fairly arbitrarily separated the string and symbol keys groups. At the specific implementation level this might complicate (or possibly simplify) things, but it seems like a useful distinction for consumers of these key lists. For example, if you don't care about symbol keys you can stop iterating over the key array as soon as you encounter a symbol.
If we didn't know that the web has property creation ordering dependencies I might have specified sorting the string keys (doesn't require recording property creation order). However, there is no natural sort order for symbols.
The proposed ordering for [[OwnPropertyKeys]] is the same as the current for..in ordering in the latest versions of most browsers.
As Allen pointed out, this group has basically declined to specify for..in ordering, and specifying an order for [[OwnPropertyKeys]] technically leaves for..in order unspecified.
However, I believe that specifying an order for [[OwnPropertyKeys]] effectively specifies the order of for..in iteration, for the practical reason that browser implementers are really unlikely to provide two possible orderings. It would require a bunch of extra data structures and indices to offer good performance for multiple possible orderings.
Given this, I would suggest that if ECMA is going to specify an order for [[OwnPropertyKeys]], it's time to go ahead and specify for..in iteration order too.
Regardless of what the ordering is (what I prefer, or what Allen proposed), if for..in ordering is technically unspecified but implementation concerns basically require a particular order, that's the worst of all outcomes: every browser is consistent and the matter is really settled, but no one can rely on it.
However, for the record, I continue to think that special treatment of numeric keys is a really bad design for property order for Objects (for Arrays, it's fine). To summarize the arguments that seemed to play best with members of this group:
- special treatment of Array indices is surprising
If we're spec'ing that order is preserved for String properties, people will continue to routinely use Objects as ordered maps [1].
This leads to the gotcha that some properties fall out of order, just because they happen to be parsable as numbers, even if you explicitly quoted the property name in your code. Consider:
var extensions = { "0110":"Marketing", "1225":"Development", "0301":"Support" };
This way of specifying phone extensions in order would work, except that "Development" is inexplicably placed first because 1225 is an array index.
To be crystal clear: I'm not raising this as a backwards compatibility issue (which some people seem to take as a cue to argue uselessly about who is to blame..).
What I'm saying is that this is a "gotcha" being permanently enshrined in the spec, that will bite people from time to time for as long as ECMAScript is in use.
- treating array index keys specially loses information
If order was preserved, an Object would be a very convenient structure for storing the list of available options in order, as a map from stored integer values to user-visible Strings:
var availableOptions = { "9" : "Fixed" "3" : "Won't Fix" "5" : "Invalid" };
Not being able to to do this is a loss of expressiveness for a very common use case in database-oriented apps.
- treating array indices as special does not match any known use case
No one has identified any application in which it would be desirable for Object (again not Array) to have this particular quirk of property ordering. It's purely an optimization detail leaking through.
[1] if anyone doubts that using Objects as ordered maps has been a long-standing practice, just look at the record-setting 178 stars and long comment thread on just one of the several bugs filed against Chrome for not preserving order for integer keys:
https://code.google.com/p/v8/issues/detail?id=164
There's no reason to assume this practice will go away, especially since savvy developers will now know that it's safe to assume order preservation for String keys.
On 4/19/14, 4:38 PM, Charles Kendrick wrote:
However, for the record, I continue to think that special treatment of numeric keys is a really bad design for property order for Objects (for Arrays, it's fine).
In practice people use plain objects as arrays (i.e. setting indexed properties on them, then getting them, and expecting performance to match that of arrays) that JS engines have explicitly moved toward having more or less the same internal representation for Object and Array, since you need the indexed access optimizations for both.
So in practice engines actually have different internal representations for indexed (in at least some cases up to some number that's much smaller than 2^32) properties and other properties, and the enumeration behavior browsers have right now falls out of those internal representations.
Now you might argue that using plain objects as arrays (in benchmarks, no less) is bad form, and I'd agree, but the code out there is what it is.
- treating array indices as special does not match any known use case
No one has identified any application in which it would be desirable for Object (again not Array) to have this particular quirk of property ordering. It's purely an optimization detail leaking through.
Indeed. The problem is that the optimization in this case involves discarding the information that you'd need to produce a different enumeration order...
On 19 April 2014 20:14, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
I'm in the middle of updating the spec. of [[OwnPropertyKeys]] to returns an Array rather than an Iterator. While doing this I realized that because [[OwnPropertyKeys]] is essentially a new MOP level operation we have the opportunity to precisely define the property key ordering it exposes for ordinary objects. This seems like a one-time opportunity that we shouldn't pass up.
Note that this doesn't necessarily mean we need to change the unspecified/weakly specified ordering of exiting library functions such as Object.keys (even though they are specified in terms of [[OwnPropertyKeys]]). But it does mean that new library functions such as Reflect.ownKeys will be defined with a fully specified specified ordering.
The ordering I propose is:
- All array index property keys, in ascending array index numeric order. Followed by:
- All other string property keys, in property creation order. Followed by:
- All symbol property keys, in property creation order
Creation order? I don't know about other VM's, but for V8 it's unlikely that we are willing to pay the price for tracking that accurately.
On Apr 22, 2014, at 12:02 AM, Andreas Rossberg wrote:
On 19 April 2014 20:14, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
The ordering I propose is:
- All array index property keys, in ascending array index numeric order. Followed by:
- All other string property keys, in property creation order. Followed by:
- All symbol property keys, in property creation order
Creation order? I don't know about other VM's, but for V8 it's unlikely that we are willing to pay the price for tracking that accurately.
My recollection of the last time we surveyed this was that all major engines did for-in enumeration of non-array index own property keys in creations/insertion order. Has something changed for V8 in this regard? My understanding is that the web depends upon that ordering for such properties.
An alternative would be to specify a sort ordering for string keyed properties. That might be possible because this is a new MOP level operation. However, some of the uses of it within the ES6 standard, such as Object.keys, require that they produce the same ordering as for-in. Also, there is no natural sort ordering for Symbols
On 22 April 2014 16:52, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Apr 22, 2014, at 12:02 AM, Andreas Rossberg wrote:
On 19 April 2014 20:14, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
The ordering I propose is:
- All array index property keys, in ascending array index numeric order. Followed by:
- All other string property keys, in property creation order. Followed by:
- All symbol property keys, in property creation order
Creation order? I don't know about other VM's, but for V8 it's unlikely that we are willing to pay the price for tracking that accurately.
My recollection of the last time we surveyed this was that all major engines did for-in enumeration of non-array index own property keys in creations/insertion order. Has something changed for V8 in this regard? My understanding is that the web depends upon that ordering for such properties.
We do track it for up to 2^24 properties, after that we give up. As far as the web is concerned, that is more than enough (and you most likely run OOM before that anyway). A specialised test might try to push it, though. I suppose you could argue that is simply an implementation limit, like stack limits.
On Apr 22, 2014, at 8:45 AM, Andreas Rossberg wrote:
On 22 April 2014 16:52, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Apr 22, 2014, at 12:02 AM, Andreas Rossberg wrote:
On 19 April 2014 20:14, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
The ordering I propose is:
- All array index property keys, in ascending array index numeric order. Followed by:
- All other string property keys, in property creation order. Followed by:
- All symbol property keys, in property creation order
Creation order? I don't know about other VM's, but for V8 it's unlikely that we are willing to pay the price for tracking that accurately.
My recollection of the last time we surveyed this was that all major engines did for-in enumeration of non-array index own property keys in creations/insertion order. Has something changed for V8 in this regard? My understanding is that the web depends upon that ordering for such properties.
We do track it for up to 2^24 properties, after that we give up. As far as the web is concerned, that is more than enough (and you most likely run OOM before that anyway). A specialised test might try to push it, though. I suppose you could argue that is simply an implementation limit, like stack limits.
I think treating this as an implementation limitation is just fine. I'd prefer not to explicitly state such a permision in the spec. (just like we don't for stack depth) because I wouldn't want to come up with an specific number and I wouldn't want any implementation to interpret the lack of a specific number as permission to set a silly limit like 8.
I would prefer that this implementation limit be treated even more like the stack limit; say by throwing an error rather than silently adding a 1+2^24 property that breaks the contract. Any reason not to throw an error on this?
That said, I agree that this should be an implementation limit rather than relaxing the contract. As an implementation limit, it is currently none of the spec's business how an implementation handles it. However, sound reasoning about security is impossible so long as such behavior in not specced. We'll get around to speccing constraints on how implementations may go wrong when they exceed this limit when we do for other limits, such as stack depth. For example, although stack depth limits are outside the spec, we'd all consider it wrong for an implementation to lose memory safety when this limit is exceeded. Eventually, likewise for some other limits and some other invariants.
I'm in the middle of updating the spec. of [[OwnPropertyKeys]] to returns an Array rather than an Iterator. While doing this I realized that because [[OwnPropertyKeys]] is essentially a new MOP level operation we have the opportunity to precisely define the property key ordering it exposes for ordinary objects. This seems like a one-time opportunity that we shouldn't pass up.
Note that this doesn't necessarily mean we need to change the unspecified/weakly specified ordering of exiting library functions such as Object.keys (even though they are specified in terms of [[OwnPropertyKeys]]). But it does mean that new library functions such as Reflect.ownKeys will be defined with a fully specified specified ordering.
The ordering I propose is:
Does anybody see any reason why we shouldn't specify (this) property ordering?