[[Extensible]]and Proxies (Was: Proxy.isProxy )
Aren't arrays [[Extensible]]:true by default?
As to why the restriction, this is to uphold the following invariants (from ES5 section 8.6.2):
-
If the value of the host object‘s [[Extensible]] internal property has been observed by ECMAScript code to be false, then if a call to [[GetOwnProperty]] describes a property as non-existent all subsequent calls must also describe that property as non-existent.
-
The [[DefineOwnProperty]] internal method of a host object must not permit the addition of a new property to a host object if the [[Extensible]] internal property of that host object has been observed by ECMAScript code to be false.
-
If the [[Extensible]] internal property of that host object has been observed by ECMAScript code to be false then it must not subsequently become true.
Cheers, Tom
2011/7/13 Allen Wirfs-Brock <allen at wirfs-brock.com>
Tom Van Cutsem wrote:
Aren't arrays [[Extensible]]:true by default?
Yes, but any array object can be set to [[Extensible]]: false and the implicit delete behavior defined in 15.4.5.1 for when length is set to a smaller value still applies. To emulate that behavior at least the set/defineProperty traps need to work even when [[Extensible]] is false.
As to why the restriction, this is to uphold the following invariants (from ES5 section 8.6.2):
If the value of the host object‘s [[Extensible]] internal property has been observed by ECMAScript code to be false, then if a call to [[GetOwnProperty]] describes a property as non-existent all subsequent calls must also describe that property as non-existent.
The [[DefineOwnProperty]] internal method of a host object must not permit the addition of a new property to a host object if the [[Extensible]] internal property of that host object has been observed by ECMAScript code to be false.
If the [[Extensible]] internal property of that host object has been observed by ECMAScript code to be false then it must not subsequently become true.
Yes, but the restriction prohibits otherwise legal behavior that doesn't violate the invariant. Isn't that as "bad" as not enforcing the invariant?
On Wed, Jul 13, 2011 at 8:38 AM, Allen Wirfs-Brock <allen at wirfs-brock.com>wrote:
Hate to start another cycle of this, but if (trapping) proxies can't be set to [[Extensible]]: false, then they can't be used to fully emulate built-ins such as Array. Why is there this restriction?
On Wed, Jul 13, 2011 at 10:30 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
Yes, but the restriction prohibits otherwise legal behavior that doesn't violate the invariant. Isn't that as "bad" as not enforcing the invariant?
What behavior of non-extensible arrays cannot be emulated by a normal non-proxy object?
Le 13/07/2011 20:52, Mark S. Miller a écrit :
On Wed, Jul 13, 2011 at 8:38 AM, Allen Wirfs-Brock <allen at wirfs-brock.com <mailto:allen at wirfs-brock.com>> wrote:
Hate to start another cycle of this, but if (trapping) proxies can't be set to [[Extensible]]: false, then they can't be used to fully emulate built-ins such as Array. Why is there this restriction?On Wed, Jul 13, 2011 at 10:30 AM, Allen Wirfs-Brock <allen at wirfs-brock.com <mailto:allen at wirfs-brock.com>> wrote:
Yes, but the restriction prohibits otherwise legal behavior that doesn't violate the invariant. Isn't that as "bad" as not enforcing the invariant?What behavior of non-extensible arrays cannot be emulated by a normal non-proxy object?
I'm not sure I understand your question since "normal non-proxy object" cannot really emuate anything. Did you mean to ask what behavior of non-extensible arrays cannot be emulated by proxies? If that's the question:
var a = Array(1,2,3,4); // a.length===4 Object.preventExtensions(a); console.log(a[3]); // 4, of course a.length = 3; // should delete a[3] under the hood. console.log(a[3]);
Logs "undefined" with a conformant ECMAScript Array. This cannot be emulated with current proxies. Current choices are:
- Throw a TypeError on Object.preventExtensions
- Accept to be transformed into a normal object fully respecting ES5.1 8.12 algorithms and log "4"
- Turn the "length" property into a getter/setter pair which violates ES5.1 15.4.5.2 which ask the length property to be a non-configurable data property.
On Thu, Jul 14, 2011 at 6:01 AM, David Bruant <david.bruant at labri.fr> wrote:
Le 13/07/2011 20:52, Mark S. Miller a écrit :
On Wed, Jul 13, 2011 at 8:38 AM, Allen Wirfs-Brock <allen at wirfs-brock.com>wrote:
Hate to start another cycle of this, but if (trapping) proxies can't be set to [[Extensible]]: false, then they can't be used to fully emulate built-ins such as Array. Why is there this restriction?
On Wed, Jul 13, 2011 at 10:30 AM, Allen Wirfs-Brock < allen at wirfs-brock.com> wrote:
Yes, but the restriction prohibits otherwise legal behavior that doesn't violate the invariant. Isn't that as "bad" as not enforcing the invariant?
What behavior of non-extensible arrays cannot be emulated by a normal non-proxy object?
I'm not sure I understand your question since "normal non-proxy object" cannot really emuate anything. Did you mean to ask what behavior of non-extensible arrays cannot be emulated by proxies?
I did mean non-proxy object. In particular, a non-extensible, non-proxy object whose length is an accessor property, so that decreasing it can cause deletion, as in your third bullet below.
If that's the question:
var a = Array(1,2,3,4); // a.length===4 Object.preventExtensions(a); console.log(a[3]); // 4, of course a.length = 3; // should delete a[3] under the hood. console.log(a[3]);
Logs "undefined" with a conformant ECMAScript Array. This cannot be emulated with current proxies. Current choices are:
- Throw a TypeError on Object.preventExtensions
- Accept to be transformed into a normal object fully respecting ES5.1 8.12 algorithms and log "4"
- Turn the "length" property into a getter/setter pair which violates ES5.1 15.4.5.2 which ask the length property to be a non-configurable data property.
So having the length be an accessor means that such an object is not a perfectly transparent emulation of an array. What is a use case this non-transparency actually matters. In all the cases I can think of, where we want precisely the behavior of an array, why not just use an array?
Le 14/07/2011 18:57, Mark S. Miller a écrit :
On Thu, Jul 14, 2011 at 6:01 AM, David Bruant <david.bruant at labri.fr <mailto:david.bruant at labri.fr>> wrote:
If that's the question: ----- var a = Array(1,2,3,4); // a.length===4 Object.preventExtensions(a); console.log(a[3]); // 4, of course a.length = 3; // should delete a[3] under the hood. console.log(a[3]); ----- Logs "undefined" with a conformant ECMAScript Array. This cannot be emulated with current proxies. Current choices are: * Throw a TypeError on Object.preventExtensions * Accept to be transformed into a normal object fully respecting ES5.1 8.12 algorithms and log "4" * Turn the "length" property into a getter/setter pair which violates ES5.1 15.4.5.2 which ask the length property to be a non-configurable data property. So having the length be an accessor means that such an object is not a perfectly transparent emulation of an array. What is a use case this non-transparency actually matters.
I guess I can turn the question into: What motivated ES5.1 15.4.5.2 to define the length property as a data property and keep the step 3 of 15.4.5.1 rather than a defining a getter/setter pair and remove step 3?
Let assume arrays length property are non-configurable getter/setter for a minute. If you hand an array to some untruted code, this code can extract the length setter and set length as it wishes. You have no way to change the setter yourself as the property is non-configurable. So, in pure ES5.1 + (assumption of length being a non-configurable getter/setter pair), as soon as you hand an array, you'd grant the right to change the length at any time to the untrusted code. (this may be the answer to my question above) Still under the same assumption, in the Harmony era (with proxies), a workaround could be to wrap your array in a proxy preventing access to the length property descriptor. But it sounds much more work than "just" exposing a data property.
In all the cases I can think of, where we want precisely the behavior of an array, why not just use an array?
Of course an array should be used. I think that the current question is rather "can proxies as currently defined can fully emulate arrays?". Being able to fully emulate arrays sounds like an "easy" use case.
On Jul 14, 2011, at 10:38 AM, David Bruant wrote:
In all the cases I can think of, where we want precisely the behavior of an array, why not just use an array? Of course an array should be used. I think that the current question is rather "can proxies as currently defined can fully emulate arrays?". Being able to fully emulate arrays sounds like an "easy" use case.
I agree, and Allen is on the record here too.
Fixing an array-proxy to an array could be enough for real-world use-cases, so this may be not a burning practical issue. Indeed it verges on Principle. But it's a good one, and I think we should have a goal (rather than a principle) for cleanly proxying arrays, including non-extensible ones.
If this goal is met by saying fixing an array proxy always results in the instance becoming a fixed Array, that could be good enough.
I do not think we should say "fix to a non-extensbile object with a length accessor" while we at the same time carefully spec Array length as a data property, precisely to ensure inter-operation. That seems to be telling JS implementors one thing, and (possibly an overlapping group) proxy users another.
On Jul 14, 2011, at 10:47 AM, Brendan Eich wrote:
On Jul 14, 2011, at 10:38 AM, David Bruant wrote:
In all the cases I can think of, where we want precisely the behavior of an array, why not just use an array? Of course an array should be used. I think that the current question is rather "can proxies as currently defined can fully emulate arrays?". Being able to fully emulate arrays sounds like an "easy" use case.
I agree, and Allen is on the record here too.
Fixing an array-proxy to an array could be enough for real-world use-cases, so this may be not a burning practical issue. Indeed it verges on Principle. But it's a good one, and I think we should have a goal (rather than a principle) for cleanly proxying arrays, including non-extensible ones.
If this goal is met by saying fixing an array proxy always results in the instance becoming a fixed Array, that could be good enough.
I do not think we should say "fix to a non-extensbile object with a length accessor" while we at the same time carefully spec Array length as a data property, precisely to ensure inter-operation. That seems to be telling JS implementors one thing, and (possibly an overlapping group) proxy users another.
/be
I think Mark and I are looking at Proxies from very different perspectives. I want Proxies to to useful to create a perfect implementation of anything that be specified as a ES built-in object or that might be implemented whether now or in the future as a host object defined in some other programming language. I hold up ES array objects, string objects, argument objects, functions objects, and various DOM implementations simply as exemplars of such objects that somebody might want to implement using Proxies.
Historically there never were any real restriction on what such objects can do. The DOM and RegExp.prototype.compile demonstrate this. The host object object restrictions added in ES5 don't apply to built-in objects and regardless are questionable in their effect. For example, the ES5 spec says "Host object may implement these internal methods in any manner unless otherwise specified" and "The [[DefineOwnProperty]] internal method of a host object must not permit the addtion of a new property...if the [[Extensible]]...has been observed...to be false". So, in that case [[DefineOwnProperty]] can't add a new property but [[Delete]], or [[Get]], or [[DefaultValue]] or any other the internal methods could as nothing specifies that they can't. Finally, in adding the restrictions to ES5 I don't think there was any serious effort to see whether or not the restrictions were consistent with what host objects actually did. Certainly they are not consistent with the commonly implemented behavior of RegExp.prototype.compile.
My sense is that Mark has some specific invariants that he wishes were guaranteed in ES implementations and he wants to make it impossible to violate those invariants using Proxies even though there is no such invariants imposed on built-in objects and the host object invariants are incomplete and hard to make impossible to violated.
If we really want to do this, then we need to really define a complete and self consistent set of invariants that we are willing to apply to all objects (include built-ins and host) now and into the future. Also in designing any guarantees into Proxies we need to make sure that they they do not impose any additional restrictions other than what is actually stated or implied by the agreed upon invariants. Conservative guarantees that go beyond the invariants are not acceptable. If we were going to enforce such upon Proxies then we need to also enforce the same for built-in and host objects.
Personally, I'm skeptical that we can agree upon such a set of invariants that are compatible with the legacy language and DOM. I'm also an skeptical of the need to do so for Proxies or for built-in and host objects. But, if we want to go for it, then we should proceed and try to define the universal object invariants and see if we can reach consensus on accepting them. If we can't, then we shouldn't be restricting Proxy semantics based upon some assumed invariants.
On Thu, Jul 14, 2011 at 12:19 PM, Allen Wirfs-Brock <allen at wirfs-brock.com>wrote:
I think Mark and I are looking at Proxies from very different perspectives.
Agreed.
I want Proxies to to useful to create a perfect implementation of anything that be specified as a ES built-in object or that might be implemented whether now or in the future as a host object defined in some other programming language.
Absent other costs, that's a fine thing to want. But because of those costs, it was never a goal of the proxies design.
I hold up ES array objects, string objects, argument objects, functions objects, and various DOM implementations simply as exemplars of such objects that somebody might want to implement using Proxies.
Historically there never were any real restriction on what such objects can do.
The DOM and RegExp.prototype.compile demonstrate this. The host object
object restrictions added in ES5 don't apply to built-in objects
Allen, you and I have been over this, both on list and off. This is simply not true. All the restrictions at the end of 8.6.2 are stated as applying only to host (i.e., non-native) objects only because we did not need to state that they apply to native objects. E5.1 specifies the behavior of native objects in enough detail that we can read the spec and verify that they must obey these constraints as well.
and regardless are questionable in their effect.
That's why we have test suites.
For example, the ES5 spec says "Host object may implement these internal methods in any manner unless otherwise specified" and "The [[DefineOwnProperty]] internal method of a host object must not permit the addtion of a new property...if the [[Extensible]]...has been observed...to be false". So, in that case [[DefineOwnProperty]] can't add a new property but [[Delete]], or [[Get]], or [[DefaultValue]] or any other the internal methods could as nothing specifies that they can't.
I agree that this is a spec bug. We should fix this in the ES5 errata.
Finally, in adding the restrictions to ES5 I don't think there was any serious effort to see whether or not the restrictions were consistent with what host objects actually did. Certainly they are not consistent with the commonly implemented behavior of RegExp.prototype.compile.
Ok, I admit I haven't paid as much attention to RegExp.prototype.compile as I should, because, thankfully, it is deletable on all major browsers. Where can I read about its cross-browser de-facto behavior?
My sense is that Mark has some specific invariants that he wishes were guaranteed in ES implementations and he wants to make it impossible to violate those invariants using Proxies
y
even though there is no such invariants imposed on built-in objects
wrong, as we've discussed numerous times.
and the host object invariants are incomplete and hard to make impossible to violated.
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
If we really want to do this, then we need to really define a complete and self consistent set of invariants that we are willing to apply to all objects (include built-ins and host) now and into the future. Also in designing any guarantees into Proxies we need to make sure that they they do not impose any additional restrictions other than what is actually stated or implied by the agreed upon invariants. Conservative guarantees that go beyond the invariants are not acceptable. If we were going to enforce such upon Proxies then we need to also enforce the same for built-in and host objects.
Yes. For a start,
- That the statechart at < bugzilla.mozilla.org/show_bug.cgi?id=630284> represents all the
states and transition possible for a property.
- 8.6.2, with the clarification that it applies to both native and non-native, and with the [[Extensible]] spec bug that you point out fixed.
- The "eternal invariants" from < esdiscuss/2011-May/014150>.
Personally, I'm skeptical that we can agree upon such a set of invariants that are compatible with the legacy language and DOM. I'm also an skeptical of the need to do so for Proxies or for built-in and host objects. But, if we want to go for it, then we should proceed and try to define the universal object invariants and see if we can reach consensus on accepting them. If we can't, then we shouldn't be restricting Proxy semantics based upon some assumed invariants.
Until and unless we can all agree on something else, we stick with what we have already agreed to. With our current agreement as a baseline, I am confident we can agree on something better -- like cleaning up the [[Extensible]] spec bug you point out above.
On Thu, Jul 14, 2011 at 1:19 PM, Mark S. Miller <erights at google.com> wrote:
On Thu, Jul 14, 2011 at 12:19 PM, Allen Wirfs-Brock <allen at wirfs-brock.com
wrote:
I think Mark and I are looking at Proxies from very different perspectives.
Agreed.
I want Proxies to to useful to create a perfect implementation of anything that be specified as a ES built-in object or that might be implemented whether now or in the future as a host object defined in some other programming language.
Absent other costs, that's a fine thing to want. But because of those costs, it was never a goal of the proxies design.
I hold up ES array objects, string objects, argument objects, functions objects, and various DOM implementations simply as exemplars of such objects that somebody might want to implement using Proxies.
Historically there never were any real restriction on what such objects can do.
The DOM and RegExp.prototype.compile demonstrate this. The host object
object restrictions added in ES5 don't apply to built-in objects
Allen, you and I have been over this, both on list and off. This is simply not true. All the restrictions at the end of 8.6.2 are stated as applying only to host (i.e., non-native) objects only because we did not need to state that they apply to native objects. E5.1 specifies the behavior of native objects in enough detail that we can read the spec and verify that they must obey these constraints as well.
and regardless are questionable in their effect.
That's why we have test suites.
For example, the ES5 spec says "Host object may implement these internal methods in any manner unless otherwise specified" and "The [[DefineOwnProperty]] internal method of a host object must not permit the addtion of a new property...if the [[Extensible]]...has been observed...to be false". So, in that case [[DefineOwnProperty]] can't add a new property but [[Delete]], or [[Get]], or [[DefaultValue]] or any other the internal methods could as nothing specifies that they can't.
I agree that this is a spec bug. We should fix this in the ES5 errata.
Finally, in adding the restrictions to ES5 I don't think there was any serious effort to see whether or not the restrictions were consistent with what host objects actually did. Certainly they are not consistent with the commonly implemented behavior of RegExp.prototype.compile.
Ok, I admit I haven't paid as much attention to RegExp.prototype.compile as I should, because, thankfully, it is deletable on all major browsers. Where can I read about its cross-browser de-facto behavior?
My sense is that Mark has some specific invariants that he wishes were guaranteed in ES implementations and he wants to make it impossible to violate those invariants using Proxies
y
even though there is no such invariants imposed on built-in objects
wrong, as we've discussed numerous times.
and the host object invariants are incomplete and hard to make impossible to violated.
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
If we really want to do this, then we need to really define a complete and self consistent set of invariants that we are willing to apply to all objects (include built-ins and host) now and into the future. Also in designing any guarantees into Proxies we need to make sure that they they do not impose any additional restrictions other than what is actually stated or implied by the agreed upon invariants. Conservative guarantees that go beyond the invariants are not acceptable. If we were going to enforce such upon Proxies then we need to also enforce the same for built-in and host objects.
Yes. For a start,
- That the statechart at < bugzilla.mozilla.org/show_bug.cgi?id=630284> represents all the states and transition possible for a property.
Oops. Copy-paste error. Should be < es3.1:attribute_states>.
On Jul 14, 2011, at 1:19 PM, Mark S. Miller wrote:
On Thu, Jul 14, 2011 at 12:19 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote: I think Mark and I are looking at Proxies from very different perspectives.
Agreed.
I want Proxies to to useful to create a perfect implementation of anything that be specified as a ES built-in object or that might be implemented whether now or in the future as a host object defined in some other programming language.
Absent other costs, that's a fine thing to want. But because of those costs, it was never a goal of the proxies design.
I think several of us have been quite consistent in expressing that we are looking for a mechanism for implementing in pure ES things that currently require host objects or enhanced object semantics of the sort that historically has been defined for built-ins such as Array objects. If that was not one of the goals for Proxies I misunderstood. Perhaps Proxy is not what we need then.
I hold up ES array objects, string objects, argument objects, functions objects, and various DOM implementations simply as exemplars of such objects that somebody might want to implement using Proxies.
Historically there never were any real restriction on what such objects can do. The DOM and RegExp.prototype.compile demonstrate this. The host object object restrictions added in ES5 don't apply to built-in objects
Allen, you and I have been over this, both on list and off. This is simply not true. All the restrictions at the end of 8.6.2 are stated as applying only to host (i.e., non-native) objects only because we did not need to state that they apply to native objects. E5.1 specifies the behavior of native objects in enough detail that we can read the spec and verify that they must obey these constraints as well.
Implementations are free to add new built-in objects (see 4.3.28). Without any stated requirements for such objects you have no guarantees.
Further more, in writing specification material such as the alternative internal methods, I was never aware of any requirements that had to be meet by those implementations other than whatever was necessary to describe the legacy or new semantics that we were trying to specify. As far as I know, there is no comprehensive normative specification of invariant that must be maintained by all objects.
and regardless are questionable in their effect.
That's why we have test suites.
For example, the ES5 spec says "Host object may implement these internal methods in any manner unless otherwise specified" and "The [[DefineOwnProperty]] internal method of a host object must not permit the addtion of a new property...if the [[Extensible]]...has been observed...to be false". So, in that case [[DefineOwnProperty]] can't add a new property but [[Delete]], or [[Get]], or [[DefaultValue]] or any other the internal methods could as nothing specifies that they can't.
I agree that this is a spec bug. We should fix this in the ES5 errata.
No it is not a bug. There was never any consensus on a larger set of invariants or even as far as I can recall a proposal for such. I don't know whether or not we would have or in the future can achieve consensus on that. For now, ES5.1 is what it is and and unless something is clearly internally inconsistent, unintended breakage from an earlier edition, or breaks long establish browser behaviors I don't think you can call it a bug.
Finally, in adding the restrictions to ES5 I don't think there was any serious effort to see whether or not the restrictions were consistent with what host objects actually did. Certainly they are not consistent with the commonly implemented behavior of RegExp.prototype.compile.
Ok, I admit I haven't paid as much attention to RegExp.prototype.compile as I should, because, thankfully, it is deletable on all major browsers. Where can I read about its cross-browser de-facto behavior?
bugzilla.mozilla.org/show_bug.cgi?id=630284 The Microsoft implementation is described in msdn.microsoft.com/en-us/library/ff955265(v=VS.85).aspx but to be sure what it means you need to write some tests.
These was off-list email earlier among various implementors about this issue. this is what Luke said at the time:
... From a quick check, it appears all browsers today violate the "non-writable, non-configurable properties must not change" rule for these RegExp properties. Technically speaking, the only mention of that rule I'm aware of in ES5.1 is 8.6.2, and only applies to host objects, so I'm not positive that this is really violating ES5. But even if it is, that particular rule is a little bit of a reach anyway - it doesn't seem fundamentally justified by the internal consistency of the spec, and I'm not sure that implementations can really rely on it for optimization given that it is easy enough for host objects (and built-ins) to violate it. With Proxies, it would become even easier to build objects that violate this rule.
Is the right solution here not just to either: a) Relax the ES5 rule that non-writable, non-configurable properties must not change (if it even applies to built-ins) b) Change the regexp instance properties to get-only accessor properties
As for interoperable behavior more generally - the only disagreement in compile semantics I have found is that IE and FF return the regexp object from re.compile(...) as well as modifying the object itself, whereas Chrome does not return the object. That seems relatively minor, and something that could easily be made interoperable in either direction. Are there any further known non-interoperabilities in compile implementations?
Luke
My sense is that Mark has some specific invariants that he wishes were guaranteed in ES implementations and he wants to make it impossible to violate those invariants using Proxies
y
even though there is no such invariants imposed on built-in objects
wrong, as we've discussed numerous times.
So we disagree WRT whether the specified behavior of features in current specification can be extrapolated as imposing limitation on future unrelated features and specifications.
and the host object invariants are incomplete and hard to make impossible to violated.
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
We could provide a defined interface mechanism that validates constraints or limits behavior in a way that guarantees the desired invariants. That's what Proxies appear to be trying to do, why not do it for host objects. If you can depend upon host objects actually supporting your invariants why does not matter whether or not Proxy objects also do so.
If we really want to do this, then we need to really define a complete and self consistent set of invariants that we are willing to apply to all objects (include built-ins and host) now and into the future. Also in designing any guarantees into Proxies we need to make sure that they they do not impose any additional restrictions other than what is actually stated or implied by the agreed upon invariants. Conservative guarantees that go beyond the invariants are not acceptable. If we were going to enforce such upon Proxies then we need to also enforce the same for built-in and host objects.
Yes. For a start,
- That the statechart at bugzilla.mozilla.org/show_bug.cgi?id=630284 represents all the states and transition possible for a property.
- 8.6.2, with the clarification that it applies to both native and non-native, and with the [[Extensible]] spec bug that you point out fixed.
The normative specification of valid state transitions is actually 8.12.9, but that only applied to objects that use that definition of [[DefineOwnProperty]]. Any extrapolation to other situations hasn't been normatively adopted.
- The "eternal invariants" from esdiscuss/2011-May/014150.
We can certainly discuss these, but TC39 hasn't adopted them.
Personally, I'm skeptical that we can agree upon such a set of invariants that are compatible with the legacy language and DOM. I'm also an skeptical of the need to do so for Proxies or for built-in and host objects. But, if we want to go for it, then we should proceed and try to define the universal object invariants and see if we can reach consensus on accepting them. If we can't, then we shouldn't be restricting Proxy semantics based upon some assumed invariants.
Until and unless we can all agree on something else, we stick with what we have already agreed to. With our current agreement as a baseline, I am confident we can agree on something better -- like cleaning up the [[Extensible]] spec bug you point out above.
And what we have already agreed to is only what is currently in the ES5.1 spec, as written. As I expressed above, I don't think there is agreement about extrapolations of the text to situations that aren't explicitly called out in the specification.
Sorry, if that seems unharmonious but it seems like the only solid footing we have.
On Jul 14, 2011, at 3:00 PM, Allen Wirfs-Brock wrote:
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
We could provide a defined interface mechanism that validates constraints or limits behavior in a way that guarantees the desired invariants. That's what Proxies appear to be trying to do, why not do it for host objects. If you can depend upon host objects actually supporting your invariants why does not matter whether or not Proxy objects also do so.
For the record, Mozilla wants its "host objects" to be only as powerful as Proxies. Current web compatibility constraints don't allow this but we want to evolve the standards, de facto (over time) and de jure, toward that goal.
Some Proxy extensions may be needed, too, so it's not all up to web compatibility to take the hit.
Here, I do not think we should keep treating host objects or the "platform provider" as priviileged by ECMA-262 to do whatever the heck they want. The spec gives host objects way more rope, as you point out (lots of lack of specification, and somewhat inconsistent and sparse attempts to constrain host object semantics). That's a historic botch due mostly to the IE DOM.
- The "eternal invariants" from esdiscuss/2011-May/014150. We can certainly discuss these, but TC39 hasn't adopted them.
We should discuss. Agree we have not adequately discussed, let alone adopted.
On Thu, Jul 14, 2011 at 3:00 PM, Allen Wirfs-Brock <allen at wirfs-brock.com>wrote:
On Jul 14, 2011, at 1:19 PM, Mark S. Miller wrote:
[...]
Absent other costs, that's a fine thing to want. But because of those costs, it was never a goal of the proxies design.
I think several of us have been quite consistent in expressing that we are looking for a mechanism for implementing in pure ES things that currently require host objects or enhanced object semantics of the sort that historically has been defined for built-ins such as Array objects. If that was not one of the goals for Proxies I misunderstood. Perhaps Proxy is not what we need then.
I'm sorry you misunderstood. I recall you were at Dynamic Languages 2010, where you gave a great keynote and Tom presented our paper on proxies < static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/36574.pdf
We have publicized that paper on es-discuss and to the committee and linked to it from the proxies wiki pages. We have hardly kept it a secret. It says:
From section section 4.4 "Temporary Intercession"
Recall from section 3.2 that ECMAScript 5 enables the creation of
tamper-proof objects. A tamper-proof object provide useful guarantees that programmers can rely upon. When designing a proxy API, care should be taken that proxies cannot break these guarantees. For example, if o is an object, then Object.freeze(o) freezes that object. If the programmer knows o is frozen, he can rely on the fact that the number of properties contained in o will no longer change. If o is a proxy, we do not want a proxy to violate such assumptions. [...] if an operation imposes strong consistency requirements on a proxy object (such as becoming immutable), intercession should be disabled for some or all of the operations intercepted by proxies.
And essentially all of section 4.6 "Selective Interception".
Historically there never were any real restriction on what such objects can do.
The DOM and RegExp.prototype.compile demonstrate this. The host object
object restrictions added in ES5 don't apply to built-in objects
Allen, you and I have been over this, both on list and off. This is simply not true. All the restrictions at the end of 8.6.2 are stated as applying only to host (i.e., non-native) objects only because we did not need to state that they apply to native objects. E5.1 specifies the behavior of native objects in enough detail that we can read the spec and verify that they must obey these constraints as well.
Implementations are free to add new built-in objects (see 4.3.28). Without any stated requirements for such objects you have no guarantees.
Quoting that section in full:
*4.3.28
**built-in method *method that is a built-in function NOTE Standard built-in methods are defined in this specification, and an ECMAScript implementation may specify and provide other additional built-in methods.
My interpretation of this is simply that implementations can provide new built-in methods that are still constrained to operate as native functions if they are to be considered native functions. As with the specified built-in native functions, they can have [[Call]] and [[Construct]] methods that are not written in JavaScript. I don't understand what else you are reading into this, unless it's captured by the peek and poke discussion below.
Further more, in writing specification material such as the alternative internal methods, I was never aware of any requirements that had to be meet by those implementations other than whatever was necessary to describe the legacy or new semantics that we were trying to specify. As far as I know, there is no comprehensive normative specification of invariant that must be maintained by all objects.
That is why I was and continue to agree with you that we should seek to arrive at agreement on such a clear normative specification.
For example, the ES5 spec says "Host object may implement these internal
methods in any manner unless otherwise specified" and "The [[DefineOwnProperty]] internal method of a host object must not permit the addtion of a new property...if the [[Extensible]]...has been observed...to be false". So, in that case [[DefineOwnProperty]] can't add a new property but [[Delete]], or [[Get]], or [[DefaultValue]] or any other the internal methods could as nothing specifies that they can't.
I agree that this is a spec bug. We should fix this in the ES5 errata.
No it is not a bug. There was never any consensus on a larger set of invariants or even as far as I can recall a proposal for such. I don't know whether or not we would have or in the future can achieve consensus on that. For now, ES5.1 is what it is and and unless something is clearly internally inconsistent, unintended breakage from an earlier edition, or breaks long establish browser behaviors I don't think you can call it a bug.
Do you really think it makes sense to allow new properties to appear on non-extensible objects? Really?
Perhaps you do. Again, unless and until we get agreement to clean up these (irregularities, inconsistencies, features, whatever you want to call them) in 8.6.2, ES5.1 is, as you say, what it is. I hope we can do better.
Finally, in adding the restrictions to ES5 I don't think there was any serious effort to see whether or not the restrictions were consistent with what host objects actually did. Certainly they are not consistent with the commonly implemented behavior of RegExp.prototype.compile.
Ok, I admit I haven't paid as much attention to RegExp.prototype.compile as I should, because, thankfully, it is deletable on all major browsers. Where can I read about its cross-browser de-facto behavior?
bugzilla.mozilla.org/show_bug.cgi?id=630284 The Microsoft implementation is described in msdn.microsoft.com/en-us/library/ff955265(v=VS.85).aspx but to be sure what it means you need to write some tests.
These was off-list email earlier among various implementors about this issue. this is what Luke said at the time: [...]
Thanks. Most useful. I will look at these and respond once I have.
My sense is that Mark has some specific invariants that he wishes were guaranteed in ES implementations and he wants to make it impossible to violate those invariants using Proxies
y
even though there is no such invariants imposed on built-in objects
wrong, as we've discussed numerous times.
So we disagree WRT whether the specified behavior of features in current specification can be extrapolated as imposing limitation on future unrelated features and specifications.
I agree that this is a good summary of our disagreement, yes.
and the host object invariants are incomplete and hard to make impossible
to violated.
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
We could provide a defined interface mechanism that validates constraints or limits behavior in a way that guarantees the desired invariants. That's what Proxies appear to be trying to do, why not do it for host objects. If you can depend upon host objects actually supporting your invariants why does not matter whether or not Proxy objects also do so.
I think this is key to the whole discussion, as you and I have discussed verbally. The main use case you seem to be thinking in terms of is Mozilla's own use, as a platform provider, of the proxy mechanism as a way to replace your current mechanisms for providing host objects. This is a laudable goal which I fully support. However, for your use as a platform provider, it is legitimate for you to provide yourself dangerous shortcuts that we cannot expose to untrusted code, given that you take responsibility for not abusing these dangerous shortcuts.
For clarity, let's take an extreme example: peek and poke. For those who missed the history, peek and poke < en.wikipedia.org/wiki/PEEK_and_POKE> were primitives in some Basics
for providing direct unchecked access to raw memory. If you, as platform providers, wanted to provide yourself with peek and poke operations in order to gain some efficiency or whatever, there's nothing fundamentally wrong with that. But clearly, no matter what the gains, that shortcut should never be exposed to untrusted code.
This also demonstrates out why the notorious Chapter 16 exemptions need to be tightened. As currently written, a platform could provide peek and poke as native methods without violating the spec. However, so far I am at a loss on how to tighten this up so that it is neither too tight for agreement nor too loose for safety. Perhaps this is the real 4.3.28 issue you were raising above?
If we really want to do this, then we need to really define a complete and self consistent set of invariants that we are willing to apply to all objects (include built-ins and host) now and into the future. [...]
Yes. For a start,
That the statechart at [< es3.1:attribute_states>] represents all the states and transition possible for a property.
8.6.2, with the clarification that it applies to both native and non-native, and with the [[Extensible]] spec bug that you point out fixed.
The normative specification of valid state transitions is actually 8.12.9, but that only applied to objects that use that definition of [[DefineOwnProperty]]. Any extrapolation to other situations hasn't been normatively adopted.
- The "eternal invariants" from < esdiscuss/2011-May/014150>.
We can certainly discuss these, but TC39 hasn't adopted them.
Understood. Those three bullets are my answer to your inquiry about what "complete and self consistent set of invariants" I'd like to see us agree on going forward. I believe they are better than our current ES5.1 baseline. If you agree, perhaps we can make some progress.
Personally, I'm skeptical that we can agree upon such a set of invariants that are compatible with the legacy language and DOM. I'm also an skeptical of the need to do so for Proxies or for built-in and host objects. But, if we want to go for it, then we should proceed and try to define the universal object invariants and see if we can reach consensus on accepting them. If we can't, then we shouldn't be restricting Proxy semantics based upon some assumed invariants.
Until and unless we can all agree on something else, we stick with what we have already agreed to. With our current agreement as a baseline, I am confident we can agree on something better -- like cleaning up the [[Extensible]] spec bug you point out above.
And what we have already agreed to is only what is currently in the ES5.1 spec, as written. As I expressed above, I don't think there is agreement about extrapolations of the text to situations that aren't explicitly called out in the specification.
Sorry, if that seems unharmonious but it seems like the only solid footing we have.
Yup. It seems we no longer have agreement on how to extrapolate to our current standards activities. That is disappointing, and seems to leave us stuck. Let us try to figure out how to go forward from here. I am quite anxious to get back to our harmonious state of progress towards a better JavaScript. This one issue aside, I've very encouraged about how well all of this has been proceeding!
On Jul 14, 2011, at 4:59 PM, Mark S. Miller wrote:
Do you really think it makes sense to allow new properties to appear on non-extensible objects? Really?
Perhaps you do. Again, unless and until we get agreement to clean up these (irregularities, inconsistencies, features, whatever you want to call them) in 8.6.2, ES5.1 is, as you say, what it is. I hope we can do better.
Mark, hope you don't mind me jumping in -- I think I see a misunderstanding. Allen is not saying you should be able to extend non-extensible objects.
He is, I believe, noting how adding one or two ad-hoc restrictions on host objects, while leaving a large (or indefinite) list of other aspects of host objects free of such restrictions, is not good for the spec or its consumers.
So we disagree WRT whether the specified behavior of features in current specification can be extrapolated as imposing limitation on future unrelated features and specifications.
I agree that this is a good summary of our disagreement, yes.
I'm not sure -- it leaves open the question of whether we should do the work to fill all the gaps, so no unwarranted extrapolation is required. I think we may agree on one of (1) no restrictions on a few host object aspects; (2) a great many restrictions, so as to reliably enforce the desired invariants.
This assumes we can specify and agree on the invariants.
I don't think anyone has written down those invariants in a way TC39 could evaluate and agree to put in the spec. It seems premature to conclude that we could not agree on the right set.
There will be some fine line-drawing. We can't just intercede everywhere without making proxies so powerful that you can't cheaply maintain certain security invariants. There's a tension between the "reflect everything and allow intercession in all operations" line advocated by David Ungar in a question to Tom at DLS last fall (about why Proxies do not intercede for === -- "did you chicken out?") and sandboxing the base and meta levels, providing a restricted subset of the language with strictly fewer meta operations.
My hope is that with modules and loaders, we can provide all the power Allen wants at the meta layer, in a way that does not break abstractions, and secure subsets can cheaply remove such capabilities.
But I admit to having mixed feelings about, e.g. Proxy.isProxy. If it's present in browsers without a subset opt-in, the web developers may start to call it on DOM NodeLists to detect whether they are running on a Firefox 7 or later that uses Proxies to implement NodeLists. That would be bad.
On Jul 14, 2011, at 4:59 PM, Mark S. Miller wrote:
On Thu, Jul 14, 2011 at 3:00 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
Implementations are free to add new built-in objects (see 4.3.28). Without any stated requirements for such objects you have no guarantees.
Quoting that section in full:
4.3.28 built-in method method that is a built-in function NOTE Standard built-in methods are defined in this specification, and an ECMAScript implementation may specify and provide other additional built-in methods.
My interpretation of this is simply that implementations can provide new built-in methods that are still constrained to operate as native functions if they are to be considered native functions. As with the specified built-in native functions, they can have [[Call]] and [[Construct]] methods that are not written in JavaScript. I don't understand what else you are reading into this, unless it's captured by the peek and poke discussion below.
sorry I meant to refer to 4.3.7 Built-in objects which using the term Native Object.
Note that the definition of Native Objects was changed in ES5 in a manner that I now think is inconsistent with web reality in that it now says that the semantics of (all) native objects are defined by the ES specification. ES3 did not say that.
On Jul 14, 2011, at 4:59 PM, Mark S. Miller wrote:
On Thu, Jul 14, 2011 at 3:00 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Jul 14, 2011, at 1:19 PM, Mark S. Miller wrote:
Do you really think it makes sense to allow new properties to appear on non-extensible objects? Really?
Perhaps you do. Again, unless and until we get agreement to clean up these (irregularities, inconsistencies, features, whatever you want to call them) in 8.6.2, ES5.1 is, as you say, what it is. I hope we can do better.
Prior to ES5, [[Extensible]] did exist. We added it in a manner that was consistent for native objects. We may have over-stepped in requiring that host objects have such an internal property when in the past they didn't. I don't think there is any fundamental necessity that such an internal property exists for host objects or that it is possible to be able to place a host object into a state where properties cannot be added. I not sure that DOMs violate this, but I'm also not sure that they don't.
If an object (whether native or host) has such a state, then it clearly is a bug if new properties can be added when the object is in such a state. But is it a critical safely bug that must be proactively detected and prevented? I not yet convinced that it is or that the benefit of proactively guaranteeing that such an invariant cannot be violated is worth the cost.
and the host object invariants are incomplete and hard to make impossible to violated.
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
We could provide a defined interface mechanism that validates constraints or limits behavior in a way that guarantees the desired invariants. That's what Proxies appear to be trying to do, why not do it for host objects. If you can depend upon host objects actually supporting your invariants why does not matter whether or not Proxy objects also do so.
I think this is key to the whole discussion, as you and I have discussed verbally. The main use case you seem to be thinking in terms of is Mozilla's own use, as a platform provider, of the proxy mechanism as a way to replace your current mechanisms for providing host objects. This is a laudable goal which I fully support. However, for your use as a platform provider, it is legitimate for you to provide yourself dangerous shortcuts that we cannot expose to untrusted code, given that you take responsibility for not abusing these dangerous shortcuts.
No, that is just one use case. However, in general I think the ability to use intercession to produce interesting object semantics is a valuable feature that has various uses include hosting other languages. However, I am content to leave the maintenance of complex invariants up to the implementors of such object semantics and to tolerate any failure to do so as user bugs alongs as they can't violate memory safety.
For clarity, let's take an extreme example: peek and poke. For those who missed the history, peek and poke en.wikipedia.org/wiki/PEEK_and_POKE were primitives in some Basics for providing direct unchecked access to raw memory. If you, as platform providers, wanted to provide yourself with peek and poke operations in order to gain some efficiency or whatever, there's nothing fundamentally wrong with that. But clearly, no matter what the gains, that shortcut should never be exposed to untrusted code.
We aren't talking about implementation private extensions and we aren't talking about peek and poke which which can be used to violate fundamental memory safety invariants in any program. I don't see the issues we are talking about as involving invariants of such universal significance or which are exploitable in the same broad manner. If there are equivalent severity exploits that you are concerned about I haven't heard what there are yet.
This also demonstrates out why the notorious Chapter 16 exemptions need to be tightened. As currently written, a platform could provide peek and poke as native methods without violating the spec. However, so far I am at a loss on how to tighten this up so that it is neither too tight for agreement nor too loose for safety. Perhaps this is the real 4.3.28 issue you were raising above?
I would be quite open to specifying memory safely invariants. I'm don't think there are many others of equal significance. Chapter 16 needs to be very loose or it would prevent most future extensions to the language.
While proxies are definitely designed to emulate some of the peculiar behavior of host objects, that does not necessarily imply that proxies should be as powerful and free of restrictions as host objects. If any JS script can create a proxy, and a proxy can violate invariants that do hold for built-in Object or Function values, the logical implication is that Javascript code, in general, can no longer rely on those invariants. Code that does can then more easily be exploited.
Consider a non-configurable, non-writable data property. The binding of such a property for regular objects is guaranteed to be immutable. Immutability can be used for caching purposes, but is equally useful to not have to adopt a defensive coding style (as a point in case, take Mark's implementation of the structured clone algorithm in JS. Such defensive coding would be unnecessary when dealing with frozen objects.) In a security context, immutability may be used to determine whether or not to allow some third-party code to access an object.
In short, if proxies could violate the semantics of e.g. frozen objects, then it seems to me that the usefulness of that concept is severely crippled. Where Object.isFrozen would have been a test for guaranteed immutability (of an object's structure), it merely becomes a test that hints at it, but cannot be relied upon.
Cheers,
Tom
2011/7/15 Allen Wirfs-Brock <allen at wirfs-brock.com>
On Jul 15, 2011, at 3:51 AM, Tom Van Cutsem wrote:
Hi Allen,
While proxies are definitely designed to emulate some of the peculiar behavior of host objects, that does not necessarily imply that proxies should be as powerful and free of restrictions as host objects. If any JS script can create a proxy, and a proxy can violate invariants that do hold for built-in Object or Function values, the logical implication is that Javascript code, in general, can no longer rely on those invariants. Code that does can then more easily be exploited.
My view is that intercession is useful in that it supports the implementation of object semantics that differ form the base language. It actually seems to me that the property attributes are part of those base object semantics that might be reasonably modified by intercession. But I guess that is really the base question here. What are the base invariants that must be maintained at all costs and what are secondary invariants that may be useful but not essential.
Consider a non-configurable, non-writable data property. The binding of such a property for regular objects is guaranteed to be immutable. Immutability can be used for caching purposes, but is equally useful to not have to adopt a defensive coding style (as a point in case, take Mark's implementation of the structured clone algorithm in JS. Such defensive coding would be unnecessary when dealing with frozen objects.) In a security context, immutability may be used to determine whether or not to allow some third-party code to access an object.
The immediate issue was about [[Extensible]] not about the broader issue of Object.freeze. I think they are probably separable issues and that part of the disagreement is that they have been conflated.
In short, if proxies could violate the semantics of e.g. frozen objects, then it seems to me that the usefulness of that concept is severely crippled. Where Object.isFrozen would have been a test for guaranteed immutability (of an object's structure), it merely becomes a test that hints at it, but cannot be relied upon.
But can't a Proxy based object do all sorts of nasty back channel stuff even while it maintains the apparent object freeze invariants?
More generally, it seems like what you really need to defend against is objects that are implemented via untrusted Proxy handlers. I have no problem with "your" defensive subsystem rejecting my non-extensible proxy based array because you don't trust my proxy handler. But to tell me that I can't even create such an object for my own purposes because you are are afraid I might pass it to you seems to be putting your use case ahead of mine.
On Jul 15, 2011, at 11:16 AM, Allen Wirfs-Brock wrote:
On Jul 15, 2011, at 3:51 AM, Tom Van Cutsem wrote:
Hi Allen,
While proxies are definitely designed to emulate some of the peculiar behavior of host objects, that does not necessarily imply that proxies should be as powerful and free of restrictions as host objects. If any JS script can create a proxy, and a proxy can violate invariants that do hold for built-in Object or Function values, the logical implication is that Javascript code, in general, can no longer rely on those invariants. Code that does can then more easily be exploited.
My view is that intercession is useful in that it supports the implementation of object semantics that differ form the base language. It actually seems to me that the property attributes are part of those base object semantics that might be reasonably modified by intercession. But I guess that is really the base question here. What are the base invariants that must be maintained at all costs and what are secondary invariants that may be useful but not essential.
Consider a non-configurable, non-writable data property. The binding of such a property for regular objects is guaranteed to be immutable. Immutability can be used for caching purposes, but is equally useful to not have to adopt a defensive coding style (as a point in case, take Mark's implementation of the structured clone algorithm in JS. Such defensive coding would be unnecessary when dealing with frozen objects.) In a security context, immutability may be used to determine whether or not to allow some third-party code to access an object.
The immediate issue was about [[Extensible]] not about the broader issue of Object.freeze. I think they are probably separable issues and that part of the disagreement is that they have been conflated.
In short, if proxies could violate the semantics of e.g. frozen objects, then it seems to me that the usefulness of that concept is severely crippled. Where Object.isFrozen would have been a test for guaranteed immutability (of an object's structure), it merely becomes a test that hints at it, but cannot be relied upon.
But can't a Proxy based object do all sorts of nasty back channel stuff even while it maintains the apparent object freeze invariants?
Duh, of course a frozen object really isn't a Proxy any more as currently defined. But is has also lost all useful intercession derived semantics.
BTW, if strawman:fixed_properties was extended to support Object.preventExtensions we might not have anything to argue about except perhaps performance issues.
On Jul 15, 2011, at 12:38 PM, Allen Wirfs-Brock wrote:
On Jul 15, 2011, at 11:16 AM, Allen Wirfs-Brock wrote:
But can't a Proxy based object do all sorts of nasty back channel stuff even while it maintains the apparent object freeze invariants?
Duh, of course a frozen object really isn't a Proxy any more as currently defined. But is has also lost all useful intercession derived semantics.
This may be ok, so long as we can distinguish preventExtensions from seal from freeze, and determine the class or constructor used for the newborn object that becomes the proxy. Yes, I'm thinking faithful Array emulation.
BTW, if strawman:fixed_properties was extended to support Object.preventExtensions we might not have anything to argue about except perhaps performance issues.
Indeed, that helps quite a bit. Glad to hear it.
But there may be more power needed in the fix trap, per the above Array-proxy test.
On Jul 15, 2011, at 12:47 PM, Brendan Eich wrote:
On Jul 15, 2011, at 12:38 PM, Allen Wirfs-Brock wrote:
On Jul 15, 2011, at 11:16 AM, Allen Wirfs-Brock wrote:
But can't a Proxy based object do all sorts of nasty back channel stuff even while it maintains the apparent object freeze invariants?
Duh, of course a frozen object really isn't a Proxy any more as currently defined. But is has also lost all useful intercession derived semantics.
This may be ok, so long as we can distinguish preventExtensions from seal from freeze, and determine the class or constructor used for the newborn object that becomes the proxy. Yes, I'm thinking faithful Array emulation.
BTW, if strawman:fixed_properties was extended to support Object.preventExtensions we might not have anything to argue about except perhaps performance issues.
Indeed, that helps quite a bit. Glad to hear it.
But there may be more power needed in the fix trap, per the above Array-proxy test.
Perhaps the fix trap should return the constructor to use or perhaps even the substitute object.
2011/7/15 Brendan Eich <brendan at mozilla.com>
On Jul 15, 2011, at 12:38 PM, Allen Wirfs-Brock wrote:
BTW, if strawman:fixed_properties was extended to support Object.preventExtensions we might not have anything to argue about except perhaps performance issues.
Indeed, that helps quite a bit. Glad to hear it.
But there may be more power needed in the fix trap, per the above Array-proxy test.
Glad to hear that too. I'll try to extend the FixedHandler with support for non-extensibility. David may beat me to it, though ;-) I'll be offline for the next couple of weeks, so this may take a while, but it's noted and I'll follow-up on suggestions later.
Le 14/07/2011 21:19, Allen Wirfs-Brock a écrit :
On Jul 14, 2011, at 10:47 AM, Brendan Eich wrote:
On Jul 14, 2011, at 10:38 AM, David Bruant wrote:
In all the cases I can think of, where we want precisely the behavior of an array, why not just use an array? Of course an array should be used. I think that the current question is rather "can proxies as currently defined can fully emulate arrays?". Being able to fully emulate arrays sounds like an "easy" use case. I agree, and Allen is on the record here too.
Fixing an array-proxy to an array could be enough for real-world use-cases, so this may be not a burning practical issue. Indeed it verges on Principle. But it's a good one, and I think we should have a goal (rather than a principle) for cleanly proxying arrays, including non-extensible ones.
If this goal is met by saying fixing an array proxy always results in the instance becoming a fixed Array, that could be good enough.
I do not think we should say "fix to a non-extensbile object with a length accessor" while we at the same time carefully spec Array length as a data property, precisely to ensure inter-operation. That seems to be telling JS implementors one thing, and (possibly an overlapping group) proxy users another.
/be I think Mark and I are looking at Proxies from very different perspectives. I want Proxies to to useful to create a perfect implementation of anything that be specified as a ES built-in object or that might be implemented whether now or in the future as a host object defined in some other programming language. I hold up ES array objects, string objects, argument objects, functions objects, and various DOM implementations simply as exemplars of such objects that somebody might want to implement using Proxies.
Historically there never were any real restriction on what such objects can do.
However, all 7 invariants present in the ES5.1 spec all address things that could not be noticed or changed in JavaScript itself, there was no way to:
- change configurability
- measure configurability (only non-deletability)
- measure or change [[Extensibility]] (because such a thing didn't exist) Current invariants seems to be following the general idea of "if I measure something (non-configurability or non-ensensibility), then, I'll have some guarantees on what I can/can't do afterward that will hold for the rest of the program". Since we had no way to measure any before ES5, I would say that what used to happen before ES5.1 is irrelevant. ES5 implementations not respecting these invariants are more problematic. So are ES5 constructs that would not respect these invariants too.
Le 15/07/2011 23:04, Tom Van Cutsem a écrit :
2011/7/15 Brendan Eich <brendan at mozilla.com <mailto:brendan at mozilla.com>>
On Jul 15, 2011, at 12:38 PM, Allen Wirfs-Brock wrote:BTW, if http://wiki.ecmascript.org/doku.php?id=strawman:fixed_properties was extended to support Object.preventExtensions we might not have anything to argue about except perhaps performance issues.Indeed, that helps quite a bit. Glad to hear it. But there may be more power needed in the fix trap, per the above Array-proxy test.Glad to hear that too. I'll try to extend the FixedHandler with support for non-extensibility. David may beat me to it, though ;-) :-p I'm only catching up now on this thread. I have indeed started some work to try to implement an invariant-enforcing implementation of preventExtension rather than fix+become. Things that happened can be followed in the "[Harmony Proxies] LazyReadCopy experiment and invariant checking for [[Extensible]]=false" thread. Long story short, I have started an implementation. It's here: DavidBruant/HarmonyProxyLab/tree/master/NonExtensibleProxies But I have not tested it or run it, because after re-reading the ES5 invariants, I realized I was making too much work. So now, I'm waiting for an answer on whether or not property enumeration traps should maintain an invariant. Also, traps also need to know non-configurable properties, so, the approach I took (independent non-extensible proxy maker) does not seem to fit. One implementation should take care of all invariants at once. It sounds easier this way. However, the code used to override Object.preventExtensions and Object.isExtensible can be kept as is (need to be run and tested to make sure).
I'll be offline for the next couple of weeks, so this may take a while, but it's noted and I'll follow-up on suggestions later.
Time may be short for me for the next week too. If I find some time, I'll redo an implementation based on yours and will give a follow-up.
Le 15/07/2011 00:16, Brendan Eich a écrit :
On Jul 14, 2011, at 3:00 PM, Allen Wirfs-Brock wrote:
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
We could provide a defined interface mechanism that validates constraints or limits behavior in a way that guarantees the desired invariants. That's what Proxies appear to be trying to do, why not do it for host objects. If you can depend upon host objects actually supporting your invariants why does not matter whether or not Proxy objects also do so.
For the record, Mozilla wants its "host objects" to be only as powerful as Proxies. Current web compatibility constraints don't allow this but we want to evolve the standards, de facto (over time) and de jure, toward that goal.
Do you have examples?
Some Proxy extensions may be needed, too, so it's not all up to web compatibility to take the hit.
Do you have such Proxy extensions in mind already?
On Jul 17, 2011, at 3:14 PM, David Bruant wrote:
Le 15/07/2011 23:04, Tom Van Cutsem a écrit :
2011/7/15 Brendan Eich <brendan at mozilla.com> On Jul 15, 2011, at 12:38 PM, Allen Wirfs-Brock wrote:
BTW, if strawman:fixed_properties was extended to support Object.preventExtensions we might not have anything to argue about except perhaps performance issues.
Indeed, that helps quite a bit. Glad to hear it.
But there may be more power needed in the fix trap, per the above Array-proxy test.
Glad to hear that too. I'll try to extend the FixedHandler with support for non-extensibility. David may beat me to it, though ;-) :-p I'm only catching up now on this thread. I have indeed started some work to try to implement an invariant-enforcing implementation of preventExtension rather than fix+become.
Just off the top of my head, sorry if I'm misunderstanding: fix based on become is very attractive to VM implementors, because it avoids extra for-life-of-fixed-object costs in the proxy or native code paths that aren't already there with Proxies as proposed and with ES5.
"becomes" lets the brain-swapped objects do what they do best. There's no hybridization as if by composition using vtbls, if statements, or whatever, that affects proxy codepaths. It's true one could "become" a proxy with the extra code paths, but that's duplicative at some level (not necessarily code inlining, but layering the extra preventExtensions enforcement somehow).
If the preventExtensions enforcement is the same as the pre-fix overhead for fixed properties, perhaps this is less of an issue.
Sticking with becomes, but (a) distinguishing preventExtensions from seal from freeze and (b) allowing the [[Class]] or ES.next equivalent to be controlled would allow, e.g., Proxy-emulated Arrays that fix to become real (pE'ed, sealed, or frozen) Arrays.
On Jul 17, 2011, at 3:18 PM, David Bruant wrote:
Le 15/07/2011 00:16, Brendan Eich a écrit :
On Jul 14, 2011, at 3:00 PM, Allen Wirfs-Brock wrote:
Host objects are part of the platform. A platform provider is free to violate any part of the spec they like, and there's nothing we can do about it other than to add tests to std test suites to make the violations obvious to the community.
We could provide a defined interface mechanism that validates constraints or limits behavior in a way that guarantees the desired invariants. That's what Proxies appear to be trying to do, why not do it for host objects. If you can depend upon host objects actually supporting your invariants why does not matter whether or not Proxy objects also do so.
For the record, Mozilla wants its "host objects" to be only as powerful as Proxies. Current web compatibility constraints don't allow this but we want to evolve the standards, de facto (over time) and de jure, toward that goal. Do you have examples?
David Flanagan is in the thick of dom.js work (andreasgal/dom.js), so he should comment. IIRC at least configuring [[Class]] name as exposed by Object.prototype.toString, and of course fixed properties on pre-fixed Proxies, have come up in the course of self-hosting the DOM.
Some Proxy extensions may be needed, too, so it's not all up to web compatibility to take the hit. Do you have such Proxy extensions in mind already?
See my previous reply, about fixed properties, the fix trap, controlling the [[Class]] of the object the proxy "becomes" upon fixing, and preserving the (preventExtensions, seal, freeze) distinction when fixing.
On Jul 17, 2011, at 6:01 PM, Brendan Eich wrote:
See my previous reply, about fixed properties, the fix trap, controlling the [[Class]] of the object the proxy "becomes" upon fixing, and preserving the (preventExtensions, seal, freeze) distinction when fixing.
Here's a possibly dumb question: why not have fix return not a pdmap for the new Object instance to be fixed at those properties, rather an object (of any class) to be fixed by the exact cause of the fix trap: preventExtensions, seal, or freeze?
Le 18/07/2011 02:58, Brendan Eich a écrit :
On Jul 17, 2011, at 3:14 PM, David Bruant wrote:
I'm only catching up now on this thread. I have indeed started some work to try to implement an invariant-enforcing implementation of preventExtension rather than fix+become.
Just off the top of my head, sorry if I'm misunderstanding: fix based on become is very attractive to VM implementors, because it avoids extra for-life-of-fixed-object costs in the proxy or native code paths that aren't already there with Proxies as proposed and with ES5.
"becomes" lets the brain-swapped objects do what they do best. There's no hybridization as if by composition using vtbls, if statements, or whatever, that affects proxy codepaths. It's true one could "become" a proxy with the extra code paths, but that's duplicative at some level (not necessarily code inlining, but layering the extra preventExtensions enforcement somehow).
If the preventExtensions enforcement is the same as the pre-fix overhead for fixed properties, perhaps this is less of an issue.
I was in doubt for some time and I realized that the fixed properties required very few checks and these checks turn out to be the same performed by some code defined in a way or another in one of the ES5.1 8.12 algorithms. By enforcing ES5 invariants, we actually are trying to enforce things that are enforced for normal objects already. If you haven't read Tom's FixedHandler implementation [1], I encourage you to, because the minimalism of it is quite interesting. Especially, in the get{Own}PropertyDescriptor and defineProperty traps, Object.defineProperty is called and as commented, the purpose is to (re)use the argument-checking code of this method to let it throw if needed. It's extremely generic and future-proof with regard to new sorts of property descriptors. We'll see what the implementation of invariant checking non-extensible proxies looks like.
Sticking with becomes, but (a) distinguishing preventExtensions from seal from freeze and (b) allowing the [[Class]] or ES.next equivalent to be controlled would allow, e.g., Proxy-emulated Arrays that fix to become real (pE'ed, sealed, or frozen) Arrays.
Distinguishing preventExtensions from seal from freeze will still be possible with an invariant-enforcing proxy if that's the chosen solution.
David
[1] code.google.com/p/es-lab/source/browse/trunk/src/proxies/FixedHandler.js
Le 18/07/2011 03:03, Brendan Eich a écrit :
On Jul 17, 2011, at 6:01 PM, Brendan Eich wrote:
See my previous reply, about fixed properties, the fix trap, controlling the [[Class]] of the object the proxy "becomes" upon fixing, and preserving the (preventExtensions, seal, freeze) distinction when fixing. Here's a possibly dumb question: why not have fix return not a pdmap for the new Object instance to be fixed at those properties, rather an object (of any class) to be fixed by the exact cause of the fix trap: preventExtensions, seal, or freeze?
Interesting. What would happen if the returned object is a proxy?
Regardless, my understanding of the fix+become design was to enforce the non-extensibility invariants (and if I'm missing something, please tell me so). I have to admit that it really feels weird to me that because I call Object.{preventExtensions|seal|freeze}, my handler should stop its existence and my proxy should "become" another object (even a native array or a DOM Node if we want to allow that). It sounds very arbitrary to me. Why does preventing extension should throw my loggers away? Or stop the forwarding of all operations to the target object if I defined a forwarder? Actually, I've raised the issue in another thread that a forwarder proxy cannot properly forward Object.{preventExtensions|seal|freeze} to the target object because within the fix trap, there is no way to distinguish between the 3.
On Jul 18, 2011, at 2:35 AM, David Bruant wrote:
Le 18/07/2011 03:03, Brendan Eich a écrit :
On Jul 17, 2011, at 6:01 PM, Brendan Eich wrote:
See my previous reply, about fixed properties, the fix trap, controlling the [[Class]] of the object the proxy "becomes" upon fixing, and preserving the (preventExtensions, seal, freeze) distinction when fixing. Here's a possibly dumb question: why not have fix return not a pdmap for the new Object instance to be fixed at those properties, rather an object (of any class) to be fixed by the exact cause of the fix trap: preventExtensions, seal, or freeze? Interesting. What would happen if the returned object is a proxy?
Good question. First impulse is to make that an error, but if proxies can be fixed, then the possibility of a fixed and inextensible proxy sufficing as return value of this hypothetical fix trap is plausible.
We would need continued, VM-level enforcement that such a fixed proxy cannot be extended, and of course that all of its properties are fixed.
Regardless, my understanding of the fix+become design was to enforce the non-extensibility invariants (and if I'm missing something, please tell me so). I have to admit that it really feels weird to me that because I call Object.{preventExtensions|seal|freeze}, my handler should stop its existence and my proxy should "become" another object (even a native array or a DOM Node if we want to allow that). It sounds very arbitrary to me.
The original proxy design had no support for fixed properties, so no way to let the handler intercede after fix without leaving too much bug/attack surface.
If we extend proxies to support fixed properties and ![[Extensible]], then we may be able to relax the design. But nothing here is truly arbitrary. It's a divide-and-conquer progression, I think.
Also, to repeat: implementation of becomes with a newborn is really quite slick and relatively easy to support, compared to more invasive fixed/inextensible enforcement.
Why does preventing extension should throw my loggers away? Or stop the forwarding of all operations to the target object if I defined a forwarder?
Because there was no way in sight to enforce the critical invariants. Possibly there is, now.
Actually, I've raised the issue in another thread that a forwarder proxy cannot properly forward Object.{preventExtensions|seal|freeze} to the target object because within the fix trap, there is no way to distinguish between the 3.
Yes indeed.
On Jul 18, 2011, at 2:06 AM, David Bruant wrote:
By enforcing ES5 invariants, we actually are trying to enforce things that are enforced for normal objects already. If you haven't read Tom's FixedHandler implementation [1], I encourage you to, because the minimalism of it is quite interesting.
Read it, very slick. Has it changed recently?
Especially, in the get{Own}PropertyDescriptor and defineProperty traps, Object.defineProperty is called and as commented, the purpose is to (re)use the argument-checking code of this method to let it throw if needed. It's extremely generic and future-proof with regard to new sorts of property descriptors.
That is a good sign, and anything else would be a "bad sign" ;-).
We'll see what the implementation of invariant checking non-extensible proxies looks like.
This is the crucial part.
Sticking with becomes, but (a) distinguishing preventExtensions from seal from freeze and (b) allowing the [[Class]] or ES.next equivalent to be controlled would allow, e.g., Proxy-emulated Arrays that fix to become real (pE'ed, sealed, or frozen) Arrays. Distinguishing preventExtensions from seal from freeze will still be possible with an invariant-enforcing proxy if that's the chosen solution.
Is there a proposal?
Hate to start another cycle of this, but if (trapping) proxies can't be set to [[Extensible]]: false, then they can't be used to fully emulate built-ins such as Array. Why is there this restriction?