On defining non-standard exotic objects
I generally support this view, although it may be too restrictive to completely disallow non-standard exotics.
Instead, I would be happy if we could just state that non-standard exotics are expected to uphold "the invariants of the ES6 object model", which will presumably be codified in Section 8.1.6.3.
Le 31/12/2012 13:43, Tom Van Cutsem a écrit :
Hi David,
I generally support this view, although it may be too restrictive to completely disallow non-standard exotics.
Instead, I would be happy if we could just state that non-standard exotics are expected to uphold "the invariants of the ES6 object model", which will presumably be codified in Section 8.1.6.3.
I disagree with this view for reasons Marc Stiegler expresses really well in his Lazy Programmer's Guide to Secure Computing talk [1].
The current situation is that ES folks would like to impose some restrictions on how the language can be extended, especially when it comes to objects. To the possible extent (and it looks possible), the limits would be the one set by what's possible by proxies. Now, there are 2 choices:
- "do whatever you want within these boundaries defined in prose"
- "proxies are the most powerful extension mechanism at your disposal"
The former requires people to read the spec carefully, become very intimate with it. The recent issue you have solved [2] shows a very subtle problem. I don't think non-standard exotic object spec writers should be expected to understand all these subtleties. And I don't think every subtlety can be documented in the spec. At every non-standard exotic object spec change, people will have to re-review if all properties are properly enforced. The latter solution is the ocap-style lazy one (in Marc Stiegler's sense). All properties that have been carefully weighted and crafted on es-discuss will apply to non-ECMAScript standard objects. Spec writer for these objects won't need to know the details. They just have to fit in the box they are provided and subtleties are taken care of by the box, by design of how the box is being designed.
One point I could understand is that maybe script proxies will not necessarily make a conveninent box for spec writers. If this is really an issue, ECMAScript could define an intermediate proxy representation that would be used to spec proxies and by other spec writers. But I think non-standard exotic objects spec writers could work easily with: "object X is a proxy which target is a new Y object and the handler is a frozen object which trap methods are [enumerate traps and their behaviors, specifically mention absent trap and that's on purpose]"
David
Ps: "non-standard exotic objects" is way too long to type. I'll be aliasing that to "host objects" from now on.
[1] www.youtube.com/watch?v=eL5o4PFuxTY [2] tvcutsem/harmony-reflect#11
On Wed, Jan 9, 2013 at 9:07 AM, David Bruant <bruant.d at gmail.com> wrote:
The current situation is that ES folks would like to impose some restrictions on how the language can be extended, especially when it comes to objects. To the possible extent (and it looks possible), the limits would be the one set by what's possible by proxies. Now, there are 2 choices:
- "do whatever you want within these boundaries defined in prose"
- "proxies are the most powerful extension mechanism at your disposal"
The former requires people to read the spec carefully, become very intimate with it.
...
They just have to fit in the box they are provided and subtleties are taken
care of by the box, by design of how the box is being designed.
One point I could understand is that maybe script proxies will not necessarily make a conveninent box for spec writers. If this is really an issue, ECMAScript could define an intermediate proxy representation that would be used to spec proxies and by other spec writers.
The crux of the matter is that the ES5 spec doesn't really allow for aspect oriented use of the internal methods. The granularity provided is pretty much at the method level: if you want to reuse the core functionality of say [[DefineOwnProperty]] then you're basically committing to reimplementing the whole thing, or at best pre- or post- processing the input/output of the standard ones. Proxies allow for deferring to the spec implementation and letting it do the heavy lifting of ensuring the internal consistency that a finely polished object protocol provides, and then stepping in to make the (usually small) adjustments needed for the exotic functionality. Proxies are only desirable as a model for implementers because of a lack of other options provided by the spec. They are actually pretty poorly suited in many ways because they're intended use is for untrusted code.
The last couple revisions of the ES6 spec directly address this problem in a much better way for implementers. The core functionality of the internal methods is being split out into separate abstract operations that can be composed with the additional exotic functionality an implementer wishes to add. The flexibility that Proxies provide can be attained without layering on the added complexity that describing something in terms of Proxies requires.
Specifically, I'm referring to things like OrdinaryGetOwnProperty, OrdinaryDefineOwnProperty, ValidateAndApplyPropertyDescriptor, OrdinaryConstruct, OrdinaryCreateFromConstructor, OrdinaryHasInstance, .etc, as well as indexed delegated objects which is a reusable solution for the most common form of exotic object. Along with these methods are the various hooks that they expose to implementers so that, for some things, it's not even required to override an internal method at all (@@create, @@hasInstance being examples).
I'm uncomfortable with the blurring of the boundary between specification and implementation that I see in this thread.
Proxy is the only "standard" extension mechanism provided by ES6 for defining non-standard exotic objects. If non-proxy based non-standard exotic objects exist within an implementation then they must be defined using some other extension mechanism that is inherently implementation dependent. It is presumably also up to the implementation (or the host environment) to determine who has access to any such extension mechanism. For example, such a mechanism might only be available to components that are statically linked with the implementation engine.
Regardless of the interfacing mechanism used to define non-standard exotic objects, all exotic object are required by the standard to conform to the essential invariants that will be defined in section 8.1.6.1 (this section number will ultimately change). An implementation specific extension mechanism might actively enforce those invariants, similarly to what the Proxy mechanism is specified to do. Or, it might place that burden upon the component that is using the extension mechanism in which case any component that defines objects that violate the 8.1.6.1 invariants would have to be considered a buggy component just as would a component that violated any other interfacing rule of extension mechanism.
David seems to be primarily concerned about people who are writing specs. for non-standard exotic objects (eg, W3C/WebIDL-based spec. writers) rather than implementors of such objects. In that case, it is probably reasonable for such a spec. writer to assume that the objects must be implementable using the Proxy mechanism. After all, that is the only extension mechanism that is guaranteed to be available in a standards compliant ES6 implementation. That still doesn't mean that such a spec. writer doesn't need to understand the ES object invariants as they shouldn't be writing any specification requirments that violates those invariants. However, it does mean that they should be able to test their specification by doing a Proxy-based prototype implementation. I can even imagine that such a prototype implementation could be made a mandatory part of the spec. development process.
In the end, specifications don't have any enforcement power and perhaps not even all that much "moral authority". If an implementation really needs to do something that is forbidden by a spec. it will do it anyway. Browser implementations and HTML5 certainly takes this perspective WRT Ecma-262 and re-specifies things such that don't match current browser requirements.
I don't see any need for an intermediate proxy representation or for attempting to limit non-proxy based extension mechanisms. However, if Proxy is not sufficiently powerful to support everything that needs to be done in the real world (and in particular by browsers) then we probably should be looking at how to fill those deficiencies.
On Jan 9, 2013, at 7:24 AM, Brandon Benvie wrote:
On Wed, Jan 9, 2013 at 9:07 AM, David Bruant <bruant.d at gmail.com> wrote: The current situation is that ES folks would like to impose some restrictions on how the language can be extended, especially when it comes to objects. To the possible extent (and it looks possible), the limits would be the one set by what's possible by proxies. Now, there are 2 choices:
- "do whatever you want within these boundaries defined in prose"
- "proxies are the most powerful extension mechanism at your disposal"
The former requires people to read the spec carefully, become very intimate with it. ... They just have to fit in the box they are provided and subtleties are taken care of by the box, by design of how the box is being designed.
One point I could understand is that maybe script proxies will not necessarily make a conveninent box for spec writers. If this is really an issue, ECMAScript could define an intermediate proxy representation that would be used to spec proxies and by other spec writers.
The crux of the matter is that the ES5 spec doesn't really allow for aspect oriented use of the internal methods. The granularity provided is pretty much at the method level: if you want to reuse the core functionality of say [[DefineOwnProperty]] then you're basically committing to reimplementing the whole thing, or at best pre- or post- processing the input/output of the standard ones. Proxies allow for deferring to the spec implementation and letting it do the heavy lifting of ensuring the internal consistency that a finely polished object protocol provides, and then stepping in to make the (usually small) adjustments needed for the exotic functionality. Proxies are only desirable as a model for implementers because of a lack of other options provided by the spec. They are actually pretty poorly suited in many ways because they're intended use is for untrusted code.
The last couple revisions of the ES6 spec directly address this problem in a much better way for implementers. The core functionality of the internal methods is being split out into separate abstract operations that can be composed with the additional exotic functionality an implementer wishes to add. The flexibility that Proxies provide can be attained without layering on the added complexity that describing something in terms of Proxies requires.
Specifically, I'm referring to things like OrdinaryGetOwnProperty, OrdinaryDefineOwnProperty, ValidateAndApplyPropertyDescriptor, OrdinaryConstruct, OrdinaryCreateFromConstructor, OrdinaryHasInstance, .etc, as well as indexed delegated objects which is a reusable solution for the most common form of exotic object. Along with these methods are the various hooks that they expose to implementers so that, for some things, it's not even required to override an internal method at all (@@create, @@hasInstance being examples).
Except that the spec. is not describing a required implementation factoring, just required results. There is no particular reason that an implementation must have the same internal factoring as the spec. or that an implementation exposes via an implementation-specific extension mechanism the equivalent of the abstract operations used within the specification. So while, the spec. refactoring hopefully makes life easier for future spec. writers (including me) and for readers of the spec. I don't think it helps implementors to use object aspects in the way you seem to be envisioning.
On the other hand, the @@hooks are intentionally design to provide extension mechanisms that operate above the level of the MOP (internal methods) and its object invariants. As much as possible, I want to use that style of extension hook and avoid extending or complicating the MOP.
Le 09/01/2013 20:30, Allen Wirfs-Brock a écrit :
David seems to be primarily concerned about people who are writing specs. for non-standard exotic objects (eg, W3C/WebIDL-based spec. writers) rather than implementors of such objects.
I am indeed. When there is a spec, implementors "only" have to read it (and ask questions/submit test cases to whoever is writing the spec in case of doubts)
In that case, it is probably reasonable for such a spec. writer to assume that the objects must be implementable using the Proxy mechanism. After all, that is the only extension mechanism that is guaranteed to be available in a standards compliant ES6 implementation.
It's also the most powerful available and by design, if I understand corretly, the most powerful that will ever be introduced to the ECMAScript language (for objects I mean). We had data properties in ES3, we have getter/setters in ES5, we're about to get proxies in ES6 and I think that's where the trains stops for objects (maybe something will be introduced for document.all falsiness... maybe not? but that would be the actual last step)
That still doesn't mean that such a spec. writer doesn't need to understand the ES object invariants as they shouldn't be writing any specification requirments that violates those invariants.
What I'm trying to get at is that these spec writers don't need to worry about the invariants if they define proxies. The invariants will take care of themselves. If spec writers define proxies, by design, they won't be able to specify requirement that violate these invariants.
However, it does mean that they should be able to test their specification by doing a Proxy-based prototype implementation. I can even imagine that such a prototype implementation could be made a mandatory part of the spec. development process.
Agreed.
In the end, specifications don't have any enforcement power and perhaps not even all that much "moral authority".
The text no, but the coordination between different specs seems to indicate that there is a form of "moral authority" in what TC39 says. If in ES6 is written that any magic behavior has to fit in a proxy box, I think it will be read and understood.
If an implementation really needs to do something that is forbidden by a spec. it will do it anyway. Browser implementations and HTML5 certainly takes this perspective WRT Ecma-262 and re-specifies things such that don't match current browser requirements.
I'm more worried of the case where spec writers do want to conform to the spec without necessarily having to understand every subtelty of the invariants. If they spec something as proxies, they can spec, prototype in code and see if it fits their intention or if they had missed something. This is not true for free-form objects. As you say, writing "you have to use proxies" isn't that authoritative anyway, so it can just be written and those who feel expert enough can go on freeride.
I don't see any need for an intermediate proxy representation or for attempting to limit non-proxy based extension mechanisms. However, if Proxy is not sufficiently powerful to support everything that needs to be done in the real world (and in particular by browsers) then we probably should be looking at how to fill those deficiencies.
Agreed.
On Jan 9, 2013, at 1:56 PM, David Bruant wrote:
Le 09/01/2013 20:30, Allen Wirfs-Brock a écrit :
...
I don't see any need for an intermediate proxy representation or for attempting to limit non-proxy based extension mechanisms. However, if Proxy is not sufficiently powerful to support everything that needs to be done in the real world (and in particular by browsers) then we probably should be looking at how to fill those deficiencies. Agreed.
I guess I should have also said, that Proxy should be viewed as the last resort solution and should seldom be needed. Method dispatch based extension hooks like the @@create, @@hasInstance, and @@ToPrimitive hooks in the current ES6 draft and the hooks described in the "Object Model Reformation" [ 1] strawman operate at a higher meta level and are probably generally preferable to proxy based solutions. It is impossible for them to violate the object invariants. If we still have self-hosting deficiencies we probably should first look for that style of solution before extending Proxy.
Allen
This was basically what I was getting out, even if it was wrong in the idea of using those specific internal methods rather than the higher level methods you describe. Proxies are better than the completely magical implementations that were used in the past, in that they will end up, by default, closer to internally consistent. But they are still capable of being completely inconsistent, and it doesn't take very much complexity in a handler's design to end up with subtle inconsistencies. Rather, as soon as you stop auto-forwarding anything you have to put a great amount of care into ensuring they do remain internally consistent. A higher level hook that exposes the ability to tweak the thing you want to tweak, while ensuring it all remains consistent automatically, is more desirable.
2013/1/9 David Bruant <bruant.d at gmail.com>
Le 09/01/2013 20:30, Allen Wirfs-Brock a écrit :
That still doesn't mean that such a spec. writer doesn't need to understand the ES object invariants as they shouldn't be writing any specification requirments that violates those invariants.
What I'm trying to get at is that these spec writers don't need to worry about the invariants if they define proxies. The invariants will take care of themselves. If spec writers define proxies, by design, they won't be able to specify requirement that violate these invariants.
Well, it's not that simple. Unfortunately direct proxies only enforce the invariants using runtime checks, so it's easy to specify something statically as a Proxy that will still fail at runtime. Of course testing helps here, but my point is that the Proxy API by itself does not prevent broken invariants. One still needs a good understanding of the invariants to define correct exotic objects.
Le 15/01/2013 20:35, Tom Van Cutsem a écrit :
2013/1/9 David Bruant <bruant.d at gmail.com <mailto:bruant.d at gmail.com>>
Le 09/01/2013 20:30, Allen Wirfs-Brock a écrit : That still doesn't mean that such a spec. writer doesn't need to understand the ES object invariants as they shouldn't be writing any specification requirments that violates those invariants. What I'm trying to get at is that these spec writers don't need to worry about the invariants if they define proxies. The invariants will take care of themselves. If spec writers define proxies, by design, they won't be able to specify requirement that violate these invariants.Well, it's not that simple. Unfortunately direct proxies only enforce the invariants using runtime checks, so it's easy to specify something statically as a Proxy that will still fail at runtime. Of course testing helps here, but my point is that the Proxy API by itself does not prevent broken invariants. One still needs a good understanding of the invariants to define correct exotic objects.
I agree the static/runtime divide is not obvious, but as you say it's possible to test and I would add it's very cheap to test. Write the core of your function as JavaScript code, test against your favorite ES6 implementation and if you misunderstood a part of the proxy API, you implementation and tests will tell you soon enough.
Describing in prose the invariant has its educational purpose, but I feel there is no loss and it's not that big of a burden in asking other specs to describe their magic in terms of the proxy API.
It's been suggested a couple times on the list that the hope for non-standard exotic objects (ES6 terminology for ES5 host objects) is to make them at most as powerful as proxies. The current definition of exotic object says: "Exotic objects may implement internal methods in any manner unless specified otherwise; for example, [example]. However, if any specified manipulation of an exotic object's internal properties is not supported by an implementation, that manipulation must throw a TypeError exception when attempted."
What about just removing this paragraph and state that any non-standard exotic object must be an standard object (which include proxies)? As a consequence, proxies would be the most powerful extension mechanism allowed by the spec and a direct consequence would be that non-standard exotic objects would just be at most as powerful as proxies and would be subject to all invariant checks, etc.
Instead of having to freely redefine all some or all internal methods (and potentially violate invariants), non-standard object specs would just have to define the target and necessary traps.