Private Slots
On Mon, Jan 14, 2013 at 10:57 AM, Kevin Smith <khs4473 at gmail.com> wrote:
When a property is added to an object using a private symbol, it creates a new kind of slot: a private slot. It is clear from the design of the Proxy API that this represents a fundamentally new extension to the ES object model.
I (as one of the people who has advocated for symbols) disagree entirely. The JS object model extension required for symbols is small -- JS objects now map from either strings or symbols to values (plus prototype inheritance, getters, setters, configurability, writeability, and the other aspects of the JS object model the quote below glosses over).
The reason that the Proxy API is complex from the perspective of private symbols is that combining reflection with encapsulation and object-capability based security principles is a tricky and less-well-explored domain. Fortunately, Tom as well as others have done an excellent job with this design, and I'm confident in it. However, that complexity is not the same as complexity in the fundamental object model.
On Jan 14, 2013, at 8:04 AM, Sam Tobin-Hochstadt wrote:
On Mon, Jan 14, 2013 at 10:57 AM, Kevin Smith <khs4473 at gmail.com> wrote:
When a property is added to an object using a private symbol, it creates a new kind of slot: a private slot. It is clear from the design of the Proxy API that this represents a fundamentally new extension to the ES object model.
I (as one of the people who has advocated for symbols) disagree entirely. The JS object model extension required for symbols is small -- JS objects now map from either strings or symbols to values (plus prototype inheritance, getters, setters, configurability, writeability, and the other aspects of the JS object model the quote below glosses over).
The reason that the Proxy API is complex from the perspective of private symbols is that combining reflection with encapsulation and object-capability based security principles is a tricky and less-well-explored domain. Fortunately, Tom as well as others have done an excellent job with this design, and I'm confident in it. However, that complexity is not the same as complexity in the fundamental object model.
As further evidence, the word "private" does not even occur in sections 8.1.6 and 8.1.6.1 of the current ES6 draft. These are the sections that define the ES6 object model. Small changes and additions had to be made to allow for property keys to be either strings or symbols but those changes are independent of whether a symbol is private or not. The only place that the privateness of a symbol comes into play (besides in proxies) is in the context of a few reflection operations whose behavior is predicated upon whether a symbol property key is a private symbol or not. This is very similar to the tests that the same or similar operations make on individual property attributes.
I (as one of the people who has advocated for symbols) disagree entirely. The JS object model extension required for symbols is small -- JS objects now map from either strings or symbols to values (plus prototype inheritance, getters, setters, configurability, writeability, and the other aspects of the JS object model the quote below glosses over).
You've misunderstood the distinction between private slots and (unique) symbols. Symbols are fine. I'm taking issue with the new kind of slot that is created when using a private symbol.
The reason that the Proxy API is complex from the perspective of
private symbols is that combining reflection with encapsulation and object-capability based security principles is a tricky and less-well-explored domain. Fortunately, Tom as well as others have done an excellent job with this design, and I'm confident in it. However, that complexity is not the same as complexity in the fundamental object model.
I disagree. The proxy design reflects the complexity of the underlying object model. And the proxy design is not the only place where the complexity is apparent. What about "freezing" that is no longer "freezing"?
In any case, my point still stands: changes to the object model should be thoroughly justified.
On Mon, Jan 14, 2013 at 11:28 AM, Kevin Smith <khs4473 at gmail.com> wrote:
I (as one of the people who has advocated for symbols) disagree entirely. The JS object model extension required for symbols is small -- JS objects now map from either strings or symbols to values (plus prototype inheritance, getters, setters, configurability, writeability, and the other aspects of the JS object model the quote below glosses over).
You've misunderstood the distinction between private slots and (unique) symbols. Symbols are fine. I'm taking issue with the new kind of slot that is created when using a private symbol.
There is no "new kind of slot". Symbols, private and otherwise, go in exactly the same kind of slot as any other object property. As Allan points out, this is clear in the object model portion of the spec, which does not distinguish any "new kind of slot".
As further evidence, the word "private" does not even occur in sections 8.1.6 and 8.1.6.1 of the current ES6 draft. These are the sections that define the ES6 object model. Small changes and additions had to be made to allow for property keys to be either strings or symbols but those changes are independent of whether a symbol is private or not. The only place that the privateness of a symbol comes into play (besides in proxies) is in the context of a few reflection operations whose behavior is predicated upon whether a symbol property key is a private symbol or not. This is very similar to the tests that the same or similar operations make on individual property attributes.
At the micro-level, it may not seem like a big deal. But at the macro level, it is a new kind of slot with new behavior and it changes the nature of objects and properties. It changes the assumptions that you can make about objects.
It may be a great idea - I just don't see the justification, yet.
Le 14/01/2013 16:57, Kevin Smith a écrit :
When a property is added to an object using a private symbol, it creates a new kind of slot: a private slot. It is clear from the design of the Proxy API that this represents a fundamentally new extension to the ES object model.
On the very first paragraph of the CoffeeScript homepage, we read:
Underneath all those awkward braces and semicolons, JavaScript has always had a gorgeous object model at its heart
Nothing against CoffeeScript, but the Coffeescript homepage doesn't hold for the unique source of truth about JavaScript. My personal take on the ES5 object model with property descriptors and [[Extensible]] is that it's a way too complicated model. But that's a necessary one given history, the DOM, the need to fix the platform underneath your feet, etc.
It seems to me that any changes or extensions to the ES object model should be justified by well-documented application-focused use cases, clearly illustrating the need for such a change.
Per-instance private state always come with absurd costs in the ES5 model.
function C(){
var s = String(Math.random()).substring(2);
this.firstLetter = function(){
return s.substring(0, 1);
};
}
In this case, each instance has its own method (each function has its own well-encapsulated scope), costing memory. This shouldn't be necessary.
It's also possible to have the method inherited:
function C(){
this.init();
}
var states = new WeakMap();
C.prototype = {
init: function(){
var state = {s: String(Math.random()).substring(2)};
states.set(this, state);
},
firstLetter(){
var state = states.get(this);
return state.s.substring(0, 1);
}
}
There is some boilerplate (all the weakmap game) making the code less easily readable and runtime costs (at least memory, because one additional state object is created per instance!) to this kind of solutions too. The additional state object will also cost something in terms of GC.
I was obviously talking about actually private state, not _properties, which have no additional runtime cost, but are also not private at all!
Let's see with symbols:
var sym = new Symbol();
function C(){
this[sym] = String(Math.random()).substring(2);
}
C.prototype = {
firstLetter: function(){
return this[sym].substring(0, 1);
}
}
No boilerplate, no additional runtime costs than what's necessary.
I would personally like to see answers to the following questions:
- Do private slots enable applications that would otherwise be impossible?
I guess applications where you need memory for your actual content and not memory to compensate the lack of language expressiveness. I have no real idea of how much can be saved in terms of memory between the 3 snippets I've shared with millions of C instances.
- What are some examples of real-world applications where the runtime security of private slots is necessary?
It really depends on what you mean by "necessary". As far as I'm concerned, it'd be all application using third-party code and a lot of websites embed a Google Analytics script, so that would be a lot of them (looking forward to the day GA is being hacked by the way :-) )
- In those applications, are there acceptable methods of achieving the same results which do not rely on extending the object model?
I showed 2 alternatives. Whether they are acceptable depends on whether you'll fill the runtime memory up to crashing it. For most cases, it won't be the case, I agree.
- If GC performance is a significant motivator, then can this supposed benefit be quantified?
I don't feel qualified to answer this point.
From the teachability perspective, I'm tired of explaining the closure hack to explain "private" properties. Even to some who are experienced webdevs, I have to explain that they can't access the "private" property through "this.". The language needs to evolve to the point where people can write "this[something]" to retrieve private state. Symbols work for that.
There is no "new kind of slot". Symbols, private and otherwise, go in exactly the same kind of slot as any other object property.
Slots keyed with private symbols have different, novel behavior. Behavior that did not exist before. At the very least, these slots are:
- Strongly non-reflective
- Unfreezable
At a conceptual level, they change the nature of "ownership" with respect to slots.
Unless I missed a beat, they are not unfreezable. They are exactly like normal properties except they are not enumerated by any existing method including gOPN.
No boilerplate, no additional runtime costs than what's necessary.
Arguably, all of those examples could be addressed by unique symbols. Where is the need for strong runtime enforcement of encapsulation?
I would personally like to see answers to the following questions:
- Do private slots enable applications that would otherwise be impossible?
I guess applications where you need memory for your actual content and not memory to compensate the lack of language expressiveness. I have no real idea of how much can be saved in terms of memory between the 3 snippets I've shared with millions of C instances.
Private symbols are not necessary for this. Unique symbols work just fine.
- What are some examples of real-world applications where the runtime
security of private slots is necessary?
It really depends on what you mean by "necessary". As far as I'm concerned, it'd be all application using third-party code and a lot of websites embed a Google Analytics script, so that would be a lot of them (looking forward to the day GA is being hacked by the way :-) )
I want to see more along these lines. What is the security model for "loading third-party" code and in what way do private slots help?
I did miss a beat now, reading back. Admittedly, if private symbol do remain mutable as was indicated in the other thread, this certainly represents a different beast.
On Jan 14, 2013, at 8:38 AM, Kevin Smith wrote:
There is no "new kind of slot". Symbols, private and otherwise, go in exactly the same kind of slot as any other object property.
Slots keyed with private symbols have different, novel behavior. Behavior that did not exist before. At the very least, these slots are:
- Strongly non-reflective
- Unfreezable
Private named slots can be "frozen" just like any slot by setting their writable and configurable attributes to false. It is just that the Object.freeze function does not do so for such properties.
On Mon, Jan 14, 2013 at 11:50 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Jan 14, 2013, at 8:38 AM, Kevin Smith wrote:
There is no "new kind of slot". Symbols, private and otherwise, go in exactly the same kind of slot as any other object property.
Slots keyed with private symbols have different, novel behavior. Behavior that did not exist before. At the very least, these slots are:
- Strongly non-reflective
- Unfreezable
Private named slots can be "frozen" just like any slot by setting their writable and configurable attributes to false. It is just that the Object.freeze function does not do so for such properties.
And again, Object.freeze can be thought of as simply enumerating the
accessible properties and freezes them, which works unchanged with
private symbols, since they are not reflectively accessible via
getOwnPropertyNames.
And again,
Object.freezecan be thought of as simply enumerating the accessible properties and freezes them, which works unchanged with private symbols, since they are not reflectively accessible via getOwnPropertyNames.
I think of this fact as rather supporting my view, because freezing is no longer freezing the whole object, as before. Thus, while the freeze operation might remain unchanged, the object model has shifted under our feet.
On Mon, Jan 14, 2013 at 12:47 PM, Kevin Smith <khs4473 at gmail.com> wrote:
And again,
Object.freezecan be thought of as simply enumerating the accessible properties and freezes them, which works unchanged with private symbols, since they are not reflectively accessible via getOwnPropertyNames.I think of this fact as rather supporting my view, because freezing is no longer freezing the whole object, as before. Thus, while the freeze operation might remain unchanged, the object model has shifted under our feet.
The concept of freezing the "whole object" is the problem. If you
call freeze on a DOM object, does the page stop changing? Of course
not -- objects can have hidden state that you, the client of that
object, have no access to and thus no way to freeze. This isn't
something new being added here.
From the teachability perspective, I'm tired of explaining the closure hack to explain "private" properties. Even to some who are experienced webdevs, I have to explain that they can't access the "private" property through "this.". The language needs to evolve to the point where people can write "this[something]" to retrieve private state. Symbols work for that.
Below is a slight variation of the closure "hack" that allows using private properties through this. The idea is to use the public 'this' as prototype for the private 'this', using 'bind' to give instance methods access to the private 'this'
Claus
var MyClass = (function () {
function MyClass(id) {
this.id = id;
var private_this = Object.create(this); // this and more
private_this.secret = Math.random().toString();
this.guess = guess.bind(private_this);
}
function guess(guess) {
var check = guess===this.secret
? "I'm not telling!"
: "That guess is wrong!";
console.log("("+this.id+"'s secret is: "+this.secret+")");
console.log(this.id+' says: '+check);
}
return MyClass;
})();
var myObj1 = new MyClass("instance1"); var myObj2 = new MyClass("instance2");
console.log(myObj1,myObj2);
var guess = Math.random().toString(); console.log("guessing: "+guess); myObj1.guess(guess); myObj2.guess(guess);
The concept of freezing the "whole object" is the problem. If you call
freezeon a DOM object, does the page stop changing? Of course not -- objects can have hidden state that you, the client of that object, have no access to and thus no way to freeze. This isn't something new being added here.
Slots whose existence is guaranteed to be completely hidden from all code that does not possess a "magic key" looks new to me.
You can argue that the change is justified (that's the kind of argument I'd like to see in fact), but arguing that it "isn't something new" isn't working for me.
Le 14 janv. 2013 à 19:29, Kevin Smith <khs4473 at gmail.com> a écrit :
The concept of freezing the "whole object" is the problem. If you call
freezeon a DOM object, does the page stop changing? Of course not -- objects can have hidden state that you, the client of that object, have no access to and thus no way to freeze. This isn't something new being added here.Slots whose existence is guaranteed to be completely hidden from all code that does not possess a "magic key" looks new to me.
You can argue that the change is justified (that's the kind of argument I'd like to see in fact), but arguing that it "isn't something new" isn't working for me.
{ Kevin }
es-discuss mailing list es-discuss at mozilla.org, mail.mozilla.org/listinfo/es-discuss
Really, there is nothing new. Try this in your favorite ES5-compliant browser:
d = new Date(2000, 0, 1) Object.freeze(d) d // output: Sat Jan 1 2000 ... d.setFullYear(2001) d // output: Mon Jan 1 2001 ... Uh? the date was not frozen!
The value of the Date object is stored in some internal "slot", which is not enumerable with Object.getOwnPropertyName or configurable with Object.defineProperty, but only accessed by some ad-hoc API.
Symbols allow you to store protected properties (just like the date of a Date object) in a natural manner, without resorting to some clever hack. And the very definition of "protected" means that you can't access if you haven't the key. It is really the raison d'être of symbols.
—Claude
Le 14 janv. 2013 à 20:46, Claude Pache <claude.pache at gmail.com> a écrit :
Le 14 janv. 2013 à 19:29, Kevin Smith <khs4473 at gmail.com> a écrit :
The concept of freezing the "whole object" is the problem. If you call
freezeon a DOM object, does the page stop changing? Of course not -- objects can have hidden state that you, the client of that object, have no access to and thus no way to freeze. This isn't something new being added here.Slots whose existence is guaranteed to be completely hidden from all code that does not possess a "magic key" looks new to me.
You can argue that the change is justified (that's the kind of argument I'd like to see in fact), but arguing that it "isn't something new" isn't working for me.
{ Kevin }
es-discuss mailing list es-discuss at mozilla.org, mail.mozilla.org/listinfo/es-discuss
Really, there is nothing new.
In fact, I meant: this sort of behaviour is already implemented in predefined objects, and symbols provide a way to implement it naturally in user-defined objects (see below). So there is hopefully something new.
Really, there is nothing new. Try this in your favorite ES5-compliant browser:
d = new Date(2000, 0, 1) Object.freeze(d) d // output: Sat Jan 1 2000 ... d.setFullYear(2001) d // output: Mon Jan 1 2001 ... Uh? the date was not frozen!
This only proves that there is relevant state that is not stored in a property. It doesn't prove the existence of private slots, except in a purely metaphorical sense.
Perhaps self-hosting the built-ins is a valid use-case for private slots. But to what extent are private slots necessary for that use case? Can we provide for that use case without making the object model more complex?
This is the kind of thing I'm after.
The common reaction I'm seeing (I knew it wouldn't be good!) takes issue with my assertion that private slots make the object model more complex.
Well they do.
Consider again the mixin scenario. A user wants to create a class with private methods:
let privateMethod = new Symbol(true);
class C {
constructor() {}
[privateMethod]() { }
publicMethod() { this[privateMethod](); }
}
And then use that class as a mixin:
class D {
constructor() { C.call(this); }
}
mixin(D.prototype, C.prototype);
The footgun fires like this:
new D().publicMethod(); // Error: no [privateMethod] on object
The answer is to use a unique symbol here instead of a private symbol. But then [privateMethod] is no longer private. Will the user be frustrated by this limitation? And do we really need private slots with strict runtime enforcement? Why?
Consider again the mixin scenario. A user wants to create a class with private methods:
let privateMethod = new Symbol(true);
class C { constructor() {} privateMethod { } publicMethod() { thisprivateMethod; } }
And then use that class as a mixin:
class D { constructor() { C.call(this); } }
mixin(D.prototype, C.prototype);
The footgun fires like this:
new D().publicMethod(); // Error: no [privateMethod] on object
The answer is to use a unique symbol here instead of a private symbol. But then [privateMethod] is no longer private. Will the user be frustrated by this limitation? And do we really need private slots with strict runtime enforcement? Why?
This is one thing that came to my mind a couple months ago, but there are a number of ways to resolve this problem, such as:
function mixin(a, b) { for (let name of Object.getOwnPropertyNames(b)) { let desc = Object.getOwnPropertyDescriptor(b, name); if (typeof desc.value == 'function') desc.value = desc.value.bind(b); Object.defineProperty(a, name, desc); } }
This, I think, makes a lot of sense with the mixin model because mixins assume they're going to be working on their own properties anyway and should be pretty self-contained.
Nathan
This is one thing that came to my mind a couple months ago, but there are a
number of ways to resolve this problem, such as:
function mixin(a, b) { for (let name of Object.getOwnPropertyNames(b)) { let desc = Object.getOwnPropertyDescriptor(b, name); if (typeof desc.value == 'function') desc.value = desc.value.bind(b); Object.defineProperty(a, name, desc); } }
In order to satisfy security guarantees, properties which are keys on private symbols are not returned by getOwnPropertyNames.
In order to satisfy security guarantees, properties which are keys on private symbols are not returned by getOwnPropertyNames.
That's why the methods are bound to the original object.
I realize, however, that non-method properties don't work correctly this way (so I probably just should have not sent my original message). Accessors could be used. You'd probably make the point that having private symbols around introduces these kinds of confusions. My opinion is I don't think it's that hard to learn how private symbols work and develop a consistent approach to mixins if you want to use them.
Nathan
In order to satisfy security guarantees, properties which are keys on
private symbols are not returned by getOwnPropertyNames.
That's why the methods are bound to the original object.
Regardless of the "this" binding, the privately keyed method will not be copied over from the source object and hence will be undefined when called.
You'd probably make the point that having private symbols around introduces these kinds of confusions.
Well, yes, there is certainly that. It's best not to have features which are superficially very similar but are quite different in actuality (e.g. unique vs. private names). But that's not my main point.
My opinion is I don't think it's that hard to learn how private symbols work and develop a consistent approach to mixins if you want to use them.
OK - but that consistent approach will be to not use private methods at all in if you want your classes to be mixable. That's a big asterisk next to the private symbol feature.
Regardless of the "this" binding, the privately keyed method will not be copied over from the source object and hence will be undefined when called.
I think you're not understanding. Here's a simpler example that runs in modern ES engines:
var A = { foo: function() { console.log('foo'); }, bar: function() { console.log('bar'); this.foo(); } };
var B = { }; B.bar = A.bar.bind(A);
B.bar();
This logs "bar" and then "foo", so foo is correctly called, even though it is not a method of B.
That's why the methods are bound to the original object.
Regardless of the "this" binding, the privately keyed method will not be copied over from the source object and hence will be undefined when called.
Sorry, I misunderstood what you were trying to get at. Your implementation
would require carrying around a fully constructed b in addition to a
fully constructed a, which I suppose is a kind of "uses-a" mixin, rather
than the inheritance-based mixin that I was trying to create.
Kevin Smith wrote:
I think of this fact as rather supporting my view, because freezing is no longer freezing the whole object, as before.
It's really none of your business when you try to freeze my object whether any of
(a) pre-existing private-symbol-named properties remain writable; (b) weakmap-encoded private state remains writable; (c) objects-as-closures environment variables remain writable.
Really. Not. Your. Business!
Thus, while the freeze operation might remain unchanged, the object model has shifted under our feet.
Did (b) or (c) shift anything under your feet? RNOB!
Le 14/01/2013 19:21, Claus Reinke a écrit :
From the teachability perspective, I'm tired of explaining the closure hack to explain "private" properties. Even to some who are experienced webdevs, I have to explain that they can't access the "private" property through "this.". The language needs to evolve to the point where people can write "this[something]" to retrieve private state. Symbols work for that.
Below is a slight variation of the closure "hack" that allows using private properties through this. The idea is to use the public 'this' as prototype for the private 'this', using 'bind' to give instance methods access to the private 'this'
Bound methods. Smart! I come to wonder why you even need the Object.create at all.
Le 14/01/2013 17:45, Kevin Smith a écrit :
No boilerplate, no additional runtime costs than what's necessary.Arguably, all of those examples could be addressed by unique symbols. Where is the need for strong runtime enforcement of encapsulation?
The variable was called "s" standing for secret. The example I gave was dummy, but examples in the wild are legion. Take your favorite node library. Anytime it defines a _property, this is where there should be a private name. One of the goal was that no one had access to the "s" part of the state. You need private symbols for that.
I would personally like to see answers to the following questions: - Do private slots enable applications that would otherwise be impossible? I guess applications where you need memory for your actual content and not memory to compensate the lack of language expressiveness. I have no real idea of how much can be saved in terms of memory between the 3 snippets I've shared with millions of C instances.Private symbols are not necessary for this. Unique symbols work just fine.
The implicit requirement was that only trusted parties were expected to have access to the "s" variable or equivalent in other examples. Unique symbols are enumerated, so do not do the job.
- What are some examples of real-world applications where the runtime security of private slots is necessary? It really depends on what you mean by "necessary". As far as I'm concerned, it'd be all application using third-party code and a lot of websites embed a Google Analytics script, so that would be a lot of them (looking forward to the day GA is being hacked by the way :-) )I want to see more along these lines. What is the security model for "loading third-party" code and in what way do private slots help?
I'm realizing now that I may be off-topic. I'm not that interested whether it's a new type of slot with the non-freezability property. I only care that it's private (non-enumerable with Object.getOwnPropertyNames)
Thinking more about the loading third-party code, I guess it's technically doable to do without private names. It comes to the cost of creating proxies doing the encapsulation for you. You provide a blacklist of properties that must not be reflected and the third party never sees them... In essence, you're listing the private properties and the proxy does the book keeping (details about non-configurability aside). It'd be more appropriate to define this private properties directly on the object if there is a commonality to what you wish keeping to your self when considering sharing the object.
I don't know whether this justifies a new type of slot. Brendan's "none of your business" argument has a point. Additionally, if you do not have access to a private symbol, why should you be provided by default the right to change related private property descriptors?
It's really none of your business when you try to freeze my object whether any of
(a) pre-existing private-symbol-named properties remain writable; (b) weakmap-encoded private state remains writable; (c) objects-as-closures environment variables remain writable.
Really. Not. Your (user). Business!
But it is Your (author) business, and turning everything into a proxy to get control over how the authored objects behave seems a little excessive. And the same thing applies to clone/ mixin.
It has been pointed out that the issue is one of implicitly called iterators: standard methods for freezing or cloning call iterators that only handle public API.
I have suggested before that it would be good to put control over object iteration into the hands of the object authors, by enabling them to override the slot iteration method.
One would need to find a way of doing so without exposing private names, but it should allow object authors to handle your a-c, as well as define what cloning/mixing should do in the presence of private state (however encoded, although private slots might make this easier/more explicit).
Claus
I have suggested before that it would be good to put control over object iteration into the hands of the object authors, by enabling them to override the slot iteration method. One would need to find a way of doing so without exposing private names, but it should allow object authors to handle your a-c, as well as define what cloning/mixing should do in the presence of private state (however encoded, although private slots might make this easier/more explicit).
I might be missing something, but isn't this basically covered with the enumerable flag? And that brings us back around - if we only had unique symbols and we defined the property using the symbol, and made it enumerable false, would it still be private? Am I missing something obvious (I probably am)? I know this is more work, but it would be more orthogonal. Sugar could be added to class to make this better.
It's really none of your business when you try to freeze my object whether any of
(a) pre-existing private-symbol-named properties remain writable; (b) weakmap-encoded private state remains writable; (c) objects-as-closures environment variables remain writable.
Really. Not. Your. Business!
But that's a change from the current object model and by (a) you're assuming the conclusion. ES has a really simple object model, as explained in the first part of the ES5 specification. That simplicity is an advantage. If you're going add complexity, then it should be justified with application-focused use cases. That request does not seem like a stretch to me.
More in reply to David...
The variable was called "s" standing for secret. The example I gave was dummy, but examples in the wild are legion. Take your favorite node library. Anytime it defines a _property, this is where there should be a private name.
Again, that's arguable. Certainly it would be better expressed with a symbol. But private? Are most node authors attempting to write "secure" code? Of course not! Node (by itself) does not provide a secure environment for untrusted code.
One of the goal was that no one had access to the "s" part of the state.
You need private symbols for that.
But why? Where is this strict runtime privacy requirement coming from? What I'm not understanding is the larger context in which mutually untrusting code supposedly shares raw (non-proxied) objects.
Where is the real world code doing this? For what applications? Really, I want to know! : )
In "Crockfordian" design, "name encapsulation" and "security encapsulation" are indistinguishable. But unique symbols address "name encapsulation". Is there really a need to address "security encapsulation" at the object property level?
Thinking more about the loading third-party code, I guess it's technically doable to do without private names. It comes to the cost of creating proxies doing the encapsulation for you. You provide a blacklist of properties that must not be reflected and the third party never sees them... In essence, you're listing the private properties and the proxy does the book keeping (details about non-configurability aside).
Sure - and approaches like this (or simpler - people are clever!) can be factored away into a neat library, without having to mess with the underlying object model.
ES6 provides WeakMaps and Proxies. Why not see what people do with those before introducing private slots?
Claus Reinke wrote:
It's really none of your business when you try to freeze my object whether any of
(a) pre-existing private-symbol-named properties remain writable; (b) weakmap-encoded private state remains writable; (c) objects-as-closures environment variables remain writable.
Really. Not. Your (user). Business!
But it is Your (author) business, and turning everything into a proxy to get control over how the authored objects behave seems a little excessive.
What have proxies to do with any of a-c? I don't follow.
And the same thing applies to clone/ mixin.
Agree proxies should not be required to do standard O-O things.
It has been pointed out that the issue is one of implicitly called iterators: standard methods for freezing or cloning call iterators that only handle public API.
I think you're missing the point. Object.freeze deals in certain observables. It does not free closure state, or weakmap-encoded state, or (per the ES6 proposal) private-symbol-named property state where such properties already exist. Those are not directly observable by the reflection API.
Your -- as in You the abstraction implementor -- may indeed make such state observable. That's a feature. Think of const methods with mutable this members in C++.
Kevin Smith wrote:
It's really none of your business when you try to freeze my object whether any of (a) pre-existing private-symbol-named properties remain writable; (b) weakmap-encoded private state remains writable; (c) objects-as-closures environment variables remain writable. Really. Not. Your. Business!But that's a change from the current object model
(b) is a change from ES5 but you don't object to weak maps -- why not?
(c) is not a change from JS's object model!
and by (a) you're assuming the conclusion.
No, I'm saying the horse already left the barn.
ES has a really simple object model, as explained in the first part of the ES5 specification. That simplicity is an advantage. If you're going add complexity, then it should be justified with application-focused use cases. That request does not seem like a stretch to me.
Of course, we justified weak maps with specific use-cases. Those do not cover private symbols, because private symbols are really "in" the object, they do not identify storage in a side table. This matters when abstracting over strings or symbol (private or unique) property names, as I showed. It matters when using the bracketing operator (a weak map would have to be exposed and its use transposes operands of []). It matters for any future private syntax.
Finally, efficiency does matter. The GC cost of weak maps is just one cost I mentioned. The property access cost is another, which you did not address.
Brendan Eich wrote:
Object.freeze deals in certain observables. It does not free
("freeze", of course.)
Sure - and approaches like this (or simpler - people are clever!) can be factored away into a neat library, without having to mess with the underlying object model.
ES6 provides WeakMaps and Proxies. Why not see what people do with those before introducing private slots?
Is it really such large change to the object model? Couldn't it be considered something akin to an extra property descriptor? Non-enumerable properties don't show up in for..in or Object.keys. This is sort of "non-reflectable". Non-reflectable properties don't show up in for..in, Object.keys, or Object.getOPN.
Nathan
(b) is a change from ES5 but you don't object to weak maps -- why not?
WeakMaps provide a new feature without extending the object model. They are "external" to objects, if you like.
(c) is not a change from JS's object model!
Sure - but I'm not objecting to closed-over state (in any form). That would be ludicrous. : )
and by (a) you're assuming the conclusion.
No, I'm saying the horse already left the barn.
Again, not objecting to private state. But just because we allow private state, it does not necessarily follow that we should mess with the object model to accomodate it. Maybe it's worth it, I don't know. That's the whole point: I don't know. And when in doubt...
Of course, we justified weak maps with specific use-cases. Those do not cover private symbols, because private symbols are really "in" the object, they do not identify storage in a side table. This matters when abstracting over strings or symbol (private or unique) property names, as I showed. It matters when using the bracketing operator (a weak map would have to be exposed and its use transposes operands of []). It matters for any future private syntax.
But is this not premature optimization? How do I know that WeakMaps and Proxies are insufficient?
Finally, efficiency does matter. The GC cost of weak maps is just one cost I mentioned. The property access cost is another, which you did not address.
Again, aren't we optimizing private state lookup without field testing? I hear people say "me wants private!" but for the common case, I'm not seeing the need for runtime security. Not yet anyway. I want to understand the security model that would motivate such a universal need.
Kevin Smith wrote:
(b) is a change from ES5 but you don't object to weak maps -- why not?WeakMaps provide a new feature without extending the object model. They are "external" to objects, if you like.
They are a kind of object, on the contrary.
(c) is not a change from JS's object model!Sure - but I'm not objecting to closed-over state (in any form). That would be ludicrous. : )
But that's part of the object model, no? If so, the CoffeeScript summary-manqué is not worth much, and your argument from it, even less so :-|.
and by (a) you're assuming the conclusion. No, I'm saying the horse already left the barn.Again, not objecting to private state. But just because we allow private state, it does not necessarily follow that we should mess with the object model to accomodate it.
I think you've chosen a cartoon-short version of "the object model" to argue selectively against other things.
Maybe it's worth it, I don't know. That's the whole point: I don't know. And when in doubt...
Hold that thought.
Of course, we justified weak maps with specific use-cases. Those do not cover private symbols, because private symbols are really "in" the object, they do not identify storage in a side table. This matters when abstracting over strings or symbol (private or unique) property names, as I showed. It matters when using the bracketing operator (a weak map would have to be exposed and its use transposes operands of []). It matters for any future private syntax.But is this not premature optimization? How do I know that WeakMaps and Proxies are insufficient?
Realistically, O-O commonplaces such as private fields and methods should not require a proxy or a weakmap, or they won't see much use.
It may be that high-integrity privacy won't see much use anyway (your point?). But it has its uses (SES is being used on the web now, details coming).
On Tue, Jan 15, 2013 at 2:01 PM, Kevin Smith <khs4473 at gmail.com> wrote:
(b) is a change from ES5 but you don't object to weak maps -- why not?
WeakMaps provide a new feature without extending the object model. They are "external" to objects, if you like.
(c) is not a change from JS's object model!
Sure - but I'm not objecting to closed-over state (in any form). That would be ludicrous. : )
Would it? I'd love to see a stronger form of freeze that "purified" an object's scope.
and by (a) you're assuming the conclusion.
No, I'm saying the horse already left the barn.
Again, not objecting to private state. But just because we allow private state, it does not necessarily follow that we should mess with the object model to accommodate it. Maybe it's worth it, I don't know. That's the whole point: I don't know. And when in doubt...
You keep objecting on the grounds of object model changes, but you seem to be totally fine with the notion of unique objects, which IMHO are the substantial object model change. Compared to this, the ability to specify keys as "non-reflectable" (as Nathan put it) seems trivial. This ability seems to me to be a natural extension of property definitions, it's just that it wasn't necessary in ES5 because it is meaningless for string keys.
[snipped the rest]
Le 15/01/2013 17:16, Kevin Smith a écrit :
The variable was called "s" standing for secret. The example I gave was dummy, but examples in the wild are legion. Take your favorite node library. Anytime it defines a _property, this is where there should be a private name.Again, that's arguable. Certainly it would be better expressed with a symbol. But private? Are most node authors attempting to write "secure" code? Of course not!
People writing Java code do use the private keyword. I think if encapsulation wasn't that hard to achieve, people would do it. I'm looking forward to see people use classes and modules without putting more effort that they do currently and yet write more secure code by default.
Node (by itself) does not provide a secure environment for untrusted code.
What I know of Node makes me think it's not that much worse than any other platform and from experience, at least much better than anything I've played wit in PHP.
One of the goal was that no one had access to the "s" part of the state. You need private symbols for that.But why? Where is this strict runtime privacy requirement coming from? What I'm not understanding is the larger context in which mutually untrusting code supposedly shares raw (non-proxied) objects.
The reason you can't write code in any blog comment these days is that people have given up on securing untrusted code. Likewise for emails. HTML is used for emails, but scripts aren't executed, because email client (web-based or not) have given up on the idea of securing email code.
Where is the real world code doing this?
I think iGoogle does this. Otherwise, close to nowhere because people have given up. My guess is that they have given up because the language does not make easy to sandbox untrusted code. As soon as you have naturally written your code in JavaScript, it's unsafe and it takes a lot of work making it safe. People give up. Private symbols are one tool to lower the barrier to writing code secure by default.
For what applications? Really, I want to know! : )
They do not exist unfortunately. That's a chicken and egg problem. As soon as it'll be easier, people will probably restart thinking of what they want to do with the new tool they have in hand. That's what happened when JS engines got faster. Developers didn't stay still happy that their websites just got faster. Websites just started to get more and more JS.
In "Crockfordian" design, "name encapsulation" and "security encapsulation" are indistinguishable. But unique symbols address "name encapsulation". Is there really a need to address "security encapsulation" at the object property level?
It seems that it lowers the barrier to writing secure code by default. If that's the only thing to gain, I'll take it.
Thinking more about the loading third-party code, I guess it's technically doable to do without private names. It comes to the cost of creating proxies doing the encapsulation for you. You provide a blacklist of properties that must not be reflected and the third party never sees them... In essence, you're listing the private properties and the proxy does the book keeping (details about non-configurability aside).Sure - and approaches like this (or simpler - people are clever!) can be factored away into a neat library, without having to mess with the underlying object model.
ES6 provides WeakMaps and Proxies. Why not see what people do with those before introducing private slots?
I wouldn't be opposed to that, but that's just my opinion. Still, private symbols allow property-like syntax. I haven't followed the latest developments of how classes and private symbols interact, but I'm not too worried it goes well. Assuming it does, it makes it easy for people to use actually private properties in their code, lowering the barrier to writing well-encapsulated code.
David Bruant wrote:
ES6 provides WeakMaps and Proxies. Why not see what people do with those before introducing private slots? I wouldn't be opposed to that, but that's just my opinion. Still, private symbols allow property-like syntax. I haven't followed the
Hm. If this would be the best benefit (and putting aside the issue of WeakMap GC cost for a moment), one may think about "property-like access, but with reversed responsibilities" which could allow for easing things like that. For example, if I abused @ character for that (as a replacement for .), I could write:
foo.bar at baz.quux to mean baz[foo.bar].quux // reversed access, ie WeakMap
and
foo.bar at baz(quux) to mean baz(foo.bar, quux) // more functional OO-like
(not necessarily the same syntax, just the idea, that, instead of private slots, one can provide "reversal" into the language allowing to use certain pattern easier)
In defense of Kevin, I too would argue that private symbols add complexity to the object model. That doesn't mean I'm arguing in favor of removing private symbols, just noting that we do pay a complexity price.
We should not deny that with the addition of private symbols, we had to reconsider every existing ES5 operation to see whether it needed to be adjusted so that it does not leak private symbols. We happen to be lucky that Object.keys and Object.freeze can be thought of as using Object.getOwnPropertyNames, so adjusting gOPN (and the Proxy API) closed all leaks identified thus far. Going forward from here, we need to continue to be vigilant about private symbol leaks.
I also think Kevin is making the fair point that for the 99% use case, unique symbols are sufficient. For software engineering purposes, they provide good enough encapsulation. "private" in Java is not really private due to java.lang.reflect, but that doesn't stop people using it to express encapsulation.
Since Kevin is merely asking for a good explanation of why private symbols are "in", I would say that it's because TC39 explicitly wanted symbols to be usable for high-integrity encapsulation scenarios.
Kevin's reply to that is "if the use case is high-integrity encapsulation, why aren't WeakMaps sufficient?"
The response to that is: WeakMaps add more boilerplate and more overhead. I don't think the WeakMap boilerplate overhead is an actual issue for people writing defensive code (such code already has to go so out-of-its-way, e.g. by using early-bound primordials, I think it can deal with WeakMaps). The performance argument still stands though.
Cheers, Tom
2013/1/15 David Bruant <bruant.d at gmail.com>
Le 15/01/2013 21:06, Herby Vojčík a écrit :
David Bruant wrote:
ES6 provides WeakMaps and Proxies. Why not see what people do with those before introducing private slots? I wouldn't be opposed to that, but that's just my opinion. Still, private symbols allow property-like syntax. I haven't followed the
Hm. If this would be the best benefit (and putting aside the issue of WeakMap GC cost for a moment), one may think about "property-like access, but with reversed responsibilities" which could allow for easing things like that. For example, if I abused @ character for that (as a replacement for .), I could write:
foo.bar at baz.quux to mean baz[foo.bar].quux // reversed access, ie WeakMap
and
foo.bar at baz(quux) to mean baz(foo.bar, quux) // more functional OO-like
(not necessarily the same syntax, just the idea, that, instead of private slots, one can provide "reversal" into the language allowing to use certain pattern easier)
I agree there is an equivalent between private symbols and weakmap. Both are a (object, object) -> value mapping (symbols can be getters, etc,
but it could be implemented with weakmaps too)
In theory, private symbols should have the same GC issues than WeakMap. Consider:
var o = {}, o2 = {};
for(var i=0, i<1000, i++){
let s = new Symbol(true /*private*/);
o[s] = o2[s] = i;
}
Here, private symbols are created, properties are added. Because of let-locality, every symbol can be GC'ed, because at the end of the loop, no one can ever access any of the symbols. The equivalent of the WeakMap GC property would want properties in o and o2 to be removed too. I'm willing to bet this will never happen in any JS engine.
The major difference is how people will use each tool. From experience in OO programming, most objects have a finite set of properties determined at object creation time. It's true even in JavaScript where objects are bags of properties. This allows to write "stable" code (you can write "o.find()" assuming there is a find property which value is an function). If people were creating eratic objects, writing what I call "stable" code would be a much more complicated exercise. This "cultural" use of objects is what makes optimizations like Shape/HiddenClass possible. WeakMaps, on the other hand, are used as a maps key'ed on objects, with the convenience that you don't need to clean the map when the object is about to disappear. There is a assumption that when you create a new weakmap, its use is largely independent of previous weakmaps and it's unlikely 2 weakmaps will have the same set of objects, so engines are unlikely to perform optimizations combining different weakmap storages so that the values of the same key will be close to one another or something like that.
WeakMap and Symbols each corresponding to a given patterns of use of (object, object) -> value mappings. Although a unique feature could be
used to emulate the other, it would be very hard for an implementation to optimize one feature for 2 patterns. If anything, choosing one feature over the other is a way to tell your coworkers and the JS engine which pattern you want to use.
Very fair response -- thanks. I want to add praise for Kevin, he asks the right questions, especially the one to which I replied "hold that thought":
Maybe it's worth it, I don't know. That's the whole point: I don't
know. And when in doubt...
We certainly did leave out something in doubt: declarative private @-name syntax. But private symbols (vs. unique ones) seem to be worth their weight for the better performance model, as Tom notes.
On Tue, Jan 15, 2013 at 4:07 PM, Brendan Eich <brendan at mozilla.com> wrote:
Very fair response -- thanks. I want to add praise for Kevin, he asks the right questions, especially the one to which I replied "hold that thought":
Maybe it's worth it, I don't know. That's the whole point: I don't know. And when in doubt...
We certainly did leave out something in doubt: declarative private @-name syntax. But private symbols (vs. unique ones) seem to be worth their weight for the better performance model, as Tom notes.
Let's not forget the better prototypal inheritance story as well.
On Tue, Jan 15, 2013 at 12:10 PM, Tom Van Cutsem <tomvc.be at gmail.com> wrote:
In defense of Kevin, I too would argue that private symbols add complexity to the object model. That doesn't mean I'm arguing in favor of removing private symbols, just noting that we do pay a complexity price.
We should not deny that with the addition of private symbols, we had to reconsider every existing ES5 operation to see whether it needed to be adjusted so that it does not leak private symbols. We happen to be lucky that Object.keys and Object.freeze can be thought of as using Object.getOwnPropertyNames, so adjusting gOPN (and the Proxy API) closed all leaks identified thus far. Going forward from here, we need to continue to be vigilant about private symbol leaks.
I also think Kevin is making the fair point that for the 99% use case, unique symbols are sufficient. For software engineering purposes, they provide good enough encapsulation. "private" in Java is not really private due to java.lang.reflect, but that doesn't stop people using it to express encapsulation.
Since Kevin is merely asking for a good explanation of why private symbols are "in", I would say that it's because TC39 explicitly wanted symbols to be usable for high-integrity encapsulation scenarios.
Kevin's reply to that is "if the use case is high-integrity encapsulation, why aren't WeakMaps sufficient?"
The response to that is: WeakMaps add more boilerplate and more overhead. I don't think the WeakMap boilerplate overhead is an actual issue for people writing defensive code (such code already has to go so out-of-its-way, e.g. by using early-bound primordials, I think it can deal with WeakMaps). The performance argument still stands though.
If the only thing keeping private symbols alive is this performance argument, then I think they could die. The performance issue with WeakMaps is easily fixed with a boolean hint that I suggested a long time ago. It was dropped as too complex, which I appreciate, but if it enables dropping something yet more complicated....
Independent of this hint, wm.set(key, value) needs to make value reachable if wm and key are reachable. In the absence of this hint, to assess the space complexity of an algorithm expressed using WeakMaps, we may assume that the value is not made reachable any longer than that[1]. With the hint, we may only assume that the value does not outlive the key. It may very well outlive the WeakMap. This allows an implementation to use exactly the same implementation techniques internally for these wimply weak maps that we expect them to use for private symbols.
[1] if not otherwise reachable of course
On Tue, Jan 15, 2013 at 12:30 PM, David Bruant <bruant.d at gmail.com> wrote: [...]
I agree there is an equivalent between private symbols and weakmap. Both are a (object, object) -> value mapping (symbols can be getters, etc, but it could be implemented with weakmaps too)
In theory, private symbols should have the same GC issues than WeakMap. [...] WeakMap and Symbols each corresponding to a given patterns of use of (object, object) -> value mappings. Although a unique feature could be used to emulate the other, it would be very hard for an implementation to optimize one feature for 2 patterns.
Without a hint.
If anything, choosing one feature over the other is a way to tell your coworkers and the JS engine which pattern you want to use.
Or the hint could be.
Although the hint and the expectations it signals is complex, it has no semantics beyond this difference in ability of a program to claim asymptotic space complexity. And really, these claims are a lie anyway strawman:gc_semantics, since
we've never made it normative that a JS engine must collect any garbage at all, much less which garbage.
Nevertheless, I am not advocating that we drop private symbols, at least at this time. My point is only that the performance issue, by itself, is not a reason to keep them.
David Bruant wrote:
In theory, private symbols should have the same GC issues than WeakMap. Consider:
var o = {}, o2 = {}; for(var i=0, i<1000, i++){ let s = new Symbol(true /*private*/); o[s] = o2[s] = i; }Here, private symbols are created, properties are added. Because of let-locality, every symbol can be GC'ed, because at the end of the loop, no one can ever access any of the symbols. The equivalent of the WeakMap GC property would want properties in o and o2 to be removed too. I'm willing to bet this will never happen in any JS engine.
This is not the "GC issue" that was raised, precisely because WeakMap does make no-cyclic-leak guarantees that Symbols as used in your example do not.
So Symbols could have the leak problem that WeakMaps cure, and that is another issue.
But the cost of allocating a WeakMap per object that wants private properties, and then GC'ing it promptly per the guarantee, is the main issue. When you need a WeakMap, this cost must be born. When you just need private properties (pace Kevin on "need"), the allocation and the GC mark/sweep overhead should not be imposed lightly.
Another issue, as mentioned: property access performance with private symbols (or unique/public ones) will be much better, out of the gates. Not a GC issue, just repeating so it doesn't get lost.
The major difference is how people will use each tool. From experience in OO programming, most objects have a finite set of properties determined at object creation time. It's true even in JavaScript where objects are bags of properties. This allows to write "stable" code (you can write "o.find()" assuming there is a find property which value is an function). If people were creating eratic objects, writing what I call "stable" code would be a much more complicated exercise. This "cultural" use of objects is what makes optimizations like Shape/HiddenClass possible.
Yup, I should have read ahead.
WeakMaps, on the other hand, are used as a maps key'ed on objects, with the convenience that you don't need to clean the map when the object is about to disappear. There is a assumption that when you create a new weakmap, its use is largely independent of previous weakmaps and it's unlikely 2 weakmaps will have the same set of objects, so engines are unlikely to perform optimizations combining different weakmap storages so that the values of the same key will be close to one another or something like that.
WeakMap and Symbols each corresponding to a given patterns of use of (object, object) -> value mappings. Although a unique feature could be used to emulate the other, it would be very hard for an implementation to optimize one feature for 2 patterns. If anything, choosing one feature over the other is a way to tell your coworkers and the JS engine which pattern you want to use.
Good point. There is inherent complexity in having two things, with some overlap. I don't see it as fatal. I think Kevin's stronger point is YAGNI re: high-integrity privacy, outside of SES use-cases.
But SES support is a goal of Harmony and ES6.
The GC cost is one problem, but the property access cost is another, and the second allocation (wm for every obj) yet another. And as Dean just reminded, prototypal inheritance favors symbols. It's not just one cost that motivates symbols over weakmaps if the goal is OO privacy.
On Tue, Jan 15, 2013 at 1:56 PM, Brendan Eich <brendan at mozilla.com> wrote:
David Bruant wrote:
In theory, private symbols should have the same GC issues than WeakMap. Consider:
var o = {}, o2 = {}; for(var i=0, i<1000, i++){ let s = new Symbol(true /*private*/); o[s] = o2[s] = i; }Here, private symbols are created, properties are added. Because of let-locality, every symbol can be GC'ed, because at the end of the loop, no one can ever access any of the symbols. The equivalent of the WeakMap GC property would want properties in o and o2 to be removed too. I'm willing to bet this will never happen in any JS engine.
This is not the "GC issue" that was raised, precisely because WeakMap does make no-cyclic-leak guarantees that Symbols as used in your example do not.
So Symbols could have the leak problem that WeakMaps cure, and that is another issue.
But the cost of allocating a WeakMap per object that wants private properties,
I'm not following this. Why would you allocate more WeakMaps than you would have allocated private symbols? If the goal is per-class privacy, which is the private symbol use case, you'd allocate one WeakMap per field per class just as you would have with private symbols.
Mark S. Miller wrote:
On Tue, Jan 15, 2013 at 1:56 PM, Brendan Eich<brendan at mozilla.com> wrote:
David Bruant wrote:
In theory, private symbols should have the same GC issues than WeakMap. Consider:
var o = {}, o2 = {}; for(var i=0, i<1000, i++){ let s = new Symbol(true /*private*/); o[s] = o2[s] = i; }Here, private symbols are created, properties are added. Because of let-locality, every symbol can be GC'ed, because at the end of the loop, no one can ever access any of the symbols. The equivalent of the WeakMap GC property would want properties in o and o2 to be removed too. I'm willing to bet this will never happen in any JS engine. This is not the "GC issue" that was raised, precisely because WeakMap does make no-cyclic-leak guarantees that Symbols as used in your example do not.
So Symbols could have the leak problem that WeakMaps cure, and that is another issue.
This stuff still stands.
But the cost of allocating a WeakMap per object that wants private properties,
I'm not following this. Why would you allocate more WeakMaps than you would have allocated private symbols? If the goal is per-class privacy, which is the private symbol use case, you'd allocate one WeakMap per field per class just as you would have with private symbols.
The savings comes from prototypal inheritance. Given one class with 2 private methods, you need only 2 symbols. These are flat frozen objects, maybe even internally more efficiently represented.
Given one class with 2 private weakmaps, you need those two weakmaps, indeed also allocations -- but then they must expand to index every instance of the class!
On Tue, Jan 15, 2013 at 2:27 PM, Brendan Eich <brendan at mozilla.com> wrote:
Mark S. Miller wrote:
On Tue, Jan 15, 2013 at 1:56 PM, Brendan Eich<brendan at mozilla.com> wrote:
David Bruant wrote:
In theory, private symbols should have the same GC issues than WeakMap. Consider:
var o = {}, o2 = {}; for(var i=0, i<1000, i++){ let s = new Symbol(true /*private*/); o[s] = o2[s] = i; }Here, private symbols are created, properties are added. Because of let-locality, every symbol can be GC'ed, because at the end of the loop, no one can ever access any of the symbols. The equivalent of the WeakMap GC property would want properties in o and o2 to be removed too. I'm willing to bet this will never happen in any JS engine.
This is not the "GC issue" that was raised, precisely because WeakMap does make no-cyclic-leak guarantees that Symbols as used in your example do not.
So Symbols could have the leak problem that WeakMaps cure, and that is another issue.
The hint I spoke of would suggest to the engine that it's ok for this weakmap to have this same leak, i.e., to allow the (wm,key)=>value
association to outlive wm, the weakmap.
This stuff still stands.
But the cost of allocating a WeakMap per object that wants private properties,
I'm not following this. Why would you allocate more WeakMaps than you would have allocated private symbols? If the goal is per-class privacy, which is the private symbol use case, you'd allocate one WeakMap per field per class just as you would have with private symbols.
The savings comes from prototypal inheritance. Given one class with 2 private methods, you need only 2 symbols. These are flat frozen objects, maybe even internally more efficiently represented.
Given one class with 2 private weakmaps, you need those two weakmaps, indeed also allocations -- but then they must expand to index every instance of the class!
There are two obvious implementations of weakmaps. The one you imagine above is indeed the one more naturally suggested by its semantics, where the (wm,key)=>value associations are stored as key=>value
associations within wm. The other, which corresponds to how you expect to implement private symbols, is to store wm=>value associations
within key. The first is better when key typically outlives wm. The second is better when wm typically outlives key. In the absence of the hint, it is not clear which would be more common. The hinted case, if we support the hint, should always use the second representation. Then there's no storage difference between the hinted wm story and the private symbol story that I see. Of course, other differences remain, but let's settle this one first.
On Jan 15, 2013, at 1:58 PM, Brendan Eich wrote:
The GC cost is one problem, but the property access cost is another, and the second allocation (wm for every obj) yet another. And as Dean just reminded, prototypal inheritance favors symbols. It's not just one cost that motivates symbols over weakmaps if the goal is OO privacy.
Let's consider the just the GC implications of private symbols versus WeakMaps for a very simple integrity use case -- branding.
Let's assume that we have a class of small, highly ephemeral objects that some important part of a system is very dependent upon. The integrity of the system requires that the instances be well-formed. Also, these objects are created and discarded at a high rate. (for example, perhaps some kind of Point object that implements some sort of algorithm optimized coordinate encoding). So, we want to "brand" valid instances so that algorithms that consume such objects can ensure that they are dealing with the real thing.
There are, at least, two ways to implement this. One way is to use a WeakMap as a registry for all valid instances of the class. The other is to use a private Symbol keyed property on each instance as the brand.
Now, before going on we need to understand a few thinks about modern high performance garbage collectors. One principle that applies to them is that when dealing with very ephemeral objects you want the complexity of the GC algorithms to be proportional to the number of retained (ie, non-garbage) objects processed rather than the number of allocated or garbage objects. This is because typical application algorithms that use short-lived objects often allocated orders of magnitude more object than actually remain alive when an GC actually occurs. This characteristic is foundational to generational GCs. When it is time to scavenge the nursery, you don't want the GC to have to look at 10s of thousands of allocated Points if only dozens are actually alive at that point in time. Scavenging ephemeral collectors are fast and low overhead because they only have to inspect or manipulate the actual survivors. It doesn't matter now many now dead objects were allocated, because the GC doesn't even look at them.
Use of a private Symbol for branding has no impact upon this GC behavior. The brand is just an object property and if an ephemeral branded object is unreachable it will never be inspected by the GC.
Now consider the impact upon the GC of using WeakMaps for branding.
To start with, WeakMaps uses ephemeron algorithms that add an extra phase to the GC process and (ignoring other overhead) the cost of this phase will be proportional to the number of allocated branded objects. You can over simplistically think about it this way: Some WeakDictionary contains an entry for every one of the allocated branded objects. As a final phase of each GC, each entry in the WeakDictionary must be examined to determine whether it reference an object that was identified as garbage (or conversely, non-garbage) by the previous phases of this GC cycle. If an object was identified as garbage the final phase of the GC cycle must removed it from the dictionary. So, even though there may only be dozens of surviving branded objects, the GC algorithm may have to explicitly process thousands that are garbage. You have essentially lost all the advantage of having a generational ephemeral collector.
Ephemeron-based data structures are great for dealing with a moderate number of moderately long-lived objects. But, if you care about overall system performance you don't want to use them to manage high volumes of very ephemeral objects. As someone who has a lot of GC implementation experience I would always choose to use a private Symbol over a WeakMap if either alternative can work for some particular use case.
On Tue, Jan 15, 2013 at 3:03 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Jan 15, 2013, at 1:58 PM, Brendan Eich wrote:
The GC cost is one problem, but the property access cost is another, and the second allocation (wm for every obj) yet another. And as Dean just reminded, prototypal inheritance favors symbols. It's not just one cost that motivates symbols over weakmaps if the goal is OO privacy.
Let's consider the just the GC implications of private symbols versus WeakMaps for a very simple integrity use case -- branding.
Let's assume that we have a class of small, highly ephemeral objects that some important part of a system is very dependent upon. The integrity of the system requires that the instances be well-formed. Also, these objects are created and discarded at a high rate. (for example, perhaps some kind of Point object that implements some sort of algorithm optimized coordinate encoding). So, we want to "brand" valid instances so that algorithms that consume such objects can ensure that they are dealing with the real thing.
There are, at least, two ways to implement this. One way is to use a WeakMap as a registry for all valid instances of the class. The other is to use a private Symbol keyed property on each instance as the brand.
Now, before going on we need to understand a few thinks about modern high performance garbage collectors. One principle that applies to them is that when dealing with very ephemeral objects you want the complexity of the GC algorithms to be proportional to the number of retained (ie, non-garbage) objects processed rather than the number of allocated or garbage objects. This is because typical application algorithms that use short-lived objects often allocated orders of magnitude more object than actually remain alive when an GC actually occurs. This characteristic is foundational to generational GCs. When it is time to scavenge the nursery, you don't want the GC to have to look at 10s of thousands of allocated Points if only dozens are actually alive at that point in time. Scavenging ephemeral collectors are fast and low overhead because they only have to inspect or manipulate the actual survivors. It doesn't matter now many now dead objects were allocated, because the GC doesn't even look at them.
Use of a private Symbol for branding has no impact upon this GC behavior. The brand is just an object property and if an ephemeral branded object is unreachable it will never be inspected by the GC.
Now consider the impact upon the GC of using WeakMaps for branding.
To start with, WeakMaps uses ephemeron algorithms
Not those with the hint. That's the point. Are you getting my messages? Are they unclear? Please ack even if you have no other response. Thanks.
On Jan 15, 2013, at 3:18 PM, Mark S. Miller wrote:
On Tue, Jan 15, 2013 at 3:03 PM, Allen Wirfs-Brock
To start with, WeakMaps uses ephemeron algorithms
Not those with the hint. That's the point. Are you getting my messages? Are they unclear? Please ack even if you have no other response. Thanks.
Sorry, where is this hint proposal described?
Regardless, the original request for information about the GC impact of using WeakMaps was presumably based upon the accepted WeakMap semantics.
On Tue, Jan 15, 2013 at 3:27 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Jan 15, 2013, at 3:18 PM, Mark S. Miller wrote:
On Tue, Jan 15, 2013 at 3:03 PM, Allen Wirfs-Brock
To start with, WeakMaps uses ephemeron algorithms
Not those with the hint. That's the point. Are you getting my messages? Are they unclear? Please ack even if you have no other response. Thanks.
Sorry, where is this hint proposal described?
My message in this thread at 1:54pm. Quoting
Independent of this hint, wm.set(key, value) needs to make value reachable if wm and key are reachable. In the absence of this hint, to assess the space complexity of an algorithm expressed using WeakMaps, we may assume that the value is not made reachable any longer than that[1]. With the hint, we may only assume that the value does not outlive the key. It may very well outlive the WeakMap. This allows an implementation to use exactly the same implementation techniques internally for these wimply weak maps that we expect them to use for private symbols.
[1] if not otherwise reachable of course
Regardless, the original request for information about the GC impact of using WeakMaps was presumably based upon the accepted WeakMap semantics.
Are space reclamation expectations semantic? We never decided to make them normative. If we do, then yes. But the hint still has no semantics beyond their space reclamation semantics. This arguably make the hint simpler than private symbols.
A note to avoid misunderstanding:
I have been very careful in this thread not to advocate that we drop private symbols. But if we keep them, it should be on grounds other than performance. I am arguing here only that the performance argument is a red herring.
On Jan 15, 2013, at 3:46 PM, Mark S. Miller wrote:
On Tue, Jan 15, 2013 at 3:27 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Jan 15, 2013, at 3:18 PM, Mark S. Miller wrote:
On Tue, Jan 15, 2013 at 3:03 PM, Allen Wirfs-Brock
To start with, WeakMaps uses ephemeron algorithms
Not those with the hint. That's the point. Are you getting my messages? Are they unclear? Please ack even if you have no other response. Thanks.
Sorry, where is this hint proposal described?
My message in this thread at 1:54pm. Quoting
Independent of this hint, wm.set(key, value) needs to make value reachable if wm and key are reachable. In the absence of this hint, to assess the space complexity of an algorithm expressed using WeakMaps, we may assume that the value is not made reachable any longer than that[1]. With the hint, we may only assume that the value does not outlive the key. It may very well outlive the WeakMap. This allows an implementation to use exactly the same implementation techniques internally for these wimply weak maps that we expect them to use for private symbols.
[1] if not otherwise reachable of course
Ok, this paragraph was so dense I didn't realize it was the proposal...
Let me see if I can interpret. For situations like branding where the "has" method is really all we care about you would implement the weak map as an inverted data structure. The WM itself would have no internal state. It is primarily an identify holder that the WM method can be invoked upon. Instead, /every/ object that might to stored in such a WM must have as part of its internal state a set whose member are the WM objects that logically hold that object as a key. The WM "set" and "has" method are implemented as: set(key, dontCareAboutValue) { $internalWMSet(key).set(this)} has(key) {return $internalWMSet(key).has(this)}
where $internalWMSet(obj) is an internal method that provides access to an encapsulated implementation level set object.
So, yes, this casts this use case of WeakMap into a mechanism that is conceptually similar to how private Symbols would be used for branding. But, at the expense of a lot more low level complexity in every object. Basically every object must carry as part of its state an internalWMSet (or the ability to dynamically acquire one when it is needed). An implementation might be able to use the normal per object property store for the internalWMSet but to do so it would have to internally allow WMs to be used as property keys and then to filter them out from any normal property access or reflection operations.
Of course, ephemeron-based WM would still have to exist to deal with situations where circular dependencies between keys and values are possible.
At the implementation object model level this all seems much more complex than having non-reflected private Symbols. If we actually want implementations to do this, we would almost surely have to explicitly specify such as I have a hard time believe that any implementor would on their own decide to do this.
Conceptually, this scheme allows people to exclusively reason about their abstractions in terms of Weakmaps. However, I don't see that as any sort of advantage. Per-property WeakMap side tables could be used to represent all object properties. But who wants to think about object instances as a bunch of tables joined by a common key.
So, for me, this further demonstrated that the combination of WeakMaps to handle circular registry dependencies and private Symbols to handle high integrity per instance state is probably exactly the right combination of features to be providing.
Regardless, the original request for information about the GC impact of using WeakMaps was presumably based upon the accepted WeakMap semantics.
Are space reclamation expectations semantic? We never decided to make them normative. If we do, then yes. But the hint still has no semantics beyond their space reclamation semantics. This arguably make the hint simpler than private symbols.
A note to avoid misunderstanding:
I have been very careful in this thread not to advocate that we drop private symbols. But if we keep them, it should be on grounds other than performance. I am arguing here only that the performance argument is a red herring.
I believe that pragmatic performance implications are a fine thing to factor into our design decisions. We are unlike to specify specification space reclamation requirements but that doesn't mean we don't care or don't expect implementors to make their best effort to be space efficient. In particular we shouldn't specify things that we don't have a reasonable expectation that in can be implemented reasonably efficiently by a typical ES implementor.
I think it is fine to argue that no reasonable ES implementor is likely to spontaneously decide to implement WMs in the manner you describe above. If we want that behavior we will have to specify it, and that isn't simpler than private symbols.
I think you now understand my proposal. By "simple", I meant for users, not implementors. I have no other disagreement with your conclusion.
Thank you Mark, for the GC hint proposal: it matches what was my (admittedly naive) intuition. And thanks, Allen, for the thorough description of GC issues. Very helpful!
Some points, in no particular order:
-
It seems likely to me that the implementation complexity of {private symbols, weakmaps} will be on par with the implementation complexity of {weakmaps with hints}.
-
Reflectivity of all property names is a useful feature that should not be discarded lightly.
-
Simplicity in the object model is an advantage for everyone attempting to reason about objects.
-
Having one kind of symbol is easier to conceptualize than two. Indeed, private symbols could be an attractive nuisance if they lead developers toward over-securing their abstractions.
-
Unique symbols provide sufficient encapsulation for software-engineering purposes.
-
It is my conjecture that high-integrity privacy will be required by a small fraction of projects (such as SES sandboxing engines), and the developer audience for these features will be "power-users". In this case, private symbols might be an over-optimization of a corner case (although it's an important corner!).
-
It looks to me like much of the work to make ES6 future compatible with private symbols has already been done.
-
During the ES7 design process, there will be plenty of user experience with weakmaps and proxies to draw from. This should make the private symbol decision much clearer.
Well stated, respect.
I think you are underweighting the utility of private symbols. The counter-argument I want to make is: what if high-integrity privacy (via weakmaps as you propose) as a popular practice would take off, but with weakmaps there's no optimization (PICs, TI) as there is for property access in top engines today, so developers avoid privacy via weakmaps.
They might use unique symbols. Those paths probably optimize for free as with string-keyed properties. But what if they just keep on truckin' with public string-keyed properties?
This is a weak argument, I admit. It may be that adoption of any kind of privacy, weak or strong, is going to be slow and soft, especially without syntax (a la TypeScript, which has no-integrity privacy!).
Not throwing in the towel, just wanted to respond with the best counter-argument I can make.
To compare the various scenarios, this is what (I believe) it looks like for SES. In ES5, SES currently censors gOPN I believe, in order to implement the equivalent of private keys and/or WeakMap. Given an ES6 runtime environment that only supported unique symbols, it would have to censor getOwnKeys and filter symbols if they were present in a blacklist Set. Given an ES6 runtime environment with private symbols, SES wouldn't have to censor either.
Below is a slight variation of the closure "hack" that allows using private properties through this. The idea is to use the public 'this' as prototype for the private 'this', using 'bind' to give instance methods access to the private 'this' Bound methods. Smart! I come to wonder why you even need the Object.create at all.
I need both a private 'this' (for the methods to use) and a public 'this' (for the constructor to return, and for the public/clients to use), and I need them to be related.
Using Object.create allows to put the private fields before the public fields in the prototype chain (the public 'this' is the prototype of the private 'this'), so instance methods can access both while the public can only access the public 'this'.
Claus
On Wed, Jan 16, 2013 at 11:47 AM, Brandon Benvie <brandon at brandonbenvie.com> wrote:
To compare the various scenarios, this is what (I believe) it looks like for SES. In ES5, SES currently censors gOPN I believe, in order to implement the equivalent of private keys and/or WeakMap.
SES includes all API defined in std ES5 including gOPN. On platforms without built-in WeakMaps, SES does virtualize gOPN as part of its emulation of WeakMaps, but that should be completely invisible to the SES user. SES currently only seeks to be like ES5+WeakMaps. It does not currently attempt to emulate private or unique symbols.
Indeed, I used SES as a kind of template for this theoretical future framework faced with the wcenarios. It was a response to the idea that private symbols are primarily valuable for a SES type use case where security is paramount, versus regular encapsulation which is handled adequately by unique symbols.
My position on private symbols.
My position on classes is and has always been that classes are worth introducing into the language only if they give us, or can be used with, an affordable means for true object encapsulation. Assuming Allen is right about what actual implementors will do (which I find plausible) then WeakMaps are not that means. Given other discussions, I am confident that the objects-as-closures pattern will not become efficient either -- it is likely to continue to cost an allocation per method per instance as its semantics naively suggests. So of the options practically on the table, I think private symbols are the only workable choice. If this is correct, then I consider private symbols to be a requirement. Am I missing anything?
Mark S. Miller wrote:
My position on private symbols.
My position on classes is and has always been that classes are worth introducing into the languageonly if they give us, or can be used with, an affordable means for true object encapsulation. Assuming Allen is right about what actual implementors will do (which I find plausible) then WeakMaps are not that means. Given other discussions, I am confident that the objects-as-closures pattern will not become efficient either -- it is likely to continue to cost an allocation per method per instance as its semantics naively suggests. So of the options practically on the table, I think private symbols are the only workable choice. If this is correct, then I consider private symbols to be a requirement. Am I missing anything?
I agree, but David Bruant is arguing for weakmap-with-hint still, and Kevin is arguing (my summary) "YAGNI" to high-integrity privacy, not exclusively but for most developers -- and for the SES minority, weakmaps are enough.
On Wed, Jan 16, 2013 at 3:48 PM, Brendan Eich <brendan at mozilla.com> wrote:
Mark S. Miller wrote:
My position on private symbols.
My position on classes is and has always been that classes are worth introducing into the languageonly if they give us, or can be used
with, an affordable means for true object encapsulation. Assuming Allen is right about what actual implementors will do (which I find plausible) then WeakMaps are not that means. Given other discussions, I am confident that the objects-as-closures pattern will not become efficient either -- it is likely to continue to cost an allocation per method per instance as its semantics naively suggests. So of the options practically on the table, I think private symbols are the only workable choice. If this is correct, then I consider private symbols to be a requirement. Am I missing anything?
I agree, but David Bruant is arguing for weakmap-with-hint still, and Kevin is arguing (my summary) "YAGNI" to high-integrity privacy, not exclusively but for most developers -- and for the SES minority, weakmaps are enough.
I think Kevin's argument is pretty strong. SES is definitely not everyone, and unique is pretty powerful, especially if you make the slot non-enumerable. The when in doubt... argument has been made. ES6 is pretty stuffed as it is. Thinking about ES7, though, I thought of a possibly interesting place that might be more fitting for private style slots.
I've thought about the tradmark( strawman:trademarks) proposal a lot, because I keep hearing the notion of a brand being used as a reason for a private symbol. Perhaps branding could be implemented with private symbols... or perhaps high-integrity privacy could be related with brand. A brand could open the door to ADT/typeclass/protocol style abstractions, and perhaps that is a better location for privacy.
On Jan 16, 2013, at 12:57 PM, Russell Leggett wrote:
On Wed, Jan 16, 2013 at 3:48 PM, Brendan Eich <brendan at mozilla.com> wrote: Mark S. Miller wrote: My position on private symbols.
My position on classes is and has always been that classes are worth introducing into the languageonly if they give us, or can be used
with, an affordable means for true object encapsulation. Assuming Allen is right about what actual implementors will do (which I find plausible) then WeakMaps are not that means. Given other discussions, I am confident that the objects-as-closures pattern will not become efficient either -- it is likely to continue to cost an allocation per method per instance as its semantics naively suggests. So of the options practically on the table, I think private symbols are the only workable choice. If this is correct, then I consider private symbols to be a requirement. Am I missing anything?
I agree, but David Bruant is arguing for weakmap-with-hint still, and Kevin is arguing (my summary) "YAGNI" to high-integrity privacy, not exclusively but for most developers -- and for the SES minority, weakmaps are enough.
I think Kevin's argument is pretty strong. SES is definitely not everyone, and unique is pretty powerful, especially if you make the slot non-enumerable. The when in doubt... argument has been made. ES6 is pretty stuffed as it is. Thinking about ES7, though, I thought of a possibly interesting place that might be more fitting for private style slots.
I've thought about the tradmark(strawman:trademarks) proposal a lot, because I keep hearing the notion of a brand being used as a reason for a private symbol. Perhaps branding could be implemented with private symbols... or perhaps high-integrity privacy could be related with brand. A brand could open the door to ADT/typeclass/protocol style abstractions, and perhaps that is a better location for privacy.
I normally a behaviorist and don't care whether the contract compliant object you pass me is one that I certifiably created or one you created. However, in some situations being able to identify "trusted" objects is an absolute requirement and in even more situations it is a perceived requirement. Being able to identify such trusted object, is what I call branding. The poster child that I hold up for this isn't SES (although it apparently does requiring branding) but the DOM. The DOM designers really believe that most DOM objects need to be branded in this manner.
An explicit goal of ES6 is to support self hosting of important native libraries including the DOM. Self hosting libraries will be practical only if the self hosted implementation can approach native performance and that includes any overhead for branding, where it is required. In my previous message I explained why we shouldn't expect WeakMap based branding, in practice, to perform competitively with private Symbol based branding. If we remove private Symbols from ES6 and assume that WeakSymbols aren't a perf competitive solution then the only branding solution (for use cases like the DOM) that I see remaining is something based upon using Proxies. But that would mean that essentially every DOM object would have to be represented using a Proxy but proxies themselves impose performance issues and currently (using private Symbol branding) most DOM objects don't need to be proxies.
Private Symbols actually have very little impact upon the core ES specification and presumably implementations. The only place they introduce any real complications is in the semantics of proxies. And even there, the complications are manageable. In fact, if I really wanted to conceptually simplify ES6, I would look at eliminating or simplifying Proxy before I would look to eliminate private Symbols.
Private Symbols are a good pragmatic engineering solution to a very real problem. We really should move forward with them. I don't think we can wait for ES7 .
Le 16/01/2013 21:48, Brendan Eich a écrit :
Mark S. Miller wrote:
My position on private symbols.
My position on classes is and has always been that classes are worth introducing into the languageonly if they give us, or can be used with, an affordable means for true object encapsulation. Assuming Allen is right about what actual implementors will do (which I find plausible) then WeakMaps are not that means. Given other discussions, I am confident that the objects-as-closures pattern will not become efficient either -- it is likely to continue to cost an allocation per method per instance as its semantics naively suggests. So of the options practically on the table, I think private symbols are the only workable choice. If this is correct, then I consider private symbols to be a requirement. Am I missing anything?
I agree, but David Bruant is arguing for weakmap-with-hint still
Let's say I'm "exploring" rather than "arguing for"
and Kevin is arguing (my summary) "YAGNI" to high-integrity privacy, not exclusively but for most developers
The current practice of everything-public in JS, in my opinion, is derived from the low-energy path developers take when they write JS code. If the natural thing to use is a high integrity tool, they'll use it and get all the benefits from it without even thinking about it. That's what happens with "private" in Java (I don't think most Java devs think that the Reflect API makes their private attributes actually not private). Java devs could make everything public, they choose not to because the cost of putting "public" is the same as the cost of putting "private" and the benefits are known. In ES5, high integrity privacy has a cost both in code readability and runtime, that's why people don't use it. It's not that they don't need it, it's just that getting to high integrity is annoying.
Most developers didn't think they needed JSON.parse so they didn't bother downloading json2.js and used eval (4 characters, very low-energy) to create objects out of XHR-retrieved strings. We know the outcome. Most developers (most human beings?) do not know what they need. They have a job to accomplish, a set of tools they're aware of and they work to find the easiest way ("low-energy") to get to the goals using the tools they have. The easy way is to attach information to an object is to use properties? That's what people use, end of story, no strong reflection on what they "should" be using or what they "need". The easy way to add a getter is to use defineGetter? That's what people use in Node.js. Fortunately, it has no consequence on security. A language like Rust fascinates me primarily because the easiest way to write something in Rust is to write safe code (it's actually the only way), while the easiest way to write C code...
I think the "most developers" argument is upside down. Most developers strive for "easy" with few thoughts on the exact properties they need. They want to have their website working, that's all most developers care about. If easy is secure by default, they get security for free, just doing their job. If easy isn't secure by default, they're in a better or worse eval(jsonString) situation. ES6 needs to change the language to make high-integrity the thing most developers will naturally use by default in their low-energy path. I think this relates to Mark's above argument about the condition on whether classes are worth it.
Every project of a decent size needs high-integrity privacy. This shouldn't have a barrier to entry. It should just be what people do by default. Security shouldn't be considered as a luxury left to experts. It should be what naturally comes from writing our code.
Take a microwave. It stops when you open the door. This is because human beings do open the door when the time isn't up yet. Microwaves have just been designed to be safe when human beings naturally interact with them. I shouldn't have to learn how to use my microwave safely. I may need to learn how to take care of it in the long term (clean it up every once in a while...), but I'm in right to expect that the microwave is designed so that how I naturally interact with it is safe. I live on the second floor. On my balcony is a barrier. When I go on my balcony, gravity says the low-energy path is to fall. With the barrier, I have to make quite an effort to fall; my balcony has been designed so that the way I naturally interact with it is safe. In my opinion, ES6 should be a microwave or a balcony and not expect people to be experts and use convoluted patterns to be used safely.
It's worth noting that private Symbols are used by the ES6 spec to solve a number of other problems currently. They are used to provide easy to use hooks for overriding builtin functionality: @@hasInstance, @@create, @@toStringTag, @@iterator, and @@ToPrimitive. Some of these (@@toStringTag, @@iterator) likely make sense as unique symbols, but the others protect access to powerful capabilities that should likely be protected. Without private symbols, it would be difficult to expose these APIs in the easily hookable manner they are provided now. Using prototypal inheritance along with private symbols to protect access is exactly how these hooks are made easy to use.
On Jan 16, 2013, at 3:20 PM, Brandon Benvie wrote:
It's worth noting that private Symbols are used by the ES6 spec to solve a number of other problems currently. They are used to provide easy to use hooks for overriding builtin functionality: @@hasInstance, @@create, @@toStringTag, @@iterator, and @@ToPrimitive. Some of these (@@toStringTag, @@iterator) likely make sense as unique symbols, but the others protect access to powerful capabilities that should likely be protected. Without private symbols, it would be difficult to expose these APIs in the easily hookable manner they are provided now. Using prototypal inheritance along with private symbols to protect access is exactly how these hooks are made easy to use.
Let's refer to all of these built-in @@symbols as language extensions points.
Upon my first reading of the above, I wasn't sure I agreed with Brandon. After all, language extension point symbols all need to be publicly available (presumably via module imports) if certain forms of extended abstractions are going to be defined by ES code. From that perspective regular non-private Symbols should be fine for representing language extension points. (BTW, note I avoid saying "unique symbol" as that isn't current part of the ES6 vocabulary. We have Symbols, some of which are private. But all of them (including private Symbols) are unique values.)
However, here is why it might make sense to make some of language extension points private: If you are creating a sandboxed environment where you would like to limit the availability of some of the language extension points and hence the ability to define new abstractions with the extended functionality. A sandbox's module loader can presumably restrict access to the modules that would export some or all of the language extension points. However, if the language extension points are represented using regular, non-private Symbols then it still might be possible to discover the language extension points via reflection and then use them to circumvent the sandbox restrictions. If they are represented as private Symbols this would not be possible.
So that is a theoretical reason why we might want language extension points to be private Symbols. But, in practice, it isn't clear to me why you would want to sandbox any of the currently identified language extension points in this manner. They all have utility in defining application level abstractions. That's why they're there. If you take them away you have a less powerful language. It also isn't clear what potential hard is exposed by them.
Does anyone have particular language extension points that they think need to be restricted in this manner?
That's exactly how I was thinking of it. @@hasInstance and @@create live on Function.prototype so the only protection a sandbox has from letting any and all comers muck with them is in them not being reflected.
Overriding Function.prototype.@@create (and the custom ones e.g.
Boolean.@@create)
would allow you to touch every single object that is created before any
other code as access to it. Overriding Function.prototype.@@hasInstance
would give you access to every object used on the LHS of instanceof and
every function on the RHS of instanceof (this inside @@hasInstance).
These seem like things that SES would absolutely want access prevented to.
The default functionality itself may be mundane, but the exposure of the
operands is itself the power those functions have.
On Jan 16, 2013, at 4:03 PM, Brandon Benvie wrote:
That's exactly how I was thinking of it. @@hasInstance and @@create live on Function.prototype so the only protection a sandbox has from letting any and all comers muck with them is in them not being reflected.
Overriding Function.prototype.@@create (and the custom ones e.g. Boolean.@@create) would allow you to touch every single object that is created before any other code as access to it. Overriding Function.prototype.@@hasInstance would give you access to every object used on the LHS of instanceof and every function on the RHS of instanceof (
thisinside @@hasInstance). These seem like things that SES would absolutely want access prevented to. The default functionality itself may be mundane, but the exposure of the operands is itself the power those functions have.
For this reason, I've been specifying all of the @@create methods for standard built-ins as non-writable, non-configurable, just like the "prototype" property has always been specified. The rationale, is that just like "prototype" the integrity of @@create is essential to any function that needs to define it.
In this light, it may also make sense to make Function.prototype.@@ and Function.prototype.@@hasInstance non-writable, non-configurable. Regardless of the defaults, SES could presumably defend itself in the same same way.
I don't see any real need to restrict access to all predefined @@create methods. If you invoke it directly (or copy it to another object) about the worse you can do is make an initialized Boolean (in this example) instance which if throw if any of the Boolean methods are applied to it. If you use it to create such an instance and then manually apply the Boolean constructor to initialize it you getting that is basically equivalent to (new Boolean).proto = whatever. It isn't clear that the object you would get is dangerous or anymore so that you would get if you evaluated: new (class extends Boolean)
In this light, it may also make sense to make Function.prototype.@@create and Function.prototype.@@hasInstance non-writable, non-configurable. Regardless of the defaults, SES could presumably defend itself in the same same way.
If you make Function.prototype.@@create non-writable, wouldn't that make it difficult to use?
I'm thinking of the "you can't override non-writable inherited properties with a write operation" ES5 decision, thus forcing you to use Object.defineProperty.
Nathan
That's true, restricting their ability to be modified quells that concern. The only other concern would be to make sure there isn't any extension points left open in the default builtins. For example, if Object.@@create wasn't specified by default (and the algorithm still ended up deferring to it, I'm not sure if it does or not for "plain" objects) then a malicious late-comer could insert their own @@create here and tap into every object literal and tag them or some such thing. As long as all the builtins have this door closed then it's not an issue.
It is difficult to use by its nature, it's the kind of thing that implementors or maybe library authors would make use of. It exposes a hook for what previously was a low level spec operation (the object allocation part of construction).
On Jan 16, 2013, at 4:34 PM, Nathan Wall wrote:
In this light, it may also make sense to make Function.prototype.@@create and Function.prototype.@@hasInstance non-writable, non-configurable. Regardless of the defaults, SES could presumably defend itself in the same same way.
If you make Function.prototype.@@create non-writable, wouldn't that make it difficult to use?
I'm thinking of the "you can't override non-writable inherited properties with a write operation" ES5 decision, thus forcing you to use Object.defineProperty.
No they's still just properties that can be shadowed during property lookup. Subclass need to use defineProperty to define over-riding methods but class declaration will do that for you. More generally definitional rather than assignment operation should be used to define subclass over-rides.
On Jan 16, 2013, at 4:34 PM, Brandon Benvie wrote:
That's true, restricting their ability to be modified quells that concern. The only other concern would be to make sure there isn't any extension points left open in the default builtins. For example, if Object.@@create wasn't specified by default (and the algorithm still ended up deferring to it, I'm not sure if it does or not for "plain" objects) then a malicious late-comer could insert their own @@create here and tap into every object literal and tag them or some such thing. As long as all the builtins have this door closed then it's not an issue.
Or any intentionally open doors are closable by subsystems such as SES.
I believe that all the language level semantics that invoke extension points have fallback that are used if no extension point property is found. Certainly that is my intent. If you find any that aren't that way, let me know.
My position on private symbols.
My position on classes is and has always been that classes are worth introducing into the language only if they give us, or can be used with, an affordable means for true object encapsulation.
Classes want encapsulation, yes. But do they want low-integrity, reflectable encapsulation (a'la Java's private modifier), or high-integrity, security-minded encapsulation (a'la closures)?
If the latter, then why? OO design clearly only needs the former, which is handily provided by unique symbols.
That's what happens with "private" in Java (I don't think most Java devs think that the Reflect API makes their private attributes actually not private). Java devs could make everything public, they choose not to because the cost of putting "public" is the same as the cost of putting "private" and the benefits are known. In ES5, high integrity privacy has a cost both in code readability and runtime, that's why people don't use it. It's not that they don't need it, it's just that getting to high integrity is annoying.
First, let's point out that Java's private members are for the most part reflectable, and as such they are more akin to unique symbols than private symbols. For OO design purposes at least, we don't need high-integrity privacy.
Every project of a decent size needs high-integrity privacy. This shouldn't have a barrier to entry. It should just be what people do by default. Security shouldn't be considered as a luxury left to experts. It should be what naturally comes from writing our code.
I would be willing to admit that every project needs high-integrity privacy, but not without proof. Volumes of code have been written for the Node platform, which is inherently low-integrity. Is the lack of security there a cause for concern among developers?
Let's take your microwave analogy. There is a clearly defined danger involved with opening the door while the thing is still running: you'll end up blasting your face with radiation. Similarly, we need to clearly define the danger of low-integrity abstractions before we can talk about changing the object model to avoid those dangers.
An explicit goal of ES6 is to support self hosting of important native libraries including the DOM. Self hosting libraries will be practical only if the self hosted implementation can approach native performance and that includes any overhead for branding, where it is required. In my previous message I explained why we shouldn't expect WeakMap based branding, in practice, to perform competitively with private Symbol based branding. If we remove private Symbols from ES6 and assume that WeakSymbols aren't a perf competitive solution then the only branding solution (for use cases like the DOM) that I see remaining is something based upon using Proxies. But that would mean that essentially every DOM object would have to be represented using a Proxy but proxies themselves impose performance issues and currently (using private Symbol branding) most DOM objects don't need to be proxies.
I would start by saying that providing a framework for self-hosting the DOM with near-native performance is quite a lofty goal for ES6. I don't think that's what the masses are asking for from ES6.
Since this is an argument from performance, I'm going to have to think carefully about it before I respond.
Le 17/01/2013 07:16, Kevin Smith a écrit :
That's what happens with "private" in Java (I don't think most Java devs think that the Reflect API makes their private attributes actually not private). Java devs could make everything public, they choose not to because the cost of putting "public" is the same as the cost of putting "private" and the benefits are known. In ES5, high integrity privacy has a cost both in code readability and runtime, that's why people don't use it. It's not that they don't need it, it's just that getting to high integrity is annoying.First, let's point out that Java's private members are for the most part reflectable, and as such they are more akin to unique symbols than private symbols.
Most Java programs do not use reflection (most of the one's I've read at least), so Java's private is as much akin to unique symbols as it is to private symbols.
For OO design purposes at least, we don't need high-integrity privacy.
I don't think the debate is about what people need. Most developers do not know or think about what they need. Give them only unique symbols, they'll use that. Give them only private symbols, they'll use that. I'm actually willing to bet that most developers won't understand the point of unique symbols, because it's easier to stick to strings to do what unique symbols enable. Actually, that's a good question, what's the benefit of unique symbols over string properties for OO programming? I see the downsides, like you can't use the dot notation or that you systematically need an extra variable to use them. But I don't see why anyone should bother changing their code to use unique symbols.
In my opinion, the only valid use case for unique symbols is collision-free extensions, whether it's for @iterator or libraries extending the platform (Array.prototype & friends), why not also jQuery "plugins". The awful-but-widely-used Express plugin could make a good use of unique symbols, because its "add properties on the req or res object" philosophy really is future-hostile. But in that case too, unique symbols wouldn't be used as objects properties (in the OO sense), they'd be used as collision-free keys on the req and res "maps".
Every project of a decent size needs high-integrity privacy. This shouldn't have a barrier to entry. It should just be what people do by default. Security shouldn't be considered as a luxury left to experts. It should be what naturally comes from writing our code.I would be willing to admit that every project needs high-integrity privacy, but not without proof. Volumes of code have been written for the Node platform, which is inherently low-integrity. Is the lack of security there a cause for concern among developers?
You said something equivalent to: "volumes of code use eval to parse JSON string. Is the lack of security a cause of concern among developers?" The answer is no, it is not a cause of concern and websites we visit daily do that. But that's not a cause of concern until a malicious user finds out about the eval and abuses it. Then the cost of the mistake becomes absurd by comparison to how much it would have taken to fix it. It's not a concern... until it becomes one.
I don't think our security should be relying on the fact that no one will find out about the leaks in our code. I know a startup who did that. Some sort of professional social network. I just had to type script elements in messages and it would execute them locally, send them to the server, distribute them to anyone I was in touch and execute the script in their browser. A carefully crafted script could infect all users I could reach after transitive closure. The lack of security wasn't a concern for these developers apparently, but I don't think their concern is relevant. I prefer to think they're happy I told them about the issue before someone malicious (a competitor who'd want to ruin their reputation?) knew about it and could steal every users work and password (needs a bit of additional social engineering, but not that hard to achieve).
What we agree on is that developers should not have to worry about security. The good thing with private symbols is that they can get their job done with a tool that's put them in a secure position by default, so that they don't need to become experts in security. It's our role as people who are experts in security (... well, or at least care about as far as I'm concerned) to put the secure easy-to-use tools in the hands of people so that they keep not worrying about security.
Let's take your microwave analogy. There is a clearly defined danger involved with opening the door while the thing is still running: you'll end up blasting your face with radiation. Similarly, we need to clearly define the danger of low-integrity abstractions before we can talk about changing the object model to avoid those dangers.
These dangers are known. Low-integrity abstraction goes against POLA (Principle Of Least Authority). That's the root danger. Now, maybe no one will abuse the extra authority granted by the language... maybe someone will. Maybe you'll use the low-integrity features in ways it can't be abused, maybe someone else won't and put themselves in danger. Maybe no one will abuse the string concatenation you do to craft SQL queries. Maybe someone will. The danger of running with a knife in my hand is low, but it's bigger than running without a knife, so, by default, I choose to run without a knife in my hands. I don't actually actively think about and the low-danger items I could run around with. I just run, because that's what I want to do. Let's have a by-default knife-free JavaScript. If people want to start cutting things, they'll find out where the knifes are.
2013/1/16 Allen Wirfs-Brock <allen at wirfs-brock.com>
Private Symbols actually have very little impact upon the core ES specification and presumably implementations. The only place they introduce any real complications is in the semantics of proxies. And even there, the complications are manageable.
Let's not forget the impact on gOPN (censoring of private symbols) (and if the spec hadn't been refactored so that other internal algorithms depended upon gOPN, they too would have required changes). That said, I agree the complexity remains manageable overall, but it's added complexity nonetheless.
On 15 January 2013 17:16, Kevin Smith <khs4473 at gmail.com> wrote:
It's really none of your business when you try to freeze my object whether any of
(a) pre-existing private-symbol-named properties remain writable; (b) weakmap-encoded private state remains writable; (c) objects-as-closures environment variables remain writable.
Really. Not. Your. Business!
But that's a change from the current object model and by (a) you're assuming the conclusion. ES has a really simple object model, as explained in the first part of the ES5 specification. That simplicity is an advantage. If you're going add complexity, then it should be justified with application-focused use cases. That request does not seem like a stretch to me.
Just to throw in one more opinion, I sympathise with Kevin to some extent. Despite Sam's argument, I think there is considerable complexity imposed by private names, and it has been increasingly unclear to me that it is really warranted at this point. It might be worth reconsidering and/or post-poning, and Kevin made a few good arguments to that end later down the thread.
(However, I don't follow your description of the ES5 object model being "really simple". With sufficient squinting, you may call the ES3 model (relatively) simple, but ES5 certainly put an end to that. Now, ES6 is adding several whole new dimensions of complexity, all of which dwarf private symbols. Let alone a potential Object.observe in ES7. In fact, there are very few languages that have an object model more complicated than that. ;) )
I have suggested before that it would be good to put control over object iteration into the hands of the object authors, by enabling them to override the slot iteration method.
I might be missing something, but isn't this basically covered with the enumerable flag?
There are several object iteration protocols, of which enumerable implicitly specifies one, and this one is (a) somewhat out of favor as the default and (b) does not cover private slots.
One could think about a 'freezable' flag, to allow private slots to participate in the iteration behind .freeze.
Claus
It's really none of your business when you try to freeze my object whether any of
(a) pre-existing private-symbol-named properties remain writable; (b) weakmap-encoded private state remains writable; (c) objects-as-closures environment variables remain writable.
Really. Not. Your (user). Business!
But it is Your (author) business, and turning everything into a proxy to get control over how the authored objects behave seems a little excessive.
What have proxies to do with any of a-c? I don't follow.
I was assuming that proxies would be able to intercept freeze and implement matching behavior for private slots.
It has been pointed out that the issue is one of implicitly called iterators: standard methods for freezing or cloning call iterators that only handle public API.
I think you're missing the point. Object.freeze deals in certain observables. It does not freeze closure state, or weakmap-encoded state, or (per the ES6 proposal) private-symbol-named property state where such properties already exist. Those are not directly observable by the reflection API.
True, private symbols as first-class objects can hide anywhere.
I was thinking in terms of private slots as limited to the instance (hoping to translate existing private symbol property names to fresh private symbols, thereby supporting mixins without exposing the existing private symbols), but even if that was the case, one would quickly end up with the complexity of a deep-cloning operation, which could only be provided as a primitive/built-in.
For the special case of freeze, perhaps a 'freezable' attribute is all that is needed to include private slots in the freeze iteration, without exposing them.
Your -- as in You the abstraction implementor -- may indeed make such state observable.
But if I do not have a way to hook into freeze, etc, how do I make my objects with hidden state behave like objects with exposed state?
Claus
Claus Reinke wrote:
Object.freeze deals in certain observables. It does not freeze closure state, or weakmap-encoded state, or (per the ES6 proposal) private-symbol-named property state where such properties already exist. Those are not directly observable by the reflection API.
True, private symbols as first-class objects can hide anywhere.
...
Your -- as in You the abstraction implementor -- may indeed make such state observable.
But if I do not have a way to hook into freeze, etc, how do I make my objects with hidden state behave like objects with exposed state?
You don't, for a non-proxy. For a proxy, you can do what you like via the whitelist and unknownPrivateSymbol (or better, if there's a better way to handle whitelist misses).
On Wed, Jan 16, 2013 at 3:53 PM, Allen Wirfs-Brock <allen at wirfs-brock.com>wrote:
On Jan 16, 2013, at 3:20 PM, Brandon Benvie wrote:
It's worth noting that private Symbols are used by the ES6 spec to solve a number of other problems currently. They are used to provide easy to use hooks for overriding builtin functionality: @@hasInstance, @@create, @@toStringTag, @@iterator, and @@ToPrimitive. Some of these (@@toStringTag, @@iterator) likely make sense as unique symbols, but the others protect access to powerful capabilities that should likely be protected. Without private symbols, it would be difficult to expose these APIs in the easily hookable manner they are provided now. Using prototypal inheritance along with private symbols to protect access is exactly how these hooks are made easy to use.
Let's refer to all of these built-in @@symbols as language extensions points.
Upon my first reading of the above, I wasn't sure I agreed with Brandon. After all, language extension point symbols all need to be publicly available (presumably via module imports) if certain forms of extended abstractions are going to be defined by ES code. From that perspective regular non-private Symbols should be fine for representing language extension points. (BTW, note I avoid saying "unique symbol" as that isn't current part of the ES6 vocabulary. We have Symbols, some of which are private. But all of them (including private Symbols) are unique values.)
However, here is why it might make sense to make some of language extension points private: If you are creating a sandboxed environment where you would like to limit the availability of some of the language extension points and hence the ability to define new abstractions with the extended functionality. A sandbox's module loader can presumably restrict access to the modules that would export some or all of the language extension points. However, if the language extension points are represented using regular, non-private Symbols then it still might be possible to discover the language extension points via reflection and then use them to circumvent the sandbox restrictions. If they are represented as private Symbols this would not be possible.
So that is a theoretical reason why we might want language extension points to be private Symbols. But, in practice, it isn't clear to me why you would want to sandbox any of the currently identified language extension points in this manner. They all have utility in defining application level abstractions. That's why they're there. If you take them away you have a less powerful language. It also isn't clear what potential hard is exposed by them.
Does anyone have particular language extension points that they think need to be restricted in this manner?
Hi Allen, do you mean the symbols in table 10:
@@create @@hasInstance @@iterator @@ToPrimitive @@toStringTag
?
I see no reason for any of these to be private. Are there additional language extension points I should look at?
When a property is added to an object using a private symbol, it creates a new kind of slot: a private slot. It is clear from the design of the Proxy API that this represents a fundamentally new extension to the ES object model.
On the very first paragraph of the CoffeeScript homepage, we read:
It seems to me that any changes or extensions to the ES object model should be justified by well-documented application-focused use cases, clearly illustrating the need for such a change.
I would personally like to see answers to the following questions:
Do private slots enable applications that would otherwise be impossible?
What are some examples of real-world applications where the runtime security of private slots is necessary?
In those applications, are there acceptable methods of achieving the same results which do not rely on extending the object model?
If GC performance is a significant motivator, then can this supposed benefit be quantified?
When a property is added to an object using a private symbol, it creates a new kind of slot: a private slot. It is clear from the design of the Proxy API that this represents a fundamentally new extension to the ES object model. On the very first paragraph of the CoffeeScript homepage, we read: > Underneath all those awkward braces and semicolons, > JavaScript has always had a gorgeous object model at > its heart. It seems to me that any changes or extensions to the ES object model should be justified by well-documented application-focused use cases, clearly illustrating the need for such a change. I would personally like to see answers to the following questions: - Do private slots enable applications that would otherwise be impossible? - What are some examples of real-world applications where the runtime security of private slots is necessary? - In those applications, are there acceptable methods of achieving the same results which do not rely on extending the object model? - If GC performance is a significant motivator, then can this supposed benefit be quantified? { Kevin } -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20130114/1bc91246/attachment.html>