WeakMap better than Private Symbols? (was: direct_proxies "problem")
Nathan Wall wrote:
// ES6 Symbols // Forgive me for not knowing what the current // correct syntax is for creating a symbol: let timestamp = new Symbol(); class SimpleDate { construct(time) { this.setTime(time); } setTime(time) { this[timestamp] = +time; } getTime() { return this[timestamp]; } } // ES6 WeakMap let timeMap = new WeakMap(), // Store WeakMap methods to maintain integrity of the internal state. WeakMapGet = Function.prototype.call.bind(WeakMap.prototype.get), WeakMapSet = Function.prototype.call.bind(WeakMap.prototype.set); class SimpleDate { construct(time) { this.setTime(time); } setTime(time) { WeakMapSet(timeMap, this, +time); } getTime() { return WeakMapGet(timeMap, this); } }AFAIK the two ES6 implementations above should function the same, except when (1) a Proxy uses a SimpleDate as the target and (2) a SimpleDate is frozen.
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap.In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think).
IIRC, private symbols are exempt from freezing.
Le 10/01/2013 17:22, Nathan Wall a écrit :
Choosing symbols or a weakmap would make a huge difference in how proxy replacement would react to DOM algorithms. If the DOM in terms of accessing private properties, then proxies can replace DOM objects transparently. Their "unknownPrivateSymbol" trap will be called [2] and if they don't throw, the access to the private property will be transparently forwarded to the target without the private symbol ever leaking (it actually wouldn't need to exist in implementations). Private Symbols are great for their syntactic advantages, but they're also more observable than WeakMaps, and can't be used as a replacement to closures for representing internal state. As a library author, I find this unhelpful. Consider the following SimpleDate implementations:
(code)
AFAIK the two ES6 implementations above should function the same, except when (1) a Proxy uses a SimpleDate as the target and (2) a SimpleDate is frozen.
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap.
The trap does not expose the symbol. That would be a bad leak. The third "whitelist" argument in the proxy constructor is here to communicate whether the proxy knows the privatesymbol or not. When [[Get]] or [[Set]] (or other property operations) happens with a whitelisted private symbol, the get or set (or other) trap is called with the symbol itself (since it's known). If the name isn't in the whitelist, the unknownPrivateSymbol trap is called with target as the only trap argument (the private name is not exposed and that's on purpose!)
In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think).
True. Note that it's not a new thing. It's also an issue with string properties. In that regard, symbols are only consistent with what currently exists.
It sounds to me like if I am writing defensive library code, I will always want to use a WeakMap instead of a Symbol.
You can also wrap all the objects you expose in a proxy that doesn't allow being frozen (throw in the preventExtensions/seal/freeze traps). Or make your properties non-configurable accessors. A bit drastic and costly, but doable if defensiveness matters over performance.
In both (1) and (2), I would want my object to behave the way the WeakMap implementation behaves, not the Symbol way. Note that, though there are other deviations the ES5 implementation makes, in both (1) and (2) the ES5 one behaves the same as the WeakMap one -- the closure hides the internal state from the unknownPrivateSymbol trap and protects it against freezing.
Initially, one of the things I loved about Symbols was that it allowed internal state to be specified without using closures, so that prototypal inheritance can be more fully relied upon. However, due to these considerations, I'm not sure Symbols have much use to me anymore.
Thoughts?
Interaction between proxies and symbols have been carefully considered and that shouldn't be a worry for you. The Proxy API has been designed with preventing private symbols leaks. I don't think there is much that can be done about the freezing problem than the 2 solutions I suggested (which can be reasonable).
I think I've described how private names can be used equivalently to weakmaps. There may be performance issues (though I don't know how weakmaps perfs are at scale against the perf issues that would come with the solutions I've suggested)
Nathan Wall wrote:
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap.
Private symbols do not trap.
In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think).
Nope, we agreed that frozen objects support setting pre-existing (added before the object was made non-extensible) properties named by private symbols.
Le 10/01/2013 17:41, Brendan Eich a écrit :
Nathan Wall wrote:
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap.Private symbols do not trap.
They do trap, but they don't leak.
In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think).Nope, we agreed that frozen objects support setting pre-existing (added before the object was made non-extensible) properties named by private symbols.
I said the opposite in my answe, my mistake. I wasn't aware of that.
At the very least, WeakMaps are potentially usable today (depending on the target audience) while symbols are nowhere in sight. I’ve gotten a lot of mileage out of this function, which as you said provides a similar effective result as symbols (they differ in that the WeakMap version isn’t inherited):
function createStorage(creator){
var store = new WeakMap;
creator || (creator = Object.create.bind(null, null, {}));
return function(obj){
var priv = store.get(obj);
if (!priv) store.set(obj, priv = creator(obj));
return o;
};
}
// usage
var _ = createStorage();
function Ctor(val){
_(this).private = val;
}
It's funny: I've been considering a similar thread, but the opposite. I.e. "private symbols better than weak maps?"
In particular, given the freezing clarification, the use cases for weak maps instead of private symbols seem to reduce to adding new symbols to extension-prevented objects.
Is that the only case, or am I missing something obvious?
They both have their place since there's many uses for both inherited and non-inherited private values.
Allen rightly has pointed out several times on es-discuss that WeakMaps require GC special handling and are costly, compared to private symbols. This is true not only for GC time, but especially for get/set runtime. Properties named by Symbols, when implemented, should have the same polymorphic-inline-caching-or-better fast paths as string-key properties.
Private symbols should be faster. No need for hashing and GC of an object with private symbols is the exact same GC as any other object. Getting the GC behavior of a WeakMap is a lot harder (and thus slower).
...also what Brendan just said.
They both have their place since there's many uses for both inherited and non-inherited private values.
Can we see some demonstrated use-cases for which WeakMaps are insufficient, and private properties are? I continue to hear that private symbols are a great idea without any discussion of the tradeoff: a more complicated (and more difficult to reason about) runtime object model.
On Thu, Jan 10, 2013 at 11:55 AM, Kevin Smith <khs4473 at gmail.com> wrote:
They both have their place since there's many uses for both inherited and
non-inherited private values.
Can we see some demonstrated use-cases for which WeakMaps are insufficient, and private properties are? I continue to hear that private symbols are a great idea without any discussion of the tradeoff: a more complicated (and more difficult to reason about) runtime object model.
Inheritance is one use case. Assume you want a "virtual" "protected" method. With private symbols you set it on the prototypes. For WM you would have to set up the map for every instance and find a strategy for storing the correct function.
From: Brandon Benvie [brandon at brandonbenvie.com] Sent: Thursday, January 10, 2013 11:50
They both have their place since there's many uses for both inherited and non-inherited private values.
Of course, (non-)inheritance is a great point.
Amusingly, using "conventional" OO terminology, it seems private symbols give you something like "protected" and weak maps give you "private".
Symbols (private or public, don't forget the latter) are crucial to ES6 and not going away :-|.
Besides the property access and GC costs of WeakMaps (which are probably fine, but at least inevitable, when you really do need a WeakMap), symbols allow coordination between friendly objects by name, without exposing a map full of names, plural.
Finally, symbols allow property access to be abstracted functionally:
function get(obj, key) { return obj[key]; } function set(obj, key, val) { obj[key] = val; }
No such abstraction with weakmaps.
Different tools in your belt are a good thing. We don't fault the screwdriver for being a poor hammer, or vice versa!
Domenic Denicola wrote:
From: Brandon Benvie [brandon at brandonbenvie.com] Sent: Thursday, January 10, 2013 11:50
They both have their place since there's many uses for both inherited and non-inherited private values.
Of course, (non-)inheritance is a great point.
Amusingly, using "conventional" OO terminology, it seems private symbols give you something like "protected" and weak maps give you "private".
Private symbols can give you true privacy. Just confine that symbol!
The distinction for OO fans that I see is class-private vs. instance-private: a well-confined private symbol for a prototype method, e.g., gives you class-private methods. A WeakMap keyed by instance identity, like the closure pattern for OO in JS, gives you instance-private.
Brendan Eich wrote:
Nathan Wall wrote:
In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think).Nope, we agreed that frozen objects support setting pre-existing (added before the object was made non-extensible) properties named by private symbols.
I wasn't aware, but that significantly improves the case for symbols. I'm glad to hear it! However, it also begs the question of why symbols should behave differently from WeakMaps/closures in the presence of Proxies (via unknownPrivateSymbol trap).
David Bruant wrote:
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap. The trap does not expose the symbol. That would be a bad leak. The third "whitelist" argument in the proxy constructor is here to communicate whether the proxy knows the privatesymbol or not. When [[Get]] or [[Set]] (or other property operations) happens with a whitelisted private symbol, the get or set (or other) trap is called with the symbol itself (since it's known). If the name isn't in the whitelist, the unknownPrivateSymbol trap is called with target as the only trap argument (the private name is not exposed and that's on purpose!)
I'm sorry, I was unclear. I understand that the symbol itself is not exposed, but the fact that some symbol was accessed is. It wold allow a proxy to throw to prevent access to internal state (right?). This is different from what would occur in the WeakMap implementation, and if I'm trying to hide internal state, then I don't want proxies to be able to mess with attempts to access it, so I'll choose to use WeakMaps anytime I want to write defensive code.
Of course, if I'm just writing something quick and dirty, I'll use Symbols because I would much prefer to use the simpler syntax Symbols provide. But again, as a library author, allowing Proxies to interfere here seems like a weaker position for a generic object than using a WeakMap to hide this.
Nathan
Nathan Wall wrote:
Brendan Eich wrote:
Nathan Wall wrote:
In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think). Nope, we agreed that frozen objects support setting pre-existing (added before the object was made non-extensible) properties named by private symbols.I wasn't aware, but that significantly improves the case for symbols. I'm glad to hear it! However, it also begs the question of why symbols should behave differently from WeakMaps/closures in the presence of Proxies (via unknownPrivateSymbol trap).
WeakMaps transpose the terms of property access:
obj[sym] <=> wm[obj]
This seems important! Do you propose, if obj is a proxy, to trap with wm as key for the latter?
David Bruant wrote:
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap. The trap does not expose the symbol. That would be a bad leak. The third "whitelist" argument in the proxy constructor is here to communicate whether the proxy knows the privatesymbol or not. When [[Get]] or [[Set]] (or other property operations) happens with a whitelisted private symbol, the get or set (or other) trap is called with the symbol itself (since it's known). If the name isn't in the whitelist, the unknownPrivateSymbol trap is called with target as the only trap argument (the private name is not exposed and that's on purpose!)I'm sorry, I was unclear. I understand that the symbol itself is not exposed, but the fact that some symbol was accessed is.
No, not if the symbol is not in the whitelist. Zero information leak is required.
As an aside, Irakli Gozashvili and I independently realized that you could use a WeakMap for parallel objects with inheritance. Very similar to Brendan’s createStorage, we put together a parallel universe constructor.
function Parallel(root) { var parallel = new WeakMap(); root = root || null; function get(object) { if (!parallel.has(object)) { parallel.set(object, Object.create(get(Object.getPrototypeOf(object)))); } return parallel.get(object); } return get; }
Irakli’s was a Namespace constructor that appeared somewhere in the Mozilla Add-on toolkit, but the links have gone stale.
Kris Kowal
Brendan Eich:
No, not if the symbol is not in the whitelist. Zero information leak is required.
That's good news too. Objection withdrawn. I guess I didn't correctly understand David's point in the direct_proxies thread then :-/. I'll have to reread it more carefully.
Thanks for the participation and clarification, everyone!
Nathan
On Jan 10, 2013, at 8:41 AM, Brendan Eich wrote:
Nathan Wall wrote:
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap.Private symbols do not trap.
In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think).Nope, we agreed that frozen objects support setting pre-existing (added before the object was made non-extensible) properties named by private symbols.
Yes, indeed.
In addition I'm now thinking we want Object.freeze to notify the object via a @@method it is bering frozen . This gives the object an opportunity to configure its private state and public behavior in a manner that actually presents an immutable abstraction. For example, a collection that holds it elements in a private symbol accessed slot and uses public accessor methods might internally set a flag that makes the collection act immutably.
Brendan Eich:
No, not if the symbol is not in the whitelist. Zero information leak is required.
That's good news too. Objection withdrawn.
Maybe I gave up too easy :). Is the unknownPrivateSymbol trap called? What's the rationale for this trap?
Symbols (private or public, don't forget the latter) are crucial to ES6 and not going away :-|.
No problem here with public symbols! They provide collision-free property naming without changing the underlying object model. Woot!
But when I ask for use cases for private symbols, I get answers that in affect say: because they make it easier to create "private" data and methods. Well, that just begs the question (h.t. @domenic ; )! Why is this strict "privacy" so important for object property access?
I can't for the life of me imagine that it would be a good idea to have two mutually untrusting bodies of code executing in the same environment, without the mediation of some strict sandboxing infrastructure presiding over them, along with strict guidelines limiting their communication. Having mutually untrusting code bases, managing their own security, and passing around objects adorned with private data and methods seems to me to be a recipe for disaster.
Unique symbols provide a sufficient encapsulation mechanism for API design by making it inconvenient to access symbol-named properties without the symbol. But private symbols go beyond this and create a new kind of slot. Is this extension to the object model really necessary? Is it necessary for ES6?
We are conflating property access (an API issue) with security, and the ramifications are bleeding out into the proxy design and into the nature of "freezing". This smells like a separation of concerns issue to me.
WeakMaps address the security use case. Unique symbols address the API use case. This is a clean separation of concerns and is easy to understand.
(Apologies for being a bit of a gadfly on this...)
On Thu, Jan 10, 2013 at 11:22 AM, Nathan Wall <nathan.wall at live.com> wrote:
// ES6 WeakMap let timeMap = new WeakMap(), // Store WeakMap methods to maintain integrity of the internalstate. WeakMapGet = Function.prototype.call.bind(WeakMap.prototype.get), WeakMapSet = Function.prototype.call.bind(WeakMap.prototype.set);
Just a heads up, this is unnecessary and actually won't work at all. WeakMap has nothing defined on its prototype property—all methods are late-bound to the instance for the same reason you mention in the comment. gist.github.com/4505255
Right, that's not how the spec. is written.
You certainly could do that on a per instance basis by copying down the methods. Perhaps even define yourself a subclass to do so. But it would be silly to force this on every instance.
On Thu, Jan 10, 2013 at 3:04 PM, Domenic Denicola < domenic at domenicdenicola.com> wrote:
Really?? people.mozilla.org/~jorendorff/es6-draft.html#sec-15.15.5
Wow, yep. For some reason, when I saw that example, I only recalled the original WeakMap proposal—which is 2 years out of date. VERY sorry for the confusion.
On Thu, Jan 10, 2013 at 3:14 PM, Allen Wirfs-Brock <allen at wirfs-brock.com>wrote:
Right, that's not how the spec. is written.
You certainly could do that on a per instance basis by copying down the methods. Perhaps even define yourself a subclass to do so. But it would be silly to force this on every instance.
Right, see my last response. I should've reviewed the harmony proposal and latest draft before responding. Again, apologies for the confusion and incorrect information.
... It's one of those days ;)
Nathan Wall wrote:
Brendan Eich:
No, not if the symbol is not in the whitelist. Zero information leak is required. That's good news too. Objection withdrawn.
Maybe I gave up too easy :). Is the
unknownPrivateSymboltrap called? What's the rationale for this trap?
I just wrote that the trap is not even called if the symbol is not in the whitelist passed in when the proxy is created.
The rationale was given clearly by Tom Van Cutsem here:
referenced by this post:
Here is a link to a documentatin for the weak map based solution that Kris mentioned:
addons.mozilla.org/en-US/developers/docs/sdk/latest/modules/sdk/core/namespace.html
We find it invaluable and use for storing private field. Inheritance works, but we did not found that feature all that useful in practice, probably because we use that only for data fields
There is also another experiment that I found to be a lot more useful for non data properties like methods gozala/method
I also honestly wish private names would just be it
Private Symbols are great for their syntactic advantages, but they're also more observable than WeakMaps, and can't be used as a replacement to closures for representing internal state. As a library author, I find this unhelpful. Consider the following SimpleDate implementations:
// ES5 function SimpleDate(timestamp) { timestamp = +timestamp;
this.setTime = function setTime(time) { timestamp = +time; }
this.getTime = function getTime() { return timestamp; } }
Trying to rewrite it in ES6 produces these possibilities:
// ES6 Symbols
// Forgive me for not knowing what the current // correct syntax is for creating a symbol: let timestamp = new Symbol();
class SimpleDate {
construct(time) { this.setTime(time); }
setTime(time) { this[timestamp] = +time; }
getTime() { return this[timestamp]; }
}
// ES6 WeakMap
let timeMap = new WeakMap(), // Store WeakMap methods to maintain integrity of the internal state. WeakMapGet = Function.prototype.call.bind(WeakMap.prototype.get), WeakMapSet = Function.prototype.call.bind(WeakMap.prototype.set);
class SimpleDate {
construct(time) { this.setTime(time); }
setTime(time) { WeakMapSet(timeMap, this, +time); }
getTime() { return WeakMapGet(timeMap, this); }
}
AFAIK the two ES6 implementations above should function the same, except when (1) a Proxy uses a SimpleDate as the target and (2) a SimpleDate is frozen.
In the case of (1), the implementation using a private symbol will have internal accesses to the
timestampsymbol exposed through the unknownPrivateSymbol trap.In the case of (2), the implementation using a private symbol will fail on a call to
setTimewhen the object is frozen (I think).It sounds to me like if I am writing defensive library code, I will always want to use a WeakMap instead of a Symbol. In both (1) and (2), I would want my object to behave the way the WeakMap implementation behaves, not the Symbol way. Note that, though there are other deviations the ES5 implementation makes, in both (1) and (2) the ES5 one behaves the same as the WeakMap one -- the closure hides the internal state from the unknownPrivateSymbol trap and protects it against freezing.
Initially, one of the things I loved about Symbols was that it allowed internal state to be specified without using closures, so that prototypal inheritance can be more fully relied upon. However, due to these considerations, I'm not sure Symbols have much use to me anymore.
Thoughts?
> Choosing symbols or a weakmap would make a huge difference in how proxy > replacement would react to DOM algorithms. > If the DOM in terms of accessing private properties, then proxies can > replace DOM objects transparently. Their "unknownPrivateSymbol" trap > will be called [2] and if they don't throw, the access to the private > property will be transparently forwarded to the target without the > private symbol ever leaking (it actually wouldn't need to exist in > implementations). Private Symbols are great for their syntactic advantages, but they're also more observable than WeakMaps, and can't be used as a replacement to closures for representing internal state. As a library author, I find this unhelpful. Consider the following SimpleDate implementations: // ES5 function SimpleDate(timestamp) { timestamp = +timestamp; this.setTime = function setTime(time) { timestamp = +time; } this.getTime = function getTime() { return timestamp; } } Trying to rewrite it in ES6 produces these possibilities: // ES6 Symbols // Forgive me for not knowing what the current // correct syntax is for creating a symbol: let timestamp = new Symbol(); class SimpleDate { construct(time) { this.setTime(time); } setTime(time) { this[timestamp] = +time; } getTime() { return this[timestamp]; } } // ES6 WeakMap let timeMap = new WeakMap(), // Store WeakMap methods to maintain integrity of the internal state. WeakMapGet = Function.prototype.call.bind(WeakMap.prototype.get), WeakMapSet = Function.prototype.call.bind(WeakMap.prototype.set); class SimpleDate { construct(time) { this.setTime(time); } setTime(time) { WeakMapSet(timeMap, this, +time); } getTime() { return WeakMapGet(timeMap, this); } } AFAIK the two ES6 implementations above should function the same, except when (1) a Proxy uses a SimpleDate as the target and (2) a SimpleDate is frozen. In the case of (1), the implementation using a private symbol will have internal accesses to the `timestamp` symbol exposed through the unknownPrivateSymbol trap. In the case of (2), the implementation using a private symbol will fail on a call to `setTime` when the object is frozen (I think). It sounds to me like if I am writing defensive library code, I will always want to use a WeakMap instead of a Symbol. In both (1) and (2), I would want my object to behave the way the WeakMap implementation behaves, not the Symbol way. Note that, though there are other deviations the ES5 implementation makes, in both (1) and (2) the ES5 one behaves the same as the WeakMap one -- the closure hides the internal state from the unknownPrivateSymbol trap and protects it against freezing. Initially, one of the things I loved about Symbols was that it allowed internal state to be specified without using closures, so that prototypal inheritance can be more fully relied upon. However, due to these considerations, I'm not sure Symbols have much use to me anymore. Thoughts?