Questioning WeakMap.prototype.clear
Le 21/01/2013 18:02, Allen Wirfs-Brock a écrit :
If you don't want to expose clear on a WeakMap, create a subclass that doesn't support it:
class WeakMapWithoutClear extends WeakMap { delete() {throw Error("Clearing this map is not allowed"); }
if you are really paranoid that somebody is going to do WeakMap.prototype.clear.call(myWeakMapWithoutClear); then don't expose myWeakMapWithoutClear to untrusted parties or only expose wrapper object that hides the WeakMap as private state.
clear is an operation that can not be otherwise synthesized for WeakMaps. Given the ambient impact of the mere existance of WeakMap entries on garbage collection algorithms, it seems likely that will be performance advantages in some situations to proactively clear a WeakMap rather than waiting for the GC to do so. Having clear enables this while it doesn't prevent using WeakMaps in ways that don't expose that operation to ambient use.
Since the use of .clear isn't mandatory, implementors have an incentive to optimize weakmaps in which clear isn't used. It's possible that 5-10 years from now, implementors realize that .clear provides no performance benefits. If performance is the main reason to introduce, why not wait until performance actually becomes a bottleneck? Something about premature optimizations.
On Jan 21, 2013, at 3:04 AM, David Bruant wrote:
Le 21/01/2013 11:58, David Bruant a écrit :
Before the clear method, weakmaps had the following property: "you can only modify a weakmap entry if you have the key". This property isn't true anymore with .clear. This may be considered as abusive ambient authority. Let's see how this feature came to appear: Jason Orendorff talked about Map.prototype.clear [1] (Oct 22nd). Seen as a good idea. No discussion on whether it's a good idea for WeakMaps specifically. Nicholas Zakas briefly mentions it in November [2]. No one replied to it specifically. I haven't seen any discussion about it in meeting notes [3]. A brief mention of Set/Map.prototype.clear [4] as a review of the Oct 26th draft [5] (note, 4 days after Jason post, which is a very short amount of time) but nothing about WeakMap.prototype.clear. Implemented in Firefox soon after [6]... I think WeakMap.prototype.clear slipped through the crack without being specifically discussed. Based on what's publicly available, I don't see anyone noticed and discussed the fact that WeakMap.prototype.clear questions the property that was true before its adoption ("you can only modify a weakmap entry if you have the key") As much as makes sense we want Map, WeakMap, and Set to expose the same interfaces. Any new method added to one is going to get added to all of them, unless it either simply makes no sense for one of them or if somebody can make a convincing argument for excluding it for one of them.
Maps and WeakMaps don't even have the same signature for the get/set/has/delete methods in which the former accepts non-objects as key while the latter doesn't! Maps and WeakMaps are very different features for different usages. I don't think I've encountered a case where I could switch one for the other. If someone has such cases, can they share them to the list? I guess it's an equivalent debate than the recent one about unique and private symbols. Unique symbols are collision-free property names and avoiding collision is the only good reason to use them, otherwise, strings will do the job fine; private symbols are about encapsulation for which neither strings nor unique symbols are relevant. Different features, for different usages; I don't think it's a sane default to map interfaces whenever possible. I think the default should be to carefully consider if a method in one case is relevant for the other. Where I agree is that similar functionality should have the same name in these sibling interfaces.
I think the property I mentioned is cricial to weakmap integrity and I think WeakMap.prototype.clear should be considered for removal... or at least proper discussion since none really happened from what I've found. TC39 doesn't have a process that requires a priori discussion on es-discuss of every design detailed before it can become part of specification draft.
I don't want discussion on es-discuss to be a requirement; there is no mention of WeakMap.prototype.clear specifically with the issues I raised in meeting notes. I guess TC39 agrees on the "Any new method added to one is going to get added to all of them, unless...". I hope TC39 would reconsider this position given what I said above.
Le 21/01/2013 18:38, Andrea Giammarchi a écrit :
somebody asked why clear() is needed, when if you have the reference you can simply ref = new WeakMap; instead of clearing, while clear() exposes undesired side effects. +1
Said that, even localStorage lets us remove all keys even if it wasn't us setting them so ... well, I guess we can survive then with clear().
are you using localStorage as a model of high integrity API? ;-) Regardless, the problem is different. localStorage methods deal with a persistent remote resource. An empty localStorage without .clear would require a lot of back and forth exchanges between disk and JS. If you want an empty weakmap, you can just create a new one and get rid of the previous one. GC will do the cleanup at a time it finds relevant; maybe even in parallel of your program one day?
This is specially for Rick, since it's holidays today and I am bored home :D Where is exactly the latest state of es6 collections? I have no idea how I should update mine accordingly, please point me to the very latest, most updated, agreed link 'cause I am confused.
Latest draft at harmony:specification_drafts
David Bruant wrote:
I guess TC39 agrees on the "Any new method added to one is going to get added to all of them, unless...".
Why do you "guess" that? I don't agree with it.
On Mon, Jan 21, 2013 at 11:44 AM, David Bruant <bruant.d at gmail.com> wrote:
Le 21/01/2013 17:16, Rick Waldron a écrit :
On Mon, Jan 21, 2013 at 6:04 AM, David Bruant <bruant.d at gmail.com> wrote:
I think WeakMap.prototype.clear slipped through the crack without being specifically discussed. Based on what's publicly available, I don't see anyone noticed and discussed the fact that WeakMap.prototype.clear questions the property that was true before its adoption ("you can only modify a weakmap entry if you have the key")
I agree and disagree. I disagree because WeakMap.prototype.clear() doesn't modify the weakmap entry in the same direct way, ie. WeakMap.prototype.set, WeakMap.prototype.delete. On the other hand, I agree because it provides (if the weakmap is exposed) a way to remove an entry that might be used without any defense, ie. WeakMap.prototype.has() or might leave the program in an unstable state due to loss of some aggregate state information stored in the weakmap.
That being said, I support the inclusion of WeakMap.prototype.clear(), because reasonable security and state defense expectations can be met by:
- Not exposing a sensitive weakmap in program code
- Defending against missing entries with has()
Subjectively, #1 mitigates without nannying.
I thought more about how I use weakmaps and #1 is a thing I do naturally indeed. I don't believe in #2, because calls to .has everytime someone may have maliciously or by mistake called .clear is quite a cost.
Agreed.
I'd like to point out, that just based on what you wrote, you do not support the inclusion of WeakMap.prototype.clear, but rather you support its non-removal based on what I said.
Fair enough!
(I actually laughed a little when I read that)
Use cases for WeakMap.prototype.clear are... unclear. In which conditions would one want to wipe out all entries? If you're the only holder of the weakmap, creating a new one is equivalent to .clear (but allows to lazy GC the entries which sync .clear doesn't as easily).
More specifically, in which conditions would one want to wipe out all the entries they're oblivious to?
This is the reality check I can get behind—I'm hard pressed to come up with a use case that isn't contrived or solvable by some other means.
Le 21/01/2013 20:30, Brendan Eich a écrit :
David Bruant wrote:
I guess TC39 agrees on the "Any new method added to one is going to get added to all of them, unless...".
Why do you "guess" that?
I didn't see anything in the notes about discussions related specifically to WeakMap.prototype.clear, but I saw that Set/Map clear methods had been discussed [1], I assumed if WeakMap.prototype.clear hadn't been discussed, it was because TC39 agreed with the above sentence.
I don't agree with it.
I'm glad I was wrong in assuming so :-)
I see that WeakMap.prototype.clear seems controversial, doesn't seem to have been discussed neither on es-discuss nor at TC39 meetings and yet it's already in the spec and in Firefox 20 as a consequence. I don't mean to slow down the spec process, but discussing before adding in drafts sounds like a reasonable idea in light of recent events.
David
[1] rwldrn/tc39-notes/blob/master/es6/2012-11/nov-27.md#review
On Jan 21, 2013, at 11:36 AM, Rick Waldron wrote:
This is the reality check I can get behind—I'm hard pressed to come up with a use case that isn't contrived or solvable by some other means.
This is easy:
I'm do phased traversals over a complex data structure. I have a number of functions that collaborative perform the function and they share access to a WeakMap to cache relationships that they identify over the course of the traversal. When I start a new traversal phase I want to flush the cache so I use the clear method to do so.
On Mon, Jan 21, 2013 at 2:52 PM, Allen Wirfs-Brock <allen at wirfs-brock.com>wrote:
On Jan 21, 2013, at 11:36 AM, Rick Waldron wrote:
This is the reality check I can get behind—I'm hard pressed to come up with a use case that isn't contrived or solvable by some other means.
This is easy:
I'm do phased traversals over a complex data structure. I have a number of functions that collaborative perform the function and they share access to a WeakMap to cache relationships that they identify over the course of the traversal. When I start a new traversal phase I want to flush the cache so I use the clear method to do so.
This use case is similar to the cases I was also thinking of; when I wrote my response, I had just read through the thread and felt that the user-land implementation, suggested by Andrea and Mark, was sufficient as "solvable by some other means". The impl I'm referring to:
class WeakMapWithClear { private let wrapped; // not const, however we say this for a field constructor() { wrapped = new WeakMap(); } get(key) => wrapped.get(key), set(key, val) => wrapped.set(key, value), has(key) => wrapped.has(key), delete(key) => wrapped.delete(key), clear() => { wrapped = new WeakMap(); } }
Subjectively, I don't like this at all and don't think it's fair to put the burden on user-code.
Le 21/01/2013 20:52, Allen Wirfs-Brock a écrit :
On Jan 21, 2013, at 11:36 AM, Rick Waldron wrote:
This is the reality check I can get behind—I'm hard pressed to come up with a use case that isn't contrived or solvable by some other means.
This is easy:
I'm do phased traversals over a complex data structure. I have a number of functions that collaborative perform the function and they share access to a WeakMap to cache relationships that they identify over the course of the traversal. When I start a new traversal phase I want to flush the cache so I use the clear method to do so.
Creating a new weakmap would work equally well to flush the cache. Note that the current WeakMap.prototype.clear method is specified as: "Set the value of M’s [[WeakMapData]] internal data property to a new empty List." which is quite close from creating a new weakmap. If you care about reusing the same object, Mark's encapsulation [1]+[2] works too.
I intuit perf differences can be made marginal by engines in both cases. If I were wrong on that, implementors or authors will tell. The Firefox precedent shows that it took ~3 weeks from the bug being filed to implementation to add .clear to an existing WeakMap implementation, so I'm not too worried about that either.
I think .clear can wait and people would need to clear can either create new weakmaps or wrap weakmaps as Mark showed.
David
[1] esdiscuss/2013-January/028370 [2] esdiscuss/2013-January/028371
On Jan 21, 2013, at 12:25 PM, David Bruant wrote:
Le 21/01/2013 20:52, Allen Wirfs-Brock a écrit :
On Jan 21, 2013, at 11:36 AM, Rick Waldron wrote:
This is the reality check I can get behind—I'm hard pressed to come up with a use case that isn't contrived or solvable by some other means.
This is easy:
I'm do phased traversals over a complex data structure. I have a number of functions that collaborative perform the function and they share access to a WeakMap to cache relationships that they identify over the course of the traversal. When I start a new traversal phase I want to flush the cache so I use the clear method to do so. Creating a new weakmap would work equally well to flush the cache.
Same arguments applies to Map clear. I'm actually more comfortable with a discussion of the utility of the clear method for maps, in general. But, if it has utility for Map then it has the same utility for WeakMap and supplying on both is a matter of API consistency.
Note that the current WeakMap.prototype.clear method is specified as: "Set the value of M’s [[WeakMapData]] internal data property to a new empty List." which is quite close from creating a new weakmap.
No it is very different. The fact that we are talking about a "List" (ie, not an actual observable object) and an internal data property means that an implementation is able to aggressively manage those resources in ways that is not possible with a vanilla reference to a observable object that may have other references
If you care about reusing the same object, Mark's encapsulation [1]+[2] works too.
I intuit perf differences can be made marginal by engines in both cases. If I were wrong on that, implementors or authors will tell. The Firefox precedent shows that it took ~3 weeks from the bug being filed to implementation to add .clear to an existing WeakMap implementation, so I'm not too worried about that either.
These techniques do not allow for the same implementation level optimizations.
I don't know how many people on this list have implemented production generational GC with ephemeron support. I have. I'm telling been trying to tell you that weak abstractions in general and ephemerons in particular will have a systemic impact upon GC throughput. Also, generational collectors can have large latencies between the time the last reference to an object is destroyed and the when the GC actually notices. Many GC cycles may occur during that period and if a populated but unneeded large WeakMap is one of these zombie object, then it can have perf impacts.
I think .clear can wait and people would need to clear can either create new weakmaps or wrap weakmaps as Mark showed.
For Map too, right?
On Mon, Jan 21, 2013 at 1:42 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Jan 21, 2013, at 12:25 PM, David Bruant wrote:
Le 21/01/2013 20:52, Allen Wirfs-Brock a écrit :
On Jan 21, 2013, at 11:36 AM, Rick Waldron wrote:
This is the reality check I can get behind—I'm hard pressed to come up with a use case that isn't contrived or solvable by some other means.
This is easy:
I'm do phased traversals over a complex data structure. I have a number of functions that collaborative perform the function and they share access to a WeakMap to cache relationships that they identify over the course of the traversal. When I start a new traversal phase I want to flush the cache so I use the clear method to do so. Creating a new weakmap would work equally well to flush the cache.
Same arguments applies to Map clear. I'm actually more comfortable with a discussion of the utility of the clear method for maps, in general. But, if it has utility for Map then it has the same utility for WeakMap and supplying on both is a matter of API consistency.
Note that the current WeakMap.prototype.clear method is specified as: "Set the value of M’s [[WeakMapData]] internal data property to a new empty List." which is quite close from creating a new weakmap.
No it is very different. The fact that we are talking about a "List" (ie, not an actual observable object) and an internal data property means that an implementation is able to aggressively manage those resources in ways that is not possible with a vanilla reference to a observable object that may have other references
If you care about reusing the same object, Mark's encapsulation [1]+[2] works too.
I intuit perf differences can be made marginal by engines in both cases. If I were wrong on that, implementors or authors will tell. The Firefox precedent shows that it took ~3 weeks from the bug being filed to implementation to add .clear to an existing WeakMap implementation, so I'm not too worried about that either.
These techniques do not allow for the same implementation level optimizations.
I don't know how many people on this list have implemented production generational GC with ephemeron support. I have. I'm telling been trying to tell you that weak abstractions in general and ephemerons in particular will have a systemic impact upon GC throughput. Also, generational collectors can have large latencies between the time the last reference to an object is destroyed and the when the GC actually notices. Many GC cycles may occur during that period and if a populated but unneeded large WeakMap is one of these zombie object, then it can have perf impacts.
That's why it's important that common patterns, such as the rights amplification pattern, which don't actually need the costs of ephemeron gc, shouldn't have to pay these costs because of the presence of a clear method they don't use.
On Jan 21, 2013, at 1:55 PM, Mark S. Miller wrote:
...
That's why it's important that common patterns, such as the rights amplification pattern, which don't actually need the costs of ephemeron gc, shouldn't have to pay these costs because of the presence of a clear method they don't use.
So what are you proposing, as an alternative? A third kind of map?
Note that even without the circularity protection provided by ephemerons, an exclusively weak keyed map still has systemic impacts upon GC and have most of the issues I discussed I discussed in esdiscuss/2013-January/028145
The clear method seems like it has very little relevance to that discussion, other than its utility in mitigating weak reference gc processing overhead in situations where it is known the entire contents is no longer needed.
On Mon, Jan 21, 2013 at 6:04 AM, David Bruant <bruant.d at gmail.com> wrote:
[...] WeakMap.prototype.clear questions the property that was true before its adoption ("you can only modify a weakmap entry if you have the key")
David, would you please elaborate your argument for this invariant? This the first I've seen it stated.
An invariant can be a powerful thing. Still, I guess my default position is that (1) the object-capabilities perspective is only one view among many; (2) even looking at things with an eye for o-c integrity and security, clearing a data structure seems like a reasonable thing to allow, treating a reference to the data structure itself as a sufficient capability. It's (2) that I would especially like you to address.
Le 22/01/2013 11:47, Jason Orendorff a écrit :
On Mon, Jan 21, 2013 at 6:04 AM, David Bruant <bruant.d at gmail.com <mailto:bruant.d at gmail.com>> wrote:
[...] WeakMap.prototype.clear questions the property that was true before its adoption ("you can only modify a weakmap entry if you have the key")David, would you please elaborate your argument for this invariant? This the first I've seen it stated.
An invariant can be a powerful thing. Still, I guess my default position is that (1) the object-capabilities perspective is only one view among many; (2) even looking at things with an eye for o-c integrity and security, clearing a data structure seems like a reasonable thing to allow, treating a reference to the data structure itself as a sufficient capability. It's (2) that I would especially like you to address.
I think Rick already suggested your (2), though phrased a bit differently [1] (that was his #1). I answered [2]: "I thought more about how I use weakmaps and [well-encapsulate my weakmaps so that I'm the only holder] is a thing I do naturally indeed." The problem may arise when you start sharing weakmaps around and some use cases require you to [3].
Regarding your (1), I don't doubt the need to clear a data structure since Allen explained a very compelling use case for that [3]. However, Mark showed an elegant way to implement .clear on top of clear-less weakmaps and the class syntax [4][5] (reproducing here the final version for clarity)
// note: implements the WeakMap API but does *not* extend WeakMap.
class WeakMapWithClear {
private let wrapped;
constructor() {
wrapped = new WeakMap();
}
get(key) => wrapped.get(key),
set(key, val) => wrapped.set(key, value),
has(key) => wrapped.has(key),
delete(key) => wrapped.delete(key),
clear() { wrapped = new WeakMap(); }
}
Now, the only thing that can differentiate both the native against this version is performance I think. Allen seems to argue that a native .clear would have better perf characteristics (related to GC). I still fail to see why the difference would be significant (but I need to re-read his recent posts about that). In all likelihood, .clear is a method that is used sporadically. At least, one needs to fill up the weakmap a bit before calling it, so I don't think a marginal perf difference would matter.
As an implementor, what is your feeling about performance characteristics of both the native and the class-based version?
David
[1] esdiscuss/2013-January/028353 [2] esdiscuss/2013-January/028357 [3] esdiscuss/2013-January/028380 [4] esdiscuss/2013-January/028370 [5] esdiscuss/2013-January/028371
On Tue, Jan 22, 2013 at 5:56 AM, David Bruant <bruant.d at gmail.com> wrote:
Le 22/01/2013 11:47, Jason Orendorff a écrit :
On Mon, Jan 21, 2013 at 6:04 AM, David Bruant <bruant.d at gmail.com> wrote:
[...] WeakMap.prototype.clear questions the property that was true before its adoption ("you can only modify a weakmap entry if you have the key")
David, would you please elaborate your argument for this invariant? This the first I've seen it stated.
An invariant can be a powerful thing. Still, I guess my default position is that (1) the object-capabilities perspective is only one view among many; (2) even looking at things with an eye for o-c integrity and security, clearing a data structure seems like a reasonable thing to allow, treating a reference to the data structure itself as a sufficient capability. It's (2) that I would especially like you to address.
I think Rick already suggested your (2), though phrased a bit differently [1] (that was his #1). I answered [2]: "I thought more about how I use weakmaps and [well-encapsulate my weakmaps so that I'm the only holder] is a thing I do naturally indeed." The problem may arise when you start sharing weakmaps around and some use cases require you to [3].
What problem exactly?
Sharing mutable data structures across abstraction (or trust) boundaries is already pretty well understood to be an integrity (or security) risk. It's easy to fix: you expose a read-only view instead.
Also, I don't understand how [3] is a use case for sharing weakmaps around. To me it looks like a use case for clearing a WeakMap.
Thanks for the response.
On Tue, Jan 22, 2013 at 5:56 AM, David Bruant <bruant.d at gmail.com> wrote:
However, Mark showed an elegant way to implement .clear on top of clear-less weakmaps and the class syntax [4][5] (reproducing here the final version for clarity)
Sure—and roughly the same code can be used to remove an unwanted .clear() method:
class WeakMapWithoutClear {
// (this is just all your code except the clear method)
private let wrapped;
constructor() {
wrapped = new WeakMap();
}
get(key) => wrapped.get(key),
set(key, val) => wrapped.set(key, value),
has(key) => wrapped.has(key),
delete(key) => wrapped.delete(key),
}
So the workaround is about equally elegant either way. (And just for the sake of kidding you a little: the Without hack is more general and seems like it ought to be pretty familiar to an o-c hacker!)
Then there are two issues to address. We already have lots of things named "1" and "2" in this thread, so:
A) Are there more WeakMap applications that will want .clear() or applications that will want .clear() not to exist? Offhand I would bet on the former, by a landslide, but if you think otherwise, or if there's some other reason to privilege .clear() not existing, let's talk about that.
B) What you said:
Now, the only thing that can differentiate both the native against this
version is performance I think. Allen seems to argue that a native .clear would have better perf characteristics (related to GC). I still fail to see why the difference would be significant (but I need to re-read his recent posts about that).
Definitely re-read them. They made sense to me. If you have questions about the implementation of GC through WeakMaps, I'll happily share what I know.
In all likelihood, .clear is a method that is used sporadically. At least,
one needs to fill up the weakmap a bit before calling it, so I don't think a marginal perf difference would matter.
The perf difference Allen is talking about is not marginal.
As an implementor, what is your feeling about performance characteristics
of both the native and the class-based version?
Heh! I'm the worst person to ask about this. I'm not comfortable with the worst-case GC performance of WeakMaps to begin with. My main coping mechanism is not thinking about it!
In our current implementation, creating a new WeakMap and dropping the old one is very nearly equivalent in performance to clear(). However that's because we don't have a generational GC today. Dead WeakMaps are promptly collected. In another year, that will change. If we end up with more than two generations, I think it'll lead to exactly the problems Allen foresees. Maybe even if we just have two generations. (To some extent, long-lived ordinary Maps and Arrays also do this in a generational GC; but WeakMaps have much, much worse worst-case GC performance.)
Having said all that, I bet we could hack around the worst-case GC performance. It'll be a pain, but GC is like that sometimes. This decision should hinge on what provides the best API for developers. I think we mainly disagree on what developers want, which is a great thing to talk about. Let's talk about that.
Le 22/01/2013 15:19, Jason Orendorff a écrit :
On Tue, Jan 22, 2013 at 5:56 AM, David Bruant <bruant.d at gmail.com <mailto:bruant.d at gmail.com>> wrote:
Le 22/01/2013 11:47, Jason Orendorff a écrit :On Mon, Jan 21, 2013 at 6:04 AM, David Bruant <bruant.d at gmail.com <mailto:bruant.d at gmail.com>> wrote: [...] WeakMap.prototype.clear questions the property that was true before its adoption ("you can only modify a weakmap entry if you have the key") David, would you please elaborate your argument for this invariant? This the first I've seen it stated. An invariant can be a powerful thing. Still, I guess my default position is that (1) the object-capabilities perspective is only one view among many; (2) even looking at things with an eye for o-c integrity and security, clearing a data structure seems like a reasonable thing to allow, treating a reference to the data structure itself as a sufficient capability. It's (2) that I would especially like you to address.I think Rick already suggested your (2), though phrased a bit differently [1] (that was his #1). I answered [2]: "I thought more about how I use weakmaps and [well-encapsulate my weakmaps so that I'm the only holder] is a thing I do naturally indeed." The problem may arise when you start sharing weakmaps around and some use cases require you to [3].What problem exactly?
I was wrong in saying "the problem". A problem may arise, this problem being that there is a risk that you were relying on some entries and that they may disappear at any time making your code harder to reason about.
[re-ordering]
Also, I don't understand how [3] is a use case for sharing weakmaps around. To me it looks like a use case for clearing a WeakMap.
I was imagining that some of the different phases could be performed by third-party code. But since the use case is about a cache, there is no reason one would rely on the existence of some entries. Maybe a more subtle use case needs to be found.
Sharing mutable data structures across abstraction (or trust) boundaries is already pretty well understood to be an integrity (or security) risk. It's easy to fix: you expose a read-only view instead.
If WeakMap.prototype.clear is part of the built-in API, an attacker (including buggy code) can do WeakMap.prototype.clear.call(yourWeakMap), so "exposing a read-only view" means wrapping pretty much the way Mark Miller implemented clear
class WeakMapWithoutClear {
private let wrapped;
constructor() {
wrapped = new WeakMap();
}
get(key) => wrapped.get(key),
set(key, val) => wrapped.set(key, value),
has(key) => wrapped.has(key),
delete(key) => wrapped.delete(key)
}
What this and my previous show is an semantics equivalence between clearable and clear-less weakmaps. Which should be chosen as default?
- clear-less weakmaps have better integrity properties.
- clearable weakmaps may have better performance characteristics (I'm still not entirely convinced) Are use cases for .clear that common that they justify being put in the native API? Or is it acceptable to ask those who want it to wrap in classes?
On Tue, Jan 22, 2013 at 2:47 AM, Jason Orendorff <jason.orendorff at gmail.com> wrote:
On Mon, Jan 21, 2013 at 6:04 AM, David Bruant <bruant.d at gmail.com> wrote:
[...] WeakMap.prototype.clear questions the property that was true before its adoption ("you can only modify a weakmap entry if you have the key")
David, would you please elaborate your argument for this invariant? This the first I've seen it stated.
An invariant can be a powerful thing. Still, I guess my default position is that (1) the object-capabilities perspective is only one view among many;
Of course.
(2) even looking at things with an eye for o-c integrity and security, clearing a data structure seems like a reasonable thing to allow, treating a reference to the data structure itself as a sufficient capability. It's (2) that I would especially like you to address.
In general, yes. Maps exist as simply a useful mutable container data structure. .clear is perfectly reasonable to allow on maps, and indeed neither I nor anyone I recall has raised any objections to this.
But WeakMaps were introduced specifically to support rights amplification[1]. Because of this special need for WeakMaps, we introduced them first. The Maps and Set strawmen came later. Since the get/set/has/delete functionality of WeakMaps and Maps seem naturally related, we proceeded considering them to be related -- both by name of the abstraction and by compatibility of their common methods. This still seems attractive to me. But this type-like relationship between them seems to be causing more confusion than clarity. Their purposes are very different.
[1] www.youtube.com/watch?v=oBqeDYETXME has a good
explanation of rights amplification, and shows the purse example as an example.
On Tue, Jan 22, 2013 at 9:01 AM, David Bruant <bruant.d at gmail.com> wrote:
class WeakMapWithoutClear {
Yep. Glad to see we are thinking along the same lines.
What this and my previous show is an semantics equivalence between clearable and clear-less weakmaps. Which should be chosen as default?
- clear-less weakmaps have better integrity properties.
- clearable weakmaps may have better performance characteristics (I'm still not entirely convinced)
Well, I disagree a little with the wording here! I'd put it like this:
-
clear-less weakmaps are more convenient when you need a particular invariant, useful in implementing rights amplification;
-
clearable weakmaps are more convenient when you want to clear a WeakMap.[*]
To me, the wording "better integrity properties" suggests an integrity property that benefits all code using WeakMaps. But as far as I can tell, very few use cases would really benefit.
Separately: APIs should make easy things easy and hard things possible, right? So far, all the proposed use cases of non-clearable weakmaps are complex and security-sensitive. If you're already doing something that exciting, you can write WeakMapWithoutClear with your eyes closed. By contrast, use cases for .clear() are pretty simple. Clearing caches. The sort of thing where having to write a wrapper class might actually be an annoying marginal cost.
Cheers, -j
[*] Also some sort of performance thing that will totally check out if you take the time to look into it! :)
On Jan 22, 2013, at 7:18 AM, Mark S. Miller wrote:
But WeakMaps were introduced specifically to support rights amplification[1].
Perhaps, from your perspective, but not from mine. The above is certainly not a phrase I would ever say, I'm not even sure of your exact technical definition of "rights amplification" ( haven't watched the video link). Early in the development of ES6 talked quite a bit about support for hash tables (including observable per object identify hash values). We also talked about providing some sort of weak or ephemeron based collection because it address a known problem with GC based storage management: Representing relationships among objects in a manner than does not impact the GC lifetime of the objects involved in the relationship.
In designing a language we should generally look for composable features that can be used to solve a variety of programming problems. Whenever possible we should avoid features that have only one use or are over optimized for one use case.
I'm happy to provide a language facility that you can use to create whatever sort of rights amplification abstraction you needed. I'm strongly opposed to a feature that is only useful for or over optimized for some specific rights amplification abstraction.
Le 21/01/2013 22:42, Allen Wirfs-Brock a écrit :
On Jan 21, 2013, at 12:25 PM, David Bruant wrote:
Le 21/01/2013 20:52, Allen Wirfs-Brock a écrit :
On Jan 21, 2013, at 11:36 AM, Rick Waldron wrote:
This is the reality check I can get behind—I'm hard pressed to come up with a use case that isn't contrived or solvable by some other means.
This is easy:
I'm do phased traversals over a complex data structure. I have a number of functions that collaborative perform the function and they share access to a WeakMap to cache relationships that they identify over the course of the traversal. When I start a new traversal phase I want to flush the cache so I use the clear method to do so. Creating a new weakmap would work equally well to flush the cache. Same arguments applies to Map clear.
The difference with maps is that one can already enumerate all the keys, the .clear is just a convenience, not a new capability
I'm actually more comfortable with a discussion of the utility of the clear method for maps, in general. But, if it has utility for Map then it has the same utility for WeakMap and supplying on both is a matter of API consistency.
Again, I don't think maps and weakmaps can be compared. They are different tools that can be used in different conditions. Like unique and private symbols. My (small) experience is that when I need to associate data with an object but don't want to do the book-keeping of which entry I care about, I use weakmaps. So far, I've only used maps in places where I used to use objects to associate string with data. So far, I've used maps as a safe object (no need to worry about inheritance or properties)
WeakMaps methods can't use anything else than objects as keys, it makes very hard to switch from a structure to another and I still haven't found a case where I would have traded one for the other.
I'll fork a new thread about GC-related performance.
Le 22/01/2013 15:59, Jason Orendorff a écrit :
A) Are there more WeakMap applications that will want .clear() or applications that will want .clear() not to exist? Offhand I would bet on the former, by a landslide, but if you think otherwise, or if there's some other reason to privilege .clear() not existing, let's talk about that.
(...)
Having said all that, I bet we could hack around the worst-case GC performance. It'll be a pain, but GC is like that sometimes. This decision should hinge on what provides the best API for developers. I think we mainly disagree on what developers want, which is a great thing to talk about. Let's talk about that.
I agree use case dominant is a crucially important answer. To date, I honestly don't know which case of want-clear and don't-want-clear would be dominant. I agree Allen showed a compelling use case but I can't judge how important it is. In my experience, I've not had the need for a .clear yet. I know that in some occurences, my code relies on the weakmap not being emptied, but I've kept the weakmap well-encapsulated in these cases so this experience is not that relevant (because intrusive .clear can't happen).
I've written a whole paragraph which is the most factual I can say on the topic, but doesn't help the debate much.
When it comes to feelings, I prefer prudence by default. I'd like to say a few words about that. I understand that features shouldn't be seen only with ocaps eyes, but I'd like to take a moment to describe what I care about when it comes to ocaps and what we call "security". Node.js is an interesting ecosystem. There are a lot of modules, it's not unusual to use 10-20 modules in a project which can make 100+ modules when counting recursively. Because it costs a lot of time, it's not possible to rewrite everything, it's not possible to contribute the necessary test coverage to modules and it's not possible to do careful security reviews of all used modules (and updates!). However, it's possible to apply POLA (Principle Of Least Authority), that is give to each module the information and capabilities it needs to do its job and no more. If WeakMap.prototype.clear gets natively in the language, it means all modules can have an irrevocable right to flush any weakmap I hand them. It's the same sort of problem than if a "free" operator was brought to JavaScript (in advance, I agree that the .clear is more acceptable) as suggested once [1] (almost ironically by Node's lead?). Suddenly, a module could free objects you hand out to them. A module thinks it's freeing one of it's own objects but actually frees one of yours because of a bug? Too bad, you'll be throwing a TypeError very soon and who knows in which state it will leave your application. Dave Herman made an equivalent case about coroutines [2]. It provides abusive authority: you call a module-imported function and for whatever good or bad reason, it can suddenly stop the stack. It makes your code harder to reason about because when you wrote the code, you probably expected the function call to return (or throw).
Back to weakmaps, the issue here is not technical, but... cultural I would say. I can decide to encapsulate my weakmap in a WeakMapWithoutClear, but doing so, I cut myself from modules which take .clear for granted. A module does rely on WeakMap clearbility? It will hand me the weakmap I'm supposed to use and I know in advance anything can happen, because I didn't create this object. If weakmaps don't have a clear, modules using language weakmaps won't take it for granted and you can be fearless about sharing your weakmap... very much like you can be hand objects today without the fear of them being free'd by mistake or malice using a free operator.
Since I'm on the topic of language-based abusive authority, I've come across a case where a logging library would throw at my face [3] anytime it would have to log about an object with a cycle in it [4]. Logging libraries are really not the one you'd expect to throw an error so we didn't wrap the call in a try-catch. So logging some objects would make the application crash. I have come to think that error-throwing with stack-unwinding is an abusive authority. Especially given that try-catch is an opt-in. I understand now that silently swallowing errors is not a good idea and the need to report errors in a different channel than return value, but I don't think stack-unwinding is a good default for that. Promises show an interesting model; to keep in mind for another language maybe.
The point I tried to make here is that POLA allows to build applications using untrusted modules without fearing them and that's really an excellent property, because we use constantly huge amounts code we don't necessarily trust or have the time to review or test besides what we test during the development period.
WeakMap.prototype.clear is at a different scale of "danger" and abusive authority than a hypothetical free operator or coroutine or abusive stack-unwinding, but it's an issue of the same family. Whether the smaller scale of danger makes it acceptable is a good question. I would answer "no" by default, but I can understand if others say "yes" for this one. To be balanced with advantages that native WeakMap.prototype.clear provide which are yet to be accurately determined.
David
[1] esdiscuss/2012-October/026007 [2] calculist.org/blog/2011/12/14/why-coroutines-wont-work-on-the-web [3] flatiron/winston#151 [4] flatiron/winston#100
I've been following the threads on WeakMap.prototype.clear with interest, and I think I may have an argument that should make the existence of WeakMap.prototype.clear irrelevant from an ocap security perspective.
It was already previously suggested in this thread that the ocap thing to do would be to prevent access from untrusted code to your WeakMap instance in the first place.
I think the suggestion that WeakMaps could be shared freely safely if only they didn't have a .clear() method is wrong.
Consider for a moment that WeakMaps don't have a .clear() method so you know no one can get at your values without having access to the corresponding keys. So you share your WeakMap freely with untrusted code. This still leaves you open to another attack. Before I can describe the attack, I think it's useful if people have some understanding of what the "rights amplification" pattern actually is.
The "rights amplification" pattern that MarkM previously referred to is often used to convert an opaque, untrusted object reference into a value your code can trust.
In my trusted code, I can write:
token = Object.freeze({}); myWeakMap.set(token, myTrustedObject); return token;
I now pass around the token to code in the wild.
The token by itself is useless. It's only useful if someone later passes it back to my code:
function doSomethingWith(token) { // token could be anything, I don't trust it var trustedObj = myWeakMap.get(token); // if trustedObj is not undefined, the token was legitimate }
This pattern really hinges on the WeakMap being properly encapsulated. If it isn't, an attacker could just mint his own tokens, add them to the WeakMap, and then fool my trusted code into thinking that the token is legitimate.
In other words: if one wants to share a WeakMap with untrusted code, one must already wrap it such that access to the "set" method is prohibited as well. In this regard, also having to remove .clear() is a marginal extra cost, compared to the more general utility of this method when using WeakMaps for caches rather than for rights amplification.
Cheers, Tom
2013/1/22 David Bruant <bruant.d at gmail.com>
I would like to humbly say that if this WeakMap#clear() method is anyhow
slowing down or reminding the adoption of the WeakMap concept itself for
all browsers, just mark it TBD and go on with the rest: WeakMap is needed
and wanted since ever, clear() gotchas can be discussed for all the time
we need to make it right, it should not stop or block WeakMap
specifications for ES6, a big IMHO in this reply and thanks for reading
'till here.
Le 22 janv. 2013 à 22:34, David Bruant <bruant.d at gmail.com> a écrit :
Back to weakmaps, the issue here is not technical, but... cultural I would say.
Indeed, the issue is cultural. We are questioning WeakMap.prototype.clear, because some people think that WeakMap should provide some strong encapsulation. But it need not be so.
The core problem is that WeakMaps are not designed as a secure feature. They are just designed as, well, weak maps, that is maps that don't prevent garbage collection of their keys and values. As such, they are not intended to be more (or less) secure than Maps or any other feature, but just less memory-leaking, which is very different. It is true that weak maps provide some degree of encapsulation if you don't want to expose GC, but this is just a side-effect of the definition. It is also true that a WeakMap without "clear" method provides some strong sort of encapsulation, but this is a limitation of the API, not a an inherent property of weak maps.
By contrast, Private Symbols are precisely designed to provide good privacy. Any feature that would threat that privacy (e.g., an Object.deleteAll() functionality which wipes all own properties of an object, even those whose keys are unknown Private Symbols) is dubious. This is because Private Symbols provide privacy by design, not by side-effect or by limitation of the API.
If you want strong encapsulation, then either WeakMap considered as "weak map" is not sufficient, or the feature is misnamed and should have been called, say, ProtectedMap.
just imagine I gave up with polyfills and shim creating my own thing that work ... called HybridMap ...
Seriously, private symbols are ES7 stuff while WeakMaps are already in ES6, right? So go for whatever weak definition it is rather than delay partial improvements for other 4 years, IMHO!
WeakMap are OK, no matter how you call them, developers will trap objects here and there somehow so no real easy win anywhere but for those developers that know what they are doing, I would really cut the philosophy and go ASAP for what works the best, then again a lot of time to discuss whatever, just saying (and last reply here),
Le 21/01/2013 17:16, Rick Waldron a écrit :
I thought more about how I use weakmaps and #1 is a thing I do naturally indeed. I don't believe in #2, because calls to .has everytime someone may have maliciously or by mistake called .clear is quite a cost.
I'd like to point out, that just based on what you wrote, you do not support the inclusion of WeakMap.prototype.clear, but rather you support its non-removal based on what I said. Use cases for WeakMap.prototype.clear are... unclear. In which conditions would one want to wipe out all entries? If you're the only holder of the weakmap, creating a new one is equivalent to .clear (but allows to lazy GC the entries which sync .clear doesn't as easily).
More specifically, in which conditions would one want to wipe out all the entries they're oblivious to?
Le 21/01/2013 17:16, Rick Waldron a écrit : > On Mon, Jan 21, 2013 at 6:04 AM, David Bruant <bruant.d at gmail.com > <mailto:bruant.d at gmail.com>> wrote: > > I think WeakMap.prototype.clear slipped through the crack without > being specifically discussed. Based on what's publicly available, > I don't see anyone noticed and discussed the fact that > WeakMap.prototype.clear questions the property that was true > before its adoption ("you can only modify a weakmap entry if you > have the key") > > > I agree and disagree. I disagree because WeakMap.prototype.clear() > doesn't modify the weakmap entry in the same direct way, ie. > WeakMap.prototype.set, WeakMap.prototype.delete. On the other hand, I > agree because it provides (if the weakmap is exposed) a way to remove > an entry that might be used without any defense, ie. > WeakMap.prototype.has() or might leave the program in an unstable > state due to loss of some aggregate state information stored in the > weakmap. > > That being said, I support the inclusion of WeakMap.prototype.clear(), > because reasonable security and state defense expectations can be met by: > > 1. Not exposing a sensitive weakmap in program code > 2. Defending against missing entries with has() > > Subjectively, #1 mitigates without nannying. I thought more about how I use weakmaps and #1 is a thing I do naturally indeed. I don't believe in #2, because calls to .has everytime someone may have maliciously or by mistake called .clear is quite a cost. I'd like to point out, that just based on what you wrote, you do not support the inclusion of WeakMap.prototype.clear, but rather you support its non-removal based on what I said. Use cases for WeakMap.prototype.clear are... unclear. In which conditions would one want to wipe out all entries? If you're the only holder of the weakmap, creating a new one is equivalent to .clear (but allows to lazy GC the entries which sync .clear doesn't as easily). More specifically, in which conditions would one want to wipe out all the entries they're oblivious to? David > > Rick > > > I think the property I mentioned is cricial to weakmap integrity > and I think WeakMap.prototype.clear should be considered for > removal... or at least proper discussion since none really > happened from what I've found. > > David > > [1] > https://mail.mozilla.org/pipermail/es-discuss/2012-October/025962.html > [2] > https://mail.mozilla.org/pipermail/es-discuss/2012-November/026114.html > [3] https://github.com/rwldrn/tc39-notes > [4] > https://github.com/rwldrn/tc39-notes/blob/master/es6/2012-11/nov-27.md#review-of-new-draft-spec > [5] > http://wiki.ecmascript.org/doku.php?id=harmony:specification_drafts > [6] https://bugzilla.mozilla.org/show_bug.cgi?id=814562 > _______________________________________________ > es-discuss mailing list > es-discuss at mozilla.org <mailto:es-discuss at mozilla.org> > https://mail.mozilla.org/listinfo/es-discuss > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.mozilla.org/pipermail/es-discuss/attachments/20130121/5a17bb95/attachment-0001.html>