Use cases for WeakMap
Can you describe those common use cases, and how they are impossible with the current specification?
Le 15/05/2011 00:05, Oliver Hunt a écrit :
The more I read the WeakMap spec, the more it seems to prevent the common use cases I know of.
What are the use cases you know of? How are they prevented by the current spec?
Could someone give a few examples of problems the current weakmap spec solves?
In my opinion, having a key->value map indexed on object identities
which has O(log(n)) lookup is a win by itself. Weakness is a plus.
I don't know if it's a problem, but WeakMaps with getters/setters allow a form of inheritance that has not really been possible before (it was with getter/setters and arrays, but with almost unavoidable memory leaks).
var o = {}; (function(){ var wm = WeakMap();
function get(){
return wm.get(this);
}
function set(val){
wm.set(this, val);
}
Object.defineProperty(o, "a", {get:get, set:set});
})();
var oo = Object.create(o); oo.a = 1;
var oo2 = Object.create(o); oo2.a = 2;
console.log(oo.a); // 1; console.log(oo2.a); // 2 // but still: console.log(Object.getOwnPropertyDescriptor(oo, "a")); // undefined console.log(Object.getOwnPropertyDescriptor(oo2, "a")); // undefined // On FF nightly, this is not undefined, but I'd say it's due to bugzilla.mozilla.org/show_bug.cgi?id=636989
Other may disagree, but I like this pattern, because it keeps properties "at the layer they belong to" rather than making the own layer a messy bag.
This is a gist that David Bruant created for me with regard to conversation in the Property Fixing thread: gist.github.com/970812
gist.github.com/970812Seen here:
It's not that they're impossible, it's that they all reduce to a strong map with automatic deletion, rather than "WeakMap".
Take any form of weak cache, say for example you want to cache the object representation that's the result of an XHR (or some such)
eg. you want to do
function getObject(url) { var result = myCache.get(url); if (result) return result; result = Object.freeze(JSON.parse(loadURLWithSynchXHR(url).responseText)); myCache.set(url, result); return result; }
Except you this doesn't work, because the map needs to have an object key, so the string is not allowed, we have to do new String(url) or some such as the key, so for a use case like this we need a map from string primitive to String object, so making the map a strong map.
Can you provide a use case where you have an object key as the usual programming idiom?
Don't mind me, I misread the spec (again) as saying that if either the key or value was live then the entire mapping remained alive.
If you want to self-host Date objects in JS you can use a WeakMap to store the [[Value]] for each Date object (similar to private names, the Date object is the key). The same goes for self-hosted DOM implementations (the content-facing wrapper is the key used to look up the internal object holding the underlying DOM node state).
Le 15/05/2011 00:37, Oliver Hunt a écrit :
It's not that they're impossible, it's that they all reduce to a strong map with automatic deletion, rather than "WeakMap".
Take any form of weak cache, say for example you want to cache the object representation that's the result of an XHR (or some such)
eg. you want to do
function getObject(url) { var result = myCache.get(url); if (result) return result; result = Object.freeze(JSON.parse(loadURLWithSynchXHR(url).responseText)); myCache.set(url, result); return result; }
Except you this doesn't work, because the map needs to have an object key, so the string is not allowed, we have to do new String(url) or some such as the key, so for a use case like this we need a map from string primitive to String object, so making the map a strong map.
Since your getObject function has a string as unique argument, there is no other choice than doing a strong map. Strings are primitive values. They are copied, not passed by reference like objects. If you want to index things on primitive values, you'll have to delete them yourself (with a timer maybe?).
No, I am wrong, if i have a key that i can ever reuse, the map is strong, because the key will keep the value live. These aren't weak maps, they are strong maps that don't leak keys that have become dead.
I can kind of see the value of this kind of structure, but I don't believe it is a WeakMap.
Le 15/05/2011 01:01, Oliver Hunt a écrit :
No, I am wrong, if i have a key that i can ever reuse, the map is strong, because the key will keep the value live. These aren't weak maps, they are strong maps that don't leak keys that have become dead.
I can kind of see the value of this kind of structure, but I don't believe it is a WeakMap.
What is your definition of a WeakMap? How is the current strawman different from this definition?
On May 14, 2011, at 4:03 PM, David Bruant wrote:
Le 15/05/2011 01:01, Oliver Hunt a écrit :
No, I am wrong, if i have a key that i can ever reuse, the map is strong, because the key will keep the value live. These aren't weak maps, they are strong maps that don't leak keys that have become dead.
I can kind of see the value of this kind of structure, but I don't believe it is a WeakMap. What is your definition of a WeakMap? How is the current strawman different from this definition?
In the definition of a weak map that I have always known, the key does not keep the mapped value alive. In the weakmaps proposal the key keeps the mapped value alive.
So if you can ever lookup a key again, the value cannot be collected, so your "cache" will hold onto every entry forever.
On Sat, May 14, 2011 at 4:01 PM, Oliver Hunt <oliver at apple.com> wrote:
No, I am wrong, if i have a key that i can ever reuse, the map is strong, because the key will keep the value live. These aren't weak maps, they are strong maps that don't leak keys that have become dead.
I can kind of see the value of this kind of structure, but I don't believe it is a WeakMap.
Does the following summary agree with your point? "A WeakMap is just a non-enumerable object identity keyed map, whose API is designed to allow implementations to transparently collect more unreachable collections. Non-normatively, implementation are expected to exploit this opportunity.".
Note that I did not say "weak" in this summary. The original proposed name was EphemeronTable, which everyone hated, including myself. But we gave it this name to distinguish it from the traditional weak-key hashtable, implemented from weak references and notification, since weak-key hashtables cannot be as non-leaky as ephemeron tables.
When someone -- wasn't it you? -- suggested "WeakMap", IIRC we all immediately fell in love with the name. Speaking only for myself, I like "WeakMap" because it suggests some connection with weak references without being specific, and clearly being distinguished from weak-key hashtable. As a new coinage, we're free to define precisely what it means, as we have.
As for genuine weak references, which enable a range of use cases that WeakMaps do not, what do you think of < strawman:weak_references>? I hope to
propose something along these lines for ES-after-next.
On Sat, May 14, 2011 at 4:21 PM, Oliver Hunt <oliver at apple.com> wrote:
On May 14, 2011, at 4:03 PM, David Bruant wrote:
Le 15/05/2011 01:01, Oliver Hunt a écrit :
No, I am wrong, if i have a key that i can ever reuse, the map is strong, because the key will keep the value live. These aren't weak maps, they are strong maps that don't leak keys that have become dead.
I can kind of see the value of this kind of structure, but I don't believe it is a WeakMap. What is your definition of a WeakMap? How is the current strawman different from this definition?
In the definition of a weak map that I have always known, the key does not keep the mapped value alive. In the weakmaps proposal the key keeps the mapped value alive.
Like < strawman:weak_references#a_weakvaluetable
?
Is there some prior system that refers to these as WeakMaps?
I suspect i did suggest WeakMap but I think I misunderstood the proposal. I felt the goal was to prevent the key from being kept around forever even when the value was gone, I did not expect the key to keep the value alive.
Le 15/05/2011 02:03, Oliver Hunt a écrit :
I suspect i did suggest WeakMap but I think I misunderstood the proposal. I felt the goal was to prevent the key from being kept around forever even when the value was gone, I did not expect the key to keep the value alive.
I may be missing something, but I think there is no way to do differently.
var wm = WeakMap(); var value = {}; var key = {};
wm.set(key, value); value = null; // value can be garbage-collected anytime now.
wm.get(key); // ?
Let's suppose "key" can be garbage-collected if "value" is garbage-collected as you suggest. What is the value returned by the get? "value" if it hasn't been garbage-collected or "undefined" if it has? This is a garbage-collector leak and introduces indeterminism.
On 5/14/11 6:37 PM, Oliver Hunt wrote:
Can you provide a use case where you have an object key as the usual programming idiom?
Attaching metadata to object without polluting the objects themselves.
For example, Firefox extensions want to do this all the time for various DOM objects (and especially Window).
Boris,
Would you mind sharing a piece of working code (as in, runs in the latest Firefox Nightly) that demonstrates your example in a real world scenario? This would be greatly appreciated
On May 15, 2011 2:55 AM, "Boris Zbarsky" <bzbarsky at mit.edu> wrote:
On 5/14/11 6:37 PM, Oliver Hunt wrote:
Can you provide a use case where you have an object key as the usual
programming idiom?
Attaching metadata to object without polluting the objects themselves.
That sounds more like private names
For example, Firefox extensions want to do this all the time for various
DOM objects (and especially Window).
On Sat, May 14, 2011 at 11:57 PM, Erik Corry <erik.corry at gmail.com> wrote:
On May 15, 2011 2:55 AM, "Boris Zbarsky" <bzbarsky at mit.edu> wrote:
On 5/14/11 6:37 PM, Oliver Hunt wrote:
Can you provide a use case where you have an object key as the usual programming idiom?
Attaching metadata to object without polluting the objects themselves.
That sounds more like private names
Private name and soft field lookup inherits. WeakMap lookup doesn't.
On May 14, 2011, at 11:59 PM, Mark S. Miller wrote:
On Sat, May 14, 2011 at 11:57 PM, Erik Corry <erik.corry at gmail.com> wrote:
On May 15, 2011 2:55 AM, "Boris Zbarsky" <bzbarsky at mit.edu> wrote:
On 5/14/11 6:37 PM, Oliver Hunt wrote:
Can you provide a use case where you have an object key as the usual programming idiom?
Attaching metadata to object without polluting the objects themselves.
That sounds more like private names
Private name and soft field lookup inherits. WeakMap lookup doesn't.
Also, depending on how they're spec'ed, private names are really naming properties "on" the object. Can't set them on frozen objects in the main (names, private names) proposals. This is a crucial difference with weak maps.
Besides attaching metadata, weak maps are important for remembering the wrapper or membrane for a given (frozen or not, built-in or "host", not to be mutated) object identity. Mark and Andreas knows too well, so I'm preaching to es-discuss in the To: line. This is not a use-case for weak references.
Weak maps are useful. I don't think they're misnamed. But we can revisit the name if we must.
On 5/14/11 9:06 PM, Rick Waldron wrote:
Would you mind sharing a piece of working code (as in, runs in the latest Firefox Nightly)
I don't have such code, sorry. I could try to write some, but pretty much anyone else is in as good a position to do that too.
If the issue is just needing an example of how this could be used, I can give more details of a use case with pseudocode; I have no idea what the current weakmap syntax even looks like.
On May 14, 2011, at 6:06 PM, Rick Waldron wrote:
Boris,
Would you mind sharing a piece of working code (as in, runs in the latest Firefox Nightly) that demonstrates your example in a real world scenario? This would be greatly appreciated
javascript:k = {}; v = "hi"; w = WeakMap(); w.set(k, v); alert(w.get(k))
just alerted "hi" in my slightly out of date tracemonkey-repo nightly build. There may be some javascript: URL stripping or blocking you have to get past, in which case use the code after the javascript: in the Web Console.
Thanks Brendan, I was looking for something that was representative of Boris's use-case, but I appreciate the snippet none-the-less.
I should clarify my intentions; while your example is very straight forward, it's also as ambiguous as the gist I had originally posted that Alex, yourself and I shared a discussion on - only in that it doesn't illustrate the behaviour of the weakmap that has had a nulled key, post-GC.
I'm coming at this from the perspective of an js application developer and thinking "how do I synthesize this into something that developers can understand the benefit of and apply to real-world uses?", in short: developer education/relations.
Either way, thanks again!
On May 15, 2011, at 11:20 AM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case, but I appreciate the snippet none-the-less.
Sorry, I should have realized what you were after (although in our gist discussion I thought I addressed it :-P).
I should clarify my intentions; while your example is very straight forward, it's also as ambiguous as the gist I had originally posted that Alex, yourself and I shared a discussion on - only in that it doesn't illustrate the behaviour of the weakmap that has had a nulled key, post-GC.
As noted, you can't illustrate that difference without exposing the non-determistic GC schedule. We don't want to do that. With an #ifdef DEBUG, REPL/console-only hook, we could.
I'm coming at this from the perspective of an js application developer and thinking "how do I synthesize this into something that developers can understand the benefit of and apply to real-world uses?", in short: developer education/relations.
It's more of a "use this unless you really know what you are doing with strong references". Using Map by default is going to make leaks in the real world, guaranteed. Using WeakMap when in doubt will cost a bit more but avoid leaks.
Brendan,
Just wanted to say that as I gather information about this and other ES.next APIs, these points of clarification are really appreciated - thanks again.
On May 15, 2011 8:59 AM, "Mark S. Miller" <erights at google.com> wrote:
On Sat, May 14, 2011 at 11:57 PM, Erik Corry <erik.corry at gmail.com> wrote:
On May 15, 2011 2:55 AM, "Boris Zbarsky" <bzbarsky at mit.edu> wrote:
On 5/14/11 6:37 PM, Oliver Hunt wrote:
Can you provide a use case where you have an object key as the usual
programming idiom?
Attaching metadata to object without polluting the objects themselves.
That sounds more like private names
Private name and soft field lookup inherits. WeakMap lookup doesn't.
In rhe case where you care about this you can use hasOwnProperty.
And I think you should be able to add private properties to frozen objects. Since the properties are private they are invisible to whoever froze the object so they can't violate the expectations of the freezer. Perhaps it also needs to be possible to freeze individual private properties (or their absence) but being able to put a blanket ban on other parts of the program performing an action that is undetectable to you feels wrong.
In terms of implementation this is no worse than keying a weak map with a frozen object since that will also involve modifying the frozen object.
For example, Firefox extensions want to do this all the time for
various DOM objects (and especially Window).
On May 14, 2011, at 5:03 PM, Oliver Hunt wrote:
I suspect i did suggest WeakMap but I think I misunderstood the proposal. I felt the goal was to prevent the key from being kept around forever even when the value was gone, I did not expect the key to keep the value alive.
--Oliver
I think I've suggest in the past that what we are currently calling "WeakMap" should just be called "ObjectMap" or something like that. I can't think of any use case where object identify is used as a map key and you don't want the "WeakMap" semantics. Essentially "WeakMap" just means "NonLeakyObjectMap".
On Sun, May 15, 2011 at 6:31 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On May 14, 2011, at 5:03 PM, Oliver Hunt wrote:
I suspect i did suggest WeakMap but I think I misunderstood the proposal. I felt the goal was to prevent the key from being kept around forever even when the value was gone, I did not expect the key to keep the value alive.
--Oliver
I think I've suggest in the past that what we are currently calling "WeakMap" should just be called "ObjectMap" or something like that. I can't think of any use case where object identify is used as a map key and you don't want the "WeakMap" semantics. Essentially "WeakMap" just means "NonLeakyObjectMap".
It's pretty easy to implement a Set datastructure on top of a Map, and if your set's equality is ===, then you'd want a strong object-identity-keyed map. WeakMap semantics would clearly be wrong here.
Of course, to do this using ES.next WeakMaps would be annoying, because only objects can be used as keys, but that's not really relevant to the strong/weak distinction.
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away.
Right now the latter is solved through Gecko-specific notifications that Firebug listens for and the former is solved in ways that are a bit hacky and error-prone, as I recall. Both would be solved with an API that lets you use the element or window as the key to look up the data of interest but doesn't keep the element or window alive.
Note that JS libraries on the web have some similar issue; right now they use various sorts of hacks to decorate elements with properties whose names are "not likely" to collide with other stuff...
On May 15, 2011, at 3:47 PM, Boris Zbarsky wrote:
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away.
Which is a use case private names would achieve much more succinctly than weakmaps.
On May 15, 2011, at 3:48 PM, Oliver Hunt wrote:
On May 15, 2011, at 3:47 PM, Boris Zbarsky wrote:
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away. Which is a use case private names would achieve much more succinctly than weakmaps.
Not if the object is frozen.
Unless you then overspecify private names as weak maps, in which case we are going in circles :-P.
On May 15, 2011, at 3:41 PM, Sam Tobin-Hochstadt wrote:
On Sun, May 15, 2011 at 6:31 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On May 14, 2011, at 5:03 PM, Oliver Hunt wrote:
I suspect i did suggest WeakMap but I think I misunderstood the proposal. I felt the goal was to prevent the key from being kept around forever even when the value was gone, I did not expect the key to keep the value alive.
--Oliver
I think I've suggest in the past that what we are currently calling "WeakMap" should just be called "ObjectMap" or something like that. I can't think of any use case where object identify is used as a map key and you don't want the "WeakMap" semantics. Essentially "WeakMap" just means "NonLeakyObjectMap".
It's pretty easy to implement a Set datastructure on top of a Map, and if your set's equality is ===, then you'd want a strong object-identity-keyed map. WeakMap semantics would clearly be wrong here.
I talked to Sam because it wasn't totally clear to me what the loss of a WeakMap entry if all clients of that Set implementation lost their keys would break. He replied "enumeration".
That's a good point but it disqualifies WeakMap out of the gate. WeakMaps are not enumerable, again to avoid leaking GC non-determinism.
Therefore I don't think ObjectMap is a better name than WeakMap. But neither is NonEnumerableEphemeronBasedWeakMap ;-).
Of course, to do this using ES.next WeakMaps would be annoying, because only objects can be used as keys, but that's not really relevant to the strong/weak distinction.
This is a good point too. Not sure we've considered a value -> value map carefully yet.
On Sun, May 15, 2011 at 7:06 PM, Brendan Eich <brendan at mozilla.com> wrote:
But neither is NonEnumerableEphemeronBasedWeakMap ;-).
NonEnumerableEphemeronBasedWeakIdentityKeyedAssociativeArraryOfObjects ?
On May 15, 2011, at 3:41 PM, Sam Tobin-Hochstadt wrote:
On Sun, May 15, 2011 at 6:31 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
I think I've suggest in the past that what we are currently calling "WeakMap" should just be called "ObjectMap" or something like that. I can't think of any use case where object identify is used as a map key and you don't want the "WeakMap" semantics. Essentially "WeakMap" just means "NonLeakyObjectMap".
It's pretty easy to implement a Set datastructure on top of a Map, and if your set's equality is ===, then you'd want a strong object-identity-keyed map. WeakMap semantics would clearly be wrong here.
Weakmap are also not enumerable which I believe you need for this sort of set implementation to work (and to be useful). You need to be able to get items that have no exterior references out of the set. If you can't you might as well collect it.
A possible sloppy Set implementation using WeakMap and an auxiliary Array:
function Set() { this.keys = new WeapMap; this.entries = []; }
Set.prototype.add = function (member) { if (!this.keys.get(member)) { this.entries.push(member); this.keys.set(member,this.entries.length-1); } }
Set.prototype.remove = function (member) { let indx = this.keys.get(member); if (indx) { delete this.entries[indx]; this.keys.delete(member); } }
Set.prototype.add = function (member) { return this.keys.has(member); }
Set.prototype.forEach= function (f) { this.entries.forEach(f); }
If you are going to have both WeakMap and ObjectMap (ie, NonweakMap) I suspect that you will have more bugs from people using an ObjectMap when they really should be using a WeakMap then visa versa. The exceptional case is wanting the map to be leaky. Rather than having Map and WeakMap it would probably be better to have Map and StrongMap.
On May 15, 2011, at 3:50 PM, Brendan Eich wrote:
On May 15, 2011, at 3:48 PM, Oliver Hunt wrote:
On May 15, 2011, at 3:47 PM, Boris Zbarsky wrote:
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away. Which is a use case private names would achieve much more succinctly than weakmaps.
Not if the object is frozen.
Unless you then overspecify private names as weak maps, in which case we are going in circles :-P.
/be
From a object design perspective there is a big difference between using a side-table to associate supplementary information with an object and adding a property to an object. This difference is independent of issues like weakness of the table or visibility of the property.
An object is suppose to represent a cohesive abstraction The building blocks of a JS object abstraction are its properties. You should only add properties to an object if they contribute to a cohesive extension of the object's abstraction. If you are simply tracking an object or using it as data to in the context of some other abstraction you should be extending to the object properties. Instead the tracking mechanism or additional data should be part of the consuming abstraction, perhaps represented as some sort of "side table".
There are only a few caveats to the above. If the abstraction is some sort of generic data collection (eg, an Array) that directly exposes its content as properties, then of course you can add properties if you are adding to the collection. Similarly, if an object exposed as part of its core abstraction some sort of open-end annotation or extension mechanism then it should be used as intended. Finally, you may be forced to use a "side-table" for what should logically be a property level extension to an if you are not allowed to extend the original object.
2011/5/16 Brendan Eich <brendan at mozilla.com>:
This is a good point too. Not sure we've considered a value -> value map carefully yet.
A value->anything map is pretty easy to do with a normal JS object.
function get(value) { if (typeof(value) == 'number') return this["NUM" + value]; if (typeof(value) == 'string') return this["STR" + value]; ... }
Weakness makes no sense in this case since a value can never really be lost.
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 3:48 PM, Oliver Hunt wrote:
On May 15, 2011, at 3:47 PM, Boris Zbarsky wrote:
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away. Which is a use case private names would achieve much more succinctly than weakmaps.
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Unless you then overspecify private names as weak maps, in which case we are going in circles :-P.
Allowing private names on frozen objects doesn't imply the GC semantics of weak maps so there is still a very important difference.
On May 15, 2011, at 11:53 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
This is a good point too. Not sure we've considered a value -> value map carefully yet.
A value->anything map is pretty easy to do with a normal JS object.
function get(value) { if (typeof(value) == 'number') return this["NUM" + value]; if (typeof(value) == 'string') return this["STR" + value]; ... }
Where's the 'object' case (excluding null)?
Weakness makes no sense in this case since a value can never really be lost.
Weakness is still necessary for an object key if you want to avoid leaks and you don't need to enumerate.
On May 15, 2011, at 11:55 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 3:48 PM, Oliver Hunt wrote:
On May 15, 2011, at 3:47 PM, Boris Zbarsky wrote:
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away. Which is a use case private names would achieve much more succinctly than weakmaps.
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Now we are going in circles. Let's try to use words already defined in ES5 and model things in observably distinguishable ways, or what's the point?
Frozen means [[Extensible]] is false, so you can't add any properties, I don't care how they are named. What you are proposing here is not observably different from soft fields via weak maps. That is one way to "implement private names" but it isn't what we have been calling "private names", and it does not help the discussion to assume one conclusion.
Further, some controversy remains around reflection. If any party (one with access to the private name) can reflect on a private-named extension to a frozen object, the objejct was therefore not really frozen, rather extensible.
Unless you then overspecify private names as weak maps, in which case we are going in circles :-P.
Allowing private names on frozen objects doesn't imply the GC semantics of weak maps so there is still a very important difference.
Not observable.
Anyway, inextensible objects cannot be decorated with new properties, whatever the privacy or GC semantics of those properties' names.
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 11:55 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 3:48 PM, Oliver Hunt wrote:
On May 15, 2011, at 3:47 PM, Boris Zbarsky wrote:
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away. Which is a use case private names would achieve much more succinctly than weakmaps.
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Now we are going in circles. Let's try to use words already defined in ES5 and model things in observably distinguishable ways, or what's the point?
Are you saying the way that private names work is off limits because the decision has already been made?
I'd appreciate if you responded to my earlier mail in this thread where I explained the thinking behind allowing private names on frozen objects, rather than just saying it's impossible by definition. In addition to the reasoning there I will add that if it is possible (and perhaps it isn't) to satisfy the important use cases of WeakMaps with a modified private names proposal then we can reduce two new features to one which AFAICS is a clear win in the keeping-complexity-manageable effort.
Frozen means [[Extensible]] is false, so you can't add any properties, I don't care how they are named. What you are proposing here is not observably different from soft fields via weak maps. That is one way to "implement private names" but it isn't what we have been calling "private names", and it does not help the discussion to assume one conclusion.
Further, some controversy remains around reflection. If any party (one with access to the private name) can reflect on a private-named extension to a frozen object, the objejct was therefore not really frozen, rather extensible.
Unless you then overspecify private names as weak maps, in which case we are going in circles :-P.
Allowing private names on frozen objects doesn't imply the GC semantics of weak maps so there is still a very important difference.
Not observable.
The GC properties of weak maps are hard to describe in the standard, but they are important to the users and the implementers so I hope they are not off limits for the discussion.
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 11:53 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
This is a good point too. Not sure we've considered a value -> value map carefully yet.
A value->anything map is pretty easy to do with a normal JS object.
function get(value) { if (typeof(value) == 'number') return this["NUM" + value]; if (typeof(value) == 'string') return this["STR" + value]; ... }
Where's the 'object' case (excluding null)?
Elided for clarity :-) It can be implemented with private names or WeakMaps. My point was we don't need to think about maps with values as keys. We have that already.
2011/5/15 Brendan Eich <brendan at mozilla.com>:
Besides attaching metadata, weak maps are important for remembering the wrapper or membrane for a given (frozen or not, built-in or "host", not to be mutated) object identity. Mark and Andreas knows too well, so I'm preaching to es-discuss in the To: line. This is not a use-case for weak references.
Is there an extra 'not' in this sentence? "weak maps are important [...] this is not a use-case for weak references"
Mark has mentioned membranes as an example of the use of WeakMaps. I can see that you don't want the membrane to keep the objects alive, but is it a problem that the objects keep the membrane alive? Are we expecting lots of membranes to come and go and the GC will need to clean up after them? I'm not saying it isn't important, I'm just trying to clarify the use case here.
On May 16, 2011, at 12:01 AM, Brendan Eich wrote:
On May 15, 2011, at 11:55 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Frozen means [[Extensible]] is false, so you can't add any properties, ...
I'll go further: frozen means the implementation should be free to move the object and anything associated with its shallow-frozen key/value state into read-only memory, so the full force of 50-year-plus OS and hardware MMU protection can ensure [[Extensible]] really is false.
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 16, 2011, at 12:01 AM, Brendan Eich wrote:
On May 15, 2011, at 11:55 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Frozen means [[Extensible]] is false, so you can't add any properties, ...
I'll go further: frozen means the implementation should be free to move the object and anything associated with its shallow-frozen key/value state into read-only memory, so the full force of 50-year-plus OS and hardware MMU protection can ensure [[Extensible]] really is false.
I think this would also preclude the most efficient implementation of Weak Maps. In fact I have a hard time seeing how you would GC weak maps at all under these conditions.
On May 16, 2011, at 12:08 AM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 11:55 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 3:48 PM, Oliver Hunt wrote:
On May 15, 2011, at 3:47 PM, Boris Zbarsky wrote:
On 5/15/11 2:20 PM, Rick Waldron wrote:
Thanks Brendan, I was looking for something that was representative of Boris's use-case
A typical example is an extension wanting to associate some state with a DOM element or a Window without polluting the DOM. For example, Adblock Plus wants to store some state per-element so that it knows when it's in the middle of unblocking something so it'll allow the load through for that one thing only. Firebug wants to store some state per-window (e.g. script bodies, etc) and discard it when the window goes away. Which is a use case private names would achieve much more succinctly than weakmaps.
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Now we are going in circles. Let's try to use words already defined in ES5 and model things in observably distinguishable ways, or what's the point?
Are you saying the way that private names work is off limits because the decision has already been made?
No, I'm saying clearly that [[Extensible]] being false means something in the spec, and in any future spec that includes private names.
I'd appreciate if you responded to my earlier mail in this thread where I explained the thinking behind allowing private names on frozen objects, rather than just saying it's impossible by definition.
I'd appreciate it if we all followed the history of the big argument about private names vs. soft fields (weak maps). There's a lot to digest, and it is not about "possible" -- it's about what words mean.
If private names are spec'ed so they can be implemented by weak maps or secretly-extensible frozen objects, and no one can tell the difference, then there's nothing to argue about. But if you read the wiki and the threads here, clearly we have disagreements on reflection vs. encapsulation, at least. We also have disagreed (and perhaps finally agreed) on whether dot and bracket syntax should be mapped to weakmap transposed get and set.
Sam Tobin-Hochstadt a while ago pointed to a crucial experiment in a message I won't dig up right now:
function get(obj, id) { return obj[id]; } function set(obj, id, val) { obj[id] = val; } var obj = Object.freeze({}); var privateName = ...; set(obj, privateName, 42); alert(get(obj, privateName));
Can this work with private names? Should it?
The private names proposals were already subject to a deathmatch attempt that ran roughshod over this disagreement. Let's not do it again.
Unless you then overspecify private names as weak maps, in which case we are going in circles :-P.
Allowing private names on frozen objects doesn't imply the GC semantics of weak maps so there is still a very important difference.
Not observable.
The GC properties of weak maps are hard to describe in the standard, but they are important to the users and the implementers so I hope they are not off limits for the discussion.
My point is not about GC semantics, it is about observable "frozen object gains property with private name" behavior. This is the bone of contention.
On May 16, 2011, at 12:11 AM, Erik Corry wrote:
2011/5/15 Brendan Eich <brendan at mozilla.com>:
Besides attaching metadata, weak maps are important for remembering the wrapper or membrane for a given (frozen or not, built-in or "host", not to be mutated) object identity. Mark and Andreas knows too well, so I'm preaching to es-discuss in the To: line. This is not a use-case for weak references.
Is there an extra 'not' in this sentence? "weak maps are important [...] this is not a use-case for weak references"
Weak references != weak maps.
Mark has mentioned membranes as an example of the use of WeakMaps. I can see that you don't want the membrane to keep the objects alive, but is it a problem that the objects keep the membrane alive? Are we expecting lots of membranes to come and go and the GC will need to clean up after them? I'm not saying it isn't important, I'm just trying to clarify the use case here.
I'm not sure what this has to do with weak references not being usable -- at all -- for membranes associated with (possibly frozen, not-to-be-mutated-in-any-event) objects.
There is no "there" in which to store a weak reference to the membrane from the object. Pigeon-hole problem, frozen object vs. mutation problem, host-object with crazy/zero storage semantics problem, the list goes on.
But in case it helps: yes, membranes need to be GC'ed ahead of their wrapped objects.
On May 16, 2011, at 12:18 AM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 16, 2011, at 12:01 AM, Brendan Eich wrote:
On May 15, 2011, at 11:55 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Frozen means [[Extensible]] is false, so you can't add any properties, ...
I'll go further: frozen means the implementation should be free to move the object and anything associated with its shallow-frozen key/value state into read-only memory, so the full force of 50-year-plus OS and hardware MMU protection can ensure [[Extensible]] really is false.
I think this would also preclude the most efficient implementation of Weak Maps. In fact I have a hard time seeing how you would GC weak maps at all under these conditions.
Why, because GC metadata has to go in the same page as the frozen object that might be a key in a weak map?
Weak maps are in Firefox nightlies. We're playing with page protection too (not for freezing, yet). This seems like a dare, but it also seems to be dodging my point in replying again: that private names cannot be used to extend frozen objects in the "[[Extensible]] = true" sense of the spec.
On May 16, 2011, at 12:10 AM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 11:53 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
This is a good point too. Not sure we've considered a value -> value map carefully yet.
A value->anything map is pretty easy to do with a normal JS object.
function get(value) { if (typeof(value) == 'number') return this["NUM" + value]; if (typeof(value) == 'string') return this["STR" + value]; ... }
Where's the 'object' case (excluding null)?
Elided for clarity :-) It can be implemented with private names or WeakMaps.
Oh, ok -- you wrote "normal JS object" and that seemed to preclude new stuff.
Yes, we can make a value -> value map as a library abstraction, but it's clunky. It has to use two kinds of maps under the hood. People shouldn't necessarily have to reinvent or discover or curate it and download those bytes all the time.
My point was we don't need to think about maps with values as keys. We have that already.
Your use of "normal JS object" and present tense to refer to either weak maps or private names (implemented via weak maps) is making me want to wait a few months and talk again :-/.
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 16, 2011, at 12:10 AM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 15, 2011, at 11:53 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
This is a good point too. Not sure we've considered a value -> value map carefully yet.
A value->anything map is pretty easy to do with a normal JS object.
function get(value) { if (typeof(value) == 'number') return this["NUM" + value]; if (typeof(value) == 'string') return this["STR" + value]; ... }
Where's the 'object' case (excluding null)?
Elided for clarity :-) It can be implemented with private names or WeakMaps.
Oh, ok -- you wrote "normal JS object" and that seemed to preclude new stuff.
Yes, we can make a value -> value map as a library abstraction, but it's clunky. It has to use two kinds of maps under the hood. People shouldn't necessarily have to reinvent or discover or curate it and download those bytes all the time.
This is where we disagree fundamentally. I think we should give people the lego bricks to build the abstractions they need. You seem to think we should give them all the abstractions they are going to need, ready built.
As for downloading all the time that's a caching/modules/libraries issue which needs to be solved in other ways. It has to be possible to write composable modules in the language that are downloaded only once for each time a new release is created. If we can't solve that then the temptation will always be to add every thinkable module to the language and the result will be constant pressure for bloat. I regard the i18n changes currently going in as an example of that.
My point was we don't need to think about maps with values as keys. We have that already.
Your use of "normal JS object" and present tense to refer to either weak maps or private names (implemented via weak maps) is making me want to wait a few months and talk again :-/.
I didn't use 'normal JS object' in that sense.
The context was you talking about a value->value map.
I presented a value->anything map which was implemented as a 'normal object'.
You complained it wasn't an anything->anything map.
I said you could do that with either private properties or weak maps. At that point you need both the 'normal object' and something else.
Perhaps we have a nomenclature problem with 'value'. Do you regard things that have typeof(o) == 'object' as values?
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 16, 2011, at 12:18 AM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 16, 2011, at 12:01 AM, Brendan Eich wrote:
On May 15, 2011, at 11:55 PM, Erik Corry wrote:
2011/5/16 Brendan Eich <brendan at mozilla.com>:
Not if the object is frozen.
That shouldn't prevent you adding private names. See earlier message in this thread.
Frozen means [[Extensible]] is false, so you can't add any properties, ...
I'll go further: frozen means the implementation should be free to move the object and anything associated with its shallow-frozen key/value state into read-only memory, so the full force of 50-year-plus OS and hardware MMU protection can ensure [[Extensible]] really is false.
I think this would also preclude the most efficient implementation of Weak Maps. In fact I have a hard time seeing how you would GC weak maps at all under these conditions.
Why, because GC metadata has to go in the same page as the frozen object that might be a key in a weak map?
I think the objects used as keys in weak maps need to be somehow annotated with this information so that the GC can clean up the weak maps when the keys die. This means that if you take an object that is frozen and use it as a key in a weak map then it will need to be mutated in some way and can't be on a read-only page.
Perhaps you have a different, efficient, implementation. I can't see us gaining much from putting frozen objects on read-only pages, thus I can't accept it as a very strong argument about the way that frozen objects should work together with a new feature.
Weak maps are in Firefox nightlies. We're playing with page protection too (not for freezing, yet). This seems like a dare, but it also seems to be dodging my point in replying again: that private names cannot be used to extend frozen objects in the "[[Extensible]] = true" sense of the spec.
Is there a description anywhere about how you have implemented GC of weak maps?
2011/5/16 Brendan Eich <brendan at mozilla.com>:
On May 16, 2011, at 12:11 AM, Erik Corry wrote:
2011/5/15 Brendan Eich <brendan at mozilla.com>:
Besides attaching metadata, weak maps are important for remembering the wrapper or membrane for a given (frozen or not, built-in or "host", not to be mutated) object identity. Mark and Andreas knows too well, so I'm preaching to es-discuss in the To: line. This is not a use-case for weak references.
Is there an extra 'not' in this sentence? "weak maps are important [...] this is not a use-case for weak references"
Weak references != weak maps.
Thanks, now I see what you meant. Don't know how I missed that.
Mark has mentioned membranes as an example of the use of WeakMaps. I can see that you don't want the membrane to keep the objects alive, but is it a problem that the objects keep the membrane alive? Are we expecting lots of membranes to come and go and the GC will need to clean up after them? I'm not saying it isn't important, I'm just trying to clarify the use case here.
I'm not sure what this has to do with weak references not being usable -- at all -- for membranes associated with (possibly frozen, not-to-be-mutated-in-any-event) objects.
Nothing, that was just me reading 'weak references' as 'weak maps'.
There is no "there" in which to store a weak reference to the membrane from the object. Pigeon-hole problem, frozen object vs. mutation problem, host-object with crazy/zero storage semantics problem, the list goes on.
Perhaps I was unclear: The semantics of Weak Maps are essentially symmetric. If you have the map and the key you can get the value. If you have just one then you can't (and the system may GC the value). My question was, do the use cases have both the GC of the map and the key triggering the GC of the value or is the GC of the key the important one and GC of the map not that common/important etc.
For the use case mentioned by Boris in this thread, where a FF extension needs to attach metadata to an object it doesn't seem likely that the mapping will get lost and need to be GCed before the objects that have the metadata attached.
On May 16, 2011, at 12:43 AM, Erik Corry wrote:
Elided for clarity :-) It can be implemented with private names or WeakMaps.
Oh, ok -- you wrote "normal JS object" and that seemed to preclude new stuff.
Yes, we can make a value -> value map as a library abstraction, but it's clunky. It has to use two kinds of maps under the hood. People shouldn't necessarily have to reinvent or discover or curate it and download those bytes all the time.
This is where we disagree fundamentally.
First, it helps to be clear about present vs. future tense -- we may not disagree at all once we straighten that out.
Second, of course we on TC39 are trying to fill semantic gaps, minimally and compositionally. We are not out to standardize libraries and higher-level abstractions in general. I've spoken and written about this often.
This is why generators are in harmony, while schedulers and deferreds built on them (or built on extensions to them, or on underlying models such as CPS conversion, but that's not the only way...) are not yet in Harmony.
It's arguably also why classes have had such a hard time.
Anyway, we don't disagree fundamentally on such short notice, with recent lack of clarity in tense and terms.
I think we should give people the lego bricks to build the abstractions they need. You seem to think we should give them all the abstractions they are going to need, ready built.
Not at all, see above and please don't exaggerate. Here, in this thread, I raised the question of value -> value maps not being fully considered:
This is a good point too. Not sure we've considered a value -> value map carefully yet.
It's still worth building the library code and weighing it. Your ellipses left out a lot.
As for downloading all the time that's a caching/modules/libraries issue which needs to be solved in other ways.
Agreed in general, but we have Map and Set on the agenda for the meeting next week. We've standardized library code such as the Array extras in ES5. De-jure cleanup and codification of de-facto standards such as Function.prototype.bind are on our permanent agenda.
The context was you talking about a value->value map.
I presented a value->anything map which was implemented as a 'normal object'.
You complained it wasn't an anything->anything map.
No, I thought you were suggesting it could be implemented without weak maps or something like them -- that's all.
I said you could do that with either private properties or weak maps. At that point you need both the 'normal object' and something else.
Right.
Perhaps we have a nomenclature problem with 'value'. Do you regard things that have typeof(o) == 'object' as values?
Certainly.
Let's get back to the observable-per-spec differences between private names (as proposed and called by that phrase in TC39) and other things based on WeakMaps, which may or may not (I think not) be competing with private names as proposed.
On May 16, 2011, at 12:47 AM, Erik Corry wrote:
I think the objects used as keys in weak maps need to be somehow annotated with this information so that the GC can clean up the weak maps when the keys die. This means that if you take an object that is frozen and use it as a key in a weak map then it will need to be mutated in some way and can't be on a read-only page.
That's already false in Firefox nightlies. We support Object.freeze. We have a WeakMap implementation. We do not mutate the frozen object. Its GC metadata does not reside in a header for it, or even in the same OS page.
Perhaps you have a different, efficient, implementation. I can't see us gaining much from putting frozen objects on read-only pages, thus I can't accept it as a very strong argument about the way that frozen objects should work together with a new feature.
This is a bit too subjective an argument, sorry.
My point about 50+ years of OS and MMU firewalling is important. Chrome (recently hacked by French spook-types, but also hacked over a year ago with a two-step attack) is a convincing example.
Sure, we have user-code isolation tools in our belts, including fancy compiler/runtime pairs. But it's hard to beat processes if you want to be sure. No silver bullet, simply "stronger isolation".
Weak maps are in Firefox nightlies. We're playing with page protection too (not for freezing, yet). This seems like a dare, but it also seems to be dodging my point in replying again: that private names cannot be used to extend frozen objects in the "[[Extensible]] = true" sense of the spec.
Is there a description anywhere about how you have implemented GC of weak maps?
hg.mozilla.org/tracemonkey/rev/7dcd0d16cc08
Look for WeakMap::mark... names. There's no need to mutate a key object. There should not be, either.
Yes, this GC can iterate. A lot, but a "fix" doesn't obviously require mutating (possibly frozen) key objects. Also, since POITROAE we are going to measure twice, Optimize once.
On May 16, 2011, at 1:52 AM, Erik Corry wrote:
My question was, do the use cases have both the GC of the map and the key triggering the GC of the value or is the GC of the key the important one and GC of the map not that common/important etc.
I'm not sure, we will have to measure to give a quantitative answer. But consider this: weak maps are used with membranes and "vats", and both the wrapped object (a key into a weak map handling out-of-vat object wrapping) and the whole vat can go away. Think window.close or DOM iframe creation/destruction via navigation.
For the use case mentioned by Boris in this thread, where a FF extension needs to attach metadata to an object it doesn't seem likely that the mapping will get lost and need to be GCed before the objects that have the metadata attached.
The add-on easily can outlive the content window or iframe containing the wrapped object, you're right. But it can go the other way too: add-on disabling or add-on specific reset clears the add-on state, including weak maps, while the content DOM wrapped objects live blithely on.
Even if you want to store weak-map specific meta data per object, nobody would store that directly in the object. Thats needlessly cruel
On 5/16/11 1:48 PM, Brendan Eich wrote:
For the use case mentioned by Boris in this thread, where a FF extension needs to attach metadata to an object it doesn't seem likely that the mapping will get lost and need to be GCed before the objects that have the metadata attached.
The add-on easily can outlive the content window or iframe containing the wrapped object, you're right. But it can go the other way too: add-on disabling or add-on specific reset clears the add-on state, including weak maps, while the content DOM wrapped objects live blithely on.
Indeed. Once you stop debugging, Firebug may well want to throw away all the state it needed to debug.
2011/5/16 Andreas Gal <gal at mozilla.com>:
Even if you want to store weak-map specific meta data per object, nobody would store that directly in the object. Thats needlessly cruel on the cache since >>99.9% of objects never end up in a weak map. Instead one would locate that meta data outside the object in a direct mapped dense area (like mark bitmaps), which is on its own page that is not write protected.
More than 99.9% of objects don't have a property called "fish". Nevertheless if someone adds a "fish" property to an object V8 will try to (and usually succeed in) storing it in the object and it won't be cruel on the cache. Quite the opposite.
On May 16, 2011 7:30 PM, "Brendan Eich" <brendan at mozilla.com> wrote:
On May 16, 2011, at 12:43 AM, Erik Corry wrote:
Perhaps we have a nomenclature problem with 'value'. Do you regard things that have typeof(o) == 'object' as values?
Certainly.
This is the source of our confusion. I was meaning 'primitive value' when I wrote 'value'. Sorry about that.
--
Erik Corry
On May 16, 2011, at 11:30 AM, Erik Corry wrote:
2011/5/16 Andreas Gal <gal at mozilla.com>:
Even if you want to store weak-map specific meta data per object, nobody would store that directly in the object. Thats needlessly cruel on the cache since >>99.9% of objects never end up in a weak map. Instead one would locate that meta data outside the object in a direct mapped dense area (like mark bitmaps), which is on its own page that is not write protected.
More than 99.9% of objects don't have a property called "fish". Nevertheless if someone adds a "fish" property to an object V8 will try to (and usually succeed in) storing it in the object and it won't be cruel on the cache. Quite the opposite.
I agree.
The same logic applies to object hashcodes -- an object must always produce the same hashcode which means it will need to store it -- having a secondary map doesn't help you, because that map will itself require a hascode. However making every object always have space to store a hashcode is wasteful of memory, so essentially some form of "private name" must be used. I think i saw someone (Erik?) say that this is what v8 does.
On May 16, 2011, at 11:37 AM, Oliver Hunt wrote:
On May 16, 2011, at 11:30 AM, Erik Corry wrote:
2011/5/16 Andreas Gal <gal at mozilla.com>:
Even if you want to store weak-map specific meta data per object, nobody would store that directly in the object. Thats needlessly cruel on the cache since >>99.9% of objects never end up in a weak map. Instead one would locate that meta data outside the object in a direct mapped dense area (like mark bitmaps), which is on its own page that is not write protected.
More than 99.9% of objects don't have a property called "fish". Nevertheless if someone adds a "fish" property to an object V8 will try to (and usually succeed in) storing it in the object and it won't be cruel on the cache. Quite the opposite.
I agree.
The same logic applies to object hashcodes -- an object must always produce the same hashcode which means it will need to store it -- having a secondary map doesn't help you, because that map will itself require a hascode. However making every object always have space to store a hashcode is wasteful of memory, so essentially some form of "private name" must be used. I think i saw someone (Erik?) say that this is what v8 does.
SpiderMonkey has "reserved slots". But this is all beside the point.
Frozen objects are not observably extensible. I argued that we want an implementation option to put their shallowly-frozen state (values at least) in read-only memory. I have not seen anything like an argument why this option should be forbidden.
On May 16, 2011, at 11:40 AM, Brendan Eich wrote:
Frozen objects are not observably extensible. I argued that we want an implementation option to put their shallowly-frozen state (values at least) in read-only memory. I have not seen anything like an argument why this option should be forbidden.
And (sorry for followup) in particular, there's no argument for frozen objects as weak map keys needing to sprout mutable private-named properties. We have an existence proof to the contrary.
On May 16, 2011, at 11:30 AM, Erik Corry wrote:
2011/5/16 Andreas Gal <gal at mozilla.com>:
Even if you want to store weak-map specific meta data per object, nobody would store that directly in the object. Thats needlessly cruel on the cache since >>99.9% of objects never end up in a weak map. Instead one would locate that meta data outside the object in a direct mapped dense area (like mark bitmaps), which is on its own page that is not write protected.
More than 99.9% of objects don't have a property called "fish". Nevertheless if someone adds a "fish" property to an object V8 will try to (and usually succeed in) storing it in the object and it won't be cruel on the cache. Quite the opposite.
That's true of JS using oneObj.fish among many, which is not the case we're arguing.
The case at hand is whether the GC has to fetch or store from an object (possibly a frozen object) used as a key in a WeakMap, to decide whether to treat the value part of the WeakMap entry as a strong reference. Touching random objects during GC is generally cache-cruel.
A GC may want to touch objects, even frozen objects, so long as this implementation detail can be concealed and performance is good.
Another JS implementation might want to avoid such key-object touching, and it might well want to write-protect frozen object memory. I said that and it seemed you thought such an implementation was impossible. We have one, but maybe there's more to discuss, a better way to GC weak maps.
Implementation issues are not the big deal here, though; this seems like a tangent. The main thread of discussion, as the Subject: says, was about use cases. WeakMaps have a strong use-case: mapping any object to per-object data that cannot be stored in the object -- in particular a membrane in the recursive pattern used for mediation, capability revocation, etc. between vats (windows, iframes, sandboxes, etc.).
This is the strongest use-case, IMHO.
There is a more generalized use-case for object-keyed indexing or memoization without memory leaks. Such indexes, memos, and caches, if implemented by strong maps, and if the key can be (reached from) a value and vice versa in the map, are terribly leak prone. Such caches generally don't need to be enumerated (except for debugging).
WeakMaps satisfy this more general non-enumerable object-keyed cache use-case well, too -- at least as far as I can tell. We'll be building on the Firefox nightly implementation to find out more.
On May 16, 2011, at 1:07 PM, Brendan Eich wrote:
WeakMaps satisfy this more general non-enumerable object-keyed cache use-case well, too -- at least as far as I can tell. We'll be building on the Firefox nightly implementation to find out more.
I think my problem with this is that WeakMap to me imply something other than what is being specified. What is being specified here is essentially a non-enumerable strong map, not a weakmap in the sense that i have ever known it.
On May 16, 2011, at 1:15 PM, Oliver Hunt wrote:
On May 16, 2011, at 1:07 PM, Brendan Eich wrote:
WeakMaps satisfy this more general non-enumerable object-keyed cache use-case well, too -- at least as far as I can tell. We'll be building on the Firefox nightly implementation to find out more.
I think my problem with this is that WeakMap to me imply something other than what is being specified. What is being specified here is essentially a non-enumerable strong map,
No, not strong -- conditionally strong or weak.
If you want to pick on the name for imprecision, you have to be more precise. That's the first rule of weakmap-club.
not a weakmap in the sense that i have ever known it.
I believe you are still thinking of weak references or weak pointers. Can you link to an example of what you mean?
On May 16, 2011, at 11:37 AM, Oliver Hunt wrote:
The same logic applies to object hashcodes -- an object must always produce the same hashcode which means it will need to store it -- having a secondary map doesn't help you, because that map will itself require a hascode. However making every object always have space to store a hashcode is wasteful of memory, so essentially some form of "private name" must be used. I think i saw someone (Erik?) say that this is what v8 does.
Old trick that works well when you have a copying collector:
Hashes are initially stored, on first demand, in a side-table that is a hash table keyed by the current address of an object. When an object is moved by the collector it checks to see if the object has a hashcode in the side table. If so, it adds a slot at the new location of the object and stores the hashcode in that slot. It also removes the entry from the side table.
This requires 2 bits in the object header to represent the following states: Object's hash has not been accessed so it is unassigned. Object's hash is in the side-table. Object's hash is in the hash slot.
If you can generate a suitable hash value based upon the object's current address you don't actually need the side-table.
If you want to have a readonly header that is ROM-able you can either always assign a hash slot to such objects or you can always keep their hash in the side-table.
The other hack is to make the hash value itself opaque as I believe weak maps could. That way you can use the address as the hash at the cost of having to lazily rehash after a GC. Typically the Cheney scan of a hash table would randomly reordered objects in memory and this hurt memory performance. The optimization was to delayed scanning hash tables until the end of the Cheney scan so linked objects would have already been placed in memory order.
This is all a bit off topic but performance does matter and folks seem to be underestimating the wealth of community knowledge that exists in this area.
On May 16, 2011, at 2:46 PM, Hudson, Rick wrote:
This is all a bit off topic but performance does matter and folks seem to be underestimating the wealth of community knowledge that exists in this area.
Who underestimates?
A bunch of us are aware of all this. Allen certainly knows all about it, and we've been talking shop with him for years, long before he joined Mozilla :-P. I recall a conversation like this one about sparse hashcode implementation with Allen, Lars Thomas Hansen (then of Opera), and Graydon Hoare from four or five years ago...
proposals:hashcodes (check the history)
However, in this thread, the issue is not optimizing hashcode or other metadata sparsely associated with objects. That's a good thing, implementations should do it. Having the hashcode in the object wins, compared to having it (initially) in a side table, but who's counting?
The issue under dispute was neither sparse hashcode nor sparse "fish" property association, where the property would be accessed by JS "user code" that referenced the containing object itself. Rather, it was whether a frozen object needed any hidden mutable state to be a key in a WeakMap. And since this state would be manipulated by the GC, it matters if it's in the object, since the GC would be touching more potentially randomly distributed memory, thrashing more cache.
So far as I can tell, there's no demonstrated need for this hidden-mutable key-in-weakmap object state. And it does seem that touching key objects unnecessarily will hurt weakmap-aware GC performance. But I may be underestimating... :-/
This is all a bit off topic but performance does matter and folks seem to be underestimating the wealth of community knowledge that exists in this area.
Who underestimates?
Sorry, this wasn't meant to slight anyone. I have spent a career standing on the shoulders of Allen and his colleagues. My respect should not be underestimated.
Interesting pointer.
-
Rick
From: Brendan Eich [mailto:brendan at mozilla.com] Sent: Monday, May 16, 2011 6:44 PM To: Hudson, Rick Cc: Allen Wirfs-Brock; Oliver Hunt; Andreas Gal; es-discuss Subject: Re: Use cases for WeakMap
On May 16, 2011, at 2:46 PM, Hudson, Rick wrote:
This is all a bit off topic but performance does matter and folks seem to be underestimating the wealth of community knowledge that exists in this area.
Who underestimates?
A bunch of us are aware of all this. Allen certainly knows all about it, and we've been talking shop with him for years, long before he joined Mozilla :-P. I recall a conversation like this one about sparse hashcode implementation with Allen, Lars Thomas Hansen (then of Opera), and Graydon Hoare from four or five years ago...
proposals:hashcodes (check the history)
However, in this thread, the issue is not optimizing hashcode or other metadata sparsely associated with objects. That's a good thing, implementations should do it. Having the hashcode in the object wins, compared to having it (initially) in a side table, but who's counting?
The issue under dispute was neither sparse hashcode nor sparse "fish" property association, where the property would be accessed by JS "user code" that referenced the containing object itself. Rather, it was whether a frozen object needed any hidden mutable state to be a key in a WeakMap. And since this state would be manipulated by the GC, it matters if it's in the object, since the GC would be touching more potentially randomly distributed memory, thrashing more cache.
So far as I can tell, there's no demonstrated need for this hidden-mutable key-in-weakmap object state. And it does seem that touching key objects unnecessarily will hurt weakmap-aware GC performance. But I may be underestimating... :-/
Off topic.
Hi Rick, my name is Rick. Nice to meet you.
The more I read the WeakMap spec, the more it seems to prevent the common use cases I know of. Could someone give a few examples of problems the current weakmap spec solves?