What is the status of Weak References?
On Thu, Jan 31, 2013 at 1:48 PM, Kevin Gadd <kevin.gadd at gmail.com> wrote:
A search shows some old discussions of the topic mentioning that they might be going in to future versions of the language, etc. But on the other hand I've been told in response to this question before that TC39 has a general policy against features that allow garbage collection to be visible to applications.
There's still a strawman up on the wiki: strawman:weak_references and while it appears to be a relatively simple, sane way to expose weak references, it looks like the strawman hasn't been touched since late 2011. Is that because it's dead? Or was it deprioritized because weak references are believed to not be needed by JS application developers?
I believe that proposal was dropped in favor of just always using WeakMaps.
We have yet to find a solution that does not leak information between two actors that are not supposed to be able to communicate. At this point we have a lot of important work to do for ES6 and until someone comes up with a solution to the security issues WeakRefs are postponed.
I'm not an expert in this field but I can try to explain the problem as I understand it.
- Given a master actor M that can communicate with two independent actors A and B (but A cannot communicate with B).
- M passes a frozen object O to A.
- A then creates a WeakRef for O.
- A also strongly holds on to O.
- M then passes the same object to B.
- B creates another weak ref to O.
- Now B will get notified when A stops holding strongly to O leading to a communication channel.
It is possible that we can accept this information leak and just have Caja etc blacklist use of WeakRefs but this is a discussion that we decided to postpone for now.
Thank you for the detailed explanation of the information leak - I had never seen an explanation of why GC visibility creates an issue.
Is there a page on the wiki somewhere that explains why information leaks are such a huge concern in JavaScript? I've never been under the impression that it is designed to be a secure/sandboxed language - I always was under the impression that for such uses you would need a layer like Caja.
Postponing work on Weak References due to an information leak feels to me like prioritizing theoretical security concerns over the real usability/implementation obstacles for existing applications that need a weak reference equivalent. Those applications will stay outside the domain of JS (in something like Native Client or the JVM instead) until it is available, or at best, end up as worse-performing JS applications that don't actually leverage any large part of the JS API and spec that you guys work so hard on. I have to imagine that's not the case though and these security concerns are a real pressing issue for a subset of the JS userbase I don't know about?
On Thu, Jan 31, 2013 at 4:48 PM, Kevin Gadd <kevin.gadd at gmail.com> wrote:
A search shows some old discussions of the topic mentioning that they might be going in to future versions of the language, etc. But on the other hand I've been told in response to this question before that TC39 has a general policy against features that allow garbage collection to be visible to applications.
There's still a strawman up on the wiki: strawman:weak_references and while it appears to be a relatively simple, sane way to expose weak references, it looks like the strawman hasn't been touched since late 2011. Is that because it's dead? Or was it deprioritized because weak references are believed to not be needed by JS application developers?
I ask this because the lack of weak references (or any suitable substitute mechanism) comes up regularly when dealing with the challenge of porting native apps to JavaScript, and it leads people to consider extremely elaborate workarounds just to build working applications (like storing all their data in a virtual heap backed by typed arrays and running their own garbage collector against it). If there is really a firm reason why this must be so, so be it, but seeing so many people do an end-run around the JS garbage collector only to implement their own in JavaScript makes me wonder if perhaps something is wrong. The presence of WeakMaps makes it clear to me that solving this general class of problems is on the table.
People are certainly solving this problem in other JS related contexts: TooTallNate/node-weak Historically the lack of weak references has resulted in various solutions in libraries like jQuery specifically designed to avoid cycles being created between event listeners and DOM objects. Many of these solutions are error-prone and require manual breaking of cycles.
Indeed! In the spirit of Erik Arvidsson's response above, I personally reached out to Nate on two separate occasions asking for a document of the semantics and an implementation experience write up... still waiting, but I'm sure he's a busy guy.
Earlier today I discussed with Arv an approach for doing WeakRefs without compromising security. (FWIW, I also posted this in my message near the end of groups.google.com/forum/m/?fromgroups#!topic/nodejs/fV8MDpkBauw.
Thanks to Rick for the pointer.)
When a WeakRef from Realm A points at an object from Realm A, it points weakly. When it points at an object from another Realm, it points strongly (or throws an error -- take your pick). Within a Realm, SES can do what E has always done -- treat the makeWeakRef function as privileged, not to be made available by default to non-privileged code within that Realm (much as we already do with document, window, XHR, etc). The problem is that SES is not in a position to police other Realms, and we had not known how to solve the cross-Realm problem. This restriction prevents the cross-Realm leak.
Arv and I came up with a nice refinement: when dealing with another Realm, if you can obtain the makeWeakRef for that Realm, then you can use it to point at its objects weakly. If you obtain makeWeakRefs for several Realms, perhaps it makes sense to provide a makeWeakRef combiner that makes a composite makeWeakRef function, which will point weakly at any of the Realms of the leaf makeWeakRef functions. However, I think this is a detail. Virtually all use cases of interest only need to point weakly within a Realm.
not quite. WeakMaps and WeakRefs address mostly disjoint use cases. WeakRefs cannot be used to fully emulate the GC virtues of WeakMaps on the one hand, and WeakMaps cannot be used to make an object accessible only until it is not otherwise referenced. We separated the two because the former is deterministic and can be made freely available to unprivileged code. The latter make non-deterministic GC decisions visible. Separating these abstractions clarifies both.
I do not recall anyone deciding to drop weak refs, as opposed to just postponing them. I hope to see them in ES7.
Le 31/01/2013 22:48, Kevin Gadd a écrit :
I ask this because the lack of weak references (or any suitable substitute mechanism) comes up regularly when dealing with the challenge of porting native apps to JavaScript, and it leads people to consider extremely elaborate workarounds just to build working applications (like storingall their data in a virtual heap backed by typed arrays and running their own garbage collector against it). If there is really a firm reason why this must be so, so be it, but seeing so many people do an end-run around the JS garbage collector only to implement their ownin JavaScript makes me wonder if perhaps something is wrong. The presence of WeakMaps makes it clear to me that solving this general class of problems is on the table.
I don't understand the connection between the lack of weak references and emulating a heap in a typed array.
Historically the lack of weak references has resulted in various solutions in libraries like jQuery specifically designed to avoid cycles being created between event listeners and DOM objects. Many of these solutions are error-prone and require manual breaking of cycles.
Garbage collectors have evolved and cycles aren't an issue any longer, weak references or not.
But on the other hand I've been told in response to this question before that TC39 has a general policy against features that allow garbage collection to be visible to applications.
I'm not part of TC39, but I'm largely opposed to anything that makes GC observable. It introduces a source of non-determinism; that is the kind of things that brings bugs that you observe in production, but unfortunately didn't notice and can't reproduce in development environment. Or if you observe them when running the program, you don't observe it in debugging mode.
On Fri, Feb 1, 2013 at 2:06 AM, David Bruant <bruant.d at gmail.com> wrote:
I don't understand the connection between the lack of weak references and emulating a heap in a typed array.
For an algorithm that needs weak references to be correct, the only way to implement that algorithm in JavaScript is to stop using the JS garbage collector and write your own collector. This is basically the model used by Emscripten applications compiled from C++ to JS - you can use a C++ weak reference type like boost::weak_ptr, but only because the entire application heap is stored inside of a typed array and not exposed to the JS garbage collector. This is great from the perspective of wanting near-native performance, because there are JS runtimes that can turn this into incredibly fast native assembly, but the resulting code barely looks like JavaScript and has other disadvantages, so that is why I bring it up - weakref support in JS would make it possible to express these algorithms in hand-written, readable, debuggable JS.
Garbage collectors have evolved and cycles aren't an issue any longer, weak references or not.
Cycles are absolutely an issue, specifically because JS applications can interact with systems that are not wholly managed by the garbage collector. The problem in this case is a cycle being broken too early because the application author has to manually break cycles. To present a couple simple examples:
I have a top-level application object that manages lower-level 'mode' objects representing screens in the application. The screens, when constructed, attach event listeners to the application object. Because the application manages modes, it needs to have a list of all the active modes.
- The event handler closures can accidentally (or intentionally) capture the mode object, creating a real cycle involving a dead mode that will not be collected by even the most sophisticated GC.
- If I am not extremely cautious, when a mode is destroyed I might forget (or fail) to remove its associated event handlers from the event handler list, causing the event handler lists to grow over time and eventually degrade the performance of the entire application.
- I have to explicitly decide when a mode has become dead and manually break cycles between the mode and the application, while also cleaning up any running code (or callbacks on pending operations) that rely on the mode. In this scenario, weak references are less essential but still tremendously valuable: An event handler list containing weak references would never form a cycle, and would continue to work correctly as long as the mode is alive. It is also trivial to prune 'dead' event handlers from a list of weak event handlers. The need to explicitly tag a mode as dead and break cycles (potentially breaking ongoing async operations like an XHR) goes away because any ongoing async operations will keep the object itself alive (even if it has been removed from the mode list), allowing it to be eventually collected when it is safe (because the GC can prove that it is safe).
I decide to build a simple pool allocator for some frequently used JS objects, because JS object construction is slow. This is what optimization guides recommend. I pull an object instance out of the pool and use it for a while, and return it to the pool.
- If I forget to return an object to the pool when I'm done with it, it gets collected and eventually the pool becomes empty.
- If I mistakenly return an object to the pool when it actually escaped into a global variable, object attribute, or closure, now the state of the object may get trampled over if it leaves the pool again while it's still in use.
- If I mess up my pool management code I might return the same object to the pool twice. In this scenario, weak references would allow you to make the pool implementation wholly automatic (though that would require the ability to resurrect collected objects - I'm not necessarily arguing for that feature). I should point out that this scenario is complicated by JS's lack of an equivalent to RAII lifetime management in C++ and the 'using' block in C# (you can vaguely approximate it with try/finally but doing so has many serious downsides) - given RAII or a 'using' equivalent, you could manually ref-count pool entries instead of using weakrefs. But I hope you can see the general gist here of solving a problem the GC should be solving?
These examples are simplified but are both based on real world applications I've personally worked on where the listed issues caused us real grief - crashes and leaks from buggy manual lifetime management, inferior performance, etc.
I'm not part of TC39, but I'm largely opposed to anything that makes GC observable. It introduces a source of non-determinism; that is the kind of things that brings bugs that you observe in production, but unfortunately didn't notice and can't reproduce in development environment. Or if you observe them when running the program, you don't observe it in debugging mode.
My argument here is not that non-determinism is good. My argument is that an application that runs non-deterministically in every web browser (because it's a JavaScript application) is superior to an application that deterministically doesn't run in any web browser because the application cannot be expressed accurately in JS. It is possible that the set of these applications is a small set, but it certainly seems of considerable size to me because I encounter these problems on a regular basis. The developers that I speak to who are building these applications are being forced to choose Native Client or Emscripten because their applications are not expressible in JS.
I'm personally developing a compiler that targets JS and the lack of weak references (or RAII/'using') dramatically limits the set of programs I can actually convert to JS because there are lots of applications out there that simply need this functionality. If this is something that can't be done in JS, or isn't possible until ES7/ES8, I understand, but I would be very disappointed if the only reasons for it are the hypothetical dread spectres of non-determinism and information leaks.
Thanks,
Le 01/02/2013 12:21, Kevin Gadd a écrit :
On Fri, Feb 1, 2013 at 2:06 AM, David Bruant <bruant.d at gmail.com> wrote:
I don't understand the connection between the lack of weak references and emulating a heap in a typed array. For an algorithm that needs weak references to be correct, the only way to implement that algorithm in JavaScript is to stop using the JS garbage collector and write your own collector. This is basically the model used by Emscripten applications compiled from C++ to JS - you can use a C++ weak reference type like boost::weak_ptr, but only because the entire application heap is stored inside of a typed array and not exposed to the JS garbage collector. This is great from the perspective of wanting near-native performance, because there are JS runtimes that can turn this into incredibly fast native assembly, but the resulting code barely looks like JavaScript and has other disadvantages, so that is why I bring it up - weakref support in JS would make it possible to express these algorithms in hand-written, readable, debuggable JS.
Sorry for repeating myself, but I still don't see the connection between the lack of weak references and emulating a heap in a typed array. Phrased as a question: Would it be possible to compile a C++ program in JS with weakrefs without emulating a heap in a typed array? Because of pointer arithmetics, I doubt it, but I'm curious to learn if that's the case.
Garbage collectors have evolved and cycles aren't an issue any longer, weak references or not. Cycles are absolutely an issue, specifically because JS applications can interact with systems that are not wholly managed by the garbage collector. The problem in this case is a cycle being broken too early because the application author has to manually break cycles. To present a couple simple examples:
I have a top-level application object that manages lower-level 'mode' objects representing screens in the application. The screens, when constructed, attach event listeners to the application object. Because the application manages modes, it needs to have a list of all the active modes.
- The event handler closures can accidentally (or intentionally)
Last I heard, it's very difficult to accidentally capture a reference in a closure because modern engines check which objects are actually used (looking at variable names), so for an object to be captured in a closure, it has to be used. So "intentionally".
capture the mode object, creating a real cycle involving a dead mode that will not be collected by even the most sophisticated GC.
The problem is not about cycles. It's about abusively holding references to objects.
- If I am not extremely cautious, when a mode is destroyed I might forget (or fail) to remove its associated event handlers from the event handler list, causing the event handler lists to grow over time and eventually degrade the performance of the entire application.
- I have to explicitly decide when a mode has become dead
Yes. I would say "understand" rather than "decide", but yes. And that's a very important point that most developers ignore or forget. GC is an undecidable problem, meaning that there will always be cases where a human being needs to figure out when in the object lifecycle it is not longer needed and either free it in languages where that's possible or make it collectable in languages with a GC. There will be such cases even in languages where there are weak references. Nowadays, making an object collectable means cutting all references (even if the object is not involved in a cycle!) that the mark-and-sweep algorithm (as far as I know, all modern engines use this algorithm) would traverse.
In this scenario, weak references are less essential but still tremendously valuable: An event handler list containing weak references would never form a cycle, and would continue to work correctly as long as the mode is alive. It is also trivial to prune 'dead' event handlers from a list of weak event handlers.
When does the GC decide to prune dead event handlers? randomly? Or maybe when you've performed some action meaning that the corresponding mode is dead?
The need to explicitly tag a mode as dead and break cycles (potentially breaking ongoing async operations like an XHR) goes away because any ongoing async operations will keep the object itself alive (even if it has been removed from the mode list), allowing it to be eventually collected when it is safe (because the GC can prove that it is safe).
I decide to build a simple pool allocator for some frequently used JS objects, because JS object construction is slow. This is what optimization guides recommend.
Are these guides aware of bump allocators? or that keeping objects alive more than they should pressures generational garbage collectors?
I pull an object instance out of the pool and use it for a while, and return it to the pool.
- If I forget to return an object to the pool when I'm done with it, it gets collected and eventually the pool becomes empty.
- If I mistakenly return an object to the pool when it actually escaped into a global variable, object attribute, or closure, now the state of the object may get trampled over if it leaves the pool again while it's still in use.
- If I mess up my pool management code I might return the same object to the pool twice.
I'm sorry, but all your examples are "if I forget, if i make a mistake...". I don't think making bugs are a good justification to add new features in a language. If you really care about memory, make your algorithms right, spend the necessary time to understand the lifecycle of your own objects to understand when to release them.
In this scenario, weak references would allow you to make the pool implementation wholly automatic (though that would require the ability to resurrect collected objects - I'm not necessarily arguing for that feature). I should point out that this scenario is complicated by JS's lack of an equivalent to RAII lifetime management in C++ and the 'using' block in C# (you can vaguely approximate it with try/finally but doing so has many serious downsides) - given RAII or a 'using' equivalent, you could manually ref-count pool entries instead of using weakrefs. But I hope you can see the general gist here of solving a problem the GC should be solving?
These examples are simplified but are both based on real world applications I've personally worked on where the listed issues caused us real grief - crashes and leaks from buggy manual lifetime management, inferior performance, etc.
I'm not part of TC39, but I'm largely opposed to anything that makes GC observable. It introduces a source of non-determinism; that is the kind of things that brings bugs that you observe in production, but unfortunately didn't notice and can't reproduce in development environment. Or if you observe them when running the program, you don't observe it in debugging mode. My argument here is not that non-determinism is good. My argument is that an application that runs non-deterministically in every web browser (because it's a JavaScript application) is superior to an application that deterministically doesn't run in any web browser because the application cannot be expressed accurately in JS. :-) Interesting argument.
It is possible that the set of these applications is a small set, but it certainly seems of considerable size to me because I encounter these problems on a regular basis. The developers that I speak to who are building these applications are being forced to choose Native Client or Emscripten because their applications are not expressible in JS.
I don't know enough languages to tell, but I wonder until which point should JS import other language features for the sake of porting programs. Where are the JS equivalent of Scala actors? There are probably some very interesting Scala programs to port to the web?
I'm personally developing a compiler that targets JS and the lack of weak references (or RAII/'using') dramatically limits the set of programs I can actually convert to JS because there are lots of applications out there that simply need this functionality.
ES6 introduces revokable proxies [1] which could be used to implement as "explicit weakrefs" (you need to say explicitely when you don't want to use an object anymore). One idea would be to add some source annotations to tell at a coarse level when some object is guaranteed to be not needed anymore. It would compile to revoking the proxy.
If this is something that can't be done in JS, or isn't possible until ES7/ES8, I understand, but I would be very disappointed if the only reasons for it are the hypothetical dread spectres of non-determinism and information leaks.
Each of these reasons seems to be valid to me.
David
It's not possible to polyfill or emulate weak references (or WeakMaps for that matter) in JS without completely disengaging from using the host JS engine's object model. In Continuum, for example, I can get close but not quite all the way to emulating WeakMaps using a meta-interpretive approach to the object model (each object in the VM corresponds to one or more objects in the host engine). Eventually I'll switch to switching to using my own typedarray-backed heap with a GC in order to fully realize the semantics of WeakMaps.
Le 01/02/2013 15:53, Brandon Benvie a écrit :
It's not possible to polyfill or emulate weak references (or WeakMaps for that matter) in JS without completely disengaging from using the host JS engine's object model.
For all practical purposes, you've done such a thing yourself Benvie/WeakMap ;-)
In Continuum, for example, I can get close but not quite all the way to emulating WeakMaps using a meta-interpretive approach to the object model (each object in the VM corresponds to one or more objects in the host engine). Eventually I'll switch to switching to using my own typedarray-backed heap with a GC in order to fully realize the semantics of WeakMaps.
There are probably minor limitations to your above implementation, but I wonder if they're worth the trouble to move to a typedarray heap. It's your call not mine anyway :-)
Indeed, and Continuum uses the same strategy for implementing WeakMap currently. To my knowledge, the circumstances that makes that implementation of WeakMap fail is unlikely to arise except in the case of a membrane. As a polyfill provided to developers devloping with ES5 as a target, this is acceptable since a membrane requires an implementation of Proxy. But since Continuum implements Proxy, this sets up the one case where the two (lack of native WeakMaps and Proxy) to meet.
And by fail I mean result in a set of objects that next gets collected by the GC correctly despite being otherwise fit to be collected.
David Bruant wrote:> >> David Bruant wrote:
Garbage collectors have evolved and cycles aren't an issue any longer, weak references or not.> >> > Kevin Gadd wrote: Cycles are absolutely an issue, specifically because JS applications can interact with systems that are not wholly managed by the garbage collector. The problem in this case is a cycle being broken too early because the application author has to manually break cycles. To present a couple simple examples:
I have a top-level application object that manages lower-level 'mode' objects representing screens in the application. The screens, when constructed, attach event listeners to the application object. Because the application manages modes, it needs to have a list of all the active modes.
- The event handler closures can accidentally (or intentionally)>> Last I heard, it's very difficult to accidentally capture a reference in a closure because modern engines check which objects are actually used (looking at variable names), so for an object to be captured in a closure, it has to be used. So "intentionally".
We had a situation recently where we needed to monitor an element with setInterval to get information about when it was resized or moved. As library authors we wanted to encapsulate this logic into the module so that it would "just work". We wanted someone to be able to call var widget = new Widget();, attach it to the document, and have it automatically size itself based on certain criteria. If a developer then moved its position in the document (using purely DOM means), we wanted it to resize itself automatically again. We didn't want to make a requirement to call a public resize method, nor did we want to impose dispose (it's an easy thing to forget to call and it doesn't feel like JavaScript). Of course, strongly referencing the element in the setInterval keeps it alive in memory even after the developer using the library has long since discarded it.
In this case, we managed to come up with a solution to refer to elements "weakly" through selectors, retrieving them out of the document only when they're attached (we have a single setInterval that always runs, but it allows objects to be GC'd). However, this solution is far from fool-proof, lacks integrity (any element can mimic our selectors and cause us grief), and not performant. In our case it's good enough, but I can imagine a case where it wouldn't be. I can also imagine a case where you wouldn't have the luxury to use DOM traversal as a "weak" mechanism for referring to objects. I think it could be useful internally in library components which make use of 3rd party components (in this case the DOM) to be able to monitor aspects of those components only when they're being consumed.
Having said that, I also understand the desire to keep the language deterministic and to not expose GC operations.
Nathan
Le 02/02/2013 06:41, Nathan Wall a écrit :
David Bruant wrote:
David Bruant wrote: Garbage collectors have evolved and cycles aren't an issue any longer, weak
references or not.
Kevin Gadd wrote: Cycles are absolutely an issue, specifically because JS applications can interact with systems that are not wholly managed by the garbage collector. The problem in this case is a cycle being broken too early because the application author has to manually break cycles. To present a couple simple examples:
I have a top-level application object that manages lower-level 'mode' objects representing screens in the application. The screens, when constructed, attach event listeners to the application object. Because the application manages modes, it needs to have a list of all the active modes.
- The event handler closures can accidentally (or intentionally)
Last I heard, it's very difficult to accidentally capture a reference in a closure because modern engines check which objects are actually used (looking at variable names), so for an object to be captured in a closure, it has to be used. So "intentionally".
We had a situation recently where we needed to monitor an element with
setIntervalto get information about when it was resized or moved. As library authors we wanted to encapsulate this logic into the module so that it would "just work". We wanted someone to be able to callvar widget = new Widget();, attach it to the document, and have it automatically size itself based on certain criteria. If a developer then moved its position in the document (using purely DOM means), we wanted it to resize itself automatically again. We didn't want to make a requirement to call a publicresizemethod, nor did we want to imposedispose(it's an easy thing to forget to call and it doesn't feel like JavaScript). Of course, strongly referencing the element in thesetIntervalkeeps it alive in memory even after the developer using the library has long since discarded it.
Since we're discussing the addition of a new feature, let's first start to see how existing or about-to-exist features can help us solve the same problem.
In an ES6 world, new Widget() can return a proxy and you, as the widget library author, can track down anytime the element is moved and resized (the handler will probably have to do some unwrapping, function binding, etc, but that's doable). DOM mutation observers [1] can be of some help to track down this, I think.
Hmm... It's been a couple of years that I have the intuition that events should be considered as part of an object interface and not some sort of external thing and I think the justification is right here. Doing setInterval polling has you do forces you to create a function unrelated to the object you want to observe and keeps a reference in that function. If you were able to attach an event listener to the object itself to be notified of just what you need, the observer function would die as soon as the object it's attached to would die.
In your particular case, events at the object level would solve your problem, I think.
[answering separately]
nor did we want to impose
dispose(it's an easy thing to forget to call and it doesn't feel like JavaScript)
I'd like to repeat something I wrote in another message: "...a very important point that most developers ignore or forget. GC is an undecidable problem, meaning that there will always be cases where a human being needs to figure out when in the object lifecycle it is not longer needed and either free it in languages where that's possible or make it collectable in languages with a GC. There will be such cases even in languages where there are weak references. " And when such a case will be found, what will be the solution? Adding a new subtler language construct which exposes a bit more of the GC?
JavaScript has an history of being the language of the client side where a web page lives for a couple of minutes; leaks were largely unnoticeable because navigating or closing a tab would make the content collectable (well... except in crappy version of IE in which JS content could make browser-wide leaks -_-#). As soon as we have long-running JavaScript, we have to start caring more about our memory usage, we have to question what we assumed/knew of JavaScript. The GC does maybe 80-100% of the job in well-written complex code, but we must never forget that the GC only does an approximation of an undecidable problem. In applications where memory matters a lot, maybe a protocol like .dispose will become necessary.
In this case, we managed to come up with a solution to refer to elements "weakly" through selectors, retrieving them out of the document only when they're attached (we have a single
setIntervalthat always runs, but it allows objects to be GC'd). However, this solution is far from fool-proof, lacks integrity (any element can mimic our selectors and cause us grief), and not performant. In our case it's good enough, but I can imagine a case where it wouldn't be. I can also imagine a case where you wouldn't have the luxury to use DOM traversal as a "weak" mechanism for referring to objects. I think it could be useful internally in library components which make use of 3rd party components (in this case the DOM) to be able to monitor aspects of those components only when they're being consumed.
Do you mean events? :-)
Having said that, I also understand the desire to keep the language deterministic and to not expose GC operations.
About weakrefs, I've read a little bit [2][3] and I'm puzzled by one thing: the return value of get is a strong reference, so if a misbehaving component keeps this strong reference around, having passed a weak reference was pointless.
David
[1] developer.mozilla.org/en-US/docs/DOM/MutationObserver [2] strawman:weak_references [3] weblogs.java.net/blog/2006/05/04/understanding
2013/2/2 David Bruant <bruant.d at gmail.com>
About weakrefs, I've read a little bit [2][3] and I'm puzzled by one thing: the return value of get is a strong reference, so if a misbehaving component keeps this strong reference around, having passed a weak reference was pointless.
For use cases where you're passing a reference to some plug-in/component and want the referred-to object to be eventually collected, we have revocable proxies. Weak references aren't the right tool when you want to express the guarantee that the component can no longer hold onto the object.
Le 02/02/2013 15:32, Tom Van Cutsem a écrit :
2013/2/2 David Bruant <bruant.d at gmail.com <mailto:bruant.d at gmail.com>>
About weakrefs, I've read a little bit [2][3] and I'm puzzled by one thing: the return value of get is a strong reference, so if a misbehaving component keeps this strong reference around, having passed a weak reference was pointless.For use cases where you're passing a reference to some plug-in/component and want the referred-to object to be eventually collected, we have revocable proxies. Weak references aren't the right tool when you want to express the guarantee that the component can no longer hold onto the object.
Indeed, it makes weak references a tool only useful within a trust boundary (when you don't need to share the object reference with an untrusted 3rd party).
Interestingly, revocable proxies require their creator to think to the lifecycle of the object to the point where they know when the object shouldn't be used anymore by whoever they shared the proxy with. I feel this is the exact same reflections that is needed to understand when an object isn't needed anymore within a trust boundary... seriously questioning the need for weak references.
David Bruant wrote:
Interestingly, revocable proxies require their creator to think to the lifecycle of the object to the point where they know when the object shouldn't be used anymore by whoever they shared the proxy with. I feel this is the exact same reflections that is needed to understand when an object isn't needed anymore within a trust boundary... seriously questioning the need for weak references.
Sorry, but this is naive. Real systems such as COM, XPCOM, Java, and C# support weak references for good reasons. One cannot do "data binding" transparently without either making a leak or requiring manual dispose (or polling hacks), precisely because the lifecycle of the model and view data are not known to one another, and should not be coupled.
See strawman:weak_refs intro, on the observer and publish-subscribe patterns.
Le 02/02/2013 20:02, Brendan Eich a écrit :
David Bruant wrote:
Interestingly, revocable proxies require their creator to think to the lifecycle of the object to the point where they know when the object shouldn't be used anymore by whoever they shared the proxy with. I feel this is the exact same reflections that is needed to understand when an object isn't needed anymore within a trust boundary... seriously questioning the need for weak references.
Sorry, but this is naive.
It is, you don't need to apologize.
Real systems such as COM, XPCOM, Java, and C# support weak references for good reasons. One cannot do "data binding" transparently without either making a leak or requiring manual dispose (or polling hacks), precisely because the lifecycle of the model and view data are not known to one another, and should not be coupled.
See strawman:weak_refs intro, on the observer and publish-subscribe patterns.
I guess manual dispose would make a lot of sense. A view knows own its lifecycle, it involves adding observers in a bunch of places. When the view lifecycle comes to an end for whatever reason, it only makes sense that it removes the observers it added. My rule of thumb would be "clean up the mess you made". Memory leaks are bugs. Like off-by-ones. People should just fix their bugs. Garbage collectors encourage the fantasy that people can forget about memory. It is a fantasy. A convenient one, but a fantasy nonetheless. A fantasy like "we can have a lifestyle that assumes oil is unlimited". </naivety>
<acceptance>
I guess it's just human nature, so weakrefs are pretty much unavoidable.
If a weakref to a function is passed to Object.observe, will it auto-get the function and unobserve automatically if the .get returns null?
I'd like to repeat something I wrote in another message: "...a very important point that most developers ignore or forget. GC is an undecidable problem, meaning that there will always be cases where a human being needs to figure out when in the object lifecycle it is not longer needed and either free it in languages where that's possible or make it collectable in languages with a GC. There will be such cases even in languages where there are weak references. " And when such a case will be found, what will be the solution? Adding a new subtler language construct which exposes a bit more of the GC?
JavaScript has an history of being the language of the client side where a web page lives for a couple of minutes; leaks were largely unnoticeable because navigating or closing a tab would make the content collectable (well... except in crappy version of IE in which JS content could make browser-wide leaks -_-#). As soon as we have long-running JavaScript, we have to start caring more about our memory usage, we have to question what we assumed/knew of JavaScript. The GC does maybe 80-100% of the job in well-written complex code, but we must never forget that the GC only does an approximation of an undecidable problem. In applications where memory matters a lot, maybe a protocol like .dispose will become necessary.
I agree that there are scenarios where .dispose() or a similar protocol may be come necessary; in those scenarios it would be great if JS had some mechanism that could be used to simplify the proper use of the protocol. Given that the need for Weak References in many scenarios would be reduced because a partially (or wholly) automatic implementation of that protocol can make it much easier to use the protocol correctly. As I mentioned earlier, I think 'using' in C# is a relatively simple feature that lets you express this, and function scoped values with destructors like in C++ are a great way to express this since you can increment a refcount in the constructor and decrement it in the destructor. The downsides still remain, though - refcounting (or other forms of non-GC management) add some overhead to all the code that uses objects managed with those lifetime policies. Manual refcounting requires the programmer to think about every function to understand where refs should be added/removed and adds noise to the code; explicit .dispose() without refcounting requires a programmer to have a global understanding of his/her application in order to determine the correct location to call it, which is very difficult for large applications.
About weakrefs, I've read a little bit [2][3] and I'm puzzled by one thing: the return value of get is a strong reference, so if a misbehaving component keeps this strong reference around, having passed a weak reference was pointless.
The return value is a strong reference because otherwise you'd have to write a check on every line to see if the object had expired in between statements. It'd also mean that any function you passed the weak object to would have to perform the check.
You can think of a weak reference as similar mechanically to a refcounted object: You 'acquire' a reference to the object, hold onto it a while to use the object, then 'release' the reference. In refcounting this is done with ++/-- to prevent destruction; for weak references, you do this by converting the WR temporarily into a strong reference and then discarding it.
David Bruant wrote:
A view knows own its lifecycle, it involves adding observers in a bunch of places. When the view lifecycle comes to an end for whatever reason, it only makes sense that it removes the observers it added.
The problem is that the notification comes from a model object to the view via the observer. If the view holds the model object strongly, it can entrain the entire model. And if there is an association from model to view somewhere (which is not unreasonable, in a mostly self-hosted system), then....
Some people would say that garbage collection is the most important advancement in computer science in the last 20 years.... www.codinghorror.com/blog/2009/01/die-you-gravy-sucking-pig-dog.html
Le 03/02/2013 06:21, Brandon Benvie a écrit :
Some people would say that garbage collection is the most important advancement in computer science in the last 20 years....www.codinghorror.com/blog/2009/01/die-you-gravy-sucking-pig-dog.html
Don't get me wrong, I didn't say nor didn't mean to say that garbage collectors as a tool aren't awesome. I'm very happy that most of the time, I don't need to worry about releasing memory and that in most of the remaining cases, null-ing out a single reference makes an entire subgraph collectable.
However, like any tool, I think it's crucially important to understand the limitations of it. From the article you posted:
Use your objects, and just walk away when you're done. The garbage collector will cruise by periodically, and when he sees stuff you're not using any more, he'll clean up behind you and deal with all that nasty pointer and memory allocation stuff on your behalf. It's totally automatic. "It's totally automatic". Here is someone who is not aware of the limitations of a garbage collector apparently.
Also from the article:
I view explicit disposal as more of an optimization than anything else
Nonsense. I'm sorry, but this is fueling the fantasy. Manual disposal is necessary in cases where the GC cannot make the decision. And the GC cannot make a decision because it is an algorithm, bound by decidability.
I feel there is a lot of misconception about what a GC can and cannot do. Here is an article I wrote about memory management [1]. Here is probably the most important part of it:
Release when the memory is not needed anymore
Most of memory management issues come at this phase. The hardest task here is to find when "the allocated memory is not needed any longer". It often requires for the developer to determine where in the program such piece of memory is not needed anymore and free it.
High-level languages interpreter embed a piece of software called "garbage collector" whose job is to track memory allocation and use in order to find when a piece of allocated memory is not needed any longer in which case, it will automatically free it. This process is an approximation since the general problem of knowing whether some piece of memory is needed is undecidable (can't be solved by an algorithm).
The most important part of this section is "approximation". Every GC algorithm in existence has to approximate conservatively an answer to the question "will this piece of memory be used again?". As an aside, I prefer that the GC is conservative and doesn't abusively free memory I would actually still need. In other words, memory leaks are the price to pay for the memory to be reliable.
Reference-counting garbage collection
This is the most naive garbage collection algorithm. This algorithm reduces the definition of "an object is not needed anymore" to "an object has no other object referencing to it". An object is considered garbage-collectable if there is zero reference pointing at this object.
Mark-and-sweep algorithm
This algorithm reduces the definition of "an object is not needed anymore" to "an object is unreachable". [then definition of reachable by explaining the root and the traversal]
And in both cases, I explain the limitations. I couldn't find a simple enough example to put in the documentation for the limitations of mark-and-sweep. Let's try to make up a simple example:
function Storage(){
var storage = []
return {
push(e){storage.push(e)},
last(){return storage[storage.length-1]};
}
}
var s = new Storage();
s.push({});
s.push({});
s.last();
In this example, we know, as human beings understanding the semantics of JavaScript, that all but last elements of storage could be collected. Because of its limited definition of "unreachable", the mark-and-sweep algorithm thinks these elements should be kept in memory. Some serious static analysis might figure out (as we do, human beings understanding the semantics of JavaScript) that all but the last element of storage aren't needed anymore... This analysis needs to prove at some point that Array.prototype.push is a frozen property. As a side note, as far as I know, all GC algorithms are runtime algorithms and deal with runtime "objects" and "references" and don't exploit static analysis/language-specific information.
Let's see how the example would be with weakrefs:
function Storage(){
var storage = []
return {
push(e){storage.push(makeWeakRef(e))},
last(){
var last = storage[storage.length-1]
return last.get(); // oops!
};
}
}
var s = new Storage();
s.push({});
s.push({});
s.last();
Interestingly, holding weakly the references in storage couldn't be of help here, it may be possible that the last element in the array has been GC'ed and that the call to .last doesn't return the element that was last pushed.
This example is dummy and unrealistic, but I hope shows the limitations of GCs. As I said, all GC algorithms are limited because the question "is this object still needed?" is undecidable. That's why there will always be cases where manually disposing is mandatory (and not an optimization like @codinghorror seems to think).
In any case, GC are awesome, the mark-and-sweep approximation goes a very very long way in helping developers not having to worry about memory, but like any GC algorithms existing and upcoming, its definition of "is this object no longer needed?" is limited and it's crucially important to understand this limitation. Since the GC is limited, there will always be cases where human beings need to manually do some work to tell the GC "hey, this is collectable"
I don't want to go too far in the self-promotion, but since I'm writing about GC, it seems on topic... For anyone who'd be interested, I'll be talking about that at Fluent in San Fransisco in May fluentconf.com/fluent2013/public/schedule/detail/27841
David
[1] developer.mozilla.org/en-US/docs/JavaScript/Memory_Management
On Sun, Feb 3, 2013 at 2:58 AM, David Bruant <bruant.d at gmail.com> wrote:
Let's see how the example would be with weakrefs:
function Storage(){ var storage = [] return { push(e){storage.push(makeWeakRef(e))}, last(){ var last = storage[storage.length-1] return last.get(); // oops! }; } } var s = new Storage(); s.push({}); s.push({}); s.last();
What problem is this example supposed to be solving? The problem here is not weakrefs, the problem is that the problem is poorly specified and the implementation may be inadequate: If the goal is to return the most recently pushed item that has not been collected, then last() must become a for loop that walks backwards through the storage looking for a live object (preferably pruning dead ones it encounters). This is a pattern you will see in applications sometimes - 'weak lists' that contain multiple weak elements and automatically prune out dead ones to create the illusion of a sequence only containing live objects. If the goal is to keep all the objects alive so that last always returns the last value you pushed, then you wouldn't use weakrefs here. The example usage of the Storage class makes no sense to begin with; the {}'s are effectively dead before they're even placed into storage because they're temporaries. last would only return something useful here in the event that the collector didn't take the opportunity to collect the temporaries (though to be fair, most collectors won't - they'd probably live in the young generation for a little bit).
When discussing issues as complex as garbage collection, the examples need to be at least vaguely real-world. Your example does not demonstrate a limitation of GCs because there's nothing for the GC to actually do.
Le 03/02/2013 12:08, Kevin Gadd a écrit :
On Sun, Feb 3, 2013 at 2:58 AM, David Bruant <bruant.d at gmail.com> wrote:
Let's see how the example would be with weakrefs:
function Storage(){ var storage = [] return { push(e){storage.push(makeWeakRef(e))}, last(){ var last = storage[storage.length-1] return last.get(); // oops! }; } } var s = new Storage(); s.push({}); s.push({}); s.last();What problem is this example supposed to be solving?
None, but that's beyond the point. My point was to explain that in some cases a human being can see that some objects aren't going to be needed any longer while the GC algorithm cannot. The reason is that nowadays, state-of-the-art GCs are oblivious to the code semantics.
The problem here is not weakrefs, the problem is that the problem is poorly specified
I didn't say the problem came from weakrefs, but that weakrefs don't help for this particular problem.
When discussing issues as complex as garbage collection, the examples need to be at least vaguely real-world. Your example does not demonstrate a limitation of GCs because there's nothing for the GC to actually do.
It's really hard to find real code, because it often means showing several pages of code and I don't want to force everyone reading to understand several pages of code just to make a point. Although my example is dummy, it supports the point that a GC is limited because it's oblivious to code semantics and limits the information it uses to "object", "reference" (and "root" for some objects in a mark-and-sweep). WeakRefs would just add a word in the GC vocabulary ("weak reference"), but wouldn't change the obliviousness.
On the obliviousness, here is another example. Let's say jQuery is imported with requireJS (otherwise, it's global and you can't do anything really). If you use only one jQuery function, the GC could probably figure out that only one function is used and release the rest. It doesn't, because you need a reference to the main jQuery object and the rest is attached and the GC has no understanding that you're using only one function of this library. Weakrefs cannot help with this problem. Hopefully, that's real world enough even without the code ;-) There will always be problems that weak references won't be able to solve. That's all I'm saying.
[answering to another point from yesterday]
I agree that there are scenarios where .dispose() or a similar protocol may be come necessary; in those scenarios it would be great if JS had some mechanism that could be used to simplify the proper use of the protocol. Given that the need for Weak References in many scenarios would be reduced because a partially (or wholly) automatic implementation of that protocol can make it much easier to use the protocol correctly.
The idea of a protocol is very interesting. I would love to see different deterministic protocols explored before bringing weakrefs if possible.
On Sat, Feb 2, 2013 at 11:02 AM, Brendan Eich <brendan at mozilla.com> wrote:
David Bruant wrote:
Interestingly, revocable proxies require their creator to think to the lifecycle of the object to the point where they know when the object shouldn't be used anymore by whoever they shared the proxy with. I feel this is the exact same reflections that is needed to understand when an object isn't needed anymore within a trust boundary... seriously questioning the need for weak references.
Sorry, but this is naive. Real systems such as COM, XPCOM, Java, and C# support weak references for good reasons. One cannot do "data binding" transparently without either making a leak or requiring manual dispose (or polling hacks), precisely because the lifecycle of the model and view data are not known to one another, and should not be coupled.
See strawman:weak_refs intro, on the observer and publish-subscribe patterns.
This is exactly right.
I'm preparing an implementation report on Object.observe for the next meeting, and in it I'll include findings from producing a general purpose observation library which uses Object.observe as a primitive and exposes the kind of semantics that databinding patterns are likely to need.
Without WeakRefs, observation will require a dispose() step in order to allow garbage collection of observed objects, which is (obviously) very far from ideal.
Le 02/03/2013 01:58, Rafael Weinstein a écrit :
On Sat, Feb 2, 2013 at 11:02 AM, Brendan Eich <brendan at mozilla.com> wrote:
David Bruant wrote:
Interestingly, revocable proxies require their creator to think to the lifecycle of the object to the point where they know when the object shouldn't be used anymore by whoever they shared the proxy with. I feel this is the exact same reflections that is needed to understand when an object isn't needed anymore within a trust boundary... seriously questioning the need for weak references.
Sorry, but this is naive. Real systems such as COM, XPCOM, Java, and C# support weak references for good reasons. One cannot do "data binding" transparently without either making a leak or requiring manual dispose (or polling hacks), precisely because the lifecycle of the model and view data are not known to one another, and should not be coupled.
See strawman:weak_refs intro, on the observer and publish-subscribe patterns. This is exactly right.
I'm preparing an implementation report on Object.observe for the next meeting, and in it I'll include findings from producing a general purpose observation library which uses Object.observe as a primitive and exposes the kind of semantics that databinding patterns are likely to need.
Without WeakRefs, observation will require a dispose() step in order to allow garbage collection of observed objects, which is (obviously) very far from ideal.
There is another approach taken by the requestAnimationFrame API that consists in one-time event listeners (Node.js also has that concept too [1]), requiring to re-subscribe if one wants to listen more than once. I wonder why this approach has been taken for requestAnimationFrame which is fired relatively often (60 times a second). I'll ask on public-webapps. I won't say it's absolutely better than WeakRefs and it may not apply to the data binding case (?), but it's an interesting pattern to keep in mind.
I'm looking forward to reading your findings in the meeting notes.
David
[1] nodejs.org/api/events.html#events_emitter_once_event_listener
I don't understand how the requestAnimationFrame approach (to registering periodic callbacks) applies to scenarios where you want Weak References (for lifetime management) or to observe an object (for notifications in response to actions by other arbitrary code that has a reference to an object). These seem to be significantly different problems with different constraints.
If anything, requestAnimationFrame is an example of an API that poorly expresses developer intent. It is rare for someone to actually only ever want to render a single animation frame; furthermore most animation scenarios in fact require rendering a series of frames on consistent timing. Furthermore, the need to manually trigger further frame callbacks is error-prone - you are essentially offloading the cost of lifetime management onto the application developer, by making them manually manage the lifetime of their callback on an ongoing basis by having to remember to say 'please keep my animation alive' at the right time every frame no matter what, which probably means a try block and auditing their rAF callback to ensure that all exit paths call rAF again. I suspect that if you were to look at most applications that use rAF, you would find very few of them intentionally stop running animation frames in any scenario other than the termination of the application. For this and other reasons, I would suggest that it is a horrible idea to use rAF as an example of how to design an API or solve developer problems - especially problems as important as those addressed by weak references.
On Sat, Mar 2, 2013 at 6:11 AM, Kevin Gadd <kevin.gadd at gmail.com> wrote:
I don't understand how the requestAnimationFrame approach (to registering periodic callbacks) applies to scenarios where you want Weak References (for lifetime management) or to observe an object (for notifications in response to actions by other arbitrary code that has a reference to an object). These seem to be significantly different problems with different constraints.
If anything, requestAnimationFrame is an example of an API that poorly expresses developer intent. It is rare for someone to actually only ever want to render a single animation frame; furthermore most animation scenarios in fact require rendering a series of frames on consistent timing. Furthermore, the need to manually trigger further frame callbacks is error-prone - you are essentially offloading the cost of lifetime management onto the application developer, by making them manually manage the lifetime of their callback on an ongoing basis by having to remember to say 'please keep my animation alive' at the right time every frame no matter what, which probably means a try block and auditing their rAF callback to ensure that all exit paths call rAF again. I suspect that if you were to look at most applications that use rAF, you would find very few of them intentionally stop running animation frames in any scenario other than the termination of the application. For this and other reasons, I would suggest that it is a horrible idea to use rAF as an example of how to design an API or solve developer problems - especially problems as important as those addressed by weak references.
One positive aspect about the rAF approach is that if an error occurs in the callback, the animation will stop instead of potentially leading the application into an inconsistent state and flooding the console, making debugging more painful. That said, I hardly ever use rAF directly, but instead usually a wrapper library that handles animation continuity, fps calculation etc. Perhaps rAF was meant to be consumed as a low-level API by animation libraries.
Le 02/03/2013 12:11, Kevin Gadd a écrit :
I don't understand how the requestAnimationFrame approach (to registering periodic callbacks) applies to scenarios where you want Weak References (for lifetime management) or to observe an object (for notifications in response to actions by other arbitrary code that has a reference to an object). These seem to be significantly different problems with different constraints.
It's not really about "periodic", but rather about the idea of a "one time listener". The reason I talked about 60 frames per second is that it's an event that's fired very often, so it may have a significant cost something at runtime to conditionally re-register.
The general problem is to know when someone listening actually wants to stop listening. Currently, the default of observing is "observe ad vitam æternam" which obviously causes the issue of "what if actually it's been long that we didn't want to observe?" The "one time listener" approach is interesting, because it doesn't say "observe ad vitam æternam", but rather "I'll call you only once, figure out the rest on your own" (so, re-subscribe if that's what you want). On the huge benefits of this approach, the problem of GC-ing observers is completely solved by the API. Well, not exactly. Maybe you registered, but don't care before the event happens, so your observer is garbage until called once. But after the event happens, it can be released.
In a way, a one-time listener can be seen as an auto-wrapped (wrapped by the event emitter, not the one who registered it) one-time reference (very weak reference?).
If anything, requestAnimationFrame is an example of an API that poorly expresses developer intent.
I've asked [1], we'll see. I'm not very interested in rAF specifically, because I mostly agree with you that from all the code snippets I've read and written, people re-subscribe unconditionally. Maybe some more complex applications don't.
Furthermore, the need to manually trigger further frame callbacks is error-prone - you are essentially offloading the cost of lifetime management onto the application developer
The lifetime management is on the application developer shoulders. It always has and will always be. GC and weakrefs are just conveniences to make this work (much!) easier. There are cases where a GC isn't sufficient. There will always be cases where manual disposal will be necessary. Even if the language gets WeakRefs. When a developer wraps an object in a WeakRef before handing it to observe an event, the developer is making a lifetime management decision.
For this and other reasons, I would suggest that it is a horrible idea to use rAF as an example of how to design an API or solve developer problems - especially problems as important as those addressed by weak references.
I feel misintrepreted. There is a long way from my "I won't say it's absolutely better than WeakRefs and it may not apply to the data binding case (?), but it's an interesting pattern to keep in mind." and your intrepretation of my post. I shared an idea that hadn't been shared yet on this thread. I didn't say it would solve all problems. I've actually been careful to say that it may not.
David
[1] lists.w3.org/Archives/Public/public-webapps/2013JanMar/0623.html
On 3/2/13 at 3:47 AM, bruant.d at gmail.com (David Bruant) wrote:
"I won't say it's absolutely better than WeakRefs and it may not apply to the data binding case (?), but it's an interesting pattern to keep in mind."
Speaking from ignorance here.
One advantage of the "resubscribe for every event" pattern is that if the events are generated in one process -- an animation process is the example used here -- and a message is sent to the observer in another process, and the observer is slow, the natural outcome will be to drop frames instead of queue up a lot of messages pertaining to events that no longer need to be processed.
Cheers - Bill
Bill Frantz | When it comes to the world | Periwinkle (408)356-8506 | around us, is there any choice | 16345 Englewood Ave www.pwpconsult.com | but to explore? - Lisa Randall | Los Gatos, CA 95032
I won't say it's absolutely better than WeakRefs and it may not apply to
the data binding case (?), but it's an interesting pattern to keep in mind.
I can't see how this would work in an observer/listener application. The listening object has no way of knowing if it will be deleted when the event occurs, so it cannot decide whether to resubscribe or not. If all references to it were still deleted, it would not go away. It would therefore need a shouldNotResubscribe flag, which must be set when it should be deleted. When the next event occurs it can react and decide not to resubscribe. This means that a listening object still needs a dispose method (to set the shouldNotResubscribe flag), and it also means that it would not be deleted until the next event occurs, which could be in a very long time.
Marius
I'm still balanced on whether/when/how the one-time listener pattern can be effective, but I feel like it's an interesting idea to discuss.
Le 02/03/2013 21:13, Marius Gundersen a écrit :
I won't say it's absolutely better than WeakRefs and it may not apply to the data binding case (?), but it's an interesting pattern to keep in mind.
I can't see how this would work in an observer/listener application. The listening object has no way of knowing if it will be deleted when the event occurs, so it cannot decide whether to resubscribe or not. If all references to it were still deleted, it would not go away. It would therefore need a shouldNotResubscribe flag, which must be set when it should be deleted. When the next event occurs it can react and decide not to resubscribe. This means that a listening object still needs a dispose method (to set the shouldNotResubscribe flag)
In the weakref case, someone has to keep a strong reference to the listener until it's not needed anymore. The decision to cut this last strong reference is exactly the same decision as deciding when not to re-subscribe.
I think the difference is that it might be that the last reference is cut because another object got released and this object got released because another object got released, etc. Unless the entire application is collectable, I think somewhere in the chain, there is an explicit strong reference cut. My point is that being garbage collectable needs an explicit action somewhere. WeakRef is a convenient construct that benefits from a cascading of the explicit action. But there is an explicit action needed somewhere anyway.
and it also means that it would not be deleted until the next event occurs, which could be in a very long time.
There can be a removeEventListener too.
In the weakref case, someone has to keep a strong reference to the
listener until it's not needed anymore. The decision to cut this last strong reference is exactly the same decision as deciding when not to re-subscribe.
Unless there are multiple references to an object, in which case you cannot know when the last strong reference is cut, unless you employ reference counting. With WeakRefs an object can be a listener, and can be passed around the application until it is no longer needed, at which point it is garbage collected and stops listening.
I think the difference is that it might be that the last reference is cut
because another object got released and this object got released because another object got released, etc. Unless the entire application is collectable, I think somewhere in the chain, there is an explicit strong reference cut.
Yes, this is the exact scenario I have trouble implementing, which WeakRefs would solve.
My point is that being garbage collectable needs an explicit action
somewhere. WeakRef is a convenient construct that benefits from a cascading of the explicit action. But there is an explicit action needed somewhere anyway.
There can be a removeEventListener too.
I don't see why there would be a need for explicit reattaching and explicit unattaching to an event. If a removeEventListener method exists, then you can use that method inside the listener function when you want to unsubscribe to the events, rather than (not) calling the addEventListener every time.
The explicit reattaching of the rAF works because you know how long it is until the next animation frame, but it won't work in a system where the time until the next event is indeterminable.
Marius Gundersen
David Bruant wrote:
There is another approach taken by the requestAnimationFrame API
rAF is really not analogous to a listener, it is like setTimeout or setImmediate. See lists.w3.org/Archives/Public/public-webapps/2013JanMar/0625.html and the whole thread there.
I see this is a very old thread, but I was wondering if there have been any new developments/plans?
My thoughts are this:
I think it's natural, if you're used to seeing people misuse GC, to be suspicious of weakrefs. With GC it's easy to create accidental memory leaks because you think the useful lifetime of your objects will always correspond to the lifetime of their references, when in fact they don't. My Java code always has more memory leaks than my C++ code, even though they tell me it shouldn't.
But in my experience people don't tend to misuse weakrefs. Weakrefs are tricky and awkward, so they're not a tempting feature; people don't tend to use them unless they have a specific compelling need. They're not a feature like C++ implicit conversions where everyone uses them and then shoots them in the foot. So I think the risk of adding weakrefs to the language from a code maintainability perspective are fairly low.
I think the one-time-listener pattern is really great. I've written some FRP code that uses it, and it makes a whole lot of things better, such as skipping over unnecessary calculations and handling listeners that return listeners gracefully. But it turns out it doesn't solve the memory problem unless you also have weakrefs. The reason is that it's far more common than you might think to create listeners that are never ever called. Think about opening a dialog with OK and Cancel buttons, listening on each. Every time you open the dialog, at least one listener will leak. What one-time-listeners does solve very nicely is the lifetime of control -- you can trust that the code associated with useless listeners will never be called, and never waste CPU cycles. But unfortunately the memory stays around unless you have weakrefs.
We must remember that our programming languages are Turing complete, so that everything that can be done with weakrefs can also be done without them. It might seem sort of like asking MS Paint to expose its flood-fill algorithm through an API -- because you know it's in there, and maybe you want to use it in your code and avoid duplication. Well, that would seem rather silly, asking MS Paint to expose an API -- go use your own flood-fill algorithm! And maybe that's what it seems like with the GC -- just because the GC is there, doesn't mean it has to make all its features available to the programmer to do what they like with.
It's just that it happens that implementing GC is just such a massive and tricky task, that it's just so tempting to use the GC that's already there. It can solve certain problems for you so that you don't have to solve them yourself. It doesn't solve every problem -- no software can -- but when it does solve one of your problems it solves it so beautifully. And it's just so frustrating to know the code is right there under the surface and you can't use it. Sure, maybe you should do the memory management yourself. But if you could just get at those weakrefs..... then you'd be in good shape.
In my experience there is a very small set of problems that really really benefit from weakrefs. Most of the code you write will never encounter one of those problems. But when you do hit one, it's so nice when the language supports it. You write your weakref code carefully -- because it's nondeterministic and hard to debug. You box it in real tight and get a solid core that uses weakrefs just the way you need it. Then you expose it as a library and never think about it again. You make sure the semantics of your library never expose a nondeterministic variable -- those stay on the inside.
The funny thing is that even when you keep your API fully deterministic, the fact that it can use the GC's nondeterminism on the inside changes what the external API can do. It's like how using a nonlinear electrical component can change the set of linear transfer functions you can implement at the boundary. And because everyone wants to work with nice APIs at the boundary, there's very little temptation to abuse weakrefs.
Just some things to consider.
This is probably best asked/stated here: tc39/proposal-weakrefs
Isiah Meadows me at isiahmeadows.com
Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com
Thank you for the link, I'm glad to see that people have been making progress on it.
A search shows some old discussions of the topic mentioning that they might be going in to future versions of the language, etc. But on the other hand I've been told in response to this question before that TC39 has a general policy against features that allow garbage collection to be visible to applications.
There's still a strawman up on the wiki: strawman:weak_references and while it appears to be a relatively simple, sane way to expose weak references, it looks like the strawman hasn't been touched since late 2011. Is that because it's dead? Or was it deprioritized because weak references are believed to not be needed by JS application developers?
I ask this because the lack of weak references (or any suitable substitute mechanism) comes up regularly when dealing with the challenge of porting native apps to JavaScript, and it leads people to consider extremely elaborate workarounds just to build working applications (like storing all their data in a virtual heap backed by typed arrays and running their own garbage collector against it). If there is really a firm reason why this must be so, so be it, but seeing so many people do an end-run around the JS garbage collector only to implement their own in JavaScript makes me wonder if perhaps something is wrong. The presence of WeakMaps makes it clear to me that solving this general class of problems is on the table.
People are certainly solving this problem in other JS related contexts: TooTallNate/node-weak Historically the lack of weak references has resulted in various solutions in libraries like jQuery specifically designed to avoid cycles being created between event listeners and DOM objects. Many of these solutions are error-prone and require manual breaking of cycles.
To make a controversial statement, I would suggest that there are some problems that cannot be solved in JavaScript unless it has an approximation of weak references. I would love to be proven wrong here because then I can use whatever your crazy computer science alternative is. :D
Thanks,