Discuss
  • Proxy objects and collection

Proxy objects and collection

# Daurnimator (11 years ago)

I'd like to submit a request for some sort of Proxy collection notification, so resources can be freed. This will greatly help with language interoperability.

I'm the maintainer of lua.vm.js[1], which allows Lua code to run in the browser. I've managed to construct a simple and robust cross-language bridge, so that Lua code can interact with Javascript objects. When the Lua no longer references the Javascript object, its garbage collector notifies me (via the __gc metamethod[2]). At this point, I drop the reference to the JS object, and the JS garbage collector can collect it if needed. However, currently, exposing a Lua function or object to Javascript does not have the same ability; there is no way (that I know of) to know when Javascript no longer needs your object.

Example use case: A Lua script wishes to have a Lua function called on an interval: timerid = window:setInterval(function() window.console:log("ping") end) Later, it wishes to clear the timer: window:clearInterval(timerid) This will stop the function from being called, but I have no way to notify Lua that the function can be collected.

The closest thing maturing in this direction seems to be Revocable proxies[3]. However, these still require a manual function call to revoke on the Javascript side of things.

So, I'd like to see some sort of "trap" that is fired when a Proxy is collected. To prevent over specifying how Javascript garbage collectors should operate, I propose that the trap may only be called at some undefined point after the object is not strongly referenced.

, Daurn

[1] kripken.github.io/lua.vm.js/repl.html [2] www.lua.org/manual/5.2/manual.html#2.5.1 [3] harmony:direct_proxies#revocable_proxies

# Brendan Eich (11 years ago)

Daurnimator wrote:

So, I'd like to see some sort of "trap" that is fired when a Proxy is collected. To prevent over specifying how Javascript garbage collectors should operate, I propose that the trap may only be called at some undefined point after the object is not strongly referenced.

What would you need in the way of an argument to be passed to the trap, at this later point?

# Daurnimator (11 years ago)

On 2 September 2014 14:41, Brendan Eich <brendan at mozilla.org> wrote:

Daurnimator wrote:

So, I'd like to see some sort of "trap" that is fired when a Proxy is collected. To prevent over specifying how Javascript garbage collectors should operate, I propose that the trap may only be called at some undefined point after the object is not strongly referenced.

What would you need in the way of an argument to be passed to the trap, at this later point?

I'd imagine the proxy itself (which would be this). The trap would not need to return anything.

There might be an argument here for some sort of object id instead of the proxy itself, to avoid resurrection? I'm not sure if Javascript implementations already deal with this elsewhere.

# Brendan Eich (11 years ago)

Daurnimator wrote:

On 2 September 2014 14:41, Brendan Eich <brendan at mozilla.org <mailto:brendan at mozilla.org>> wrote:

Daurnimator wrote:

    So, I'd like to see some sort of "trap" that is fired when a
    Proxy is collected.
    To prevent over specifying how Javascript garbage collectors
    should operate,
    I propose that the trap *may* only be called at some
    *undefined* point after the object is not strongly referenced.


What would you need in the way of an argument to be passed to the
trap, at this later point?

I'd imagine the proxy itself (which would be this). The trap would not need to return anything.

There might be an argument here for some sort of object id instead of the proxy itself, to avoid resurrection?

Indeed we do not want post-mortem resurrection, but any id would have the problem too, if it were strongly linked; and would have a similar problem if weak.

In terms of LuaVM, can you say very concretely what you need? A handle to the LuaVM peer of the proxy, the object the proxy reflects into JS?

I'm not sure if Javascript implementations already deal with this elsewhere.

GC is quite involved, and as you suggest, not to be exposed naively to JS "user code".

Have you seen

strawman:weak_references

which has evolved on es-discuss:

esdiscuss.org/topic/what-is-the-status-of-weak-references, esdiscuss.org/topic/es6-es7-es8-and-beyond-a-proposed-roadmap

and probably other threads I'm forgetting.

# Daurnimator (11 years ago)

On 2 September 2014 15:19, Brendan Eich <brendan at mozilla.org> wrote:

Indeed we do not want post-mortem resurrection, but any id would have the

problem too, if it were strongly linked; and would have a similar problem if weak.

Would it? If the object is freed before the trap, then the trap just gets some sort of object id. Then it can call out to the external resource manager: "hey, resource id 123 isn't needed any more". Which could then free it.

In terms of LuaVM, can you say very concretely what you need? A handle to

the LuaVM peer of the proxy, the object the proxy reflects into JS?

Currently in Lua.vm.js when a Lua object is exposed to Javascript, we place it in the registry (an associative array), under an unused integer key. This integer key is placed inside of a Javascript object. This object then has methods '.get()' '.set()' '.invoke()', etc. When one of these methods is called, it uses the integer id to look the object up in the lua registry, then performs the selected operation on that looked up value.

For the implementation, check out: kripken/lua.vm.js/blob/6a3b8fb99b84017de61f763e2541389d4d14577b/src/lua.js#L527

I'm hoping that in future, this can be re-written to use Javascript Proxys, this way, you don't need ob.get(foo) but can index the Proxy directly: ob[foo].

GC is quite involved, and as you suggest, not to be exposed naively to JS "user code".

Have you seen

strawman:weak_references

which has evolved on es-discuss:

esdiscuss.org/topic/what-is-the-status-of-weak-references, esdiscuss.org/topic/es6-es7-es8-and-beyond-a-proposed-roadmap

and probably other threads I'm forgetting.

Yeah I have, you could implement post-mortem finalizers by polling a WeakValue Map, though I don't think that's the best course of action. I thought narrowing the scope to just Proxy objects might be a bit more palatable.

The discussions around security issues are somewhat interesting, though I think allayed by my proposal of an undefined interval (i.e. possibly delayed, or random) between the reference being dropped, and the finaliser getting called.

# David Bruant (11 years ago)

Le 02/09/2014 20:07, Daurnimator a écrit :

So, I'd like to see some sort of "trap" that is fired when a Proxy is collected. To prevent over specifying how Javascript garbage collectors should operate, I propose that the trap may only be called at some undefined point after the object is not strongly referenced.

As Brendan said, what you want has been discussed as Weak References on the list, not really proxies.

The question of not wanting to over-specify upfront has come in other places in the past. Sometimes, even when the spec leaves freedom to implementors, it happens that implemetors make some common choices, then people rely on the shared browser behavior of that spec-undefined functionality. Then, the feature has to be standardized de facto as commonly implemented afterwards.

My point here being that not specifying up front does not guarantee that the details won't have to be ever specified. The enumeration order of object keys comes to mind.

I'm not saying that it's what will or even may happen in this case, but just remind that leaving things undefined can fire back and generate the opposite of what was intended.

# Brendan Eich (11 years ago)

Daurnimator wrote:

On 2 September 2014 15:19, Brendan Eich <brendan at mozilla.org <mailto:brendan at mozilla.org>> wrote:

Indeed we do not want post-mortem resurrection, but any id would have the problem too, if it were strongly linked; and would have a similar problem if weak.

Would it? If the object is freed before the trap, then the trap just gets some sort of object id. Then it can call out to the external resource manager: "hey, resource id 123 isn't needed any more". Which could then free it.

That's cool, but what if the id died on the Lua side? You'd have to avoid recycling it as another object. That makes it very much like a strong reference.

The discussions around security issues are somewhat interesting, though I think allayed by my proposal of an undefined interval (i.e. possibly delayed, or random) between the reference being dropped, and the finaliser getting called.

The trouble comes when one implementation is first or has the most market power. Then developers (intentionally or not, doesn't matter) come to depend on its implementation-defined interval.

The next-event-loop-turn thinking I remember from the weak refs threads may be ok. I don't think there's a fast path for proxies, though. Weak refs are wanted in ES7 or whatever annual edition after it that they fit in.

# Daurnimator (11 years ago)

On 2 September 2014 16:40, Brendan Eich <brendan at mozilla.org> wrote:

Daurnimator wrote:

If the object is freed before the trap, then the trap just gets some

sort of object id.

Then it can call out to the external resource manager: "hey, resource id

123 isn't needed any more".

Which could then free it.

That's cool, but what if the id died on the Lua side? You'd have to avoid

recycling it as another object. That makes it very much like a strong reference.

I don't think I understand what you mean here? Lua will hold onto the object until it is told it can let go. If (for some reason) the lua vm is destroyed, then the other traps will fail e.g. the get() trap could throw "VM not available"

I should note here that this is not unique to lua.vm.js. I could easily come up with cases where a WebWorker would want to know when the main thread was done with an object.

The discussions around security issues are somewhat interesting, though I think allayed by my proposal of an undefined interval (i.e.

possibly delayed, or random)

between the reference being dropped, and the finaliser getting called.

The trouble comes when one implementation is first or has the most market

power. Then developers (intentionally or not, doesn't matter) come to depend on its implementation-defined interval.

One of the 'tricks' I had in mind was that we already have this 'undefined' amount of time behaviour in the browser today; It just happens to be 'never'!

For the current/old use case of Javascript where the VM is regularly thrown away, these memory leaks don't matter that much. But looking to the future, knowing about collection at some undetermined point in the future would be nice :)

The next-event-loop-turn thinking I remember from the weak refs threads

may be ok. I don't think there's a fast path for proxies, though. Weak refs are wanted in ES7 or whatever annual edition after it that they fit in.

That sounds like a okay choice, but wouldn't this mean browsers have to run a full gc every time they get back to their main loop? This would seem undesirable.

Is it bad to wait and see what the browser authors do, and then just standardize on that?

# Brendan Eich (11 years ago)

Daurnimator wrote:

On 2 September 2014 16:40, Brendan Eich <brendan at mozilla.org <mailto:brendan at mozilla.org>> wrote:

Daurnimator wrote:

If the object is freed before the trap, then the trap just gets some sort of object id.

Then it can call out to the external resource manager: "hey, resource id 123 isn't needed any more".

Which could then free it.

That's cool, but what if the id died on the Lua side? You'd have to avoid recycling it as another object. That makes it very much like a strong reference.

I don't think I understand what you mean here? Lua will hold onto the object until it is told it can let go.

Oh, if you hang onto the object on the Lua side just because there's a reflecting Proxy for JS, even though there may be zero JS refs, then you'll indeed keep the id alive and not have replay bugs.

But in this case I believe you will have memory leaks, due to Lua waiting for JS GC when it could know the object lost all JS refs and then died from loss of Lua refs.

We've had these in Mozilla code using C++ and JS heaps to refer to C++ objects and their JS wrappers. We fixed them by being sufficient eager no matter which side's refs die first.

I should note here that this is not unique to lua.vm.js. I could easily come up with cases where a WebWorker would want to know when the main thread was done with an object.

Yes, and if the worker is process-isolated there is a distributed GC problem.

But when workers are threads in one process (on most browsers today?), it's easier.

That sounds like a okay choice, but wouldn't this mean browsers have to run a full gc every time they get back to their main loop? This would seem undesirable.

I think not -- GCs are not only generational but incremental -- or heading that way. So they should be able to collect often, in increments.

Is it bad to wait and see what the browser authors do, and then just standardize on that?

No, see the bad game theory I outlined. Winner takes all and spec follows that, which overspecifies GC.

# Jason Orendorff (11 years ago)

On Tue, Sep 2, 2014 at 1:07 PM, Daurnimator <quae at daurnimator.com> wrote:

I'm the maintainer of lua.vm.js[1], which allows Lua code to run in the browser. I've managed to construct a simple and robust cross-language bridge, so that Lua code can interact with Javascript objects. When the Lua no longer references the Javascript object, its garbage collector notifies me (via the __gc metamethod[2]). At this point, I drop the reference to the JS object, and the JS garbage collector can collect it if needed. However, currently, exposing a Lua function or object to Javascript does not have the same ability; there is no way (that I know of) to know when Javascript no longer needs your object.

Example use case: A Lua script wishes to have a Lua function called on an interval: timerid = window:setInterval(function() window.console:log("ping") end) Later, it wishes to clear the timer: window:clearInterval(timerid) This will stop the function from being called, but I have no way to notify Lua that the function can be collected.

If the JS function contains the only strong reference to the Lua function, then when the JS function is collected, the Lua function will be collected too. Why isn't that sufficient? Does the Lua runtime keep a list of all functions or something? What for?