Question about joined function object of ECMA-262 3rd edition
Shijun He scripsit:
function A() { function B(x) { return x*x } return B }
var b1 = A(); var b2 = A();
Spec says: b1 and b2 can be joined, implementation may make b1 and b2 the same object because [[scope]] of them have no difference.
Two call of A() will produce a function from the same FunctionBody, so they are equated and the result function objects can be joined, am I right?
It is not because b1 and b2 come from the same FunctionBody, but because the function B has no persistent state, that they can be joined (which is just a way of saying they are the same object). Because A has no local variables, B doesn't have to care that it was nested inside A.
function C(x) { function D() { return x*x } return D }
var d1 = C(1); var d2 = C(2);
Are these two call of A() also are equated uses of the same source?
No. In this case, d1 and d2 have to be different objects, because they have different persistent states: within d1, the variable x (which is free in D) is bound to 1, but within d2, the variable x is bound to 2. Thus d1 and d2 must be different objects.
In the A-B case, it is not a requirement that b1 and b2 are the same object, but interpreters are permitted to make this optimization to save on allocation. Whether any of them actually do the optimization is another question. It may be more trouble than it's worth. However, a proper ES compiler would probably want to do so.
Shijun He scripsit:
function A() { function B(x) { return x*x } return B }
var b1 = A(); var b2 = A();
Spec says: b1 and b2 can be joined, implementation may make b1 and b2 the same object because [[scope]] of them have no difference.
Two call of A() will produce a function from the same FunctionBody, so they are equated and the result function objects can be joined, am I right?
On 26/07/07, John Cowan <cowan at ccil.org> wrote:
It is not because b1 and b2 come from the same FunctionBody, but because the function B has no persistent state, that they can be joined (which is just a way of saying they are the same object). Because A has no local variables, B doesn't have to care that it was nested inside A.
A could have any number of local variables, b1 and b2 could still be joined. The important factor is that it doesn't use variables from the containing scope(s), so there is no observable difference.
Shijun He scripsit:
function C(x) { function D() { return x*x } return D }
var d1 = C(1); var d2 = C(2);
Are these two call of A() also are equated uses of the same source?
On 26/07/07, John Cowan <cowan at ccil.org> wrote:
No. In this case, d1 and d2 have to be different objects, because they have different persistent states: within d1, the variable x (which is free in D) is bound to 1, but within d2, the variable x is bound to 2. Thus d1 and d2 must be different objects.
Only because they actually use the variable x, however. They don't have to have identical scope to be joined, but they have to be outwardly indistinguishable. Any nested function which only uses variables from it's own variable object can be joined. It's also possible to join function objects in some cases where they do use variables other than from their own variable object, given that all variables used are guaranteed to be the same - while doing this joining is a memory footprint optimisation, it will make for more complex (i.e. slower) compilers however.
On 26/07/07, John Cowan <cowan at ccil.org> wrote:
In the A-B case, it is not a requirement that b1 and b2 are the same object, but interpreters are permitted to make this optimization to save on allocation. Whether any of them actually do the optimization is another question. It may be more trouble than it's worth. However, a proper ES compiler would probably want to do so.
If any implementation would chose to do the joined objects behaviour, are they not more likely to actually just use a reference to the same function object instead of joining two different objects? Joining different function objects doesn't make sense if you can actually use the same object - after all, each removal or addition of a property on a joined function object will take extra time depending on how many other function objects it's joined to, each creation of a joined function object will require one enumeration of all properties of the original function object, and each joined function object will take up memory.
However, I think the compiler complexity added for implementing this will give a negliable reduction in footprint, so most implementors haven't found it worthwile to do it.
liorean scripsit:
A could have any number of local variables, b1 and b2 could still be joined. The important factor is that it doesn't use variables from the containing scope(s), so there is no observable difference.
Yes, that is what I meant.
If any implementation would chose to do the joined objects behaviour, are they not more likely to actually just use a reference to the same function object instead of joining two different objects?
Quite so; as I said, the difference between joined and being the same is irrelevant in this context, and indeed an internal detail of the implementation.
However, I think the compiler complexity added for implementing this will give a negliable reduction in footprint, so most implementors haven't found it worthwile to do it.
So far. Eventually, someone will write an "ahead-of-time" compiler in the style of typical Lisp compilers, where such things matter.
Wait, am I following this correctly in that:
function A() { function B() {} return B; }
var x = A(); var y = A();
x.foo = 1; y.foo = 2;
alert(x.foo + y.foo);
would show "3" in current compliant implementations, but
theoretically in the future an implementation could exist that would
show "4" and still be compliant? If I'm surmising correctly, then
that seems...buggy.
On 7/27/07, Neil Mix <nmix at pandora.com> wrote:
Wait, am I following this correctly in that:
function A() { function B() {} return B; }
var x = A(); var y = A();
x.foo = 1; y.foo = 2;
alert(x.foo + y.foo);
would show "3" in current compliant implementations, but theoretically in the future an implementation could exist that would show "4" and still be compliant? If I'm surmising correctly, then that seems...buggy.
That's why I am confused...
Moreover, the spec says (13.1.2):
NOTE Two or more objects joined to each other are effectively indistinguishable except that they may have different internal properties. The only such internal property that may differ in this specification is [[Scope]].
As I understand, that means, if x and y are joined, then: x === y returns true, but x() == y() may return false, because their [[Scope]] may differ.
I also post the question in the google groups such as comp.lang.javascript and netscape.public.mozilla.jseng.
One reponse from mozilla.dev.tech.js-engine: groups.google.com/group/mozilla.dev.tech.js-engine/browse_thread/thread/16e22c189c9ea5f6/5383179969284d97#5383179969284d97
Jason Orendorff Jul 24, 1:10 am
Yes, it was a mistake. I think this "feature" is being dropped for ECMAScript 4.
On 26/07/07, Neil Mix <nmix at pandora.com> wrote:
Wait, am I following this correctly in that: [snip] var x = A(); var y = A();
x.foo = 1; y.foo = 2;
alert(x.foo + y.foo);
would show "3" in current compliant implementations, but theoretically in the future an implementation could exist that would show "4" and still be compliant? If I'm surmising correctly, then that seems...buggy.
Yes, you're following it correctly. Yes, this feature of the spec is "buggy", in the sense that you have two incompatible behaviours that are both correct according to the spec. So in a way, we're fortunate the most wide spread engines don't use joined function objects.
On 27/07/07, Shijun He <hax.sfo at gmail.com> wrote:
Moreover, the spec says (13.1.2):
NOTE Two or more objects joined to each other are effectively indistinguishable except that they may have different internal properties. The only such internal property that may differ in this specification is [[Scope]].
As I understand, that means, if x and y are joined, then: x === y returns true, but x() == y() may return false, because their [[Scope]] may differ.
x()===y() may return false the same way x()===x() may return false. Functions in JavaScript may have side effects, and the same input doesn't necessarily give the same output in cosecutive uses of the function.
Functions with different scope may be joined, but only if the scope difference will lead to no externally observable difference. For example:
function mkFn(x){
return function fn(y){
return y*y;
}
}
var
fn1=mkFn(1),
fn2=mkFn(2);
The variable x in mkFn will give no externally observable difference in fn1 and fn2, thus fn1 and fn2 may be joined. If fn did use x however, they couldn't be joined because then that would lead to an externally observable difference.
On 7/27/07, liorean <liorean at gmail.com> wrote:
x()===y() may return false the same way x()===x() may return false. Functions in JavaScript may have side effects, and the same input doesn't necessarily give the same output in cosecutive uses of the function.
No, I don't mean side effect, and there is no side effect in this case. The result may be not equal just because they have diff [[Scope]].
Functions with different scope may be joined, but only if the scope difference will lead to no externally observable difference.
Could you point out in where the spec defines such condition?
The spec only says: ...an implementation may detect when the differences in the [[Scope]] properties of two or more joined Function objects are not externally observable and in those cases reuse the same Function object rather than making a set of joined Function objects.
My impression of it is: if the differences in the [[Scope]] are externally observable, then the implementation can't reuse the same object, but can make a set of joined Function objects.
On 7/27/07, liorean <liorean at gmail.com> wrote:
x()===y() may return false the same way x()===x() may return false. Functions in JavaScript may have side effects, and the same input doesn't necessarily give the same output in cosecutive uses of the function.
On 27/07/07, Shijun He <hax.sfo at gmail.com> wrote:
No, I don't mean side effect, and there is no side effect in this case. The result may be not equal just because they have diff [[Scope]].
The function objects may not be joined if the they are dependent on different scopes. They may only be joined under the circumstance that the difference in scopes does not give a difference in observable behaviour.
On 7/27/07, liorean <liorean at gmail.com> wrote:
Functions with different scope may be joined, but only if the scope difference will lead to no externally observable difference.
On 27/07/07, Shijun He <hax.sfo at gmail.com> wrote:
Could you point out in where the spec defines such condition?
The spec only says: ...an implementation may detect when the differences in the [[Scope]] properties of two or more joined Function objects are not externally observable and in those cases reuse the same Function object rather than making a set of joined Function objects.
My impression of it is: if the differences in the [[Scope]] are externally observable, then the implementation can't reuse the same object, but can make a set of joined Function objects.
I think the meaning is rather that of: "If implementing joined functions, engines may instead simply reuse the same function object" than of "For this special case of joined functions, an engine may reuse the same function object" because of the wording of the prior and next sections.
13.1.1 Equated Grammar Productions
• Both uses obtained their FunctionBody from the same location in the
source text of the same ECMAScript program. This source text consists
of global code and any contained function codes according to the
definitions in section 10.1.2.
/.../
10.1.2 Types of Executable Code
There are three types of ECMAScript executable code:
• Global code is source text that is treated as an ECMAScript
Program. The global code of a particular Program does not include any
source text that is parsed as part of a FunctionBody.
/.../
• Function code is source text that is parsed as part of a
FunctionBody. The function code of a particular FunctionBody does not
include any source text that is parsed as part of a nested
FunctionBody. /.../
I.e. if a nested function relies on the containing function, the code it relies on is not part of either the global code or it's contained function code and in other words can't be equated.
13.2 Creating Function Objects
/ - - - /
Step 1 allows an implementation to optimise the common case of a
function A that has a nested function B where B is not dependent on A.
In this case the implementation is allowed to reuse the same object
for B instead of creating a new one every time A is called. Step 13
makes this optimisation optional; an implementation that chooses not
to implement it will go to step 2.
Note "where B is not dependent on A" and "an implementation that chooses not to implement it will go to step 2".
As I understand it, this part of the spec was written with that particular optimisation in mind. Joined objects are just a way for the specification to allow it.
Also, IMO it's one part of the spec that should be removed in ES4. It makes the semantics of certain valid ES3 programs ambiguous.
So, what the spec meant to say is: If the compiler writer can think
of an optimization that will save time or space and not break the
semantics of Javascript, they are permitted to make that
optimization. :)
On 7/30/07, P T Withington <ptw at pobox.com> wrote:
So, what the spec meant to say is: If the compiler writer can think of an optimization that will save time or space and not break the semantics of Javascript, they are permitted to make that optimization. :)
Joining is more sinister than that and several messages in this thread have approached the problem from various angles. The worst of it is that side effects on function objects are unpredictable, as Neil Mix wrote earlier. Consider:
function f() { function g() {} if (!("x" in g)) g.x = 12; g.x += 10; return g.x; }
f(); f();
Observe the inner function g. By 13.1.1 bullet 1, the two uses of the body of g are equated. Therefore by 13.2 items 1, 13, and 14, the two uses of g creates two objects that are joined: they have different internal properties, but by 13.2 act as "one object" -- side effects on one are visible on the other.
Therefore the first call to f() will return 22 but the second call may return 22 or 32 -- either is "right". This does not quite seem like a feature.
What the joining spec actually means in practical terms is that "two internal function definitions should not be used as mutable values unless you're really careful."
Normally an implementation would care about avoiding creating closures for nested functions that are only used as functions (ie they are only ever called, never used as values). There are well-known techniques for that that that do not require joining.
We haven't discussed this in the working group. My opinion: Implementations that perform the optimization will probably want to continue performing it for backwards compatibility, so the behavior may need to be brought forward for that reason. But only if there are important implementations that do perform it; I haven't tested and I haven't asked. I'm guessing the language is in there because some implementation did perform it, at the time the previous spec was written.
Input from members of the ES3 committee would be helpful at this point. The text does not appear in ES1 or ES2.
On 2007-07-31, at 07:52 EDT, Lars T Hansen wrote:
On 7/30/07, P T Withington <ptw at pobox.com> wrote:
So, what the spec meant to say is: If the compiler writer can think of an optimization that will save time or space and not break the semantics of Javascript, they are permitted to make that optimization. :)
Joining is more sinister than that and several messages in this thread have approached the problem from various angles. The worst of it is that side effects on function objects are unpredictable, as Neil Mix wrote earlier.
Indeed. I was suggesting that the spec was broken; that it meant to
prescribe an optimization to avoid unnecessary closures, but that it
got it wrong (perhaps because it overlooked the mutability of the
function object itself?). Surely backwards compatibility should not
trump correctness? You don't want to have to force users to contrive
to create a closure just to be able to add properties to a function?
Oh. Ha, ha. Now I understand a piece of code in our runtime that
did exactly that (since removed). I think the test case you are
looking for might be the SWF version 5 interpreter.
On Jul 31, 2007, at 5:41 AM, P T Withington wrote:
Indeed. I was suggesting that the spec was broken; that it meant to prescribe an optimization to avoid unnecessary closures, but that it got it wrong (perhaps because it overlooked the mutability of the function object itself?). Surely backwards compatibility should not trump correctness? You don't want to have to force users to contrive to create a closure just to be able to add properties to a function?
No, none of that (breaking backward compatibility, requiring closures
for mutability) was desired.
I wasn't around for Edition 3 except for one or two meetings (pitched
sharp variables and uneval/toSource), but I talked to Waldemar about
this at some point. The goal was to allow an optimization that would
be implementation dependent. I believe mutability was forgotten. So
we should just remove all this joined function language.
On 7/27/07, liorean <liorean at gmail.com> wrote:
Functions with different scope may be joined, but only if the scope difference will lead to no externally observable difference.
I would like to say my opinion again: though your words is very reasonable, and I believe that may be the original meaning of the authors of es3, but I can't read it from the specification. The spec only say if there is no observable diff, then the impl could not only join them but also use the same object at all.
You know english is not my native language, so may be I misunderstand it.
On 04/08/07, Shijun He <hax.sfo at gmail.com> wrote:
On 7/27/07, liorean <liorean at gmail.com> wrote:
Functions with different scope may be joined, but only if the scope difference will lead to no externally observable difference.
I would like to say my opinion again: though your words is very reasonable, and I believe that may be the original meaning of the authors of es3, but I can't read it from the specification. The spec only say if there is no observable diff, then the impl could not only join them but also use the same object at all.
You're right, the specification doesn't say it outright. The specification is made for implementators though. The implementing joined functions for more than the special case of when the difference is not externally observable is not possible without breaking the semantics of the scoping and closure mechanisms of the language.
In fact, even joined functions for the special case of no externally observable differences is tricky - if "externally observable" only refers to variable scoping, then certain valid ES3 programs will have ambiguous semantics - those programs that use global objects or the joined function object itself as storage. Both Neil Mix and Lars T Hansen gave examples which show the ambiguity of the language due to allowing joined function objects even in the case of only doing them for the case of no externally observable differences.
You know english is not my native language, so may be I misunderstand it.
No, you don't misunderstand it. I just read more between the lines than you do.
I read your post again today. And I found the key point is the definition of "Equated".
13.1.1 Both uses obtained their FunctionBody from the same location in the source text of the same ECMAScript program. This source text consists of global code and any contained function codes according to the definitions in section 10.1.2.
Q1: what is "source text of the same ECMAScript program" or "this source text"? Q2: what is global code and what is contained function codes?
Let's give a example here:
function A() { return function B(x) { return x * x; } } var b1 = A(); var b2 = A();
Is the "source text" means all code or just means "return x * x"? I think it means all codes, then the global code of it is: function A() {...} var b1 = A(); var b2 = A(); The contained function codes are: return function B(x) {...} and return x * x;
You said: "if a nested function relies on the containing function, the code it relies on is not part of either the global code or it's contained function code"
So I don't understand it. Do you mean the source text should be "return x * x"?
In my opinion, the optimisation such as closure prototype in the spidermonkey is enough and joined function optimisation is useless (and wrong). If the developer want reuse the same function object, they can write like this:
function A() { ... } A.B = function (x) { return x * x; }
On 05/08/07, Shijun He <hax.sfo at gmail.com> wrote:
I read your post again today. And I found the key point is the definition of "Equated".
Yeah, but my argument was flawed since the spec never actually says what I read out from it.
13.1.1 Both uses obtained their FunctionBody from the same location in the source text of the same ECMAScript program. This source text consists of global code and any contained function codes according to the definitions in section 10.1.2.
Q1: what is "source text of the same ECMAScript program" or "this source text"?
"this source text" refers to this: "... FunctionBody from the same location in the source text of the same ECMAScript program"
"the same location in the source text of the same ECMAScript program" refers to the source code being from the same place in the same program (or file), not different places in the same program, or from different programs.
Q2: what is global code and what is contained function codes?
I included the definition of global code and function code right after the equated grammar productions quote:
10.1.2 Types of Executable Code
There are three types of ECMAScript executable code:
• Global code is source text that is treated as an ECMAScript
Program. The global code of a particular Program does not include any
source text that is parsed as part of a FunctionBody.
/.../
• Function code is source text that is parsed as part of a
FunctionBody. The function code of a particular FunctionBody does not
include any source text that is parsed as part of a nested
FunctionBody. /.../
Global code is straightly as defined. By "any contained function codes" they refer to the function codes of the functions that are joined, and from the use of the word "any" presumably their nested function codes as well.
Let's give a example here:
function A() { return function B(x) { return x * x; } } var b1 = A(); var b2 = A();
Is the "source text" means all code or just means "return x * x"? I think it means all codes, then the global code of it is: function A() {...} var b1 = A(); var b2 = A(); The contained function codes are: return function B(x) {...} and return x * x;
You said: "if a nested function relies on the containing function, the code it relies on is not part of either the global code or it's contained function code"
So I don't understand it. Do you mean the source text should be "return x * x"?
The contained function code of B is "return x * x", yes, and the joined functions are B, but from different calls to A, so they have different [[Scope]]. Now, consider this code:
function A(y) {
return function B(x) {
return x * y;
}
}
var
b1 = A(1),
b2 = A(2);
Here the contained function code of B is "return x * y". The function bodies are in the terms of ES3 equated - their only difference is [[Scope]]. Which means that b1 and b2 formally may be joined here, but an implementation cannot reuse the same function object since the y variable from closure is used (i.e.B depends on the containing function A). The engine would have to actually implement the joining mechanism or some equivalent*.
My argument was wrong here - since y is part of A's variable object and not B's variable object, I considered it part of the containing function code (A's code). But it really isn't, so the flaw in my argument.
- Actually implementing the joining mechanism is not an optimisation by any means - I cannot see it actually improving the engine in any area, neither memory footprint, byte code size nor performance. The optimisation lies in the cases where you can skip having different objects at all, not in the case of having two objects that are always in sync. Keeping two distinct but identical objects in sync after they have been created is a net loss performance wise and footprint wise, not a net gain. Every change to a propety of one of the joined function objects have to be duplicated on the other object.
Do you mean Flash 5 ActionScript implement joined function? I have downloaded flash 5, but this old software can't be installed on win xp...
all:
First, I'm sorry because the question is mainly about es3 not es4, but I don't know other place to get a authority answer about my question. We are talking about closure in javascript, and we have some questions about ECMA-262 3rd edition: What is joined function object(ecma-262 13.1)?
The spec has given a example:
function A() { function B(x) { return x*x } return B }
var b1 = A(); var b2 = A();
Spec says: b1 and b2 can be joined, implementation may make b1 and b2 the same object because [[scope]] of them have no difference.
Two call of A() will produce a function from the same FunctionBody, so they are equated and the result function objects can be joined, am I right? What about this code:
function C(x) { function D() { return x*x } return D }
var d1 = C(1); var d2 = C(2);
Are these two call of A() also are equated uses of the same source? And can d1 anb d2 be joined in this case even their [[scope]] is different?
If they can be joined, and as the definition of joined object(13.1.2), d1 === d2 should return true. That's so strange because d1 === d2, but d1() != d2()
I've read spec several times, but still confused.
In fact, I tested many js engine and no implementation join b1 and b2 (and which will make b1 === b2 = true) as spec (except dmdscript, but it not support closure at all!). I know joining them is optional, but if there is any implementation which join them, then it and current impl may get diff result from the same code. Example:
function A() { return function () { return arguments.callee.test; }
}
var x = A(); var y = A(); x.test = 1; y.test = 2; print(x == y); print(x()); print(y());
x and y can be joined, and their [[scope]] are equal, so impl can make x and y the same object, so this code will print true, 2 and 2. But current implements choose to not join them, print false, 1 and 2.
I believe optimization should never change the semantics of a program, is joined object a mistake of the es3 spec so that no impl support this bug feature. Or, maybe I misunderstand the spec.
BTW, I know in spidermonkey b1.proto == b2.proto, it some like joined function, but they are not real joined object because b1 != b2.
Hi, all: First, I'm sorry because the question is mainly about es3 not es4, but I don't know other place to get a authority answer about my question. We are talking about closure in javascript, and we have some questions about ECMA-262 3rd edition: What is joined function object(ecma-262 13.1)? The spec has given a example: function A() { function B(x) { return x*x } return B } var b1 = A(); var b2 = A(); Spec says: b1 and b2 can be joined, implementation may make b1 and b2 the same object because [[scope]] of them have no difference. Two call of A() will produce a function from the same FunctionBody, so they are equated and the result function objects can be joined, am I right? What about this code: function C(x) { function D() { return x*x } return D } var d1 = C(1); var d2 = C(2); Are these two call of A() also are equated uses of the same source? And can d1 anb d2 be joined in this case even their [[scope]] is different? If they can be joined, and as the definition of joined object(13.1.2), d1 === d2 should return true. That's so strange because d1 === d2, but d1() != d2() I've read spec several times, but still confused. In fact, I tested many js engine and no implementation join b1 and b2 (and which will make b1 === b2 = true) as spec (except dmdscript, but it not support closure at all!). I know joining them is optional, but if there is any implementation which join them, then it and current impl may get diff result from the same code. Example: function A() { return function () { return arguments.callee.test; } } var x = A(); var y = A(); x.test = 1; y.test = 2; print(x == y); print(x()); print(y()); x and y can be joined, and their [[scope]] are equal, so impl can make x and y the same object, so this code will print true, 2 and 2. But current implements choose to not join them, print false, 1 and 2. I believe optimization should never change the semantics of a program, is joined object a mistake of the es3 spec so that no impl support this bug feature. Or, maybe I misunderstand the spec. BTW, I know in spidermonkey b1.__proto__ == b2.__proto__, it some like joined function, but they are not real joined object because b1 != b2. -- hax