return when desugaring to closures
Lex Spoon wrote:
(function(x, y) { print(y); })(2, x) We've been over this. What if you replace print(y) with return y?
Desugaring to closures is highly worthwhile, so it looks worth solving these. Defining good variable-definition semantics is hard, as the experience with var shows, so it's good to reuse function literals if at all possible.
The problems you describe are actually not hard to fix. All that has to happen is that return is labeled with the function it applies to. If at parse time no label is supplied, then the parser can manufacture a label to use.
The simplest implementation is that if whenever you create a function object with a return in it, you record in the object the stack frame that that return would apply to. This stack frame will always be either the current stack frame, or one that was passed in from a surrounding frame. When a return actually happens, you can then pop the stack until you get to that frame, or else signal an error if that frame has already popped.
Common Lisp works this way, and as far as I know it has worked out nicely. Scala faced this exact problem, and it added non-local returns to address them. Smalltalk has non-local returns, presumably because it, too, uses a lot of nested anonymous functions, and they work well there, too.
So, I agree that return, etc., need to be fixed up if you desugar to closures, but they can indeed be fixed up. Further, you probably want non-local return anyway if people are going to use a lot of function literals in their code.
Likewise for break and continue (which are already labeled), and arguments (which could certainly be labeled).
and 'var'? (See the discussion of let-statements in the other thread.)
Waldemar
On Thu, Aug 21, 2008 at 11:09 AM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 21, 2008, at 10:50 AM, Lex Spoon wrote:
Desugaring to closures is highly worthwhile,
Why?
When it doesn't work, it doesn't work. But when it does work, it's highly worthwhile because it's the closest thing we've got to proper lambda abstraction.
On Thu, Aug 21, 2008 at 11:24 AM, Mark S. Miller <erights at google.com> wrote:
When it doesn't work, it doesn't work. But when it does work, it's highly worthwhile because it's the closest thing we've got to proper lambda abstraction.
... and to my mind, the benefit of (defining the semantics as) desugaring to something well-understood (be it proper lambda abstraction or otherwise) is that the primitives of the language are now more amenable to formal -- or even semi-formal -- reasoning.
From our point of view in trying to build secure JavaScript variants,
what I consider to be a huge breakthrough moment was when MarkM realized that we can support existing JavaScript patterns (fondly known as "monkey patching") by essentially desugaring one JavaScript flavor to another JavaScript flavor, and doing all our security reasoning about the second flavor.
And I'm harping on security -- and support for clear reasoning this entails -- because I do believe it is crucial. Our work on the Caja group is not unique; there are several JavaScript dialects out there all trying to support the ubiquitous sharing of active content. The browser's same origin policy no longer works for the brave new world of social networks; gadgets; content aggregators; active advertisements; and what not.
"Language as UI" issues are important, yes, but the most important thing is to be very careful and minimalistic about any new foundations that EcmaScript drives into in the ground -- ones that cannot be built on top of existing foundations. These have proven to be impossible to pull up again.
Ihab
On Aug 21, 2008, at 11:24 AM, Mark S. Miller wrote:
On Thu, Aug 21, 2008 at 11:09 AM, Brendan Eich
<brendan at mozilla.org> wrote:On Aug 21, 2008, at 10:50 AM, Lex Spoon wrote:
Desugaring to closures is highly worthwhile,
Why?
When it doesn't work, it doesn't work.
Glad to hear it.
But when it does work, it's highly worthwhile because it's the closest thing we've got to proper lambda abstraction.
But it's not, for the reasons just cited by Waldemar and me. We can
either limp along (strict mode is not enough) or add better forms.
That choice is an open one in Harmony, with a bias toward conserving
existing runtime semantics, but not at the price of obfuscation or
integrity bugs.
On Aug 21, 2008, at 11:45 AM, ihab.awad at gmail.com wrote:
On Thu, Aug 21, 2008 at 11:24 AM, Mark S. Miller
<erights at google.com> wrote:When it doesn't work, it doesn't work. But when it does work, it's highly worthwhile because it's the closest thing we've got to proper lambda abstraction.
... and to my mind, the benefit of (defining the semantics as) desugaring to something well-understood (be it proper lambda abstraction or otherwise) is that the primitives of the language are now more amenable to formal -- or even semi-formal -- reasoning.
We have experienced people on the committee who've formalized JS
subset semantics (Dave Herman, Cormac Flanagan). But the problem I
keep hammering on is that JS function != Lambda.
From our point of view in trying to build secure JavaScript variants, what I consider to be a huge breakthrough moment was when MarkM realized that we can support existing JavaScript patterns (fondly known as "monkey patching") by essentially desugaring one JavaScript flavor to another JavaScript flavor, and doing all our security reasoning about the second flavor.
This is common practice in the PLT world, right?
And I'm harping on security -- and support for clear reasoning this entails -- because I do believe it is crucial. Our work on the Caja group is not unique; there are several JavaScript dialects out there all trying to support the ubiquitous sharing of active content. The browser's same origin policy no longer works for the brave new world of social networks; gadgets; content aggregators; active advertisements; and what not.
We've all heard this in committee, somewhat to excess. The solution
does not, in my opinion, preclude better semantics for new forms.
"Language as UI" issues are important, yes, but the most important thing is to be very careful and minimalistic about any new foundations that EcmaScript drives into in the ground -- ones that cannot be built on top of existing foundations. These have proven to be impossible to pull up again.
You don't have to tell me. I've been around this block more than
anyone now involved, starting in May 1995.
I think you're fighting the last war. There's a risk in not adopting
what implementations have already proven, beyond what was written
down (with lots of errata) in 1999.
At best, your position is yin to my yang. I don't think it's a trump
card.
[ ... agreed with the preceding ... ]
On Thu, Aug 21, 2008 at 12:09 PM, Brendan Eich <brendan at mozilla.org> wrote:
At best, your position is yin to my yang.
Anything different would be worse.
Ihab
I should add --
On Thu, Aug 21, 2008 at 12:09 PM, Brendan Eich <brendan at mozilla.org> wrote:
We've all heard this in committee, somewhat to excess. ...
Not having been there, I cannot comment on the quantitative aspects :) but: it's never too strongly emphasized for me, and I did not start out as a security person. I just want to build better UIs that people can understand, and these are built using objects not just plain old data. Getting the objects to behave themselves is therefore the priority that I see.
But again: I agree there are two sides to this, and I'm happy that this is the case.
On Aug 21, 2008, at 2:29 PM, Lex Spoon wrote:
(- es-discuss, which I'm not on)
Why not join? es3.x-discuss is not the place if its purpose is
focused discussion about the ES3.1 spec work that's before TC39. This
kind of thread belongs in es-discuss, so I'm cross-posting with reply-
to set.
On Thu, Aug 21, 2008 at 2:45 PM, <ihab.awad at gmail.com> wrote:
On Thu, Aug 21, 2008 at 11:24 AM, Mark S. Miller
<erights at google.com> wrote:When it doesn't work, it doesn't work. But when it does work, it's highly worthwhile because it's the closest thing we've got to proper lambda abstraction.
... and to my mind, the benefit of (defining the semantics as) desugaring to something well-understood (be it proper lambda abstraction or otherwise) is that the primitives of the language are now more amenable to formal -- or even semi-formal -- reasoning.
I would add that there is a long history of variable-definition forms in PLs that look okay at first but turn out to be bad to program with. JavaScript's var is just one example. The dynamically bound variables of Lisp haunted that community for decades.
It's uncanny how people keep reinventing Lisp, compleat with dynamic
scope. In the nineties, Perl, TCL, and JS all flirted with disaster.
The maddening mix of dynamic scope (the hopeless global top-level,
with, eval) and lexical scope (with warts like hoisting of vars,
along with better-motivated let rec bindings of function definitions)
in JS is something I believe the committee should address in Harmony.
Doing it compatibly and compositionally will require new forms that
opt into true lexical scope.
This problem is especially disturbing because the problems are so subtle. Once you've convinced yourself that JavaScript's var is problematic, it's really hard to summarize the argument concisely. You can show coding examples where programmers get surprised, but many will respond and simply say don't do that. You can show term-rewriting rules that should be but aren't equivalences, but people's eyes glaze over. I can give a rule of thumb, though, for avoiding these problems to begin with: go with an existing design. The desugaring of lets into functions and function calls is not some obscure new invention, but a common one that has stood the test of time well.
No, it hasn't, because you are talking about JS functions, with |
this|, arguments, and so on. I don't know why you changed the
argument from decrying var binding to treating JS functions as if
they were well-behaved lambda expresions or Scheme procedures.
Also, Brendan, please be aware that the languages I listed are not some obscure collection of random languages.
Hey, did I just fall off the turnip truck?
Please assume knowledge of the languages you cite, and deal with the
point I made in reply: these do not all bear on any evolution of JS
that's backward-compatible and not a Frankenstein's monster language
(like Objective C, say).
I fully agree that different languages aren't necessarily comparable to each other. However, those I named have the same semantics as JavaScript for the relevant features: nested functions, function calls, parameters, and the proposed let-bound variables.
No, they do not. Common Lisp, Scala, and Smalltalk do not have the
same semantics in detail as JS does with respect to functions, calls,
parameters, and any let proposal I've seen or implemented.
If you look from 100,000 feet, you could say they're all similar, but
they are not the same. Just one example: Smalltalk has block objects,
quoted code you can activate by sending a message. You could say
that's just like a function, except where it is quite different
(c2.com/cgi/wiki?SmalltalkBlockReturn). Never mind JS
functions having variable arity, unbound |this| parameterized by call
expression, arguments object, etc.
Arguing by authority is a fallacy. We need specific solutions from
other languages that could compatibly fit into JS as extensions. I
welcome such specific proposals. The committee has before it the let-
as-better-var proposal, which was favorably received by many in Oslo.
The approach you are saying won't work, is in fact the chosen and working solution for other languages that have gone this route. My understanding is that programmers using these languages have been happy with the arrangement.
I don't know what you mean by "The approach". Can you be concrete?
Brendan Eich wrote:
Arguing by authority is a fallacy. We need specific solutions from
other languages that could compatibly fit into JS as extensions. I
welcome such specific proposals. The committee has before it the let- as-better-var proposal, which was favorably received by many in Oslo.
Lex Spoon wrote:
The approach you are saying won't work, is in fact the chosen and working solution for other languages that have gone this route. My understanding is that programmers using these languages have been happy with the arrangement.
We all know about the benefits of specification by desugaring. (I do
know a thing or two about macros, ya know.) But desugaring only works
when the feature you want really does layer properly on top of an
existing feature. The simple reason we haven't specified let' as a desugaring intofunction' is because we want them to behave differently.
Brendan Eich wrote:
The maddening mix of dynamic scope (the hopeless global top-level,
with, eval) and lexical scope [...]
'with' is lexically scoped. groups.google.com/group/google-caja-discuss/browse_thread/thread/edc12e29e2cfba9f/c2ea493e83a6bf1f?lnk=gst#c2ea493e83a6bf1f
(Please don't interpret this as a defence of 'with'.)
On Aug 21, 2008, at 4:05 PM, David-Sarah Hopwood wrote:
Brendan Eich wrote:
The maddening mix of dynamic scope (the hopeless global top-level, with, eval) and lexical scope [...]
'with' is lexically scoped. <groups.google.com/group/google-caja-discuss/browse_thread thread/edc12e29e2cfba9f/c2ea493e83a6bf1f?lnk=gst#c2ea493e83a6bf1f>
There, you wrote
"The first example illustrates (but does not prove) that 'with' actually provides lexical scoping, provided that all free identifiers in the scope of the 'with' exist as properties of the supplied scope object. "
The provided is not guaranteed by 'with', so if it's not satisfied,
then by your own argument 'with' does not provide lexical scoping.
Perhaps you meant to write "'with' may but not must be lexically
scoped"?
Consider this counter-example:
var opt_A = false, opt_B = 0, opt_C = ""; // defaults
function setOptions(opts) { with (opts) { if (opt_A) setOptionA(true); if (opt_B) setOptionB(opt_B); if (opt_C) setOptionC(opt_C); } } setOptions({opt_A: true, opt_C: "hi"}); setOptions({opt_B: 42, opt_C: "bye"});
I'm not recommending this kind of code, but IIRC Lars Hansen pointed
out the pattern, based on sightings in the wild.
Is this "dynamic scope" in the classical sense? No, but it smells as
bad and has similar undesirable properties ;-).
These old docs may be worth a read:
clarification:lexical_scope, proposals:reformed_with
(Please don't interpret this as a defence of 'with'.)
No worries.
Dave Herman wrote:
We all know about the benefits of specification by desugaring. (I do know a thing or two about macros, ya know.) But desugaring only works when the feature you want really does layer properly on top of an existing feature. The simple reason we haven't specified
let' as a desugaring intofunction' is because we want them to behave differently.
Who is "we"?
Brendan seemed to invite discussion about the semantics of let.
If the committee has reached consensus then those outside the committee could save time and keystrokes and focus on other things where there is need.
How do outsiders know which proposals have reached committee consensus?
Peter
On Aug 21, 2008, at 5:39 PM, Peter Michaux wrote:
Dave Herman wrote:
We all know about the benefits of specification by desugaring. (I do know a thing or two about macros, ya know.) But desugaring only works when the feature you want really does layer properly on top of an existing feature. The simple reason we haven't specified
let' as a desugaring intofunction' is because we want them to behave
differently.Who is "we"?
That is clear from context: "we haven't specified ...".
Brendan seemed to invite discussion about the semantics of let.
I was trying to get comments from users of JS1.7's let forms. The
keyword argument was not interesting to me. But this is es-discuss,
we can talk about whatever we want to do with ECMAScript futures.
If the committee has reached consensus then those outside the
committee could save time and keystrokes and focus on other things where
there is need.
Rather than change the topic to complain about something that hasn't
happened (no ES4 spec yet), why not respond to the substantive point:
let != function-expression-call?
How do outsiders know which proposals have reached committee
consensus?
I said that many on the committee were in favor, notionally and in
some details, of much of JS1.7's "convenience features", and some
beyond:
let as new var destructuring assignment and binding forms generators (modulo keyword issue) expression closures spread operator rest parameters optional parameters
So not everything is a blank slate in TC39 for ES-Harmony, nor should
it be.
On Aug 21, 2008, at 5:48 PM, Brendan Eich wrote:
On Aug 21, 2008, at 5:39 PM, Peter Michaux wrote:
How do outsiders know which proposals have reached committee consensus?
I said that many on the committee were in favor, notionally and in some details, of much of JS1.7's "convenience features", and some beyond:
let as new var destructuring assignment and binding forms generators (modulo keyword issue) expression closures spread operator rest parameters optional parameters
Of course, consensus (meaning "general agreement") and "many ... were
in favor" are not the same thing. It's not over till the spec is
frozen by Ecma well in advance of a General Assembly vote, and then
on to ISO. Still, it's important to build consensus where possible,
not throw everything open. That's what I meant by "not everything is
a blank slate."
You mentioned how consensus could be spoiled easily in as large a
committee as we have, and you're right: that's a risk. I hope we'll
avoid it. If we get stuck, we'll have to back away from the sticky
proposal. Some members may implement it to prove fitness and test
against real developers (Decimal should be done this way, IMHO).
If all proposals get stuck, I'll cry shenanigans, because I do not
believe that the strong technical contributors we have on committee,
who all share values around lexical scope and dynamic typing, will
get stuck without someone putting glue on chairs -- if you get what I
mean :-/.
On Thu, Aug 21, 2008 at 5:48 PM, Brendan Eich <brendan at mozilla.org> wrote:
I said that many on the committee were in favor, notionally and in some details, of much of JS1.7's "convenience features", and some beyond:
let as new var destructuring assignment and binding forms generators (modulo keyword issue) expression closures spread operator rest parameters optional parameters
I am in favor of all these including "destructuring binding forms". But I don't know what "destructuring assignment" is. I'm also in favor of some kind of class sugar and some kind of dynamic type-like annotation on variables, properties, and function return values.
The two I'm most nervous about are generators and the type-like annotations. Since generators are shallow, it seems clear that there should be some local cps-style transform of generators into the remainder of the language. However, it would be a relief to know what that cps-style transform actually looks like, and what edge cases it has. Anyone care to post a proposal?
Regarding type-like annotations, I think the proposal at harmony:types is heading in a
good direction.
On Aug 21, 2008, at 6:19 PM, Mark S. Miller wrote:
I am in favor of all these including "destructuring binding forms". But I don't know what "destructuring assignment" is.
Same thing without the binding keyword (let, var, const):
js> function foo() { return [1,2,3] }
js> let [a,b,c] = foo()
js> print(a,b,c)
1 2 3 function print() { [native code] } js> [d,e,f] = foo()
1,2,3
The destructuring pattern here is an array, but objects work too:
js> let {length: len} = foo()
js> len
3
The assignment version of the above requires parentheses, to avoid
the { being taken for start of block statement:
js> ({length:len} = foo())
1,2,3
You could parenthesize just the left-hand side, of course.
There's a shorthand, familiar to SML users, for the common case where
you want the property name and the binding name to be the same:
js> function bar() { return {p:42, q:"hi"} }
js> let {p, q} = bar()
js> p
42 js> q
hi
The destructuring patterns work in formal parameter position and
catch clauses too. They nest too, so you can extract deep property
values and bind them to variable names.
Our experience in JS1.7 and 1.8 is that destructuring is extremely
pleasant and popular, with no nasty surprises (not even the
parenthesization requirement for object patterns in destructuring
assignment).
Brendan Eich wrote:
On Aug 21, 2008, at 4:05 PM, David-Sarah Hopwood wrote:
Brendan Eich wrote:
The maddening mix of dynamic scope (the hopeless global top-level, with, eval) and lexical scope [...]
'with' is lexically scoped. groups.google.com/group/google-caja-discuss/browse_thread/thread/edc12e29e2cfba9f/c2ea493e83a6bf1f?lnk=gst#c2ea493e83a6bf1f
There, you wrote
"The first example illustrates (but does not prove) that 'with' actually provides lexical scoping, provided that all free identifiers in the scope of the 'with' exist as properties of the supplied scope object. "
The provided is not guaranteed by 'with', so if it's not satisfied, then by your own argument 'with' does not provide lexical scoping.
My statement in that post was imprecise: 'with' always provides lexical scoping for all of the identifiers that it affects. It does not affect the scoping of identifiers that do not exist as properties of the scope object: those have the same scoping as they would have had without the 'with'.
Of course if that set of properties changes, then the effect on the scope is "dynamic" in some sense, but that's still not "dynamic scoping".
I made the "provided" caveat just because at that point I hadn't done sufficient testing and careful analysis of the spec to confirm that 'with' actually is always lexically scoped. But the design of Jacaranda (the object-capability subset I'm working on) depends on that, so I certainly hope it's correct.
In your example below, the opt_* variables are globally scoped in the case where they are not affected by the 'with'. The global scoping is not fully lexical (and I fully agree with your dismissal of the global scope semantics as "hopeless"), but it isn't the 'with' construct that is causing that.
Perhaps you meant to write "'with' may but not must be lexically scoped"?
Consider this counter-example:
var opt_A = false, opt_B = 0, opt_C = ""; // defaults
function setOptions(opts) { with (opts) { if (opt_A) setOptionA(true); if (opt_B) setOptionB(opt_B); if (opt_C) setOptionC(opt_C); } } setOptions({opt_A: true, opt_C: "hi"}); setOptions({opt_B: 42, opt_C: "bye"});
This is lexical scoping with conditional shadowing. If it were dynamic, then you'd expect:
function setOptionB(x) { print(x == opt_B); }
setOptions({opt_B: 42});
to print true, but it actually prints false.
I'm not recommending this kind of code, but IIRC Lars Hansen pointed out the pattern, based on sightings in the wild.
Is this "dynamic scope" in the classical sense? No, but it smells as bad and has similar undesirable properties ;-).
Oh, I'm certainly not disputing the latter. My discussion of 'with' is made in the spirit of "know your enemy".
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org> wrote:
js> [d,e,f] = foo() 1,2,3
Is this a simultaneous assignment of d,e,f, or does it declare variables d,e,f?
If the first, can you do a swap as
[d, e] = [e, d]
?
On Thu, Aug 21, 2008 at 5:48 PM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 21, 2008, at 5:39 PM, Peter Michaux wrote:
Dave Herman wrote:
We all know about the benefits of specification by desugaring. (I do know a thing or two about macros, ya know.) But desugaring only works when the feature you want really does layer properly on top of an existing feature. The simple reason we haven't specified
let' as a desugaring intofunction' is because we want them to behave differently.Who is "we"?
That is clear from context: "we haven't specified ...".
Is there a list of individuals those who will ratify the final proposal? It isn't apparent reading emails on this list who is on the committee and who is not.
[snip]
If the committee has reached consensus then those outside the committee could save time and keystrokes and focus on other things where there is need.
Rather than change the topic to complain
I don't mean it as a complaint. It is simply a matter of efficiency.
about something that hasn't happened (no ES4 spec yet), why not respond to the substantive point: let != function-expression-call?
What I was trying to determine is if the "we want them to behave differently" means there is already consensus that let should not desugar to a function-expression call. If that consensus exists then there really is nothing to be gained by outsiders chipping in comments and doing so is just lost time and energy.
I think a let expression could work either as a function-expression call or not. It makes sense to me that let desugars to a function-expression call because those are known semantics. I really don't think the arguments, this, break, continue, return issues are that big a deal but I can see someone else might think differently. If let expressions don't desugar to a function-expression call we will all survive.
Peter
On Aug 21, 2008, at 6:45 PM, David-Sarah Hopwood wrote:
In your example below, the opt_* variables are globally scoped in the case where they are not affected by the 'with'. The global scoping is not fully lexical (and I fully agree with your dismissal of the global scope semantics as "hopeless"), but it isn't the 'with' construct that is causing that.
Wrap my example in a function and then what kind of scoping do you have?
This is lexical scoping with conditional shadowing. If it were
dynamic, then you'd expect:function setOptionB(x) { print(x == opt_B); }
setOptions({opt_B: 42});
to print true, but it actually prints false.
I don't see how. Dynamic scope means the caller's environment is
visible to the callee. But an object initialiser passed as an actual
argument does not bind any of its properties in the caller's
environment.
I've never heard of lexical scoping with conditional shadowing.
Lexical scoping as I understand it means the binding environment can
be judged statically (at compile time -- interesting contrast with
dynamic typing, but one does find the two paired often). Is there a
programming language of note with lexical scoping with conditional
shadowing?
On Aug 21, 2008, at 6:47 PM, Mark S. Miller wrote:
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org>
wrote:js> [d,e,f] = foo() 1,2,3
Is this a simultaneous assignment of d,e,f, or does it declare
variables d,e,f?
The first.
If the first, can you do a swap as
[d, e] = [e, d]
Sure can (variable values as in previous mail):
js> [a,b] = [b,a]
2,1 js> a
2 js> b
1 js> [d,e,f] = [e,f,d]
2,3,1 js> d
2 js> e
3 js> f
1
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org> wrote:
[snip]
The destructuring pattern here is an array, but objects work too:
js> let {length: len} = foo()
The above has always instinctively looked to me like the variable "length" is being assigned the value of the "len" property of the object returned by foo. That is the way an object literal works: the right value is assigned to the left property.
[snip]
Peter
On Thu, Aug 21, 2008 at 6:54 PM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 21, 2008, at 6:47 PM, Mark S. Miller wrote:
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org> wrote:
js> [d,e,f] = foo() 1,2,3
Is this a simultaneous assignment of d,e,f, or does it declare variables d,e,f?
The first.
If the first, can you do a swap as
[d, e] = [e, d]
Sure can (variable values as in previous mail):
Nice. I don't see any problems with that. With the caveat that I've only thought about it for less than an hour, I'm in favor.
On Aug 21, 2008, at 7:04 PM, Peter Michaux wrote:
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org>
wrote:[snip]
The destructuring pattern here is an array, but objects work too:
js> let {length: len} = foo()
The above has always instinctively looked to me like the variable "length" is being assigned the value of the "len" property of the object returned by foo. That is the way an object literal works: the right value is assigned to the left property.
Probably not instinct (no survival advantage ;-); possibly lack of
familiarity with the new form in context. Try using it in more
realistic code for a few days (especially the shorthand form) and
ping me if it still reads like an object initialiser.
On Thu, Aug 21, 2008 at 7:04 PM, Peter Michaux <petermichaux at gmail.com> wrote:
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org> wrote:
The destructuring pattern here is an array, but objects work too:
js> let {length: len} = foo()
The above has always instinctively looked to me like the variable "length" is being assigned the value of the "len" property of the object returned by foo. [...]
Not me. For me, the correct reading was the intuitive one. Did anyone else find this confusing?
On Thu, Aug 21, 2008 at 7:08 PM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 21, 2008, at 7:04 PM, Peter Michaux wrote:
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org> wrote:
[snip]
The destructuring pattern here is an array, but objects work too:
js> let {length: len} = foo()
The above has always instinctively looked to me like the variable "length" is being assigned the value of the "len" property of the object returned by foo. That is the way an object literal works: the right value is assigned to the left property.
Probably not instinct (no survival advantage ;-); possibly lack of familiarity with the new form in context. Try using it in more realistic code for a few days (especially the shorthand form) and ping me if it still reads like an object initialiser.
It works in practice as context of syntax sometime matters.
This is definitely an example of the same syntax having semantically different meaning (actually opposite meanings) which was an objection to the use of return in an let statement if a let statement desugars to a function-expression call.
Peter
Brendan Eich wrote:
On Aug 21, 2008, at 6:45 PM, David-Sarah Hopwood wrote:
In your example below, the opt_* variables are globally scoped in the case where they are not affected by the 'with'. The global scoping is not fully lexical (and I fully agree with your dismissal of the global scope semantics as "hopeless"), but it isn't the 'with' construct that is causing that.
Wrap my example in a function and then what kind of scoping do you have?
Lexical with conditional shadowing.
This is lexical scoping with conditional shadowing. If it were dynamic, then you'd expect:
function setOptionB(x) { print(x == opt_B); }
setOptions({opt_B: 42});
to print true, but it actually prints false.
I don't see how. Dynamic scope means the caller's environment is visible to the callee.
Yes, exactly. In your example, the caller's environment is not visible to the callee for any of the calls.
But an object initialiser passed as an actual argument does not bind any of its properties in the caller's environment.
No, but the 'with' hypothetically would, if it provided dynamic scoping. It's quite easy to imagine a 'with'-like construct that could work in this way, in a language that was otherwise dynamically scoped, or if the use of 'with' were to insert a scope in the lexical chain not dependent on the with's lexical position. (Yes, that would be awful.)
I've never heard of lexical scoping with conditional shadowing.
I just made up the term, but it's as good a description as any.
Lexical scoping as I understand it means the binding environment can be judged statically (at compile time -- interesting contrast with dynamic typing, but one does find the two paired often).
The distinction between lexical and dynamic scoping is (for the definitions I prefer, anyway) decided by the algorithm that is used to search for bindings:
- for pure lexical scoping, look for a potential binding in the immediate lexically surrounding scope, then the next surrounding scope, and so on.
- for pure dynamic scoping, look in the current activation record, then its caller's activation record, and so on; and finally the global scope if any.
In ECMAScript, the search for bindings is lexical. The reason why that doesn't imply static analysability (besides other reasons like use of 'eval') is that when the search reaches a scope introduced by a 'with', it may or may not find a match in that scope depending on the existence of a property at run-time. But it's the sequence of scopes that are searched that is the primary difference between the two categories of scoping mechanism. (Hybrid mechanisms are also possible that use a partly lexical and partly dynamic search, or that use different searches for different variables -- Common Lisp does the latter.)
Of course, you can always adopt a definition of lexical scoping that requires static analysability (and some sources do). In that case what 'with' does would be neither lexical nor dynamic scoping. But note that by this definition lexical scoping would differ from dynamic scoping along two logically distinct axes. It makes more sense to me to use the term "static scoping" for scoping that is necessarily statically analysable.
Is there a programming language of note with lexical scoping with conditional shadowing?
ECMAScript :-)
I don't know of anything that works like 'with' in any other programming language. (Not directly as a language construct, I mean; you can obviously simulate it in languages with eval, unhygienic macros, or computational reflection.)
The name for the regularity that we all wish JavaScript closure have, but don't, is "Tennent Correspondence":
gafter.blogspot.com/2006/08/tennents-correspondence-principle-and.html
It is the absence of this regularity that blocks so many attempts to desugar other constructs into functions.
We don't have Tennent Correspondence because of misfeatures like the mechanics of break, return, continue, this, arguments, and var. We can't fix these, but we should avoid making the problem worse. When you find yourself in a hole, first stop digging. The old es4 "this function" violated Tennent Correspondence and would make the hole deeper. I'm against.
On Aug 21, 2008, at 7:26 PM, Peter Michaux wrote:
It works in practice as context of syntax sometime matters.
Context matters in many ways in JS and other languages.
This is definitely an example of the same syntax having semantically different meaning (actually opposite meanings)
The syntax is not the same because of the left-hand side of
assignment (=) vs. right. Mutation in JS is a concern; you have to
know whether you're assigning or using a variable; you have to pay
attention to context.
which was an objection to the use of return in an let statement if a let statement desugars to a function-expression call.
I anticipated this when I wrote in an earlier reply:
"Different semantics should have distinct syntax; similarity or symmetry may motivate reuse of syntactic forms or ideas. Here, there is no function to return y from, to be assigned to z. You could say "let desugars", but it doesn't say what it means."
You are proposing the same semantics for different syntax (let vs.
function expression that's called right away). Destructuring patterns
use the same (or similar, in the case of the object shorthand) syntax
for related (complementary, symmetric about =) semantics. See the
difference?
That 'return' is the common syntax inside your let-as-lambda-call is
not the source of similarity, since the outer syntax (let vs.
function) is dissimilar. The cognitive load comes from the question:
"where does return go?", since it is a control flow break. The
Smalltalk page I linked to earlier talks about this.
Another example in favor of similar syntax for similar or related use-
cases: JS uses function forms in expressions and definitions. The
contexts and effects differ, but the similarity motivates syntax re-use.
Not so the case with the same (not similar, same -- desugaring)
semantics for your different outer syntaxes, especially when the cost
is loss of pellucid return, break, continue, plus the bigger loss of
purely lexical scope, which by contrast let in JS1.7 provides.
On Aug 21, 2008, at 7:55 PM, David-Sarah Hopwood wrote:
If it were dynamic, then you'd expect:
function setOptionB(x) { print(x == opt_B); }
setOptions({opt_B: 42});
to print true, but it actually prints false.
I don't see how. Dynamic scope means the caller's environment is
visible to the callee.Yes, exactly. In your example, the caller's environment is not
visible to the callee for any of the calls.
I know. But in your example above, opt_B in the _ context in
setOptions({_: 42}); is not a binding. So if dynamic scope were the
rule for JS, the print would still print false because the outer
opt_B is initialized to 0.
Probably I'm missing your meaning.
The dynamic part here is that with means you can't make static
binding judgments, as you can in lexically scoped languages such as
Scheme or JS without with, eval, and the global object.
But an object initialiser passed as an actual argument does not bind any of its properties in the caller's environment.
No, but the 'with' hypothetically would, if it provided dynamic
scoping.
Oh, I see. That is not what I thought you meant. 'with' is not a
binding form, so the hypothesis fails in my view.
It's quite easy to imagine a 'with'-like construct that could work in this way, in a language that was otherwise dynamically scoped, or if the use of 'with' were to insert a scope in the lexical chain not dependent on the with's lexical position. (Yes, that would be awful.)
Gotcha, and agreed.
I've never heard of lexical scoping with conditional shadowing.
I just made up the term, but it's as good a description as any.
I don't think it's lexical scoping, period.
en.wikipedia.org/wiki/Lexical_scope#Static_versus_dynamic_scoping
I don't know of anything that works like 'with' in any other
programming language.
Various Borland languages and BASIC dialects had a WITH, if memory
serves. In fact some ex-Borland people at Netscape advocated for with
in spring, 1995, citing long property references starting with
document. -- and I gave it to them... :-/
Since generators are shallow, it seems clear that there should be some local cps-style transform of generators into the remainder of the language. However, it would be a relief to know what that cps-style transform actually looks like, and what edge cases it has. Anyone care to post a proposal?
Can't be done without defining the entire language in CPS. Yes, it's shallow, but a captured activation can involve any arbitrary combination of expressions other than function call.
Since generators are shallow, it seems clear that there should be some local cps-style transform of generators into the remainder of the language. However, it would be a relief to know what that cps-style transform actually looks like, and what edge cases it has. Anyone care to post a proposal?
Can't be done without defining the entire language in CPS. Yes, it's shallow, but a captured activation can involve any arbitrary
combination of expressions other than function call.
I've done meta-circular implementations of generators (or something
like them) twice now. It's amateur work ;) so put it in the FWIW
category, but in case you find it useful "research":
A) www.neilmix.com/narrativejs/doc This does source
transformation of code into suspendable/resumable "partial
continuations" (see groups.google.com/group/narrativejs/browse_thread/thread/9e4ade2924bd8ab0
)
This project was more ambitious than implementing generators since it
was capturing multiple stack frames. Even so, I found the most
difficult part of the transformation to be handling flow control
without goto. I'm pretty sure that I never got it quite right,
especially in exception handlers that contain break or continue.
B) neilmix.googlecode.com/svn/trunk/generators/js.js A meta-
circular implementation of a JS evaluator that supported generators
(intended to be used with function.toString(), so the parser is overly
forgiving, ignore the parser.)
My strategy in this one was to parse into "bytecode" and run in a mini-
interpreter for the bytecode. This was way easier because you get to
use goto in the interpreter and the current state of the interpreter
can be captured in a closure, making it easily suspendable/resumable.
It was also much easier to discover and fix the bugs that I never even
considered in narrativejs.
Brendan Eich wrote:
On Aug 21, 2008, at 7:04 PM, Peter Michaux wrote:
On Thu, Aug 21, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.org>
wrote:The destructuring pattern here is an array, but objects work too: js> let {length: len} = foo() The above has always instinctively looked to me like the variable "length" is being assigned the value of the "len" property of the object returned by foo. That is the way an object literal works: the right value is assigned to the left property.
I find that it's hard to get used to. I keep interpreting this...
var {a: x, b: y, c: z} = fn();
...as...
with (fn())
{ temporary_object = {a: x, b: y, c: z}
}
// Splash out the object.
...(but restricted to returned values). Also, my brain wants to allow the following, or at least reserve as a future possibility...
var {a: x, b: y+z} = fn();
...but with the chosen inverted semantics it would have to be written like this...
var {x: a, y+z: b} = fn();
...which looks weird.
Probably not instinct (no survival advantage ;-); possibly lack of
familiarity with the new form in context. Try using it in more
realistic code for a few days (especially the shorthand form)
The shorthand form would be the same (or am I missing something?)
and
ping me if it still reads like an object initialiser.
Stockholm syndrome!
People can get used to anything. Just look at Perl. :-)
On Aug 22, 2008, at 7:28 AM, Ingvar von Schoultz wrote:
Probably not instinct (no survival advantage ;-); possibly lack
of familiarity with the new form in context. Try using it in
more realistic code for a few days (especially the shorthand form)The shorthand form would be the same (or am I missing something?)
var {a, b, c} = fn();
binds a, b, and c to the same-named properties of the returned object.
and ping me if it still reads like an object initialiser.
Stockholm syndrome!
People can get used to anything. Just look at Perl. :-)
True, and apart from duress (which was what I was getting at about
Stockholm syndrome -- the web was hostage for a long while and no one
had a choice), people have legitimate differences of opinion on
what's good. JS is not Perl, but it is also not on the "there must be
one and only one way to do it" side of the spectrum. It's multi-
paradigm already.
Ingvar von Schoultz wrote:
...(but restricted to returned values). Also, my brain wants to allow the following, or at least reserve as a future possibility...
var {a: x, b: y+z} = fn();
your form confuses me a lot, because it's not clear to me where 'y' and 'z' are coming from. what if the thing returned by fn() doesn't have a 'y', is it going to pull 'y' from the lexical scope, or is it going to use 'undefined' for 'y'?
the es-harmony form var {x: a, y: b} = fn(); makes intuitive sense to me because the thing on the left is a pattern-matching template. I'm saying that I expect the rhs object to have that shape, and if the shape does match, pull out values from it.
On Thu, Aug 21, 2008 at 6:24 PM, Brendan Eich <brendan at mozilla.org> wrote:
This problem is especially disturbing because the problems are so subtle. Once you've convinced yourself that JavaScript's var is problematic, it's really hard to summarize the argument concisely. You can show coding examples where programmers get surprised, but many will respond and simply say don't do that. You can show term-rewriting rules that should be but aren't equivalences, but people's eyes glaze over. I can give a rule of thumb, though, for avoiding these problems to begin with: go with an existing design. The desugaring of lets into functions and function calls is not some obscure new invention, but a common one that has stood the test of time well.
No, it hasn't, because you are talking about JS functions, with |this|, arguments, and so on. I don't know why you changed the argument from decrying var binding to treating JS functions as if they were well-behaved lambda expresions or Scheme procedures.
You didn't really respond to my point. Using lexical binding has worked out well, and equating lexical binding with function calls has, too. There might be something about JavaScript that causes an issue. You raised return, break, continue, and arguments before, and I addressed those. So is there anything that is really an issue with the proposed desugaring?
I fully agree that different languages aren't necessarily comparable to each other. However, those I named have the same semantics as JavaScript for the relevant features: nested functions, function calls, parameters, and the proposed let-bound variables.
No, they do not. Common Lisp, Scala, and Smalltalk do not have the same semantics in detail as JS does with respect to functions, calls, parameters, and any let proposal I've seen or implemented.
If you look from 100,000 feet, you could say they're all similar, but they are not the same. Just one example: Smalltalk has block objects, quoted code you can activate by sending a message. You could say that's just like a function, except where it is quite different (c2.com/cgi/wiki?SmalltalkBlockReturn).
Brendan, that is exactly the non-local return I was talking about. The success of this construct in multiple languages proves that it is at least workable. Its repeated selection proves that others have found it desirable. Now, there might be something special about JavaScript that would cause it not to work well. What difference would that be, though?
Felix wrote:
Ingvar von Schoultz wrote:
...(but restricted to returned values). Also, my brain wants to allow the following, or at least reserve as a future possibility...
var {a: x, b: y+z} = fn();your form confuses me a lot, because it's not clear to me where 'y' and 'z' are coming from. what if the thing returned by fn() doesn't have a 'y', is it going to pull 'y' from the lexical scope, or is it going to use 'undefined' for 'y'?
That's what I meant by "restricted to returned values": I would expect that the lexical scope would not be visible.
the es-harmony form var {x: a, y: b} = fn(); makes intuitive sense to me because the thing on the left is a pattern-matching template. I'm saying that I expect the rhs object to have that shape, and if the shape does match, pull out values from it.
Thanks for the description, it helps.
Is this the actual semantics? Does it require a matching shape and throw an error otherwise?
On Aug 22, 2008, at 4:02 PM, Lex Spoon wrote:
You raised return, break, continue, and arguments before, and I addressed those.
Where? Your syntax-free suggestion of labels (as in break to label
and continue to label) does not address the problems with |this| or
arguments.
So is there anything that is really an issue with the proposed
desugaring?
Of course. First, it's unnecessary and confusing to have two ways to
write
(function (x, y) {...})(a, b)
one of which has a clear target for return in ..., the other not. I'm
still waiting to see a proposal for unambiguous return-value-to-label
syntax. The dynamic binding of |this| means in ... it will be the
global object, even if it is not in the outer context. And arguments
[0] aliases x. Break compatibility? Then you break
"desugaring" (which is a sacred cow anyway).
If you look from 100,000 feet, you could say they're all similar,
but they are not the same. Just one example: Smalltalk has block objects,
quoted code you can activate by sending a message. You could say that's just
like a function, except where it is quite different (c2.com/cgi/wiki?SmalltalkBlockReturn).Brendan, that is exactly the non-local return I was talking about. The success of this construct in multiple languages proves that it is at least workable. Its repeated selection proves that others have found it desirable. Now, there might be something special about JavaScript that would cause it not to work well. What difference would that be, though?
I still don't know exactly what you mean. Instead of hand-waving,
please write explicit syntax. How would non-local return work? How
would it desugar? How would you deal with |this| and arguments?
You can stop addressing me by name, I'm the only To: addressee.
On Sat, Aug 23, 2008 at 7:15 PM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 22, 2008, at 4:02 PM, Lex Spoon wrote:
So is there anything that is really an issue with the proposed desugaring?
Of course. First, it's unnecessary and confusing to have two ways to write
(function (x, y) {...})(a, b)
That statement seems like an anti-sugar statement in general which I don't think you are, are you? (I don't think I'm quoting that statement out of context.)
If you do think some sugar is ok, where do you draw the line? How much boilerplate needs to be eliminated to make the sugar sweet enough?
If no new semantics are added to JavaScript then certainly having sugar for writing
(function (x, y) {...})(a, b)
would be quite welcome. It is clear people like this pattern and it is confusing when the formals and actuals are more than a couple and more than a couple lines apart.
If the problem JavaScripters are trying to solve with that pattern is addressed by adding new semantics then that is a different story.
Peter
On Sat, Aug 23, 2008 at 8:36 PM, Peter Michaux <petermichaux at gmail.com> wrote:
(function (x, y) {...})(a, b)
would be quite welcome. It is clear people like this pattern and it is confusing when the formals and actuals are more than a couple and more than a couple lines apart.
As Lars pointed out, using ES-Harmony's optional parameters with defaults, you can keep the actuals and formals together by writing
(function (x=a, y=b){...})()
Given this, I don't see any need for let blocks or let expressions that justifies their added complexity. Especially since, as we've established, they can't be added as sugar that desugars to anything like the above code.
On Aug 23, 2008, at 8:36 PM, Peter Michaux wrote:
On Sat, Aug 23, 2008 at 7:15 PM, Brendan Eich <brendan at mozilla.org>
wrote:On Aug 22, 2008, at 4:02 PM, Lex Spoon wrote:
So is there anything that is really an issue with the proposed desugaring?
Of course. First, it's unnecessary and confusing to have two ways to write
(function (x, y) {...})(a, b)
That statement seems like an anti-sugar statement in general which I don't think you are, are you? (I don't think I'm quoting that statement out of context.)
First, let's settle the hash over whether any desugaring without
extensions such as return-to-label, reformed lexical scope, tamed
this, banished arguments, etc. etc., trumps adding a new binding
form, to wit: let as block scoped var. That's between Lex and me, and
it's not a philosophical disagreement but a practical one (possibly
factual, since the problems in JS may not be appreciated; talking
about a static language like Scala, or dear old Smalltalk, really
does not help).
Second, you can invent syntax using macros in many languages, but one
hacker's sugar is another's vinegar. ES may get macros, but before
then it does not need every addition to consist of syntax that maps
directly onto the (bad old compatible and broken) core semantics.
Finally, I wrote that "it's unnecessary and confusing to have two
ways to write ..." and you seemed to seize that statement as "anti-
sugar". Which do you disagree with, "unnecessary" or "confusing"?
I call it unnecessary to make let (x = a, y = b) {...} be shorthand
for (function (x, y) {...})(a,b). Count characters: 24 vs. 28. Come
on! It also abuses and preempts 'let', a binding form in many
languages, imposing unwanted costs and visible effects by mandating
desugaring to JS functions (with their |this|, arguments, return/
break/continue isolation, etc. natures). Saying "oh, fix those with
extensions" breaks the desugaring deal.
I call it confusing to have return from within such a let-as-lambda-
call not return to the nearest explicit enclosing function. If the
only benefit is four fewer characters to type, then any burden in
learning or re-learning is not worth it. We do not write-optimize JS
in any event. It's fairly keyword-y already.
If you do think some sugar is ok, where do you draw the line?
You are ignoring my specific arguments, given before and repeated
above, in order to set up a straw man. I'll knock it down for you
anyway:
There's no question that some sugar is good sugar: avoiding repeated
let {p:p, q:q, r:q} = ... by using the shorthand let {p, q, r} = ...
is a clear win. Redundancy for no good reason (such as avoiding a
grammatical ambiguity) is not justified.
What you seem to call sugar is simply a four-character savings that
would be a personal macro, if we had macros. It would not be part of
the standard. It's not justified by four chars of saved typing, even
if you ignore the substantive problems with mapping let-bindings onto
function parameters.
How much boilerplate needs to be eliminated to make the sugar sweet enough?
No way is 28 instead of 24 chars too much boilerplate.
If no new semantics are added to JavaScript
Who says no new semantics are being added to JS? Did you miss the
news about generated Name objects? JS1.7 generators also represent
some (obvious) extensions to activation semantics. Even ES3.1 adds
semantics, with things like seal.
then certainly having sugar for writing
(function (x, y) {...})(a, b)
would be quite welcome. It is clear people like this pattern and it is confusing when the formals and actuals are more than a couple and more than a couple lines apart.
Then write a function definition and call it, if you really need to
return a value early from its body.
Otherwise, use let as implemented in JS1.7 and you can initialize the
bindings in the let-block head. Or use let-as-the-better-var, in a
block.
Again, people aren't using "this pattern" because they like
everything about it but the separation of the actual arguments from
the formal parameters, and the four extra characters of boilerplate.
They're using it because it's all they have. Do not jump to the
conclusion, or try to plant it as an axiom, that every user of this
pattern must have early returns or other such things in the body.
If the problem JavaScripters are trying to solve with that pattern is addressed by adding new semantics then that is a different story.
You need new semantics to avoid all the problems I keep citing with
mapping onto the old semantics.
On Aug 23, 2008, at 10:23 PM, Mark S. Miller wrote:
On Sat, Aug 23, 2008 at 8:36 PM, Peter Michaux
<petermichaux at gmail.com> wrote:(function (x, y) {...})(a, b)
would be quite welcome. It is clear people like this pattern and
it is confusing when the formals and actuals are more than a couple and
more than a couple lines apart.As Lars pointed out, using ES-Harmony's optional parameters with defaults, you can keep the actuals and formals together by writing
(function (x=a, y=b){...})()
Good point, although I say it's no fair to talk desugaring but then
special-plead extensions into the mix. Optional parameters are not
just syntax, especially in their evaluation and scope rules.
Given this, I don't see any need for let blocks or let expressions that justifies their added complexity.
They may indeed not be justified on complexity grounds, but I'd like
to argue against your reason:
Especially since, as we've established, they can't be added as sugar that desugars to anything like the above code.
The above code has issues in the ... you didn't show. |this|,
arguments aliasing, the arguments name shadowing an outer arguments
that would otherwise be visible, e.g., in a JS.1.7 let block, and
probably other compatibility burdens I'm forgetting atm, all make it
undesirable to map a binding form with explicitly delimited scope and
let-not-letrec rules for initialization (JS1.7's let block or let
expression) onto a lambda-call.
So let blocks and expressions indeed can't desugar -- that could be
the end of it. But that argument does not prevail in all cases,
particularly in the case of let declarations (let as the new var),
which I believe you support.
My counter-argument is this: If the existing semantics have their own
complexity problems, even though we are stuck with them for
compatibility, we could make the language better overall, and some
day hope to move programmers off bad old forms, by adding some
carefully designed better forms. Desugaring is not the only good to
consider. Reforming the core semantics while keeping compatibility
may require new runtime semantics.
I agree with you that conserving runtime semantics when adding syntax
is a good discipline and an appropriate bias. But the lexical scope
breaks, the extra gewgaws such as |this| and arguments that afflict
JS functions should not be ignored.
2008/8/24 Brendan Eich <brendan at mozilla.org>:
I call it unnecessary to make let (x = a, y = b) {...} be shorthand for (function (x, y) {...})(a,b). Count characters: 24 vs. 28. Come on!
And in the case of the let expressions using the shorthand (it is proposed for ES4 and available in FireFox 3.0) that allows to replace
function optionalName(...) { return expr; }
by
function optionalName(...) expr
the situation is
let (x = a, y = b) expr
versus
(function(x, y) expr)(a, b)
That is again just 4 extra characters, 23 versus 27 and there is nothing to sweeten with sugar.
Now, if those proposal of having "fun" or even "ƛ" as a shorthand for the "function" keyword would be resurrected, then the explicit lambda forms would be shorter then the let forms. Plus, compared with the let blocks or expressions, such shorthands could be used with each and every function definition.
, Igor
On Sun, Aug 24, 2008 at 12:31 AM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 23, 2008, at 10:23 PM, Mark S. Miller wrote:
On Sat, Aug 23, 2008 at 8:36 PM, Peter Michaux <petermichaux at gmail.com> wrote:
(function (x, y) {...})(a, b)
would be quite welcome. It is clear people like this pattern and it is confusing when the formals and actuals are more than a couple and more than a couple lines apart.
As Lars pointed out, using ES-Harmony's optional parameters with defaults, you can keep the actuals and formals together by writing
(function (x=a, y=b){...})()
Good point, although I say it's no fair to talk desugaring but then special-plead extensions into the mix. Optional parameters are not just syntax, especially in their evaluation and scope rules.
Sorry I hadn't responded to this earlier.
I am not a "no new semantics -- desugaring only" absolutist; just close. I recognize there are tradeoffs. When new semantics has a high enough benefit and a low enough cost when compared with alternatives, then we should add new semantics. Of course, there are no objective measures here, so all we can do is to strive to set very high bars and to err on the side of simplicity.
Optional and (especially) rest parameters pass my subjective test. They are new semantics that I believe should appear in Harmony. As you have pointed out, "new Date(args...)" demonstrates that they are not equivalent to any desugaring.
Given this, I don't see any need for let blocks or let expressions that justifies their added complexity.
They may indeed not be justified on complexity grounds, but I'd like to argue against your reason:
Especially since, as we've established, they can't be added as sugar that desugars to anything like the above code.
The above code has issues in the ... you didn't show. |this|, arguments aliasing, the arguments name shadowing an outer arguments that would otherwise be visible, e.g., in a JS.1.7 let block, and probably other compatibility burdens I'm forgetting atm, all make it undesirable to map a binding form with explicitly delimited scope and let-not-letrec rules for initialization (JS1.7's let block or let expression) onto a lambda-call.
What I'm saying is that, given optional args, the let expression and let block no longer have enough differential benefit to justify the cost of adding them. I'm not invoking any absolute principles. I'm arguing about relative costs and benefits.
So let blocks and expressions indeed can't desugar -- that could be the end of it. But that argument does not prevail in all cases,
Of course. It depends.
particularly in the case of let declarations (let as the new var), which I believe you support.
I do indeed. The benefits of this feature are overwhelming.
My counter-argument is this: If the existing semantics have their own complexity problems, even though we are stuck with them for compatibility, we could make the language better overall, and some day hope to move programmers off bad old forms, by adding some carefully designed better forms. Desugaring is not the only good to consider. Reforming the core semantics while keeping compatibility may require new runtime semantics.
Agreed in principle. We are even in agreement on many particular applications of this principle. However, we must strive -- much harder than we have historically -- to keep needless complexity out and to only admit features whose benefits clearly exceed their costs.
I agree with you that conserving runtime semantics when adding syntax is a good discipline and an appropriate bias. But the lexical scope breaks, the extra gewgaws such as |this| and arguments that afflict JS functions should not be ignored.
Rest assured that I am not ignoring them. They have been a source of a great deal of pain. I expect they will continue to be ;).
On Sun, Aug 24, 2008 at 12:31 AM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 23, 2008, at 10:23 PM, Mark S. Miller wrote:
On Sat, Aug 23, 2008 at 8:36 PM, Peter Michaux <petermichaux at gmail.com> wrote:
(function (x, y) {...})(a, b)
would be quite welcome. It is clear people like this pattern and it is confusing when the formals and actuals are more than a couple and more than a couple lines apart.
As Lars pointed out, using ES-Harmony's optional parameters with defaults, you can keep the actuals and formals together by writing
(function (x=a, y=b){...})()
Good point, although I say it's no fair to talk desugaring but then special-plead extensions into the mix. Optional parameters are not just syntax, especially in their evaluation and scope rules.
Sorry I hadn't responded to this earlier.
I am not a "no new semantics -- desugaring only" absolutist; just close. I recognize there are tradeoffs. When new semantics has a high enough benefit and a low enough cost when compared with alternatives, then we should add new semantics. Of course, there are no objective measures here, so all we can do is to strive to set very high bars and to err on the side of simplicity.
Optional and (especially) rest parameters pass my subjective test. They are new semantics that I believe should appear in Harmony. As you have pointed out, "new Date(args...)" demonstrates that they are not equivalent to any desugaring.
Given this, I don't see any need for let blocks or let expressions that justifies their added complexity.
They may indeed not be justified on complexity grounds, but I'd like to argue against your reason:
Especially since, as we've established, they can't be added as sugar that desugars to anything like the above code.
The above code has issues in the ... you didn't show. |this|, arguments aliasing, the arguments name shadowing an outer arguments that would otherwise be visible, e.g., in a JS.1.7 let block, and probably other compatibility burdens I'm forgetting atm, all make it undesirable to map a binding form with explicitly delimited scope and let-not-letrec rules for initialization (JS1.7's let block or let expression) onto a lambda-call.
What I'm saying is that, given optional args, the let expression and let block no longer have enough differential benefit to justify the cost of adding them. I'm not invoking any absolute principles. I'm arguing about relative costs and benefits.
So let blocks and expressions indeed can't desugar -- that could be the
end
of it. But that argument does not prevail in all cases,
Of course. It depends.
particularly in the case of let declarations (let as the new var), which I believe you
support.
I do indeed. The benefits of this feature are overwhelming.
My counter-argument is this: If the existing semantics have their own complexity problems, even though we are stuck with them for compatibility, we could make the language better overall, and some day hope to move programmers off bad old forms, by adding some carefully designed better forms. Desugaring is not the only good to consider. Reforming the core semantics while keeping compatibility may require new runtime semantics.
Agreed in principle. We are even in agreement on many particular applications of this principle. However, we must strive -- much harder than we have historically -- to keep needless complexity out and to only admit features whose benefits clearly exceed their costs.
I agree with you that conserving runtime semantics when adding syntax is a good discipline and an appropriate bias. But the lexical scope breaks, the extra gewgaws such as |this| and arguments that afflict JS functions
should
not be ignored.
Rest assured that I am not ignoring them. They have been a source of a great deal of pain. I expect they will continue to be ;).
I wrote:
Since generators are shallow, it seems clear that there should be some local cps-style transform of generators into the remainder of the language. However, it would be a relief to know what that cps-style transform actually looks like, and what edge cases it has. Anyone care to post a proposal?
On Thu, Aug 21, 2008 at 11:31 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Can't be done without defining the entire language in CPS. Yes, it's shallow, but a captured activation can involve any arbitrary combination of expressions other than function call.
I don't yet understand this and I need to. Can you take us through a simple example? Thanks.
On Fri, Aug 22, 2008 at 7:27 AM, Neil Mix <nmix at pandora.com> wrote:
Since generators are shallow, it seems clear that there should be some local cps-style transform of generators into the remainder of the language. However, it would be a relief to know what that cps-style transform actually looks like, and what edge cases it has. Anyone care to post a proposal?
Can't be done without defining the entire language in CPS. Yes, it's shallow, but a captured activation can involve any arbitrary combination of expressions other than function call.
I've done meta-circular implementations of generators (or something like them) twice now. It's amateur work ;) so put it in the FWIW category, but in case you find it useful "research":
A) www.neilmix.com/narrativejs/doc This does source transformation of code into suspendable/resumable "partial continuations" (see groups.google.com/group/narrativejs/browse_thread/thread/9e4ade2924bd8ab0 )
Neil, I think Narrative JS is really cool. Kudos.
This project was more ambitious than implementing generators since it was capturing multiple stack frames.
Yes, this would correspond to deep generators or coroutines, which have many of the virtues and many of the hazards of continuations. It isn't appropriate for future standard EcmaScripts, but again, it is really cool.
However, this reopens the question. If deep generators are implementable by a global cps transform, are shallow generators implementable by a local cps transform? If not, can anyone explain the issues using a simple example? Thanks.
B) neilmix.googlecode.com/svn/trunk/generators/js.js A meta-circular implementation of a JS evaluator that supported generators (intended to be used with function.toString(), so the parser is overly forgiving, ignore the parser.)
My strategy in this one was to parse into "bytecode" and run in a mini-interpreter for the bytecode. This was way easier because you get to use goto in the interpreter and the current state of the interpreter can be captured in a closure, making it easily suspendable/resumable. It was also much easier to discover and fix the bugs that I never even considered in narrativejs.
This approach would seem relevant to adding generators by adding new semantic state. But would it help us understand how to add generators by a local code transformation?
On Aug 25, 2008, at 4:10 PM, Mark S. Miller wrote:
are shallow generators implementable by a local cps transform?
Yes, the local evaluation (inside the generator function) is the only
place that needs to be CPS'ed to allow yield in any expression or
statement context.
This approach would seem relevant to adding generators by adding new semantic state. But would it help us understand how to add generators by a local code transformation?
Why does this matter? Changing any future ES spec to use CPS is a
semantic state shift, and a very big one. The plan (designed and
implemented by Dave) for specifying generators in ES4 was to use
delimited continuations in the SML reference implementation.
It's not pretty, but you can CPS-convert generators. Here's a
concrete "desugaring" ("souring"? :-P) example based on JS1.7, not
ES4 (no iterator namespace, ugly iterator name instead):
function simple_range(n) { for (let i = 0; i < n; i++) yield i; }
function simple_range(n) { let i = 0; return { iterator: function () { return { next: function () { if (i == n) throw StopIteration; return i++; } } } }; }
Object.freeze on the returned iterator object omitted ;-). Adding
send, throw, and close also left as an exercise.
On Mon, Aug 25, 2008 at 4:33 PM, Brendan Eich <brendan at mozilla.org> wrote:
On Aug 25, 2008, at 4:10 PM, Mark S. Miller wrote:
are shallow generators implementable by a local cps transform?
Yes, the local evaluation (inside the generator function) is the only place that needs to be CPS'ed to allow yield in any expression or statement context.
This approach would seem relevant to adding generators by adding new semantic state. But would it help us understand how to add generators by a local code transformation?
Why does this matter? Changing any future ES spec to use CPS is a semantic state shift, and a very big one. The plan (designed and implemented by Dave) for specifying generators in ES4 was to use delimited continuations in the SML reference implementation.
I find continuations, delimited or otherwise, very hard to reason about. Generally, my best route to understanding what's going on is by CPS transform anyway. YMMV. I didn't know SML has delimited continuations. But I'd still rather not try to understand generators by CPS-transforming the relevant bits of the RI.
As for which technique should be used to specify generators, I'll postpone forming an opinion until I understand them ;).
It's not pretty, but you can CPS-convert generators. Here's a concrete "desugaring" ("souring"? :-P) example based on JS1.7, not ES4 (no iterator namespace, ugly iterator name instead):
function simple_range(n) { for (let i = 0; i < n; i++) yield i; }
function simple_range(n) { let i = 0; return { iterator: function () { return { next: function () { if (i == n) throw StopIteration; return i++; } } } }; }
Object.freeze on the returned iterator object omitted ;-). Adding send, throw, and close also left as an exercise.
Thanks, that helps a lot. But shouldn't that "i == n" be "!(i < n)" since it's a transform of the original "i < n"?
On Thu, Aug 21, 2008 at 11:31 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Can't be done without defining the entire language in CPS. Yes, it's shallow, but a captured activation can involve any arbitrary combination of expressions other than function call.
Dave, is the violent transform of the for-loop above the kind of rest-of-the-language transform you were referring to? If so, isn't this only an issue for control structures (including &&, ||, and ?:)? What other elements of the language might need to be turned inside out this way? Or have I misunderstood what you're getting at.
On Aug 25, 2008, at 6:15 PM, Mark S. Miller wrote:
Thanks, that helps a lot. But shouldn't that "i == n" be "!(i < n)" since it's a transform of the original "i < n"?
Sure -- was going fast (other work competing ;-).
On Thu, Aug 21, 2008 at 11:31 PM, Dave Herman <dherman at ccs.neu.edu>
wrote:Can't be done without defining the entire language in CPS. Yes, it's shallow, but a captured activation can involve any arbitrary
combination of expressions other than function call.Dave, is the violent transform of the for-loop above the kind of rest-of-the-language transform you were referring to?
Answering for Dave: yes.
If so, isn't this only an issue for control structures (including &&, ||, and ?:)? What other elements of the language might need to be turned inside out this way? Or have I misunderstood what you're getting at.
As in Python 2.5, JS1.7 and up support yield expressions too. These
return values and resume with new values when the generator is
continued via next (which sends undefined) or send(v) (arbitrary v).
The simple_range example I showed happened to use yield as an
expression-statement.
I don't expect we'll CPS-transform any future ES spec, but you are
entitled to pain if you like it ;-).
Brendan Eich wrote:
Dave, is the violent transform of the for-loop above the kind of rest-of-the-language transform you were referring to?
Answering for Dave: yes.
Sorry for the delay-- yes, any kind of construct in the language other than function/method/constructor call would require CPS. This makes for a pretty violent change to the language, though admittedly less violent than if generators captured more than one activation frame.
In essence, you can think of the language as split between two kinds of continuation: the "function activation stack" and the "local expression stack". Only the latter can be suspended by `yield', so only the latter needs to be representable in the semantics as a storable data structure. (If you're familiar with Landin's SECD machine, I think the activation stack corresponds to the [D]ump and the local expression continuation to the [S]tack.)
If so, isn't this only an issue for control structures (including &&, ||, and ?:)? What other elements of the language might need to be turned inside out this way? Or have I misunderstood what you're getting at.
It would be an issue for all the other binary operators, too (=, ==, ===, +, *, >>>, %, ^, etc.). Also sequences, blocks, and declarations.
Any context that a `yield' expression may occur is fodder for capture.
I do understand what you're saying; it's not as big a change as having to CPS function calls. But in light of the (useful!) exhortations to emphasize specification by desugaring, I want to make it clear that a CPS transform is not a desugaring, i.e. a local rewrite; it's a reimplementation of (a portion of) the semantics.
Mark S. Miller wrote:
I find continuations, delimited or otherwise, very hard to reason about. Generally, my best route to understanding what's going on is by CPS transform anyway. YMMV.
I hear you. Advanced control constructs can definitely get thorny. I find operational semantics with evaluation contexts much easier to understand than CPS, but as you say tastes vary.
As it turns out, the use of delimited continuations for generators is
pretty straightforward-- you install a delimiter (reset') when you enter the generator function (think of this like installing an exception handler) and capture the continuation (shift') upon `yield'. This
effectively captures and aborts the local continuation up to the
delimiter, popping back up to the caller. That captured continuation,
then, is in essence just a function; when you invoke it later, you're
just pushing that suspended function activation back on the control stack.
I didn't know SML has delimited continuations.
It doesn't officially; SML/NJ and MLton implement callcc, which you can use to implement shift and reset. So it's admittedly on the edge of ML's semantics. But these are all very well-understood control operators and easy to add to the formal semantics of SML. We went that route so we could write the reference implementation in direct style.
But I'd still rather not try to understand generators by CPS-transforming the relevant bits of the RI.
A little confused here-- was that an errant "not"?
On Mon, Aug 25, 2008 at 7:24 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
But I'd still rather not try to understand generators by CPS-transforming the relevant bits of the RI.
A little confused here-- was that an errant "not"?
No. I meant that, since I understand continuation by CPS-transform, if the RI and/or the spec explains generators by absorbing them into SML's delimited continuations, then I'd need try to understand generators by CPS-transforming that portion of the RI's implementation. Given a choice between understanding generators that way vs. understanding them by CPS transforming the ES generator code, the latter seems preferable. Again, this may just be a personal idiosyncrasy. I do not yet have an opinion about how this should be speced.
Thank you and Brendan for your explanations. They truly did help a lot.
How important is it that yield be an expression rather than a statement? It seems like a sibling of return and throw, so I don't think anyone coming to them afresh would be surprised if yield were a statement.
On Aug 25, 2008, at 8:51 PM, Mark S. Miller wrote:
How important is it that yield be an expression rather than a statement? It seems like a sibling of return and throw, so I don't think anyone coming to them afresh would be surprised if yield were a statement.
This is and was not a clean-slate exercise. We both followed and gave
feedback to Python in doing this work, because the Python and JS
communities overlap significantly, and the languages share certain
stylistic and substantial features (not all, but enough). The goals
include reusing the Python open-source/python-dev experience and the
developer brainprint.
yield as an expression form is important for so-called "co-routines",
where you can call gen.send(val) to resume a generator the val
becoming the result of the last yield expression; you can also
gen.throw(exc).
Finally, and this is possibly the most useful, gen.close() will be
called in JS when a generator used in a for-in construct. With close
called from for-in, cleanup can be automated that otherwise can't be
done without Python-style ref-counting/GC, Java-style finalization,
or worse -- complexities we wish to avoid in any ES standard.
Calling gen.close() on an open generator forces a return from the
last yield. This has the desirable property that if the yield is in a
try, any relevant finally clauses will be run. (Return from generator
throws StopIteration, as usual for iterators; a generator can't
return a value.) Background reading:
mail.python.org/pipermail/python-dev/2006-August/068497.html, mail.python.org/pipermail/python-dev/2006-August/068498.html, mail.python.org/pipermail/python-dev/2006-August/068450.html
One use of generators, which Neil Mix wrote: "Threads" in JS:
www.neilmix.com/2007/02/07/threading-in-javascript-17
Source to read:
www.neilmix.com/demos/js17threading/example.js, www.neilmix.com/demos/js17threading/Thread.js
These use send and yield expressions heavily. The es- discuss at mozilla.org archives at
best searched with Google site:mail.mozilla.org search, have a number
of threads tracing the evolution of generators. Besides Igor
Bukanov's work to simplify the close mechanism by eliminating
GeneratorExit, Chris Hansen of Google made the crucial proposal to
automate close only from for-in constructs (not from GC).
Generators are in JS1.7 (Firefox 2). Sugar in the form of generator
expressions, as in Python 2.4, are in JS1.8 (Firefox 3). My port of
Peter Norvig's Sudoku solver, which uses genexps:
bugzilla.mozilla.org/attachment.cgi?id=266577
I believe Pythonic generators have been the most useful, if not most
widely used, of the extensions that we have shipped since JS1.5 (ES3
- getters and setters).
On Sun, Aug 24, 2008 at 3:17 AM, Brendan Eich <brendan at mozilla.org> wrote:
First, let's settle the hash over whether any desugaring without extensions such as return-to-label, reformed lexical scope, tamed this, banished arguments, etc. etc., trumps adding a new binding form, to wit: let as block scoped var.
With no extensions, it is true, return would end up returning from a different method under the proposed rewrite. Likewise, this and arguments would cause trouble. Possibly break and continue would, depending on what their precise semantics are.
However, they work under some specific extensions that appear to benefit JavaScript anyway. I suspect that most languages with both return expressions and nested functions will eventually want a way to return from other than the innermost function. More generally, it would be really nice if programmers could safely add a nested function without losing access to important things from their surrounding scope. That goes not just for return, but also arguments, this, break, and continue. It's an orthoganality problem. With JavaScript's current limited return expression, programmers have trouble when they try to use both function nesting and return at the same time. Using one feature makes it harder to use the other.
That's a philosophical argument, and so people will disagree. That's why I tried to pull in experience with existing languages that have both of these features. Not all that many do, actually. The languages that push nested functions the hardest are functional languages, and most of them do not have return. Of those that have both, though, it's notable that most also have non-local return. I earlier listed Scala, Smalltalk, and Lisp as examples. On the flip side, I can only think of two languages: Java (counting anonymous inner classes as nested functions) and X10 (which is based closely on Java). Can anyone else think of other language experiences to compare against?
Looking at this experience, it's fair to say that non-local return works out fine in practice. Further, the two languages in my list do not have this feature, also make it annoying to write nested functions that also want to return. I guess it's a matter of taste, but I have to say that Java (and X10), despite their many other advantages, don't make it concise nor elegant to nest anonymous classes very much. Java's designers seem to agree, having written their collections library not to support a style where you'd use a lot of nested anonymous classes.
Stepping back, let me emphasize that I'm not religious about desugaring as a definition approach. I'm more concerned that an intuitive-looking rewrite doesn't work. Programmers are going to do intuitive-seeming things to their code, and it's a nicer language when those intuitive-seeming things are in fact safe. Of course, if it can't be done, it can't be done, and the wart has to stay. In this case, though, the wart can be removed by adding a feature that has merits of its own.
Thus, I don't know what the Harmony process is, but I would recommend a careful look at accessing "return", "this", and "arguments" for functions surrounding the immediately enclosing function. If those work out as well as they appear to, then programmers would have one less thing to worry about when they introduce nested functions into their programs.
Finally, there was a little bit of question about what the semantics would be. Let me go into just a little more detail. The idea is that instead of just "return foo", you could also put a label on the return. I don't know what the syntax would be, but as a straw man, maybe it would be "return:bar foo". This notation means to return "foo" from the function named "bar". If "bar" happens to be the immediately enclosing function, then you return exactly as in JavaScript right now. Otherwise, you pop stack frames until you get to the correct "bar" stack frame. You then return from that one. If that frame has already popped, then the return fails, much like an exception that is never handled. Does that explain it well enough, or should I go on?
On Tue, Sep 2, 2008 at 2:16 PM, Lex Spoon <spoon at google.com> wrote:
On Sun, Aug 24, 2008 at 3:17 AM, Brendan Eich <brendan at mozilla.org> wrote:
First, let's settle the hash over whether any desugaring without extensions such as return-to-label, reformed lexical scope, tamed this, banished arguments, etc. etc., trumps adding a new binding form, to wit: let as block scoped var.
With no extensions, it is true, return would end up returning from a different method under the proposed rewrite. Likewise, this and arguments would cause trouble. Possibly break and continue would, depending on what their precise semantics are.
Wouldn't any Completion Type cause problems?
(function() { throw Error("help."); })();
?
Wouldn't the caller, stack, et c, be all messed up?
However, they work under some specific extensions that appear to benefit JavaScript anyway. I suspect that most languages with both return expressions and nested functions will eventually want a way to return from other than the innermost function. More generally, it would be really nice if programmers could safely add a nested function without losing access to important things from their surrounding scope. That goes not just for return, but also arguments, this, break, and continue.
Those are not lost if you store them in a variable in the containing scope.
It wouldn't make sense to have - return: outerFunction - because the outer function might not have been the caller. In fact, it probably won't be in the majority of cases.
It's an orthoganality problem. With JavaScript's current limited return expression, programmers have trouble when they try to use both function nesting and return at the same time. Using one feature makes it harder to use the other.
I've never wanted to do that; to make the inner function make the outer function return.
Finally, there was a little bit of question about what the semantics would be. Let me go into just a little more detail. The idea is that instead of just "return foo", you could also put a label on the return.
What would you use that for?
On Tue, Sep 2, 2008 at 5:48 PM, Garrett Smith <dhtmlkitchen at gmail.com> wrote:
On Tue, Sep 2, 2008 at 2:16 PM, Lex Spoon <spoon at google.com> wrote:
On Sun, Aug 24, 2008 at 3:17 AM, Brendan Eich <brendan at mozilla.org> wrote:
First, let's settle the hash over whether any desugaring without extensions such as return-to-label, reformed lexical scope, tamed this, banished arguments, etc. etc., trumps adding a new binding form, to wit: let as block scoped var.
With no extensions, it is true, return would end up returning from a different method under the proposed rewrite. Likewise, this and arguments would cause trouble. Possibly break and continue would, depending on what their precise semantics are.
Wouldn't any Completion Type cause problems?
(function() { throw Error("help."); })();
?
Wouldn't the caller, stack, et c, be all messed up?
That sounds like a good rule. At a glance, though, I don't see an immediate problem with throw. In your example, the first thing that happens is the top-most stack level is popped (because there is no exception handler), and then you're back at the original stack level and will start popping from there. So it looks like you get the same behavior whether or not you add a function wrapper and immediately call it.
If you did introduce an exception handler, that would be a different story. But introducing an exception handler would be a separate step from introducing the function wrapper. The function wrapper itself wouldn't, as far as I can see, cause trouble for throw and catch.
By the way, the above rewrite is precisely the one I am thinking it would be nice to make available to programmers. It doesn't work out of the box, but with labeled return, and with the proper interpretation of break and continue, it would work with the only burden being to label the returns. That's a local change that would not require rethinking the rest of the algorithm.
However, they work under some specific extensions that appear to benefit JavaScript anyway. I suspect that most languages with both return expressions and nested functions will eventually want a way to return from other than the innermost function. More generally, it would be really nice if programmers could safely add a nested function without losing access to important things from their surrounding scope. That goes not just for return, but also arguments, this, break, and continue.
Those are not lost if you store them in a variable in the containing scope.
True. It's mostly return that is in trouble. You could work around arguments and this by making new variables to refer to them.
Finally, there was a little bit of question about what the semantics would be. Let me go into just a little more detail. The idea is that instead of just "return foo", you could also put a label on the return.
What would you use that for?
I earlier described why you would want it in principle: if you have both return and nested functions, you otherwise are hampered in using them both at the same time. Further, there is no reason to believe that using one means you won't want to use the other. Their purposes are orthogonal, so you would think there would be code that wants to do both.
As a specific example, consider the style of collection library where each collection type supplies methods for looping through the collection, searching for items, etc. For example, suppose all of the collections in the library implement a foreach() method that goes through the elements of the collection. It's true that many people don't like this style of library, but many people do, and those people I would bet overlap heavily with those who like having nested functions at all. (I freely grant that if you hate nested functions, this whole discussion is unexciting, because your answer will always be, well don't nest functions--problem solved!)
Given such a library, you could implement a find() method by calling foreach(). The contract of find() is to take a predicate as argument (itself a function), apply that predicate to each element of a supplied collection, and then return the matching item that is found. If no matching item is found it returns null.
Here's how it looks if you have return from outer function:
function find(collection, predicate) { collection.foreach(function (each) { if (predicate(each)) return:find each; } return null; }
So that's one example where you want both nested functions and return from outer function in the same place. There are lots of other collection operations, too. Further, there are lots of libraries other than collection libraries that can benefit from implementing control-structure-like methods such as foreach. This whole style of library, one of the major benefits of nested functions, is in trouble if you don't have return from outer function.
To back up, this is all just a general comment. As far as I can tell, the feature is sound and would be helpful in practice. However, I don't know what all are the aims of Harmony, nor the timelines, etc., and I'm certainly not volunteering to do any of the leg work. The language obviously can live without the feature, because it has for years, and at best it makes programming in Harmony just a little bit nicer.
Mark S. Miller wrote:
On Mon, Aug 25, 2008 at 7:24 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
But I'd still rather not try to understand generators by CPS-transforming the relevant bits of the RI. A little confused here-- was that an errant "not"?
No. I meant that, since I understand continuation by CPS-transform, if the RI and/or the spec explains generators by absorbing them into SML's delimited continuations, then I'd need try to understand generators by CPS-transforming that portion of the RI's implementation. Given a choice between understanding generators that way vs. understanding them by CPS transforming the ES generator code, the latter seems preferable. Again, this may just be a personal idiosyncrasy.
It's not just your personal idiosyncrasy. I also find it almost impossible to understand the effects of (even delimited) continuations directly. The semantics of CPS-transformed code is straightforward, no matter how ugly the code may be.
Dave Herman <dherman at ccs.neu.edu> wrote:
Mark S. Miller wrote:
I didn't know SML has delimited continuations.
It doesn't officially; SML/NJ and MLton implement callcc, which you can use to implement shift and reset. So it's admittedly on the edge of ML's semantics. But these are all very well-understood control operators and easy to add to the formal semantics of SML.
Hmm. I thought the motivation for using SML is that it had a definite formal semantics (for which the last revision was published a decade ago), that had already been looked at and understood by many people. Handwaving about the possibility of adding delimited continuations to that formalization is not the same thing, even if you are right in saying it is "easy".
Hmm. I thought the motivation for using SML is that it had a definite formal semantics (for which the last revision was published a decade ago),
Not really; I suppose it's nice to know that the underlying meta-language is formalized, but this would really only provide practical utility if we were proving theorems about ES. While there are interesting theorems that you might like to prove, it's not as high a priority as the actual design, specification, and reference implementation work. Being able to execute the spec and run test cases is an enormous leap forward in terms of quality assurance; being able to prove correctness properties would be even better but not as much better as just being able to run code.
In practice, proving theorems about PL's means either choosing stylized subsets or using mechanized frameworks. It's just not efficient or cost-effective to prove theorems manually about artifacts as large as real-world languages. Mechanized frameworks like ACL2, Coq, Twelf, and Agda are making great progress, but they are still unwieldy and too challenging. We settled on ML as a simple, well-understood language that is well-suited for implementing PL's.
Down the road, if we were to use one of these mechanized frameworks, we'd have to transliterate the RI code into the meta-language of the framework we'd chosen, which would be stylistically similar to ML but wouldn't be ML. And the language wouldn't have any of the side effects of ML, so then we'd still have to add shift/reset on top of the meta-language, as well as mutation and exceptions (and possibly even non-termination).
that had already been looked at and understood by many people. Handwaving about the possibility of adding delimited continuations to that formalization is not the same thing, even if you are right in saying it is "easy".
I've written a paper that shows you can implement shift/reset in ML as a library using call/cc [1]. And call/cc is very well-understood -- it's been around for decades in Scheme and has been formalized repeatedly, and it's been in SML/NJ for over a decade. It's been added to models of ML in the literature and proved type-sound. More directly, shift/reset has been formalized as well.
But that's for the academic pin-heads. :) In much more conceptually appealing terms, this is my favorite explanation to date of delimited continuations:
blog.moertel.com/articles/2008/05/05/olegs-great-way-of-explaining-delimited-continuations
I like to imagine that demonstrates that as programmers we actually already have good intuitions about delimited continuations, and that they aren't as scary as they sound.
But you're right that this comes down to a question of clarity. Which option is more obfuscated: specifying the entire language in CPS just for one feature, or using an esoteric low-level language construct that isn't well-known outside of the research world? It's definitely a trade-off. But I don't think the real concern is whether this makes the spec "less formal." It's just a concern about clarity.
Dave
[1] www.ccs.neu.edu/home/dherman/research/papers/icfp07-great-escape.pdf
On Sep 2, 2008, at 2:16 PM, Lex Spoon wrote:
On Sun, Aug 24, 2008 at 3:17 AM, Brendan Eich <brendan at mozilla.org>
wrote:First, let's settle the hash over whether any desugaring without extensions such as return-to-label, reformed lexical scope, tamed this, banished arguments, etc. etc., trumps adding a new binding form, to wit: let as block scoped var.
With no extensions, it is true, return would end up returning from a different method under the proposed rewrite. Likewise, this and arguments would cause trouble.
Sorry for the very tardy reply. You make good points in the abstract,
and the messy language-specific details of existing semantics for
functions not being clean enough deserves a better response than just
"don't desugar".
I stand by "don't desugar let to functions as-is". I'm also pretty
certain "don't add more modes or subsets to try to fix existing forms"
is sound, since versionitis does not help us either keep the spec
simple or specify the backward-compatible semantics in the full
language.
So, to avoid trouble, we've been thinking of new forms including a
better function, call it lambda, that has none of the compatibility
baggage. I say "we" but really Dave Herman deserves credit for
championing this. A "lambda" form has been a topic now and then for a
while, on this list and in committee, and sometimes only as syntactic
sugar (which would miss the opportunity for semantic reform) -- yet
without it getting the breathing room it needs.
Dave is working now in the
space on the wiki. Don't throw stones, this is not in the harmony:
namespace for good reason. Constructive comments welcome. And I still
owe the list a story on wiki access that keeps Ecma happy and doesn't
throw open the edit wars doors.
Among the new strawman pages, the following are relevant and (I hope)
helpful:
strawman:lambdas, strawman:lexical_scope, strawman:return_to_label
Possibly break and continue would, depending on what their precise semantics are.
JS has break from labeled statement, and continue to labeled loop
bottom, a la Java. These look trouble-free to me. Let me know if you
see a hard case. Thanks,
On Thu, Oct 9, 2008 at 5:31 PM, Brendan Eich <brendan at mozilla.org> wrote:
JS has break from labeled statement, and continue to labeled loop bottom, a la Java. These look trouble-free to me. Let me know if you see a hard case. Thanks,
My question was whether the semantics of break and continue would support the following:
while(true) { (function() { if (--x == 0) break; })(); }
I honestly don't know, but it shouldn't cause any real trouble to allow it. The implementation would be analogous to that for labeled return. For example, if the appropriate while loop is no longer on the stack, the "break" would turn into an exception.
As my usual disclaimer, I am not closely following the different ES trends including Harmony. I'm only commenting about what could possibly make the language more consistent and orthogonal.
On Oct 9, 2008, at 3:05 PM, Lex Spoon wrote:
On Thu, Oct 9, 2008 at 5:31 PM, Brendan Eich <brendan at mozilla.org>
wrote: JS has break from labeled statement, and continue to labeled
loop bottom, a la Java. These look trouble-free to me. Let me know
if you see a hard case. Thanks,My question was whether the semantics of break and continue would
support the following:while(true) { (function() { if (--x == 0) break; })(); }
That's currently a specified error (possibly at runtime; chapter 16 of
ES3 allows it to be at compile time).
So a future edition could allow it, probably without opt-in
versioning. Our compatibility model does not guarantee exceptions,
since it allows contemporaneous extensions that remove those
exceptions (see ES3 chapter 16 again, second bulleted list, first
bullet -- these lists need their own sub-sections and numbers!).
I honestly don't know, but it shouldn't cause any real trouble to
allow it. The implementation would be analogous to that for labeled
return.
Right, and break to label outside the function's body, but lexically
in scope, would be completely analogous (or just "the same" ;-)):
L: while (true) {
(function () {
... // stuff possibly including a loop or switch that brackets
the next line
if (--x == 0) break L;
...
})();
}
For example, if the appropriate while loop is no longer on the
stack, the "break" would turn into an exception.
Yes, this new runtime exception is the price of admission.
The exception seems to a major source of grief in the Java BGGA
closures controversy, or at least it did when I last looked. But it
comes up with escape continuations in Scheme, and it is inevitable if
we want these kinds of program equivalences.
I'm interested in the reactions of others on the list to such "return/
break/continue from already-deactivated statement/frame" exceptions.
They could be caught and handled, of course. Feature and bug, dessert
topping and floor wax ;-).
On Thu, Oct 9, 2008 at 2:31 PM, Brendan Eich <brendan at mozilla.org> wrote:
[snip]
Sorry for the very tardy reply. You make good points in the abstract, and the messy language-specific details of existing semantics for functions not being clean enough deserves a better response than just "don't desugar".
I stand by "don't desugar let to functions as-is".
I argued for "let" desugaring to "function" and I understand the problems with "arguments", "this" and "return". In light of the "lambda" idea below and that "let" could desugar to that more intuitively (i.e. Tenent's principle) to "lambda", I think what I was really asking/arguing for was an axiomatic definition of the language. That is, "let" should should desugar to something more fundamental.
I'm also pretty certain "don't add more modes or subsets to try to fix existing forms" is sound, since versionitis does not help us either keep the spec simple or specify the backward-compatible semantics in the full language.
This makes really good sense to me. Things like modes that affect semantics of forms makes me uncomfortable.
So, to avoid trouble, we've been thinking of new forms including a better function, call it lambda,
Please call it "lambda"! :)
that has none of the compatibility baggage. I say "we" but really Dave Herman deserves credit for championing this.
Dave Herman for president!
A "lambda" form has been a topic now and then for a while, on this list and in committee, and sometimes only as syntactic sugar (which would miss the opportunity for semantic reform) -- yet without it getting the breathing room it needs.
Dave is working now in the
space on the wiki. Don't throw stones, this is not in the harmony: namespace for good reason. Constructive comments welcome. And I still owe the list a story on wiki access that keeps Ecma happy and doesn't throw open the edit wars doors.
Among the new strawman pages, the following are relevant and (I hope) helpful:
strawman:lambdas, strawman:lexical_scope, strawman:return_to_label
I wasn't sure if there was going to be a day when I saw these exciting ideas as potential/maybe ideas that might/perhaps end up in ECMAScript. I'm going to cross my fingers and start holding my breath now.
Peter
On Oct 9, 2008, at 3:31 PM, Peter Michaux wrote:
So, to avoid trouble, we've been thinking of new forms including a
better function, call it lambda,Please call it "lambda"! :)
We do. Hard to beat, IMHO -- even if it originated in a typographical
compromise (dn.codegear.com/article/33336 cites sources
averring that Church wrote something like a circumflex).
that has none of the compatibility baggage. I say "we" but really Dave Herman deserves credit for championing this.
Dave Herman for president!
He has my vote! ;-)
My question was whether the semantics of break and continue would support the following:
Yes, this is another good case to consider. Thanks for pointing it out; I'll add this to the strawman:lambdas proposal. Essentially this is another aspect of the semantics of 'function' that is implicit -- that it cancels out the scope of break/continue labels -- and it's precisely these implicit elements of a language feature that break expected equivalences. (They are essentially "unhygienic" -- if you push me, I can explain the connection to macros.)
while(true) { (function() { if (--x == 0) break; })(); }
Note that the 'lambda' proposal is intended to be separate from 'function', whose semantics would be left unchanged. So your example would still be disallowed, but this:
while(true) {
(lambda() {
if (--x == 0) break;
})();
}
would be allowed.
I honestly don't know, but it shouldn't cause any real trouble to allow it. The implementation would be analogous to that for labeled return. For example, if the appropriate while loop is no longer on the stack, the "break" would turn into an exception.
That's correct.
I argued for "let" desugaring to "function" and I understand the problems with "arguments", "this" and "return". In light of the "lambda" idea below and that "let" could desugar to that more intuitively (i.e. Tenent's principle) to "lambda", I think what I was really asking/arguing for was an axiomatic definition of the language. That is, "let" should should desugar to something more fundamental.
Exactly -- and from our standpoint, we weren't arguing that "let" shouldn't desugar to something more fundamental, just that 'function' wasn't the right primitive. My intention was for 'lambda' to be the "dialectical synthesis" to our debate. Glad to hear it has appeal.
Please call it "lambda"! :)
Knowing full well that we've already had long, silly, drawn-out, painful bike-shedding sessions on alternative names for 'lambda' on es-discuss, and at the risk of picking at old wounds... <anticipatory wince>
How would people feel about the declaration form being 'define' instead of lambda? As in:
define const(x) {
lambda(y) x
}
Maybe I'm just accustomed to Scheme, but it looks awkward to me for the declaration form to be called lambda. Dylan also used 'define'.
On Oct 9, 2008, at 4:28 PM, David Herman wrote:
How would people feel about the declaration form being 'define'
instead of lambda? As in:define const(x) { lambda(y) x }
Maybe I'm just accustomed to Scheme, but it looks awkward to me for
the declaration form to be called lambda. Dylan also used 'define'.
For named functions, it's less cryptic, it has clearer connotations.
For anonymous functions, e.g.:
(define (x) {...})(x)
or
return foo(define (y) {...}, z);
your mileage will vary, but it seems worse by a hair to me. But I'm
used to "lambda" as a term of art.
The obscurity of "lambda" helps it avoid collisions (we have ways of
unreserving keywords in property-name contexts, but these do not work
for formal parameters and variables named "define", which seem
likelier at a guess than "lambda" -- spidering the web could help
confirm this guess).
The obscurity also arguably partners "lambda" better with "function".
Setting up "define" as a cleaner "function" seems to switch domains of
discourse. Concretely, we have in ES3.1 Object.defineProperty and
similarly named functions. These "define" APIs were prefigured by
Object.prototype.defineGetter_, etc.. This sense of "define" has
meant "bind property name to value or getter/setter".
On the other side, Python, E, etc. use "def". But we would be verbose
like Scheme and Dylan. So "define" vs. "lambda".
End of my bike-shedding ruminations.
Sorry, I was unclear. I meant 'lambda' for the expression form and 'define' for the definition form.
On Thu, Oct 9, 2008 at 4:28 PM, David Herman <dherman at ccs.neu.edu> wrote:
[snip]
How would people feel about the declaration form being 'define' instead of lambda? As in:
define const(x) { lambda(y) x }
Maybe I'm just accustomed to Scheme, but it looks awkward to me for the declaration form to be called lambda. Dylan also used 'define'.
I think there is a good reason for concern that may be intuitively behind your "looks awkward". (That little voice in my head saying "ewwwwww" is usually correct.)
From Scheme's r5rs specification, two of the three declaration forms are
(define <variable> <expression>) (define (<variable> <formals>) <body>)
The second form for declaring lambdas is just shorthand for
(define <variable> (lambda (<formals>) <body>))
So there is a reason the declaration form for lambdas uses "define": it desugars to the first form which also uses "define". It gives a tidy parallel syntax for both kinds of declarations.
If we try to make the same parallelism in JavaScript with "var" we would have
var <variable> = <expression>
var <variable>(<formals>) <body>
or if we use "let" we have the following parallelism
let <variable> = <expression>
let <variable>(<formals>) <body>
So in your suggestion for using "define", does "define" mean "var" or "let" scoping of the declaration? Since it should probably mean one or the other, it might be better to allow the declaration forms for lambdas to just use either "var" or "let" depending how the programmer wants the declaration scoped. If new scoping keywords appear in the language (e.g. "our", "my") then new lambda declaration forms can use the new keywords and scoping of various declaration forms will be naturally parallel.
This keyword/scoping problem must already have appeared for functions as function declarations have "var" scoping and obtaining "let" scoping requires using something like "let a = function(){}". This is pretty ugly for functions to have "let" scoping but the good news is the door has been left open for real lambdas to snatch up the available "var a(){}" and "let a(){}" syntaxes.
Peter
On Oct 9, 2008, at 6:44 PM, David Herman wrote:
Sorry, I was unclear.
No, my fault for missing "declaration form".
I meant 'lambda' for the expression form and 'define' for the
definition form.
Do keywords cost more than concepts?
If people think define name(x) x and lambda (x) x are different
beasts, then different names win.
If people think of lambdas (what else to call them? not "functions")
as being named or not when used as expressions, and named when
declarations (parallel to the existing function forms are in ES3), but
being different (albeit similar; cleaned-up) function-like beasts,
then one name for the three forms that parallel the ES3 function forms
seems better.
It's hard to argue about from first principles outside of the
specifics of ES3 and modern JS implementations and teaching. The
specific JS books I know talk about functions having several forms, so
it seems better to me to keep lambda parallel. I see the appeal of
define for the declaration (definition, rather) form, though.
On Oct 9, 2008, at 8:57 PM, Peter Michaux wrote:
This keyword/scoping problem must already have appeared for functions as function declarations have "var" scoping and obtaining "let" scoping requires using something like "let a = function(){}". This is pretty ugly for functions to have "let" scoping
An agreement from TC39 this past sprint was that function definitions
directly nested in blocks, not specified by ES3, defined block-local
(let) bindings.
There was general aversion to 'let function f() ...', an earlier ES4
idea already on the ropes. Separating binding forms from function
definition and other options (such as const) avoids too many binding
forms ('let const', 'let function', etc.). But too many binding forms
is just too many, and the committee strongly favored using grammatical
placement to avoid adding more syntactic complexity.
but the good news is the door has been left open for real lambdas to snatch up the available "var a(){}" and "let a(){}" syntaxes.
There's no reason to add var a() {} given function a() {} as a direct
child of a program or function body. It seems to me let a(){} is
Dave's define. So we're back to function vs. define/lambda.
The idea of a desugaring let statement and let expression require
lambda, the reformed function (whether define wins or not). But let
declarations as "the new var" do not desugar to lambdas. They hoist,
even if forward references (use before set) are errors.
We haven't found a reformed var; I don't think there is one. This does
not mean let declarations are somehow not worth adding. They're a big
improvement on var declarations in our experience with let in JS1.7
and up.
Peter Michaux wrote:
This keyword/scoping problem must already have appeared for functions as function declarations have "var" scoping and obtaining "let" scoping requires using something like "let a = function(){}". This is pretty ugly for functions to have "let" scoping but the good news is the door has been left open for real lambdas to snatch up the available "var a(){}" and "let a(){}" syntaxes.
... and "const a(){}" or "let const a(){}", which are what you would want in preference to "var" or "let" in the majority of cases for function definitions.
(Is it just me, or is "let const" as proposed in proposals:block_expressions
a bit too verbose? E uses "def" for constant declarations, and IMHO it does matter that this is just as concise as "var".)
Brendan Eich wrote:
On Oct 9, 2008, at 3:05 PM, Lex Spoon wrote:
On Thu, Oct 9, 2008 at 5:31 PM, Brendan Eich <brendan at mozilla.org>
wrote: JS has break from labeled statement, and continue to labeled
loop bottom, a la Java. These look trouble-free to me. Let me know
if you see a hard case. Thanks,My question was whether the semantics of break and continue would
support the following:while(true) { (function() { if (--x == 0) break; })(); }
That's currently a specified error (possibly at runtime; chapter 16 of
ES3 allows it to be at compile time).So a future edition could allow it, probably without opt-in
versioning. Our compatibility model does not guarantee exceptions,
since it allows contemporaneous extensions that remove those
exceptions (see ES3 chapter 16 again, second bulleted list, first
bullet -- these lists need their own sub-sections and numbers!).
So what should f(5, 0) do?
function f(x, h) { while (true) { try { if (h == 0) h = function() {break}; if (x != 0) f(x-1, h); else h(); } catch (e) { alert("caught " + e + " on " + x); } finally { alert("f called finally on " + x); } alert("f looping on " + x); } alert("f exited on " + x); }
WaldemarOn Oct 10, 2008, at 1:25 PM, Waldemar Horwat wrote:
So what should f(5, 0) do?
function f(x, h) { while (true) { try { if (h == 0) h = function() {break};
Just to repeat something Dave wrote, we don't propose to allow break
in a function where the break is not in a labeled statement, switch,
or loop within that function. Only lambda would support such novelties.
if (x != 0) f(x-1, h); else h();
This will break from the while (true) in the outermost (x = 5)
activation of f.
In Scheme implementations that support it, the analogue is call/ec --
call-with-escape-continuation (weak continuation is another name for
escape continuation) -- where the caller's continuation is the
argument to the procedure passed to call/ec. Escape continuations are
cheaper to implement and simpler to reason about than full call/cc
continuations because of the dynamic error (exception) you get if you
call the escape continuation outside of the dynamic extent of the
current call.
Sorry if this is already known; Dave should wipe my chin as needed
since he is the adult Schemer here and I'm the toddler.
} catch (e) { alert("caught " + e + " on " + x); } finally { alert("f called finally on " + x); } alert("f looping on " + x); } alert("f exited on " + x); }
The break itself does not propagate as an exception, just to be clear.
If the statement being broken from is inactive, then an exception will
be thrown from the break evaluation in the function that was assigned
to h. The call to h would have to come after control flow had left the
outermost while (true), via a statement after that loop, or some other
call made via a returned or heap reference (upward funarg).
On Oct 10, 2008, at 3:31 PM, Brendan Eich wrote:
} catch (e) { alert("caught " + e + " on " + x); } finally { alert("f called finally on " + x); }
Skipping the intervening active finally clauses is bad, though -- a
bug in the current wiki rough draft that I should have mentioned. Dave
will post a follow-up soon.
Brendan Eich wrote:
On Oct 10, 2008, at 3:31 PM, Brendan Eich wrote:
} catch (e) { alert("caught " + e + " on " + x); } finally { alert("f called finally on " + x); }
Skipping the intervening active finally clauses is bad, though -- a bug in the current wiki rough draft that I should have mentioned. Dave will post a follow-up soon.
The only sensible thing would be to run all of them; local "break" even does that in ES3.
Of course that raises the question of what happens if a "break" whose matching activation frame is no longer live would generate an error but a finally clause would have caught and diverted it.
Waldemar
David Herman wrote:
My question was whether the semantics of break and continue would support the following:
Yes, this is another good case to consider. Thanks for pointing it out; I'll add this to the strawman:lambdas proposal. Essentially this is another aspect of the semantics of 'function' that is implicit -- that it cancels out the scope of break/continue labels -- and it's precisely these implicit elements of a language feature that break expected equivalences. (They are essentially "unhygienic" -- if you push me, I can explain the connection to macros.)
First off, I'm really glad that "clean" functions are being considered for ES-Harmony - another step toward hygienic macros!
I read through the strawman:lambdas proposal and saw that it did not mention anything about |var|, e.g.
(function() { lambda { var x = 10; }(); return x; })()
Does the |var| within the lambda define a var in the function body, and does that var declaration hoist to the top of the function body?
Brendan Eich wrote:
There was general aversion to 'let function f() ...', an earlier ES4
idea already on the ropes. Separating binding forms from function
definition and other options (such as const) avoids too many binding
forms ('let const', 'let function', etc.). But too many binding forms
is just too many, and the committee strongly favored using grammatical
placement to avoid adding more syntactic complexity.but the good news is the door has been left open for real lambdas to snatch up the available "var a(){}" and "let a(){}" syntaxes.
There's no reason to add var a() {} given function a() {} as a direct
child of a program or function body. It seems to me let a(){} is
Dave's define. So we're back to function vs. define/lambda.
I wonder if we really need named lambdas in the first place. I consider the whole named function/let/var thing to be messy enough already. Is it really that too difficult to do this:
let f = lambda() { ... }
And if predictable recursion is an issue, you can always do this (assuming this form is in ES-Harmony):
let (f = lambda() { ... }) ...
I don't consider keeping symmetry with named functions to be a convincing argument, since the proposed named lamda works on the block level rather than the function level, unlike named functions. There's bound to be confusion regardless.
Does the |var| within the lambda define a var in the function body, and does that var declaration hoist to the top of the function body?
Good catch, thank you. Since lambda' is designed to have no implicit semantics, it can't introduce a newvar' frame. IOW, the var' would still be hoisted to the nearest enclosingfunction', not `lambda'. I'll add this to the wiki page.
Thanks,
if (h == 0) h = function() {break};
Did you mean if (x == 0)? That's been confusing me in trying to read your example.
The only sensible thing would be to run all of them; local "break" even does that in ES3.
Yes.
Of course that raises the question of what happens if a "break" whose matching activation frame is no longer live would generate an error but a finally clause would have caught and diverted it.
Here's roughly the semantics of return-to-label:
- return-to-label first checks to see if the label is live on the stack
- if not, it raises an exception from the point where the return was attempted
- but if so, it attempts to unwind the stack to the point where the label was executed, passing through any intervening finally clauses
- if any of the finally clauses has its own non-local exit, this interrupts and aborts the unwinding
Thank you for pointing out, though, that try/catch isn't so easily defined on top of return-to-label, since it still needs special handling for finally. The options are either to define a lower-level primitive underlying try/finally (akin to Scheme's dynamic-wind), or to leave exceptions -- or at least try/finally -- as primitive. I lean towards the latter; dynamic-wind is a subtle beast.
[For those interested in dynamic-wind, Flatt et al's ICFP 07 paper has a nice investigation into its semantics: www.cs.utah.edu/plt/publications/icfp07-fyff.pdf]
Yuh-Ruey Chen wrote:
David Herman wrote:
My question was whether the semantics of break and continue would support the following: Yes, this is another good case to consider. Thanks for pointing it out; I'll add this to the strawman:lambdas proposal. Essentially this is another aspect of the semantics of 'function' that is implicit -- that it cancels out the scope of break/continue labels -- and it's precisely these implicit elements of a language feature that break expected equivalences. (They are essentially "unhygienic" -- if you push me, I can explain the connection to macros.)
Consider yourself pushed.
First off, I'm really glad that "clean" functions are being considered for ES-Harmony - another step toward hygienic macros!
I read through the strawman:lambdas proposal and saw that it did not mention anything about |var|, e.g.
(function() { lambda { var x = 10; }(); return x; })()
Does the |var| within the lambda define a var in the function body, and does that var declaration hoist to the top of the function body?
To satisfy Tennent's correspondence, all var statements must hoist in the same way ignoring lambdas -- i.e. to the top of the innermost enclosing function body (if not in the global scope).
On Sat, Oct 11, 2008 at 7:12 AM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
Yuh-Ruey Chen wrote:
I read through the strawman:lambdas proposal and saw that it did not mention anything about |var|, e.g.
(function() { lambda { var x = 10; }(); return x; })()
Does the |var| within the lambda define a var in the function body, and does that var declaration hoist to the top of the function body?
To satisfy Tennent's correspondence, all var statements must hoist in the same way ignoring lambdas -- i.e. to the top of the innermost enclosing function body (if not in the global scope).
How to define a variable that is local to the enclosing lambda? Isn't the ability to do that essential?
Peter
On Fri, Oct 10, 2008 at 7:28 PM, Yuh-Ruey Chen <maian330 at gmail.com> wrote:
I wonder if we really need named lambdas in the first place. I consider the whole named function/let/var thing to be messy enough already. Is it really that too difficult to do this:
let f = lambda() { ... }
And if predictable recursion is an issue, you can always do this (assuming this form is in ES-Harmony):
let (f = lambda() { ... }) ...
I don't consider keeping symmetry with named functions to be a convincing argument, since the proposed named lamda works on the block level rather than the function level, unlike named functions. There's bound to be confusion regardless.
I think it would be ok to have only unnamed lambdas. (It would be ok to have named lambdas too.)
As it stands, I always write the following in ES3
var f = function() {};
and now that arguments.callee is on the chopping block, I've started writing recursion as the painful contortion below
var f = (function() { var callee = function() { callee(); }; return callee; })();
Peter
On Sat, Oct 11, 2008 at 9:11 AM, Peter Michaux <petermichaux at gmail.com> wrote:
As it stands, I always write the following in ES3
var f = function() {};
and now that arguments.callee is on the chopping block, I've started writing recursion as the painful contortion below
var f = (function() { var callee = function() { callee(); }; return callee; })();
I don't get it. Why not write
var f = function() { f(); };
?
On Sat, Oct 11, 2008 at 10:02 AM, Mark S. Miller <erights at google.com> wrote:
On Sat, Oct 11, 2008 at 9:11 AM, Peter Michaux <petermichaux at gmail.com> wrote:
As it stands, I always write the following in ES3
var f = function() {};
and now that arguments.callee is on the chopping block, I've started writing recursion as the painful contortion below
var f = (function() { var callee = function() { callee(); }; return callee; })();
I don't get it. Why not write
var f = function() { f(); };
var f = function() {f();}; var g = f; f = function(){alert('broke your recursion :)');}; g();
The call to g will not create an infinite loop which is the intended behavior of the function object defined in the first line. The late binding of the recursive call to whatever f happens to reference is the problem.
Peter
David Herman wrote:
Does the |var| within the lambda define a var in the function body, and does that var declaration hoist to the top of the function body?
Good catch, thank you. Since
lambda' is designed to have no implicit semantics, it can't introduce a newvar' frame. IOW, thevar' would still be hoisted to the nearest enclosingfunction', not `lambda'. I'll add this to the wiki page.Thanks, Dave
Also, I wonder why lambda in it's block-less form is restricted to expressions. I mean, why can't the following be valid?
lambda() for (p in x) ... lambda() if (cond) ... else ...
The reason block-less functions are restricted to expressions is that they have an implicit return. With lambdas, there is no implicit return
- the lamba's returned value is always going to be the value of the tail position expression.
With this change, this would simply the grammar down to:
Expression ::= ... | lambda Formals Statement
The advantage of this is that it will allow in the future more seamless "statement" macros that follow C conventions, i.e. a statement can include a nested block-less statement, e.g. stmtMacro() stmt.
Peter Michaux wrote:
On Sat, Oct 11, 2008 at 10:02 AM, Mark S. Miller <erights at google.com> wrote:
On Sat, Oct 11, 2008 at 9:11 AM, Peter Michaux <petermichaux at gmail.com> wrote:
As it stands, I always write the following in ES3
var f = function() {};
and now that arguments.callee is on the chopping block, I've started writing recursion as the painful contortion below
var f = (function() { var callee = function() { callee(); }; return callee; })(); I don't get it. Why not write
var f = function() { f(); };
var f = function() {f();}; var g = f; f = function(){alert('broke your recursion :)');}; g();
/const/ var f = function() { f(); };
Then just treat /const/ variables as if they were 'const', until at some point you no longer care about old versions of JScript and can globally replace '/const/ var' -> 'const'.
On Oct 11, 2008, at 9:05 AM, Peter Michaux wrote:
How to define a variable that is local to the enclosing lambda? Isn't the ability to do that essential?
Use let (the var replacement declaration form).
Peter Michaux wrote:
On Sat, Oct 11, 2008 at 7:12 AM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
Yuh-Ruey Chen wrote:
I read through the strawman:lambdas proposal and saw that it did not mention anything about |var|, e.g.
(function() { lambda { var x = 10; }(); return x; })()
Does the |var| within the lambda define a var in the function body, and does that var declaration hoist to the top of the function body? To satisfy Tennent's correspondence, all var statements must hoist in the same way ignoring lambdas -- i.e. to the top of the innermost enclosing function body (if not in the global scope).
How to define a variable that is local to the enclosing lambda?
Use 'let'. The body of a lambda is a block, so a variable declared with 'let' will be local to that block (this still satisfies Tennent's correspondence, in both the expression and statement forms as described in gafter.blogspot.com/2006/08/tennents-correspondence-principle-and.html.)
Isn't the ability to do that essential?
Yes.
On Sat, Oct 11, 2008 at 11:59 AM, Brendan Eich <brendan at mozilla.com> wrote:
On Oct 11, 2008, at 9:05 AM, Peter Michaux wrote:
How to define a variable that is local to the enclosing lambda? Isn't the ability to do that essential?
Use let (the var replacement declaration form).
Sounds good to me but it is a little confusing to keep track if "let" is either in or out of ES-Harmony and if it is partly in then which of the several JavaScript 1.7 uses are in and if there will be "let", "let*", "letrec" semantics.
Peter
How to define a variable that is local to the enclosing lambda? Isn't the ability to do that essential?
No. With all due respect to Brendan, var' hoisting to the top of a function body is one of the more problematic aspects of ES's semantics. If you want a local variable, uselet' -- it'll be local to its containing block. If you want a variable that is local to the entire body of a lambda', uselet' at the top level of the `lambda' body.
Also, I wonder why lambda in it's block-less form is restricted to expressions.
I'm with you... but I'd want to check with the experts on the ES grammar to see whether this introduces any nasty ambiguities.
Sounds good to me but it is a little confusing to keep track if "let" is either in or out of ES-Harmony and if it is partly in then which of the several JavaScript 1.7 uses are in and if there will be "let", "let*", "letrec" semantics.
I've got no crystal ball, but I'd say it'd be unlikely (and terribly silly) that we'd have lambda' without havinglet'.
(The lambda' form would fail in its requirement as an equivalence-preserving primitive if it became a target ofvar'-hoisting.)
On Sat, Oct 11, 2008 at 12:52 PM, Peter Michaux <petermichaux at gmail.com> wrote:
Use let (the var replacement declaration form).
Sounds good to me but it is a little confusing to keep track if "let" is either in or out of ES-Harmony and if it is partly in then which of the several JavaScript 1.7 uses are in and if there will be "let", "let*", "letrec" semantics.
The let declaration is in. Like const and function declarations, it has block-level letrec lexical scoping. I continue to oppose let expressions and let statements, as they don't provide enough additional power to justify their cost. For example, if lambda supports optional args, as I think we all agree it should, then, adapting a suggestion of Lars, we should define the scope in which the default value expressions are evaluated so that
var x = 2;
(lambda (x = 3, y = x) (x+y))()
returns 5, not 6. This gives us an easy enough let. The builtin let declarations provide an easy letrec. Given these, no one will miss let*.
On Sat, Oct 11, 2008 at 1:21 PM, Mark S. Miller <erights at google.com> wrote:
The let declaration is in. Like const and function declarations, it has block-level letrec lexical scoping. I continue to oppose let expressions and let statements, as they don't provide enough additional power to justify their cost. For example, if lambda supports optional args, as I think we all agree it should, then, adapting a suggestion of Lars, we should define the scope in which the default value expressions are evaluated so that
var x = 2;
(lambda (x = 3, y = x) (x+y))()
Simplifying
(lambda (x = 3, y = x) (x+y))()
to just
let (x = 3, y = x) (x+y)
makes it much more clear when reading code what the intention of the programmer was. The reader doesn't need to interpret the code with as many steps and doesn't need to check at the end of the expression to see if the lambda is immediately called or not. This checking is an issue when an immediately called lambda spans more than a screen. Repeatedly writing the whole immediately-called, anonymous function is boilerplate for such a common pattern. I admit it is not much character boiler plate but it is the fact that the pattern is so common that I believe the concept is worth abstracting. It is the mental boilerplate that is the waste.
Peter
On Sat, Oct 11, 2008 at 1:36 PM, Peter Michaux <petermichaux at gmail.com> wrote:
Simplifying
(lambda (x = 3, y = x) (x+y))()
to just
let (x = 3, y = x) (x+y)
makes it much more clear when reading code what the intention of the programmer was.
If the need for this case were common, that would be something. But how often do cases come up for which the letrec-like let declaration is not adequate? In hand written source, I think it's very rare. In machine generated code, including explanatory expansions, it's quite common. But these can use the lambda form without confusion.
On Sat, Oct 11, 2008 at 9:11 AM, Peter Michaux <petermichaux at gmail.com> wrote:
I think it would be ok to have only unnamed lambdas. (It would be ok to have named lambdas too.)
I think we should not introduce named lambdas because then we'd need to decide whether the scoping of a lambda name works the same as the scoping of a function name. Do we really want to reproduce the following confusions:
- Distinction between named function expressions vs named function declarations?
- The name of a name function expression is only in scope within the function, whereas the name of a named function declaration is in (letrec) scope in the containing block.
- Because these look the same, a named function expression cannot be used naked as the start of an expression statement. It must wear protective parentheses.
- For consistency, an anonymous function expression also cannot be used as a naked as the start of an expression statement.
If lambdas had optional names, we would either need to reproduce the above confusions or deviate from them. Either choice is terribly unpleasant.
On Oct 11, 2008, at 12:55 PM, David Herman wrote:
How to define a variable that is local to the enclosing lambda? Isn't the ability to do that essential?
No. With all due respect to Brendan, `var' hoisting to the top of a
function body is one of the more problematic aspects of ES's
semantics.
I agree, it's no skin off my nose -- 'var' hoisting was an artifact of
function implementation in Netscape 2, and did not apply to global
vars then. It was standardized as hoisting in all kinds of code
(global, function, and eval). We are stuck with it. However, hoisting
still applies to let:
If you want a local variable, use
let' -- it'll be local to its containing block. If you want a variable that is local to the entire body of alambda', uselet' at the top level of thelambda' body.
While let is local to containing block, the let-as-new-var proposal
(implemented in Firefox 2 and up) hoists to top of block. So you
cannot initialize the inner x using the outer x's value:
{ let x = 42; { let x = x; // undefined, not 42. alert(x); } alert(x); // 42, of course }
We've discussed making use-before-set a strict error, but we've
avoided it. The initialiser is not mandatory, and we do not wish to
impose costly analysis on small implementations.
On Oct 11, 2008, at 12:52 PM, Peter Michaux wrote:
On Sat, Oct 11, 2008 at 11:59 AM, Brendan Eich <brendan at mozilla.com>
wrote:On Oct 11, 2008, at 9:05 AM, Peter Michaux wrote:
How to define a variable that is local to the enclosing lambda?
Isn't the ability to do that essential?Use let (the var replacement declaration form).
Sounds good to me but it is a little confusing to keep track if "let" is either in or out of ES-Harmony
I do not see why you are confused. I wrote, in the original
"ECMAScript Harmony" post:
I heard good agreement on low-hanging "de-facto standard" fruit, particularly let as the new var, to match block-scoped const as still proposed (IIRC) in 3.1.
See esdiscuss/2008-August/006837 .
and if it is partly in then which of the several JavaScript 1.7 uses are in and if there will be "let", "let*", "letrec" semantics.
It's something else. See my reply about hoisting, just sent.
On Oct 11, 2008, at 2:43 PM, Brendan Eich wrote:
On Oct 11, 2008, at 12:52 PM, Peter Michaux wrote:
and if it is partly in then which of
the several JavaScript 1.7 uses are in and if there will be "let", "let*", "letrec" semantics.
It's something else. See my reply about hoisting, just sent.
Mark and others have described the let declaration (let as new var) as
like letrec. Close, but since forward references are possible and
initiialization in the declaration is not mandatory, it's analogous,
not identical. The function definition form in JS is letrec. But let
as new var? I still say it's "something else"; I admit that "like
letrec" or "letrec-ilke" works.
On Sat, Oct 11, 2008 at 2:42 PM, Brendan Eich <brendan at mozilla.com> wrote:
We've discussed making use-before-set a strict error, but we've avoided it. The initialiser is not mandatory, and we do not wish to impose costly analysis on small implementations.
Since "const" use-before-set is an unconditional error (i.e., not dependent on strictness), implementations already have to pay for a read-barrier mechanism. Since we'd like to support type-annotation-constraints on initialized let variables, I think initialized let variables should have an unconditional read barrier as well.
On Oct 11, 2008, at 2:24 PM, Mark S. Miller wrote:
On Sat, Oct 11, 2008 at 1:36 PM, Peter Michaux
<petermichaux at gmail.com> wrote:Simplifying
(lambda (x = 3, y = x) (x+y))()
to just
let (x = 3, y = x) (x+y)
makes it much more clear when reading code what the intention of the programmer was.
If the need for this case were common, that would be something. But how often do cases come up for which the letrec-like let declaration is not adequate? In hand written source, I think it's very rare.
One easily inspected source repository search (there are other
repositories) is
mxr.mozilla.org/firefox/search?string=let+(
yielding the results appended at the bottom of this message (mxr is
slow, so I thought I'd include them and save others the wait).
The reason people like let blocks and expressions as opposed to let
declarations, from what I can tell, is precisely the lack of hoisting
to top of block done for let declarations, and the greater clarity of
head-scoped initialization (possibly using outer names that will be
shadowed in the block or expression, but usually not) of the 'let
(head) ...' syntax.
Of course everything could be lambda-coded, but that's not harmonious
or usable. Just let as new var is agreed upon, and let blocks and
expressions could be let coded if there is no name shadowing (else
they'd need lambda-coding). But we should not dismiss the use-cases
for let blocks/expressions out of hand. There is experience in JS1.7
and 1.8 to consider here, not just hypothetical arguments about how
often hand-written cases come up.
Based on the lambda proposal, we could unify let expressions with
blocks, avoiding two forms. That is, you could use let (x = y) { /*
statements here */ } as an expression or in a statement context. TC
would want return, etc. to work as for lambda. Thus let with a head
would become lambda-call sugar (a bit more than a spoonful when you
add up the three more letters and the () to call).
(This is incompatible with JS1.7+ but if standardized, JS1.x would
track the standard and carry compatibility costs under the old
versions.)
/be
/toolkit/mozapps/downloads/src/DownloadUtils.jsm (View CVS log or CVS
annotations)
* line 103 -- let ([n, v] = [name, value])
* line 178 -- let (transfer =
DownloadUtils.getTransferTotal(aCurrBytes, aMaxBytes)) {
* line 184 -- let ([rate, unit] =
DownloadUtils.convertByteUnits(aSpeed)) {
* line 192 -- let ([timeLeft, newLast] =
DownloadUtils.getTimeLeft(seconds, aLastSec)) {
* line 271 -- let (diff = aSeconds - aLastSec) {
/toolkit/mozapps/downloads/content/downloads.js (View CVS log or CVS
annotations)
* line 438 -- let (sb =
document.getElementById("downloadStrings")) { * line 946 -- let (stateSize = {}) { * line 1127 -- let (empty = gDownloadsView.cloneNode(false)) { * line 1190 -- let (referrer = gStmt.getString(7)) {
/toolkit/content/widgets/autocomplete.xml (View CVS log or CVS
annotations)
* line 1268 -- regions = regions.sort(function(a, b) let (start =
a[0] - b[0])
/browser/base/content/browser.js (View CVS log or CVS annotations)
* line 6695 -- let (bundle =
document.getElementById("bundle_browser")) {
/browser/components/preferences/applications.js (View CVS log or CVS
annotations)
* line 1848 -- let (preferredApp =
aHandlerInfo.preferredApplicationHandler) {
/js/src/xpconnect/tests/mochitest/test_wrappers.html (View CVS log or
CVS annotations)
* line 32 -- let (obj = { a: 3 }) {
* line 50 -- let (obj = {}) {
/netwerk/test/unit/test_bug414122.js (View CVS log or CVS annotations)
* line 39 -- let (rest = line.substring(2))
/content/xbl/test/test_bug389322.xhtml (View CVS log or CVS annotations)
* line 17 -- let (x=1) (x);
* line 28 -- let (x=1) (x);
* line 37 -- let (x=1) (x);
* line 48 -- let (x=1) (x);
* line 60 -- let (x=1) (x);
* line 75 -- let (x=1) (x);
* line 109 -- let (x=1) (x);
* line 117 -- let (x=1) (x);
On Sat, Oct 11, 2008 at 5:24 PM, Mark S. Miller <erights at google.com> wrote:
On Sat, Oct 11, 2008 at 1:36 PM, Peter Michaux <petermichaux at gmail.com> wrote:
Simplifying
(lambda (x = 3, y = x) (x+y))()
to just
let (x = 3, y = x) (x+y)
makes it much more clear when reading code what the intention of the programmer was.
If the need for this case were common, that would be something. But how often do cases come up for which the letrec-like let declaration is not adequate? In hand written source, I think it's very rare.
This may be true of ES (though, really, it's impossibly to say, since ES doesn't yet have 'let,' and we don't know what idioms will become popular). In Scheme, however, this sort of thing is very common, especially in the named-let form:
(define (frob-list lst) (let loop ((lst lst)) ...))
... is a common idiom. A necessary one? Of course not. Beside the point, since ES has explicit iteration forms? Maybe, but there certainly are cases where tail recursion is more useful in ES than 'for' or 'while' iteration. This is particularly true when constructing closures in a loop, where each closure needs to refer to a distinct element of the iterated sequence. (You really don't want to be mutating an induction variable in these cases.)
At any rate, this is, at best, a side issue. I'd just like a non-hoisting 'let.' Based on what you wrote about hoisting earlier in this thread, I'm surprised you don't want one, too.
first, my compliments on your lambda proposal. This should significantly simplify the core language after expanding sugars -- especially if you succeed at redefining function as desugaring to lambdas. That would be awesome!
On Sat, Oct 11, 2008 at 5:34 AM, David Herman <dherman at ccs.neu.edu> wrote:
Here's roughly the semantics of return-to-label:
- return-to-label first checks to see if the label is live on the stack
- if not, it raises an exception from the point where the return was attempted
- but if so, it attempts to unwind the stack to the point where the label was executed, passing through any intervening finally clauses
- if any of the finally clauses has its own non-local exit, this interrupts and aborts the unwinding
Thank you for pointing out, though, that try/catch isn't so easily defined on top of return-to-label, since it still needs special handling for finally. The options are either to define a lower-level primitive underlying try/finally (akin to Scheme's dynamic-wind), or to leave exceptions -- or at least try/finally -- as primitive. I lean towards the latter; dynamic-wind is a subtle beast.
[For those interested in dynamic-wind, Flatt et al's ICFP 07 paper has a nice investigation into its semantics: www.cs.utah.edu/plt/publications/icfp07-fyff.pdf]
I haven't looked at the Flatt paper yet. But the semantics you explain above corresponds exactly to E's escape expressions. And E also treats try/finally primitive for similar reasons. (Historical note: The E escape construct and the call/ec previously explained both originate in an "escape" statement by Reynolds from a paper in the late 60s or early 70s. I don't recall whether Reynolds had any mechanism analogous to try/finally.)
On Oct 11, 2008, at 3:16 PM, Brendan Eich wrote:
Of course everything could be lambda-coded, but that's not harmonious or usable. Just let as new var is agreed upon, and let blocks and expressions could be let coded if there is no name shadowing (else they'd need lambda-coding).
Of course, let expressions would need lambda-coding no matter what
names were shadowed. The experience gained in JS1.7+ shows more let
block usage than let expression, but expression temporaries (lacking
macros and ignoring automatically generated code) are uncommon in
today's JS. The function-expression-immediately-applied cliché usually
has statements for effect, if not a return -- not a single expression
in its body.
On Oct 11, 2008, at 2:55 PM, Mark S. Miller wrote:
On Sat, Oct 11, 2008 at 2:42 PM, Brendan Eich <brendan at mozilla.com>
wrote:We've discussed making use-before-set a strict error, but we've avoided it. The initialiser is not mandatory, and we do not wish to impose costly analysis on small implementations.
Since "const" use-before-set is an unconditional error (i.e., not dependent on strictness), implementations already have to pay for a read-barrier mechanism. Since we'd like to support type-annotation-constraints on initialized let variables, I think initialized let variables should have an unconditional read barrier as well.
If using an uninitialized let binding is an error, then hoisting is
pointless except to make the statements between start of block and the
let declaration a dead zone for the binding name. This fits the
ancient, weak but not entirely worthless post-hoc rationale for var
hoisting (to avoid confusion among novice or inexperienced programmers
by making many scopes, each implicitly opened by var), but it's not
particularly useful.
What's more, as discussed here and in TC39, repeated let declarations
for the same binding name within the same block should be allowed.
Anything else is user- and refactoring-hostile. So the non-initial let
declarations for a given name in the same block would be ignored.
This leaves efficiency as a concern. If implementations do not do the
analysis required to catch use before set at compile time, then let as
well as const pays the read barrier price. It's non-trivial. Today let
(and var in the absence of eval) can be optimized to use a "read" in
the VM, possibly even a load instruction -- no getter barrier
required. Whatever the GC write barrier cost, reads dominate and this
is a significant savings.
On the other hand, computing SSA including dominance relations while
parsing (in one pass) is possible so long as we do not add goto to the
language. So maybe the analysis is acceptable to modern
implementations, which are increasingly sophisticated.
Still, it is not yet agreed that let use before set shall be an error.
It certainly is not the case for var, and working code counts on the
undefined default initial value.
On Sat, Oct 11, 2008 at 3:25 PM, Jon Zeppieri <jaz at bu.edu> wrote:
[...] I'd just like a non-hoisting 'let.' Based on what you wrote about hoisting earlier in this thread, I'm surprised you don't want one, too.
History aside, my preference would be what E does: A variable is in scope textually starting at the point of declaration, left-to-right, until the close of its enclosing scope block. I would love it if this applied to lets, consts, and functions. However, given that ES3 functions hoist (within the top-level function body), I think the least surprising least asymmetric compromise is to generalize functions to be per-block, and to have const and let scope like functions will.
On Sat, Oct 11, 2008 at 3:30 PM, Brendan Eich <brendan at mozilla.com> wrote:
What's more, as discussed here and in TC39, repeated let declarations for the same binding name within the same block should be allowed. Anything else is user- and refactoring-hostile. So the non-initial let declarations for a given name in the same block would be ignored.
What if repeated let declarations constrain the same variable with different type-annotation-constraints?
If we want to avoid the read-barrier, we should not hoist either const or let. If we are to consider not hoisting const, WE NEED TO DECIDE THIS NOW, before ES3.1 mandates a hoisting const.
On Sat, Oct 11, 2008 at 6:38 PM, Mark S. Miller <erights at google.com> wrote:
On Sat, Oct 11, 2008 at 3:25 PM, Jon Zeppieri <jaz at bu.edu> wrote:
[...] I'd just like a non-hoisting 'let.' Based on what you wrote about hoisting earlier in this thread, I'm surprised you don't want one, too.
History aside, my preference would be what E does: A variable is in scope textually starting at the point of declaration, left-to-right, until the close of its enclosing scope block. I would love it if this applied to lets, consts, and functions. However, given that ES3 functions hoist (within the top-level function body), I think the least surprising least asymmetric compromise is to generalize functions to be per-block, and to have const and let scope like functions will.
I agree. The let declaration has to hoist, or else it isn't the new var, and then we'll have to throw out all those tee shirts. That's why I want a let block. (Actually, as I posted some time ago, I'd prefer to have the let block instead of the let declaration, but I realize that's a non-starter with most of the TC members.)
On Oct 11, 2008, at 3:45 PM, Mark S. Miller wrote:
On Sat, Oct 11, 2008 at 3:30 PM, Brendan Eich <brendan at mozilla.com>
wrote:What's more, as discussed here and in TC39, repeated let
declarations for the same binding name within the same block should be allowed.
Anything else is user- and refactoring-hostile. So the non-initial let
declarations for a given name in the same block would be ignored.What if repeated let declarations constrain the same variable with different type-annotation-constraints?
The agreement from the May TC39 meeting was that the declarations
implicit (:*) and explicit annotations must normalize to the same
type, or there's an error.
If we want to avoid the read-barrier, we should not hoist either const or let. If we are to consider not hoisting const, WE NEED TO DECIDE THIS NOW, before ES3.1 mandates a hoisting const.
A few messages back you nicely repeated the least-concepts/least-
astonishment/most-symmetric case for hoisting to block top that has
carried so far. I don't see how we can backtrack here. We'll flail hard.
Scheme has s-expression boundaries for every binding form. C-like
languages do not, and that makes invisible start of scope boundaries
for new bindings that appear mid-block (for C++ and languages that do
not require declarations before statements). Indeed C++ has implicit
copy constructions scoped within expressions. We don't want any of
that, in my view. We want explicit forms that look different when they
scope differently.
Either the read barrier must be swallowed (implemented using analysis
or runtime checking), or else untyped let (as untyped var) can be used
before set, resulting in undefined.
On Sat, Oct 11, 2008 at 3:26 PM, Brendan Eich <brendan at mozilla.com> wrote:
Of course, let expressions would need lambda-coding no matter what names were shadowed. The experience gained in JS1.7+ shows more let block usage than let expression, but expression temporaries (lacking macros and ignoring automatically generated code) are uncommon in today's JS. The function-expression-immediately-applied cliché usually has statements for effect, if not a return -- not a single expression in its body.
Here's an odd idea. (I'm not yet advocating this; just brainstorming)
What if the body of a lambda were always a block (curlies enclosing a statement list), but, by analogy to eval, if the last statement executed is an expression statement, then invoking the lambda returns the value of that last expression statement. Let's say, further, that a lambda's parameter list is syntactically optional. Together, this would result in
lambda{ .... }
being a first-class control-flow block as in Smalltalk. Then, if we did want let-blocks that desugar to lambda, with no additional complexity, a let block could be an expression even though its body had statements. One construct would serve both as let expressions and let blocks, and would "for free" turn JavaScript into an expression language.
3 + let{ while(...){...}; 4; }
would either not terminate or evaluate to 7. None of this would violate TC.
Read the proposal again: the statement form of lambdas does return
the value of its last expression; this is what ES3 calls the
"completion value."
On Sat, Oct 11, 2008 at 4:05 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Read the proposal again: the statement form of lambdas does return the value of its last expression; this is what ES3 calls the "completion value."
Cool! So why are we still discussing proposed let expressions and let blocks as distinct constructs?
On Sat, Oct 11, 2008 at 4:01 PM, Brendan Eich <brendan at mozilla.com> wrote:
If we want to avoid the read-barrier, we should not hoist either const or let. If we are to consider not hoisting const, WE NEED TO DECIDE THIS NOW, before ES3.1 mandates a hoisting const.
A few messages back you nicely repeated the least-concepts/least-astonishment/most-symmetric case for hoisting to block top that has carried so far. I don't see how we can backtrack here. We'll flail hard. [...other good stuff snipped...]
I'm convinced; thanks. I agree it's too late to consider non-hoisting const even if we come to regret it. Given that const hoists, let declarations must as well, and we can argue about let-read-barriers and repeated declarations later.
On Sat, Oct 11, 2008 at 4:05 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Read the proposal again: the statement form of lambdas does return the value of its last expression; this is what ES3 calls the "completion value."
At strawman:lambdas:
Expression ::= ... | lambda Formals Block | lambda Formals Expression Declaration ::= ... | lambda Identifier Formals Block | lambda Identifier Formals Expression
How about just
Expression ::= ... | lambda Formals? Block
?
On Oct 11, 2008, at 7:13 PM, Mark S. Miller wrote:
Cool! So why are we still discussing proposed let expressions and let blocks as distinct constructs?
I'm open to just having a single form, which is an expression whose
body is a statement. In fact, waaaaay back (about 2 to 2.5 years ago)
the original proposal I wrote for let' was a single form which was an expression whose body was a block. And it used the completion value. It spun off into the let declaration form (which I personally think is a good thing) but withoutlambda' it seemed more natural to have
separate expression and statement forms.
I am most definitely in favor of steps towards unifying statements and
expressions, and I think the completion value is the glue between
them. It'll never be a perfect marriage, because some C-isms are just
so darned imperative (e.g., switch cases requiring break; e.g., loop
forms working via mutation), and C's expression forms are so clunky
(e.g., ?:). But those are minor warts compared to the benefit of
taking advantage of the completion value.
How about just
Expression ::= ... | lambda Formals? Block
?
Or even
Expression ::= ... | lambda Formals? Statement
as Yuh-Ruey suggested. I don't know if that causes nasty ambiguities,
but I'm open to it. Also, the expression-body form falls out naturally
as an instance of ExpressionStatement.
[Hm, one ambiguity is "lambda(x)(x)" could be read as either a
headless lambda with the body (x) being applied to (x) or as a headful
lambda with the body (x). I'm not in love with the optional formals,
since it doesn't have Smalltalk's incredibly lightweight syntax. I
think being able to leave off the formals list is the biggest win when
the syntax for closures is ultra-cheap. Otherwise the savings of the
two characters '(' ')' isn't really that important.]
On Oct 11, 2008, at 4:02 PM, Mark S. Miller wrote:
Let's say, further, that a lambda's parameter list is syntactically optional.
This makes for ambiguities if we do not require a braced expression,
as Dave pointed out. Requiring braces always goes against the
expression/statement unification thrust of the proposal. Letting
(formals) be omitted only if a { follows lambda seems like too much
special-casing.
Most lambdas will take at least one parameter, judging from existing
languages and use of functions in JS. It is not obviously worth the
complexity in avoiding ambiguity just to cater to the zero-formals
case. I say it's not, since if () is two chars too many, then -bda is
still three too many (lam- ties let).
The let block/expression idea, however it shakes out, should be viewed
as sugar for (lambda (bindings) body)(...). There the savings is at
least (bda)() or seven chars, four of them noisy and shifty (RSI-
inducing, but who's counting given our C heritage :-P) punctuation.
We can argue about the merits of sugar (tooth decay? longevity via
reduced stress? natural sweeteners combined with complete nutrition
are not associated with tooth decay -- ok, I'll stop now ;-)), but if
we keep this discussion separate from lambda's syntax and semantics,
then I think both lambda and any let block/expr sugar that emerges
will benefit.
On Sat, Oct 11, 2008 at 3:16 PM, Brendan Eich <brendan at mozilla.com> wrote:
The reason people like let blocks and expressions as opposed to let declarations, from what I can tell, is precisely the lack of hoisting to top of block done for let declarations, and the greater clarity of head-scoped initialization (possibly using outer names that will be shadowed in the block or expression, but usually not) of the 'let (head) ...' syntax.
Of course everything could be lambda-coded, but that's not harmonious or usable. Just let as new var is agreed upon, and let blocks and expressions could be let coded if there is no name shadowing (else they'd need lambda-coding). But we should not dismiss the use-cases for let blocks/expressions out of hand. There is experience in JS1.7 and 1.8 to consider here, not just hypothetical arguments about how often hand-written cases come up.
I agree with all of the above. The clarity of the let expressions and blocks is valuable for understanding code as the hoisting issue is gone and it is explicitly clear what is going on with variable scopes.
Peter
On Sat, Oct 11, 2008 at 4:33 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Expression ::= ... | lambda Formals? Statement
Statement or SubStatement? If Statement, what meaning do you propose for
{ const f = (lambda () const x = 3;); .... x .... }
Does the const export its binding into the lambda's enclosing block, such that the x on the next line refers to that x? I hope not.
If the answer is SubStatement, then changing the answer to Block in order to require the curlies may very well help people understand the scope relationships across the lambda boundary.
Mark S. Miller wrote:
On Sat, Oct 11, 2008 at 4:33 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Expression ::= ... | lambda Formals? Statement
Statement or SubStatement? If Statement, what meaning do you propose for
{ const f = (lambda () const x = 3;); .... x .... }
Does the const export its binding into the lambda's enclosing block, such that the x on the next line refers to that x? I hope not.
If the answer is SubStatement, then changing the answer to Block in order to require the curlies may very well help people understand the scope relationships across the lambda boundary.
Now that I think about it, would it truly be necessary for lambda to create an implicit block scope in the first place? |lambda() return 10| would not require such a block scope. Why not have the block scope only created if there are curly brackets? That would follow the precedent set by the rest of the language with to block scope with the exception of |for (let x...) ...|.
Yuh-Ruey Chen wrote:
Mark S. Miller wrote:
On Sat, Oct 11, 2008 at 4:33 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Expression ::= ... | lambda Formals? Statement
Statement or SubStatement? If Statement, what meaning do you propose for
{ const f = (lambda () const x = 3;); .... x .... }
Does the const export its binding into the lambda's enclosing block, such that the x on the next line refers to that x? I hope not.
If the answer is SubStatement, then changing the answer to Block in order to require the curlies may very well help people understand the scope relationships across the lambda boundary.
Now that I think about it, would it truly be necessary for lambda to create an implicit block scope in the first place?
It's not strictly necessary, but it's quite ugly not to. We are intending to restrict 'eval' precisely to remove the ability to inject bindings into the surrounding scope (although 'eval' is worse because the scoping it creates isn't statically analysable).
|lambda() return 10| would not require such a block scope. Why not have the block scope only created if there are curly brackets? That would follow the precedent set by the rest of the language with to block scope
No, that would be entirely inconsistent with the rest of the language:
{ while (...) let x = ...; }
is equivalent to
{ while (...) { let x = ...; } }
not
{ let x; while (...) { x = ...; } }
with the exception of |for (let x...) ...|.
I don't think that such an "unscoped" lambda provides enough (if any) value to justify its complexity.
David-Sarah Hopwood wrote:
Yuh-Ruey Chen wrote:
Now that I think about it, would it truly be necessary for lambda to create an implicit block scope in the first place?
It's not strictly necessary, but it's quite ugly not to. We are intending to restrict 'eval' precisely to remove the ability to inject bindings into the surrounding scope (although 'eval' is worse because the scoping it creates isn't statically analysable).
|lambda() return 10| would not require such a block scope. Why not have the block scope only created if there are curly brackets? That would follow the precedent set by the rest of the language with to block scope
No, that would be entirely inconsistent with the rest of the language:
{ while (...) let x = ...; }
is equivalent to
{ while (...) { let x = ...; } }
not
{ let x; while (...) { x = ...; } }
I was under the impression that such statements should be disallowed, following the example of JS1.8. In JS1.8 (Fx3), the following are all syntax errors:
while (...) let x = ...; do let x = ...; while (...); if (...) let x = ...; for (...) let x = ...;
Actually, this might also apply to JS1.7, but I don't have a copy of Fx2 handy.
On Sun, Oct 12, 2008 at 2:19 PM, Yuh-Ruey Chen <maian330 at gmail.com> wrote:
David-Sarah Hopwood wrote:
{ while (...) let x = ...; } I was under the impression that such statements should be disallowed, following the example of JS1.8. In JS1.8 (Fx3), the following are all syntax errors:
while (...) let x = ...; do let x = ...; while (...); if (...) let x = ...; for (...) let x = ...;
These are unconditionally disallowed starting in ES3.1 because such declarations are Statements but not SubStatements and only SubStatements are allowed by the grammar at the positions above.
Mark S. Miller wrote:
On Sun, Oct 12, 2008 at 2:19 PM, Yuh-Ruey Chen <maian330 at gmail.com> wrote:
David-Sarah Hopwood wrote:
{ while (...) let x = ...; } I was under the impression that such statements should be disallowed, following the example of JS1.8. In JS1.8 (Fx3), the following are all syntax errors:
while (...) let x = ...; do let x = ...; while (...); if (...) let x = ...; for (...) let x = ...;
These are unconditionally disallowed starting in ES3.1 because such declarations are Statements but not SubStatements and only SubStatements are allowed by the grammar at the positions above.
Excellent. (I had thought these were only disallowed in strict mode, based on MarkM's comment at groups.google.com/group/google-caja-discuss/browse_thread/thread/ca99f5af671a7dc1/7121d2e6d732f0fc#7121d2e6d732f0fc,
but disallowing them unconditionally is even better IMHO.)
So why are we arguing about whether lambdas should be allowed without braces, when the direction being taken for the rest of the language is to make the braces mandatory around all forms that can potentially declare variables?
Mark S. Miller wrote:
On Sun, Oct 12, 2008 at 2:19 PM, Yuh-Ruey Chen <maian330 at gmail.com> wrote:
David-Sarah Hopwood wrote:
{ while (...) let x = ...; } I was under the impression that such statements should be disallowed, following the example of JS1.8. In JS1.8 (Fx3), the following are all syntax errors:
while (...) let x = ...; do let x = ...; while (...); if (...) let x = ...; for (...) let x = ...;
These are unconditionally disallowed starting in ES3.1 because such declarations are Statements but not SubStatements and only SubStatements are allowed by the grammar at the positions above.
Since LabelledStatement is included in SubStatement, the following is allowed in the current ES3.1 draft:
while (...) foo: var x = ...;
I would suggest that it shouldn't be, i.e. that LabelledStatement should be removed from SubStatement.
[Jacaranda has similar, but stricter restrictions on substatements. See the rationale for group SIMPLE_STATEMENTS in the Jacaranda spec, www.jacaranda.org/jacaranda-spec-0.3.txt.]
On Sun, Oct 12, 2008 at 7:34 PM, David-Sarah Hopwood < david.hopwood at industrial-designers.co.uk> wrote:
So why are we arguing about whether lambdas should be allowed without braces, when the direction being taken for the rest of the language is to make the braces mandatory around all forms that can potentially declare variables?
What "rest of the language" are you talking about? |if|, |while|, and |for| do not require braces and thus do not necessarily create block scopes. I can't imagine a situation where |lambda(...) non_let_statement| would require creating a block scope for the lambda, unless that non_let_statement is a block itself.
On Sun, Oct 12, 2008 at 5:45 PM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
Since LabelledStatement is included in SubStatement, the following is allowed in the current ES3.1 draft:
while (...) foo: var x = ...;
This will be corrected in the next version. But the agreed fix is not to make LabelledStatement a Statement. The fix is to make a LabelledStatement contain only a SubStatement. So the above will still be rejected, but
while (...) foo: {var x = ...;}
will be fine.
2008/10/12 YR Chen <maian330 at gmail.com>:
On Sun, Oct 12, 2008 at 7:34 PM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
So why are we arguing about whether lambdas should be allowed without braces, when the direction being taken for the rest of the language is to make the braces mandatory around all forms that can potentially declare variables?
What "rest of the language" are you talking about? |if|, |while|, and |for| do not require braces and thus do not necessarily create block scopes. I can't imagine a situation where |lambda(...) non_let_statement| would require creating a block scope for the lambda, unless that non_let_statement is a block itself.
David-Sarah said "that can potentially declare variables". If the then branch of an |if| declares variables then it needs to be enclosed in curlies in order to be a SubStatement.
YR Chen wrote:
On Sun, Oct 12, 2008 at 7:34 PM, David-Sarah Hopwood < david.hopwood at industrial-designers.co.uk> wrote:
So why are we arguing about whether lambdas should be allowed without braces, when the direction being taken for the rest of the language is to make the braces mandatory around all forms that can potentially declare variables?
What "rest of the language" are you talking about? |if|, |while|, and |for| do not require braces and thus do not necessarily create block scopes.
|if|, |while|, |do..while|, |for|, and |with| do necessarily create block scopes. Their bodies are always scopes regardless of whether they have braces around them, even in ES3. (When the body is a statement that does not introduce any variables, then whether a scope is created isn't observable, and so we might as well consider them as creating a scope, for regularity.)
In ES3.1, this will mean that they normally require braces whenever a body can introduce variables. There are two classes of exceptions, shown by these examples:
a) while (...) foo: var x = ...; b) while (...) for (var x = ...; ...; ...) {}
I think these are bugs. a) can certainly be disallowed. There might possibly be existing code that is relying on b), confusing though it is, but it can be disallowed in strict mode at least.
On Sun, Oct 12, 2008 at 7:55 PM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
In ES3.1, this will mean that they normally require braces whenever a body can introduce variables. There are two classes of exceptions, shown by these examples:
a) while (...) foo: var x = ...; b) while (...) for (var x = ...; ...; ...) {}
I think these are bugs. a) can certainly be disallowed. There might possibly be existing code that is relying on b), confusing though it is, but it can be disallowed in strict mode at least.
ES3.1 will disallow #a because a LabelledStatement can only contain a SubStatement. A variable declaration is a Statement but not a SubStatement.
Your point about #b is interesting. I do not know if it has previously been raised.
Mark S. Miller wrote:
On Sun, Oct 12, 2008 at 7:55 PM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
In ES3.1, this will mean that they normally require braces whenever a body can introduce variables. There are two classes of exceptions, shown by these examples:
a) while (...) foo: var x = ...; b) while (...) for (var x = ...; ...; ...) {}
I think these are bugs. a) can certainly be disallowed. There might possibly be existing code that is relying on b), confusing though it is, but it can be disallowed in strict mode at least.
ES3.1 will disallow #a because a LabelledStatement can only contain a SubStatement. A variable declaration is a Statement but not a SubStatement.
Note that although there is currently no plausible use for
{ foo: let x = ...; }
because expressions cannot 'break' or 'continue' to a label outside the expression, that reason will no longer hold if/when we add lambdas. In that case "..." might contain (after expansion) a lambda that says "break foo;" or "continue foo;". But that doesn't really cause any problem since,
- if the let is written explicitly by the programmer, it's easy enough to add extra braces in this (presumably rare) case;
- if the let is introduced by an expansion, the expansion should include braces around it if it is intended to be a SubStatement.
<snip contribution added by cat walking on keyboard>
[Reminder to self: In order for Jacaranda to continue to be a subset of ES3.1, I will have to change it to reject "label: var ...", which was allowed in draft 0.3.]
Your point about #b is interesting. I do not know if it has previously been raised.
Actually the while loop is a red herring in that example: "for (var ...)" implicitly introduces a block whether or not it is a substatement. This is a wart of C++/C99/Java syntax that we have to live with, since too much code relies on it.
On Mon, Oct 13, 2008 at 8:05 AM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
"for (var ...)" implicitly introduces a block whether or not it is a substatement. This is a wart of C++/C99/Java syntax that we have to live with, since too much code relies on it.
Yes, but how do we live with it? The only currently allowed case, "var" as above, is no problem, since it hoists to the function body anyway. But what about "let". Do we all agree that in
for (let x = ...) {...x...}
... x ...
the x after the for loop does not refer to the x defined by the for loop? In that case, no problem with for being a SubStatement.
A remaining interesting question is whether the for loop reassigns to a single per-for-loop-entry x, or whether it initializes a fresh per-iteration x. If the for loop body has a closure that captures x, it makes a difference. I recommend the per-iteration view. If we can agree quickly on per-iteration, then
for (const x = ...) {...x...}
should be allowed in ES3.1 (whether or not const hoists to block start). After ES3.1
for (const i :T[i] = ...) {...; a[i] = function(){...i...}; ...}
would then mean what it should mean. Cool.
On Mon, Oct 13, 2008 at 11:43 AM, Mark S. Miller <erights at google.com> wrote:
On Mon, Oct 13, 2008 at 8:05 AM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
"for (var ...)" implicitly introduces a block whether or not it is a substatement. This is a wart of C++/C99/Java syntax that we have to live with, since too much code relies on it.
Yes, but how do we live with it? The only currently allowed case, "var" as above, is no problem, since it hoists to the function body anyway. But what about "let". Do we all agree that in
for (let x = ...) {...x...} ... x ...
the x after the for loop does not refer to the x defined by the for loop?
Yes. Definitely.
In that case, no problem with for being a SubStatement.
A remaining interesting question is whether the for loop reassigns to a single per-for-loop-entry x, or whether it initializes a fresh per-iteration x. If the for loop body has a closure that captures x, it makes a difference. I recommend the per-iteration view.
I like your intention here -- I brought up the iteration variable / closure issue earlier in the thread -- but this seems rather messy.
In the 'var' case, x must be the same throughout, no?
function foo() { for (var x = ... ) { ... x ... } ... x ... }
But then it becomes extremely awkward for the 'let' or 'const' case to use a fresh variable on every iteration. It ties the semantics of 'for' to its initialization clause in a very strange way.
On Mon, Oct 13, 2008 at 9:13 AM, Jon Zeppieri <jaz at bu.edu> wrote:
I like your intention here -- I brought up the iteration variable / closure issue earlier in the thread -- but this seems rather messy.
In the 'var' case, x must be the same throughout, no?
yes.
function foo() { for (var x = ... ) { ... x ... } ... x ... }
But then it becomes extremely awkward for the 'let' or 'const' case to use a fresh variable on every iteration. It ties the semantics of 'for' to its initialization clause in a very strange way.
I think it falls out of the natural desugaring of for to lambda with no special cases. The "var" itself causes its own hoisting and joining.
On Mon, Oct 13, 2008 at 12:47 PM, Mark Miller <erights at gmail.com> wrote:
On Mon, Oct 13, 2008 at 9:13 AM, Jon Zeppieri <jaz at bu.edu> wrote:
But then it becomes extremely awkward for the 'let' or 'const' case to use a fresh variable on every iteration. It ties the semantics of 'for' to its initialization clause in a very strange way.
I think it falls out of the natural desugaring of for to lambda with no special cases. The "var" itself causes its own hoisting and joining.
Sorry -- I must be particularly dense today. What desugaring do you have in mind?
Mark S. Miller wrote:
On Mon, Oct 13, 2008 at 8:05 AM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
"for (var ...)" implicitly introduces a block whether or not it is a substatement. This is a wart of C++/C99/Java syntax that we have to live with, since too much code relies on it.
Yes, but how do we live with it? The only currently allowed case, "var" as above, is no problem, since it hoists to the function body anyway.
Yes, I should have used "for (let ...)" as the example.
But what about "let". Do we all agree that in
for (let x = ...) {...x...} ... x ...the x after the for loop does not refer to the x defined by the for loop?
Yes, if this form is allowed. The other options are either to require:
let x in { for (x = 0; x < 10; x++) {...x...} } or let x = 0 in { for (; x < 10; x++) {...x...} }
or to add new syntax:
for x = 0 while {x < 10} with {x++} do { ...x... }
The latter is longer, but it makes more sense to me, because the scoping is absolutely consistent with the braces. I don't know what other C/Java/JS-trained programmers will make of it, though.
A remaining interesting question is whether [in the case of "for (let" or "for (const"] the for loop reassigns to a single per-for-loop-entry x, or whether it initializes a fresh per-iteration x. If the for loop body has a closure that captures x, it makes a difference. I recommend the per-iteration view. If we can agree quickly on per-iteration, then
for (const x = ...) {...x...}should be allowed in ES3.1 (whether or not const hoists to block start). After ES3.1
for (const i :T[i] = ...) {...; a[i] = function(){...i...}; ...}would then mean what it should mean. Cool.
Not so fast :-) Consider:
for (let i = 0; i < 10; i++) { ... }
In the "i++", which iteration's 'i' is the LeftHandSideExpression referring to? Or does this expand to something like:
let ($i = 0) { for (; let (i = $i) {i < 10}; let (i = $i) {{i++;} $i = i;}} { let (i = $i) {...}; } }
? That would work as I think you want, but it seems a bit magical to me. I could be convinced, though -- do you want to try and make the case for why there should be a fresh variable per iteration with some concrete examples?
David-Sarah Hopwood wrote:
Mark S. Miller wrote:
But what about "let". Do we all agree that in
for (let x = ...) {...x...} ... x ...the x after the for loop does not refer to the x defined by the for loop?
Yes, if this form is allowed. The other options are either to require:
let x in { for (x = 0; x < 10; x++) {...x...} } or let x = 0 in { for (; x < 10; x++) {...x...} }
I meant:
let (x) { for (x = 0; x < 10; x++) {...x...} }
or let (x = 0) { for (; x < 10; x++) {...x...} }
('let ... in' is similar to the 'local ... in' syntax that Oz uses, and I think it reads better, but it can't be used in ECMAScript because the part to the right of the identifier is ambiguous with an application of the 'in' operator.)
David Herman wrote:
if (h == 0) h = function() {break};Did you mean if (x == 0)? That's been confusing me in trying to read your example.
No. I meant the code as it was originally written.
WaldemarOn Oct 12, 2008, at 12:48 PM, Yuh-Ruey Chen wrote:
Now that I think about it, would it truly be necessary for lambda to create an implicit block scope in the first place? |lambda() return
10| would not require such a block scope.
Implementation, or optimization, detail.
Why not have the block scope only created if there are curly brackets?
What about the lambda's formal parameters? They are in scope. Would a
braced body of the lambda be able to shadow them, uselessly?
Per ES1-3, in
function f(x) { var x; ... }
the var has no effect, since x is a property of the variable object
already.
After some experiments, we decided for ES4 to make let and var the
same at top level in a function or global code.
This helped avoid implementation pain: very long scripts on the web,
consisting of many statements in a row, motivate statement-wise
parsing and cumulative code generation, which is straightforward in
the absence of goto. Yet scope changes (e.g. due to a tardy rogue let
x; after thousands of statements the first of which uses x) require a
separate pass to generate the necessary block set-up code before the
first use of x.
Unifying let and var at top level also reduces the cognitive load
(number of scopes in mind) and eliminates useless name shadowing
opportunities.
But such let/var unification at top level in a function body does
leave bad old arguments[0] aliasing x in the above example, allowing
mutation of otherwise lexical let x; (change var to let and fill in
the ... with arguments[0] = 42; return x). The answer that we chose
for ES4, and the one that's already agreeable in committee for
Harmony, was "deprecate arguments by providing optional and rest
parameters".
That would follow the precedent set by the rest of the language with to block scope with the exception of |for (let x...) ...|.
There is no syntactically valid way to write let x ... except as the
direct child of a body, block, or for head, in any of the complete
proposals of let declarations I've read. One implementation, JS1.7 in
Firefox 2, did misread the proposal as allowing sentences such as
if (x) let y = 42;
and hoisting y to the block containing the if statement. This bug is
fixed in JS1.8 / Firefox 3.
Some folks have expressed the strong desire for a fresh block scope
each time through a for (let x...) loop. See
bugzilla.mozilla.org/show_bug.cgi?id=449811
Comments (here) welcome.
On Oct 12, 2008, at 1:39 PM, David-Sarah Hopwood wrote:
|lambda() return 10| would not require such a block scope. Why not have the block scope
only created if there are curly brackets? That would follow the
precedent set by the rest of the language with to block scopeNo, that would be entirely inconsistent with the rest of the language:
{ while (...) let x = ...; }
is equivalent to
{ while (...) { let x = ...; } }
Not in any let proposal -- specifically, grammar -- that I know of.
See my last post.
In the case of for (let x...) ... this seems like a bug, cited in last
message, here it is again:
bugzilla.mozilla.org/show_bug.cgi?id=449811
not
{ let x; while (...) { x = ...; } }
Rather, the proposed let declaration grammar requires that let be a
direct child of a block, function body, for-loop head, or top-level
program -- anywhere else and it's a syntax error.
I don't think that such an "unscoped" lambda provides enough (if any) value to justify its complexity.
I agree, "unscoped" lambda is not worth specifying as such, and begs
questions about what scope any formal parameters go in (could be the
block induced by {...} comes back if there are formals, but then
again, there's no point in talking about the unscoped degenerate
case). As an optimization, implementations will erase unnecessary
scopes. But the spec shouldn't worry about this.
On Oct 12, 2008, at 9:34 PM, Mark S. Miller wrote:
On Sun, Oct 12, 2008 at 7:55 PM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
In ES3.1, this will mean that they normally require braces whenever a body can introduce variables. There are two classes of exceptions, shown by these examples:
a) while (...) foo: var x = ...; b) while (...) for (var x = ...; ...; ...) {}
I think these are bugs. a) can certainly be disallowed. There might possibly be existing code that is relying on b), confusing though it is, but it can be disallowed in strict mode at least.
ES3.1 will disallow #a because a LabelledStatement can only contain a SubStatement. A variable declaration is a Statement but not a SubStatement.
Your point about #b is interesting. I do not know if it has previously been raised.
Why is it interesting?
We allow var in for heads, no matter how nested. This has been in the
spec since the first edition. It is not a bug. We should not break it
in strict mode.
On Oct 13, 2008, at 8:43 AM, Mark S. Miller wrote:
On Mon, Oct 13, 2008 at 8:05 AM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
"for (var ...)" implicitly introduces a block whether or not it is a substatement. This is a wart of C++/C99/Java syntax that we have to live with, since too much code relies on it.
Yes, but how do we live with it? The only currently allowed case, "var" as above, is no problem, since it hoists to the function body anyway. But what about "let". Do we all agree that in
for (let x = ...) {...x...} ... x ...
the x after the for loop does not refer to the x defined by the for loop? In that case, no problem with for being a SubStatement.
No one ever proposed otherwise for for (let...), and it would be nuts
to do anything like that.
A remaining interesting question is whether the for loop reassigns to a single per-for-loop-entry x, or whether it initializes a fresh per-iteration x. If the for loop body has a closure that captures x, it makes a difference. I recommend the per-iteration view. If we can agree quickly on per-iteration, then
for (const x = ...) {...x...}
should be allowed in ES3.1 (whether or not const hoists to block start). After ES3.1
for (const i :T[i] = ...) {...; a[i] = function(){...i...}; ...}
would then mean what it should mean. Cool.
Agreed. See bugzilla.mozilla.org/show_bug.cgi?id=449811 --
which is a bug we'd like to fix by following a spec.
On Oct 13, 2008, at 1:16 PM, David-Sarah Hopwood wrote:
In the "i++", which iteration's 'i' is the LeftHandSideExpression referring to? Or does this expand to something like:
let ($i = 0) { for (; let (i = $i) {i < 10}; let (i = $i) {{i++;} $i = i;}} { let (i = $i) {...}; } }
? That would work as I think you want, but it seems a bit magical to
me.
On the contrary, what's magical is when one stores i in elements of an
array, or calls setTimeout (example in bugzilla.mozilla.org/show_bug.cgi?id=449811)
, and the results are all the last value of the loop variable.
This has been a frequent source of confusion and complaints -- a bug,
in short.
Doing it the way Mark proposes fixes the bug, and has no other bad
effects that I can see (but we'll have to implement and user-test to
be sure).
I could be convinced, though -- do you want to try and make the case
for why there should be a fresh variable per iteration with some concrete examples?
Brendan Eich wrote:
After some experiments, we decided for ES4 to make let and var the
same at top level in a function or global code.This helped avoid implementation pain: very long scripts on the web,
consisting of many statements in a row, motivate statement-wise
parsing and cumulative code generation, which is straightforward in
the absence of goto. Yet scope changes (e.g. due to a tardy rogue let
x; after thousands of statements the first of which uses x) require a
separate pass to generate the necessary block set-up code before the
first use of x.Unifying let and var at top level also reduces the cognitive load
(number of scopes in mind) and eliminates useless name shadowing
opportunities.But such let/var unification at top level in a function body does
leave bad old arguments[0] aliasing x in the above example, allowing
mutation of otherwise lexical let x; (change var to let and fill in
the ... with arguments[0] = 42; return x). The answer that we chose
for ES4, and the one that's already agreeable in committee for
Harmony, was "deprecate arguments by providing optional and rest
parameters".
You can't do that.
function f() { x = 15; ... let t = some_runtime_expression; ... let x:t = ... }
What is the type of x at the beginning of the function?
WaldemarBrendan Eich wrote:
The agreement from the May TC39 meeting was that the declarations
implicit (:*) and explicit annotations must normalize to the same
type, or there's an error.
That was back when the language had lots of requirements for compile-time expressions, including on all types. We agreed that that's not part of ES-Harmony, and this condition doesn't make sense when type expressions are evaluated at run time.
WaldemarOn Oct 13, 2008, at 3:51 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
After some experiments, we decided for ES4 to make let and var the same at top level in a function or global code.
This helped avoid implementation pain: very long scripts on the web, consisting of many statements in a row, motivate statement-wise parsing and cumulative code generation, which is straightforward in the absence of goto. Yet scope changes (e.g. due to a tardy rogue let x; after thousands of statements the first of which uses x) require a separate pass to generate the necessary block set-up code before the first use of x.
Unifying let and var at top level also reduces the cognitive load (number of scopes in mind) and eliminates useless name shadowing opportunities.
But such let/var unification at top level in a function body does leave bad old arguments[0] aliasing x in the above example, allowing mutation of otherwise lexical let x; (change var to let and fill in the ... with arguments[0] = 42; return x). The answer that we chose for ES4, and the one that's already agreeable in committee for Harmony, was "deprecate arguments by providing optional and rest parameters".
You can't do that.
Which "that" do you mean?
function f() { x = 15; ... let t = some_runtime_expression; ... let x:t = ... }
What is the type of x at the beginning of the function?
The example had better fail to compile.
I hope my point about implementations optimizing for very long "AST
hedge" programs on the web is mixed up here. Such implementations
build a full AST for function bodies, and function bodies tend to be
reasonably short on the web (Tennent be damned). Long global code
consisting of repeated assignment statements, OTOH, can suck up too
much time and space if compiled as a whole instead of incrementally.
But I still don't know which "that" you meant.
On Oct 13, 2008, at 3:56 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
The agreement from the May TC39 meeting was that the declarations implicit (:*) and explicit annotations must normalize to the same type, or there's an error.
That was back when the language had lots of requirements for compile- time expressions, including on all types. We agreed that that's not
part of ES-Harmony, and this condition doesn't make sense when type
expressions are evaluated at run time.
True enough -- but even without normalization, multiple equivalent (at
runtime, depending on flow) annotations could be allowed. Should they
be?
Brendan Eich wrote:
If using an uninitialized let binding is an error, then hoisting is
pointless except to make the statements between start of block and the
let declaration a dead zone for the binding name. This fits the
ancient, weak but not entirely worthless post-hoc rationale for var
hoisting (to avoid confusion among novice or inexperienced programmers
by making many scopes, each implicitly opened by var), but it's not
particularly useful.
This was our agreement from the ES4 days. It's very useful, in that it allows mutually recursive lambdas.
What's more, as discussed here and in TC39, repeated let declarations
for the same binding name within the same block should be allowed.
You can't do that in ES-Harmony. There is no way to tell if the two let declarations have the same type. For orthogonality you'd also need to allow multiple const declarations within the same scope, and I just don't want to go there.
Anything else is user- and refactoring-hostile. So the non-initial let
declarations for a given name in the same block would be ignored.
This is loaded language, but I can't tell how requiring there to be a unique point of definition for a const or let is user-hostile.
WaldemarOn Oct 13, 2008, at 4:01 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
If using an uninitialized let binding is an error, then hoisting is pointless except to make the statements between start of block and
the let declaration a dead zone for the binding name. This fits the ancient, weak but not entirely worthless post-hoc rationale for var hoisting (to avoid confusion among novice or inexperienced
programmers by making many scopes, each implicitly opened by var), but it's not particularly useful.This was our agreement from the ES4 days. It's very useful, in that
it allows mutually recursive lambdas.
For function bindings, yes, of course -- but we were talking about let
bindings (I thought). For let (there is no 'let function') bindings,
how? Could you give an example?
What's more, as discussed here and in TC39, repeated let declarations for the same binding name within the same block should be allowed.
You can't do that in ES-Harmony. There is no way to tell if the two
let declarations have the same type.
There has to be a way to answer the question "are these two terms the
same type" at run-time.
For orthogonality you'd also need to allow multiple const
declarations within the same scope, and I just don't want to go there.
I don't agree -- const and let are different keywords, they obviously
differ in read-only vs. writable binding. They could differ otherwise.
Anything else is user- and refactoring-hostile. So the non-initial
let declarations for a given name in the same block would be ignored.This is loaded language, but I can't tell how requiring there to be
a unique point of definition for a const or let is user-hostile.
My words were about let, not const. If let is the new var, then you
either accept that re-declaration in the same block (usually function
body) of the same name via var is common for good reason, and allow
let to be used instead; or you raise the migration tax.
I do not mean to over-load the language, but we've seen a lot of code
that redeclares using var (e.g. for (var i...) in adjacent for loops).
We tried making a strict warning (error console spew) for
redeclarations of same kind (var vs. var) four years ago (IIRC --
Firefox 1 betas) and felt the heat from developers. We fell back on
mixed var vs. function (const vs. anything is an error of course).
People on this list have argued similarly based on refactoring and var
hoisting.
It may be that the migration tax must rise here, but I'd like to know
why type annotations can't be equated at runtime.
On Mon, Oct 13, 2008 at 6:34 PM, Brendan Eich <brendan at mozilla.com> wrote:
On the contrary, what's magical is when one stores i in elements of an array, or calls setTimeout (example in bugzilla.mozilla.org/show_bug.cgi?id=449811) , and the results are all the last value of the loop variable.
This has been a frequent source of confusion and complaints -- a bug, in short.
Yes, and binding a fresh induction variable on every iteration makes sense for a 'for-each' loop (as in the bug report you cited), where the user is not in charge of updating the induction variable by means of explicit assignment. In a plain 'for' loop, however, it is magic if an assignment results in a fresh binding. And it's unexpected magic.
Mark said that there was a desugaring for 'for' to 'lambda,' without special cases, where this all works out, but I haven't been able to figure out what rewrite he had in mind.
Doing it the way Mark proposes fixes the bug, and has no other bad effects that I can see (but we'll have to implement and user-test to be sure).
Turning an assignment into a non-assignment is bad.
I could be convinced, though -- do you want to try and make the case for why there should be a fresh variable per iteration with some concrete examples?
Like I said, it makes sense for 'for-each.'
Brendan Eich wrote:
On Oct 13, 2008, at 3:56 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
The agreement from the May TC39 meeting was that the declarations implicit (:*) and explicit annotations must normalize to the same type, or there's an error.
That was back when the language had lots of requirements for compile-time expressions, including on all types. We agreed that that's not part of ES-Harmony, and this condition doesn't make sense when type expressions are evaluated at run time.
True enough -- but even without normalization, multiple equivalent (at runtime, depending on flow) annotations could be allowed. Should they be?
No. Just because the type matched the last five times you evaluated it doesn't mean that it will match the next time. Also, with multiple lets it's too easy to mistake the case of shadowing (when multiple let declarations are in nested blocks) with sharing (when they are far apart in the same block). You might refactor:
{ let x = "outer"; ... use x ... if (always_true) { let x = "inner"; ... use x ... } ... use x ... }
into:
{ let x = "outer"; ... use x ... let x = "inner"; ... use x ... ... use x ... }
WaldemarOn Oct 13, 2008, at 4:14 PM, Jon Zeppieri wrote:
Yes, and binding a fresh induction variable on every iteration makes sense for a 'for-each' loop (as in the bug report you cited), where the user is not in charge of updating the induction variable by means of explicit assignment. In a plain 'for' loop, however, it is magic if an assignment results in a fresh binding.
Why is the assignment operator relevant? The question is the binding
scope of i in
for (let i = 0; i < N; i++) ...
No curly braces required, we already have this in JS1.7+ and the let
is scoped to the for head except for the initializer of i (you can
write let i = x, j = y; too -- x and y are evaluated in the outer
scope). There's scope magic in this form even though it uses = for
assignment and creates only one block scope for the loop, no how many
times the loop iterates.
And it's unexpected magic.
Users differ on this point, but we've had long-standing confusion and
complaints about closures capturing the last value of i in
let a = [1,2,3,4]; let b = []; for (let i = 0; i < a.length; i++) b[i] = function () { return i*i; }
and the like. Getting 16 back from b0 is unexpected bad magic.
Users may be modeling closures as capturing bindings, not scope chains
of mutable objects, one per for (let...) statement or explicitly
braced block. If so, could we make let declaration capture this way?
Again, I'm proceeding from real users' complaints, not idle wishes.
Mark said that there was a desugaring for 'for' to 'lambda,' without special cases, where this all works out, but I haven't been able to figure out what rewrite he had in mind.
Tail-recursive lambda rewrite of a C-style for loop should be easy for
you :-P.
Doing it the way Mark proposes fixes the bug, and has no other bad effects that I can see (but we'll have to implement and user-test to be sure).
Turning an assignment into a non-assignment is bad.
Assignment is not the issue, the binding's scope is.
Like I said, it makes sense for 'for-each.'
Progress!
On Oct 13, 2008, at 4:17 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
True enough -- but even without normalization, multiple equivalent
(at runtime, depending on flow) annotations could be allowed. Should
they be?No. Just because the type matched the last five times you evaluated
it doesn't mean that it will match the next time. Also, with
multiple lets it's too easy to mistake the case of shadowing (when
multiple let declarations are in nested blocks) with sharing (when
they are far apart in the same block). You might refactor: ...
Same could happen for function definitions nested in blocks,
refactored to erase the blocks. It would be an error to redefine a
function with the same name and any type annotations (assuming we have
annotated functions -- function types), I take it?
If the answer is no, then I agree. The consequences of runtime types
have not been something I've thought a lot about, but it seems like
they make all binding forms like const: you cannot redefine a name in
the same block with any binding form.
David-Sarah Hopwood wrote:
Mark S. Miller wrote:
[...] I recommend the per-iteration view. If we can agree quickly on per-iteration, then
for (const x = ...) {...x...}should be allowed in ES3.1 (whether or not const hoists to block start). After ES3.1
for (const i :T[i] = ...) {...; a[i] = function(){...i...}; ...}would then mean what it should mean. Cool.
Not so fast :-) Consider:
for (let i = 0; i < 10; i++) { ... }
In the "i++", which iteration's 'i' is the LeftHandSideExpression referring to? Or does this expand to something like:
let ($i = 0) { for (; let (i = $i) {i < 10}; let (i = $i) {{i++;} $i = i;}} { let (i = $i) {...}; } }
?
This expansion is wrong for the case where the body updates i. I'll have to think about it some more.
Brendan Eich wrote:
On Oct 13, 2008, at 4:01 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
If using an uninitialized let binding is an error, then hoisting is pointless except to make the statements between start of block and the let declaration a dead zone for the binding name. This fits the ancient, weak but not entirely worthless post-hoc rationale for var hoisting (to avoid confusion among novice or inexperienced programmers by making many scopes, each implicitly opened by var), but it's not particularly useful.
This was our agreement from the ES4 days. It's very useful, in that it allows mutually recursive lambdas.
For function bindings, yes, of course -- but we were talking about let bindings (I thought). For let (there is no 'let function') bindings, how? Could you give an example?
I am talking about let bindings. Lars brought up at that meeting. I did not find the use cases particularly convincing, but the dead zone is compelling. There are four ways to do this: A1. Lexical dead zone. References textually prior to a definition in the same block are an error. A2. Lexical window. References textually prior to a definition in the same block go to outer scope. B1. Temporal dead zone. References temporally prior to a definition in the same block are an error. B2. Temporal window. References temporally prior to a definition in the same block go to outer scope.
Let's take a look at an example:
let x = "outer"; function g() {return "outer"}
{ g(); function f() { ... x ... g ... g() ... } f(); var t = some_runtime_type; const x:t = "inner"; function g() { ... x ... } g(); f(); }
B2 is bad because then the x inside g would sometimes refer to "outer" and sometimes to "inner".
A1 and A2 introduce extra complexity but doesn't solve the problem. You'd need to come up with a value for x to use in the very first call to g(). Furthermore, for A2 whether the window occurred or not would also depend on whether something was a function or not; users would be surprised that x shows through the window inside f but g doesn't.
That leaves B1, which matches the semantic model (we need to avoid referencing variables before we know their types and before we know the values of constants).
What's more, as discussed here and in TC39, repeated let declarations for the same binding name within the same block should be allowed.
You can't do that in ES-Harmony. There is no way to tell if the two let declarations have the same type.
There has to be a way to answer the question "are these two terms the same type" at run-time.
I'm sure there will be, but that's not the issue. What we need to know is that everyone agrees on the type of the variable. You might write code assuming that the type is set by a nearby let statement, when in fact it's some other let statement that set it.
For orthogonality you'd also need to allow multiple const declarations within the same scope, and I just don't want to go there.
I don't agree -- const and let are different keywords, they obviously differ in read-only vs. writable binding. They could differ otherwise.
They shouldn't differ any more than that. If they do, it would just add gratuitous hair to the language.
WaldemarOn Oct 13, 2008, at 4:09 PM, Brendan Eich wrote:
On Oct 13, 2008, at 4:01 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
If using an uninitialized let binding is an error, then hoisting is pointless except to make the statements between start of block and the let declaration a dead zone for the binding name. This fits the ancient, weak but not entirely worthless post-hoc rationale for var hoisting (to avoid confusion among novice or inexperienced programmers by making many scopes, each implicitly opened by var), but it's not particularly useful.
This was our agreement from the ES4 days. It's very useful, in that it allows mutually recursive lambdas.
For function bindings, yes, of course -- but we were talking about let bindings (I thought). For let (there is no 'let function') bindings, how? Could you give an example?
Obvious example:
{ let even = function (n) n == 0 || odd(n - 1); let odd = function (n) n != 0 && even(n - 1); print(even(42), odd(42)); print(even(99), odd(99)); }
My words about function binding meant to suggest: why not require
users to write
{ function even(n) n == 0 || odd(n - 1); function odd(n) n != 0 && even(n - 1); ... }
But I'm not arguing that we shouldn't hoist let to top of block, only
trying to justify my "not particularly useful" assertion ;-).
Mark S. Miller wrote:
On Sat, Oct 11, 2008 at 9:11 AM, Peter Michaux <petermichaux at gmail.com> wrote:
I think it would be ok to have only unnamed lambdas. (It would be ok to have named lambdas too.)
I think we should not introduce named lambdas because then we'd need to decide whether the scoping of a lambda name works the same as the scoping of a function name. Do we really want to reproduce the following confusions:
- Distinction between named function expressions vs named function declarations?
- The name of a name function expression is only in scope within the function, whereas the name of a named function declaration is in (letrec) scope in the containing block.
- Because these look the same, a named function expression cannot be used naked as the start of an expression statement. It must wear protective parentheses.
- For consistency, an anonymous function expression also cannot be used as a naked as the start of an expression statement.
If lambdas had optional names, we would either need to reproduce the above confusions or deviate from them. Either choice is terribly unpleasant.
The main benefit of named lambdas is intelligible stack traces in a debugger. What would you replace them with? In a lot of places there is no other good place to stick a name onto; I don't want to assign a lambda to a variable just so I can name it.
WaldemarOn Mon, Oct 13, 2008 at 7:39 PM, Brendan Eich <brendan at mozilla.com> wrote:
On Oct 13, 2008, at 4:14 PM, Jon Zeppieri wrote:
Yes, and binding a fresh induction variable on every iteration makes sense for a 'for-each' loop (as in the bug report you cited), where the user is not in charge of updating the induction variable by means of explicit assignment. In a plain 'for' loop, however, it is magic if an assignment results in a fresh binding.
Why is the assignment operator relevant? The question is the binding scope of i in
for (let i = 0; i < N; i++) ...
How is scope the issue? As far as I know, we don't disagree about scope.
The assignment I'm referring to, in this example, is the 'i++' part. Mark is proposing that this does not mean "increment i by one," but rather "rebind i with the value of i+1" -- which is completely different and not what the user wrote.
No curly braces required, we already have this in JS1.7+ and the let is scoped to the for head except for the initializer of i (you can write let i = x, j = y; too -- x and y are evaluated in the outer scope). There's scope magic in this form even though it uses = for assignment and creates only one block scope for the loop, no how many times the loop iterates.
I think you misread my message. We do not disagree at all about the scope of the initialization clause, but rather the meaning of the update clause.
This is why the 'for-each' loop is not problematic; it doesn't have an update clause.
And it's unexpected magic.
Users differ on this point, but we've had long-standing confusion and complaints about closures capturing the last value of i in
let a = [1,2,3,4]; let b = []; for (let i = 0; i < a.length; i++) b[i] = function () { return i*i; }
and the like. Getting 16 back from b0 is unexpected bad magic.
I understand this, but I don't see how the answer is to change the meaning of 'i++' when used in the update clause of a 'for' loop.
Users may be modeling closures as capturing bindings, not scope chains of mutable objects, one per for (let...) statement or explicitly braced block. If so, could we make let declaration capture this way? Again, I'm proceeding from real users' complaints, not idle wishes.
Mark said that there was a desugaring for 'for' to 'lambda,' without special cases, where this all works out, but I haven't been able to figure out what rewrite he had in mind.
Tail-recursive lambda rewrite of a C-style for loop should be easy for you :-P.
That's not the point. I'm talking about a rewrite from 'for' to 'lambda' that satisfies the following properties:
- for (var i = 0; i < len; i++) ... continues to mean what it means in ES3.
- for (let i = 0; i < len; i++) ... has the proper scope for 'i' (which you reiterated above), and 'i' is rebound -- not mutated -- on each iteration.
- The rewrite rules are the same, regardless of whether it's a "for (var ...)" or a "for (let ...)" loop.
At least, that's what I took Mark to mean. He can correct me if I'm wrong.
On Oct 13, 2008, at 5:00 PM, Jon Zeppieri wrote:
On Mon, Oct 13, 2008 at 7:39 PM, Brendan Eich <brendan at mozilla.com>
wrote:On Oct 13, 2008, at 4:14 PM, Jon Zeppieri wrote:
Yes, and binding a fresh induction variable on every iteration makes sense for a 'for-each' loop (as in the bug report you cited), where the user is not in charge of updating the induction variable by
means of explicit assignment. In a plain 'for' loop, however, it is
magic if an assignment results in a fresh binding.Why is the assignment operator relevant? The question is the binding scope of i in
for (let i = 0; i < N; i++) ...
How is scope the issue? As far as I know, we don't disagree about
scope.
Probably we're at cross purposes (I often am ;-) because of the primal
sin in ECMAScript of specifying scope via object, and closure via
scope chain capture.
The assignment I'm referring to, in this example, is the 'i++' part. Mark is proposing that this does not mean "increment i by one," but rather "rebind i with the value of i+1" -- which is completely different and not what the user wrote.
Gotcha -- I agree, this is a problem for a proposal that tries to
desugar for(;;). Curses.
The user expectation with a closure in a for(var i = 0; i < N; i++)
loop, that each closure captures i's current value, remains,
completely parallel to the for (i in o) ... case. Any attempt to solve
it via let instead of var -- other than by making multiple (blech)
scope (barf) objects, one per iteration -- must respecify how closures
work. Hmm.
David Herman wrote:
Also, I wonder why lambda in it's block-less form is restricted to expressions.
I'm with you... but I'd want to check with the experts on the ES grammar to see whether this introduces any nasty ambiguities.
Please specify what you are proposing. The one proposal I've seen is:
Expression ::= ... | lambda Formals Statement
This is not particularly useful because then even assign a lambda to a variable would be a syntax error, and you'd introduce a bizarre asymmetry into the ExpressionNoIn case. The questions you'd have to answer are:
- Where in the expression grammar would you introduce a lambda?
- Is lambda a reserved word?
- How does it interact with semicolon insertion? As written, if the Statement were an expression statement then the semicolon would be mandatory.
- Do you really want to have extra semicolons in the middle of statements? Now you're forced to accept things like the following:
for (; lambda() x++;, lambda() y++;,; lambda() z++;) ...
Overall, this syntax does not look promising. It's likely to get you into complexity trouble.
WaldemarOn Oct 13, 2008, at 4:54 PM, Waldemar Horwat wrote:
I am talking about let bindings. Lars brought up at that meeting.
I did not find the use cases particularly convincing, but the dead
zone is compelling. There are four ways to do this: A1. Lexical dead zone. References textually prior to a definition
in the same block are an error. A2. Lexical window. References textually prior to a definition in
the same block go to outer scope. B1. Temporal dead zone. References temporally prior to a definition
in the same block are an error. B2. Temporal window. References temporally prior to a definition in
the same block go to outer scope.Let's take a look at an example:
let x = "outer"; function g() {return "outer"}
{ g(); function f() { ... x ... g ... g() ... } f(); var t = some_runtime_type; const x:t = "inner"; function g() { ... x ... } g(); f(); }
B2 is bad because then the x inside g would sometimes refer to
"outer" and sometimes to "inner".A1 and A2 introduce extra complexity but doesn't solve the problem.
You'd need to come up with a value for x to use in the very first
call to g(). Furthermore, for A2 whether the window occurred or not
would also depend on whether something was a function or not; users
would be surprised that x shows through the window inside f but g
doesn't.That leaves B1, which matches the semantic model (we need to avoid
referencing variables before we know their types and before we know
the values of constants).
Agreed, this is compelling, it shows the insufficiency of lexical order.
I missed this somehow (where are those meeting minutes? Still being
vetted by TC39?). Thanks for restating it.
Jon Zeppieri wrote:
Why is the assignment operator relevant? The question is the binding scope of i in
for (let i = 0; i < N; i++) ...
How is scope the issue? As far as I know, we don't disagree about scope.
The assignment I'm referring to, in this example, is the 'i++' part. Mark is proposing that this does not mean "increment i by one," but rather "rebind i with the value of i+1" -- which is completely different and not what the user wrote.
Really? I'm having a hard time believing that.
I agree that a for-in loop should generate a new binding for each iteration, but there is no sensible way to do it for a regular for(;;) loop unless we introduce a new form of for-loop that has explicit rebinding expressions.
WaldemarBrendan Eich wrote:
On Oct 13, 2008, at 3:51 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
After some experiments, we decided for ES4 to make let and var the same at top level in a function or global code.
This helped avoid implementation pain: very long scripts on the web, consisting of many statements in a row, motivate statement-wise parsing and cumulative code generation, which is straightforward in the absence of goto. Yet scope changes (e.g. due to a tardy rogue let x; after thousands of statements the first of which uses x) require a separate pass to generate the necessary block set-up code before the first use of x.
Unifying let and var at top level also reduces the cognitive load (number of scopes in mind) and eliminates useless name shadowing opportunities.
But such let/var unification at top level in a function body does leave bad old arguments[0] aliasing x in the above example, allowing mutation of otherwise lexical let x; (change var to let and fill in the ... with arguments[0] = 42; return x). The answer that we chose for ES4, and the one that's already agreeable in committee for Harmony, was "deprecate arguments by providing optional and rest parameters".
You can't do that.
Which "that" do you mean?
Turn let into var at the top level of a function body.
function f() { x = 15; ... let t = some_runtime_expression; ... let x:t = ... }
What is the type of x at the beginning of the function?
The example had better fail to compile.
Because presumably the "let x:t" became "var x:t" and var can't have types? You can't be serious that let would be broken unless enclosed by an independent block!
WaldemarOn Oct 13, 2008, at 5:29 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
On Oct 13, 2008, at 3:51 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
After some experiments, we decided for ES4 to make let and var the same at top level in a function or global code. ... You can't do that.
Which "that" do you mean?
Turn let into var at the top level of a function body.
Ok then!
function f() { x = 15; ... let t = some_runtime_expression; ... let x:t = ... }
What is the type of x at the beginning of the function?
The example had better fail to compile.
Because presumably the "let x:t" became "var x:t" and var can't have
types?
Why can't var have a type annotation?
You can't be serious that let would be broken unless enclosed by an
independent block!
I didn't say that :-P.
Brendan Eich wrote:
On Oct 13, 2008, at 5:29 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
On Oct 13, 2008, at 3:51 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
After some experiments, we decided for ES4 to make let and var the same at top level in a function or global code. ... You can't do that.
Which "that" do you mean?
Turn let into var at the top level of a function body.
Ok then!
function f() { x = 15; ... let t = some_runtime_expression; ... let x:t = ... }
What is the type of x at the beginning of the function?
The example had better fail to compile.
Because presumably the "let x:t" became "var x:t" and var can't have types?
Why can't var have a type annotation?
Because a function can have many var declarations for the same variable and because you can use the variable before any of the var declarations are evaluated. You can work out the implications easily enough. This has been brought up at meetings before.
You can't be serious that let would be broken unless enclosed by an
independent block!
I didn't say that :-P.
Actually, you did. You wrote that you'd like let to become var (and unlike let as used within an independent block) if used at the top level of a function:
After some experiments, we decided for ES4 to make let and var the
same at top level in a function or global code.This helped avoid implementation pain: very long scripts on the web,
consisting of many statements in a row, motivate statement-wise
parsing and cumulative code generation, which is straightforward in
the absence of goto. Yet scope changes (e.g. due to a tardy rogue let
x; after thousands of statements the first of which uses x) require a
separate pass to generate the necessary block set-up code before the
first use of x.
What did you mean by "had better fail to compile"? Other than the type annotation, there is nothing about
function f() { x = 15; ... var t = some_runtime_expression; ... var x:t = ... }
that ought to fail to compile.
WaldemarOn Oct 13, 2008, at 6:08 PM, Waldemar Horwat wrote:
Brendan Eich wrote:
Because presumably the "let x:t" became "var x:t" and var can't have types?
Why can't var have a type annotation?
Because a function can have many var declarations for the same
variable and because you can use the variable before any of the var
declarations are evaluated. You can work out the implications
easily enough. This has been brought up at meetings before.
Of course, ES4 had var x:t all over, but as you noted, different rules
for evaluating t. Still, with new syntax comes the opportunity for
reformed semantics, including restrictions on any "bad" var abusages
we would like to move away from.
So my question remains (amended to be clear about what is the same):
why does this mean we cannot equate let and var scope at the top
level? I'm not talking about allowing them to mix in bad ways, or mix
at all (see below). I'm talking about not having an implicit block
around top-level function and global code.
You wrote that you'd like let to become var (and unlike let as used
within an independent block) if used at the top level of a function:
Ok, sorry for being unclear. I do not mean to translate top-level
'let' to 'var' and free let bindings from necessary restrictions. I do
mean that let binds in the variable object, and let usage restricts
other usage.
We can forbid mixed var x and let x at top level. We can require the
same single definition for any let x at any level. We can forbid
arguments usage if arguments[i] could alias a let binding, since let
is new (along with rest and optional parameters to enable arguments
deprecation).
But must we have an implicit block around programs and function bodies
that contain let declarations, distinct from the variable scope
(object)?
What did you mean by "had better fail to compile"? Other than the
type annotation, there is nothing aboutfunction f() { x = 15; ... var t = some_runtime_expression; ... var x:t = ... }
that ought to fail to compile.
The assignment to x in that temporal dead zone before t's initializer
has been evaluated.
Why is this different if you s/var x/let x/?
Waldemar Horwat wrote:
I am talking about let bindings. Lars brought up at that meeting. I did not find the use cases particularly convincing, but the dead zone is compelling. There are four ways to do this: A1. Lexical dead zone. References textually prior to a definition in the same block are an error. A2. Lexical window. References textually prior to a definition in the same block go to outer scope. B1. Temporal dead zone. References temporally prior to a definition in the same block are an error. B2. Temporal window. References temporally prior to a definition in the same block go to outer scope.
Let's take a look at an example:
let x = "outer"; function g() {return "outer"}
{ g(); function f() { ... x ... g ... g() ... } f(); var t = some_runtime_type; const x:t = "inner"; function g() { ... x ... } g(); f(); }
B2 is bad because then the x inside g would sometimes refer to "outer" and sometimes to "inner".
A1 and A2 introduce extra complexity but doesn't solve the problem.
I already suggested a solution to the problem based on a refinement of A1:
www.mail-archive.com/[email protected]/msg00899.html
That solution is entirely compatible with runtime types. The same identifier can be declared with different types in different parts of a block, and each reference to it uses the one that is textually in scope, so there is no need to check compatibility of types.
David-Sarah Hopwood wrote:
Waldemar Horwat wrote:
I am talking about let bindings. Lars brought up at that meeting. I did not find the use cases particularly convincing, but the dead zone is compelling. There are four ways to do this: A1. Lexical dead zone. References textually prior to a definition in the same block are an error. A2. Lexical window. References textually prior to a definition in the same block go to outer scope. B1. Temporal dead zone. References temporally prior to a definition in the same block are an error. B2. Temporal window. References temporally prior to a definition in the same block go to outer scope.
Let's take a look at an example:
let x = "outer"; function g() {return "outer"}
{ g(); function f() { ... x ... g ... g() ... } f(); var t = some_runtime_type; const x:t = "inner"; function g() { ... x ... } g(); f(); }
B2 is bad because then the x inside g would sometimes refer to "outer" and sometimes to "inner".
A1 and A2 introduce extra complexity but doesn't solve the problem.
I already suggested a solution to the problem based on a refinement of A1:
This post was more directly relevant: www.mail-archive.com/[email protected]/msg00889.html
On Oct 13, 2008, at 6:39 PM, Brendan Eich wrote:
Users may be modeling closures as capturing bindings, not scope chains of mutable objects, one per for (let...) statement or explicitly braced block. If so, could we make let declaration capture this way? Again, I'm proceeding from real users' complaints, not idle wishes.
Are you suggesting that closures over let capture bindings in the
general case? I hope not. I think for loops are a special case,
otherwise let is not the new var. ;)
WRT for loops, it's important to remember that let provides an
alternative that wasn't possible with var. Suppose for moment we do
not rebind on every iteration:
for (let i = 0; i < objects.length; i++) { let j = i; objects[i].callback = function() { print(j); } }
The syntactic overhead for such a workaround is much less than for var:
for (var i = 0; i < objects.length; i++) { objects[i].callback = buildCallback(i); }
function buildCallback(i) { print(i); }
The for/closures "bug" is definitely a newbie trap, but its pain is
not its discovery, but the difficulty of working around it. To me
this could be a winning argument against re-binding on each loop,
since re-binding precludes the (admittedly dubious) use-case of
updating inside the body:
for (let i = 0; i < 100; i++) { if (skipAhead) { i += 9; continue; } ... }
Neil Mix wrote:
The for/closures "bug" is definitely a newbie trap, but its pain is
not its discovery, but the difficulty of working around it. To me
this could be a winning argument against re-binding on each loop,
since re-binding precludes the (admittedly dubious) use-case of
updating inside the body:for (let i = 0; i < 100; i++) { if (skipAhead) { i += 9; continue; } ... }
The expansions that MarkM and I posted work correctly in that case.
("Correctly" means what a C programmer would expect :-) To be more specific, updates within the body take effect for the instance of the variable that is seen by the update expression for the next iteration.)
David-Sarah Hopwood wrote:
Neil Mix wrote:
The for/closures "bug" is definitely a newbie trap, but its pain is
not its discovery, but the difficulty of working around it. To me
this could be a winning argument against re-binding on each loop,
since re-binding precludes the (admittedly dubious) use-case of
updating inside the body:for (let i = 0; i < 100; i++) { if (skipAhead) { i += 9; continue; } ... }
The expansions that MarkM and I posted work correctly in that case.
("Correctly" means what a C programmer would expect :-) To be more specific, updates within the body take effect for the instance of the variable that is seen by the update expression for the next iteration.)
What I meant to say was: Updates within the body affect the value of the variable instance that is seen by the update expression for the next iteration.
On Oct 14, 2008, at 7:38 AM, Neil Mix wrote:
On Oct 13, 2008, at 6:39 PM, Brendan Eich wrote:
Users may be modeling closures as capturing bindings, not scope
chains of mutable objects, one per for (let...) statement or explicitly braced block. If so, could we make let declaration capture this way? Again, I'm proceeding from real users' complaints, not idle wishes.Are you suggesting that closures over let capture bindings in the general case? I hope not. I think for loops are a special case, otherwise let is not the new var. ;)
What about for-in loops?
I'm proceeding from user expectations being confounded. Something
needs help here, possibly not for (;;) loops -- but almost certainly
for-in loops containing closures capturing the loop variable.
WRT for loops, it's important to remember that let provides an alternative that wasn't possible with var. Suppose for moment we do not rebind on every iteration:
for (let i = 0; i < objects.length; i++) { let j = i; objects[i].callback = function() { print(j); } }
Yes, this is in the bug I cited (bugzilla.mozilla.org/show_bug.cgi?id=449811 ). It is still a PITA.
The syntactic overhead for such a workaround is much less than for
var:for (var i = 0; i < objects.length; i++) { objects[i].callback = buildCallback(i); }
function buildCallback(i) { print(i); }
True, which makes it even more painful in some ways. "Why doesn't JS
do it for me?" Closer, almost-but-not-quite-right, is torturous.
The for/closures "bug" is definitely a newbie trap, but its pain is not its discovery, but the difficulty of working around it. To me this could be a winning argument against re-binding on each loop, since re-binding precludes the (admittedly dubious) use-case of updating inside the body:
for (let i = 0; i < 100; i++) { if (skipAhead) { i += 9; continue; } ... }
So we can have our cake and eat it too with a tail-recursive
desugaring that forwards the previous iteration's binding value to the
next iteration's binding. Of course any practical implementation will
have to do more analysis than is currently done in the top open-source
implementations I've looked at. Could be worth the cost (VM hackers
can take the cost for the larger team of users). What do you think?
What about for-in loops?
I'm proceeding from user expectations being confounded. Something
needs help here, possibly not for (;;) loops -- but almost certainly
for-in loops containing closures capturing the loop variable.
Whatever is done should be consistent between for and for-in. My
concern was that you were suggesting that a closure over any let
variable binds the value and not the scope. i.e.:
let x = 5; let f = function() { print(x); } x = 6; f(); // prints 5??
I don't think that's what you were proposing, but I wasn't entirely
clear.
So we can have our cake and eat it too with a tail-recursive
desugaring that forwards the previous iteration's binding value to
the next iteration's binding. Of course any practical implementation
will have to do more analysis than is currently done in the top open- source implementations I've looked at. Could be worth the cost (VM
hackers can take the cost for the larger team of users). What do you
think?
Yeah, I got lost in the thread and missed the subtlety of that follow-
up. I'm having a hard time seeing any downside, other than perhaps a
mild inconsistency between var and let behavior here. Worth it, just
as long as any form with the general syntax of
<keyword> ( ... <let declaration/assigment> ... ) { <body> }
behaves in the same manner. (My point being that it's syntactically
strange that let appearing outside of the braces is bound inside the
braces --thinking like a language user, not a language implementer --
but that's good behavior as long as it's consistent.)
On 2008-10-11, at 08:34EDT, David Herman wrote:
Thank you for pointing out, though, that try/catch isn't so easily
defined on top of return-to-label, since it still needs special
handling for finally. The options are either to define a lower-level
primitive underlying try/finally (akin to Scheme's dynamic-wind), or
to leave exceptions -- or at least try/finally -- as primitive. I
lean towards the latter; dynamic-wind is a subtle beast.[For those interested in dynamic-wind, Flatt et al's ICFP 07 paper
has a nice investigation into its semantics:www.cs.utah.edu/plt/publications/icfp07-fyff.pdf ]
My 2p: If introducing a lower-level primitive would mean the language
could someday have handler-binding (the ability to handle a condition
in the frame it is signaled in), I think that would be a big win.
Brendan Eich wrote:
What did you mean by "had better fail to compile"? Other than the type annotation, there is nothing about
function f() { x = 15; ... var t = some_runtime_expression; ... var x:t = ... }
that ought to fail to compile.
The assignment to x in that temporal dead zone before t's initializer has been evaluated.
Why is this different if you s/var x/let x/?
Ah, ok. That's good news! For a long time I wanted to make statically detectable uses of a variable before its let or const declaration runs to be compile-time errors. You can't detect all of them due to closures, but you can detect most in practice. I'm glad you agree.
WaldemarDavid-Sarah Hopwood wrote:
David-Sarah Hopwood wrote:
Waldemar Horwat wrote:
I am talking about let bindings. Lars brought up at that meeting. I did not find the use cases particularly convincing, but the dead zone is compelling. There are four ways to do this: A1. Lexical dead zone. References textually prior to a definition in the same block are an error. A2. Lexical window. References textually prior to a definition in the same block go to outer scope. B1. Temporal dead zone. References temporally prior to a definition in the same block are an error. B2. Temporal window. References temporally prior to a definition in the same block go to outer scope.
Let's take a look at an example:
let x = "outer"; function g() {return "outer"}
{ g(); function f() { ... x ... g ... g() ... } f(); var t = some_runtime_type; const x:t = "inner"; function g() { ... x ... } g(); f(); }
B2 is bad because then the x inside g would sometimes refer to "outer" and sometimes to "inner".
A1 and A2 introduce extra complexity but doesn't solve the problem. I already suggested a solution to the problem based on a refinement of A1:
This post was more directly relevant: www.mail-archive.com/[email protected]/msg00889.html
This won't work for many reasons. Making the scope of a function depend on the details of what it refers to inside is too subtle of a rule to be practical. You'd need to examine the body of the entire function before you know when you can refer to its name from the outside. Also, it interacts badly with existing ES3 semantics.
Here's an example:
var f;
f(5);
const x = 2;
function f(a) { const g = function() {return x;} if (a != 5) g(); if (false) x; return 3; }
This had better work, or else you're violating ES3 semantics. There's nothing wrong with the call to f.
WaldemarOn Oct 14, 2008, at 11:19 AM, Waldemar Horwat wrote:
Brendan Eich wrote:
function f() { x = 15; ... var t = some_runtime_expression; ... var x:t = ... }
that ought to fail to compile.
The assignment to x in that temporal dead zone before t's initializer has been evaluated.
Why is this different if you s/var x/let x/?
Ah, ok. That's good news! For a long time I wanted to make
statically detectable uses of a variable before its let or const
declaration runs to be compile-time errors. You can't detect all of
them due to closures, but you can detect most in practice. I'm glad
you agree.
But before you change the assumptions of our argument, what was
hazardous about that example with var, but not with let, assuming no
static analysis requirement to flag an error? Or did I misunderstand
your point?
I mentioned in email to you this paper:
portal.acm.org/citation.cfm?id=197331
I'm not so worried about analyses that require computing dominance
relations within a function. Computing name uses and intersecting
scopes across functions are a different order of work (implementation
complexity). Advanced implementations are already doing such things
(where with and eval don't create hopeless ambiguities) but the spec
should not require anything approaching whole-program or all-nesting/
nested-functions analyses.
Other implementors on the list should sound off.
Please specify what you are proposing. The one proposal I've seen is:
Expression ::= ... | lambda Formals Statement
Yes, that's what I meant, or at least what I thought Yuh-Ruey meant.
This is not particularly useful because then even assign a lambda to a variable would be a syntax error,
Why is that?
and you'd introduce a bizarre asymmetry into the ExpressionNoIn case.
Can you explain?
The questions you'd have to answer are:
- Where in the expression grammar would you introduce a lambda?
I assume you mean what is the precedence. I'm not sure, although presumably it would be about the same as `let' expressions.
- Is lambda a reserved word?
This is a relevant question for any number of alternative syntaxes for lambda. I would think it would be the same as `let'.
- How does it interact with semicolon insertion? As written, if the Statement were an expression statement then the semicolon would be mandatory.
Why is that? If that were so then I would be absolutely opposed to Yuh-Ruey's syntax: the appeal of the idea was unifying the expression-body and statement-body form into one single definition.
- Do you really want to have extra semicolons in the middle of statements? Now you're forced to accept things like the following:
for (; lambda() x++;, lambda() y++;,; lambda() z++;) ...
Thanks, these were the kinds of nastinesses I was worried about. Arbitrarily mixing statements into expression contexts does indeed appear to get pretty convoluted. Perhaps just enforcing the block form of lambda is enough to keep things from getting too mixed up, specifically, by enforcing syntactic delimiters between expressions.
Overall, this syntax does not look promising. It's likely to get you into complexity trouble.
Do you prefer (lambda Formals Block | lambda Formals Expression)? [Personally I'm fine with that.] Or do you oppose any lambda expressions at all? Or did you have something else in mind?
Thanks,
On Wed, Oct 15, 2008 at 12:31 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
Do you prefer (lambda Formals Block | lambda Formals Expression)? [Personally I'm fine with that.] Or do you oppose any lambda expressions at all? Or did you have something else in mind?
My preference is
"lambda" Formals? Block
Since an invocation of the closure evaluates to its completion value, the only savings of introducing the Expression form would be a pair of curlies. This is too small a benefit to justify another case in the grammar. Further, in exchange for always requiring the curlies, we can make the formals optional. Once we've got lambda, we'll start using them for control abstractions, in which case the no-parameter form will become quite common. I'd rather be able to leave out the parens than the curlies. YMMV. This is admittedly subjective, but you did ask about preferences.
Dave Herman wrote:
Please specify what you are proposing. The one proposal I've seen is:
Expression ::= ... | lambda Formals Statement
Yes, that's what I meant, or at least what I thought Yuh-Ruey meant.
This is not particularly useful because then even assign a lambda to a variable would be a syntax error,
Why is that?
Because there exists no sequence of grammar production expansions that would expand to "a = lambda() x;".
and you'd introduce a bizarre asymmetry into the ExpressionNoIn case.
Can you explain?
Look at how the ExpressionNoIn cases are handled in the existing grammar. They are necessary. If you have an expression that contains a statement that contains an expression that contains the keyword "in" then you'll introduce a grammatical ambiguity into for-in statements.
The questions you'd have to answer are:
- Where in the expression grammar would you introduce a lambda?
I assume you mean what is the precedence. I'm not sure, although presumably it would be about the same as `let' expressions.
There is no such thing as a "let expression".
I don't mean precedence. I mean which rule in the expression grammar would describe a lambda. Precedence is a side effect of that, but obviously not all syntax can be captured by the concept of precedence.
- Is lambda a reserved word?
This is a relevant question for any number of alternative syntaxes for lambda. I would think it would be the same as `let'.
- How does it interact with semicolon insertion? As written, if the Statement were an expression statement then the semicolon would be mandatory.
Why is that? If that were so then I would be absolutely opposed to Yuh-Ruey's syntax: the appeal of the idea was unifying the expression-body and statement-body form into one single definition.
It's mandatory because the grammar and semicolon insertion rules say so. Btw, to properly terminate a lambda expression you'd need two semicolons. Here's why one would be insufficient:
f = lambda(x) x; (a + b) && c;
would parse the body of the lambda as the expression "x;", the "(a + b)" as an argument to the lambda, and the rest as applying to the result of calling the lambda. What you'd want to write instead would be:
f = lambda(x) x;;
- Do you really want to have extra semicolons in the middle of statements? Now you're forced to accept things like the following:
for (; lambda() x++;, lambda() y++;,; lambda() z++;) ...
Thanks, these were the kinds of nastinesses I was worried about. Arbitrarily mixing statements into expression contexts does indeed appear to get pretty convoluted. Perhaps just enforcing the block form of lambda is enough to keep things from getting too mixed up, specifically, by enforcing syntactic delimiters between expressions.
Overall, this syntax does not look promising. It's likely to get you into complexity trouble.
Do you prefer (lambda Formals Block | lambda Formals Expression)?
Speaking purely from a syntactic perspective:
- The block form is fine.
- The expression form has problems. It's hard to distinguish an expression that's part of the lambda from an expression that's part of the surrounding expression.
[Personally I'm fine with that.] Or do you oppose any lambda expressions at all? Or did you have something else in mind?
I don't see enough of a benefit to warrant inclusion of a new lambda concept. It's just another way of doing what local functions can do. If there is a bug with local functions, then fix that one instead instead of sowing confusion by having two concepts.
WaldemarI know you are well aware of the LetExpression form that was in proposed ES4 and has been implemented in Firefox since JS 1.7 -- why are you sounding like you've never heard of it?
In the grammar for proposed ES4, LetExpression was under PrimaryExpression. That's where I'm suggesting LambdaExpression might fit. IOW:
PrimaryExpression ::= ... | LetExpression | LambdaExpression
LetExpression ::= let ( LetBindingList ) CommaExpression LambdaExpression ::= lambda FunctionSignature LambdaExpressionBody
Look at how the ExpressionNoIn cases are handled in the existing grammar. They are necessary. If you have an expression that contains a statement that contains an expression that contains the keyword "in" then you'll introduce a grammatical ambiguity into for-in statements.
Okay, sure. This is already a bizarre corner case of the grammar. So it means in that case you'd have to parenthesize the lambda expression. Let's look at what we're talking about here; you can't write something like this:
for (x = lambda() foo in bar; ....)
I don't imagine this being all that common; how often do loop variables get bound to procedures? In fact, this is probably more likely to come up with let-expressions:
for (x = let(tmp = f()) tmp in obj; ...)
In any case, parenthesization resolves the issue. A corner case, yes, but IMO this is just part of the wartiness of the for-in syntax, and not reason enough to ban new expression forms.
What you'd want to write instead would be:
f = lambda(x) x;;
It might be possible to patch up this problem with more cleverness in the semicolon-insertion algorithm, but that way madness surely lies.
Anyway, I'm already convinced that (lambda Formals Statement) is not going to work out. I think we're in agreement there.
Do you prefer (lambda Formals Block | lambda Formals Expression)?
Speaking purely from a syntactic perspective:
- The block form is fine.
- The expression form has problems. It's hard to distinguish an expression that's part of the lambda from an expression that's part of the surrounding expression.
This is a problem with all compound expression forms that end in an expression (e.g., yield and let). I don't feel that's reason enough to ban them all.
I don't see enough of a benefit to warrant inclusion of a new lambda concept. It's just another way of doing what local functions can do.
As I see it, the major benefits of `lambda' are
a) introducing a low-level and compositional primitive that is useful for code generation (as a target language for compilers, macros, or other language features defined by desugaring), and
b) creating a clearer place in the language syntax to enforce tail calling by eliminating `return'
These needs aren't addressed by `function' and almost certainly can't be.
If there is a bug with local functions, then fix that one instead instead of sowing confusion by having two concepts.
Recall that lambda' does not includereturn', this',var', or
arguments'. We're not about to eliminate those fromfunction'! (Nor
would I advocate doing so-- even if we could somehow rewrite the entire
web.)
Maybe you don't feel that the issues addressed by lambda' are important enough to warrant the new feature, but "just changefunction'" is a not
a realistic argument.
Please specify what you are proposing. The one proposal I've seen is:
Expression ::= ... | lambda Formals Statement
Dave Herman wrote:
Yes, that's what I meant, or at least what I thought Yuh-Ruey meant.
This is not particularly useful because then even assign a lambda to a variable would be a syntax error,
Why is that?
2008/10/15 Waldemar Horwat <waldemar at google.com>:
Because there exists no sequence of grammar production expansions that would expand to "a = lambda() x;".
Only if you disallow newlines in the lambda syntax. This is certainly allowed in ES3:
a=lambda()
x;
Same goes for
a=lambda
{...}
or
a=lambda(...)
{...}
(...)
or any number of similar plays with the semicolon insertion rules.
On Oct 15, 2008, at 2:36 PM, Waldemar Horwat wrote:
There is no such thing as a "let expression".
Let expressions in JS1.7 (Firefox 2+), based on the ES4 proposal. ES3- ish grammar:
LetExpression :
let ( VariableDeclarationList ) [lookahead ∉ {{}]
AssignmentExpression
produced from PrimaryExpression.
It's mandatory because the grammar and semicolon insertion rules say
so. Btw, to properly terminate a lambda expression you'd need two
semicolons. Here's why one would be insufficient:f = lambda(x) x; (a + b) && c;
would parse the body of the lambda as the expression "x;", the "(a +
b)" as an argument to the lambda, and the rest as applying to the
result of calling the lambda. What you'd want to write instead
would be:f = lambda(x) x;;
Expression closures in JS1.8 (Firefox 3+) do not have this problem.
Using ES3's notation:
FunctionDeclaration : function Identifier ( FormalParameterListopt ) FunctionBody FunctionExpression : function Identifieropt ( FormalParameterListopt ) FunctionBody FormalParameterList : Identifier FormalParameterList , Identifier FunctionBody : { SourceElements } [lookahead ∉ {{}] AssignmentExpression
On Oct 15, 2008, at 6:13 PM, Brendan Eich wrote:
On Oct 15, 2008, at 2:36 PM, Waldemar Horwat wrote:
There is no such thing as a "let expression".
Let expressions in JS1.7 (Firefox 2+), based on the ES4 proposal.
ES3-ish grammar:LetExpression : let ( VariableDeclarationList ) [lookahead ∉ {{}]
AssignmentExpression
Rather,
LetExpression : let ( VariableDeclarationList ) AssignmentExpression
Brendan Eich wrote:
On Oct 15, 2008, at 2:36 PM, Waldemar Horwat wrote:
There is no such thing as a "let expression".
Let expressions developer.mozilla.org/en/New_in_JavaScript_1.7#let_expressions in JS1.7 (Firefox 2+), based on the ES4 proposal proposals:block_expressions. ES3-ish grammar:
/LetExpression /:
let*/ /*( VariableDeclarationList/ /*) [lookahead ∉{{}] AssignmentExpression**
produced from PrimaryExpression.
That's not a valid grammar. You can't have an AssignmentExpression terminating a PrimaryExpression. It leads to trouble such as:
let a = b + c
being interpreted as both:
(let a = b) + c
and:
let a = (b + c)
It's mandatory because the grammar and semicolon insertion rules say so. Btw, to properly terminate a lambda expression you'd need two semicolons. Here's why one would be insufficient:
f = lambda(x) x; (a + b) && c;
would parse the body of the lambda as the expression "x;", the "(a + b)" as an argument to the lambda, and the rest as applying to the result of calling the lambda. What you'd want to write instead would be:
f = lambda(x) x;;
Expression closures developer.mozilla.org/en/New_in_JavaScript_1.8#Expression_closures in JS1.8 (Firefox 3+) do not have this problem. Using ES3's notation:
/FunctionDeclaration /:
function */Identifier /*( */FormalParameterList//opt /*)/FunctionBody/ /FunctionExpression /:*
function */Identifier//opt /*( */FormalParameterList//opt /*)/FunctionBody/ /FormalParameterList /:* / Identifier/ / FormalParameterList /, /Identifier/ /FunctionBody /: / { SourceElements /}
*/[lookahead ∉ {*{*}] AssignmentExpression/
That's not a valid grammar. For example,
a = function(x) b+c
can parse as, among other things:
a = (function(x) b) + c
which is probably not what you want.
WaldemarOn Oct 16, 2008, at 11:38 AM, Waldemar Horwat wrote:
Brendan Eich wrote:
That's not a valid grammar.
It is unambiguous. How do you define "valid"?
You can't have an AssignmentExpression terminating a
PrimaryExpression. It leads to trouble such as:let a = b + c
being interpreted as both:
(let a = b) + c
No, it clearly parses as:
let a = (b + c)
and we've shipped this for several releases, and it sees use -- it has
not been problematic. Parenthesize if you mean (let a = b) + 1.
Same goes for expression closures. Bottom up parsers will shift, top
down will consume greedily. So what's the real problem?
Brendan Eich wrote:
On Oct 16, 2008, at 11:38 AM, Waldemar Horwat wrote:
Brendan Eich wrote:
That's not a valid grammar.
It is unambiguous. How do you define "valid"?
You can't have an AssignmentExpression terminating a PrimaryExpression. It leads to trouble such as:
let a = b + c
being interpreted as both:
(let a = b) + c
No, it clearly parses as:
let a = (b + c)
and we've shipped this for several releases, and it sees use -- it has not been problematic. Parenthesize if you mean (let a = b) + 1.
Same goes for expression closures. Bottom up parsers will shift, top down will consume greedily.
Huh? There is notion of shifting or consuming greedily in ES3. The ES3 grammar is unambiguous.
I don't think you can come up with a consistent "shift" or "greedy" notion. The one you may be thinking of will happily accept code such as
let (a = 5) x + y.foo = 2;
yet the Firefox code gives a syntax error for it despite it being parsable by your "grammar".
So what's the real problem?
I said it already. The problem is that you don't have a valid grammar. This one is invalid, so the code of the Firefox implementation is effectively the specification, and it's hard to reason about that.
Other examples: What does the following do?
for (a = let (b = c) b in d) ... vs. for (a = let (b = c) b in d;;) ... vs. for (a = let (b = c) b in d in d) ...
WaldemarOn Wed, Oct 15, 2008 at 4:49 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
In the grammar for proposed ES4, LetExpression was under PrimaryExpression. That's where I'm suggesting LambdaExpression might fit. IOW:
PrimaryExpression ::= ... | LetExpression | LambdaExpression
LetExpression ::= "let" "(" LetBindingList ")" CommaExpression LambdaExpression ::= "lambda" FunctionSignature LambdaExpressionBody [quotes inserted above]
As we argue about the pros and cons of this proposed expression-body-based grammar and similar variations, does anyone see any problems at all with the following block-based alternative grammar for them?
LetExpression ::= "let" ("(" LetBindingList ")")? Block LambdaExpression ::= "lambda" FunctionSignature? Block
The only potential advantage I have seen claimed for any of the other syntaxes over the above block-based syntax is the occasional savings of two curly brackets -- arguably in exchange for possible confusion. Are there any other problems with this block-based proposal? If we do adopt this form of lambda, would there be any remaining need for non-block functions? Everywhere one might think to write
function(...) expr
one could just write
lambda (...) {expr}
instead.
On Oct 16, 2008, at 1:20 PM, Waldemar Horwat wrote:
I don't think you can come up with a consistent "shift" or "greedy"
notion.
Funny, yacc has had one for decades, used to resolve dangling-else.
The one you may be thinking of will happily accept code such as
let (a = 5) x + y.foo = 2;
yet the Firefox code gives a syntax error for it despite it being
parsable by your "grammar".
Cut the misattribution of your ideas to me, based on misinterpretation
of your "experiment". Here's what's going on:
Firefox (SpiderMonkey, shown here via its REPL) throws an error:
js> y = {} [object Object] js> let (a = 5) x + y.foo = 2
typein:2: SyntaxError: invalid assignment left-hand side: typein:2: let (a = 5) x + y.foo = 2 typein:2: ......................^
This is an error required by ES3, but it is not a syntax error
according to the ES3 grammar -- it is an error due to semantic
checking done in the spec by PutValue:
11.13.1 Simple Assignment ( = )
The production AssignmentExpression : LeftHandSideExpression =
AssignmentExpression is evaluated as follows:
- Evaluate LeftHandSideExpression.
- Evaluate AssignmentExpression.
- Call GetValue(Result(2)).
- Call PutValue(Result(1), Result(3)).
- Return Result(3).
8.7.2 PutValue (V, W)
- If Type(V) is not Reference, throw a ReferenceError exception.
- Call GetBase(V).
- If Result(2) is null, go to step 6.
- Call the [[Put]] method of Result(2), passing GetPropertyName(V)
for the property name and W for the value. - Return.
- Call the [[Put]] method for the global object, passing
GetPropertyName(V) for the property name and W for the value. - Return.
SpiderMonkey historically used SyntaxError, not ReferenceError, and
throw at compile-time. This pre-dates ES1. Another example not
involving let expressions:
js> a + b = c
typein:1: SyntaxError: invalid assignment left-hand side: typein:1: a + b = c typein:1: ......^
From ES3 chapter 16:
An implementation may treat any instance of the following kinds of
runtime errors as a syntax error and therefore
report it early:
• Improper uses of return, break, and continue.
• Using the eval property other than via a direct call.
• Errors in regular expression literals.
• Attempts to call PutValue on a value that is not a reference (for
example, executing the assignment statement
3=4).
You may object that we should throw ReferenceError not SyntaxError --
that is not entirely clear from the chapter 16 wording, but it is at
most a bug unrelated to our disagreement, and it doesn't prove any
claim that primary expressions ending in assignment expressions are
ambiguous or unusable.
So what's the real problem?
I said it already. The problem is that you don't have a valid
grammar.
You have not demonstrated that claim.
This one is invalid, so the code of the Firefox implementation is
effectively the specification, and it's hard to reason about that.
It's easy, you can do it ;-).
Other examples: What does the following do?
for (a = let (b = c) b in d) ...
SyntaxError because no ; after first expression in for (;;) loop head.
vs. for (a = let (b = c) b in d;;) ...
Valid syntax.
vs. for (a = let (b = c) b in d in d) ...
SyntaxError because no ; after first expression in for (;;) loop head.
Yes, you can chain in (relational) expressions within an
AssignmentExpression. No, users are not flummoxed, or as far as we can
tell in over two years even bothered, by this. Yes, it can be
specified unambiguously.
On Oct 15, 2008, at 1:28 PM, Mark S. Miller wrote:
On Wed, Oct 15, 2008 at 12:31 PM, Dave Herman <dherman at ccs.neu.edu>
wrote:Do you prefer (lambda Formals Block | lambda Formals Expression)? [Personally I'm fine with that.] Or do you oppose any lambda
expressions at all? Or did you have something else in mind?My preference is
"lambda" Formals? Block
Since an invocation of the closure evaluates to its completion value, the only savings of introducing the Expression form would be a pair of curlies. This is too small a benefit to justify another case in the grammar. Further, in exchange for always requiring the curlies, we can make the formals optional. Once we've got lambda, we'll start using them for control abstractions, in which case the no-parameter form will become quite common. I'd rather be able to leave out the parens than the curlies. YMMV. This is admittedly subjective, but you did ask about preferences.
Hi Mark,
That's fair and closely reasoned, but I question the likelihood of
zero-argument lambdas emerging in control abstractions favored by all
that many users. For now I favor the lambda expression form as well as
the lambda block form. You're right that much of this is subjective,
not just personal but also speculative about what users will tend to
use.
Expression closures are already insanely popular in JS1.8, of course
used only in Mozilla-specific code. They'd be even more winning as
lambdas, whether zero args is common or not (six letter keyword
instead of eight), but losing the braces is part of the win. The two
shifted chars are not easy on some peoples' hands (ask Python hackers
who are spared such RSI-inducing chords). The visual weight is less,
to boot.
Definitely my two cents, no more (possibly fewer ;-).
On Oct 16, 2008, at 3:33 PM, Brendan Eich wrote:
SpiderMonkey historically used SyntaxError, not ReferenceError, and
throw at compile-time. This pre-dates ES1. Another example not
involving let expressions:js> a + b = c typein:1: SyntaxError: invalid assignment left-hand side: typein:1: a + b = c typein:1: ......^
From ES3 chapter 16:
An implementation may treat any instance of the following kinds of
runtime errors as a syntax error and therefore report it early: • Improper uses of return, break, and continue. • Using the eval property other than via a direct call. • Errors in regular expression literals. • Attempts to call PutValue on a value that is not a reference (for
example, executing the assignment statement 3=4).You may object that we should throw ReferenceError not SyntaxError
-- that is not entirely clear from the chapter 16 wording,
"An implementation may treat any instance of the following kinds of
runtime errors as a syntax error and therefore report it early" seems
clear enough: an implementation that does choose ("may") to report the
ReferenceError from 8.7.2 PutValue step 1 must treat that runtime
error as a syntax error -- throw a SyntaxError, in other words.
There's no SpiderMonkey bug here, and it would be incorrect to do
other than change ReferenceError to SyntaxError when reporting the
PutValue error at compile time.
Again, this is not a grammatical error. The ES3 grammar is not enough
to decide what syntax errors might result from feeding a given program
source to an implementation.
Mark S. Miller wrote:
On Wed, Oct 15, 2008 at 4:49 PM, Dave Herman <dherman at ccs.neu.edu> wrote:
In the grammar for proposed ES4, LetExpression was under PrimaryExpression. That's where I'm suggesting LambdaExpression might fit. IOW:
PrimaryExpression ::= ... | LetExpression | LambdaExpression
LetExpression ::= "let" "(" LetBindingList ")" CommaExpression LambdaExpression ::= "lambda" FunctionSignature LambdaExpressionBody [quotes inserted above]
As we argue about the pros and cons of this proposed expression-body-based grammar and similar variations, does anyone see any problems at all with the following block-based alternative grammar for them?
LetExpression ::= "let" ("(" LetBindingList ")")? Block LambdaExpression ::= "lambda" FunctionSignature? Block
You read my mind; that is exactly what I was about to propose.
Note also that ("(" LetBindingList ")") and FunctionSignature are almost identical, and could probably be merged. In that case
let ... { body }
could be sugar for
(lambda ... { body })()
if the binding of default arguments to a lambda has the same semantics as the binding of variables in a let. (There are several reasonable choices for that semantics; I suggest reading www.cs.indiana.edu/~dyb/pubs/fixing-letrec.pdf to start with.)
The degenerate syntax "let {...}" allowed by this grammar at first-sight doesn't seem very useful, until you realize that it has a similar effect (apart from not preventing hoisting of 'var' declarations past the 'let') to the Crockford module pattern "(function() {...})()".
On Oct 16, 2008, at 3:33 PM, Brendan Eich wrote:
On Oct 16, 2008, at 1:20 PM, Waldemar Horwat wrote:
I don't think you can come up with a consistent "shift" or "greedy"
notion.Funny, yacc has had one for decades, used to resolve dangling-else.
Which ES1-3 also use:
12.5 The if Statement
Syntax
IfStatement :
if ( Expression ) Statement else Statement
if ( Expression ) Statement
Each else for which the choice of associated if is ambiguous shall be
associated with the nearest possible if that would otherwise have no
corresponding else.
The grammar is not enough in ES3 without let expressions. You need at
least this shift-reduce conflict resolution rule (I'm ignoring ASI, /
as division vs. regexp delimiter, etc.). Adding let expressions does
not add novel requirements for extra-grammatical disambiguation rules.
The ambiguity between blocks and object initialisers is another well- known case, resolved by "negative lookahead":
12.4 Expression Statement Syntax ExpressionStatement : [lookahead ∉ {{, function}] Expression ;
Brendan Eich wrote:
On Thu, Oct 16, 2008 at 5:49 PM, David-Sarah Hopwood <david.hopwood at industrial-designers.co.uk> wrote:
[lots of good stuff snipped]
I agree.
The degenerate syntax "let {...}" allowed by this grammar at first-sight doesn't seem very useful, until you realize that it has a similar effect (apart from not preventing hoisting of 'var' declarations past the 'let') to the Crockford module pattern "(function() {...})()".
Since non-var declarations will now be lexically block scoped, all these non-var good features but one of Crock's module pattern will also be present for simple blocks. Since lambda preserves TC, it must be thus.
The one additional feature provided both by Crock's module pattern and by degenerate let is turn EcmaScript effectively into an expression language. Rather than saying
...(3 + function(){while(...){...}; return 4;}())...
we'd now be able to say
...(3 + lambda{while(...){...}; 4}())...
or even ...(3 + let{while(...){...}; 4})...
Of course, if you want a begin/end that works for vars as well, you'd still have to use Crock's module pattern.
On Oct 16, 2008, at 7:01 PM, Brendan Eich wrote:
On Oct 16, 2008, at 3:33 PM, Brendan Eich wrote:
On Oct 16, 2008, at 1:20 PM, Waldemar Horwat wrote:
I don't think you can come up with a consistent "shift" or
"greedy" notion.Funny, yacc has had one for decades, used to resolve dangling-else.
Which ES1-3 also use:
12.5 The if Statement Syntax IfStatement : if ( Expression ) Statement else Statement if ( Expression ) Statement Each else for which the choice of associated if is ambiguous shall
be associated with the nearest possible if that would otherwise have
no corresponding else.The grammar is not enough in ES3 without let expressions. You need
at least this shift-reduce conflict resolution rule (I'm ignoring
ASI, / as division vs. regexp delimiter, etc.). Adding let
expressions does not add novel requirements for extra-grammatical
disambiguation rules.The ambiguity between blocks and object initialisers is another well- known case, resolved by "negative lookahead":
12.4 Expression Statement Syntax ExpressionStatement : [lookahead ∉ {{, function}] Expression ;
This one can be solved without negative lookahead, we have separate
NoBF (no brace or function) and NoIn productions in our grammar to
solve these without ambiguity:
Expr: AssignmentExpr | Expr ',' AssignmentExpr ;
ExprNoIn: AssignmentExprNoIn | ExprNoIn ',' AssignmentExprNoIn ;
ExprNoBF: AssignmentExprNoBF | ExprNoBF ',' AssignmentExpr ;
With the NoIn / NoBF variants propagated through the rest of the
grammar if needed. Kind of brute force, but it does the job without
ambiguity. JavaScriptCore's grammar runs through bison with no shift/
reduce or reduce/reduce conflicts.
I think it might be better to write the official ES3.1 grammar in this
way, even though it is a little annoying and repetitive, so it can
more readily be verified that the language grammar has no ambiguities
by running through a parser-generator like yacc or bison or
boost::spirit.
As to the else issue, I don't think that ambiguity can be avoided, but
bison lets you solve that with %nonassoc, which is a sound
disambiguation mechanism.
, Maciej
On Oct 16, 2008, at 7:04 PM, Waldemar Horwat wrote:
The parser is required to backtrack until it either finds an
expansion of the grammar that doesn't generate a syntax error or
until it discovers that they all do. You can choose to make
additional syntax errors as per chapter 16, but that does not
relieve you of the backtracking requirement.
You're right that a bottom up parser will have a reduce-reduce
conflict. For a top-down parser, it's not an issue.
Other examples: What does the following do?
for (a = let (b = c) b in d) ...
SyntaxError because no ; after first expression in for (;;) loop
head.It can't be a SyntaxError. It's a perfectly valid for-in statement.
Is this a perfectly valid for-in statement?
for (a = b in c);
Not according to ES3's grammar. An assignment expression is not valid
on the left of the for-in's "in":
IterationStatement : ... for ( LeftHandSideExpression in Expression ) Statement for ( var VariableDeclarationNoIn in Expression ) Statement
LeftHandSideExpression does not produce an unparenthesized
AssignmentExpression, and if you parenthesize then PutValue will
throw on the non-Reference result of the assignment, the
ReferenceError at runtime which again can become SyntaxError at
compile time.
Brendan Eich wrote:
On Oct 16, 2008, at 7:04 PM, Waldemar Horwat wrote:
The parser is required to backtrack until it either finds an expansion of the grammar that doesn't generate a syntax error or until it discovers that they all do. You can choose to make additional syntax errors as per chapter 16, but that does not relieve you of the backtracking requirement.
You're right that a bottom up parser will have a reduce-reduce conflict. For a top-down parser, it's not an issue.
Other examples: What does the following do?
for (a = let (b = c) b in d) ...
SyntaxError because no ; after first expression in for (;;) loop head.
It can't be a SyntaxError. It's a perfectly valid for-in statement.
Is this a perfectly valid for-in statement?
for (a = b in c);
Not according to ES3's grammar. An assignment expression is not valid on the left of the for-in's "in":
/IterationStatement /:
...*for ( */LeftHandSideExpression /*in */Expression /*) */Statement/for ( var */VariableDeclarationNoIn /*in */Expression /*)*/Statement/
LeftHandSideExpression does not produce an unparenthesized AssignmentExpression, and if you parenthesize then PutValue will throw on the non-Reference result of the assignment, the ReferenceError at runtime which again can become SyntaxError at compile time.
I accidentally took out the "var" in editing the message. It should have been
for (var a = ...)
WaldemarMaciej Stachowiak wrote:
As to the else issue, I don't think that ambiguity can be avoided, but bison lets you solve that with %nonassoc, which is a sound disambiguation mechanism.
It can. I have a machine-validated ES3 (and ES4 from earlier proposals) grammar that contains no ambiguities and no handwaving. The if-else rule is handled by having a marker on Statement productions just like there is a NoIn marker on Expression productions. See:
www.mozilla.org/js/language/old-es4/core/statements.html#N-IfStatement
I need a grammar with no ambiguities to do things like verify that semicolon insertion works and that the / regexp-vs-division resolution is always uniquely resolvable in favor of one or the other: you don't ever want a parser state which combines rules that have division with rules that have a regexp in the same spot because the lookahead depends on how you lex the /-token.
WaldemarMaciej Stachowiak wrote:
I think it might be better to write the official ES3.1 grammar in this way, even though it is a little annoying and repetitive, so it can more readily be verified that the language grammar has no ambiguities by running through a parser-generator like yacc or bison or boost::spirit.
I machine-verified the ES3 grammar directly and have a parser that supports the negative lookaheads. I'm using those in addition to the "NoIn" markers because otherwise there'd be an exponential blowup in the number of productions in the grammar to handle all the cases in ES4.
Yacc or bison or such are insufficient. You also need to verify that semicolon insertion is sound and that there exist no grammar states combining a / division in some branches with a / regexp in other branches in the same spot. This would break the lexer because you'd need to look ahead to resolve the ambiguity but can't because lexing depends on how you resolve that token.
Also, the ES4 grammar turned out to be LR(1), not LALR(1) as needed by yacc.
WaldemarOn Oct 17, 2008, at 11:05 AM, Waldemar Horwat wrote:
Brendan Eich wrote:
Is this a perfectly valid for-in statement?
for (a = b in c);
Not according to ES3's grammar. An assignment expression is not
valid on the left of the for-in's "in":/IterationStatement /:
...*for ( */LeftHandSideExpression /*in */Expression /*) */Statement/
for ( var */VariableDeclarationNoIn /*in */Expression /*)
(Note VariableDeclarationNoIn after var.)
*/Statement/
LeftHandSideExpression does not produce an unparenthesized AssignmentExpression, and if you parenthesize then PutValue will
throw on the non-Reference result of the assignment, the ReferenceError at runtime which again can become SyntaxError at compile time.I accidentally took out the "var" in editing the message. It should
have beenfor (var a = ...)
Waldemar _
In that case the -NoIn sub-grammar should apply, extended to
LetExpressionNoIn. So
for (let (a = b) c in d);
would do what you want. We would have to change SpiderMonkey for this
edge case, but that is our problem (and unlikely to break anyone).
On Oct 17, 2008, at 11:30 AM, Brendan Eich wrote:
In that case the -NoIn sub-grammar should apply, extended to LetExpressionNoIn. So
for (let (a = b) c in d);
Sorry, of course that should have been
for (var a = let (b = c) b in d);
On Oct 16, 2008, at 3:40 PM, Brendan Eich wrote:
For now I favor the lambda expression form as well as the lambda
block form.
Good thing I added "For now". Since I'm convinced by Waldemar's
argument against ambiguity, I have to drop support for the lambda
expression form. It could be rescued, maybe -- but at what cost? I'd
rather get on with harmonious lambda and let work.
In that light, having let {...} be sugar for lambda () { ... } () is
good.
Making lambda's parenthesized formal parameter list optional still
rubs me the wrong way. It's unambiguous given the block's braces. The
symmetry break with function means lambda{x}() returns x where
function(){return x}() (in an expression context) has matching () for
formals and actuals. But others reasonably object to the extra
insignificant parens. So I'll go with the flow and endorse lambda
{...} for nullary lambda -- for now. :-)
On Oct 17, 2008, at 11:17 AM, Waldemar Horwat wrote:
Maciej Stachowiak wrote:
As to the else issue, I don't think that ambiguity can be avoided,
but bison lets you solve that with %nonassoc, which is a sound
disambiguation mechanism.It can. I have a machine-validated ES3 (and ES4 from earlier
proposals) grammar that contains no ambiguities and no handwaving.
The if-else rule is handled by having a marker on Statement
productions just like there is a NoIn marker on Expression
productions. See:www.mozilla.org/js/language/old-es4/core/statements.html#N-IfStatement
I need a grammar with no ambiguities to do things like verify that
semicolon insertion works and that the / regexp-vs-division
resolution is always uniquely resolvable in favor of one or the
other: you don't ever want a parser state which combines rules that
have division with rules that have a regexp in the same spot because
the lookahead depends on how you lex the /-token.
In that case I would definitely prefer to see the official spec have
an unambiguous grammar.
(Your grammar still does include the [lookahead∉{function, {}]
construct which can be handled in an analogous way to NoIn.)
, Maciej
The parser is required to backtrack until it either finds an expansion of the grammar that doesn't generate a syntax error or until it discovers that they all do. You can choose to make additional syntax errors as per chapter 16, but that does not relieve you of the backtracking requirement.
You're right that a bottom up parser will have a reduce-reduce conflict. For a top-down parser, it's not an issue.
Are you sure about that, because you using hand written top-down parser, Is it confirmed by a top-down parser generator like ANTLR?
I think top-down parser has no issue to parse following code:
function() { }();
but this is not a valid statement because:
ExpressionStatement ::= [lookahead ! {{, function}] Expression ;
This kind question I asked long time ago, but no one answed,
and now seems write a bottom up JS parser is mission impossible.
,
Eric Suen
BTW
function() return b ? c : d
should be
function() b ? c : d
,
Eric Suen
On Oct 17, 2008, at 7:36 PM, Eric Suen wrote:
Are you sure about that, because you using hand written top-down
parser, Is it confirmed by a top-down parser generator like ANTLR?
I haven't used ANTLR or any other LL(*) parser generator, no.
I found code.google.com/p/antlr-javascript and wondered if
anyone on the list has evaluated it.
I think top-down parser has no issue to parse following code:
function() { }();
Agreed.
but this is not a valid statement because:
ExpressionStatement ::= [lookahead ! {{, function}] Expression ;
That's a good example. See bugzilla.mozilla.org/show_bug.cgi?id=376052 if you are interested in why people want to evaluate such a statement.
This kind question I asked long time ago, but no one answed,
Sorry, Jeff may have missed your question. I don't understand it
myself (no one ever proposed any syntax of the form x = function ()
return y).
and now seems write a bottom up JS parser is mission impossible.
Why do you say that? Maciej already cited the Bison grammar that
WebKit's JS engine uses. Waldemar has an out of date but ambitious
bottom-up grammar checker. The question before us is what to
standardize and how to check it.
An extension like let expressions is not a foregone conclusion,
likewise expression closures (which some folks in TC39 at the Oslo
meeting favored). If we don't want ambiguities that create prove-a-
negative hazards, we won't standardize these as-is. For example, we
could require parenthesization of the AssignmentExpression body; or
we could do something else, such as the braced lambda body that's been
discussed here recently.
If you mean bottom-up parsing of JS1.7 as implemented in SpiderMonkey
is impossible, I still wouldn't go that far. C and C++ are not easy to
parse bottom-up, but even ignoring lexer/parser feedback, it can be
done with enough hacking, AFAIK.
The C-like languages including JS do not inherit bottom-up grammar
bones from bad old C. The C top-down grammar (dmr's original C
compiler used recursive descent and operator precedence parsing) shows
through pretty clearly. I agree we shouldn't make things worse. I
don't think all hope is lost.
Eric Suen wrote:
I think top-down parser has no issue to parse following code:
function() { }();
but this is not a valid statement because:
ExpressionStatement ::= [lookahead ! {{, function}] Expression ;
There is a good reason for that. It's because of semicolon insertion. You'd break things if you tried to "fix" it.
Folks write code such as:
function f() { ... }
(a + b)....
and this would turn the expression into a function call.
Another issue is that the function name (the "f" in this case) does entirely different things depending on whether it's a function expression or a function definition.
WaldemarFollowup to discuss some open issues from the thread, evident in the
message cited below:
-
Unifying var scope and let scope at top level of a function could
be done, with tolerable restrictions: given let x at top level in a
function body, existence of formal parameter x, and redeclaration of
let x by top-level var x, are static errors. -
Unifying var scope and let scope at top level in global code cannot
work since var x creates a DontDelete property of the global object if
no property x exists, else redeclares an existing property x having
arbitrary attributes.
We could make let at top level in global code an error. Making such a
let bind in an implicit block seems bad because inconsistent with 1,
because implicit, and because it's more complicated to implement and
specify than making global top-level let an error.
Any pragma of the "use lexical scope" kind:
would remove this restriction.
Comments?
Brendan Eich wrote:
Followup to discuss some open issues from the thread, evident in the message cited below:
Unifying var scope and let scope at top level of a function could be done, with tolerable restrictions: given let x at top level in a function body, existence of formal parameter x, and redeclaration of let x by top-level var x, are static errors.
Unifying var scope and let scope at top level in global code cannot work since var x creates a DontDelete property of the global object if no property x exists, else redeclares an existing property x having arbitrary attributes.
We could make let at top level in global code an error. Making such a let bind in an implicit block seems bad because inconsistent with 1, because implicit, and because it's more complicated to implement and specify than making global top-level let an error.
Any pragma of the "use lexical scope" kind:
would remove this restriction.
Comments?
Presumably the same rationale applies to const and this means that we can't use const at the top level either, not even in ES3.1?
What does ES3.1 do if you have both const x and var x at the top level?
WaldemarOn Tue, Oct 21, 2008 at 3:42 PM, Waldemar Horwat <waldemar at google.com> wrote:
What does ES3.1 do if you have both const x and var x at the top level?
It just so happens that we discussed top level const in this morning's ES3.1 phone meeting. It never occurred to us to ban top level const. We did discuss making top level const scoped just to the program unit vs adding a property to the global object. We also discussed the const read barrier issue. Of your taxonomy of four choices, we had been writing the spec on the assumption of dynamic dead zone, but we think we would prefer static dead zone if its spec is simple enough and if it's simple enough to understand in practice. Given the number of choices and the complexity of choosing among them, especially our ignorance of the implications of "static dead zone", we decided to pull "const" from ES3.1. If we got in wrong in ES3.1, we couldn't fix it later. All block scoping constructs -- const, let, and nested function declarations -- now await Harmony.
Mark S. Miller wrote:
On Tue, Oct 21, 2008 at 3:42 PM, Waldemar Horwat <waldemar at google.com> wrote:
What does ES3.1 do if you have both const x and var x at the top level?
It just so happens that we discussed top level const in this morning's ES3.1 phone meeting. It never occurred to us to ban top level const. We did discuss making top level const scoped just to the program unit vs adding a property to the global object. We also discussed the const read barrier issue. Of your taxonomy of four choices, we had been writing the spec on the assumption of dynamic dead zone, but we think we would prefer static dead zone if its spec is simple enough and if it's simple enough to understand in practice. Given the number of choices and the complexity of choosing among them, especially our ignorance of the implications of "static dead zone", we decided to pull "const" from ES3.1.
I understand the implications of "static dead zone"; I just worked through them for Jacaranda 0.4. It is definitely possible to guarantee static initialization safety for 'let' and 'const' declarations, checkable using a simple and efficient algorithm (linear in the size of the block), and without any read or write barriers. The resulting function hoisting behaviour is a fairly simple generalization of ES3. I'll post a fleshed-out proposal either tomorrow or Thursday.
If we got in wrong in ES3.1, we couldn't fix it later.
That's true, but I still hope to convince you and the group that we can get it right for ES3.1.
On Oct 21, 2008, at 5:56 PM, David-Sarah Hopwood wrote:
If we got in wrong in ES3.1, we couldn't fix it later.
That's true, but I still hope to convince you and the group that we
can get it right for ES3.1.
With zero implementations due to lack of a spec that hangs together in
full, ES3.1 is already at risk for finalization next spring. Keep
pushing mission-creep (or jump) and you'll fall into 2010. That is not
in the interest of anyone using or implementing the language.
On Aug 21, 2008, at 10:50 AM, Lex Spoon wrote:
Why?
Why do you assume function expressions (not literals in any immutable
sense) have good variable definition semantics for their arguments?
I don't agree, but we'll see how Java closures work out. In the mean
time:
Propose some unambiguous syntax, since return may have an expression
on its right.
arguments.
This has never in my hearing been requested, and people use lots of
closures in JS. Experience does not back the desire here to invent
something new, or to borrow from languages with different evaluation
models.
TC39 is not going to invent, so the likely outcome in Harmony is
something actually implemented, with user feedback, with small
adjustments also implemented at least in open-source nightly if not
beta-tested fashion.
That's not an invitation to submit patches, btw. I'd rather have some
agreement in the committee before trying more experiments. Mozilla
has done a number over the years. They were worthwhile, but far from
cost-free.
On Aug 21, 2008, at 10:50 AM, Lex Spoon wrote: >>> (function(x, y) { >>> print(y); >>> })(2, x) >> >> We've been over this. What if you replace print(y) with return y? > > Desugaring to closures is highly worthwhile, Why? > so it looks worth solving > these. Defining good variable-definition semantics is hard, as the > experience with var shows, so it's good to reuse function literals if > at all possible. Why do you assume function expressions (not literals in any immutable sense) have good variable definition semantics for their arguments? > The problems you describe are actually not hard to fix. I don't agree, but we'll see how Java closures work out. In the mean time: > All that has > to happen is that return is labeled with the function it applies to. > If at parse time no label is supplied, then the parser can manufacture > a label to use. Propose some unambiguous syntax, since return may have an expression on its right. > So, I agree that return, etc., arguments. > need to be fixed up if you desugar to > closures, but they can indeed be fixed up. Further, you probably want > non-local return anyway if people are going to use a lot of function > literals in their code. This has never in my hearing been requested, and people use lots of closures in JS. Experience does not back the desire here to invent something new, or to borrow from languages with different evaluation models. TC39 is not going to invent, so the likely outcome in Harmony is something actually implemented, with user feedback, with small adjustments also implemented at least in open-source nightly if not beta-tested fashion. That's not an invitation to submit patches, btw. I'd rather have some agreement in the committee before trying more experiments. Mozilla has done a number over the years. They were worthwhile, but far from cost-free. /be