Introduction, comprehensions beyond arrays
On Thu, May 9, 2013 at 11:42 PM, Mike Stay <metaweta at gmail.com> wrote:
In Scala, this is desugared into expr1.flatMap(x => expr2(x).flatMap(y => ... exprN(x, y, ...).map(z => result(x, y, ..., z) ) ) )
Currently comprehensions use the same protocol as for-of statements, namely iterators. I think we definitely want them to use the same protocol.
.map() is appealing, but to work with for-of statements, it would have to support break, continue, and early return, either using exceptions (like Scala's Breaks) or something new. Part of the appeal of the iterator protocol is that it doesn't complicate break/continue/return.
Separately, for the list: are arrow-functions lexically transparent to super and arguments? I hope so! For this kind of desugaring, if nothing else.
Jason Orendorff wrote:
.map()is appealing, but to work with for-of statements, it would have to supportbreak,continue, and earlyreturn, either using exceptions (like Scala's Breaks) or something new. Part of the appeal of the iterator protocol is that it doesn't complicatebreak/continue/return.
Agreed. JS ain't Scala.
Note for Mike Stay, in case it helps a bit (just syntax): we agreed to use LTR order, so:
[for (x of expr1) for (y of expr2(x)) result(x, y)]
to shorten your example a bit.
Separately, for the list: are arrow-functions lexically transparent to
superandarguments? I hope so! For this kind of desugaring, if nothing else.
super like this is lexical -- the outer function's same-named
keyword meaning.
arguments per
harmony:arrow_function_syntax is
an error. Perhaps we should change this to match the keywords. Allen?
On May 10, 2013, at 8:48 AM, Jason Orendorff wrote:
On Thu, May 9, 2013 at 11:42 PM, Mike Stay <metaweta at gmail.com> wrote: In Scala, this is desugared into expr1.flatMap(x => expr2(x).flatMap(y => ... exprN(x, y, ...).map(z => result(x, y, ..., z) ) ) )
Currently comprehensions use the same protocol as for-of statements, namely iterators. I think we definitely want them to use the same protocol.
.map() is appealing, but to work with for-of statements, it would have to support break, continue, and early return, either using exceptions (like Scala's Breaks) or something new. Part of the appeal of the iterator protocol is that it doesn't complicate break/continue/return.
We previously explored having lambdas expressions that included support abnormal exits, including method level returns. I even did most of the spec. work that would be needed to define them. But the ultimate consensus was formed around arrow functions without such control escapes.
Separately, for the list: are arrow-functions lexically transparent to super and arguments? I hope so! For this kind of desugaring, if nothing else.
super is lexically resolved through the defining method so, just like this, it isn't rebound within an arrow function. Follow the spec. logic starting in 11.2.4.
I see that I have a TODO note in 10.5.3 that says "don't create an arguments binding for arrow functions". However, I'm not sure that we actually had consensus on that point. I'm with you on in thinking that arrows shouldn't rebind arguments but I think there may have been some push back on that.
On Fri, May 10, 2013 at 12:36 PM, Brendan Eich <brendan at mozilla.com> wrote:
Agreed. JS ain't Scala.
I wouldn't want it to be! And I agree that it makes sense for the array comprehensions to match the for-of semantics.
Note for Mike Stay, in case it helps a bit (just syntax): we agreed to use LTR order, so:
[for (x of expr1) for (y of expr2(x)) result(x, y)]to shorten your example a bit.
OK, thanks. I suppose that with the arrow-functions, the boilerplate load drops tremendously. Is the ES6 spec too far along to introduce some other construct that would desugar to the above, or is there still time to create a proposal?
On May 10, 2013, at 11:36 AM, Brendan Eich wrote:
'arguments' per harmony:arrow_function_syntax is an error. Perhaps we should change this to match the keywords. Allen?
Oy yeah, look at that... I don't like these sorts of static semantics restrictions on expression elements. PrimaryExpression does't have the context to make the determination and it's a pain to restrict it at the level of ArrowFunction (although I do something like that for yield). I think this spec. complications reflect a user perspective that it is a nanny restriction. We're basically saying: you might not understand the semantics of arguments within an arrow (whatever we decided it was) so we won't let you use it at all.
Personally, I prefer Jason's perspective. Arrows don't have their own arguments object and arguments just lexically binds.
Allen
Allen Wirfs-Brock wrote:
We previously explored having lambdas expressions that included support abnormal exits, including method level returns. I even did most of the spec. work that would be needed to define them. But the ultimate consensus was formed around arrow functions without such control escapes.
And the reason (just to catch people up) was that JS has statements, the C curse, and so it's too late to graft full TCP on. In arrow function bodies, we have as much as TCP we can mix in at this late date, namely, in expressions: this, super, I expect arguments.
Separately, for the list: are arrow-functions lexically transparent to super and arguments? I hope so! For this kind of desugaring, if nothing else.
super is lexically resolved through the defining method so, just like this, it isn't rebound within an arrow function. Follow the spec. logic starting in 11.2.4.
Cool.
I see that I have a TODO note in 10.5.3 that says "don't create an arguments binding for arrow functions". However, I'm not sure that we actually had consensus on that point.
We never had an alternative that I know of. The wiki went further and said 'arguments' is an error; this was based on meeting feedback.
I'm with you on in thinking that arrows shouldn't rebind arguments but I think there may have been some push back on that.
No one wants arguments in arrows.
The question is, should an outer arguments binding be visible? I think so, now that Jason raises the question.
Mike Stay wrote:
Is the ES6 spec too far along to introduce some other construct that would desugar to the above, or is there still time to create a proposal?
Way late.
What's wrong with the iteration protocol?
Again, we're not doing control effects the way you might want given a clean slate.
Allen Wirfs-Brock wrote:
Personally, I prefer Jason's perspective. Arrows don't have their own arguments object and "arguments" just lexically binds.
+1000.
On Fri, May 10, 2013 at 1:07 PM, Brendan Eich <brendan at mozilla.com> wrote:
Mike Stay wrote:
Is the ES6 spec too far along to introduce some other construct that would desugar to the above, or is there still time to create a proposal?
Way late.
Oh, well.
What's wrong with the iteration protocol?
Nothing at all for arrays. For every other monad, there's no sugar: the only monad multiplication that's supported is list concatenation.
Again, we're not doing control effects the way you might want given a clean slate.
Sure, but array.map has never been pure. Anyone who wants purity won't be using JavaScript. I was just hoping that since we have special syntax for one monad, we could use it for all of them.
Mike Stay wrote:
On Fri, May 10, 2013 at 1:07 PM, Brendan Eich<brendan at mozilla.com> wrote:
What's wrong with the iteration protocol?
Nothing at all for arrays. For every other monad, there's no sugar: the only monad multiplication that's supported is list concatenation.
Iteration is much more general than "arrays".
Can you give an example (with the latest LRT syntax, and arrows) that shows what is lost?
Monads, I know 'em, but they remind me of these interminable Future/Promise threads, and then I want to lie down till the feeling goes away. :-/
Again, we're not doing control effects the way you might want given a clean slate.
Sure, but array.map has never been pure. Anyone who wants purity won't be using JavaScript. I was just hoping that since we have special syntax for one monad, we could use it for all of them.
What special syntax for what monad?
Monads with pretend purity are nice until runtime bugs bite.
Brendan Eich wrote:
Mike Stay wrote:
On Fri, May 10, 2013 at 1:07 PM, Brendan Eich<brendan at mozilla.com>
wrote:What's wrong with the iteration protocol?
Nothing at all for arrays. For every other monad, there's no sugar: the only monad multiplication that's supported is list concatenation.
Iteration is much more general than "arrays".
Oh, sorry -- you mean comprehensions (not the iteration protocol, which is used by all for-of syntax) are only about creating fresh arrays.
Yes, that's true. One alternative is a generator expression, which doesn't build any eager data structure, rather iterates and yields values on demand.
As Jason pointed out, arrows sweeten the explicit monadic patterns, e.g. monadic Promises. To close this thread it would be helpful to see an example where you use all the ES6 sugar, and then say why you'd still want another special form.
Apologies again, I'm a beat or three behind today. I think you gave the example already:
expr1.flatMap(x =>
expr2(x).flatMap(y => ...
exprN(x, y, ...).map(z =>
result(x, y, ..., z)
)
)
)
Nice when Scala and JS look alike!
I don't think we'd make comprehensions return a monad, specifically a Promise, even if we had the silly things (Promises, I mean -- just teasing about monads) in the language.
JS is eager and mutational. Although Mark and I have discussed privately trying to get E-like "featureless" promises, which become their fulfillment value, we think it's too big a stretch. Especially in light of existing libraries, but even starting fresh.
Without E-like featureless/"becoming" promises, I wouldn't foist them on syntax for eager computations such as comprehensions and generator expressions.
So is the explicit functional style, with arrows for sugar, really too much?
On Fri, May 10, 2013 at 12:59 PM, Brendan Eich <brendan at mozilla.com> wrote:
So is the explicit functional style, with arrows for sugar, really too much?
It was too much for Haskellers, who have a similarly light (or even lighter) lambda syntax. ^_^ That rightward march and the leftward paren tail is a bit frustrating.
But given the primacy of iterables in programming, and the existing Python legacy, I'm all for keeping the comprehension syntax focused on iterables. We can always add do-expressions later, after all, and maybe even sugar the lifting operation as cleanly as Haskell does.
To make this a little more concrete, just for fun...
Suppose I make an object representing "all future clicks" and give it a
map() method. This much, of course, is possible already:
var clicks = {
map: function (f) {
document.addEventListener("click", f);
}
};
With Mike's proposal, you could write:
for (let e of clicks) {
alert(e.clientX + ", " + e.clientY);
}
This would desugar to:
clicks.map(e => {
alert(e.clientX + ", " + e.clientY);
});
And a listener would be attached to the document. The body of the loop would run each time the user clicked.
Iterators don't do this.
Jason Orendorff wrote:
To make this a little more concrete, just for fun...
Suppose I make an object representing "all future clicks" and give it a map() method. This much, of course, is possible already:
var clicks = { map: function (f) { document.addEventListener("click", f); } };With Mike's proposal, you could write:
for (let e of clicks) { alert(e.clientX + ", " + e.clientY); }
So, just to be clear, now the issue isn't that comprehensions make arrays, it's that iteration isn't X. What is X? Quiz rule: don't say "Monad" or "Monadic" in your answer. :-|
This would desugar to:
clicks.map(e => { alert(e.clientX + ", " + e.clientY); });
I don't see why for-of doing magic flatMap*;map is better than this.
And a listener would be attached to the document. The body of the loop would run each time the user clicked.
That's just weird!
Iterators don't do this.
And rightly so!
I see that I have a TODO note in 10.5.3 that says "don't create an
argumentsbinding for arrow functions". However, I'm not sure that we actually had consensus on that point.We never had an alternative that I know of. The wiki went further and said
argumentsis an error; this was based on meeting feedback.I'm with you on in thinking that arrows shouldn't rebind
argumentsbut I think there may have been some push back on that.No one wants arguments in arrows.
The question is, should an outer arguments binding be visible? I think so, now that Jason raises the question.
This is a different position from 1, isn't it? The notes from January 2 as well as 3 might be of interest, too.
André Bargull wrote:
/ I see that I have a TODO note in 10.5.3 that says "don't create an />/ arguments binding for arrow functions". However, I'm not sure that we />/ actually had consensus on that point. / We never had an alternative that I know of. The wiki went further and said 'arguments' is an error; this was based on meeting feedback.
/ I'm with you on in thinking that arrows shouldn't rebind arguments />/ but I think there may have been some push back on that. / No one wants arguments in arrows.
The question is, should an outer arguments binding be visible? I think so, now that Jason raises the question.
This is a different position from [1], isn't it? The notes from January [2] as well as [3] might be of interest, too.
Thanks for the reminder -- I should have remembered this.
My appeal to arguments.callee was kind of a torture-test for why arrow bodies should not be strict by fiat. But it shouldn't prevail if we think "expression TCP" is more valuable. I'd be interested in Jason's thoughts here.
I keep forgetting to hit "Reply to all", sorry.
On Fri, May 10, 2013 at 2:34 PM, Brendan Eich <brendan at mozilla.com> wrote:
So, just to be clear, now the issue isn't that comprehensions make arrays, it's that iteration isn't X. What is X? Quiz rule: don't say "Monad" or "Monadic" in your answer. :-|
Actually, if we could provide a different flatten operation, that would suffice, since the iterator could simply provide a single value of the appropriate class. For example, given an iterator that produced integers, we could do matrix multiplication if flatten summed all the elements of the inner lists instead of concatenating the inner lists.
If [for (x of foo) for (y of bar(x)) for (z of baz(x,y)) expr(x, y, z)] iterated as usual over foo, bar(x) and baz(x,y) but used the flatten() method of bar(x) on the innermost list of lists and the flatten() method of foo on the outermost list of lists, that would do everything I was hoping for.
In particular, the expression
[ for (w of obj) for (x of foo(w)) for (y of bar(w,x)) for (z of baz(w,x,y)) if (cond(w,x,y,z)) expr(w,x,y,z) ]
would translate as
let resultObj = []; let onceObj = obj; for (let w of onceObj) { let resultFoo = []; let onceFoo = foo(w); for (let x of onceFoo) { let resultBar = []; let onceBar = bar(w,x); for (let y of onceBar) { let resultBaz = []; let onceBaz = baz(w,x,y); for (let z of onceBaz) { if (cond(w,x,y,z)) { resultBaz.push(expr(w,x,y,z)); } } resultBar.push(resultBaz); } resultBar = (onceBar.flatten || [].flatten).call(resultBar); resultFoo.push(resultBar); } resultFoo = (onceFoo.flatten || [].flatten).call(resultFoo); resultObj.push(resultFoo); } resultObj = (onceObj.flatten || [].flatten).call(resultObj);
No change to the syntax, no change to the iterator semantics, and a compatible change to the flattening semantics (assuming the existence of Array.prototype.flatten).
P.S. For those who do like monads, instead of binding functions of type A->TB and B->TC, we'd just be binding functions of type A->[TB]
and B->[TC].
Mike Stay wrote:
No change to the syntax, no change to the iterator semantics, and a compatible change to the flattening semantics (assuming the existence of Array.prototype.flatten).
The nested arrays cost, and no one wants 'em for the single [] comprehension case. If you want a tree of arrays, you can nest comprehensions. Therefore this will be an optimization burden for no gain in the common case.
What's wrong with functional style when you mean it? Comprehensions are about sugaring the common case.
In addition to what Brendan said about the cost of actually generating nested arrays in the common case, I don't think this desugaring does what you want - if you have only a single for-of, you'll just get an array back out, without a chance to call the custom flatten on it. You can't handle this generically, either, because the innermost for-of is handled as a map rather than a flatMap. You really do need to go monadic all the way, and as Brendan said, that's too much cost for the common case of Arrays.
Let's just add do-expressions, or something similar, in the future. We can leave list comprehensions to their array specialization for now.
On Sat, May 11, 2013 at 11:34 PM, Tab Atkins Jr. <jackalmage at gmail.com> wrote:
In addition to what Brendan said about the cost of actually generating nested arrays in the common case, I don't think this desugaring does what you want - if you have only a single for-of, you'll just get an array back out, without a chance to call the custom flatten on it.
You can't handle this generically, either, because the innermost for-of is handled as a map rather than a flatMap.
That's intentional. As I wrote, for-of would bind functions of the form A->[TB], not A->TB.
You really do need to go monadic all the way, and as Brendan said, that's too much cost for the common case of Arrays.
There's a way to accomplish it that wouldn't require creating new arrays, but the flatten operation becomes far less intuitive and loses a lot of the attractiveness of a special syntax. It's essentially inserting a "reified comma" for each loop.
Let's just add do-expressions, or something similar, in the future. We can leave list comprehensions to their array specialization for now.
OK.
On Sat, May 11, 2013 at 9:04 AM, Brendan Eich <brendan at mozilla.com> wrote:
André Bargull wrote:
No one wants arguments in arrows.
The question is, should an outer arguments binding be visible? I think so, now that Jason raises the question.
This is a different position from [1], isn't it? The notes from January [2] as well as [3] might be of interest, too.
Thanks for the reminder -- I should have remembered this.
My appeal to arguments.callee was kind of a torture-test for why arrow bodies should not be strict by fiat. But it shouldn't prevail if we think "expression TCP" is more valuable. I'd be interested in Jason's thoughts here.
No strong opinion. "Expression TCP" is nice, but not very valuable.
Assuming we do not want arguments bindings in arrows, I guess using
arguments in an arrow is almost certainly a mistake and should be a
SyntaxError.
We shouldn't be motivated either way by a desire to support arguments.callee!
Jason Orendorff wrote:
On Sat, May 11, 2013 at 9:04 AM, Brendan Eich <brendan at mozilla.com <mailto:brendan at mozilla.com>> wrote:
André Bargull wrote: No one wants arguments in arrows. The question is, should an outer arguments binding be visible? I think so, now that Jason raises the question. This is a different position from [1], isn't it? The notes from January [2] as well as [3] might be of interest, too. Thanks for the reminder -- I should have remembered this. My appeal to arguments.callee was kind of a torture-test for why arrow bodies should not be strict by fiat. But it shouldn't prevail if we think "expression TCP" is more valuable. I'd be interested in Jason's thoughts here.No strong opinion. "Expression TCP" is nice, but not very valuable.
Nevertheless, that was where TC39 ended up. It was a good conclusion, since trying for "statement TCP" was threatening to suck up time and lose the consensus I forged for arrows.
Assuming we do not want
argumentsbindings in arrows,
I sense we do not want, but we'll discuss it at next week's meeting, I'm sure!
I guess using arguments in an arrow is almost certainly a mistake and should be a SyntaxError.
That's the wiki'ed proposal ;-).
We shouldn't be motivated either way by a desire to support arguments.callee!
I know, I just used it as a stress-test. And then promptly forgot about it!
Just confirming: SpiderMonkey prototypes ES6 arrow function syntax without any arguments binding in the arrow body expression or body block:
js> f = (x) => arguments[0] (x) => arguments[0]
js> f(42)
js> g = (x) => { return arguments[0]; } (x) => { return arguments[0]; }
js> g(43)
js> arguments[0]
js> arguments[0] = 44
44 js> f(45)
44
(Onlookers not in the know: the js shell has a global arguments object capturing command-line arguments, which is an empty array in this cited session.)
My name is Mike Stay; I was a principal developer on Caja and worked with Mark Miller on designing the security features that he worked to get into ES5. I've looked through the archive for stuff on array comprehensions and didn't see any discussion about the following point, but I could easily have missed it.
In Scala, "for comprehensions" look roughly like this:
which in ES harmony would look like
[result(x, y, ..., z) for (x of expr1) for (y of expr2(x)) ... for (z of exprN(x, y, ...))]In Scala, this is desugared into
[Note the final method is map(), not flatMap().]
For arrays, flatMap is map followed by flatten, which concatenates a list of lists into a single list. This is a relatively simple transformation that would grant a lot of power, while not changing anything (so far as I can tell) about the current semantics. It enables a programmer to use the same syntax for any monad, so pick your favorite: promises, membranes, parsers, chaining UI actions, etc.
For example,
[JSON.parse(x) for (x of fetch("http://example.com/myData"))]could be a promise for the result of an asynchronous fetch, much like
Q.when().Chaining user events is easy:
[ menuItem.display() for (pos of MouseClick()) for (menuItem of MenuItem(pos)) ]etc.
My name is Mike Stay; I was a principal developer on Caja and worked with Mark Miller on designing the security features that he worked to get into ES5. I've looked through the archive for stuff on array comprehensions and didn't see any discussion about the following point, but I could easily have missed it. In Scala, "for comprehensions" look roughly like this: for ( x <- expr1; y <- expr2(x); ... ; z <- exprN(x, y, ...) ) { yield (result(x, y, ..., z)) } which in ES harmony would look like [result(x, y, ..., z) for (x of expr1) for (y of expr2(x)) ... for (z of exprN(x, y, ...))] In Scala, this is desugared into expr1.flatMap(x => expr2(x).flatMap(y => ... exprN(x, y, ...).map(z => result(x, y, ..., z) ) ) ) [Note the final method is map(), not flatMap().] For arrays, flatMap is map followed by flatten, which concatenates a list of lists into a single list. This is a relatively simple transformation that would grant a lot of power, while not changing anything (so far as I can tell) about the current semantics. It enables a programmer to use the same syntax for any monad, so pick your favorite: promises, membranes, parsers, chaining UI actions, etc. For example, [JSON.parse(x) for (x of fetch("http://example.com/myData"))] could be a promise for the result of an asynchronous fetch, much like Q.when(). Chaining user events is easy: [ menuItem.display() for (pos of MouseClick()) for (menuItem of MenuItem(pos)) ] etc. -- Mike Stay - metaweta at gmail.com http://www.cs.auckland.ac.nz/~mike http://reperiendi.wordpress.com