yield syntax (diverging from: How would shallow generators compose with lambda?)
On May 14, 2009, at 1:14 PM, Neil Mix wrote:
On May 14, 2009, at 2:38 PM, Brendan Eich wrote:
Indeed the low-precedence (same as assignment) nature of the yield
unary caused JS1.7 to follow Python in requiring parentheses around
yield expressions in lists: in comma expressions and actual
parameter lists (although we diverged from Python by allowing an
unparenthesized yield expression as the last actual argument in a
call or new expression's parameter list -- Python requires
foo((yield bar)) instead of foo(yield bar)).I have this idea that it would be better for yield expressions to
look like function calls: 'yield' '(' expression? ')'. (Note that
this is only a syntactical suggestion; clearly an implementation
wouldn't actually treat it like a function.) That would eliminate
the precedence issues Brendan cites while also making the syntax
backward compatible with earlier ES parsers. Is there any technical
reason why that wouldn't be possible?
The syntax could work but we need to reserve yield contextually. It
can't be a user-defined function name and a built-in function. The
compiler must unambiguously know that yield (the built-in) is being
called in order to make the enclosing function a generator.
This is reason enough in my view to keep yield a prefix operator and
reserve it. I wish I had done that with eval in JS1.0!
Another reason is your duck/cow point, which I think is a separate
point from compiler analyzability. Really, no one writes yield(...) in
Python, and extra parens hurt (I know RSI sufferers who benefit from
lack of shifting in Python and Ruby).
Now you could argue that we have eval already, and a cheap-ish way to
detect direct calls to it (see esdiscuss/2009-January/008656)
, so why not add something like that again? One reason is that the
cost is not zero -- cheap-ish, not cheap and not free. Adding more
distinguished special-form instructions that fall back on call
behavior if the callee is not the magic built-in will annoy
implementors.
But you do make a good no-new-syntax case, ignoring the above
considerations. I still don't buy it, given let not being doable the
same way. We need new keywords and special forms (not many, but it's
absurd to freeze the set from ES3 in 1999).
Brendan Eich wrote:
On May 14, 2009, at 1:14 PM, Neil Mix wrote:
I have this idea that it would be better for yield expressions to look like function calls: 'yield' '(' expression? ')'. (Note that this is only a syntactical suggestion; clearly an implementation wouldn't actually treat it like a function.) That would eliminate the precedence issues Brendan cites while also making the syntax backward compatible with earlier ES parsers. Is there any technical reason why that wouldn't be possible?
The syntax could work but we need to reserve yield contextually. It can't be a user-defined function name and a built-in function. The compiler must unambiguously know that yield (the built-in) is being called in order to make the enclosing function a generator.
This is reason enough in my view to keep yield a prefix operator and reserve it.
But that doesn't help: the argument to yield is an arbitrary expression, so 'yield (foo)' could be either a function call or a yield-expression. That means that this approach can at best be no simpler to implement or specify than the function call syntax.
With the function call syntax, it would be sufficient to keep the existing ES5 grammar for function calls, and then check after parsing whether a MemberExpression or CallExpression followed by Arguments is the string "yield". With the operator syntax, it's more complicated than that because there are more syntactic contexts to consider.
Another reason is your duck/cow point, which I think is a separate point from compiler analyzability. Really, no one writes yield(...) in Python, and extra parens hurt (I know RSI sufferers who benefit from lack of shifting in Python and Ruby).
Yes, those are separate points that I am not arguing against here.
On May 14, 2009, at 5:13 PM, David-Sarah Hopwood wrote:
Brendan Eich wrote:
On May 14, 2009, at 1:14 PM, Neil Mix wrote:
I have this idea that it would be better for yield expressions to
look like function calls: 'yield' '(' expression? ')'. (Note that this
is only a syntactical suggestion; clearly an implementation wouldn't actually treat it like a function.) That would eliminate the precedence issues Brendan cites while also making the syntax
backward compatible with earlier ES parsers. Is there any technical reason
why that wouldn't be possible?The syntax could work but we need to reserve yield contextually. It can't be a user-defined function name and a built-in function. The compiler must unambiguously know that yield (the built-in) is being called in order to make the enclosing function a generator.
This is reason enough in my view to keep yield a prefix operator and reserve it.
But that doesn't help: the argument to yield is an arbitrary
expression, so 'yield (foo)' could be either a function call or a yield- expression. That means that this approach can at best be no simpler to implement
or specify than the function call syntax.
Were you responding to Neil instead of me? I'm not advocating Neil's
proposal, but it seems to me he's arguing for it to avoid the
mandatory parentheses around the entire yield expression in almost any
surrounding expression in which it could be embedded. He is right that
requiring parentheses around yield's operand avoids mandatory parens
around yield expressions in list contexts.
Simpler to implement is not the issue, but it's a wash as you say.
With the function call syntax, it would be sufficient to keep the existing ES5 grammar for function calls, and then check after parsing whether a MemberExpression or CallExpression followed by Arguments is the string "yield". With the operator syntax, it's more complicated than that because there are more syntactic contexts to consider.
Yes, but it's not that complicated. SpiderMonkey and Rhino do it. Code
size burden is in the noise.
Another reason is your duck/cow point, which I think is a separate
point from compiler analyzability. Really, no one writes yield(...) in
Python, and extra parens hurt (I know RSI sufferers who benefit from lack of shifting in Python and Ruby).Yes, those are separate points that I am not arguing against here.
Ok.
I was replying to Neil, not to you (hadn't seen any message from you
on this sub-thread).
Brendan Eich wrote:
On May 14, 2009, at 5:13 PM, David-Sarah Hopwood wrote:
Brendan Eich wrote:
On May 14, 2009, at 1:14 PM, Neil Mix wrote:
I have this idea that it would be better for yield expressions to look like function calls: 'yield' '(' expression? ')'. [...]
The syntax could work but we need to reserve yield contextually. It can't be a user-defined function name and a built-in function. The compiler must unambiguously know that yield (the built-in) is being called in order to make the enclosing function a generator.
This is reason enough in my view to keep yield a prefix operator and reserve it.
But that doesn't help: the argument to yield is an arbitrary expression, so 'yield (foo)' could be either a function call or a yield-expression. That means that this approach can at best be no simpler to implement or specify than the function call syntax.
Were you responding to Neil instead of me?
I was responding to "This is reason enough in my view to keep yield a prefix operator and reserve it".
My point was that the example of 'yield (foo)' (that is, yield as a prefix operator applied to the expression '(foo)') shows that the prefix operator syntax cannot possibly be easier to specify than the function call syntax -- contrary to what you appeared to be arguing above. In fact I think it is much harder to do correctly.
With the function call syntax, it would be sufficient to keep the existing ES5 grammar for function calls, and then check after parsing whether a MemberExpression or CallExpression followed by Arguments is the string "yield". With the operator syntax, it's more complicated than that because there are more syntactic contexts to consider.
Yes, but it's not that complicated. SpiderMonkey and Rhino do it. Code size burden is in the noise.
Hmm. SpiderMonkey and Rhino use ad-hoc parsers. Show me an unambiguous grammar for the prefix yield operator, and then I'll concede the point :-)
This may be breaking a butterfly on a wheel, but I am game if it
improves the state of the strawman proposals.
On May 15, 2009, at 7:16 PM, David-Sarah Hopwood wrote:
My point was that the example of 'yield (foo)' (that is, yield as a prefix operator applied to the expression '(foo)') shows that the
prefix operator syntax cannot possibly be easier to specify than the function call syntax -- contrary to what you appeared to be arguing above.
Analogous to direct vs. indirect eval in ES5 (15.1.2.1.1), there is no
purely syntactic specification for what Neil proposes. A runtime check
is required. So I don't see why you are focusing only on syntax here.
All the costs count, but considering "who pays" can trump "how much";
see the postscript below. I'm more concerned with usability than
implementability, but the latter is not hard and there's no soundness
issue.
In fact I think it is much harder to do correctly.
See below, it's quite easy.
Yes, but it's not that complicated. SpiderMonkey and Rhino do it.
Code size burden is in the noise.Hmm. SpiderMonkey and Rhino use ad-hoc parsers. Show me an unambiguous grammar for the prefix yield operator, and then I'll concede the
point :-)
Python 2.5 has essentially the same issues, and it has a formal
grammar (several, the one CPython uses and at least one other at
antlr.org). From CPython's Grammar/Grammar EBNF:
$ grep yield Grammar
expr_stmt: testlist (augassign (yield_expr|testlist) |
('=' (yield_expr|testlist))*)
flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt |
yield_stmt
yield_stmt: yield_expr
atom: ('(' [yield_expr|testlist_gexp] ')' |
yield_expr: 'yield' [testlist]
What we are discussing is almost trivial in terms of usability, but
the difference is big enough to care about, IMHO. Either you have to
write, in Neil's approach:
yield (E) (or just yield() to yield undefined)
or in the Pythonic approach:
(yield E) (or just (yield))
with the parentheses in the Pythonic approach not required when the
yield expression is either (a) the entire expression in an expression
statement; (b) the right-hand side of an assignment.
Python differs from JS by having only assignment statements, not
assignment expressions, which simplifies the problem a bit, as the
grep output cited above shows.
In any case, yield E; as a statement is the most common kind of yield
in real-world Python code, both because it came first and because
sending values to generators is less commonly done than iterating
generators.
So with either approach you have some "insignificant, silly
parentheses" -- but with Neil's approach you have "lots" more because
the expression-statement yield must have at least () after the keyord.
This tips the usability contest in favor of the Pythonic approach.
In no case are there ambiguities for a bottom up grammar. The right
parenthesis avoids nasty automatic semicolon insertion problems that
plagued the unbracketed low-precedence expression on the right of a
high-precedence prefix in the case of expression-bodied functions, AKA
"expression closures" (see esdiscuss/2008-October/007888)
.
To confirm this, I patched WebKit's Bison grammar for JS to support
yield //a la// JS1.7. I didn't hook up the semantic actions, just
verified zero reduce-reduce conflicts. Patch attached below --
comments welcome (including from you WebKit lurkers :-).
/be
P.S. Bean-counting productions in a bottom-up grammar is counting the
wrong cost, given the few implementors and their greater skills,
compared to the many users who need to be spared those painful and
fruitless parentheses.
Bean-counting productions in a traditional bottom-up grammar is also
silly given how many are required for ES1-3. From the patch, notice
all the existing specialized NoIn/NoBF/NoNode variants in parser/
Grammar.y (NoBF = No Brace at Front, as far as I can tell).
Much of that production-forking could be avoided by using either
parameterized LR parsing (www.cs.lth.se/Research/LDTA2004/d09_TheimannNeubauer.pdf)
, or else top-down ("ad-hoc" or not :-P) parsing.
On May 17, 2009, at 11:00 AM, Brendan Eich wrote:
Python differs from JS by having only assignment statements, not
assignment expressions, which simplifies the problem a bit, as the
grep output cited above shows.
Of course this makes trouble with automatic semicolon insertion,
unless assignment statements are split from assignment expressions.
D'oh!
js> function g() {z = yield a ? f : x++()}
typein:2: SyntaxError: missing ; before statement: typein:2: function g() {z = yield a ? f : x++()} typein:2: .................................^ js> function g() {(z = yield a ? f : x++)()}
To confirm this, I patched WebKit's Bison grammar for JS to support
yield //a la// JS1.7. I didn't hook up the semantic actions, just
verified zero reduce-reduce conflicts. Patch attached below --
comments welcome (including from you WebKit lurkers :-).
Patching this is harder than I thought it would be. After hacking at
the Bison grammar a bit, I concede your point about implementation
difficulty!
Usability still favors the Python approach, at least for yield
statements.
On May 17, 2009, at 11:00 AM, Brendan Eich wrote:
Bean-counting productions in a traditional bottom-up grammar is also
silly given how many are required for ES1-3. From the patch, notice
all the existing specialized NoIn/NoBF/NoNode variants in parser/ Grammar.y (NoBF = No Brace at Front, as far as I can tell).
Correction:
%type <expressionNode> MemberExpr MemberExprNoBF /* BF => brace or
function */
On Sun, May 17, 2009 at 11:00 AM, Brendan Eich <brendan at mozilla.com> wrote:
Analogous to direct vs. indirect eval in ES5 (15.1.2.1.1), there is no purely syntactic specification for what Neil proposes. A runtime check is required. So I don't see why you are focusing only on syntax here.
I don't follow. What runtime check? For the eval operator, the runtime check is whether the value of the eval variable is the original global eval function. It makes no sense to have a corresponding global yield function value.
On May 17, 2009, at 12:43 PM, Mark S. Miller wrote:
On Sun, May 17, 2009 at 11:00 AM, Brendan Eich <brendan at mozilla.com>
wrote:Analogous to direct vs. indirect eval in ES5 (15.1.2.1.1), there is
no purely syntactic specification for what Neil proposes. A runtime
check is required. So I don't see why you are focusing only on syntax here.I don't follow. What runtime check? For the eval operator, the runtime check is whether the value of the eval variable is the original global eval function. It makes no sense to have a corresponding global yield function value.
If we reserve yield then you're right. One of the appealing (at least
to me) aspects of Neil's suggestion was that it would avoid opt-in
versioning required by reserving yield (which is used in extant web
content, or was when we tried reserving it without opt-in versioning
-- the particular use was as a formal parameter name, used as a flag
not a function).
On May 17, 2009, at 1:11 PM, Brendan Eich wrote:
On May 17, 2009, at 12:43 PM, Mark S. Miller wrote:
On Sun, May 17, 2009 at 11:00 AM, Brendan Eich
<brendan at mozilla.com> wrote:Analogous to direct vs. indirect eval in ES5 (15.1.2.1.1), there
is no purely syntactic specification for what Neil proposes. A runtime
check is required. So I don't see why you are focusing only on syntax here.I don't follow. What runtime check? For the eval operator, the
runtime check is whether the value of the eval variable is the original
global eval function. It makes no sense to have a corresponding global yield function value.If we reserve yield then you're right.
It's goofy to have
return w; delete x.y; typeof z; yield a; => SyntaxError! must use yield(a)
The mandatory parentheses could be avoided by breaking from Python's
precedent and making yield a canonical unary (that is,
high-)precedence operator like delete, !, etc. But then almost any
algebraic or logical expression computing the value to yield would
need parentheses, and people would make mistakes such as yield a + b
where they meant yield(a + b) -- as in Python -- but got yield(a) + b.
This is one reason, if I recall some PEP text correctly, why Python
requires parentheses around all yield expressions except those on the
right of assignment operators in assignment statements, and in yield
statements. It seems best to require explicit parentheses when yield
is part of a larger expression, no matter the precedence of its
optional argument/operand.
But when there's no larger expression, why require gratuitous
parentheses?
On May 17, 2009, at 7:01 PM, Brendan Eich wrote:
The mandatory parentheses could be avoided by breaking from Python's
precedent and making yield a canonical unary (that is,
high-)precedence operator like delete, !, etc. But then almost any
algebraic or logical expression computing the value to yield would
need parentheses, and people would make mistakes such as yield a + b
where they meant yield(a + b) -- as in Python -- but got yield(a) + b.
I'm going to make the argument that this is about where the
parenthesis go -- not if -- but where.
- we could always allow parenthesis to be dropped when the yield is
the entire expression of an expression statement or the right-hand
side of an assignment. - in my experience with JS 1.7 I almost always had to parenthesize the
yield expression when it was in some other kind of expression. An in
the cases where parenthesis weren't required, I parenthesized anyway
to avoid ambiguity and maintain coding style consistency. (And
because I got tired of predicting incorrectly whether or not parens
would be required in a particular context.)
So I would argue that there are two syntactical forms of yield, yield
E and (yield E), and that the rules regarding the requirement for
parenthesis are hard to predict (from personal experience).
Therefore, I argue that it would make sense to simplify a bit:
- the yield E form may be used when it is the entire expression of an
expression statement - all other times it must be parenthesized
Which is kind of a way of saying, if you're ignoring the send value,
you don't have to parenthesize. But if you use the send value, you
must parenthesize.
And now that we've made clear the definition of parenthesized and non-
parenthesized forms of yield, we can proceed to argue that yield(E) is
a valid form of parenthesis, as much so as (yield E).
Pros for yield(E):
- backward compatible
- easier to read (to my eye)
- it "feels" more correct to me in context of the when-using-send- value rule
Pros for (yield E):
- consistent with python
- doesn't present any is-it-a-function? ambiguities
As a follow on to my argument I would cite two examples for why it
would be good to always require parenthesis when the send value is used:
Example 1 I saw somewhere else in this thread:
yield a ? b : c
Is it (yield a) ? b : c | yield(a) ? b : c, or (yield a ? b : c) | yield(a ? b : c)
example 2:
let x = yield-5;
is that let x = (yield) - 5; | let x = yield() - 5; or is it let x = (yield -5); | let x = yield(-5);
I know the rules for both examples are very clear, but programmers
don't always read the specs. :P To someone not intimately familiar
with JS order of precedence, either one presents ambiguity (or worse,
doesn't!) Why play with fire?
On May 17, 2009, at 11:48 AM, Brendan Eich wrote:
On May 17, 2009, at 11:00 AM, Brendan Eich wrote:
Python differs from JS by having only assignment statements, not
assignment expressions, which simplifies the problem a bit, as the
grep output cited above shows.Of course this makes trouble with automatic semicolon insertion,
unless assignment statements are split from assignment expressions.
D'oh!js> function g() {z = yield a ? f : x++()} typein:2: SyntaxError: missing ; before statement: typein:2: function g() {z = yield a ? f : x++()} typein:2: .................................^ js> function g() {(z = yield a ? f : x++)()}
I was wrong, there's no issue here (apart from SpiderMonkey's
diagnostic being suboptimal). This is not a case of a high-precedence
prefix operator whose following operand is a low-precedence expression
non-terminal. That's unsound.
In this example, yield is just like a nested assignment:
js> function g() {z = w = a ? f : x++()}
typein:1: SyntaxError: missing ; before statement: typein:1: function g() {z = w = a ? f : x++()} typein:1: ...............................^
where the body (z = w = a ? f : x++()) === (z = (w = (a ? f : x++()))).
If PrimaryExpression: yield AssignmentExpression were a production in
the grammar, then we would have the bug I was worried about, the one
that Waldemar pointed out let expressions, which also afflicts
expression closures (ASI makes the bug harder to spot, was Waldemar's
further point -- but ASI is not an issue here).
Patching this is harder than I thought it would be. After hacking at
the Bison grammar a bit, I concede your point about implementation
difficulty!
I still concede greater burden on implementor, but here's an updated
WebKit/JavaScriptCore/parser patch:
On May 17, 2009, at 6:39 PM, Neil Mix wrote:
On May 17, 2009, at 7:01 PM, Brendan Eich wrote:
The mandatory parentheses could be avoided by breaking from
Python's precedent and making yield a canonical unary (that is,
high-)precedence operator like delete, !, etc. But then almost any
algebraic or logical expression computing the value to yield would
need parentheses, and people would make mistakes such as yield a +
b where they meant yield(a + b) -- as in Python -- but got yield(a)
- b.
I'm going to make the argument that this is about where the
parenthesis go -- not if -- but where.
Yes, this is the issue.
- we could always allow parenthesis to be dropped when the yield is
the entire expression of an expression statement or the right-hand
side of an assignment.
Right-hand side of assignment is ok without parens in Python because
assignment is a statement.
In JS if you allow assignment expressions ending in unparenthesized
yields, then you can have unparenthesized yields in argument and
initialiser lists, comma expressions, and in the middle and final
operand positions in ternary (?:) expressions.
- in my experience with JS 1.7 I almost always had to parenthesize
the yield expression when it was in some other kind of expression.
An in the cases where parenthesis weren't required, I parenthesized
anyway to avoid ambiguity and maintain coding style consistency.
(And because I got tired of predicting incorrectly whether or not
parens would be required in a particular context.)
The only contexts we allow you not to parenthesize in JS1.7 are
assignment expressions and final argument in list. But see above --
the assignment expression loophole is big enough to allow
foo(a = yield b, c);
One argument, or two?
Comma is low enough precedence that users (with or without Python
exposure) don't view it as an operator.
So I would argue that there are two syntactical forms of yield,
yield E and (yield E), and that the rules regarding the requirement
for parenthesis are hard to predict (from personal experience).
Therefore, I argue that it would make sense to simplify a bit:
- the yield E form may be used when it is the entire expression of
an expression statement- all other times it must be parenthesized
Agreed; this closes the assignment expression loophole.
Which is kind of a way of saying, if you're ignoring the send
value, you don't have to parenthesize. But if you use the send
value, you must parenthesize.And now that we've made clear the definition of parenthesized and
non-parenthesized forms of yield, we can proceed to argue that
yield(E) is a valid form of parenthesis, as much so as (yield E).
Nothing prevents you from writing yield(E) of course -- but you're
arguing that foo(a = yield(b), c) should be enough, no extra parens
required -- no foo(a = (yield(b)), c). Right?
Pros for yield(E):
- backward compatible
But for this to be true, we would need to use the direct-eval
detection hack I mentioned previously.
- easier to read (to my eye)
- it "feels" more correct to me in context of the when-using-send- value rule
These are subjective enough there's no point in arguing. I hear ya.
Pros for (yield E):
- consistent with python
- doesn't present any is-it-a-function? ambiguities
These are more objective (no look & feel ;-). There is a borrowing
from Python. There isn't a function call going on.
2009/5/17 Brendan Eich <brendan at mozilla.com>:
One of the appealing (at least to me) aspects of Neil's suggestion was that it would avoid opt-in versioning required by reserving yield (which is used in extant web content, or was when we tried reserving it without opt-in versioning -- the particular use was as a formal parameter name, used as a flag not a function).
This yield-like-eval proposal also requires to have a special function to create a generator since yield (as implemented in SpiderMonkey and in Python) is used both to implement the yielding action and to indicate that the function is a generator.
The latter is necessary as a call to a function with yield somewhere in the body creates a generator, it does not execute one. For that reason the compiler must know what yield means, it can not wait for a runtime check.
The remedy for this is simple - the generator can be created using explicit call like Generator(f, arg1, ... argN). This would turn any function into a generator and would allow for runtime checks for eval. The plus side of this is that an empty generator can be created with a straightforward:
Generator(function() {})
and not with a rather unnatural
(function() { if (false) yield; })()
Igor
On 2009-05-17, at 14:00EDT, Brendan Eich wrote:
those painful and fruitless parentheses
Sometimes those parentheses are comforting, not painful. How many
times in JS do you see:
typeof(foo)
or in C.*
sizeof(boo)
On May 18, 2009, at 5:56 AM, P T Withington wrote:
On 2009-05-17, at 14:00EDT, Brendan Eich wrote:
those painful and fruitless parentheses
Sometimes those parentheses are comforting, not painful. How many
times in JS do you see:typeof(foo)
or in C.*
sizeof(boo)
Or even return(wahhh); -- a sign of weakness in the old days, among
kernel hackers :-P.
Making them mandatory is the issue. You can derive whatever comfort
you want from 'em, but they are not required and we're not going to
start requiring them for return, delete, or typeof. So mandating
parentheses for yield is kind of wrong. Obviously the issue would go
away if the precedence where the same as delete and typeof. Hence
"kind of".
On May 18, 2009, at 2:25 AM, Igor Bukanov wrote:
The remedy for this is simple - the generator can be created using explicit call like Generator(f, arg1, ... argN). This would turn any function into a generator and would allow for runtime checks for eval.
You mean yield, not eval, right?
The plus side of this is that an empty generator can be created with a straightforward:
Generator(function() {})
and not with a rather unnatural
(function() { if (false) yield; })()
No one makes empty generators.
I think this cure is worse than the disease! The problem of yield's
precedence can be solved by mandating parenthesization as Python does
-- or if ugliness prevails, as Neil proposes :-P. It doesn't need more
runtime magic machinery and global constructors that are otherwise
unmotivated.
Nothing prevents you from writing yield(E) of course -- but you're
arguing that foo(a = yield(b), c) should be enough, no extra parens
required -- no foo(a = (yield(b)), c). Right?
Yes that's correct.
Pros for yield(E):
- backward compatible
But for this to be true, we would need to use the direct-eval
detection hack I mentioned previously.
On the plus side, this would allow for feature detection of generator
support, right? (Is there any other way to detect generator support?)
- easier to read (to my eye)
- it "feels" more correct to me in context of the when-using-send- value rule
These are subjective enough there's no point in arguing. I hear ya.
Yes, one man's opinion and experience. If others chimed in claiming
the opposite experience, obviously my argument would be moot. But I
hope I'm providing a decent "average hacker" POV for you.
Pros for (yield E):
- consistent with python
- doesn't present any is-it-a-function? ambiguities
These are more objective (no look & feel ;-). There is a borrowing
from Python. There isn't a function call going on.
Agreed. If look-and-feel weren't factors, I wouldn't have brought it
up. ;)
2009/5/18 Brendan Eich <brendan at mozilla.com>:
On May 18, 2009, at 2:25 AM, Igor Bukanov wrote:
The remedy for this is simple - the generator can be created using explicit call like Generator(f, arg1, ... argN). This would turn any function into a generator and would allow for runtime checks for eval.
You mean yield, not eval, right?
Right, that was a typo.
The plus side of this is that an empty generator can be created with a straightforward:
Generator(function() {})
and not with a rather unnatural
(function() { if (false) yield; })()
No one makes empty generators.
For me the problem with the way the generators are defined is that a dead code like that "if (0) yield;" affects the semantic by mere presence of it. Surely, this is not the first feature in ES that has that property - "if (0) var a;" is another example. But "if (0) yield;" sets a new record affecting the nature of the whole function.
It doesn't need more runtime magic machinery and global constructors that are otherwise unmotivated.
That is true, but then runtime-only generators allows to avoid opt-in tax for all the users. How high is that tax I do not know, but surely it prevents some user cases for the feature.
Igor
On May 18, 2009, at 11:53 AM, Neil Mix wrote:
But for this to be true, we would need to use the direct-eval
detection hack I mentioned previously.On the plus side, this would allow for feature detection of
generator support, right? (Is there any other way to detect
generator support?)
In JS1.7 you could object-detect:
if (this.Iterator) ...
or similarly.
But I'm in the midst of writing up strawman:iterators proposal that
doesn't add Iterator to the global object (nor any iterator
getter, no double underscores).
One generalized idea for object detection of keywords that could be
used as if they named functions in code that would be work in old
browsers:
if (Object.implementation &&
Object.implementation.supports('yield'))) { ... function gen() { ... yield(E); .... } ... } else { ... do something else ... }
I'm abusing Object as ES5 does, but harder, to avoid polluting the
global object. I'm supposing Object.implementation could be useful for
other properties than the supports method. This reads well enough,
although its a bit long-winded altogether (but individual names are
short enough). Comments?
On May 18, 2009, at 12:23 PM, Brendan Eich wrote:
Making them mandatory is the issue. You can derive whatever comfort
you want from 'em, but they are not required and we're not going to
start requiring them for return, delete, or typeof. So mandating
parentheses for yield is kind of wrong. Obviously the issue would go
away if the precedence where the same as delete and typeof. Hence
"kind of".
Ah, but delete and typeof are different in that they both require an
argument (terminology be damned, hopefully you understand what I mean
by that.) And return is different because it can't be used in an
expression. See my previous let x = yield-5; example -- let x =
typeof-5; is unambiguous.
I'm tempted to argue that yield should require an argument, and then
you could give it precedence equivalent to delete and typeof. But
that would make a statement like
yield x - 1;
utterly confusing.
Set aside for a moment the argument of which form of parenthesis for
a moment, and assume (yield E) wins for now. Could we redefine yield
so that it has two forms, a statement form and an expression form?
The statement form would be
yield E;
where E is optional. And the expression form would be
yield E
where E is required, and yield has precedence equivalent to delete and
typeof. Is that possible? Doing so would render any argument about
parenthesis moot, since they'd rarely be required, right?
On May 18, 2009, at 12:15 PM, Neil Mix wrote:
Ah, but delete and typeof are different in that they both require
an argument (terminology be damned, hopefully you understand what I
mean by that.) And return is different because it can't be used in
an expression. See my previous let x = yield-5; example -- let x =
typeof-5; is unambiguous.I'm tempted to argue that yield should require an argument, and then
you could give it precedence equivalent to delete and typeof. But
that would make a statement likeyield x - 1;
utterly confusing.
Yes; I pointed that out earlier in this thread:
"But then almost any algebraic or logical expression computing the
value to yield would need parentheses, and people would make mistakes
such as yield a + b where they meant yield(a + b) -- as in Python --
but got yield(a) + b."
But this objection applies to yield expressions in larger expressions
too.
Set aside for a moment the argument of which form of parenthesis
for a moment, and assume (yield E) wins for now. Could we redefine
yield so that it has two forms, a statement form and an expression
form? The statement form would be yield E; where E is optional. And the expression form would be yield E where E is required, and yield has precedence equivalent to delete
and typeof. Is that possible? Doing so would render any argument
about parenthesis moot, since they'd rarely be required, right?
Not clear. Using yield in a larger expression may nevertheless want to
yield an expression with precedence lower than unary (delete, !, etc.)
precedence. ES3 grammar:
UnaryExpression : PostfixExpression delete UnaryExpression void UnaryExpression typeof UnaryExpression ++ UnaryExpression -- UnaryExpression + UnaryExpression - UnaryExpression ~ UnaryExpression ! UnaryExpression
Since yield has some some similarity to return I'm still in favor of
what Python did: low precedence, mandatory parenthesization of the
whole yield expression in any non-statement context. JS1.7 tried to
relax this, but it's not worth it as your experience says.
Brendan:
I'm abusing Object as ES5 does, but harder, to avoid polluting the global object. I'm supposing Object.implementation could be useful for other properties than the supports method. This reads well enough, although its a bit long-winded altogether (but individual names are short enough). Comments?
Have i got the wrong end of the stick on this... With ES6 i thought modules would provide some namespace functionality. eg
import meta // or whatever the syntax is meta.supports("yield")
And a home for any ast functionality: import ast let x = ast.parse("3 + 3")
It's interesting that Lua puts its yield/coroutine functionality in a module: coroutine.yield()
It's function format but at least it's not necessary to worry about the global namespace (with modules).
On May 14, 2009, at 2:38 PM, Brendan Eich wrote:
I have this idea that it would be better for yield expressions to look
like function calls: 'yield' '(' expression? ')'. (Note that this is
only a syntactical suggestion; clearly an implementation wouldn't
actually treat it like a function.) That would eliminate the
precedence issues Brendan cites while also making the syntax backward
compatible with earlier ES parsers. Is there any technical reason why
that wouldn't be possible?
Perhaps the "looks-like-a-duck, quacks-like-a-cow" objection would
apply here -- a corner-case thingy that looks like a function but
isn't should be avoided. But I have to say, in practice the required
parenthesizing around yield expressions in JS 1.7 is an eyesore I'd
like to see go away.
On May 14, 2009, at 2:38 PM, Brendan Eich wrote: > Indeed the low-precedence (same as assignment) nature of the yield > unary caused JS1.7 to follow Python in requiring parentheses around > yield expressions in lists: in comma expressions and actual > parameter lists (although we diverged from Python by allowing an > unparenthesized yield expression as the last actual argument in a > call or new expression's parameter list -- Python requires > foo((yield bar)) instead of foo(yield bar)). I have this idea that it would be better for yield expressions to look like function calls: 'yield' '(' expression? ')'. (Note that this is only a syntactical suggestion; clearly an implementation wouldn't actually treat it like a function.) That would eliminate the precedence issues Brendan cites while also making the syntax backward compatible with earlier ES parsers. Is there any technical reason why that wouldn't be possible? Perhaps the "looks-like-a-duck, quacks-like-a-cow" objection would apply here -- a corner-case thingy that looks like a function but isn't should be avoided. But I have to say, in practice the required parenthesizing around yield expressions in JS 1.7 is an eyesore I'd like to see go away.