Removal of NoIn grammar productions and for-loop parsing
Allen, are you doing this some other way? Static semantics can't do it, we need parametric productions or else ye olde NoIn splitting.
Right, this point didn't come up when we talked about eliminating the NoIn productions at the last TC39 meeting: rwaldron/tc39-notes/blob/master/es6/2013-07/july-23.md#41-es6-status-report
Presumably the original motivation for the NoIn productions was to resolve the ambiguity that the introduction of the ES3 'in' operator created for the ES1 production:
IterationStatement : 'for' '(' 'var' Identifier [Initializer] 'in' Expression ')' Statement
when parsing something like:
for (var x = a in b in c) ;
it could be either:
for (var x = (a in b) in c) ;
or
for (var x = a in (b in c)) ;
Because we agreed for ES6 to eliminate the optional Initializer from for-in/for-of statements this ambiguity is no longer a possibility and the NoIn productions are not needed to resolve them. At the last meeting the discussion was about the grammar simplification benefits of completely eliminating the NoIn productions. I did this in the rev17 ES6 draft.
However, at the meeting, we did not discussion the fact that in ES3 NoIn was also used in:
IterationStatement : 'for' '(' [ExpressionNoIn] ';' [Expression] ';' [Expression] ')' Statement
This makes statements like this:
for (a in b ;;) ;
illegal in ES3. This was presumably done for nanny reasons as using the 'in' operator in this position isn't ambiguous. The availability of the NoIn productions also made it easy to express such a restriction.
But if we eliminate the NoIn productions it's no longer so easy to impose that restriction. I may be able to come up with some other static semantic mechanism to express that restriction but it will have complexity similar to the NoIn productions.
My preference is to simply allow the use of the 'in' operator in the first expression of a for(;;) statement. This is what the rev17 grammer does. As it is currently illegal in ES<=5.1, allowing 'in' use in that context is an extension rather than a breaking change. 'a in 'b may not be very useful in that position but neither is 'a + b'. The simplification of the expression grammar is a pretty big win both now and for future extensions.
Another item from the day of the July meeting that I happened to miss.
Allen Wirfs-Brock wrote:
This makes statements like this:
for (a in b ;;) ;illegal in ES3. This was presumably done for nanny reasons
No, not for nanny reasons.
as using the 'in' operator in this position isn't ambiguous. The availability of the NoIn productions also made it easy to express such a restriction.
Consider this toy grammar:
%token FOR
%token IDENT
%token IN
%token NUMBER
%%
Program:
Statement
| Program Statement
;
Statement:
FOR '(' Expression ';' Expression ';' Expression ')' Statement
| FOR '(' Expression IN Expression ')' Statement
| Expression ';'
;
Expression:
Expression IN Primary
| Primary
;
Primary:
NUMBER
| IDENT
;
This grammar is ambiguous: as bison(1) says,
state 18
6 Expression: Expression IN Primary .
7 | Primary .
IN reduce using rule 6 (Expression)
IN [reduce using rule 7 (Expression)]
')' reduce using rule 7 (Expression)
$default reduce using rule 6 (Expression)
See attached file for full output.
But if we eliminate the NoIn productions it's no longer so easy to impose that restriction. I may be able to come up with some other static semantic mechanism to express that restriction but it will have complexity similar to the NoIn productions.
You can'd do this via static semantics, as I said in my last message. We need an LR(1) grammar, that was always a consensus requirement.
My preference is to simply allow the use of the 'in' operator in the first expression of a for(;;) statement.
Ambiguity is not a matter of preference. We need to validate the ES6
grammar. Until then, please put back the NoIn productions. They were not
there only because of the silly and unwanted initialiser option for for (var x = y of z). They were there because for-in and for(;;) have prefixes in common up to arbitrary lookahead.
How is this different than destructuring assignment? We need a cover grammar in this case too.
Erik Arvidsson wrote:
How is this different than destructuring assignment? We need a cover grammar in this case too.
The toy (reduced) grammar I showed is ambiguous, just as ES3's grammar would be if you were to remove the NoIn productions.
Destructuring is ironically already covered (cover-grammar sense) by ES3's grammar already, without ambiguity (allowing the lookahead restriction by which an expression statement cannot start with '{' of course). LeftHandSideExpression indirectly produces ArrayLiteral and ObjectLiteral. Those are ruled out as legal LHS forms only by PutValue.
By now it should be clear that while a semantic restriction (static or not) can make an error of one case or another, it cannot disambiguate a grammar. Parsing comes first and must be unambiguous to be sound. See esdiscuss/2008-October/007883.html:
This is why ambiguous grammars are bad and unsuitable for our spec. In an unambiguous grammar, if you find a series of production expansions that matches a source program, then you know that those are the expansions that will be taken. In an ambiguous grammar, you also need to prove the negative: no other expansion can match that source program and shadow your expansions. Proving the negative causes trouble because a language extension could turn the mismatch into a match and because sometimes, as in the above case, you expected some other part of the grammar to shadow your expansions but it didn't.
After this thread, we agreed following vigorous debate to stick with LR(1) (sans ASI) for ECMA-262. I don't think we should go back on this decision.
On Sep 1, 2013, at 7:42 PM, Brendan Eich wrote:
Ambiguity is not a matter of preference. We need to validate the ES6 grammar. Until then, please put back the NoIn productions. They were not there only because of the silly and unwanted initialiser option for
for (var x = y of z). They were there because for-in and for(;;) have prefixes in common up to arbitrary lookahead.
Yup, we went off-track on this at the meeting. But Waldemar's point about arrow functions is still valid. We are going to need to have ArrowFunctionNoIn to disallow things like:
for (f = x => x in foo;;)
For sure.
The NoIn grammar productions have been removed in rev17. Does this mean that
for (a in b;;);is now a valid (C-style) for-loop instead of a SyntaxError?