Discuss
  • Uniform block scoping

Uniform block scoping

# Andreas Rossberg (10 years ago)

ES6 will have block scoping. However, it turns out that there are at least two different forms of blocks in the language:

  • normal block statements
  • block-like bodies hardwired into other constructs (try/catch bodies, function bodies)

In the current draft spec, these come with different rules regarding shadowing. For example:

let x; { let x }  // fine, shadowing

for (let x...) { let x }  // fine, shadowing

try {} catch(x) { let x }  // redeclaration error

function(x) { let x }  // redeclaration error

(function x() { let x })  // fine, shadowing

I think this subtle discrepancy is both unfortunate and unnecessary (ecmascript#3005). Moreover, with ES7 do expressions, I would like it to hold that

(...) => {...}    ≡    (...) => do {...}

so the former shouldn't impose extra restrictions. Therefor, I suggest we implement the uniform principle that every braced statement list behaves like a proper block (i.e., allows lexical declarations to shadow outer variables, regardless of how those are bound).

In terms of spec, this simply amounts to dropping several bullet points imposing syntactic restrictions about LexicallyDeclaredNames (in current Sections 13.14.1, 14.1.2, 14.2.1, 14.3.1, 14.4.1) -- i.e., simplification :). (Note: The legacy rule that legal ES5 function declarations are var-like, not lexical, would of course be unaffected.)

PS: With respect to try/catch/finally, I also wonder why we cannot simply relax the syntax and drop the hardwired block syntax. Then the block-scoping rules would follow naturally.

# Rick Waldron (10 years ago)

Here are some relevant discussions from past meetings:

# Allen Wirfs-Brock (10 years ago)

(Note that this topic is on the agenda for this month's TC39 meeting) (Also, note that as far as I can tell, the disagreement is only about the early errors described below. The use of a separate parameter scope that limits what closures in parameter expressions can capture has been agreed upon and is already in the ES6 draft spec.)

Everything would be so much cleaner if all we had were strictly lexically scoped declarations and no legacy to deal with...

Consider,

We've decided that that we don't want to allow multiple declarations for the same name in a scope:

function f() {
    let x;
    const x;  //early redeclaration error
}

function f() {
  let x;
  var x;  //early redeclaration error
}

function f() {
    let x;
    function x() {};  //early redeclaration error
}

We don't even have a runtime semantics for any of the above duplicate declarations.

Except that for legacy reasons we have to allow:

function f() {
    var x;
    function x() {}; 
    function x() {};  
    var x;
}

using the legacy ES semantics.

The legacy ES semantics also requires that:

function f(x) {
   var x;   //not an error
   console.log(x);
}
f(1);  //logs 1, not undefined

just like:

function g(x) {
   console.log(x);
}
g(1);  //logs 1

In other words, from the perspective of the body of the function, the following two declarations appear to be equivalent:

function f(x) {
}
function f(x) {
   var x;  
}

But we've already established this is an error:

function f(x) {
   var x;
   let x;  //early error, duplicate definition
}

and if that is an error, then its equivalent alternative form should also be an error:

function f(x) {
   let x;  //early error, duplicate definition
}

And that is the crux of the disagreement.

Andreas would like to reason about ES scoping as simple nested block contours with shadowing. But the reality is more complex than that. We have vars declarations the hoist to the top level. We have interactions between formal parameters and var declarations that interact with outer scopes (that are inner relative to the parameters if you think of the parameter list is a distinct scope contour). In fact, in my current working draft, FunctionDeclarationInstantiation is almost 3 pages of detailed algorithmic specification. (and adding a parameter scope didn't simplify things...).

We really don't want the average (and certainly not the novice) ES programmer to have to understand all the technical subtleties of these interactions. It's much easer to have a few simple rules that states which declarations are legal and which declarations are not. The rules are:

  • It is illegal for let/const/class declarations at the top level of a function to multiply define the same name.
  • It is illegal for a let/const/class declaration at the top level of a function to define the same name as a top level function declaration.
  • It is illegal for a let/const/class declaration at the top level of a function to define the same name as a var declaration that occurs anywhere within the function body.
  • It is illegal for a let/const/class declaration at the top level of a function to define the same name as a formal parameter of the function.

This is what the ES6 spec. currently says (using slightly different words). Andreas would like to eliminate that lat rule. I think it should remain, both for the specific equivalence discussed above and for overall simplicity.

In terms of spec, this simply amounts to dropping several bullet points imposing syntactic restrictions about LexicallyDeclaredNames [2] -- i.e., simplification :). (Note: The legacy rule that legal ES5 function declarations are var-like, not lexical, would of course be unaffected.)

It's actually not quite that simple. There are a number of places where the specified runtime semantics are simplified because it is known that early error rules have eliminated various possibilities, such as parameter/let naming conflicts. So, if we made this changeI will need to review all the relevant runtime semantics and look for newly exposed cases that need to be handled.

More on catch clauses and catch parameter scoping in a seperate message.

# Andreas Rossberg (10 years ago)

On 16 July 2014 18:38, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

(Note that this topic is on the agenda for this month's TC39 meeting)

Unfortunately, I won't be at the meeting. Do you think we can afford to defer this until September?

(Also, note that as far as I can tell, the disagreement is only about the early errors described below. The use of a separate parameter scope that limits what closures in parameter expressions can capture has been agreed upon and is already in the ES6 draft spec.)

Yes, though the scope separation that this already implies is what makes my suggestion natural and easy (I believe), see below.

Everything would be so much cleaner if all we had were strictly lexically scoped declarations and no legacy to deal with...

Yes indeed, but var scoping being what it is, we can still try to make lexically scoping itself as clean as possible. Which is what I am proposing. :)

Consider,

We've decided that that we don't want to allow multiple declarations for the same name in a scope:

function f() {
    let x;
    const x;  //early redeclaration error
}

function f() {
  let x;
  var x;  //early redeclaration error
}

function f() {
    let x;
    function x() {};  //early redeclaration error
}

We don't even have a runtime semantics for any of the above duplicate declarations.

Agreed.

Except that for legacy reasons we have to allow:

function f() {
    var x;
    function x() {};
    function x() {};
    var x;
}

using the legacy ES semantics.

The legacy ES semantics also requires that:

function f(x) {
   var x;   //not an error
   console.log(x);
}
 f(1);  //logs 1, not undefined

just like:

function g(x) {
   console.log(x);
}
g(1);  //logs 1

In other words, from the perspective of the body of the function, the following two declarations appear to be equivalent:

function f(x) {
}
function f(x) {
   var x;
}

Agreed as well.

But we've already established this is an error:

function f(x) {
   var x;
   let x;  //early error, duplicate definition
}

and if that is an error, then its equivalent alternative form should also be an error:

function f(x) {
   let x;  //early error, duplicate definition
}

And that is the crux of the disagreement.

Yes, that is were I disagree. My suggestion simply amounts to these examples behaving as if you had written, under the current draft:

function f(x) {{
  var x;  // fine, still same x
}}

function f (x) {{
  var x;
  let x;  // still a redeclaration error
}}

function f(x) {{
  let x;  // shadowing
}}

In particular, we already have a separate scope for the function body (due to default parameters). The special rules for var names in this scope aliasing parameter names are merely necessary for backwards compatibility (and moreover, only apply to ES5 cases, i.e., simple parameter lists).

I don't see a good reason to extend this special-case mechanism to lexical declarations. It is not consistent with either lexical scoping nor with default parameter semantics, which is why we already have to limit it to legacy cases. We can perfectly well choose to treat it as the legacy compatibility special-case it already is, and not infect other future design choices with it.

Andreas would like to reason about ES scoping as simple nested block contours with shadowing. But the reality is more complex than that. We have vars declarations the hoist to the top level. We have interactions between formal parameters and var declarations that interact with outer scopes (that are inner relative to the parameters if you think of the parameter list is a distinct scope contour). In fact, in my current working draft, FunctionDeclarationInstantiation is almost 3 pages of detailed algorithmic specification. (and adding a parameter scope didn't simplify things...).

We really don't want the average (and certainly not the novice) ES programmer to have to understand all the technical subtleties of these interactions. It's much easer to have a few simple rules that states which declarations are legal and which declarations are not. The rules are:

  • It is illegal for let/const/class declarations at the top level of a function to multiply define the same name.
  • It is illegal for a let/const/class declaration at the top level of a function to define the same name as a top level function declaration.
  • It is illegal for a let/const/class declaration at the top level of a function to define the same name as a var declaration that occurs anywhere within the function body.
  • It is illegal for a let/const/class declaration at the top level of a function to define the same name as a formal parameter of the function.

This is what the ES6 spec. currently says (using slightly different words). Andreas would like to eliminate that lat rule. I think it should remain, both for the specific equivalence discussed above and for overall simplicity.

I see, but don't think this consistency argument trumps the ones I gave. It's probably true that you have to choose between one or the other, but shouldn't forward-facing consistency be preferred over backward-facing consistency?

You haven't convinced me yet of the simplicity argument, which in fact goes the opposite way, I believe, see below.

In terms of spec, this simply amounts to dropping several bullet points imposing syntactic restrictions about LexicallyDeclaredNames [2] -- i.e., simplification :). (Note: The legacy rule that legal ES5 function declarations are var-like, not lexical, would of course be unaffected.)

It's actually not quite that simple. There are a number of places where the specified runtime semantics are simplified because it is known that early error rules have eliminated various possibilities, such as parameter/let naming conflicts. So, if we made this changeI will need to review all the relevant runtime semantics and look for newly exposed cases that need to be handled.

Yes, you are right, a change is necessary to the algorithm in 9.2.13. But AFAICS it is a net simplification as well: isn't all you need to do getting rid of the needsParameterEnvironment special casing and always create the separate environment for the body? (*) Everything else is already perfectly taken care of by the existing algorithm, namely by the instantiatedVarNames list.

(*) Concretely, you get rid of steps 6 and the condition 28.0 in the current algorithm.

Am I missing something? Is there anything else you have in mind?

(And before anybody gets concerned: always having this extra environment is purely a spec device, not a runtime cost. An implementation will of course be able to merge the scopes in almost all practical cases, as is the case in the presence of default parameters already. The current draft effectively tries to implement this very optimisation in the spec itself, but I don't think it belongs there.)

# Andreas Rossberg (10 years ago)

On 17 July 2014 19:29, Brendan Eich <brendan at mozilla.org> wrote:

We should discuss at this meeting and try to reach agreement. I can proxy for you, since I agree with you :-).

OK, thanks. (If it can be scheduled for Thursday morning, I might even be able to call in.)

# Mark S. Miller (10 years ago)

Does anyone see any impediment to scheduling this Thursday morning? If not, we will do so.

# Brendan Eich (10 years ago)

Andreas Rossberg wrote:

I think this subtle discrepancy is both unfortunate and unnecessary [1]. Moreover, with ES7 do expressions, I would like it to hold that

(...) => {...} ≡ (...) => do {...}

I channeled you as best I could, and Dmitry Lomov kindly channeled you on this point, but more than a few TC39ers objected that the left arrow function, with a body block instead of a body expression, has different semantics already, ignoring whether let x; in the body block could shadow a parameter x. First, 'return' is the only way to return a result in the left example, whereas thanks to do-expression being an expression, the completion value (reformed) of the right ... is the return value, even without 'return'.

I wanted to pass this back ASAP. More is being recorded in the meeting notes, but here you go. We'll keep channeling you as best we can!

# Brendan Eich (10 years ago)

The general problem is that body blocks are not exactly blocks, due to legacy cruft -- and this legacy cannot be separated from 'let' ideals because we want programmers to refactor from 'var' to 'let'.

So we must give greater weight (compared to the ideal case of a body block just being a block) to refactoring hazards that result in silent but deadly bugs rather than early errors.

In particular,

function f(a) {
   ... // TDZ
   let a = ...;
   ...
}

and

function g() {
   try {
     ...
   } catch (e) {
     ... // TDZ
     let e = ...;
     ...
   }
}

should be early errors because there's no useful shadowing going on with 'let' -- the TDZ means the outer binding cannot be used in the commented places -- but any prior version using 'var' would have worked and possibly allowed (coverage-dependent) uses of the "outer" (in catch's case, var hoists across the catch head in sloppy code; in the parameter case there's only ever one 'a' binding) use in the TDZ.

So an early error does no actual harm in either case, and helps avoid bugs slipping past incomplete test coverage.

# Andreas Rossberg (10 years ago)

On 31 July 2014 00:05, Brendan Eich <brendan at mozilla.org> wrote:

I channeled you as best I could, and Dmitry Lomov kindly channeled you on this point, but more than a few TC39ers objected that the left arrow function, with a body block instead of a body expression, has different semantics already, ignoring whether let x; in the body block could shadow a parameter x. First, 'return' is the only way to return a result in the left example, whereas thanks to do-expression being an expression, the completion value (reformed) of the right ... is the return value, even without 'return'.

Yeah, I glossed over the return value, but to be precise, it would simply be:

(...) =>  {...}    ≡    (...) =>  do {...; undefined}

The general problem is that body blocks are not exactly blocks, due to legacy cruft -- and this legacy cannot be separated from 'let' ideals because we want programmers to refactor from 'var' to 'let'.

So we must give greater weight (compared to the ideal case of a body block just being a block) to refactoring hazards that result in silent but deadly bugs rather than early errors.

In particular,

function f(a) {
  ... // TDZ
  let a = ...;
  ...
}

and

function g() {
  try {
    ...
  } catch (e) {
    ... // TDZ
    let e = ...;
    ...
  }
}

should be early errors because there's no useful shadowing going on with 'let' -- the TDZ means the outer binding cannot be used in the commented places -- but any prior version using 'var' would have worked and possibly allowed (coverage-dependent) uses of the "outer" (in catch's case, var hoists across the catch head in sloppy code; in the parameter case there's only ever one 'a' binding) use in the TDZ.

So an early error does no actual harm in either case, and helps avoid bugs slipping past incomplete test coverage.

Well, this is a valid concern, but I'd like to point out that it is a flawed argument for defending the status quo.

If we really cared about legacy refactoring mistakes going unnoticed, then the suitable (and consistent) semantics would be to make all intra-function shadowing (or more precisely, multiple declarations) an error. Otherwise, we are just picking random cases. For example, I don't see how the above argument can justify the current discrepancy between these two:

for (let x ...) { let x; ... }  // fine
try ... catch(x) { let x; ... }  // error

The former is just as much a potential bug as the latter, or as all other cases of redeclarations.

# Brendan Eich (10 years ago)

Andreas Rossberg wrote:

The former is just as much a potential bug as the latter, or as all other cases of redeclarations.

You're right, if the issue is shadowing an immediately enclosing binding. The param vs. var-ported-to-let case is different because the var does not shadow and this is observable via sloppy mode arguments object element / formal parameter aliasing.

But even in the catch case, in sloppy code with var x in the catch block, latent bugs are likely to lurk due to hoisting.

So the rationale is not that immediate-shadowing per se should be an error -- it is rather that var-cum-let bindings that tangle with or cross over a legacy danger zone (formal parameter aliased by var; implicit let binding in catch head) should be errors.

Macros and similar systems could very well want immediate let-bound shadowing within a block that is self-contained, in new code. We don't believe all such cases should be errors.

Does this help?