Generalize do-expressions to statements in general?

# Isiah Meadows (9 years ago)

I was reading a recent thread esdiscuss.org/topic/allow-try-catch-blocks-to-return-a-value where

do-expressions simplified a common try-catch use case, and I was wondering if do could be simplified to an expression? It would allow for this to be solved very easily, but also add a lot more flexibility in this proposal, as well as avoiding some ugly nested braces.

I know it would cause an ambiguity with do-while loops, but that could be resolved with a single token lookahead of "if the next token is the keyword while, then the block body is the body of a do-while loop, else it is the body of the block statement in a do expression".

As for the EBNF, do-expressions could be parsed with a goal symbol of either +While or -While, with do-while statements spec-wise effectively being treated as do-expressions without an init part run repetitively, but mandated to be statements.

// Do expression
let foo = do {
  foo(0)
};

let tried = do try {
  foo(0)
} catch (e) {
  throw e
};

// Do-while statement
let i = 0;
do {
  foo(i)
} while (i++ < 10);

// Combined:
let i = 0;
let foo9 = do do {
  foo(i) // can have side effects, foo9 = foo(9)
} while (i++ < 10);

Another example of where this could come in handy: simplifying asynchronous code.

function readConfig() {
  fs.readFileAsync('config.json', 'utf8')
    .then(JSON.parse)
    .then(contents => do if (contents.unexpectedProperty) {
      throw new Error('Bad property') // rejects the promise
    } else {
      doSomething(contents)
    })
    .catch(err => process.domain.emit('err', error))
}

// With only block statement
function readConfig() {
  fs.readFileAsync('config.json', 'utf8')
    .then(JSON.parse)
    .then(contents => do {
      if (contents.unexpectedProperty) {
        throw new Error('Bad property') // rejects the promise
      } else {
        doSomething(contents)
      }
    })
    .catch(err => process.domain.emit('err', error))
}

// Without do-expressions
function readConfig() {
  fs.readFileAsync('config.json', 'utf8')
    .then(JSON.parse)
    .then(contents => {
      if (contents.unexpectedProperty) {
        throw new Error('Bad property') // rejects the promise
      } else {
        doSomething(contents)
      }
    })
    .catch(err => process.domain.emit('err', error))
}

As you can see, the more general version does simplify things a little.

Also, if-statements look better than long ternaries IMHO, and are less repetitive than their counterpart, repeated assignment (us lazy typists...):

let foo = do if (someCondition) {
  value
} else if (someOtherCondition) {
  value + 1
} else if (someEdgeCase) {
  addressEdgeCase(value)
} else {
  value
}
# Mark S. Miller (9 years ago)

Interesting. Got me thinking. Here's an alternate proposal I'll call "do expressions without the 'do'."

At people.mozilla.org/~jorendorff/es6-draft.html#sec-expression-statement we have the syntax of the expression statement. Ignoring sloppy "let" nonsense, this says that an expression statement cannot begin with "{", "function", or "class".

At people.mozilla.org/~jorendorff/es6-draft.html#sec-ecmascript-language-statements-and-declarations are the legal ES6 statements. Note that most of these begin with a keyword that cannot possibly be legal at the beginning of an expression. Therefore, adding all these initial-statement-keywords to the list of things that cannot begin an expression statement would break nothing. They already cannot begin an expression statement.

With the expression statement prohibition in place, now we can allow all these forms to be expressions. As with "{", "function", or "class", if you want to state such an expression in expression-statement position, surround it with parens.

Because all these new forms will look bizarre and confusing, at least at first, let's say these always need surrounding parens to be expressions. I think that would help minimize confusion.

If we do this, the oddest duck is "{", since it begins an object literal expression. This proposal gives us no straightforward way to express an block expression. "function" and "class" are less odd, since their existing expression forms mean what you almost might expect by this new rule -- even though they are initial-declaration-keywords rather than initial-statement-keywords.

The remaining initial-declaration-keywords are "let" and "const". We already made "let" insane regarding these issues in sloppy mode, so I'm going to ignore that. But let's consider "const" and strict "let". These already cannot appear at the beginning of an expression, so it would not break anything to add them to the prohibition list for the beginning of expression statements.

No current expression can add any binding to the scope in which the expression appears. Let's examine the consequences of having parens -- rather than containing a "{"-block to create a nested scope with a value (which would conflict with object literals), instead simply define a block-like nested scope with a value. This would allow declarations and statements within the parens, much like the current "do" proposal. It would even be consistent enough with the existing semantics of paren-surrounded function and class expressions: Someone who sees these as a function or class declaration within its own nested scope, whose value was the value being declared, would rarely be surprised by the subtle difference between that story and the current semantics.

Having parens accept a list of declarations and statements rather than just an expressions seems like a radical change that must break something, but I can't find a problem. Am I missing something?

Examples inline:

On Mon, Jul 13, 2015 at 5:47 PM, Isiah Meadows <impinball at gmail.com> wrote:

I was reading a recent thread esdiscuss.org/topic/allow-try-catch-blocks-to-return-a-value where do-expressions simplified a common try-catch use case, and I was wondering if do could be simplified to an expression? It would allow for this to be solved very easily, but also add a lot more flexibility in this proposal, as well as avoiding some ugly nested braces.

I know it would cause an ambiguity with do-while loops, but that could be resolved with a single token lookahead of "if the next token is the keyword while, then the block body is the body of a do-while loop, else it is the body of the block statement in a do expression".

As for the EBNF, do-expressions could be parsed with a goal symbol of either +While or -While, with do-while statements spec-wise effectively being treated as do-expressions without an init part run repetitively, but mandated to be statements.

// Do expression
let foo = do {
  foo(0)
};
let foo = (foo(0));

This seems as broken as the original. In both cases, unless I'm missing something, this is a TDZ violation when the right side evaluates foo. Mistake?

let tried = do try {
  foo(0)
} catch (e) {
  throw e
};
let tried = (try { foo(0) } catch (e) { throw e });
// Do-while statement
let i = 0;
do {
  foo(i)
} while (i++ < 10);

// Combined:
let i = 0;
let foo9 = do do {
  foo(i) // can have side effects, foo9 = foo(9)
} while (i++ < 10);
let i = 0;
let foo9 = (do { foo(i) } while (i++ < 10));

Another example of where this could come in handy: simplifying asynchronous code.

function readConfig() {
  fs.readFileAsync('config.json', 'utf8')
    .then(JSON.parse)
    .then(contents => do if (contents.unexpectedProperty) {
      throw new Error('Bad property') // rejects the promise
    } else {
      doSomething(contents)
    })
    .catch(err => process.domain.emit('err', error))
}
...
.then(contents => (if (contents.unexpectedProperty) {
...
}))
...
// With only block statement
function readConfig() {
  fs.readFileAsync('config.json', 'utf8')
    .then(JSON.parse)
    .then(contents => do {
      if (contents.unexpectedProperty) {
        throw new Error('Bad property') // rejects the promise
      } else {
        doSomething(contents)
      }
    })
    .catch(err => process.domain.emit('err', error))
}

// Without do-expressions
function readConfig() {
  fs.readFileAsync('config.json', 'utf8')
    .then(JSON.parse)
    .then(contents => {
      if (contents.unexpectedProperty) {
        throw new Error('Bad property') // rejects the promise
      } else {
        doSomething(contents)
      }
    })
    .catch(err => process.domain.emit('err', error))
}

As you can see, the more general version does simplify things a little.

Also, if-statements look better than long ternaries IMHO, and are less repetitive than their counterpart, repeated assignment (us lazy typists...):

let foo = do if (someCondition) {
  value
} else if (someOtherCondition) {
  value + 1
} else if (someEdgeCase) {
  addressEdgeCase(value)
} else {
  value
}
let foo = (if (someCondition) {
...
})
# Isiah Meadows (9 years ago)

To be perfectly honest, though, I'm not entirely sure the specifics of the do-expression proposal, since Google is failing me here (can't find a thing giving more detail than this mailing list). And as for what my proposal here is, I forgot to mention that expression statements would be explicitly prohibited as the body of a do-expression.

As for yours, I like it too, except if we keep adding all these extra parentheses, we might as well make JavaScript into a Lisp...(well, except LispyScript lispyscript.com kinda has...) ;)

In all seriousness, I like your idea as well, but the parsing would have to take into account a similar distinction between expressions and other statements. And that problem with objects vs blocks would result in a similar situation we previously ran into with the same ambiguity (in reverse) esdiscuss.org/topic/may-24-26-rough-meeting-notes#content-3 in

arrow function syntax. The other issue is that your proposal, because of that ambiguity, would likely bring a break in backwards compatibility, one that is definitely not worth it:

// Is this a block or object literal?
let foo = ({ bar: 1 });
# Isiah Meadows (9 years ago)

On Mon, Jul 13, 2015 at 7:33 PM, Mark S. Miller <erights at google.com> wrote:

This seems as broken as the original. In both cases, unless I'm missing something, this is a TDZ violation when the right side evaluates foo. Mistake?

Yep...s/foo/anything but foo/ ;)

# Mark Miller (9 years ago)

On Mon, Jul 13, 2015 at 6:53 PM, Isiah Meadows <impinball at gmail.com> wrote:

To be perfectly honest, though, I'm not entirely sure the specifics of the do-expression proposal, since Google is failing me here (can't find a thing giving more detail than this mailing list). And as for what my proposal here is, I forgot to mention that expression statements would be explicitly prohibited as the body of a do-expression.

As for yours, I like it too, except if we keep adding all these extra parentheses, we might as well make JavaScript into a Lisp...(well, except LispyScript lispyscript.com kinda has...) ;)

In all seriousness, I like your idea as well, but the parsing would have to take into account a similar distinction between expressions and other statements. And that problem with objects vs blocks would result in a similar situation we previously ran into with the same ambiguity (in reverse) esdiscuss.org/topic/may-24-26-rough-meeting-notes#content-3 in arrow function syntax. The other issue is that your proposal, because of that ambiguity, would likely bring a break in backwards compatibility, one that is definitely not worth it:

// Is this a block or object literal?
let foo = ({ bar: 1 });

It is an object literal. My proposal is not technically ambiguous, at least for this case, since I am not proposing we change the current meaning of "({", "(function", or "(class" at all. So, under this proposal, these three (and the damn sloppy "let" would need to be called out as special cases. This is necessary so that this proposal does not change the meaning of programs that are already syntactically legal.

However, there is a human-factors ambiguity, i.e., a violation of the principle of least surprise. For "(function" and "(class", the semantic difference is subtle and would very rarely trip anyone up. Having reduced the practical hazard to only one special case, we simply need to teach that, when you wanted to say "({...})" meaning a block, just remove the curies and say instead "(...)".

With a bit of lookahead, we can even include labeled statements, since an expression cannot currently begin with <identifier>":". So your flip side

of your example becomes

let foo = (bar: 1);

where "bar: 1" is therefore a labeled statement. I admit this seems weird, but perhaps that's only because we're not yet used to it.

# Andreas Rossberg (9 years ago)

All you are proposing is to allow the braces to be dropped from a do-expression, right? That's an obvious tweak, although I'm not sure if it really improves readability. (Other than that, do-expressions are already intended to work as you describe, using the completion value of the statement list. That's their whole point, after all.)

I had something else in mind. Once we have do expressions, we can introduce syntactic sugar that effectively makes any statement syntax into legal expressions. E.g.:

  throw expr  ~>  do { throw expr; }
  try expr catch (pat) expr  ~>  do { try { expr; } catch (pat) { expr; } }
  if (expr) expr else expr  ~>  do { if (expr) expr; else expr; }
  etc

At least for those statements for which it makes sense. To avoid ambiguity, all you need to do is extend the existing restriction that none of the initial keywords may start an expression statement.

But I intend to propose that separately from (or as an optional part of) the do-expressions proposal, since it might be more controversial.

# Andreas Rossberg (9 years ago)

I don't see why you need parens at all, see my previous post. But I wouldn't make the do-less forms the base syntax,; rather, only short-hands for the general thing. In particular, because the ability to have an actual block inside an expression is one primary motivation for having do-expressions in the first place.

...Ah, it's 2015, and we still have to come up with ways to overcome the archaic statement/expression distinction from the stone ages. :)

# Matthew Robb (9 years ago)

The only gripes I have with do expressions is the inability to specify the value produced in an obvious and uniform way, also are do expressions capable of being labelled?

# Andreas Rossberg (9 years ago)

On 14 July 2015 at 15:04, Matthew Robb <matthewwrobb at gmail.com> wrote:

The only gripes I have with do expressions is the inability to specify the value produced in an obvious and uniform way,

Well, the completion value is fairly uniform. You mean an analogue to a return in a function body?

also are do expressions capable of being labelled?

No, but they can of course contain a labelled statement, which is equivalently expressive.

# Mark S. Miller (9 years ago)

On Tue, Jul 14, 2015 at 2:31 AM, Andreas Rossberg <rossberg at google.com> wrote:

I don't see why you need parens at all, see my previous post. But I wouldn't make the do-less forms the base syntax,; rather, only short-hands for the general thing. In particular, because the ability to have an actual block inside an expression is one primary motivation for having do-expressions in the first place.

Ah. Take a look at my full proposal. The bizarre observation is that extending the syntax of parens to contain approx a block-body, and extending its meaning to creating a block-like scope for evaluating that block-body, in addition to returning a value. In that case, we simply don't need the "do" expression at all. Don't propose something unnecessarily complex just because you expect that something simpler would be too controversial. If it actually is too controversial, that's another matter.

...Ah, it's 2015, and we still have to come up with ways to overcome the archaic statement/expression distinction from the stone ages. :)

Between Gedanken, Smalltalk, and Actors, almost everything we do in oo dynamic language design was already conceived right by the early '70s. Retrofitting without breaking things takes much longer than invention ;)

# Andreas Rossberg (9 years ago)

On 14 July 2015 at 16:48, Mark S. Miller <erights at google.com> wrote:

Ah. Take a look at my full proposal. The bizarre observation is that extending the syntax of parens to contain approx a block-body, and extending its meaning to creating a block-like scope for evaluating that block-body, in addition to returning a value. In that case, we simply don't need the "do" expression at all. Don't propose something unnecessarily complex just because you expect that something simpler would be too controversial. If it actually is too controversial, that's another matter.

I would very much dislike introducing a second syntax for blocks, though -- which is essentially what you are suggesting. Especially when curly braces provide a much better visual clue for the extent of a scope than innocent plain parens do. It's the natural expectation for a C-like language, too.

In the design of any modern language the whole notion of block of course is totally obsolete. But JavaScript has its C heritage, and I doubt bolting on something alien would make it a prettier language.

Between Gedanken, Smalltalk, and Actors, almost everything we do in oo dynamic language design was already conceived right by the early '70s. Retrofitting without breaking things takes much longer than invention ;)

Well, statements vs expressions was already found unnecessary before OO, in the early 60s -- consider Algol 68. (Let alone Lisp, which is late 50s.)

# Mark S. Miller (9 years ago)

On Tue, Jul 14, 2015 at 10:59 AM, Andreas Rossberg <rossberg at google.com> wrote:

I would very much dislike introducing a second syntax for blocks, though -- which is essentially what you are suggesting. Especially when curly braces provide a much better visual clue for the extent of a scope than innocent plain parens do. It's the natural expectation for a C-like language, too.

I can see that. I'm torn.

Well, statements vs expressions was already found unnecessary before OO, in the early 60s -- consider Algol 68. (Let alone Lisp, which is late 50s.)

For that part specifically, sure. Gedanken also had full indefinite extent lexical closures. It might have been the first to do so -- Lisp was dynamically scoped at the time and Actors had not yet been invented. I've always been puzzled why Gedanken has not gotten more attention -- especially since it was mainly by John Reynolds. Check it out -- you'll be impressed.

# Bob Myers (9 years ago)

With all "do" respect, none of this syntax tinkering makes any sense to me.

I've been programming JS for 15 years and never noticed I needed a try block that returns a value.

Long ago I programmed in a language called AED that had valued blockl, which I was quite fond of, but never felt the need for that in JS for whatever reason.

For looping over something and getting both the index and the element, forEach does just fine for me; exiting in the middle of the loop is an edge case. When I want to use for, it never bothered me to say let elt = arr[i];.

What we need are powerful, simple, generic, clean notions that work together to provide building blocks to allow succinct, logical, innovative notational solutions to real-world problems.

# Andreas Rossberg (9 years ago)

I've been programming in C++ for 25 years, and didn't have much need for a try expression or nested binding either.

I've also been programming in functional languages for 20 years, and need them on a regular basis.

It all depends on how high-level your programming style is. Also, Sapir Whorf applies as usual.

# Mark S. Miller (9 years ago)

I echo this. E is a dynamic language with many similarities with JS, including a similarly C-like syntax. In E I use everything-is-a-pattern-or-expression all the time. When I first moved to JS I missed it. Now that I am used to the JS statements-are-not-expressions restrictions, I no longer do, with one exception:

When simply generating simple JS code from something else, this restriction is a perpetual but minor annoyance. By itself, I would agree that this annoyance is not important enough to add a new feature. However, if rather than "adding a feature", we can explain the change as "removing a restriction", then JS would get both simpler and more powerful at the same time. Ideally, the test would be whether, when explaining the less restrictive JS to a new programmer not familiar with statement languages, this change results in one less thing to explain rather than one more.

# Herby Vojčík (9 years ago)

I like the idea though it seems a bit dense and strange on the first look. One breaking change is, though, that before the change, semicolon inside parentheses is an error, which often catches the missing parenthesis; after the change it is not (and manifests itself only at the end of the file; or even two errors can cancel each other and make conforming JS but with different semantics).

# Herby Vojčík (9 years ago)

(typo correction)

# Andreas Rossberg (9 years ago)

On 16 July 2015 at 17:29, Mark S. Miller <erights at google.com> wrote:

When simply generating simple JS code from something else, this restriction is a perpetual but minor annoyance.

Indeed, one motivation for do-expressions is better support for compilers targeting JS. And for some of those, not being able to mix statements and expressions, not having try inside expressions, and not having support for nested bindings, can be very tough, because it prevents compositional translation.

By itself, I would agree that this annoyance is not important enough to add

a new feature. However, if rather than "adding a feature", we can explain the change as "removing a restriction", then JS would get both simpler and more powerful at the same time. Ideally, the test would be whether, when explaining the less restrictive JS to a new programmer not familiar with statement languages, this change results in one less thing to explain rather than one more.

I doubt that will work, because there still will be plenty of artefacts and irregularities of a statement language that they will have to understand. Pretending it's an expression language will rather cause more confusion than less, because it isn't (for one, you can't get rid of the 'return' statement).

# Mark S. Miller (9 years ago)

I don't see a conflict between return and being an expression language.

Smalltalk and E both have return. In Scheme terms, this is simply call-with-escape-continuation. Gedanken again was probably the first to have this right with their "escape" construct, which E borrowed. E's method syntax desugars into a more primitive method syntax plus a top level escape form defining an escape continuation. E's return desugars into calling the escape continuation.

The problem with return is orthogonal -- that we chose to define arrow functions so that are their own return point, rather that having return be TCP to the enclosing method. In escape continuation terms, we made the mistake of shadowing the outer escape continuation binding. But making JS into an expression language would not make this worse.

# Matthew Robb (9 years ago)

Just wanted to say I was playing around with the idea of using parens as block-expressions this morning and I REALLY like it. It doesn't feel like an added feature at all it just seems natural since blocks don't normally produce a value.

The questions I think that remain are:

  1. return?
  2. yield?
  • Matthew Robb
# Waldemar Horwat (9 years ago)

On 07/16/2015 13:35, Herby Vojčík wrote:

I like the idea those it seems a bit dense and strange on the first look. One breaking change is, though, that before the change, semicolon inside parentheses is an error, which often catches the missing parenthesis; after the change it is not (and manifests itself only at the end of the file; or even two errors can cancel each other and make conforming JS but with different semantics).

That's my concern as well. We'd be significantly complicating the syntax (and not in a clean way because the rules are not orthogonal), and densifying the space of valid but bizarre syntaxes. More cases that used to be a simple syntax error can now turn into something grammatically correct but wrong.

This can also have adverse implications for lexing (the old / start-of-regexp-vs-division tokenization issue) and the potential for experimenting with macro systems, which are strongly negatively affected by anything that complicates the / issue in lexing.