Discuss
  • Proposal: Generator returning a value should throw SyntaxError

Proposal: Generator returning a value should throw SyntaxError

# Adam Ahmed (11 years ago)

Long-time lurker, first-time poster. Profuse apologies if this was mentioned before and I failed to find it.

I've been using V8's generator implementation in Node 0.11.x recently, and have come across what I believe is a footgun with generators currently. That is - the ability to return a value, not just yield a value. I am proposing that while return; is still allowed, return value; becomes a syntax error within generators.

An alternative proposal is to implicitly treat return value within a generator as yield value; return;, however this would create a difference in the semantics between return; and return undefined; that currently doesn't exist and I believe such a thing has been proposed before unsuccessfully in esdiscuss.org/topic/void-as-a-value

There are a couple reasons I believe this should be a syntax error.

Firstly, for-of doesn't support it, thereby making it a bit halfway-there to start with:

function * upToThree() {
    yield 1;
    yield 2;
    return 3;
}

for(var num of upToThree()) {
    console.log(num); // 1, then 2. Never logs 3
}

It also makes it impossible to return an empty iteration (or at least, impossible to distinguish an empty iteration from an iterator that has a single undefined value):

function* empty() {
    return; // or is it
    // return undefined; ?
}

Supporting that style also has knock-on effects for consuming generators like takeUntil (which will execute a function on each value, and yield them until it reaches one where the function returns true). Functions like this will have to precompute the next value to see if they should yield or return the current value. For example:

function * takeUntilThree(iterator) {
    var curr, next;
    next = iterator.next();
    while(!next.done) {
        curr = next;
        next = iterator.next();
        if (next === 3) { // only know which to do based on the _next_

value) return curr.value; } else { yield curr.value; } } return curr && curr.value; // Note: the undefined vs empty problem is still not fixed. }

var untilThree = takeUntilThree(upToThree());
untilThree.next(); // { value : 1, done : false }
untilThree.next(); // { value : 2, done : true }

For these reasons, I find the ability to return a value from a generator is not useful (though I may have missed a use case).

Given that any generator with an occurrence of return value; can be reimplemented cleanly as

yield value;
return;

I suspect we can remove the footgun by throwing early when an attempt to return a value is encountered within a generator.

I must admit I'm not across how difficult this would be for implementers. But I believe it will remove potential danger for developers without harming any real usages.

Thanks for listening! Apologies for length!

Adam

# Brendan Eich (11 years ago)

We deliberated long and hard, after going through a design state without return expr;, on this. We agree with

www.python.org/dev/peps/pep-0380

# Brendan Eich (11 years ago)

Sorry, premature send!

Brendan Eich <mailto:brendan at mozilla.com> September 26, 2013 6:57 PM We deliberated long and hard, after going through a design state without return expr;, on this. We agree with

www.python.org/dev/peps/pep-0380

We want return expr; -- not for the 'for-of' special forms, which clients use knowingly to consume in-band values, not the OOB return value -- but for the case of generators used as tasks (see ), where the scheduler can make good use of the return value:

mozilla/task.js

# Adam Ahmed (11 years ago)

Thanks, I see. The usage is for TaskResult sorts of things where the scheduling is done via calls to yield, and the output of the function is done via return. Fair enough. It still seems like a huge footgun for the other cases which I'd expect are far more common than cooperative scheduling, but I can respect that decision.

# Brandon Benvie (11 years ago)

On 9/26/2013 7:18 PM, Adam Ahmed wrote:

Thanks, I see. The usage is for TaskResult sorts of things where the scheduling is done via calls to yield, and the output of the function is done via return. Fair enough. It still seems like a huge footgun for the other cases which I'd expect are far more common than cooperative scheduling, but I can respect that decision.

An example use case is delegating yield:

function* foo() {
   yield 'what';
   yield 'ever';
   return "DONE";
}

function* bar() {
   console.log(yield* foo());
}

This console.logs "DONE". Can't do this without a way to handle a return value from generators.

# Brandon Benvie (11 years ago)

On 9/26/2013 10:40 PM, Brandon Benvie wrote:

function* foo() {
  yield 'what';
  yield 'ever';
  return "DONE";
}

function* bar() {
  console.log(yield* foo());
}

Err, this logs "DONE" when you do:

var gen = bar();
gen.next();
gen.next();
gen.next();

but you got the idea...

# Adam Ahmed (11 years ago)

In light of the recent thread discussing async and await keywords, I thought it'd be appropriate to raise this point again, understanding it may be too late to make a change.

As my original post details, the concept of return within a generator is surprising in its difference in behavior from yield.

This does not do as 'expected' in a for-in:

function * threeCount() {
  yield 1;
  yield 2;
  return 3;
}

The argument for allowing return values was that usages in the vein of task.js will use the return value as a real return value and the yields for scheduling.

If we' re going to have async and await serve the scheduling purpose as well, can we remove the 'return' foot gun from generators? It sounds like it's just a stopgap until async-await, and a painful one, IMO. A syntax error on a generator that returns values would make the scheduling (async-await) vs iteration (generator) use cases much more clear. It'll be much easier for new JS devs to understand generators.

Happy to be shutdown again, just thought it was worth reconsidering with new async-await keywords in play.

# David Herman (11 years ago)

Here are several ways to think about return:

  • A generator function is like a normal function but it can be paused. The act of pausing can send an intermediate value out to the caller (yield's argument) and the caller can send an intermediate value back in when resuming (yield's result). None of this changes the fact that, like ordinary functions, there are still arguments passed into the function and a result passed out. Refusing return values just breaks down this generalization.

  • A generator function produces an iterator object, which produces a record on each iteration that has a .value and a .done flag indicating whether the iteration is done. Refusing return values eliminates the .value field in this special case, making things less consistent.

Finally, task.js is just an example of building a control abstraction out of iterators. It happens that the for-of control flow form is imperative and doesn't have a use for a return value. That doesn't mean other control flow operations won't.

# Brendan Eich (11 years ago)

David Herman wrote:

Finally, task.js is just an example of building a control abstraction out of iterators. It happens that the for-of control flow form is imperative and doesn't have a use for a return value. That doesn't mean other control flow operations won't.

+1. The need for an affordance for some use-cases and lack of need for others does not undermine the affordance's value.

PEP-380 is worth a read, IMHO, for anyone who values Python's experience. You have to grok some history and custom jargon.