Discuss
  • A syntax alternative for generators (was: Generator issue: exceptions while initializing arguments)

A syntax alternative for generators (was: Generator issue: exceptions while initializing arguments)

# Allen Wirfs-Brock (13 years ago)

The initializing generator arguments thread and related discussions has got me questioning our use of function declaration/expression syntax as our basis for defining generators. Changing this would be a departure from what Firefox has supported in recent years, but in ES6 we have more constructs to build upon than FF did in 2006. The conclusion I reach is that it might be better to build generator definition upon a foundation of arrow functions rather than classic JS function definitions. What follow is an attempt to sketch the thought process that lead to that conclusion:

On Sep 9, 2012, at 11:32 PM, Brendan Eich wrote:

function dataSnapshot(aCollection) { let snapshot = aCollection.clone(); return function*() { for (let i = 0; i < snapshot.length; i++){ yield snapshot[i]; } }(); }

Issues: If a generator needs to capture creation time state it needs to be wrapped in a factory function/method, similar to Brendan's example above. If the factory is a method the wrapping creates confusion about this/super binding. For example: class MyClass { dataSnapshot(aCollection) { let snapshot = aCollection.clone(); return function*() { for (let i = 0; i < snapshot.length; i++){ yield thls.doSomething(snapshot[i]); //this is not what was probably expected, super would be illegal } }(); } }

Generator function definitions need to be called to create an actual iterator object (a generator instance) It is likely that a common error will be forgetting the invocation of the generator in return statements such as the above. There is a potential for confusion about the evaluation time of generator parameter default value expression. If a parameterized generator needs default value initialization it is probably better practice to accomplish that via a wrapper factory: function IteratorCol(col = CollectionManager.default()) { return function*() {for (let i = 0; i < col.length; i++) yield col[i]}(); }

is argubaly clearer and more likely to be correct than

function *IteratorCol(col = CollectionManager.default()) { for (let i = 0; i < col.length; i++) yield col[i]; } But the wrapper form is less concise than the unwrappered form. Concise method declarations for generators are misleading about the interface of the containing class/object because they place emphasis on an implementation detail (use of a generator to provide an Iterator implementation) rather than the more important fact that the method returns a object that can be used as an iterator.

A different approach: eliminate formal parameters from generator definitions -- if arguments are needed use a wrapper function/method eliminate the need to call a generator before before using it as an iterator. Generator definitions create generator instances rather than generator constructor functions. use lexical this/super binding within generator definitions base generator definition syntax off of arrow function syntax rather than function definition syntax eliminate concise generator methods

Examples:

function dataSnapshot(aCollection) { let snapshot = aCollection.clone(); return *=>{ for (let i = 0; i < snapshot.length; i++){ yield snapshot[i]; } }; }

or more concisely:

function dataSnapshot(aCollection) { let snapshot = aCollection.clone(); return *=>for (let i = 0; i < snapshot.length; i++) yield snapshot[i]; }

as a method:

class MyClass { dataSnapshot(aCollection = CollectionManager.default()) { let snapshot = aCollection.clone(); return *=>for (let i = 0; i < this.length; i++) yield this[i]; } }

Proposal summary: Generator literals -- Base generators syntax on arrow functions rather than function declarations/expressions lexical this/super expression or curly body Generator literals don't have formal parameters Generators aren't called -- generator literals create generator instances (a kind of iterator) when evaluated No concise generator methods -- use a concise method returning a generator literal

# Jason Orendorff (13 years ago)

I'd appreciate a less contrived motivating example. The best way to implement this dataSnapshot() function is simply:

function dataSnapshot(aCollection) {
    return aCollection.clone();
}
# Allen Wirfs-Brock (13 years ago)

On Sep 11, 2012, at 7:48 PM, Jason Orendorff wrote:

I'd appreciate a less contrived motivating example. The best way to implement this dataSnapshot() function is simply:

function dataSnapshot(aCollection) { return aCollection.clone(); }

-j

My original example was something like:

function *dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); for (let i=0; i<snapshot.length; i++) yield snapshot[i] }

It wasn't actually quite as intentionally clear as the above and that allowed it to evolve within the thread into something that could arguably be restated as you show. But that was why I created the original example.

The intent of the original was to show a situation where:

there is a dynamically changing data source you want to analyze You need to capture it for the analysis while it is in a stable state The data capture process does not produce a "clone" but a different (perhaps more efficient for readonly access) You know how to navigate this captured data structure and want to provide a generator-based iteration that a analysis process can you to access its elements You need to be sure that the data is captured when at the exact point in program where "dataSnapshot" was invoked, not at some arbietrary later time when "next" is first called.

I think these requirements are pretty typical of a common situation that occurs when you want to perform stable traversal over dynamically mutable object structures and the example was contrived to show some pitfalls that could occur if you try to do this using a generator.

The original example fails for at least the last bullet. To fix that, it would have to be rewritten, something like:

function dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); return function *() { for (let i=0; i<snapshot.length; i++) yield snapshot[i] } }

Note that this rewrite eliminates the need for an formal parameters on the actual generator definition.

If dataSnapshot was a method on some object and the iteration results depended upon that object, you might have instead originally coded it as:

class DataSourceAnalyzer { ... //a constructor and other methods *dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); for (let i=0; i<snapshot.length; i++) yield this.analyzeElement(snapshot[i]); // <--- note use of this } }

This also falls the last requirement in the bullet list above. So, you might refactor similarly to what I did for the non-method form:

class DataSourceAnalyzer { ... //a constructor and other methods dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); return function *() { for (let i=0; i<snapshot.length; i++) yield this.analyzeElement(snapshot[i]); // <--- note erroneous use of this } } }

and forget that the function * expression uses a dynamic this. If you remembered that you would instead have had to revert to ES<6 style this capture:

class DataSourceAnalyzer { ... //a constructor and other methods dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); let self = this; return function *() { for (let i=0; i<snapshot.length; i++) yield self.analyzeElement(snapshot[i]); // <--- note use of self instead of this } } }

Using my alternative generator syntax, this would be written as:

class DataSourceAnalyzer { ... //a constructor and other methods dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); return *=> for (let i=0; i<snapshot.length; i++) yield this(snapshot[i]); // <--- note valid of lexical this } }

# Allen Wirfs-Brock (13 years ago)

oops, I forgot to force calls to the inner function * definitions. Another hazard of the plan of record that my alternative proposal doesn't have. I fixed occurrence of this bug below:

# Jason Orendorff (13 years ago)

Allen, even with all the requirements you listed, I wouldn't write that example code. There's a more natural way to factor the problem.

The original example fails for at least the last bullet. To fix that, it would have to be rewritten, something like:

function dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); return function *() { for (let i=0; i<snapshot.length; i++) yield snapshot[i] } }

The algorithm for walking the snapshot should be a method of the snapshot object:

DataSnapshot.prototype[iterator] = function *() {
    for (let i = 0; i < this.length; i++)
        yield this[i];
};

Then dataSnapshot could be written in a straightforward way; but in practice you wouldn't bother, because the function isn't doing anything useful anymore.

Now for the example using 'this':

class DataSourceAnalyzer { ... //a constructor and other methods *dataSnapshot(aDataSource) { let snapshot = aDataSource.captureCurrentState(); for (let i=0; i<snapshot.length; i++) yield this.analyzeElement(snapshot[i]); // <--- note use of this } }

I would write:

*dataSnapshot(aDataSource) {
    let snapshot = aDataSource.captureCurrentState();
    return [this.analyzeElement(e) for (e of snapshot)];
}

The behavior is a little different; each element is analyzed eagerly instead of lazily. That's ordinarily OK; computers are fast. But if not, you can factor this function into two parts, eager and lazy. Or, assuming 'this' propagates into generator-expressions, you could just use parentheses instead of square brackets around the comprehension.

The additional requirements make the example seem more contrived, not less; and yet even with all that, we're looking at a two-line function under the current proposal, and the pitfalls never appeared. This is what my experience with these features has been like in Python.

When generators are used for iteration, the loc1/loc2 distinction rarely matters in practice. The generators are typically called from the RHS of a for-loop, often implicitly, and so the first iter.next() call comes immediately. loc1 and loc2 are effectively the same.

I think Dave Herman once pursued alternate syntax something like this. What did he conclude?