Check out Dart's iterators
This is like C#'s current / moveNext, which was discussed at the May 2011 TC39 meeting, and then again last September. See here
Rehashed Java (hasMore/getNext) and C# (Current/MoveNext) protocols involving two methods, which costs more for implementors (groups 2 and 3, however ordered) and adds incoherence hazard (two methods get out of sync; C# tries to mitigate common mistake with Java's, but still has dual out-of-sync hazard to Java's).
MM agreed with BE that Python's is simplest given other constraints we can't change, or "least bad".
Changing from hasMore/getNext to current/moveNext does not eliminate two methods that can get out of sync. You can imagine one is a property, not a method, but the general case is a getter or C#-style Current method.
-----Original Message----- From: Brendan Eich [mailto:brendan at mozilla.com] Sent: Sunday, February 10, 2013 03:20
Changing from hasMore/getNext to current/moveNext does not eliminate two methods that can get out of sync. You can imagine one is a property, not a method, but the general case is a getter or C#-style Current method.
Ah, the fact that it could be a getter does reduce it to the original two-out-of-sync-methods case, right.
FWIW, there continue to be strong misgivings about the pythonesqe design we have now, but Mozilla insists on the back of their shipping implementation. Many feel that exceptions for control-flow are a missdesign, myself included, but at this point the ship us nearly past the lighthouse on its way to sea and the effort involved in recalling it not worth the pain.
Le 10/02/2013 13:21, Alex Russell a ?crit :
FWIW, there continue to be strong misgivings about the pythonesqe design we have now, but Mozilla insists on the back of their shipping implementation. Many feel that exceptions for control-flow are a missdesign, myself included
I agree and also think return-true/false protocols aren't any better. In an ideal world, [idealworld] an extensible way to end a frame would be better for this kind of function-based protocols.
function(){
if(last())
return next();
else
throw StopIteration;
}
// would become
function(){
if(last())
return next();
else
endframe as StopIteration
}
Return and throw would be mere sugar for "endframe as return(value)" and "endframe untilcaught as throw(value)". untilcaught would indicate that this termination value propagates until being try-caught (though in my ideal world, there would be no throw, because I find it too agressive) What I'm describing here is nothing more than a generic mechanism to create new completion value types. I actually find fascinating that the SpiderMonkey debugger API completion value documentation has a specific note to explain how to recognize the end of an iterator frame.
In this ideal world, the iterator consumer story would be as follow:
// ES6 snippet:
try{
var value = it.next();
// code to manipulate the value
}
catch(e){
if(e instanceof StopIteration){
// code to run when out of elements
}
}
// would become:
var complValue = completion it.next()
if(complValue.type === 'return'){
// code playing with complValue.return;
}
if(complValue.type === 'StopIteration'){
// code to run when out of elements
}
// or something that looks more legit than the try/catch thing
The proposed "throw ForwardToTarget" would be nothing less than "endframe as ForwardToTarget" in this world.
In this ideal world, function protocols are based not on what a function released (return/throw value), but rather on how the function ended. [/idealworld]
But we do not live in the "endframe as"+"completion" world. "throw StopIteration" is probably as close as we can get in JavaScript given the 3 way to complete a frame that we have (return/throw/yield). If anything, it's very explicit about what it does ("stop iteration"). More than a return true/false protocol.
Maybe Dart could consider something like "endframe as"+"completion" though...
Alex Russell wrote:
FWIW, there continue to be strong misgivings about the pythonesqe design we have now,
See below for how "strong".
but Mozilla insists on the back of their shipping implementation.
No, that's completely false. There's no meeting notes to back you. If you objected all along, where was the record? Where was the alternative design? The C#-based idea from Peter Hallam, who was participating briefly in spring 2011, was never written up -- but it was hacked into early Traceur and then hacked out.
It's dis-harmonious to operate this way. You have a complaint, you bring it up at the meeting, it gets into the notes, we deal with restoring consensus.
For the umpteenth-time record, SpiderMonkey's ES4-era prototypes can and must change to adapt to ES6, whatever it ends up specifying.
The plain fact remains that no one in TC39, certainly not you, ever did the work or made the case for a better design, so the committee took iterators into ES6 consensus in May 2011. They remain there, with review as noted in the September 2012 notes (esdiscuss/2012-September/025241 and previoius in thread).
Many feel that exceptions for control-flow are a missdesign, myself included, but at this point the ship us nearly past the lighthouse on its way to sea and the effort involved in recalling it not worth the pain.
Are you really just bitching in es-discuss, uselessly and (see above about "Mozilla insists") inaccurately, so you can have the option to point the finger later?
Way to rise above inessential disagreement!
Le 10/02/2013 16:21, David Bruant a écrit :
Le 10/02/2013 13:21, Alex Russell a écrit :
FWIW, there continue to be strong misgivings about the pythonesqe design we have now, but Mozilla insists on the back of their shipping implementation.
I have made a mistake in keeping that part of the quote in my reply. I actually disagree with this statement.
Many feel that exceptions for control-flow are a missdesign, myself included
That's the only part I disagree with and my answer applied to.
I wrote:
But we do not live in the "endframe as"+"completion" world. "throw StopIteration" is probably as close as we can get in JavaScript given the 3 way to complete a frame that we have (return/throw/yield). If anything, it's very explicit about what it does ("stop iteration"). More than a return true/false protocol.
As I said at the end of my reply, "throw StopIteration" is probably the best thing that can be designed given the backward-compat constraints that JavaScript has, so I agree with Mozilla's implementation and bringing its design to ES6.
I apologize for the confusion.
Le 10/02/2013 16:50, David Bruant a écrit :
Le 10/02/2013 16:21, David Bruant a écrit :
Many feel that exceptions for control-flow are a missdesign, myself included That's the only part I disagree with and my answer applied to.
s/disagree/agree...
Just a couple of questions on this wonderous topic:
- how is 'e instanceof StopIteration' intended to work across multiple global objects?
- how firmly are we wedded to this? I can't imagine there is too much code that currently depends on manually catching StopIteration given ES6 is not finalized yet, and iterators aren't widely available.
I do dislike the exception based termination, I think i'd prefer next() and hasNext() style iteration over exceptions, especially given that for the most part these are hidden by clean syntax. My personal concern with all of these is how they deal with nested iterators.
I have continued my wanderings on that topic elsewhere. Sharing as food for thought:
Le 10/02/2013 16:21, David Bruant a écrit :
<idealworld>
I initially thought that yield could be the sugar of "endframe as yield(value)", but yield and return/throw are different. In the former case, the frame can (and likely will) be re-entered which is not the case for the latter. This begs for 2 different keywords. Let's call them "endframe" and "yield". yield could come in 2 forms: // generic yield as return(value) yield as throw(value) // which is impossible today?
// sugar
yield value
// which desugars naturally to
yield as return(value)
To re-enter a frame, the following could be used: reenter generator as return("yo") // equivalent of current generator.send('yo') reenter generator as throw("yo") // equivalent of current generator.throw('yo')
This raises an error if "generator" is not re-entrable (that is if it didn't end with yield or one of its sugar).
What is lost is the ability to pass around the send/throw/close/next functions. I would consider this a win. From what I've seen of generators, there is no loss. At least, task.js doesn't seem to pass these functions around.
Since every generator additional methods would be reimplemented with syntax, I think that having yield/reenter keywords (and additional sugar for usability), generators wouldn't need to be their own new function type. But that's in an ideal world, of course.
Le 10/02/2013 20:55, Oliver Hunt a écrit :
Just a couple of questions on this wonderous topic:
- how is 'e instanceof StopIteration' intended to work across multiple global objects?
StopIteration has a special "StopIteration" [[Brand]] [1], so the cross-global story shouldn't be a problem for the for-of loop. Exposing the brand can solve the problem for manual use of iterators. (you'd check if the object has a particular brand instead of "e instanceof StopIteration").
StopIteration could also be a deeply frozen constructor with same identity across globals.
- how firmly are we wedded to this? I can't imagine there is too much code that currently depends on manually catching StopIteration given ES6 is not finalized yet, and iterators aren't widely available.
I do dislike the exception based termination, I think i'd prefer next() and hasNext() style iteration over exceptions, especially given that for the most part these are hidden by clean syntax.
The "for the most part these are hidden by clean syntax" argument applies to throwing StopIteration too, no?
My personal concern with all of these is how they deal with nested iterators.
I don't see the concern. Can you provide a use case/code sample where nested iterators would be a problem?
I have to note that there is a minor security hazard in code using iterators naively: import process from "m";
var a = [1, 2, 3, 4, 5];
var next = 0;
var it = {
next: function(){
if(next < a.length){
// If the call to "process" throws StopIteration
because it's malicious/buggy, // so does this code and that's largely unexpected. return process(a[next++]); } else{ throw StopIteration; } } }
You can always protect yourself by wrapping the call to "process" with a try/catch block. I'm still on the side of preferring "throw StopIteration" for its better readability compared to "return false". Dart has "implements Iterator<T>" to help, but JavaScript doesn't.
David
On Feb 10, 2013, at 1:01 PM, David Bruant <bruant.d at gmail.com> wrote:
Le 10/02/2013 20:55, Oliver Hunt a écrit :
Just a couple of questions on this wonderous topic:
- how is 'e instanceof StopIteration' intended to work across multiple global objects? StopIteration has a special "StopIteration" [[Brand]] [1], so the cross-global story shouldn't be a problem for the for-of loop. Exposing the brand can solve the problem for manual use of iterators. (you'd check if the object has a particular brand instead of "e instanceof StopIteration").
StopIteration could also be a deeply frozen constructor with same identity across globals.
- how firmly are we wedded to this? I can't imagine there is too much code that currently depends on manually catching StopIteration given ES6 is not finalized yet, and iterators aren't widely available.
I do dislike the exception based termination, I think i'd prefer next() and hasNext() style iteration over exceptions, especially given that for the most part these are hidden by clean syntax. The "for the most part these are hidden by clean syntax" argument applies to throwing StopIteration too, no?
Yes, but exception handling for control flow is generally problematic, and you end up with things like:
function foo() { try { ... yield ... ... } catch(...) { } }
Maybe this will be caught, maybe it won't. If we say that yield acts as an exception (eg. manual iteration is performed via catch()) then the intuition is that the exception handler will break yield.
You get other things like manual generator usage being astonishingly ugly:
while (true) { try { value = generator.next(); ... } catch (e instanceof StopIteration) { break; } }
vs.
while (generator.hasNext()) { value = generator.next(); ... }
If we're not too concerned about the syntax of using generators manually, then I'd just say make it a non-goal. You can pass a generator around but you can only iterate using iteration syntax. This would allow implementations to use whatever behaviour works most efficiently for them internally, without exposing it to developers.
The more I think about this, the more I like it. Let's assume you can't directly call the iterator - for(of), comprehensions, etc work.
If it becomes sufficiently important to someone to make it "callable" then they can do:
Iterator.prototype.next = function() { for (value of this) return value; throw StopIterator; // Or return magic value, etc }
My personal concern with all of these is how they deal with nested iterators. I don't see the concern. Can you provide a use case/code sample where nested iterators would be a problem?
You know what? I don't think I actually care. I think I was swayed by a random blog post about how nested iterators suck :D - i was thinking of things like n-ary tree walkers and realised that most "solutions" end up being hideous crimes against code anyway :D
On Sun, Feb 10, 2013 at 1:55 PM, Oliver Hunt <oliver at apple.com> wrote:
I do dislike the exception based termination, I think i'd prefer next() and hasNext() style iteration over exceptions, especially given that for the most part these are hidden by clean syntax.
Yuck. Python's iterator protocol may be a bit ugly to consume, but it is nicer to implement than next/hasNext. (moveNext/current also seems a little easier to implement than next/hasNext.)
It turns out (at least in Python) that while you rarely have to directly implement the raw iterator protocol, you seriously almost never have to directly consume it. So it was the right design decision, in Python at least, to optimize the protocol for ease of use on the implementing side.
For example. In Python, file objects are iterators. It works like this (only actually this is implemented in C, not Python):
class file:
...
def __iter__(self):
return self
def next(self):
line = self.readline()
if line == '':
throw StopIteration
return line
...
It's nice that the iterator protocol does not require the iterator to store the value across method calls, from hasNext() to next(). It's kind of nice, generally, not to have to worry, in the implementation of hasNext(), about whether or not next() was called since the last time hasNext() was called.
Probably all this is down in the noise though.
My personal concern with all of these is how they deal with nested
iterators.
I see you've already taken this back, but here's what my answer would've been:
def iter_tree(node):
if node is not None:
for v in node.left:
yield v
yield node.value
for v in node.right:
yield v
The low-level protocol just doesn't enter into it. The real APIs for working with Python iterators are yield, for-in, and itertools.
On Feb 10, 2013, at 1:55 PM, Jason Orendorff <jason.orendorff at gmail.com> wrote:
On Sun, Feb 10, 2013 at 1:55 PM, Oliver Hunt <oliver at apple.com> wrote: I do dislike the exception based termination, I think i'd prefer next() and hasNext() style iteration over exceptions, especially given that for the most part these are hidden by clean syntax.
Yuck. Python's iterator protocol may be a bit ugly to consume, but it is nicer to implement than next/hasNext. (moveNext/current also seems a little easier to implement than next/hasNext.)
I'm fine with moveNext+current - i was simply meaning two functions are superior to one that may or may not throw, and may or may not be throwing something that may or may not be generator related.
It turns out (at least in Python) that while you rarely have to directly implement the raw iterator protocol, you seriously almost never have to directly consume it. So it was the right design decision, in Python at least, to optimize the protocol for ease of use on the implementing side.
Sorry, I don't follow what you're saying here - which implementer? the language implementer or the developer? using the python example below your iterator would be:
function myCustomGenerator() { while (line = this.readline()) yield line }
The runtime then will ensure correct semantics.
Le 10 févr. 2013 à 22:01, David Bruant <bruant.d at gmail.com> a écrit :
<snip>
I have to note that there is a minor security hazard in code using iterators naively: import process from "m";
var a = [1, 2, 3, 4, 5]; var next = 0; var it = { next: function(){ if(next < a.length){ // If the call to "process" throws StopIteration because it's malicious/buggy, // so does this code and that's largely unexpected. return process(a[next++]); } else{ throw StopIteration; } } }
You can always protect yourself by wrapping the call to "process" with a try/catch block. <snip>
Note that the same issue arises with generators:
function* gen(a) {
var next = 0;
if (next < a.length) {
// the iterator will end prematurely if "process" throws a StopIteration
yield process(a[next++]);
}
}
In order to mitigate the problem, instead of throwing a generic StopIteration, I think we ought to throw a specific StopIteration instance with information on which iterator has thrown. More precisely, inside a generator function, a return statement will throw a StopIteration instance with its "source" property set to the generator iterator which was terminated.
For manually throwing a StopIteration from inside a "next" method of an iterator, we could use:
throw new StopIteration(this)
And instead of "e instanceof StopIteration", we may use a more precise check:
e instanceof StopIteration && e.source === it
Has the iteration protocol been given a close look since unique symbols hit the scene? I know the iterator key was changed to a symbol (which makes great sense) but ISTM symbols offer up some design flexibility that, as far as I know, just isn't possible in any of the other languages mentioned WRT prior art.
For example, the first argument to the iterator function could be a symbol
that acts as a StopIteration semaphore for that iteration run. Obviously
consuming an iterator with for of or comprehensions would implement the
protocol for you (generating the symbol and testing it's identity under the
covers), but it'd still be just as easy (perhaps more so) to run off an
iterator run manually. If you happen know the full alphabet of the sequence
you're iterator over you don't even have to use a symbol. I think it's safe
to say very few sequences won't have holes, so providing nothing at all
would do the Right Thing in the vast majority of cases.
I know it's too late in the game for new features. Though, this is really a small tweak to the current proposal -- it's essentially the same protocol with a different signaling mechanism. There sure seem to be an awful lot of pros. It's easier to get right than the throw StopIteration dance, and I couldn't imagine a way it could be less efficient. AFAICT it's not subject to the deficiencies I've seen raised about the current spec -- no instanceof StopIteration confusion and no unnecessary catch blocks, for instance. I'm not aware of any cons.
This is exactly what's proposed for ES6, except s/.source/.value/. See also PEP-380.
Dean Landolt wrote:
Has the iteration protocol been given a close look since unique symbols hit the scene?
Previously: esdiscuss/2012-November/026647
Domenic Denicola wrote:
-----Original Message----- From: Brendan Eich [mailto:brendan at mozilla.com] Sent: Sunday, February 10, 2013 03:20
Changing from hasMore/getNext to current/moveNext does not eliminate two methods that can get out of sync. You can imagine one is a property, not a method, but the general case is a getter or C#-style Current method.
Ah, the fact that it could be a getter does reduce it to the original two-out-of-sync-methods case, right. Thanks!
Rather, current must be a getter (or really, a method). The iteration protocol abstracts over all implementations using structural type(s), where the signature(s) must be the same for all iterators. Not all iterators want to expose a writable data property, .current.
Or did you mean a non-configurable, non-writable data property named current that is updated by moveNext? That would make all iterators be exotic objects, which is a non-starter.
From: Brendan Eich [mailto:brendan at mozilla.com]
Domenic Denicola wrote:
-----Original Message----- From: Brendan Eich [mailto:brendan at mozilla.com] Sent: Sunday, February 10, 2013 03:20
Changing from hasMore/getNext to current/moveNext does not eliminate two methods that can get out of sync. You can imagine one is a property, not a method, but the general case is a getter or C#-style Current method.
Ah, the fact that it could be a getter does reduce it to the original two-out-of-sync-methods case, right. Thanks!
Rather, current must be a getter (or really, a method). The iteration protocol abstracts over all implementations using structural type(s), where the signature(s) must be the same for all iterators. Not all iterators want to expose a writable data property, .current.
Or did you mean a non-configurable, non-writable data property named current that is updated by moveNext? That would make all iterators be exotic objects, which is a non-starter.
I don't see why all iterators would have to have the same "kind" of property descriptor? I.e. if one iterator wanted a getter, and others wanted a writable, configurable data property, what's the problem there? It seems your structural type(s)/signature(s) comment is heading in this direction, but I don't understand why.
Le 11 févr. 2013 à 06:25, Brendan Eich <brendan at mozilla.com> a écrit :
This is exactly what's proposed for ES6, except s/.source/.value/. See also PEP-380.
/be
By reading [1] (and PEP-380), it seems to me that the "value" property is rather set to be the value of the return statement. Thus, to be more complete and to avoid confusion: Inside a generator function, a return statement will throw a StopIteration instance with its "value" property set to the value of the return statement and its "source" property set to the generator iterator which is being terminated. You can create manually such a StopIteration with: new StopIteration(source, value)
Claude Pache wrote:
Le 11 févr. 2013 à 06:25, Brendan Eich<brendan at mozilla.com> a écrit :
This is exactly what's proposed for ES6, except s/.source/.value/. See also PEP-380.
/be
By reading [1] (and PEP-380), it seems to me that the "value" property is rather set to be the value of the return statement.
Yes; also the StopIteration constructor's parameter for explicit constructions.
Thus, to be more complete and to avoid confusion: Inside a generator function, a return statement will throw a StopIteration instance with its "value" property set to the value of the return statement and its "source" property set to the generator iterator which is being terminated. You can create manually such a StopIteration with: new StopIteration(source, value)
Is 'source' worth its weight by default? Python manages without.
Le 11 févr. 2013 à 03:44, Dean Landolt <dean at deanlandolt.com> a écrit :
Has the iteration protocol been given a close look since unique symbols hit the scene? I know the iterator key was changed to a symbol (which makes great sense) but ISTM symbols offer up some design flexibility that, as far as I know, just isn't possible in any of the other languages mentioned WRT prior art.
For example, the first argument to the iterator function could be a symbol that acts as a StopIteration semaphore for that iteration run. Obviously consuming an iterator with
for ofor comprehensions would implement the protocol for you (generating the symbol and testing it's identity under the covers), but it'd still be just as easy (perhaps more so) to run off an iterator run manually. If you happen know the full alphabet of the sequence you're iterator over you don't even have to use a symbol. I think it's safe to say very few sequences won't have holes, so providing nothing at all would do the Right Thing in the vast majority of cases.I know it's too late in the game for new features. Though, this is really a small tweak to the current proposal -- it's essentially the same protocol with a different signaling mechanism. There sure seem to be an awful lot of pros. It's easier to get right than the throw StopIteration dance, and I couldn't imagine a way it could be less efficient. AFAICT it's not subject to the deficiencies I've seen raised about the current spec -- no instanceof StopIteration confusion and no unnecessary catch blocks, for instance. I'm not aware of any cons.
So, if I understand your idea, a custom iterator would be of the following form:
myIterator = { next: function(stopIterationSemaphore) {
// instead of throw StopIteration, you should write:
return stopIterationSemaphore
}}
The idea pleased me, but I see one problem: inside a generator, the return statement may have a value, which is significant when you use delegation, since the value of the return statement of the delegated generator becomes the value of the yield* expression of the delegating generator. Now if you return just a semaphore-symbol, you have no way to transmit the value of the return statement.
In order to resolve the issue, you could use a fresh object instead of a symbol (and it will be as unique as any symbol):
myIterator = { next: function(stopIterationSemaphore) {
// instead of throw StopIteration(value), you should write:
stopIterationSemaphore.value = value
return stopIterationSemaphore
}}
and
stopIterationSemaphore = {}
result myIterator.next(stopIterationSemaphore)
if (result === stopIterationSemaphore) {
// the iteration is over.
// and the return value is stopIterationSemaphore.value
}On Sun, Feb 10, 2013 at 4:05 PM, Oliver Hunt <oliver at apple.com> wrote:
It turns out (at least in Python) that while you rarely have to directly implement the raw iterator protocol, you seriously almost never have to directly consume it. So it was the right design decision, in Python at least, to optimize the protocol for ease of use on the implementing side.
Sorry, I don't follow what you're saying here - which implementer? the language implementer or the developer?
Developer.
using the python example below your iterator would be:
function myCustomGenerator() { while (line = this.readline()) yield line }
Agreed. The raw iterator protocol only ever matters if you're trying to avoid the generator overhead or something.
On Feb 10, 2013, at 10:40 PM, Domenic Denicola wrote:
From: Brendan Eich [mailto:brendan at mozilla.com]
Domenic Denicola wrote:
-----Original Message----- From: Brendan Eich [mailto:brendan at mozilla.com] Sent: Sunday, February 10, 2013 03:20
Changing from hasMore/getNext to current/moveNext does not eliminate two methods that can get out of sync. You can imagine one is a property, not a method, but the general case is a getter or C#-style Current method.
Ah, the fact that it could be a getter does reduce it to the original two-out-of-sync-methods case, right. Thanks!
Rather, current must be a getter (or really, a method). The iteration protocol abstracts over all implementations using structural type(s), where the signature(s) must be the same for all iterators. Not all iterators want to expose a writable data property, .current.
Or did you mean a non-configurable, non-writable data property named current that is updated by moveNext? That would make all iterators be exotic objects, which is a non-starter.
I don't see why all iterators would have to have the same "kind" of property descriptor? I.e. if one iterator wanted a getter, and others wanted a writable, configurable data property, what's the problem there? It seems your structural type(s)/signature(s) comment is heading in this direction, but I don't understand why.
To me, Domenic appears correct on this point. The difference between a "method" (a function valued property that is intended to be explicitly called) and a "state property" (not explicitly called, might be implemented as either a accessor or data property) is the only thing that matters at the signature description level.
Also, a non-configurable, non-writable by clients property doesn't require an exotic object. It can be implemented using a get accessor and some private state (pick you favorite mechanism). Iterators must have other private state to operate so a private "current" doesn't seem like it would be much of an added burden.
Finally, I don't see why we would even worry about a client modifying .current. So what if a client of an iterator (of the moveNext/current model) modifies current. What are we afraid of? That somebody is going to modify current and a few lines later forget that they did so? Seems like a mundane client bug, nothing that needs to be protected against.
Finally, why the big concern about out of sync iterator methods. Being out of sync is just a bug, and one that is not likely to go undiscovered. Why is this any worse than any other object with multiple methods that are supposed to present a consistent view of some state? This isn't something that we generally worry about, so why is it an issue here?
On Feb 11, 2013, at 8:54 AM, Jason Orendorff <jason.orendorff at gmail.com> wrote:
On Sun, Feb 10, 2013 at 4:05 PM, Oliver Hunt <oliver at apple.com> wrote:
It turns out (at least in Python) that while you rarely have to directly implement the raw iterator protocol, you seriously almost never have to directly consume it. So it was the right design decision, in Python at least, to optimize the protocol for ease of use on the implementing side.
Sorry, I don't follow what you're saying here - which implementer? the language implementer or the developer?
Developer.
using the python example below your iterator would be:
function myCustomGenerator() { while (line = this.readline()) yield line }
Agreed. The raw iterator protocol only ever matters if you're trying to avoid the generator overhead or something.
Saving the cost of "generator overhead" by making all iteration technically slower by depending on exceptions (correct semantics mean even optimising out the throw is tricky for default iteration generators).
Additionally get the correct behaviour for StopIterator is non-trivial as you aren't simply doing "throw StopIterator" you have to do "throw StopIterator(magic value that you know consumer will check for)"
For now I would say that we shouldn't expose the internal implementation behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7
From: es-discuss-bounces at mozilla.org [es-discuss-bounces at mozilla.org] on behalf of Oliver Hunt [oliver at apple.com]
For now I would say that we shouldn't expose the internal implementation behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7
Would this prevent TaskJS from working in ES6?
If so, I'm so sorry I started this thread, I take it all back!!!!!! :P
On Feb 11, 2013, at 10:35 AM, Domenic Denicola <domenic at domenicdenicola.com> wrote:
From: es-discuss-bounces at mozilla.org [es-discuss-bounces at mozilla.org] on behalf of Oliver Hunt [oliver at apple.com]
For now I would say that we shouldn't expose the internal implementation behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7
Would this prevent TaskJS from working in ES6?
If so, I'm so sorry I started this thread, I take it all back!!!!!! :P
I'm not sure what you're talking about here - if a framework is using for(of) or yield they shouldn't need to create custom generators (at a very basic overview a JS implementation of pseudo yield is unlikely to be able to beat what the JS engine+runtime will be able to do), and if you're unable to use yield or for(of) due to backwards compat concerns then you are already implementing custom generator like behaviour that has no engine/runtime/syntax dependencies.
Domenic Denicola wrote:
From: Brendan Eich [mailto:brendan at mozilla.com]
Domenic Denicola wrote:
-----Original Message----- From: Brendan Eich [mailto:brendan at mozilla.com] Sent: Sunday, February 10, 2013 03:20 Changing from hasMore/getNext to current/moveNext does not eliminate two methods that can get out of sync. You can imagine one is a property, not a method, but the general case is a getter or C#-style Current method. Ah, the fact that it could be a getter does reduce it to the original two-out-of-sync-methods case, right. Thanks! Rather, current must be a getter (or really, a method). The iteration protocol abstracts over all implementations using structural type(s), where the signature(s) must be the same for all iterators. Not all iterators want to expose a writable data property, .current.
Or did you mean a non-configurable, non-writable data property named current that is updated by moveNext? That would make all iterators be exotic objects, which is a non-starter.
I don't see why all iterators would have to have the same "kind" of property descriptor? I.e. if one iterator wanted a getter, and others wanted a writable, configurable data property, what's the problem there?
That might be ok, depending on point of view -- ES5 makes the difference observable. But I agree it would be strange to come to depend on it.
My point was more that a writable configurable data property is an anti-pattern for defensively coded iterators.
It seems your structural type(s)/signature(s) comment is heading in this direction, but I don't understand why.
Hope that helps,
Allen Wirfs-Brock wrote:
To me, Domenic appears correct on this point. The difference between a "method" (a function valued property that is intended to be explicitly called) and a "state property" (not explicitly called, might be implemented as either a accessor or data property) is the only thing that matters at the signature description level.
Maybe, but the difference is observable, and a data property has a defensive-coding smell.
Also, a non-configurable, non-writable by clients property doesn't require an exotic object. It can be implemented using a get accessor
I specified data property, after allowing for the getter alternative. Let's not go in circles.
and some private state (pick you favorite mechanism). Iterators must have other private state to operate so a private "current" doesn't seem like it would be much of an added burden.
Are you actually trying to change the accepted ES6 proposal here?
"doesn't seem like it would be much of an added burden..." -- hmm, where have I heard that before.
"It's not much of a problem" === "It's a problem."
Finally, I don't see why we would even worry about a client modifying .current. So what if a client of an iterator (of the moveNext/current model) modifies current. What are we afraid of? That somebody is going to modify current and a few lines later forget that they did so? Seems like a mundane client bug, nothing that needs to be protected against.
In a single-trust-domain small-world code network, sure -- whatever.
In a programming-in-the-large setting, a writable data property is inviting Murphy's Law. I'm not talking about security in a mixed-trust environment specifically. Large programs become "mixed trust", even when it's just me, myself, and I (over time) hacking the large amount of code.
Finally, why the big concern about out of sync iterator methods. Being out of sync is just a bug, and one that is not likely to go undiscovered. Why is this any worse than any other object with multiple methods that are supposed to present a consistent view of some state? This isn't something that we generally worry about, so why is it an issue here?
We've been over this, it's in the meeting notes and the es-discuss post that I linked to up thread. I get tired of re-searching and re-posting these links. You should get tired of reading them, for some balance :-|.
The problem of out-of-sync doesn't exist with ES6 iterators or Python iterators. More, the bigger point to make is that developers who do code implementors don't have to write two methods, as Jason pointed out.
Oliver Hunt wrote:
Saving the cost of "generator overhead" by making all iteration technically slower by depending on exceptions (correct semantics mean even optimising out the throw is tricky for default iteration generators).
What is slower?
Domenic Denicola wrote:
From: es-discuss-bounces at mozilla.org [es-discuss-bounces at mozilla.org] on behalf of Oliver Hunt [oliver at apple.com]
For now I would say that we shouldn't expose the internal implementation behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7
Would this prevent TaskJS from working in ES6?
Sorry, we are not going to do that.
Oliver was around at at least one of the meetings where iterators got consensus or kept it. He's been AWOL enough that I'm going to say right here that he doesn't get to throw a veto.
If so, I'm so sorry I started this thread, I take it all back!!!!!! :P
Yeah, what was your point?
It's fine if people want to bitch about iterators or StopIteration. It's not like es-discuss has enough noise, right?
But where is the stand-up work on a better proposal, prototype-implemented in multiple engines and sold and re-sold to TC39, achieving recorded consensus? It's not there. So keep bitching, but don't expect ES6 to change without you or someone else actually putting in the sweat equity.
On Mon, Feb 11, 2013 at 12:30 PM, Oliver Hunt <oliver at apple.com> wrote:
Saving the cost of "generator overhead" by making all iteration technically slower by depending on exceptions (correct semantics mean even optimising out the throw is tricky for default iteration generators).
We are epicly talking past each other in this thread, but if you're saying that the raw iterator protocol will rarely be used by developers, no argument.
For now I would say that we shouldn't expose the internal implementation
behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7
Can you be more concrete? Generator-iterators must at least be first-class values that can be created and then passed around.
On Feb 11, 2013, at 1:29 PM, Brendan Eich <brendan at mozilla.com> wrote:
Domenic Denicola wrote:
From: es-discuss-bounces at mozilla.org [es-discuss-bounces at mozilla.org] on behalf of Oliver Hunt [oliver at apple.com]
For now I would say that we shouldn't expose the internal implementation behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7
Would this prevent TaskJS from working in ES6?
Sorry, we are not going to do that.
I still don't know what is being talked about here :D
Oliver was around at at least one of the meetings where iterators got consensus or kept it. He's been AWOL enough that I'm going to say right here that he doesn't get to throw a veto.
So what you're saying is that people who attend in person meetings are able to veto proposals, and input from people who don't attend those meetings don't, and get much less input into the standard? That's a fairly closed approach, especially when compared to more or less every other web standard.
People aren't required to attend TPAC to have input at least considered for inclusion in W3 specs.
On Feb 11, 2013, at 1:33 PM, Jason Orendorff <jason.orendorff at gmail.com> wrote:
On Mon, Feb 11, 2013 at 12:30 PM, Oliver Hunt <oliver at apple.com> wrote:
For now I would say that we shouldn't expose the internal implementation behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7
Can you be more concrete? Generator-iterators must at least be first-class values that can be created and then passed around.
I was meaning you don't have the ability to directly executing a generator - you can pass it around, hang properties of it, etc but not directly trigger enumeration
so you couldn't go generator.next() or generator() or whatever, but you could go for (... of generator) or [ ... generator], etc
Oliver Hunt wrote:
On Feb 11, 2013, at 1:29 PM, Brendan Eich<brendan at mozilla.com> wrote:
Domenic Denicola wrote:
From: es-discuss-bounces at mozilla.org [es-discuss-bounces at mozilla.org] on behalf of Oliver Hunt [oliver at apple.com]
For now I would say that we shouldn't expose the internal implementation behaviour of yield (which is what being able to explicitly create or call a generator produces). That fairly trivially resolves the StopIteration behaviour by pushing it out of the ES6 spec. If there's enough demand for manually creating or 'calling' a generator then we can fix it in ES6.x/7 Would this prevent TaskJS from working in ES6? Sorry, we are not going to do that.
I still don't know what is being talked about here :D
That makes all of us :-P.
Oliver was around at at least one of the meetings where iterators got consensus or kept it. He's been AWOL enough that I'm going to say right here that he doesn't get to throw a veto.
So what you're saying is that people who attend in person meetings are able to veto proposals,
No, I didn't say that. You are good at logic, I've seen your code. I know you grok P -> Q does not imply !P -> !Q.
and input from people who don't attend those meetings don't, and get much less input into the standard? That's a fairly closed approach, especially when compared to more or less every other web standard.
We've been open on this stuff for years. Seven, by my count. You've been around enough of this, in person or on es-discuss, to participate.
People aren't required to attend TPAC to have input at least considered for inclusion in W3 specs.
They don't get to throw unilaterla vetos there either.
On Feb 11, 2013, at 1:58 PM, Brendan Eich <brendan at mozilla.com> wrote:
Oliver Hunt wrote:
So what you're saying is that people who attend in person meetings are able to veto proposals,
No, I didn't say that. You are good at logic, I've seen your code. I know you grok P -> Q does not imply !P -> !Q.
It seems reasonable to interpret your statement as saying that vetos are possible, but they aren't available to people unless they attend meetings.
Or are you saying that vetos aren't possible in the meetings?
People aren't required to attend TPAC to have input at least considered for inclusion in W3 specs.
They don't get to throw unilaterla vetos there either.
I'm not actually sure where I threw a veto.
Oliver Hunt wrote:
On Feb 11, 2013, at 1:58 PM, Brendan Eich<brendan at mozilla.com> wrote:
Oliver Hunt wrote:
So what you're saying is that people who attend in person meetings are able to veto proposals, No, I didn't say that. You are good at logic, I've seen your code. I know you grok P -> Q does not imply !P -> !Q.
It seems reasonable to interpret your statement as saying that vetos are possible, but they aren't available to people unless they attend meetings.
Nope, not reasonable -- a fallacy as diagrammed!
Or are you saying that vetos aren't possible in the meetings?
People in the meetings try not to die on a hill (or throw themselves in front of a train) unless they view it as vitally important. Given all the joint work, it's not a situation where someone will lightly (without participating regularly and contributing blood, sweat, and tears, as noted) say "veto!"
In the U.S. system, the CEO (that's the President) can veto, explicitly or implicitly (pocket veto), but again with some political care and restraint (could overdo it and get unpopular; could get overridden).
TC39 doesn't have a CEO, though. We all try to keep consensus, and the Chair tries to keep everyone making "progress". If this takes us up a suboptimal hill because someone (you?) wasn't there and engaged enough to take us up a better hill, that's the price of standardization.
BTW this is true of every standards body and indeed every collaborative work environment, ever.
People aren't required to attend TPAC to have input at least considered for inclusion in W3 specs. They don't get to throw unilateral vetos there either.
I'm not actually sure where I threw a veto.
I'm glad you wrote that!
Brendan Eich wrote:
More, the bigger point to make is that developers who do code implementors
"iterators", of course.
don't have to write two methods, as Jason pointed out.
/be (fat fingers)
I know
StopIterationis pretty baked in by now. But, most of the arguments I can recall for it are that it?s better thanhasNext+next, wherein you have to keep two methods in sync. I saw the Dart iterator API the other day and it provided a third alternative I thought the list might enjoy contemplating:www.dartlang.org/articles/m3-whats-new/iterables.html#iterator
Translated to ES6, an iterable that counts to 5 it would look like:
import { current, next } from "@iter"; let iterable = { [next]() { if (this[current] < 5) { this[current]++; } return this[current] <= 5; }, [current]: 0 };That is,
nextdoes all the work, returningtrue/falseto signal whether the iteration should proceed. It then keepscurrentin sync with the current value.Interesting, if perhaps too late for ES6. I'd of course love to hear why
StopIterationis better than this, so I can understand and defend it better.I know `StopIteration` is pretty baked in by now. But, most of the arguments I can recall for it are that it’s better than `hasNext` + `next`, wherein you have to keep two methods in sync. I saw the Dart iterator API the other day and it provided a third alternative I thought the list might enjoy contemplating: https://www.dartlang.org/articles/m3-whats-new/iterables.html#iterator Translated to ES6, an iterable that counts to 5 it would look like: import { current, next } from "@iter"; let iterable = { [next]() { if (this[current] < 5) { this[current]++; } return this[current] <= 5; }, [current]: 0 }; That is, `next` does all the work, returning `true`/`false` to signal whether the iteration should proceed. It then keeps `current` in sync with the current value. Interesting, if perhaps too late for ES6. I'd of course love to hear why `StopIteration` is better than this, so I can understand and defend it better.