Array extras functionality for iterators
Sorry for the resend. Meant to include the list.
I like this idea a lot! However, what would be the correct behavior of a method like 'every' on an infinite generator?
-Michael A. Smith
I'd say every and other methods like that should take another argument that specifies the number if times to test.
You could say that every returns:
- true if the generator is finite and the function always returned true
- false if one of the finite (either because the generator is or because a number was given as argument) number of calls of the function returned false
- undefined if the generator is infinite and none of the finite number of calls returned false
OR
return a generator that tests a new value and returns either true or false every time it is called on the nth call, it tests n values
map and filter clearly would be very useful. I'm not sure about the others.
I'd say every and other methods like that should take another argument that specifies the number if times to test.
This seems reasonable. Preferably that parameter would be optional, since the majority of the time my iterators will not be derived from infinite generators (and I know this). Otherwise I'd just keep passing Infinity as the third argument to my methods.
Is it possible to detect infinite generators ahead of time? My intuition says no.
map and filter clearly would be very useful. I'm not sure about the others.
Losing the others would be a shame. As I mentioned, much of the time I want to expose something with the semantics of an iterator, even if internally to my module or class I am storing the data as an array. So even if infinite generators cause problems for implementing every et al., the vast majority of real-world iterators would not be derived from infinite generators. Thus my above desire for the extra argument being optional.
Currently in our codebase, we have a number of these methods (like fetchAllProducts()) that return slice()ed copies of internal arrays, since we don't want to expose the arrays directly. (Iterators would of course be the best solution here.) Over the course of 77K LOC, the only methods we don't use on the products collection are join, reduceRight, reverse, and toString. And we do use join on some of the simpler collections (fetchProductTags()). So they're all useful!
Has anyone checked out or used docs.python.org/py3k/library/itertools.html ? Seems worth considering since ES6 generators are derived from Python generators.
Indeed, there's no way to detect an infinite generator ahead of time. Some array extras do not make sense even for very large arrays. Again I cite Peter Norvig's constraint-propagating-search-based Sudoku solver, which I ported to JS1.8:
bug380237.bugzilla.mozilla.org/attachment.cgi?id=266577
It definitely requires generators not arrays in places, to avoid exponential space blow-up.
On a related note, there are many ways in which some of these methods would overlap with the capabilities of array comprehensions and generator expressions. As far as I can tell, the one thing you cannot do (with, say, map) is take an array as input and process it lazily.
For example: someVeryLargeArray.map(someFunction); // guarantees an array result, cannot be lazy
vs
(someFunction(someVeryLargeArray(i), i, someVeryLargeArray) for (i in someVeryLargeArray)) // Lazy, likely to perform better in some important cases.
I think there's a case to be made for direct implementations of such methods. Something like
someVeryLargeArray.iMap(someFunction); // Lazy, guaranteed only to be iterable
(No apologies to the email protocol.)
What do you think?
-Michael A. Smith
someVeryLargeArray.iMap(someFunction); // Lazy, guaranteed only to be iterable
How about someVeryLargeArray.asIterator().map(someFunction)?
Catching up on this discussion.
Personally, I'd like to see us come to quick consensus on forEach for
collections, because without something like forEach, it's impossible to
implement useful versions of these collections in user-space in browsers
that do not implement for-of.
Indeed, the current browsers that implement Map and Set do not implement
for-of, which means that even though browsers implement the collections
(and Mozilla has for a bit now), they are not very useful because they
cannot be iterated over. Had forEach been part of the initial spec, I
would be using them today.
One last thing: forEach on Map should probably pass the key and value
to the function.
Yehuda Katz (ph) 718.877.1325
On Tue, Feb 28, 2012 at 12:06 PM, Yehuda Katz <wycats at gmail.com> wrote:
Catching up on this discussion.
Personally, I'd like to see us come to quick consensus on
forEachfor collections, because without something likeforEach, it's impossible to implement useful versions of these collections in user-space in browsers that do not implement for-of.Indeed, the current browsers that implement Map and Set do not implement for-of, which means that even though browsers implement the collections (and Mozilla has for a bit now), they are not very useful because they cannot be iterated over. Had
forEachbeen part of the initial spec, I would be using them today.
I've had the same experience that Yehuda describes - it's almost comical that Chrome blog published an announcement for the support of Map and Set, beyond...
var s = new Set() "undefined" s.add("foo") "undefined" s.has("foo")
true
s.delete("foo") "undefined"
...They are useless.
One last thing:
forEachonMapshould probably pass the key and value to the function.
+1 to Map.prototype.forEach receiving key and value as arguments
ES5's existing array extras make working with arrays a joy.
However, sometimes arrays are not the right tool for the job. Perhaps you want lazy evaluation semantics (generators). Or perhaps you want to communicate that the list is immutable (compare .NET's
IEnumerable<T>or Java'sIterable<T>). ES Harmony seems to have the answer: iterators! LikeIEnumerable<T>orIterable<T>, they are the most basic primitive of iteration. Yay!But, if my
fetchAllProducts()method returns an iterator, I don't get my array extras. Sad.This may be solvable in library-space, but the iterator proposal doesn't seem to have an Iterator.prototype I could extend. So we end up with unfortunate underscore-style wrappers:
_(iterator).chain().map(mapper).some(predicate).value() .some(.map(iterator, mapper), predicate)
I propose adding the array extras to any iterator (in some way), such that we can have syntax similar to the following:
iterator.map(mapper).some(predicate) // returns an iterator
The methods I would like to see are:
What do you think?