Array subclassing, .map and iterables (Re: Jan 30 TC39 Meeting Notes)
One of the problems with map is that the return type of the function might be different. With some pseudo syntax for types
function f(x : T) : V { ... } var a : Array.<T> = ...
var b = a.map(f);
b is actually of type Array.<V> and not of Array.<T>
This becomes an issue with typed arrays at least.
On Sat, Feb 9, 2013 at 5:16 AM, Claus Reinke <claus.reinke at talk21.com> wrote:
I am trying to understand the discussion and resolution of 'The Array Subclassing "Kind" Issue'. The issue (though not its solution) seemed simple enough
class V extends Array { ... } m = (new V()).map(val => val); console.log( m instanceof V ); // false :(
and I was expecting solutions somewhere along this path:
.map should work for Array subclasses, preserving class
.map is independent of Array and its subclasses, there are lots of types for which it makes sense (Sets, EventEmitters, ..)
there should be an interface Mapable, implemented by Array and its subclasses, but also by other relevant classes, such that
class M implements Mapable { ... } m = (new M()).map(val => val); console.log( m instanceof M ); // true
(in typed variants of JS, this would call for generics, to separate structure class -supporting map- from element class -being mapped)
Instead, the accepted approach -if I understood it correctly- focuses on conversion and iterables:
Array.from( iterable ) => Array.from( iterable, mapFn )
such that
SubArray.from( iterable, val => val ) instanceof SubArray
This seems very odd to me, because
it introduces a second form of .map, in .from
instead of limiting to Array, .from-map is now limited to iterables (it would work for Set, which is really OrderedSet, but it wouldn't work for WeakMap)
it doesn't address the general problem: how to inherit structural functionality (such as mapping over all elements or a container/ iterable) while preserving class
With a general solution to the issue, I would expect to write
SubArray.from( iterable ).map( val => val ) instanceof SubArray
while also getting
new Mapable().map( val => val ) instanceof Mapable
Could someone please elaborate why the committee went with an additional map built into structure conversion instead?
Claus
PS. What about array comprehensions and generator expressions?
es-discuss mailing list es-discuss at mozilla.org, mail.mozilla.org/listinfo/es-discuss
-- erik
On Feb 9, 2013, at 2:16 AM, Claus Reinke wrote:
I am trying to understand the discussion and resolution of 'The Array Subclassing "Kind" Issue'. The issue (though not its solution) seemed simple enough
class V extends Array { ... } m = (new V()).map(val => val); console.log( m instanceof V ); // false :(
and I was expecting solutions somewhere along this path:
.map should work for Array subclasses, preserving class
.map is independent of Array and its subclasses, there are lots of types for which it makes sense (Sets, EventEmitters, ..)
there should be an interface Mapable, implemented by Array and its subclasses, but also by other relevant classes, such that
the issue is that a map function can broaden the domain of array elements. For example,
var intArray = new Int32Array([42,85,127649,32768]); //create a typed array from a regular array var strArray = intArray.map(v=>v.toString());
If intArray.map() produces a new intArray then the above map function is invalid. If intArray.map() produces an Array instance then you intArray.map instance of intArray.constructor desire won't hold. We can't have it both ways without provide some additional mechanism that probably involves additional parameters to some methods or new methods.
The choice we agreed to, at the meeting is
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example
m instance of V will be true.
intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
- If you want to map the elements of an array to different kind of array use <ArrayClass>.from with a map function as the second parameter:
var strArray = Array.from(intArray, v=>v.toString());
This seemed like a less invasive change then adding additional target kind parameters to Array.prototype.map. Also it seems like a very clear way for programmers to state their intent.
class M implements Mapable { ... } m = (new M()).map(val => val); console.log( m instanceof M ); // true
(in typed variants of JS, this would call for generics, to separate structure class -supporting map- from element class -being mapped)
ES isn't Java or C#. We don't have formalized interfaces (although it is useful to think and talk about informal interfaces) and since we are dynamically typed we don't need to get sucked into the tar pit of generics.
Instead, the accepted approach -if I understood it correctly- focuses on conversion and iterables:
It's not about conversion as much as giving the programmer a way of choosing the kind of array that map generates.
Array.from( iterable ) => Array.from( iterable, mapFn )
such that
SubArray.from( iterable, val => val ) instanceof SubArray
This seems very odd to me, because
- it introduces a second form of .map, in .from
How would use produce an Array of strings from an Int32Array?
- instead of limiting to Array, .from-map is now limited to iterables (it would work for Set, which is really OrderedSet, but it wouldn't work for WeakMap)
We already have Array.from that works with iterables, how does adding a map function change anything related to the <ArrayClass>.from result domains
- it doesn't address the general problem: how to inherit structural functionality (such as mapping over all elements or a container/ iterable) while preserving class
See above, Array.prototype.map will preserve the receiver's "class".
With a general solution to the issue, I would expect to write
SubArray.from( iterable ).map( val => val ) instanceof SubArray
yes, the above will produce an instance of SubArray. But the above also has the cost of an extra copy and the map function doesn't get to see the original iterable's values.
while also getting
new Mapable().map( val => val ) instanceof Mapable
I don't even know how to interpret the above, as we don't have a class or constructor named Mapable.
Could someone please elaborate why the committee went with an additional map built into structure conversion instead?
Claus
PS. What about array comprehensions and generator expressions?
What about them? Array comprehensions are a for of Array initializer and always produce an Array instance. Generaltor expressions produce iterators (which are iterable).
Allen Wirfs-Brock wrote:
On Feb 9, 2013, at 2:16 AM, Claus Reinke wrote:
I am trying to understand the discussion and resolution of 'The Array Subclassing "Kind" Issue'. The issue (though not its solution) seemed simple enough
class V extends Array { ... } m = (new V()).map(val => val); console.log( m instanceof V ); // false :(
and I was expecting solutions somewhere along this path:
.map should work for Array subclasses, preserving class
.map is independent of Array and its subclasses, there are lots of types for which it makes sense (Sets, EventEmitters, ..)
there should be an interface Mapable, implemented by Array and its subclasses, but also by other relevant classes, such that
the issue is that a map function can broaden the domain of array elements. For example,
var intArray = new Int32Array([42,85,127649,32768]); //create a typed array from a regular array var strArray = intArray.map(v=>v.toString());
If intArray.map() produces a new intArray then the above map function is invalid. If intArray.map() produces an Array instance then you intArray.map instance of intArray.constructor desire won't hold. We can't have it both ways without provide some additional mechanism that probably involves additional parameters to some methods or new methods.
The choice we agreed to, at the meeting is
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example m instance of V will be true. intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
- If you want to map the elements of an array to different kind of array use<ArrayClass>.from with a map function as the second parameter:
var strArray = Array.from(intArray, v=>v.toString());
This seemed like a less invasive change then adding additional target kind parameters to Array.prototype.map. Also it seems like a very clear way for programmers to state their intent.
You chose one default, but I think it was not the simplest one. It is good to see that map is often transforming types, so "the same type" may not be the best default (filter is another story).
I think the best would be (it is dead simple):
- to always use Array in map result
- to leave Array.from (as well as Map.from, V.from etc.) as is, generator comprehension does the mapping for you if you wish one.
So, the examples would be
V.from(x for x in new V); // you say you want the results in V intArray.map(v=>v.toString()); // collect them in default Array
On Feb 9, 2013, at 3:01 PM, Herby Vojčík wrote:
Allen Wirfs-Brock wrote:
ethods.
The choice we agreed to, at the meeting is
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example m instance of V will be true. intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
- If you want to map the elements of an array to different kind of array use<ArrayClass>.from with a map function as the second parameter:
var strArray = Array.from(intArray, v=>v.toString());
This seemed like a less invasive change then adding additional target kind parameters to Array.prototype.map. Also it seems like a very clear way for programmers to state their intent.
You chose one default, but I think it was not the simplest one. It is good to see that map is often transforming types, so "the same type" may not be the best default (filter is another story).
I think the best would be (it is dead simple):
- to always use Array in map result
In your previous post you said:
- .map should work for Array subclasses, preserving class
are you changing that position?
Also, there is another subtlety that is on slide 25 of the deck I present. Existing ES<6 code may create objects that inherit from Array.prototype. When running on an ES6 implementation uses of Array.prototype.map (etc.) in such code can't change their behavior. So, the selection of the result "class" can't be based solely on the [[Prototype]] inheritance chain.
- to leave Array.from (as well as Map.from, V.from etc.) as is, generator comprehension does the mapping for you if you wish one.
So, the examples would be
V.from(x for x in new V); // you say you want the results in V
I don't understand? This produces the same result as new V; but with an extra V allocation, creation of a generator, etc.
Another issue, is that some array-like "classes" instances must have their size fixed at allocation. This is the case for all the TypedArrays. For iterators automatically derived from most arrays, we can make the size information available. But for a generator, there is no way to know how many iterations it will make without actually running it. For specification purposes, we may specify the "from" method as accumulating the element values into a List, allocatininge most TypedArray uses) that the double copy can be optimized away. That means that in the usual case the size must be available at the beginning which precludes using a generator expression as the usual case pattern.
Thanks for the explanations and additional details. Let me first try to rephrase, to see whether I've understood your reasoning:
The problem comes from the partial integration of types in ES, specifically having typed arrays but no easy way to express and control the types of the functions mapped over them.
And your solution is to fix Array.map to being type-preserving, and to use an auxiliary map in Container.from instead of Array.map when type changing mappings have to be expressed.
Using <> for type parameters, => for function types, and suppressing
a few details (such as prototype, this, index parameter), we can write the types of the two groups of operations as
Array<Elem>.map :
(<Elem> => <Elem>) =>
Array<Elem>
Container<Elem2>.from :
Iterable<Elem1> =>
(<Elem1> => <Elem2>) =>
Container<Elem2>
where the ES5 Array is seen as Array<Any> (so arbitrary mappings
are still supported at that level), and Array<Int32>, etc are written
as type-level constants Int32Array, for lack of type-level constructor syntax (the parameterized Interface Iterable<> is also inexpressible).
Since ES cannot guarantee that the mappings have the expected types, an implicit conversion of the mapped elements to the expected element type will be enforced (probably with a type check to avoid unexpected conversions?).
So
int32Array.map( f )
will be read as roughly
int32Array.map( (elem) => Number( f(elem) ) )
and
Int32Array.from( iterable, f )
as
Int32Array.from( iterable, (elem) => Number( f(elem) ) )
Do I have this right, so far?
var intArray = new Int32Array([42,85,127649,32768]); //create a typed array from a regular array var strArray = intArray.map(v=>v.toString());
If intArray.map() produces a new intArray then the above map function is invalid. If intArray.map() produces an Array instance then you intArray.map instance of intArray.constructor desire won't hold. We can't have it both ways without provide some additional mechanism that probably involves additional parameters to some methods or new methods.
It is this additional mechanism which I'm after. In typed languages, it is long-established practice to put the additional parameters at the type level and to hang the interface on the type-level constructor, and I wonder how much of that could be translated for use in ES.
For instance, being able to specify an overloaded map function was the motivating example for introducing type constructor classes in Haskell
A system of constructor classes: overloading and implicit
higher-order polymorphism
Mark P. Jones,
In FPCA '93: Conference on Functional Programming Languages
and Computer Architecture, Copenhagen, Denmark, June 1993.
http://web.cecs.pdx.edu/~mpj/pubs/fpca93.html
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example m instance of V will be true.
intArray.map(v=>v.toSring()) produces an Int32Array.
The strings produced by the map function get converted back to numbers.
Given the history of implicit conversions in ES, have you considered just doing runtime type checks, without those new implicit conversions?
- If you want to map the elements of an array to different kind of array use <ArrayClass>.from with a map function as the second parameter:
var strArray = Array.from(intArray, v=>v.toString());
This seemed like a less invasive change then adding additional target kind parameters to Array.prototype.map. Also it seems like a very clear way for programmers to state their intent.
ES isn't Java or C#. We don't have formalized interfaces (although it is useful to think and talk about informal interfaces) and since we are dynamically typed we don't need to get sucked into the tar pit of generics.
If a programming concept is as useful as interfaces are, it usually pays to think about language support for it. And I was certainly not thinking of Java or C#, more of TypeScript, where the team seems to be working on JavaScript-suited generics for the next version, to be able to type current JavaScript library code.
Btw, parametric polymorphism in ML and its refinements and extensions in Haskell were elegant and concise tools before they got watered down in a multi-year process to fit into Java. If you have bad experience with generics, they probably come from Java's adaption.
How would use produce an Array of strings from an Int32Array?
Somewhat like
Array.from( int32Array ).map( (elem) => elem.toString() )
Implementations would be free to replace the syntactic pattern with an optimized single pass (in more conventional optimizing language implementations, such fusion of implicit or explicit loops is standard, but even ES JIT engines -with their limited time for optimization- should be able to spot the syntactic pattern).
- instead of limiting to Array, .from-map is now limited to iterables (it would work for Set, which is really OrderedSet, but it wouldn't work for WeakMap)
We already have Array.from that works with iterables, how does adding a map function change anything related to the <ArrayClass>.from result domains
My point was that map is far more widely useful, not limited to Array (Array.prototype.map), and not limited to Array construction from Iterables (Array.prototype.from with second parameter). Consider map on event emitters, on promises, on exceptions, on generators, ..
I don't have an alternative solution that would cover all use cases in ES uniformly, because the existing solutions in other languages do not translate directly.
However, I wanted to ring a warning bell that adding a different partial solution for every new use case is not going to scale well (especially with things being so difficult to change once they are in ES), and misses potential for writing generic library code.
If I have to write different code, depending on whether I need to map over a constructed Array, an Array under construction, an Array or an Int32Array, a generator, a promise, etc., then that will result in duplicate code instead of generic code.
With a general solution to the issue, I would expect to write
SubArray.from( iterable ).map( val => val ) instanceof SubArray
yes, the above will produce an instance of SubArray. But the above also has the cost of an extra copy and the map function doesn't get to see the original iterable's values.
Implementations can fuse .from().map() as well as .map().from(); if you want access to the unconverted elements, you want to map over the iterable, not the resulting Array (again, with loop fusion in the implementation):
SubArray.from( iterable.map( val => f(val) ) )
Of course, since map isn't part of a standard Array-independent interface, I have to write that as a generator expression (not sure whether I'm up to date on their syntax) instead of a map.
SubArray.from( (for ( elem of iterable ) in f(elem) ) )
while also getting
new Mapable().map( val => val ) instanceof Mapable
I don't even know how to interpret the above, as we don't have a class or constructor named Mapable.
fair enough - Mapable being intended as an interface for classes implementing map wouldn't be new-able, so this was pseudo code for instantiating any class that implements such a Mapable. I think a full solution for polymorphic map will require some way of specifying interfaces independent of the classes (Array, Int32Array, Promise, generators, ..) implementing them.
PS. What about array comprehensions and generator expressions?
What about them? Array comprehensions are a for of Array initializer and always produce an Array instance. Generaltor expressions produce iterators (which are iterable).
Array comprehensions are a full array processing language, which can be desugared to explicit array operations like map and filter, so they tend to have the same problems as map and filter do. Will I be able to use array comprehensions with Int32Arrays, or to convert between array types? Will I be able to call .map on iterators/generator expressions, without using case-specific syntax?
Claus
Allen Wirfs-Brock wrote:
On Feb 9, 2013, at 3:01 PM, Herby Vojčík wrote:
Allen Wirfs-Brock wrote:
ethods.
The choice we agreed to, at the meeting is
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example m instance of V will be true. intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
- If you want to map the elements of an array to different kind of array use<ArrayClass>.from with a map function as the second parameter:
var strArray = Array.from(intArray, v=>v.toString());
This seemed like a less invasive change then adding additional target kind parameters to Array.prototype.map. Also it seems like a very clear way for programmers to state their intent. You chose one default, but I think it was not the simplest one. It is good to see that map is often transforming types, so "the same type" may not be the best default (filter is another story).
I think the best would be (it is dead simple): - to always use Array in map result
In your previous post you said:
- .map should work for Array subclasses, preserving class
are you changing that position?
That's probably not me who said it :-)
Also, there is another subtlety that is on slide 25 of the deck I present. Existing ES<6 code may create objects that inherit from Array.prototype. When running on an ES6 implementation uses of Array.prototype.map (etc.) in such code can't change their behavior.
I am not TC39 member, so I did not see that slide, but if you say ".map always produces Array", it does not change the behaviour.
Could you elaborate more, please?
So, the selection of the result "class" can't be based solely on the [[Prototype]] inheritance chain.
- to leave Array.from (as well as Map.from, V.from etc.) as is, generator comprehension does the mapping for you if you wish one.
So, the examples would be
V.from(x for x in new V); // you say you want the results in V I don't understand? This produces the same result as new V; but with an extra V allocation, creation of a generator, etc.
It was rewrite of Claus' example you were discussing about:
class V extends Array { ... } m = (new V()).map(val => val); console.log( m instanceof V ); // false
Another issue, is that some array-like "classes" instances must have their size fixed at allocation. This is the case for all the TypedArrays. For iterators automatically derived from most arrays, we can make the size information available. But for a generator, there is no way to know how many iterations it will make without actually running it. For specification purposes, we may specify the
Hm. Yes, this is problem, then.
"from" method as accumulating the element values into a List, allocatininge most TypedArray uses) that the double copy can be optimized away. That means that in the usual case the size must be available at the beginning which precludes using a generator expression as the usual case pattern.
Yes, then the second point is not-starter.
But I still think it is simpler to specify ".map" returning array, always. You know you get an Array, you don't have to worry about magic of Set, Map etc., if you wish to have it in other kind of container, use Container.from(...).
Of course, I don't know what that slide 25 was about, but for now I argue map is different from concat, filter etc. do not transform contents, just manipulate containers.
On Feb 10, 2013, at 3:40 AM, Herby Vojčík wrote:
Allen Wirfs-Brock wrote:
On Feb 9, 2013, at 3:01 PM, Herby Vojčík wrote:
... I think the best would be (it is dead simple): - to always use Array in map result
In your previous post you said:
- .map should work for Array subclasses, preserving class
are you changing that position?
That's probably not me who said it :-)
oops, sorry for the mis-attribution
Also, there is another subtlety that is on slide 25 of the deck I present. Existing ES<6 code may create objects that inherit from Array.prototype. When running on an ES6 implementation uses of Array.prototype.map (etc.) in such code can't change their behavior.
I am not TC39 member, so I did not see that slide, but if you say ".map always produces Array", it does not change the behaviour.
Could you elaborate more, please?
All the presentation decks from the meeting are publicly available at meetings:meeting_jan_29_2013
the short form:
In ES5, today you can use Array.prototype as the [[Prototype]] of non-Array objects or apply the Array.prototype methods to non-arrays. The behaviors they get are non-necessarily what you would expect if you would really subclass Array. We need to preserve the existing behaviors for these existing use cases but would like to provide the more reasonable behavior when class extends Array is used to create a subclass.
Array.prototype.concat is the most problematic of the existing methods in this regard.
...
Another issue, is that some array-like "classes" instances must have their size fixed at allocation. This is the case for all the TypedArrays. For iterators automatically derived from most arrays, we can make the size information available. But for a generator, there is no way to know how many iterations it will make without actually running it. For specification purposes, we may specify the
Hm. Yes, this is problem, then.
"from" method as accumulating the element values into a List, allocatininge most TypedArray uses) that the double copy can be optimized away. That means that in the usual case the size must be available at the beginning which precludes using a generator expression as the usual case pattern.
Yes, then the second point is not-starter.
But I still think it is simpler to specify ".map" returning array, always. You know you get an Array, you don't have to worry about magic of Set, Map etc., if you wish to have it in other kind of container, use Container.from(...).
But wouldn't you want:
var my16bitVector = Uint16Array.from([1,2,3,4,5,6]);
...
var y = my16bitVector.map(v=>v+1)
....
someExternalAPI(y);
to create a Uint16Array for y? It seems like the most common use cases want to produce the same kind of elements. Mapping to a different or broader kind of element is also common, but I think less common. So it's the case that should take to more to express.
On Feb 10, 2013, at 1:33 AM, Claus Reinke wrote:
Thanks for the explanations and additional details. Let me first try to rephrase, to see whether I've understood your reasoning:
The problem comes from the partial integration of types in ES, specifically having typed arrays but no easy way to express and control the types of the functions mapped over them.
ECMAScript has various kinds of values and objects that are distinguished in various ways. However, these variations of kind are quite unlike the concept of type as used in PL theory and statically typed languages. For that reason I generally try to avoid use of the word "type" in the context of ES (although sometimes it is unavoidable, eg "TypedArrays") because it can lead to thinking that reflects static typing experience rather than dynamic language experiences that are often more appropriate for ES.
And your solution is to fix Array.map to being type-preserving, and to use an auxiliary map in Container.from instead of Array.map when type changing mappings have to be expressed. Using <> for type parameters, => for function types, and suppressing a few details (such as prototype, this, index parameter), we can write the types of the two groups of operations as
Array<Elem>.map : (<Elem> => <Elem>) => Array<Elem>
Container<Elem2>.from : Iterable<Elem1> => (<Elem1> => <Elem2>) => Container<Elem2>
where the ES5 Array is seen as Array<Any> (so arbitrary mappings are still supported at that level), and Array<Int32>, etc are written as type-level constants Int32Array, for lack of type-level constructor syntax (the parameterized Interface Iterable<> is also inexpressible).
Like I said above. This is a type theory view of the word. Static type systems require some sort parameterization or genercity in order to be sufficiently expressive. Few programmers actually understand the subtleties of the type systems. Dynamic languages typically don't use such type systems and achieve equivalent (arguably greater) expressiveness using different approaches. We don't want to turn ES into Java or C#.
Since ES cannot guarantee that the mappings have the expected types, an implicit conversion of the mapped elements to the expected element type will be enforced (probably with a type check to avoid unexpected conversions?).
Right, dynamic languages typically use dynamic constraint checks or dynamic coercions when specific kinds of objects are required.
So int32Array.map( f ) will be read as roughly
int32Array.map( (elem) => Number( f(elem) ) )
and
Int32Array.from( iterable, f )
as
Int32Array.from( iterable, (elem) => Number( f(elem) ) )
Do I have this right, so far?
yes, except that the Number coercion takes place at a much deeper layer -- If is part of the [[Put]] operation that actually stores values into the Int32Array
var intArray = new Int32Array([42,85,127649,32768]); //create a typed array from a regular array var strArray = intArray.map(v=>v.toString()); If intArray.map() produces a new intArray then the above map function is invalid. If intArray.map() produces an Array instance then you intArray.map instance of intArray.constructor desire won't hold. We can't have it both ways without provide some additional mechanism that probably involves additional parameters to some methods or new methods.
It is this additional mechanism which I'm after. In typed languages, it is long-established practice to put the additional parameters at the type level and to hang the interface on the type-level constructor, and I wonder how much of that could be translated for use in ES.
There is also plenty of dynamic language experience for the pattern I proposed. For example, withAll: is pretty much Smalltalk's equivalent of ES for. The conceptual model of this is not that the for method is being parameterized by the programmer. Instead, the model is that the programmer has chosen a specific kind of collection object that will be populated using for and it is the responsibility of that object to impose what ever constraints upon its elements as are appropriate for it.
For instance, being able to specify an overloaded map function was the motivating example for introducing type constructor classes in Haskell
A system of constructor classes: overloading and implicit higher-order polymorphism Mark P. Jones, In FPCA '93: Conference on Functional Programming Languages and Computer Architecture, Copenhagen, Denmark, June 1993. web.cecs.pdx.edu/~mpj/pubs/fpca93.html
Right, but this is all starting rom a static typing perspective rather than dynamic typing.
- Array.prototype.map produces the same kind of array that it was applied to, so: for the above example m instance of V will be true. intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
Given the history of implicit conversions in ES, have you considered just doing runtime type checks, without those new implicit conversions?
TypedArrays have already been implemented in browsers doing this sort of implicit conversion. It probably can't be changed.
- If you want to map the elements of an array to different kind of array use <ArrayClass>.from with a map function as the second parameter: var strArray = Array.from(intArray, v=>v.toString()); This seemed like a less invasive change then adding additional target kind parameters to Array.prototype.map. Also it seems like a very clear way for programmers to state their intent.
ES isn't Java or C#. We don't have formalized interfaces (although it is useful to think and talk about informal interfaces) and since we are dynamically typed we don't need to get sucked into the tar pit of generics.
If a programming concept is as useful as interfaces are, it usually pays to think about language support for it. And I was certainly not thinking of Java or C#, more of TypeScript, where the team seems to be working on JavaScript-suited generics for the next version, to be able to type current JavaScript library code. Btw, parametric polymorphism in ML and its refinements and extensions in Haskell were elegant and concise tools before they got watered down in a multi-year process to fit into Java. If you have bad experience with generics, they probably come from Java's adaption.
TypeScript is, for the most part, a advisory compile-time only static type system. It isn't, at all, a part of the runtime model of the language. At runtime, the language still has to deal with all the runtime situations where the "wrong type" is passed.
Interfaces are a useful conceptual concept, but in a dynamic language you have to think about them as static approximations of dynamic behavior. They are useful for abstracting the design of a system in order to make it easier to comprehend. But, they don't need to be expressive enough to prove theorems about the system.
How would use produce an Array of strings from an Int32Array?
Somewhat like
Array.from( int32Array ).map( (elem) => elem.toString() )
Implementations would be free to replace the syntactic pattern with an optimized single pass (in more conventional optimizing language implementations, such fusion of implicit or explicit loops is standard, but even ES JIT engines -with their limited time for optimization- should be able to spot the syntactic pattern).
Even if such an optimization was likely in ES, why would you want to write that instead of the shorter single call form. Does it come down to wanting to explicitly name the "map" parameter. Would you think differently if ES had keyword parameters:
Array.from(int32Array, map: elem=>elem.toString())
- instead of limiting to Array, .from-map is now limited to iterables (it would work for Set, which is really OrderedSet, but it wouldn't work for WeakMap) We already have Array.from that works with iterables, how does adding a map function change anything related to the <ArrayClass>.from result domains
My point was that map is far more widely useful, not limited to Array (Array.prototype.map), and not limited to Array construction from Iterables (Array.prototype.from with second parameter). Consider map on event emitters, on promises, on exceptions, on generators, ..
I don't have an alternative solution that would cover all use cases in ES uniformly, because the existing solutions in other languages do not translate directly. However, I wanted to ring a warning bell that adding a different partial solution for every new use case is not going to scale well (especially with things being so difficult to change once they are in ES), and misses potential for writing generic library code. If I have to write different code, depending on whether I need to map over a constructed Array, an Array under construction, an Array or an Int32Array, a generator, a promise, etc., then that will result in duplicate code instead of generic code.
Something that dynamic language support is the use of informal polymorphism. You can assign a name to a concept (like map) without requiring type consistent behavior among various uses of that name. As long as the actual behavior makes sense in context it doesn't cause any conceptual problems.
With a general solution to the issue, I would expect to write SubArray.from( iterable ).map( val => val ) instanceof SubArray yes, the above will produce an instance of SubArray. But the above also has the cost of an extra copy and the map function doesn't get to see the original iterable's values.
Implementations can fuse .from().map() as well as .map().from(); if you want access to the unconverted elements, you want to map over the iterable, not the resulting Array (again, with loop fusion in the implementation):
SubArray.from( iterable.map( val => f(val) ) )
Of course, since map isn't part of a standard Array-independent interface, I have to write that as a generator expression (not sure whether I'm up to date on their syntax) instead of a map.
SubArray.from( (for ( elem of iterable ) in f(elem) ) )
while also getting new Mapable().map( val => val ) instanceof Mapable I don't even know how to interpret the above, as we don't have a class or constructor named Mapable.
fair enough - Mapable being intended as an interface for classes implementing map wouldn't be new-able, so this was pseudo code for instantiating any class that implements such a Mapable. I think a full solution for polymorphic map will require some way of specifying interfaces independent of the classes (Array, Int32Array, Promise, generators, ..) implementing them.
To me, you are describing steps into the static typing rat hole where to adequately express even simple concepts like mapping an Int32 array to an Array of strings require adding more complex language mechanisms and abstractions. A dynamic language can be conceptually much simpler because we just don't need those mechanism.
PS. What about array comprehensions and generator expressions? What about them? Array comprehensions are a for of Array initializer and always produce an Array instance. Generaltor expressions produce iterators (which are iterable).
Array comprehensions are a full array processing language, which can be desugared to explicit array operations like map and filter, so they tend to have the same problems as map and filter do. Will I be able to use array comprehensions with Int32Arrays, or to convert between array types?
Nope, Array comprehensions create instances of Array. The syntax provides no mechanism for syntactically specify the kind of the generated object. Of course you can say:
Int32Array.from([for (i in thing) i]) //note that order of array comprehensions have changed
but this generates an extra object and copying unless an implementation is able to perform the sort of optimizations you hope they can do.
Will I be able to call .map on iterators/generator expressions, without using case-specific syntax?
you mean:
[].map.call(iterator, elem=>elem.prime)
Generally, no. [].map and other Array.prototype methods are a method on generic array-likes, not on iterators. It requires some behaviors that the generic iterator interface doesn't provide. Of course, you could define your own map function that worked on iterators.
On Sun, Feb 10, 2013 at 4:33 AM, Claus Reinke <claus.reinke at talk21.com>wrote:
(snip)
How would use produce an Array of strings from an Int32Array?
Somewhat like
Array.from( int32Array ).map( (elem) => elem.toString() )
Implementations would be free to replace the syntactic pattern with an optimized single pass (in more conventional optimizing language implementations, such fusion of implicit or explicit loops is standard, but even ES JIT engines -with their limited time for optimization- should be able to spot the syntactic pattern).
(Assuming a future where the DOM's NodeList inherits from Array)
How would you produce a NodeList from an arbitrary array of strings?
NodeList.from( [ "div", "span", "p" ], nodeName =>
document.createElement(nodeName) );
Because...
NodeList.from( strings );
Would try to make a NodeList, but with items that of a type that it disallows, meaning:
NodeList.from( strings ).map( nodeName =>
document.createElement(nodeName) );
Would call .map() on an empty NodeList, since the string value items had been rejected. Of course, you might argue that I could just call it like:
NodeList.from( [ "div", "span", "p" ].map(nodeName =>
document.createElement(nodeName)) );
...But the "arraylike" or "iterable" might not have a .map() method of its own, which will cause issues if I'm in a JS-target transpilation scenario...
(function( root ) { root.converter = function( ctor, iterable, map ) { return this[ ctor ].from( iterable, map ); }.bind(root); }( this ));
(just assume that |this| is the global object ;) )
- instead of limiting to Array, .from-map is now limited to iterables
Where do you see this documented? Array.from (and any subclass) accepts both array-likes and iterables.
(it would work for Set, which is really OrderedSet, but it wouldn't
work for WeakMap)
This discussion is irrelevant to WeakMap.
We already have Array.from that works with iterables, how does adding a map function change anything related to the <ArrayClass>.from result domains
My point was that map is far more widely useful, not limited to Array (Array.prototype.map), and not limited to Array construction from Iterables (Array.prototype.from with second parameter). Consider map on event emitters, on promises, on exceptions, on generators, ..
I don't have an alternative solution that would cover all use cases in ES uniformly, because the existing solutions in other languages do not translate directly. However, I wanted to ring a warning bell that adding a different partial solution for every new use case is not going to scale well (especially with things being so difficult to change once they are in ES), and misses potential for writing generic library code.
Can you show an example of this?
If I have to write different code, depending on whether I need to map over a constructed Array, an Array under construction, an Array or an Int32Array, a generator, a promise, etc., then that will result in duplicate code instead of generic code.
With a general solution to the issue, I would expect to write
SubArray.from( iterable ).map( val => val ) instanceof SubArray
yes, the above will produce an instance of SubArray. But the above also has the cost of an extra copy and the map function doesn't get to see the original iterable's values.
Implementations can fuse .from().map() as well as .map().from();
.from() is a static method, so .map().from() won't work.
if you want access to the unconverted elements, you want to map over the iterable, not the resulting Array (again, with loop fusion in the implementation):
SubArray.from( iterable.map( val => f(val) ) )
The from-map-function does map over the iterable, not the resulting Array (or SubArray, or NodeList or whatever)
(See NodeList example above)
Of course, since map isn't part of a standard Array-independent interface, I have to write that as a generator expression (not sure whether I'm up to date on their syntax) instead of a map.
SubArray.from( (for ( elem of iterable ) in f(elem) ) )
So... you'd get a "SubArray" whose elements where the result of (for ( elem of iterable ) in f(elem) ), but that's the same as:
SubArray.from( iterable, f );
...Which is much nicer to read.
On Sun, Feb 10, 2013 at 11:18 AM, Rick Waldron <waldron.rick at gmail.com> wrote:
(Assuming a future where the DOM's NodeList inherits from Array)
How would you produce a NodeList from an arbitrary array of strings?
NodeList.from( [ "div", "span", "p" ], nodeName => document.createElement(nodeName) );
Because...
NodeList.from( strings );
Would try to make a NodeList, but with items that of a type that it disallows, meaning:
NodeList.from( strings ).map( nodeName => document.createElement(nodeName) );
Would call .map() on an empty NodeList, since the string value items had been rejected. Of course, you might argue that I could just call it like:
NodeList.from( [ "div", "span", "p" ].map(nodeName => document.createElement(nodeName)) );
...But the "arraylike" or "iterable" might not have a .map() method of its own, which will cause issues if I'm in a JS-target transpilation scenario...
From what I've seen in the examples in this topic so far, it looks
like Array#from takes a second optional argument, which is a function that you map the iterable elements through first, before giving them to the constructor. So, you would write:
NodeList.from( ["div", "span", "p"], (nodeName)=>document.createElement(nodeName) );
On Sun, Feb 10, 2013 at 3:04 PM, Tab Atkins Jr. <jackalmage at gmail.com>wrote:
On Sun, Feb 10, 2013 at 11:18 AM, Rick Waldron <waldron.rick at gmail.com> wrote:
(Assuming a future where the DOM's NodeList inherits from Array)
How would you produce a NodeList from an arbitrary array of strings?
NodeList.from( [ "div", "span", "p" ], nodeName => document.createElement(nodeName) );
Because...
NodeList.from( strings );
Would try to make a NodeList, but with items that of a type that it disallows, meaning:
NodeList.from( strings ).map( nodeName => document.createElement(nodeName) );
Would call .map() on an empty NodeList, since the string value items had been rejected. Of course, you might argue that I could just call it like:
NodeList.from( [ "div", "span", "p" ].map(nodeName => document.createElement(nodeName)) );
...But the "arraylike" or "iterable" might not have a .map() method of its own, which will cause issues if I'm in a JS-target transpilation scenario...
From what I've seen in the examples in this topic so far, it looks like Array#from takes a second optional argument, which is a function that you map the iterable elements through first, before giving them to the constructor.
Yes, and I said just as much (not same words) further down the thread.
So, you would write:
NodeList.from( ["div", "span", "p"], (nodeName)=>document.createElement(nodeName) );
This is exactly the same as the very first solution I offered:
(Assuming a future where the DOM's NodeList inherits from Array)
How would you produce a NodeList from an arbitrary array of strings?
NodeList.from( [ "div", "span", "p" ], nodeName =>
document.createElement(nodeName) );
Allen Wirfs-Brock wrote:
On Feb 10, 2013, at 3:40 AM, Herby Vojčík wrote:
But I still think it is simpler to specify ".map" returning array, always. You know you get an Array, you don't have to worry about magic of Set, Map etc., if you wish to have it in other kind of container, use Container.from(...).
But wouldn't you want: var my16bitVector = Uint16Array.from([1,2,3,4,5,6]); ... var y = my16bitVector.map(v=>v+1) .... someExternalAPI(y);
to create a Uint16Array for y? It seems like the most common use cases want to produce the same kind of elements. Mapping to a different or broader kind of element is also common, but I think less common. So it's the case that should take to more to express.
Well, here's the difference. Nearly every kind of .map use I remember
(or collect: in Smalltalk, maybe I should not mix them but they are
mixed in my mind) are transformation of the object: map objects to only
some specific property, wrap collection of primitives to objects that
are created from them, etc. I honestly can't remember any use of .map
concept that preserves element types beyond educatory (1 to: 10) collect: [ :x | x*x ].
That is, my conviction is, .map is mainly used for bigger transformations of its elements (I think your v=>v+1 example much more
often happens in-place, that is, forEach; map use-case seems rare for me).
On Feb 10, 2013, at 1:04 PM, Tab Atkins Jr. wrote:
From what I've seen in the examples in this topic so far, it looks like Array#from takes a second optional argument, which is a function that you map the iterable elements through first, before giving them to the constructor. So, you would write:
NodeList.from( ["div", "span", "p"], (nodeName)=>document.createElement(nodeName) );
Exactly.
Of course, in defining class NodeList extends Arrar {...} you could decide to over-ride the inherited definition of .from with a version that implicitly did the above coercion on string valued elements. It up to the designer of the class interface.
Thank you for this explanation. This is very interesting! If you don't mind, I have some questions/concerns I'd like to clear up. (In particular I want to make sure I understand; I don't intend to argue for one side or the other ATM, but this change may require me to refactor some old code to be future-ready).
Allen Wirfs-Brock wrote:
The choice we agreed to, at the meeting is
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example m instance of V will be true. intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
Sounds cool! Does it work?
I rely on the genericness of Array methods quite a lot. For example, it's fairly common for me to write something like the following:
var map = Function.prototype.call.bind(Array.prototype.map), select = document.querySelectorAll.bind(document);
var ids = map(select('div'), function(el) { return el.id; });
Currently this would get me an array of string ids for every div on a page. In a future ES6 with a future DOM where NodeList extends Array, will ids no longer hold an array of strings but try to remain a NodeList?
There's a good chance this will break some of my code. I'm capable of changing my code and writing this to be more future-friendly from now on (I'm not one who prefers backwards compatibility over a better language). But I would have always assumed I was doing things correctly before, and I'm curious if the rest of the internet will be okay..?
Nathan
On Monday, February 11, 2013, Nathan Wall wrote:
Thank you for this explanation. This is very interesting! If you don't mind, I have some questions/concerns I'd like to clear up. (In particular I want to make sure I understand; I don't intend to argue for one side or the other ATM, but this change may require me to refactor some old code to be future-ready).
Allen Wirfs-Brock wrote:
The choice we agreed to, at the meeting is
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example m instance of V will be true. intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
Sounds cool! Does it work?
I rely on the genericness of Array methods quite a lot. For example, it's fairly common for me to write something like the following:
var map = Function.prototype.call.bind(Array.prototype.map), select = document.querySelectorAll.bind(document); var ids = map(select('div'), function(el) { return el.id; });Currently this would get me an array of string ids for every div on a page. In a future ES6 with a future DOM where NodeList extends Array, will
idsno longer hold an array of strings but try to remain a NodeList?There's a good chance this will break some of my code. I'm capable of changing my code and writing this to be more future-friendly from now on (I'm not one who prefers backwards compatibility over a better language). But I would have always assumed I was doing things correctly before, and I'm curious if the rest of the internet will be okay..?
Since you're using Array.prototype.map there, your code will indeed give you an array of string IDs, as it always did.
Rick Waldron wrote:
On Monday, February 11, 2013, Nathan Wall wrote:
Thank you for this explanation. This is very interesting! If you don't mind, I have some questions/concerns I'd like to clear up. (In particular I want to make sure I understand; I don't intend to argue for one side or the other ATM, but this change may require me to refactor some old code to be future-ready). Allen Wirfs-Brock wrote: > The choice we agreed to, at the meeting is > > 1) Array.prototype.map produces the same kind of array that it was applied to, so: > > for the above example > m instance of V will be true. > intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers. Sounds cool! Does it work? I rely on the genericness of Array methods quite a lot. For example, it's fairly common for me to write something like the following: var map = Function.prototype.call.bind(Array.prototype.map), select = document.querySelectorAll.bind(document); var ids = map(select('div'), function(el) { return el.id <http://el.id>; }); Currently this would get me an array of string ids for every div on a page. In a future ES6 with a future DOM where NodeList extends Array, will `ids` no longer hold an array of strings but try to remain a NodeList? There's a good chance this will break some of my code. I'm capable of changing my code and writing this to be more future-friendly from now on (I'm not one who prefers backwards compatibility over a better language). But I would have always assumed I was doing things correctly before, and I'm curious if the rest of the internet will be okay..?Since you're using Array.prototype.map there, your code will indeed give you an array of string IDs, as it always did.
No it won't. In the current proposal (which I'd like to change, I think Array is better default container for map), if NodeList would subclass Array, applying (generic) Array.prototype.map on instance of NodeList would create a new NodeList and would try to put ids on there.
Le 10/02/2013 19:05, Allen Wirfs-Brock a écrit :
All the presentation decks from the meeting are publicly available at meetings:meeting_jan_29_2013
the short form:
In ES5, today you can use Array.prototype as the [[Prototype]] of non-Array objects or apply the Array.prototype methods to non-arrays. The behaviors they get are non-necessarily what you would expect if you would really subclass Array. We need to preserve the existing behaviors for these existing use cases but would like to provide the more reasonable behavior when class extends Array is used to create a subclass.
Array.prototype.concat is the most problematic of the existing methods in this regard.
After reading this thread and the slides, what is the plan for Typed Arrays (dedicated array like methods or Array subclassed like) ?
I see the constraints for subclassing, but the solution a.from(b,map) seems a little bit strange (even if I agree that it should return the same instance as a) , and what about concat, slice, etc as you mention? Couldn't we add something like a parameter to the methods : example.map(f,thisArg, true) or example.map(f,true) returns example's instance instead of Array (same processing as .of, .from), so you can explicitely say that you want to return the same instance and don't have backward compatilities issues (hopefully...) ?
I have used a lot Typed Arrays for [1] [2] [3] and all along wondering why Array like optimized methods were not available, especially concat, and why it was speced entirely in TC39 while a spec already exists (which specifies slice/subarray but surprisingly not concat), but it will be extremely usefull to have the Array like methods in the ES specs for Typed Arrays.
,
[1] www.github.com/Ayms/node-Tor [2] www.github.com/Ayms/iAnonym [3] www.github.com/Ayms/node
[to limit the length of my reply, I had to avoid responding to every detail, trying to answer the gist of your message instead; please let me know if I missed anything important]
Of course, you might argue that I could just call it like:
NodeList.from( [ "div", "span", "p" ].map(nodeName => document.createElement(nodeName)) );
Indeed, this would be my preferred choice. It would be more modular than packaging the combination of container change and element conversion into a single operation.
However, I understand the conflict between type-changing element maps and element-type-constrained containers.
Let me change the example to bring out this point: if we convert from an array of Int32 to an array of Double, we cannot map on the source array nor can we map on the target array. So we do have to map the elements in transit, after extracting them from the Int32 source and before placing them in the Double target, preferably without creating an intermediate Array.
Since the container change (.from()) is specced via iterators, I suggest to ensure support for .map() on iterators and map the elements in the iteration, after extraction from source, before integration into target.
Integrating the element map into the container conversion (.from()) instead, which is a static method to enforce target-type specification, solves the issue you were trying to address, for Array subclasses, but it leaves us with a host of residual issues:
-
there are now two operations for one generic task, Array.prototype.map( <function> ) // type-preserving <TargetClass>.from( <source>, <function> ) // type-changing
-
the two -clearly related- operations do not have a common interface, in fact, one is an object method, the other a static/class method
-
the latter operation is really a family of operations, and the static type prefixes of the family members are difficult to abstract over (do I try to get the target type from the target object context or from the function result, or do I force the user to pass it in at every call site)
-
even with this additional complexity, we still do not have support for mapping over the elements of other, non-Array containers
I suspect that the problem of establishing target container types is separate from the element mapping, so I would like to keep .from() and .map() separate. But even in the merged design programming will be awkward.
...But the "arraylike" or "iterable" might not have a .map() method of its own, which will cause issues if I'm in a JS-target transpilation scenario...
And I would like not only to root out any "arraylike" or "iterable" that do not support .map(), but would also like to extend the reach of .map() to other cases where it makes sense (I've listed examples in previous messages).
(function( root ) { root.converter = function( ctor, iterable, map ) { return this[ ctor ].from( iterable, map ); }.bind(root); }( this ));
What you're saying here is that (1) .from() should support .map()-like functionality for all iterables (even if they do not support .map()), that (2) we can't use .map() because it may not be supported for all iterables and the 'map' parameter might be type-changing, and that (3) you don't know how to get the target type generically, so it'll have to be passed in at each call site.
None of this is promising when I think of writing generic code that employs mapping over different container types, even if we assume that the mapping .from() replaces .map() as the general interface.
Are you going to pass around 'ctor's as dynamic type hints? Since we need the target classes, we can't even extract the class from the source arrays. This, and the inability to de-structure things like Int32Array into its components, are among the outstanding language design issues generated in this area.
My point was that map is far more widely useful, not limited to Array (Array.prototype.map), and not limited to Array construction from Iterables (Array.prototype.from with second parameter). Consider map on event emitters, on promises, on exceptions, on generators, ..
I don't have an alternative solution that would cover all use cases in ES uniformly, because the existing solutions in other languages do not translate directly. However, I wanted to ring a warning bell that adding a different partial solution for every new use case is not going to scale well (especially with things being so difficult to change once they are in ES), and misses potential for writing generic library code.
Can you show an example of this?
Example of what (can't resolve 'this';-)? I listed several examples of classes that I'd like to see map() support on. You gave an example of how you couldn't write generic code using map() because not all relevant classes support that method (using .from() on iterables doesn't work, either). If you mean difficulties of evolving ES designs after release, think no further than existing code inheriting from Array.prototype.
If you mean the code pattern and language support in general, it is just "programming to interfaces", applied to the interface supporting map. The pattern was popularized for OOP in the "Design Patterns" book ( en.wikipedia.org/wiki/Design_Patterns ), but is also standard in functional languages.
Readers here tend to be familiar with OOP, so I listed an example earlier of language support for the map interface (the Functor typeclass) in Haskell, which is possibly the most widely implemented interface there (and shows language support for fine-grained interface-based reuse - a type can support interfaces based on being a container, based on containing a certain kind of element, or based on supporting other interfaces; e.g., if you have a type that is a container, supports 'map', and contains elements that support '+', then you can write generic code for adding the container elements together, without having to commit to any specific container- or element-type; and that isn't half of it).
Implementations can fuse .from().map() as well as .map().from();
.from() is a static method, so .map().from() won't work.
Thanks, I meant .from().map() (map after from) and .from( .map() ) (from after map).
Claus
On Feb 11, 2013, at 7:13 PM, Nathan Wall wrote:
Thank you for this explanation. This is very interesting! If you don't mind, I have some questions/concerns I'd like to clear up. (In particular I want to make sure I understand; I don't intend to argue for one side or the other ATM, but this change may require me to refactor some old code to be future-ready).
Allen Wirfs-Brock wrote:
The choice we agreed to, at the meeting is
- Array.prototype.map produces the same kind of array that it was applied to, so:
for the above example m instance of V will be true. intArray.map(v=>v.toSring()) produces an Int32Array. The strings produced by the map function get converted back to numbers.
Sounds cool! Does it work?
I rely on the genericness of Array methods quite a lot. For example, it's fairly common for me to write something like the following:
var map = Function.prototype.call.bind(Array.prototype.map), select = document.querySelectorAll.bind(document); var ids = map(select('div'), function(el) { return el.id; });Currently this would get me an array of string ids for every div on a page. In a future ES6 with a future DOM where NodeList extends Array, will
idsno longer hold an array of strings but try to remain a NodeList?
Anything that works with ES5 semantics should continue to work, assuming nothing else changes.
If NodeList was turned into an actual subclass of Array, then the default behavior of Array.prototype.map when applied to a NodeList will be to create a new NodeList, assuming that NodeList supports all the mechanism that map uses to create its result object. Of course, the design of this new NodeList has the option of over-riding the inherited definition of map and/or opt-ing out of the create something other than Array instance behavior.
Regardless, because of backwards compatibility concerns, I'm skeptical that NodeList could ever be made into an Array subclass. It seems more likely that a new kind of DOM Node collection that was an Array subclass might be introduced
There's a good chance this will break some of my code. I'm capable of changing my code and writing this to be more future-friendly from now on (I'm not one who prefers backwards compatibility over a better language). But I would have always assumed I was doing things correctly before, and I'm curious if the rest of the internet will be okay..?
We can't break existing code. Where things get messy is when old code is combined with new code that uses new features. For example, you map function will produce a SubArray instance if it is called with a SubArray instance assuming that SubArray is a real ES6 subclass of Array as its first argument.
On Feb 12, 2013, at 4:54 AM, Aymeric Vitte wrote:
Le 10/02/2013 19:05, Allen Wirfs-Brock a écrit :
All the presentation decks from the meeting are publicly available at meetings:meeting_jan_29_2013
the short form:
In ES5, today you can use Array.prototype as the [[Prototype]] of non-Array objects or apply the Array.prototype methods to non-arrays. The behaviors they get are non-necessarily what you would expect if you would really subclass Array. We need to preserve the existing behaviors for these existing use cases but would like to provide the more reasonable behavior when class extends Array is used to create a subclass.
Array.prototype.concat is the most problematic of the existing methods in this regard.
After reading this thread and the slides, what is the plan for Typed Arrays (dedicated array like methods or Array subclassed like) ?
As we discussed at the meeting, to rationally place Typed Arrays into the same class hierarchy as Array would require refactoring Array.prototype into multiple abstract superclasses. This seems like too large of a change. Instead the plan that we agreed to is to introduce TypedArray as an abstract superclass of all of the specific Typed Array constructors. TypedArray.prototype will be specified to have all of the Array.prototype methods that don't dynamic change the length of an array. (TypedArray.prototype will also define set and subarray from the Khronos Typed Array spec.)
How I hope to specify TypedArray.prototype is to say that any of its properties that are the same as the corresponding Array.prototype property may, at the discretion of the implementation, be implemented using the same function objects as Array.prototype or by using a different function that conforms to the same specification. This would give an implementation an option of using either the same Array.prototype methods or of providing versions of the methods that are optimized for use with Typed Arrays.
It's possible that this may not fly. Such a choice of implementation approach allows more freedom then we normally give implementations and the choice would be detectible by user code doing something like: Array.prototype.map === TypedArray.prototype.map
If we find we can't allow that flexibility my next choice is to to say that both Array.prototype and TypedArray.prototype share in common those function objects and leave it to implementations to hide any Type Array specific optimizations inside of the functions. The reasons this is my preference, over simply saying they are all different function objects that share a common specification, is that I'm generally trying to minimize the number of new built-in functions that need to be created every-time a Realm (eg, iframe) is created.
I see the constraints for subclassing, but the solution a.from(b,map) seems a little bit strange (even if I agree that it should return the same instance as a) , and what about concat, slice, etc as you mention? Couldn't we add something like a parameter to the methods : example.map(f,thisArg, true) or example.map(f,true) returns example's instance instead of Array (same processing as .of, .from), so you can explicitely say that you want to return the same instance and don't have backward compatilities issues (hopefully...) ?
note it is A.from(b,mapfn), eg; Array.from(b, mapfn) or Uint32Array.from(b,mapfn). A boolean valued parameter isn't sufficient because you need the flexibility to identify the specific kind of array you want to map into.
Either of the above read perfectly fine to me: I want to create a new Array (or Uint32Array) from the elements of b transformed by mapfn. All the essential pieces of information are there and the conceptual model seems just fine.
I have used a lot Typed Arrays for [1] [2] [3] and all along wondering why Array like optimized methods were not available, especially concat, and why it was speced entirely in TC39 while a spec already exists (which specifies slice/subarray but surprisingly not concat), but it will be extremely usefull to have the Array like methods in the ES specs for Typed Arrays.
The intent of TC39 adopting Typed Array is to allow us to fully integrate it into the language, rather than it being a "add on" that doesn't follow the normal ES conventions. Make the (applicable) Array.prototype methods available is part of that integration.
Hey Allen, thanks for clarifying.
What will happen to other Array methods which currently return Arrays? filter is the primary one that comes to mind. Will uint32array.filter((v) => v != 0) return a Uint32Array? (I think it should behave the same way map does.)
Additionally, what will happen with the return values of slice, splice, and reverse?.. not to mention concat, which I know is a much more complex beast.
Nathan
On 14 February 2013 02:35, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
As we discussed at the meeting, to rationally place Typed Arrays into the same class hierarchy as Array would require refactoring Array.prototype into multiple abstract superclasses. This seems like too large of a change. Instead the plan that we agreed to is to introduce TypedArray as an abstract superclass of all of the specific Typed Array constructors. TypedArray.prototype will be specified to have all of the Array.prototype methods that don't dynamic change the length of an array. (TypedArray.prototype will also define set and subarray from the Khronos Typed Array spec.)
I'm still not convinced that introducing a separate class only for masking a handful of length-modifying array methods is worthwhile. After all, ordinary arrays can be sealed or frozen, too, while still providing those methods, so typed arrays are not new or different in that regard.
I'd say that either we properly clean up the Array hierarchy, or we leave it alone. A half-baked solution that only applies to typed arrays, and divorces them from the Array hierarchy, seems less attractive than just doing the naive thing, i.e., TypedArray < Array.
Nathan Wall wrote:
Hey Allen, thanks for clarifying.
What will happen to other Array methods which currently return Arrays?
filteris the primary one that comes to mind. Willuint32array.filter((v) => v != 0)return a Uint32Array? (I think it should behave the same waymapdoes.)
I know I already said that, but I repeat: map transforms elements,
filter just selects a subset. Therefore I am 100% for filter doing
the same kind of collection as the receiver. Bit not necessarily for map.
So I think, not, filter should not behave same as map.
Additionally, what will happen with the return values of
slice,splice, andreverse?.. not to mentionconcat, which I know is a much more complex beast.
I think filter should behave the same as slice, splice, reverse
and concat. They have clear rationale to use same kind of container.
Andreas Rossberg wrote:
I'd say that either we properly clean up the Array hierarchy, or we leave it alone. A half-baked solution that only applies to typed arrays, and divorces them from the Array hierarchy, seems less attractive than just doing the naive thing, i.e., TypedArray< Array.
Agree with that, and I'll go further: we should leave alone what's already shipped and in use for a long time.
TypedArray < Array sounds good to me.
I'd say that either we properly clean up the Array hierarchy, or we leave it alone. A half-baked solution that only applies to typed arrays, and divorces them from the Array hierarchy, seems less attractive than just doing the naive thing, i.e., TypedArray< Array.
Agree with that, and I'll go further: we should leave alone what's already shipped and in use for a long time.
TypedArray < Array sounds good to me.
The question is how to clean up/refine the class hierarchy with the existing language means. Consider a hypothetical
FixedLengthArray < Array
and a FixedLengthTypedArray that inherits from both branches.
More immediately relevant for this thread, I would like to see
Array < Container
with map, from, filter, and perhaps some others, moving from Array to Container. Then Map and Set would be Containers, supporting operations currently limited to Array (WeakMap is probably too special to be a normal Container).
Claus
On Feb 14, 2013, at 8:47 AM, Nathan Wall wrote:
Hey Allen, thanks for clarifying.
What will happen to other Array methods which currently return Arrays?
filteris the primary one that comes to mind. Willuint32array.filter((v) => v != 0)return a Uint32Array? (I think it should behave the same waymapdoes.)
filter, splice, splice, reverse, etc. all copy elements that from an existing array or array subclass so the result can/should be the same class is the source array. Map is different because it can produce new element values of an arbitrary kind that may not fit into the same kind of array as the source array
Additionally, what will happen with the return values of
slice,splice, andreverse?.. not to mentionconcat, which I know is a much more complex beast.
WRT to concat, see last three slides of meetings:subclassing_builtins.pdf
Claus Reinke wrote:
More immediately relevant for this thread, I would like to see
Array < Container
with map, from, filter, and perhaps some others, moving from Array to Container. Then Map and Set would be Containers, supporting operations currently limited to Array
This is not gonna happen for several reasons, one being backward incompatibility.
It's also unnecessary. The generic methods could (and should, some think -- I prototyped this years ago in SpiderMonkey) have their |this| parameters uncurried and be provided as functions. That would be a better route than OOP tyranny of hierarchy.
More immediately relevant for this thread, I would like to see
Array < Container
with map, from, filter, and perhaps some others, moving from Array to Container. Then Map and Set would be Containers, supporting operations currently limited to Array
This is not gonna happen for several reasons, one being backward incompatibility.
It's also unnecessary. The generic methods could (and should, some think -- I prototyped this years ago in SpiderMonkey) have their |this| parameters uncurried and be provided as functions. That would be a better route than OOP tyranny of hierarchy.
That particular suggestion was somewhat tongue-in-cheek, to get the thread back to technical issues;-) - I don't want the operations limited to Array (and subclasses) but I also don't think that single- inheritance prototype hierarchy is offering a good solution here.
To begin with, the usefulness of map isn't limited to container-like classes, and map's implementation cannot always be inherited, so we're really talking about interfaces (or abstract classes).
And if existing code was written to interfaces, rather than taking a specific prototype chain into account, backward incompatibility would be less of a problem.
The whole thread is about using a specific and acute example decision (what to do with .map/.from) to trigger a discussion of the larger language design issues and options that would help addressing this kind of problem (which combines programming to interfaces, generic methods outside the class hierarchy, and selection of such methods based on the class of not-yet-existing objects). Preferably before individual cases are fixed that might not fit a later general solution pattern.
I mentioned type constructor classes not because of the type system they are embedded in (or, rather, built over), but because they offer a logic and implementation pattern for dealing with interfaces and with generic code written to interfaces.
The un-optimized implementation of generic code in that system is to pass dictionaries of methods around, combining the method implementations in a way analogous to the way the types are constructed. It is a systematic generalization of the target-class- passing illustrated in Rick's code fragment earlier in this thread.
I'd be interested to see your alternative design suggestion (or, in fact, any more general approach to the issue at hand that would fit into ES).
Claus
Claus Reinke wrote:
I'd be interested to see your alternative design suggestion (or, in fact, any more general approach to the issue at hand that would fit into ES).
From ES4, proposals:static_generics.
I'd be interested to see your alternative design suggestion (or, in fact, any more general approach to the issue at hand that would fit into ES).
From ES4, proposals:static_generics.
Thanks for the pointer. However, I'd like to have even more generic methods - e.g., map is useful for structures that do not have .length or indexed elements.
To give you an idea of the possibilities I've got in mind, I've sketched an overly simplistic system where classes can implement interfaces, generic methods can be written over these implementations, and generic code can be written using only the generic methods. The code is available as
https://gist.github.com/clausreinke/4997274
(you can run the "html" in a browser or with nodejs)
It does implement map for Array, String and Int32Array (also for makePromise, if you're running this in nodejs, with q installed), without extending the classes or their objects (no mixins, and the generic map is class-independent and extensible).
To understand why this is simplistic, think about the copy&modify involved with doing this for all typed array variations - that is where class-level constructors and de-structurable classes would come in handy, so that one could separately reuse the array- and element-type-specific interface implementations (*).
Claus
(*) for those who do not switch off when they hear the word 'types' - even without a static type system, many of the ideas could be carried over from how Haskell implements type classes; it'll just be a little less convenient, and a little more explicit code to write; and it'll require some language design work to translate ideas away from the static typing context, into EcmaScript concepts;
if you want to read Haskell papers for ES language design ideas,
here are the beginnings of a little dictionary:
for 'type', read 'class'
for 'type class', read 'interface'
for 'type class instance', read 'interface implementation'
With this translation and the paper I referred to earlier in this
thread, one notices that interfaces in Haskell tend to relate
multiple classes, and that interface-implementing classes can
often be de-structured (e.g., Array<Int32> instead of Int32Array),
to facilitate access to the interfaces implemented by the class
components - that avoids a lot of duplicated code;
I am trying to understand the discussion and resolution of 'The Array Subclassing "Kind" Issue'. The issue (though not its solution) seemed simple enough
and I was expecting solutions somewhere along this path:
.map should work for Array subclasses, preserving class
.map is independent of Array and its subclasses, there are lots of types for which it makes sense (Sets, EventEmitters, ..)
there should be an interface Mapable, implemented by Array and its subclasses, but also by other relevant classes, such that
class M implements Mapable { ... } m = (new M()).map(val => val); console.log( m instanceof M ); // true
(in typed variants of JS, this would call for generics, to separate structure class -supporting map- from element class -being mapped)
Instead, the accepted approach -if I understood it correctly- focuses on conversion and iterables:
such that
This seems very odd to me, because
it introduces a second form of .map, in .from
instead of limiting to Array, .from-map is now limited to iterables (it would work for Set, which is really OrderedSet, but it wouldn't work for WeakMap)
it doesn't address the general problem: how to inherit structural functionality (such as mapping over all elements or a container/ iterable) while preserving class
With a general solution to the issue, I would expect to write
while also getting
Could someone please elaborate why the committee went with an additional map built into structure conversion instead?
Claus
PS. What about array comprehensions and generator expressions?
I am trying to understand the discussion and resolution of 'The Array Subclassing "Kind" Issue'. The issue (though not its solution) seemed simple enough class V extends Array { ... } m = (new V()).map(val => val); console.log( m instanceof V ); // false :( and I was expecting solutions somewhere along this path: 1. .map should work for Array subclasses, preserving class 2. .map is independent of Array and its subclasses, there are lots of types for which it makes sense (Sets, EventEmitters, ..) 3. there should be an interface Mapable, implemented by Array and its subclasses, but also by other relevant classes, such that class M implements Mapable { ... } m = (new M()).map(val => val); console.log( m instanceof M ); // true (in typed variants of JS, this would call for generics, to separate structure class -supporting map- from element class -being mapped) Instead, the accepted approach -if I understood it correctly- focuses on conversion and iterables: Array.from( iterable ) => Array.from( iterable, mapFn ) such that SubArray.from( iterable, val => val ) instanceof SubArray This seems very odd to me, because - it introduces a second form of .map, in .from - instead of limiting to Array, .from-map is now limited to iterables (it would work for Set, which is really OrderedSet, but it wouldn't work for WeakMap) - it doesn't address the general problem: how to inherit structural functionality (such as mapping over all elements or a container/ iterable) while preserving class With a general solution to the issue, I would expect to write SubArray.from( iterable ).map( val => val ) instanceof SubArray while also getting new Mapable().map( val => val ) instanceof Mapable Could someone please elaborate why the committee went with an additional map built into structure conversion instead? Claus PS. What about array comprehensions and generator expressions?