Array.from and sparse arrays

# medikoo (8 years ago)

Currently per spec Array.from doesn't produce perfect copies of sparse arrays:

Array.from([1,,2]); // [1, undefined, 2]

I know it's been discussed but there's not much feedback.

It doesn't seem right for a few reasons:

  1. Array.from already produces sparse arrays from array-likes:

    Array.from({ 0: 1, 2: 2, length: 3 }); // [1,,2]
    

    So why it doesn't from sparse arrays?

  2. Array.from can't be used as generic plain array copy producer.

In ES5 this can be achieved, with arr.slice(). However as in ES6 slice will return object of same type as one on which it is called, so it is no longer that straightforward, especially if we're strongly after plain array.

The only 100% bulletproof solutions we have in ES6 are either:

var plainCopy = new Array(arr.length)
arr.forEach(function (v, i) { plainCopy[i] = v; });

or other quite dirty way:

Array.from(Object.assign({ length: arr.length}, arr));

Other related question:

Why do array iterators go through not defined indexes? It seems not consistent with other iteration methods we have since ES5, are there any plans to use sparse iteration kinds 2 or are they defined just to reserve eventual future use?

# Allen Wirfs-Brock (8 years ago)

Also see ecmascript#2416

On Feb 21, 2014, at 8:13 AM, medikoo wrote:

  1. Array.from already produces sparse arrays from array-likes:

    Array.from({ 0: 1, 2: 2, length: 3 }); // [1,,2]
    

    So why it doesn't from sparse arrays?

Perhaps, this is an inconsistency that should be corrected by changing the spec. to produce [1,2,undefined] in the above case.

The current definition was derived from the legacy algorithms such as A.p.slice which preserve holes. But as the current consensus is Array.from does not preserve hole for Iterable arrays then perhaps is also should preserve them for non-iterable array-likes,

  1. Array.from can't be used as generic plain array copy producer.

When this was most recently discussed at 1 the case was made that in JS sparseness is seldom what anybody actually wants, hence the current Array.from behavior will be what's desired most of the time. Do you have any real world use cases in mind that are driving the desire for Array.from to preserve sparseness?

In ES5 this can be achieved, with arr.slice(). However as in ES6 slice will return object of same type as one on which it is called, so it is no longer that straightforward, especially if we're strongly after plain array.

On the other hand it is fairly straightforward to define:

class SparseArray extends Array {
   static from(collection) { ... /* over-ride to preserve holes */}
   *keys() {for (let k of super.keys()) if (Object.hasOwnProperty(this,k)) yield k}
   *entries() {for (let [k,v] of super.entries()) if (Object.hasOwnProperty(this,k)) yield [k,v]}
}

are there any plans to use sparse iteration kinds [2] or are they defined just to reserve eventual future use?

That's an rement that is now gone. When I first define Array Iterators I allowed for the possibility of doing sparse iterations. However, there was no public API for doing so and nobody ever proposed or strongly advocated for one the spare iteration support was removed from the spec. The mention of spare iteration in Table 42 is something I originally missed removing, but it's now gone.

# Brendan Eich (8 years ago)

Allen Wirfs-Brock wrote:

Perhaps, this is an inconsistency that should be corrected by changing the spec. to produce [1,2,undefined] in the above case.

No way. The object has length 3 and index 2 has value 2. Why in the world would Array.from re-index?

The current definition was derived from the legacy algorithms such as A,p.slice which preserve holes. But as the current consensus is Array.from does not preserve hole for Iterable arrays then perhaps is also should preserve them for non-iterable array-likes,

"also should not"?

At the last TC39 meeting, we agreed holes are freakish and should be discounted in designing new APIs like Array.from (I think; I may be overstating slightly, but that's the effect).

Whatever we do, we should be consistent among sparse arrays and sparse arraylikes, it seems to me. Want a bug?

When this was most recently discussed at [1] the case was made that in JS sparseness is seldom what anybody actually wants, hence the current Array.from behavior will be what's desired most of the time. Do you have any real world use cases in mind that are driving the desire for Array.from to preserve sparseness?

Use-cases for sparse arrays + copying them would be really helpful.

# Allen Wirfs-Brock (8 years ago)

On Feb 21, 2014, at 11:49 AM, Brendan Eich wrote:

No way. The object has length 3 and index 2 has value 2. Why in the world would Array.from re-index?

Sorry, fuzzy eyes after early morning eye checkup. I meant: [1,undefined, 2], just like what Array.from([1,,2]) is currently spec'ed to produce.

"also should not"?

right, not

# medikoo (8 years ago)

Brendan Eich wrote

Use-cases for sparse arrays + copying them would be really helpful.

It was more call for a consistency, as all other methods are gentle to sparse arrays, this one suddenly isn't and I was wondering what's the reason for that.

It also brings confusion. I try to follow same patterns designing methods that are not part of a standard, and now it's hard to decide how to handle sparse arrays case, as it's in map or as in from (?)

However if you ask me for real world cases for sparse arrays, none comes to my mind now. Personally I don't really use them (I think), but it'll be good to hear from others.

# C. Scott Ananian (8 years ago)

There are a number of differences between ES5-style methods and ES6 methods. This is unfortunate, but probably inevitable.

  • ES5 methods typically use "argument is present", while ES6 methods treat an undefined argument as the same as a missing argument. For example, compare Array#reduce to Array#fill. This is done to be consistent with the new ES6 syntax for optional arguments.

  • ES5 methods typically respect sparse arrays, ES6 fill in holes. Improvements to subclass support in ES6 mean that sparse arrays can be implemented as subclasses where they are actually wanted.

...probably the gurus on the list can supply more examples.

# C. Scott Ananian (8 years ago)

On Fri, Feb 21, 2014 at 9:49 AM, Brendan Eich <brendan at mozilla.com> wrote:

Whatever we do, we should be consistent among sparse arrays and sparse arraylikes, it seems to me. Want a bug?

Filed ecmascript#2562

# Allen Wirfs-Brock (8 years ago)

On Feb 21, 2014, at 1:15 PM, C. Scott Ananian wrote:

There are a number of differences between ES5-style methods and ES6 methods. This is unfortunate, but probably inevitable.

  • ES5 methods typically use "argument is present", while ES6 methods treat an undefined argument as the same as a missing argument. For example, compare Array#reduce to Array#fill. This is done to be consistent with the new ES6 syntax for optional arguments.

I think you're jump to a false conclusion here by comparing the arguments of methods have have distinct argument usage patterns.

A better comparison would be between fill and slice. They both have start and end arguments which may be missing and they both handle them in the same way:

slice: 6. Let relativeStart be ToInteger(start). 9. If end is undefined, let relativeEnd be len; else let relativeEnd be ToInteger(end).

fill: 6. Let relativeStart be ToInteger(start). 9. If end is undefined, let relativeEnd be len; else let relativeEnd be ToInteger(end).

# C. Scott Ananian (8 years ago)

I actually just responded in more depth over at esdiscuss.org/topic/what-does-is-not-present-mean-in-spec-algorithms#content-9

Let's continue the discussion over there.

# Nick Krempel (8 years ago)

A sparse array is useful whenever the index represents an externally meaningful piece of data (like an ID).

This use case could be replaced with a Map which just uses integer keys, but I believe JavaScript engines are better optimized working with Arrays here, where the keys are known to be integers. So in some cases when you care about performance, you may want to use a sparse array. (For some JavaScript engines and some ranges of array size you may in fact be better off with a dense array for performance, even if it means spending time filling it with undefined. That may not be an option if 'undefined' is a meaningful value differing from missing.)

# Nick Krempel (8 years ago)

And here's another sort of case where sparse arrays are useful, a more concrete example:

A partial application function which takes a function (as the 'this' argument in this example) and a (sparse) array specifying which parameters to bind: f.partial(['a', , 'b']) is then different from f.partial(['a', undefined, 'b']) in that the former binds 'a' to the first parameter and 'b' to the third parameter (leaving the second parameter unbound), while the latter binds 'a' to the first parameter, undefined to the second parameter, and 'b' to the third parameter. You could use an object like {0: 'a', 2: 'c'} but this is clearly clunkier syntax (and likely a problem in performance-critical code too, as number->string conversions are happening in many engines here). You could use a Map with only integer keys, but this is also clunky and is more weakly typed in that non-integer keys potentially have to be checked for.

# Claude Pache (8 years ago)

Here is an idea for easily defining an "iterator with holes", so that Array.from could reconstruct a sparse array from an iterable:

Iterators yield IteratorResult objects, which are of the form {done: <boolean>, value: <any>}. The idea is to produce results of the form {done: true} (omitting the value key) in order to indicate a "hole". For consumers that don't want holes (e.g. for/of loops), it will be equivalent to {done: true, value: undefined}, but consumers that are willing to make the difference between holes and undefined can distinguish the two cases.

For instance, successive application of the next method on [1, , 2].values() will produce the results:

{done: false, value: 1}  ;  {done: false}  ;  {done: false, value: 2}  ;  {done true, value: undefined}

whereas [1, undefined, 2].values() will produce

{done: false, value: 1}  ;  {done: false, value: undefined}  ;  {done: false, value: 2}  ;  {done true, value: undefined}

(Naturally, generators won't be able to produce "iterators with holes": if you really want such a silly iterable, the punishment is that you are obliged to construct it "by hand".)

# Nick Krempel (8 years ago)

You presumably mean "... of the form {done: false} ...".

The possibility of a "yield" without an assignment expression meaning "yield a hole" might be left open for the future (so some clients would treat it the same as "yield undefined", but those who care to distinguish could do). There would need to be a disambiguation rule for expressions beginning "yield /" however.

# Nick Krempel (8 years ago)

On 24 February 2014 23:30, Nick Krempel <ndkrempel at google.com> wrote:

The possibility of a "yield" without an assignment expression meaning "yield a hole" might be left open for the future (so some clients would treat it the same as "yield undefined", but those who care to distinguish could do). There would need to be a disambiguation rule for expressions beginning "yield /" however.

...and presumably "yield +", "yield -", so maybe it's too ugly.

# Claude Pache (8 years ago)

Personally, I consider that the impossibility to "yield a hole" must be considered as a feature, not a bug. Holes are useful in order to have consistent results for Array.from([1, , 3]) (i.e., getting an exact copy), but their use should not be encouraged. (Note that, if you really want to, you can always (painfully) wrap a generator producing sentinel values with a hand-made iterable that forwards the results, transforming sentinel values into holes in the process.)

# Allen Wirfs-Brock (8 years ago)

On Feb 24, 2014, at 4:18 PM, Claude Pache wrote:

Personally, I consider that the impossibility to "yield a hole" must be considered as a feature, not a bug. Holes are useful in order to have consistent results for Array.from([1, , 3]) (i.e., getting an exact copy), but their use should not be encouraged. (Note that, if you really want to, you can always (painfully) wrap a generator producing sentinel values with a hand-made iterable that forwards the results, transforming sentinel values into holes in the process.)

It easy enough to write an keys or entries iterator that ignores holes:

function *sparseKeys(array) {
   for (indx of array.keys()) if (Object.hasOwnProperty(array, key)) yield indx;
}

function *sparseEntries(array) {
   for (entry of array.entries()) if (Object.hasOwnProperty(array, entry[0])) yield entry;
}

The same thing could be done for values but that seems less useful.

# Claude Pache (8 years ago)

Indeed, but the question was about producing/forwarding holes rather than skip them, so that, e.g., Array.from could replace the elements at the correct positions.

# C. Scott Ananian (8 years ago)

You'd probably want Array.fromEntries then, and pass in an appropriate entries iterator. Note that Map() already takes an entries iterator; maybe you'd want an Object.fromEntries as well?

The more this is discussed, the more I am convinced that the current "holes are evil" behavior of Array.from and the ArrayIterator is the correct thing. Once you start introducing holes, they crop up everywhere. Just mentally tell yourself that Array.from is a method to fill in holes, not a method to clone an array. Use one of the discussed workarounds if you need holes.

# Brendan Eich (8 years ago)

+1

That's our story and we're sticking to it, I think.