Discuss
  • holes in spread elements/arguments

holes in spread elements/arguments

# Sean Eagan (14 years ago)

Sorry to drag on about array holes, but:

Should holes in spread elements (e.g. [...holeyArray]) and arguments (e.g. f(...holeyArray)) reflect as holes in the array they are spliced into to avoid information loss ? The current proposal [1] reflects them as undefined values.

Spread arguments will reflect in both the arguments array and the rest parameter binding if there is one, so this would allow them to have holes. The only issue I see is that Function.prototype.apply reflects holes as undefined values in the arguments array, so if f is a function whose code loops over arguments skipping holes (most likely via ES5 Array extras), then the following migration could cause holes to be skipped which were previously not:

f.apply(holeyArray) ->

f(...holeyArray)

This should be rare, since for one thing holey arrays are rare, and it's unlikely anything important was being done with the undefined values which are now being skipped. If this is an issue, the holes could only be reflected in rest parameter bindings since they are meant to replace arguments in the long run anyways.

On a separate note, I'm not sure if argument lists should be made symmetric to array element lists, by allowing them to have holes (e.g. f(1, , 3)), but it would essentially allow undefined arguments to be ommitted, if that would be considered useful.

[1] harmony:spread

Thanks, Sean Eagan

# Allen Wirfs-Brock (14 years ago)

On Sep 29, 2011, at 11:07 AM, Sean Eagan wrote:

Sorry to drag on about array holes, but:

Should holes in spread elements (e.g. [...holeyArray]) and arguments (e.g. f(...holeyArray)) reflect as holes in the array they are spliced into to avoid information loss ? The current proposal [1] reflects them as undefined values.

You need to read the draft ES6 spec. It specifies that spread used in array literals preserves holes. I haven't written the spec. for spread in argument lists but clearly interior holes need to be passed as undefined. Trailing holes in a spread in the final argument position could be trimmed but that would mean that:

foo.apply(undefined, new Array(5))

and foo(...(new Array(5))

wouldn't pass the same thing which seems like it would be an issue.

# Sean Eagan (14 years ago)

On Thu, Sep 29, 2011 at 1:31 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

On Sep 29, 2011, at 11:07 AM, Sean Eagan wrote:

Sorry to drag on about array holes, but:

Should holes in spread elements (e.g. [...holeyArray]) and arguments (e.g. f(...holeyArray)) reflect as holes in the array they are spliced into to avoid information loss ?  The current proposal [1] reflects them as undefined values.

You need to read the draft ES6 spec.  It specifies that spread used in array literals preserves holes.

Sorry. I agree.

I haven't written the spec. for spread in argument lists but clearly interior holes need to be passed as undefined.  Trailing holes in a spread in the final argument position could be trimmed but that would mean that:

So you don't think holes should appear in rest parameters or arguments ? The only reason I can think of to require that is that old code may expect arguments not to contain holes, but then it could be spec'ed that holes only appear in rest parameters. Beyond that is seems like it would make sense to be consistent with array literal hole preservation.

Thanks, Sean Eagan

# Allen Wirfs-Brock (14 years ago)

On Sep 29, 2011, at 12:14 PM, Sean Eagan wrote:

On Thu, Sep 29, 2011 at 1:31 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

...

I haven't written the spec. for spread in argument lists but clearly interior holes need to be passed as undefined. Trailing holes in a spread in the final argument position could be trimmed but that would mean that:

So you don't think holes should appear in rest parameters or arguments ? The only reason I can think of to require that is that old code may expect arguments not to contain holes, but then it could be spec'ed that holes only appear in rest parameters. Beyond that is seems like it would make sense to be consistent with array literal hole preservation.

No I don't. Currently there is no ways to syntactically generate an actual argument list with holes and I think that allowing ,, in argument lists is not particularly desirable. Also apply replaces holes with undefined so there is currently no way to create an arguments object that initially contains holes (somebody could delete elements after it was created). It seems highly desirable that we preserve the existing semantics of arguments and that: function foo(...args) { assert(length.arguments === length.args}; for (let i in arguments) assert(arguments.hasOwnProperty(i) === args.hasOwnProperty(i) && arguments[i] === args[i]); }

regardless of how foo was called. In other words, baz(...bar) and baz.apply(undefined,bar) should always produce the same effect.

Also, note that I've currently spec'd ES6 so that 'arguments' can be used as a rest parameter name. In other words function foo(...args) {return args} and function foo(...arguments) {return arguments} are semantically equivalent.

I did this to enable to make it easier to migrate code such as: function foo() {baz(arguments)} to function foo(...arguments) {baz(arguments)}

# Sean Eagan (14 years ago)

On Thu, Sep 29, 2011 at 2:53 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

It seems highly desirable that we preserve the existing semantics of arguments

I agree, for backward compatability.

and that:    function foo(...args) {       assert(length.arguments === length.args};       for (let i in arguments) assert(arguments.hasOwnProperty(i) === args.hasOwnProperty(i) && arguments[i] === args[i]);    }

I disagree, rest parameters are intended as an eventual permanent replacement for arguments, and thus it is more important to make them have the correct and consistent with array literal spread element hole preservation behavior, than consistency with what will be the legacy arguments to aid a one time migration to rest parameters. To illustrate the problems I see with filling holes in rest parameters, assume we are working with a holey array:

let holeyArray = []; holeyArray[99] = true;

and some code which uses it:

let spreadTarget = [a, ...holeyArray, b]; // 99 holes holeyArray.forEach(expensiveOperation); // 1 iteration

at some point we decide to abstract this code into functions:

function f(...arr) { return [a, b, ...arr]; } function g(...arr) { arr.forEach(foo); // holes now filled leading to extra iterations } ... let spreadTarget = f(...holeyArray); // 99 unwanted undefineds now g(...holeyArray); // 100 iterations now

regardless of how foo was called.  In other words,  baz(...bar) and baz.apply(undefined,bar) should always produce the same effect.

I agree since one form is not intended to replace the other. The effect IMHO should be that if baz uses arguments, holes are not preserved for backward compatability, but if baz has upgraded to rest parameters, then holes are preserved.

Also, I think not filling holes can be optimized since the undefined values don't need to be added to the rest parameter argument.

Thanks, Sean Eagan

# Erik Arvidsson (14 years ago)

On Thu, Sep 29, 2011 at 12:53, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

Also, note that I've currently spec'd ES6 so that 'arguments' can be used as a rest parameter name.  In other words    function foo(...args) {return args} and   function foo(...arguments) {return arguments} are semantically equivalent. I did this to  enable to make it easier to migrate code  such as:    function foo() {baz(arguments)} to    function foo(...arguments) {baz(arguments)}

Minus the Arguments class I hope.

# Allen Wirfs-Brock (14 years ago)

On Sep 30, 2011, at 9:42 AM, Erik Arvidsson wrote:

On Thu, Sep 29, 2011 at 12:53, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

Also, note that I've currently spec'd ES6 so that 'arguments' can be used as a rest parameter name. In other words function foo(...args) {return args} and function foo(...arguments) {return arguments} are semantically equivalent. I did this to enable to make it easier to migrate code such as: function foo() {baz(arguments)} to function foo(...arguments) {baz(arguments)}

Minus the Arguments class I hope.

I'm missing you point. What do you mean by "Arguments" class. as currently spec'd function foo(...arguments) {baz(arguments) } is semantically the same as function foo(...args) {baz(args) }

The only difference is the name binding for the rest parameter. In both cases it is a regular Array object, not an ES5 arguments objects.

Did you have something else in mind?

# Erik Arvidsson (14 years ago)

On Fri, Sep 30, 2011 at 15:27, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

On Sep 30, 2011, at 9:42 AM, Erik Arvidsson wrote:

On Thu, Sep 29, 2011 at 12:53, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

Also, note that I've currently spec'd ES6 so that 'arguments' can be used as a rest parameter name.  In other words    function foo(...args) {return args} and   function foo(...arguments) {return arguments} are semantically equivalent. I did this to  enable to make it easier to migrate code  such as:    function foo() {baz(arguments)} to    function foo(...arguments) {baz(arguments)}

Minus the Arguments class I hope.

I'm missing you point. What do you mean by "Arguments" class. as currently spec'd   function foo(...arguments) {baz(arguments) } is semantically the same as   function foo(...args) {baz(args) }

The only difference is the name binding for the rest parameter.  In both cases it is a regular Array object, not an ES5 arguments objects.

Did you have something else in mind?

Nope. That covers it.

# Sean Eagan (14 years ago)

So I realized hole preservation is actually relevant in both array structuring and destructuring. Array structuring and destructuring ellipses forms are both specified in the draft spec to preserve holes.

Argument lists are merely array structuring patterns, and parameter lists are merely array destructuring patterns, so to me, their syntax and semantics should be the same. Thus, ellipses forms for argument lists and parameter lists should be hole preserving as with array structuring / destructuring. Take this example:

// rest has 98 holes [a, b, ...rest] = [...Array(100)];

// extract into function form ->

// rest should still have 98 holes, not 98 undefined's (function(a, b, ...rest){})(...Array(100));

On Fri, Sep 30, 2011 at 8:41 AM, Sean Eagan <seaneagan1 at gmail.com> wrote:

On Thu, Sep 29, 2011 at 2:53 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

It seems highly desirable that we preserve the existing semantics of arguments

I agree, for backward compatability.

and that:    function foo(...args) {       assert(length.arguments === length.args};       for (let i in arguments) assert(arguments.hasOwnProperty(i) === args.hasOwnProperty(i) && arguments[i] === args[i]);    }

I disagree, rest parameters are intended as an eventual permanent replacement for arguments, and thus it is more important to make them have the correct and consistent with array literal spread element hole preservation behavior, than consistency with what will be the legacy arguments to aid a one time migration to rest parameters.  To illustrate the problems I see with filling holes in rest parameters, assume we are working with a holey array:

let holeyArray = []; holeyArray[99] = true;

and some code which uses it:

let spreadTarget = [a, ...holeyArray, b]; // 99 holes holeyArray.forEach(expensiveOperation); // 1 iteration

at some point we decide to abstract this code into functions:

function f(...arr) {  return [a, b, ...arr]; } function g(...arr) {  arr.forEach(foo); // holes now filled leading to extra iterations } ... let spreadTarget = f(...holeyArray); // 99 unwanted undefineds now g(...holeyArray); // 100 iterations now

regardless of how foo was called.  In other words,  baz(...bar) and baz.apply(undefined,bar) should always produce the same effect.

I agree since one form is not intended to replace the other.  The effect IMHO should be that if baz uses arguments, holes are not preserved for backward compatability, but if baz has upgraded to rest parameters, then holes are preserved.

Also, I think not filling holes can be optimized since the undefined values don't need to be added to the rest parameter argument.

Thanks, Sean Eagan

Thanks, Sean Eagan

# Erik Arvidsson (14 years ago)

But Function.prototype.apply doesn't preserve holes. On Oct 3, 2011 6:50 AM, "Sean Eagan" <seaneagan1 at gmail.com> wrote:

So I realized hole preservation is actually relevant in both array structuring and destructuring. Array structuring and destructuring ellipses forms are both specified in the draft spec to preserve holes.

Argument lists are merely array structuring patterns, and parameter lists are merely array destructuring patterns, so to me, their syntax and semantics should be the same. Thus, ellipses forms for argument lists and parameter lists should be hole preserving as with array structuring / destructuring. Take this example:

// rest has 98 holes [a, b, ...rest] = [...Array(100)];

// extract into function form ->

// rest should still have 98 holes, not 98 undefined's (function(a, b, ...rest){})(...Array(100));

On Fri, Sep 30, 2011 at 8:41 AM, Sean Eagan <seaneagan1 at gmail.com> wrote:

On Thu, Sep 29, 2011 at 2:53 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

It seems highly desirable that we preserve the existing semantics of

arguments

# Sean Eagan (14 years ago)

On Mon, Oct 3, 2011 at 10:30 AM, Erik Arvidsson <erik.arvidsson at gmail.com> wrote:

But Function.prototype.apply doesn't preserve holes.

Function.prototype.apply doesn't preserve holes for the the arguments object, but rest parameters aren't yet standardized, and are intended to replace the arguments object, so they could, and in my view, should preserve holes for both Function.prototype.apply and foo(...args) activations, at least if array structuring and destructuring preserve holes as currently specified in the ES.next draft spec.

Thanks, Sean Eagan

# Allen Wirfs-Brock (14 years ago)

On Oct 3, 2011, at 6:49 AM, Sean Eagan wrote:

So I realized hole preservation is actually relevant in both array structuring and destructuring. Array structuring and destructuring ellipses forms are both specified in the draft spec to preserve holes.

Argument lists are merely array structuring patterns, and parameter lists are merely array destructuring patterns, so to me, their syntax and semantics should be the same.

This is your assertion, but it is debatable whether it is correct or desirable. In the evolution of the ES language, argument lists predates the existence of destructuring. In the legacy language neither the syntactic or applicative (apply function) support holes. In addition, the specification of the semantics of function invocation made no allowance for the possibility of holes in an argument list. So your assertion, to be considered true, requires several changes in the existing language semantics (and arguably the syntax of argument lists.

One concern that I've already mentioned is that recognizing holes in spread would create an inconsistency with the existing semantics of apply and that changing apply would be a breaking change that might impact existing code.

We also need to look at the interaction with default argument values:

function f(...argsf) { g(...argsf); g(argsf[0],argsf[1]); } function g(a='a', b='b') {console.log(a+" "+b)}

as currently specified: f(...[,1]); outputs "undefined 1", "undefined 1". With hole preservation it would output "a 1", "undefined 1".

It seems undesirable that two such calls to g would yield different results. Also as currently specified, if a format parameter is assigned its default value then it is guaranteed that all formals to its right will also be assigned their default value. This seems like a useful invariant which, as shown above, is lost if holes are preserved in this manner.

I think I understand the logic of your position, but I don't think there are other factors to consider that make your conclusion less clear cut.

# Allen Wirfs-Brock (14 years ago)

On Oct 3, 2011, at 8:45 AM, Sean Eagan wrote:

On Mon, Oct 3, 2011 at 10:30 AM, Erik Arvidsson <erik.arvidsson at gmail.com> wrote:

But Function.prototype.apply doesn't preserve holes.

Function.prototype.apply doesn't preserve holes for the the arguments object, but rest parameters aren't yet standardized, and are intended to replace the arguments object, so they could, and in my view, should preserve holes for both Function.prototype.apply and foo(...args) activations, at least if array structuring and destructuring preserve holes as currently specified in the ES.next draft spec.

In general, a call site doesn't have any knowledge of format of the formal parameter list of the callee and the callee doesn't have any knowledge of how the caller constructed the argument list. So the caller and callee argument semantics can't be interdependent.

Also, we probably need to factor in the fact that most current implementations probably don't have a mechanism for passing holes in an argument and that adding such a mechanism might be a deep and expensive change. Would the cost to implementors really be worth the benefit to JS developers?

# Sean Eagan (14 years ago)

On Mon, Oct 3, 2011 at 11:31 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

On Oct 3, 2011, at 6:49 AM, Sean Eagan wrote:

So I realized hole preservation is actually relevant in both array structuring and destructuring.  Array structuring and destructuring ellipses forms are both specified in the draft spec to preserve holes.

Argument lists are merely array structuring patterns, and parameter lists are merely array destructuring patterns, so to me, their syntax and semantics should be the same.

This is your assertion, but it is debatable whether it is correct or desirable.

Sorry, I should have said "should merely be" rather than "are merely".

In the evolution of the ES language, argument lists predates the existence of destructuring.  In the legacy language neither the syntactic or applicative (apply function) support holes.

apply supports holes...

f.apply(Array(5));

syntactic calls will support holes...

f(...Array(5))

What doesn't support holes is the arguments object which is on its way out.

In addition, the specification of the semantics of function invocation made no allowance for the possibility of holes in an argument list.

The specification of arguments dictates this, not function invocation as a whole.

So your assertion, to be considered true, requires several changes in the existing language semantics (and arguably the syntax of argument lists.

My assertion is that argument/parameter lists and array {de}structuring should have as much as possible the same syntax and semantics, considering that currently as far as i know:

  • the only syntactic difference is that array {de}structuring supports Elision, e.g. [,,,]
  • the only semantic difference is that array {de}structuring supports hole preservation
  • i.e. the only difference is that array {de}structuring supports holes

What about holes makes them applicable to arrays in general but not argument arrays ? Everything else in ES.next points to reification of arguments as plain old arrays, why be different here ?

I would actually be ok with or without both Elision syntax and hole preservation in ES.next, I just want consistency.

One concern that I've already mentioned is that recognizing holes in spread would create an inconsistency with the existing semantics of apply and that changing apply would be a breaking change that might impact existing code.

apply does not have existing semantics for rest parameters since they don't yet exist, I'm not suggesting to change the semantics for arguments while it's still around

We also need to look at the interaction with default argument values:

function f(...argsf) {   g(...argsf);   g(argsf[0],argsf[1]); } function g(a='a', b='b') {console.log(a+" "+b)}

as currently specified:   f(...[,1]); outputs "undefined 1", "undefined 1".  With hole preservation it would output "a 1",  "undefined 1".

It seems undesirable that two such calls to g would yield different results.  Also as currently specified, if a format parameter is assigned its default value then it is guaranteed that all formals to its right will also be assigned their default value.  This seems like a useful invariant which, as shown above, is lost if holes are preserved in this manner.

Holes would not need to trigger default values. I was only suggesting holes be preserved in rest parameters to match array {de}structuring behavior. The more important thing to me is that the behavior matches array destructuring which as currently specified has the exact same default value behavior you described above:

function f(arg) { [a='a', b='b'] = arg; console.log(a+" "+b)} let sparse = [,1]; f(sparse); // "a 1" f([sparse[0],sparse[1]]); // "undefined 1"

# Sean Eagan (14 years ago)

On Mon, Oct 3, 2011 at 11:35 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

On Oct 3, 2011, at 8:45 AM, Sean Eagan wrote:

On Mon, Oct 3, 2011 at 10:30 AM, Erik Arvidsson <erik.arvidsson at gmail.com> wrote:

But Function.prototype.apply doesn't preserve holes.

Function.prototype.apply doesn't preserve holes for the the arguments object, but rest parameters aren't yet standardized, and are intended to replace the arguments object, so they could, and in my view, should preserve holes for both Function.prototype.apply and foo(...args) activations, at least if array structuring and destructuring preserve holes as currently specified in the ES.next draft spec.

In general, a call site doesn't have any knowledge of format of the formal parameter list of the callee and the callee doesn't have any knowledge of how the caller constructed the argument list. So the caller and callee argument semantics can't be interdependent.

I'm not suggesting the semantics of the caller and callee be interdependent, just that the semantics of a function are dependent on whether it has yet upgraded from arguments to rest parameters.

Also, we probably need to factor in the fact that most current implementations probably don't have a mechanism for passing holes in an argument and that adding such a mechanism might be a deep and expensive change.  Would the cost to implementors really be worth the benefit to JS developers?

I have no implementation experience, but I don't see what cost you are referring to. It seems likely to me that there would be a performance benefit since implementations don't need to add undefined values to the rest parameter, and since user code can skip holes if desired such as via ES5 Array extras.

Thanks, Sean Eagan

# Allen Wirfs-Brock (14 years ago)

On Oct 3, 2011, at 12:42 PM, Sean Eagan wrote:

On Mon, Oct 3, 2011 at 11:31 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

On Oct 3, 2011, at 6:49 AM, Sean Eagan wrote:

...

In the evolution of the ES language, argument lists predates the existence of destructuring. In the legacy language neither the syntactic or applicative (apply function) support holes.

apply supports holes...

f.apply(Array(5));

Look at the specification of apply in section 15.3.4.3 of the ES5/5.1 spec. It does a [[Get]] on each element of the argArray to get the value that passed in the argument list. [[Get]] converts holes to undefined.

syntactic calls will support holes...

f(...Array(5))

by syntactic calls I meant f(a,b,c). the legacy language does not allow for f(a,,c)

What doesn't support holes is the arguments object which is on its way out.

In addition, the specification of the semantics of function invocation made no allowance for the possibility of holes in an argument list.

The specification of arguments dictates this, not function invocation as a whole.

Sorry that is incorrect, see 15.3.4.3

So your assertion, to be considered true, requires several changes in the existing language semantics (and arguably the syntax of argument lists.

My assertion is that argument/parameter lists and array {de}structuring should have as much as possible the same syntax and semantics, considering that currently as far as i know:

  • the only syntactic difference is that array {de}structuring supports Elision, e.g. [,,,]
  • the only semantic difference is that array {de}structuring supports hole preservation
  • i.e. the only difference is that array {de}structuring supports holes

What about holes makes them applicable to arrays in general but not argument arrays ? Everything else in ES.next points to reification of arguments as plain old arrays, why be different here ?

Because we aren't talking about reification of arguments. We are talking about the semantics of an argument list in a function call expression and the semantics of the spread operator.

I would actually be ok with or without both Elision syntax and hole preservation in ES.next, I just want consistency.

One concern that I've already mentioned is that recognizing holes in spread would create an inconsistency with the existing semantics of apply and that changing apply would be a breaking change that might impact existing code.

apply does not have existing semantics for rest parameters since they don't yet exist, I'm not suggesting to change the semantics for arguments while it's still around

Apply is on the caller side. It doesn't have any parameter semantics at all. It is only responsible for passing on a valid argument list.

Because apply and normal function application can't depend knowledge to the callee, it is reasonable to expect that: foo.apply(undefined,x) means exactly the same thing as foo(...x)

We also need to look at the interaction with default argument values:

function f(...argsf) { g(...argsf); g(argsf[0],argsf[1]); } function g(a='a', b='b') {console.log(a+" "+b)}

as currently specified: f(...[,1]); outputs "undefined 1", "undefined 1". With hole preservation it would output "a 1", "undefined 1".

It seems undesirable that two such calls to g would yield different results. Also as currently specified, if a format parameter is assigned its default value then it is guaranteed that all formals to its right will also be assigned their default value. This seems like a useful invariant which, as shown above, is lost if holes are preserved in this manner.

Holes would not need to trigger default values.

then what would be the value of x in function(x=1) {}(...[]) ??

I was only suggesting holes be preserved in rest parameters to match array {de}structuring behavior. The more important thing to me is that the behavior matches array destructuring which as currently specified has the exact same default value behavior you described above:

function f(arg) { [a='a', b='b'] = arg;

The above line is not legal syntax according to the current specification draft. The destructuring binding pattern and a destructuring assignment pattern have both different syntax and semantics.

In a destructuring binding, a scalar BindingElement is defined as: SingleNameBinding : BindingIdentifier Initializer-opt while in a destructuring assignment, a scalar AssignmentElement is defined as AssignmentElement : LeftHandSideExpression

Note that an AssignmentElement does not have an Initializer. The reason I defined it this way is that unlike a BindingIdentifier, a LeftHandSideExpression can be an arbitrary complex expression. I didn't want to allow expressions such as: [new foo.bar[baz(x=5)].bam=0] = obj; where the trailing initializer looks like it is just part of the element expression. I'm willing to discuss the merits of that decision, but the fact remains that the semantics of destructuring binding and destructuring assignment are quite different so I consider suspect any argument that starts by assuming that they are the same.

# Allen Wirfs-Brock (14 years ago)

On Oct 3, 2011, at 1:00 PM, Sean Eagan wrote:

On Mon, Oct 3, 2011 at 11:35 AM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

On Oct 3, 2011, at 8:45 AM, Sean Eagan wrote:

On Mon, Oct 3, 2011 at 10:30 AM, Erik Arvidsson <erik.arvidsson at gmail.com> wrote:

But Function.prototype.apply doesn't preserve holes.

Function.prototype.apply doesn't preserve holes for the the arguments object, but rest parameters aren't yet standardized, and are intended to replace the arguments object, so they could, and in my view, should preserve holes for both Function.prototype.apply and foo(...args) activations, at least if array structuring and destructuring preserve holes as currently specified in the ES.next draft spec.

In general, a call site doesn't have any knowledge of format of the formal parameter list of the callee and the callee doesn't have any knowledge of how the caller constructed the argument list. So the caller and callee argument semantics can't be interdependent.

I'm not suggesting the semantics of the caller and callee be interdependent, just that the semantics of a function are dependent on whether it has yet upgraded from arguments to rest parameters.

as it currently stands, a function is allowed to use both:

function f(a,b,c,...rest) { g(...arguments); h(...rest); }

Also, we probably need to factor in the fact that most current implementations probably don't have a mechanism for passing holes in an argument and that adding such a mechanism might be a deep and expensive change. Would the cost to implementors really be worth the benefit to JS developers?

I have no implementation experience, but I don't see what cost you are referring to. It seems likely to me that there would be a performance benefit since implementations don't need to add undefined values to the rest parameter, and since user code can skip holes if desired such as via ES5 Array extras.

In a high perf implementation, the arguments to a function call will typically be passed as a dense sequence of values on the processor stack. Not as an actual array object. Such a sequence normally does not include any way to encode the existence of "holes" so a new mechanism would have to be added by implementors. Probably a distinguished value. Such mechanisms typically carry a cost. For example, such a distinguished internal marker value must never be exposed to JS code as the value of formal parameter. So, the callee might have to scrub each argument position and replace each hole with undefined. It already has to do something like that for default value parameters (based on the size of the argument list) but now it would have to do it for every parameter.

# Sean Eagan (14 years ago)

On Mon, Oct 3, 2011 at 5:11 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

as it currently stands, a function is allowed to use both:

function f(a,b,c,...rest) {    g(...arguments);    h(...rest); }

Rest parameters are capable of anything arguments objects are and more, so what use case would there ever be for using both? Your example can be rewritten using only rest parameters as either:

function f(a,b,c,...rest) { g(...[a,b,c,...rest]);   h(...rest); }

or

function f(...arguments) { [,,,...rest] = arguments; g(...arguments);   h(...rest); }

Rest parameters are intended to deprecate arguments, if both are allowed simultaneously then this deprecation will be much harder to accomplish. I would even go so far as to disallow the use of arguments altogether in "extended code", to encourage the use of rest parameters and accelerate the deprecation of arguments.

In a high perf implementation, the arguments to a function call will typically be passed as a dense sequence of values on the processor stack.  Not as an actual array object. Such a sequence normally does not include any way to encode the existence of "holes" so a new mechanism would have to be added by implementors.  Probably a distinguished value.  Such mechanisms typically carry a cost.  For example, such a distinguished internal marker value must never be exposed to JS code as the value of formal parameter.  So, the callee might have to scrub each argument position and replace each hole with undefined.  It already has to do something like that for default value parameters (based on the size of the argument list) but now it would have to do it for every parameter.

Hopefully I'm not getting in way over my head here, but I did some digging in spider monkey code, and here's what I found:

Thanks, Sean Eagan

# Sean Eagan (14 years ago)

On Tue, Oct 4, 2011 at 4:09 PM, Sean Eagan <seaneagan1 at gmail.com> wrote:

On Mon, Oct 3, 2011 at 5:11 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

as it currently stands, a function is allowed to use both:

function f(a,b,c,...rest) {    g(...arguments);    h(...rest); }

Rest parameters are capable of anything arguments objects are and more, so what use case would there ever be for using both?  Your example can be rewritten using only rest parameters as either:

function f(a,b,c,...rest) {  g(...[a,b,c,...rest]);   h(...rest); }

or

function f(...arguments) {  [,,,...rest] = arguments;  g(...arguments);   h(...rest); }

Rest parameters are intended to deprecate arguments, if both are allowed simultaneously then this deprecation will be much harder to accomplish.  I would even go so far as to disallow the use of arguments altogether in "extended code", to encourage the use of rest parameters and accelerate the deprecation of arguments.

In a high perf implementation, the arguments to a function call will typically be passed as a dense sequence of values on the processor stack.  Not as an actual array object. Such a sequence normally does not include any way to encode the existence of "holes" so a new mechanism would have to be added by implementors.  Probably a distinguished value.  Such mechanisms typically carry a cost.  For example, such a distinguished internal marker value must never be exposed to JS code as the value of formal parameter.  So, the callee might have to scrub each argument position and replace each hole with undefined.  It already has to do something like that for default value parameters (based on the size of the argument list) but now it would have to do it for every parameter.

Hopefully I'm not getting in way over my head here, but I did some digging in spider monkey code, and here's what I found:

The sources got mangled, should be:

[1] dxr.mozilla.org/mozilla/mozilla-central/js/src/jsfun.cpp.html#l1866 [2] dxr.mozilla.org/mozilla/mozilla-central/js/src/jsfun.cpp.html#l1913 [3] dxr.mozilla.org/mozilla/mozilla-central/js/src/jsarray.cpp.html#l441 [4] dxr.mozilla.org/mozilla/mozilla-central/js/src/jsval.h.html#l259

Thanks, Sean Eagan

# Sean Eagan (14 years ago)

On Mon, Oct 3, 2011 at 4:37 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

apply supports holes...

f.apply(Array(5));

Look at the specification of apply in section 15.3.4.3 of the ES5/5.1 spec.  It does a [[Get]] on each element of the argArray to get the value that passed in the argument list.  [[Get]] converts holes to undefined.

The only way this is observable in ES5 is if argArray has getters for any of the indexes between 0 and argArray.length. I'm not aware of any cases of objects defining getters for indexed properties, much less those objects being passed to Function.prototype.apply. So the only consequence of this is that the hole preservation algorithms in the draft spec only call [[Get]] when [[HasProperty]] returns true. So we could either tweak the hole preservation algorithm for rest parameters to call [[Get]] regardless, or the better option IMHO, match the other algorithms by no longer calling [[Get]] for holes in argArray, at least when Function.prototype.apply is called from extended code.

syntactic calls will support holes...

f(...Array(5))

by syntactic calls I meant f(a,b,c).  the legacy language does not allow for f(a,,c)

This can be said of any any syntax being added in ES.next. Also, I'm only suggesting to add that syntax if it continues to be available in array literals.

What doesn't support holes is the arguments object which is on its way out.

In addition, the specification of the semantics of function invocation made no allowance for the possibility of holes in an argument list.

The specification of arguments dictates this, not function invocation as a whole.

Sorry that is incorrect, see 15.3.4.3

You are correct, sorry. However, this is merely an unobservable specification detail, so what relevance does it have to what is best for the future of the language ?

What about holes makes them applicable to arrays in general but not argument arrays ?  Everything else in ES.next points to reification of arguments as plain old arrays, why be different here ?

Because we aren't talking about reification of arguments.  We are talking about the semantics of an argument list in a function call expression and the semantics of the spread operator.

The spread operator in argument lists reifies the argument list as an array in my view. But mostly I just care that the semantics are the same as when the spread oeprator occurs in an array literal, i.e. that holes are preserved for the receiver. If a caller does not want holes to be observed by a potential callee rest argument, they can use a dense argument list, however I'm not sure why they would care about this. If a caller does want to allow a potential callee rest parameter to utilize holes for optimization purposes, why prevent them from doing so?

I would actually be ok with or without both Elision syntax and hole preservation in ES.next, I just want consistency.

One concern that I've already mentioned is that recognizing holes in spread would create an inconsistency with the existing semantics of apply and that changing apply would be a breaking change that might impact existing code.

apply does not have existing semantics for rest parameters since they don't yet exist, I'm not suggesting to change the semantics for arguments while it's still around

Apply is on the caller side.  It doesn't have any parameter semantics at all.  It is only responsible for passing on a valid argument list. Because apply and normal function application can't depend knowledge to the callee, it is reasonable to expect that:    foo.apply(undefined,x) means exactly the same thing as    foo(...x)

Yes, I intend for them to mean the same thing, which is that holes in x which fall within a range covered by a rest parameter in foo are preserved. If foo uses arguments they continue to show up as undefined values there.

function f(arg) {  [a='a', b='b'] = arg;

The above line is not legal syntax according to the current specification draft.  The destructuring binding pattern and a destructuring assignment pattern have both different syntax and semantics. In a destructuring binding, a scalar BindingElement is defined as:    SingleNameBinding : BindingIdentifier Initializer-opt while in a destructuring assignment, a scalar AssignmentElement is defined as    AssignmentElement : LeftHandSideExpression Note that an AssignmentElement does not have an Initializer.  The reason I defined it this way is that unlike a BindingIdentifier, a LeftHandSideExpression can be an arbitrary complex expression. I didn't want to allow expressions such as:    [new foo.bar[baz(x=5)].bam=0] = obj; where the trailing initializer looks like it is just part of the element expression.  I'm willing to discuss the merits of that decision, but the fact remains that the semantics of destructuring binding and destructuring assignment are quite different so I consider suspect any argument that starts by assuming that they are the same.

Sorry, should have read the draft spec more thoroughly, I agree that assignment destructuring should have syntactic and semantic differences from binding destructuring.

However, I don't see why variable declaration array destructuring binding and parameter lists should be different in any way. The only current syntactic difference between them is elision:

// allowed function f([,,,,,,x]){} // disallowed function f(,,,,,,x){}

The same syntactic inconsistency exists between array structuring and argument lists:

// allowed f([,,,,,,x]); // disallowed f(,,,,,,x);

The only semantic inconsistencies are:

  • hole preservation (if function rest parameters do not, but array destructuring rest parameters do, as you suggest)
  • default value triggering:

// default not used (function(x = 0){})(...[,1]) // default used var [x = 0] = [,1];

To me, eliminating these inconsistencies is a clear win, since it eliminates separate syntax and semantics for FormalParameterList and ArgumentList in the spec, implementations, and user's minds. I would prefer to have elision, hole preservation, and defaulting based on input array length rather than holes, but only if they were consistently applied.

Thanks, Sean Eagan

# Sean Eagan (14 years ago)

Removing these inconsistencies would also allow argument and parameter lists to be explained in terms of desugaring into array structuring and destructuring patterns respectively: function(a, , c = 0, ...d){  //...} would desugar to: function(){  var [a, , c = 0, ...d] = arguments; // even better would be "let"!  //...} and: f(a, , c, ...d) would desugar to: Function.prototype.apply.call(f, undefined, [a, , c, ...d]);

On Wed, Oct 5, 2011 at 2:19 PM, Sean Eagan <seaneagan1 at gmail.com> wrote:

On Mon, Oct 3, 2011 at 4:37 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:

apply supports holes...

f.apply(Array(5));

Look at the specification of apply in section 15.3.4.3 of the ES5/5.1 spec.  It does a [[Get]] on each element of the argArray to get the value that passed in the argument list.  [[Get]] converts holes to undefined.

The only way this is observable in ES5 is if argArray has getters for any of the indexes between 0 and argArray.length.  I'm not aware of any cases of objects defining getters for indexed properties, much less those objects being passed to Function.prototype.apply.  So the only consequence of this is that the hole preservation algorithms in the draft spec only call [[Get]] when [[HasProperty]] returns true. So we could either tweak the hole preservation algorithm for rest parameters to call [[Get]] regardless, or the better option IMHO, match the other algorithms by no longer calling [[Get]] for holes in argArray, at least when Function.prototype.apply is called from extended code.

syntactic calls will support holes...

f(...Array(5))

by syntactic calls I meant f(a,b,c).  the legacy language does not allow for f(a,,c)

This can be said of any any syntax being added in ES.next.  Also, I'm only suggesting to add that syntax if it continues to be available in array literals.

What doesn't support holes is the arguments object which is on its way out.

In addition, the specification of the semantics of function invocation made no allowance for the possibility of holes in an argument list.

The specification of arguments dictates this, not function invocation as a whole.

Sorry that is incorrect, see 15.3.4.3

You are correct, sorry.  However, this is merely an unobservable specification detail, so what relevance does it have to what is best for the future of the language ?

What about holes makes them applicable to arrays in general but not argument arrays ?  Everything else in ES.next points to reification of arguments as plain old arrays, why be different here ?

Because we aren't talking about reification of arguments.  We are talking about the semantics of an argument list in a function call expression and the semantics of the spread operator.

The spread operator in argument lists reifies the argument list as an array in my view.  But mostly I just care that the semantics are the same as when the spread oeprator occurs in an array literal, i.e. that holes are preserved for the receiver.  If a caller does not want holes to be observed by a potential callee rest argument, they can use a dense argument list, however I'm not sure why they would care about this.  If a caller does want to allow a potential callee rest parameter to utilize holes for optimization purposes, why prevent them from doing so?

I would actually be ok with or without both Elision syntax and hole preservation in ES.next, I just want consistency.

One concern that I've already mentioned is that recognizing holes in spread would create an inconsistency with the existing semantics of apply and that changing apply would be a breaking change that might impact existing code.

apply does not have existing semantics for rest parameters since they don't yet exist, I'm not suggesting to change the semantics for arguments while it's still around

Apply is on the caller side.  It doesn't have any parameter semantics at all.  It is only responsible for passing on a valid argument list. Because apply and normal function application can't depend knowledge to the callee, it is reasonable to expect that:    foo.apply(undefined,x) means exactly the same thing as    foo(...x)

Yes, I intend for them to mean the same thing, which is that holes in x which fall within a range covered by a rest parameter in foo are preserved.  If foo uses arguments they continue to show up as undefined values there.

function f(arg) {  [a='a', b='b'] = arg;

The above line is not legal syntax according to the current specification draft.  The destructuring binding pattern and a destructuring assignment pattern have both different syntax and semantics. In a destructuring binding, a scalar BindingElement is defined as:    SingleNameBinding : BindingIdentifier Initializer-opt while in a destructuring assignment, a scalar AssignmentElement is defined as    AssignmentElement : LeftHandSideExpression Note that an AssignmentElement does not have an Initializer.  The reason I defined it this way is that unlike a BindingIdentifier, a LeftHandSideExpression can be an arbitrary complex expression. I didn't want to allow expressions such as:    [new foo.bar[baz(x=5)].bam=0] = obj; where the trailing initializer looks like it is just part of the element expression.  I'm willing to discuss the merits of that decision, but the fact remains that the semantics of destructuring binding and destructuring assignment are quite different so I consider suspect any argument that starts by assuming that they are the same.

Sorry, should have read the draft spec more thoroughly, I agree that assignment destructuring should have syntactic and semantic differences from binding destructuring.

However, I don't see why variable declaration array destructuring binding and parameter lists should be different in any way.  The only current syntactic difference between them is elision:

// allowed function f([,,,,,,x]){} // disallowed function f(,,,,,,x){}

The same syntactic inconsistency exists between array structuring and argument lists:

// allowed f([,,,,,,x]); // disallowed f(,,,,,,x);

The only semantic inconsistencies are:

  • hole preservation (if function rest parameters do not, but array destructuring rest parameters do, as you suggest)
  • default value triggering:

// default not used (function(x = 0){})(...[,1]) // default used var [x = 0] = [,1];

To me, eliminating these inconsistencies is a clear win, since it eliminates separate syntax and semantics for FormalParameterList and ArgumentList in the spec, implementations, and user's minds.  I would prefer to have elision, hole preservation, and defaulting based on input array length rather than holes, but only if they were consistently applied.

Thanks, Sean Eagan

Thanks, Sean Eagan

# Andreas Rossberg (14 years ago)

On 5 October 2011 21:19, Sean Eagan <seaneagan1 at gmail.com> wrote:

However, I don't see why variable declaration array destructuring binding and parameter lists should be different in any way. The only current syntactic difference between them is elision:

// allowed function f([,,,,,,x]){} // disallowed function f(,,,,,,x){}

Only apropos of semantics, but I really don't like this syntax at all. It is far, far too easy to overlook a hole. I think we should forbid this syntax in Harmony.

If we want to support "holes" in patterns -- and I'm all for it! -- then we should do what all other languages with proper pattern matching do and introduce explicit syntax for wildcards, namely "_". That simplifies both syntax and semantics (because it's more compositional) and increases readability:

function f([_, _, _, x]){}

function f(_, _, _, x){}

This has been suggested before, but I want to reinforce the point.

(I'm far less convinced about allowing holes in expressions, but an argument could be made that "_" is simply syntax for undefined in expressions. No more writing "void 0".)

# David Herman (14 years ago)

I don't think we can get away with repurposing _ as a pattern sigil, since it's already a valid identifier and used by popular libraries:

http://documentcloud.github.com/underscore/

In my strawman for pattern matching, I used * as the "don't-care" pattern:

http://wiki.ecmascript.org/doku.php?id=strawman:pattern_matching
# Andreas Rossberg (14 years ago)

On 7 October 2011 17:47, David Herman <dherman at mozilla.com> wrote:

I don't think we can get away with repurposing _ as a pattern sigil, since it's already a valid identifier and used by popular libraries:

http://documentcloud.github.com/underscore/

In my strawman for pattern matching, I used * as the "don't-care" pattern:

http://wiki.ecmascript.org/doku.php?id=strawman:pattern_matching

I reckoned that _ would be infeasible. But * is fine too, although it might cause more headache in a fused pattern/expression grammar.

# Brendan Eich (14 years ago)

I think you make a good case for hole preservation. However, holes in actual and formal parameter lists are not that wanted. You want them for consistency with holes in array pattern destructuring parameters. That's unusual, but I grant that if they're useful for destructuring, they could be useful for parameter lists too. In some languages, you would use a dummy name such as _.

The thing that gives me pause about extending the grammar for ArgumentList and FormalParameterList is the reports from Dean Landolt about "comma first" style users vs. "comma last" creating unintended holes in array literals. This is horrifying. It makes me abominate comma-first, but it also makes me want to be cautious about spreading holey syntax to parameter lists.

There are many kinds of consistencies to advance, along different dimensions. Some trade against others, directly or at an angle. Is this really worth the potential trouble?