Discuss
  • Operator Overloading Strawman for Harmony

Operator Overloading Strawman for Harmony

# Mark S. Miller (16 years ago)

First, let me acknowledge that there's been a long history of attempts to add operator overloading to EcmaScript, that these have all died for various reasons leaving much documentation in its wake, and that I have not read this documentation at the present time. If I am simply rehashing ideas which are already well refuted, my apologies for wasting everyone's time. But I did verbally discuss these ideas with Waldemar, who has lived through this history, and he thought this was plausible enough to be worth proposing.

Why raise this difficult issue again? We some may remember from the decimal wars, my motivation in this is to enable library designers to create new numeric types -- like rational, complex, quaternions, vector, matrix, surreal numbers, whatever -- so that not every numeric type need come from the EcmaScript committee. If we do our job as language designers -- of providing adequate abstraction mechanisms -- then library authors can use these to build a diversity of abstractions. For efficiency reasons, some scalar numeric types, like integers and decimal, may still be provided primitively. But once we have a framework such that these could be provided as a library, then adopting them as directly supported primitives becomes merely an optimization. (Indeed, efficiency considerations in the presence of multicore may demand that vectors and matricies be primitives, but let's not worry about that yet.)

At the request of the rest of the committee, one of Sam's early decimal attempts did exactly this. It defined a decimal API equivalent to one that could be provided as a library on ES3.1. ES3.1 has no operator overloading whatsoever, so we thought we could do this and be downwards compatible from a future language 9ES4 at the time) which would allow the operators to be used directly on decimal values. These plans ran aground on "+", which clarifies what problems need to be solved.

              Strawman proposal

Define a new nominal type named, let's say, "Operable". (I'm not stuck on this name. Perhaps "Numeric"?) We don't yet know what nominal type system Harmony will have, so for concreteness, and to separate issues, let's use "instanceof" as our nominal type test, where only functions whose "prototype" property is frozen can be treated as types. (Otherwise, instanceof is not monotonic.) Let's say that Operable is ES3.1-like function acting as an purely abstract class (or in Java, a marker interface). Operable throws if called (hey, it's abstract), it's prototype property is frozen (non-writable, non-enumerable, non-configurable), and points at an empty object that inherits from Object.prototype. We redefine the original Number.prototype so that it inherits from Operable.prototype, so all numbers are Operable. Crucially, strings remain non-Operable. (I'm dangerously ignoring here the distinction between primitives and wrappers. In fact, in ES3 and ES3.1, "e instanceof Number" is false. Fixing the strawman accordingly presents no problems but makes it tedious. I will continue ignoring this for now.)

We are concerned only about the "normal" operators, which exclude at least &&, ||, ?:, ++, --, ., or ===. For each of the normal operators, we insert the following tests at the beginning of their definition after all operands are evaluated and GetValue()d but before they are ToPrimitive()d. For example, for infix "+" at 11.6.1 The Addition Operator in the ES3.1 spec, after step 4, taking a few self-hosting notational shorthands:

Let Left = Result(2)
Let Right = Result(4)
If isOperable(Left) and isOperable(Right) then
    harmony code: return Left['+'](Right)
// else fall through to the current behavior.

To make this work, for each of the operators that operate on numbers, we'd add a corresponding built-in method to the original Number.prototype. In self-hosting style:

Number.prototype['+'] = function(arg) { if (isNumber(arg)) { return primAdd(this, arg); } else { return arg'reverse+'; } };

Number.prototype['reverse+'] = function(receiver) { if (isNumber(receiver)) { return primAdd(receiver, this); } else { throw ...; } };

This trick is adapted from Smalltalk, which did something similar. Newer numeric abstraction understand the build in ones and some number of widely adopted older ones. But the older ones don't understand the new ones. If X is a primitive Number and Y and Z are user-defined Complex numbers, then

Y + Z // fine, since complex Y knows what to do with complex Z Y + X // fine, since complex Y knows what to do with number X X + Y // number X doesn't know what to do with Y, so it asks Y to handle it by // Y'reverse+', which Y can successfully handle.

What if we compose together two new numeric types, say Rational and Complex, where each was defined in ignorance of the other. Say R is a Rational.

Y + R // Y is clueless, so it asks R: R'reverse+' // R is clueless, so it gives up rather than re-reversing

This distinction between "+" and "reverse+" also handles the non-commutativity of some operators.

An elaboration suggested by Mike Samuel:

Rather than have Operable.prototype be empty, since one expects various operators to have algebratic relationships to each other, we can install default methods defining some operators in terms of a smaller set than new numerics will still need to override. For example

Operable.prototype['+'] = function(arg) { return this - -arg; };

A related issue: Should various internal methods have similar tests for Operable, and invoke the operable to handle this. Waldemar raised coercion to boolean. Perhaps ToBoolean(foo), if foo is an Operable, should invoke foo['!!'] or something. We'd then install default methods for these in Operable.prototype.

Mike also raised the issue of whether we should split Operable (+,-,*,/,...) vs Comparable (<,<=,==,>=,>). I find that plausible, given that we have a

nominal type system that can handle non-tree subtype relationships, i.e., something other than instanceof.

So, is there some reason why this won't work?

# Mark Miller (16 years ago)

2009/1/8 Mark S. Miller <erights at google.com>:

(I'm dangerously ignoring here the distinction between primitives and wrappers. In fact, in ES3 and ES3.1, "e instanceof Number" is false. Fixing the strawman accordingly presents no problems but makes it tedious. I will continue ignoring this for now.)

I meant "3 instanceof Number".

# Michael Daumling (16 years ago)

This is very close to the pattern that Adobe's ExtendScript already offers, and that has been rejected before, unfortunately. The differences are:

  1.  There is no "reverse+" (or other construct). Instead, ExtendScript has a second argument "reverse" that is a Boolean, and is true if the operator is reverse.

  2.  For unary operators, the argument is undefined.

  3.  If the function returns undefined, it invokes the default behavior of the operator.

Example (very brief), assuming that toString() returns "5": ... MyClass.prototype['+'] = function(arg, reverse) { return reverse ? arg + " plus " + this : this + " plus " + arg; }

var obj = new MyClass (5); +obj; // prints "5 plus undefined" obj + 10; // prints "5 plus 10" 10 + obj; // prints "10 plus 5"

Michael

From: es-discuss-bounces at mozilla.org [mailto:es-discuss-bounces at mozilla.org] On Behalf Of Mark S. Miller Sent: Thursday, January 08, 2009 9:13 PM To: es-discuss Subject: Operator Overloading Strawman for Harmony

First, let me acknowledge that there's been a long history of attempts to add operator overloading to EcmaScript, that these have all died for various reasons leaving much documentation in its wake, and that I have not read this documentation at the present time. If I am simply rehashing ideas which are already well refuted, my apologies for wasting everyone's time. But I did verbally discuss these ideas with Waldemar, who has lived through this history, and he thought this was plausible enough to be worth proposing.

Why raise this difficult issue again? We some may remember from the decimal wars, my motivation in this is to enable library designers to create new numeric types -- like rational, complex, quaternions, vector, matrix, surreal numbers, whatever -- so that not every numeric type need come from the EcmaScript committee. If we do our job as language designers -- of providing adequate abstraction mechanisms -- then library authors can use these to build a diversity of abstractions. For efficiency reasons, some scalar numeric types, like integers and decimal, may still be provided primitively. But once we have a framework such that these could be provided as a library, then adopting them as directly supported primitives becomes merely an optimization. (Indeed, efficiency considerations in the presence of multicore may demand that vectors and matricies be primitives, but let's not worry about that yet.)

At the request of the rest of the committee, one of Sam's early decimal attempts did exactly this. It defined a decimal API equivalent to one that could be provided as a library on ES3.1. ES3.1 has no operator overloading whatsoever, so we thought we could do this and be downwards compatible from a future language 9ES4 at the time) which would allow the operators to be used directly on decimal values. These plans ran aground on "+", which clarifies what problems need to be solved.

              Strawman proposal

Define a new nominal type named, let's say, "Operable". (I'm not stuck on this name. Perhaps "Numeric"?) We don't yet know what nominal type system Harmony will have, so for concreteness, and to separate issues, let's use "instanceof" as our nominal type test, where only functions whose "prototype" property is frozen can be treated as types. (Otherwise, instanceof is not monotonic.) Let's say that Operable is ES3.1-like function acting as an purely abstract class (or in Java, a marker interface). Operable throws if called (hey, it's abstract), it's prototype property is frozen (non-writable, non-enumerable, non-configurable), and points at an empty object that inherits from Object.prototype. We redefine the original Number.prototype so that it inherits from Operable.prototype, so all numbers are Operable. Crucially, strings remain non-Operable. (I'm dangerously ignoring here the distinction between primitives and wrappers. In fact, in ES3 and ES3.1, "e instanceof Number" is false. Fixing the strawman accordingly presents no problems but makes it tedious. I will continue ignoring this for now.)

We are concerned only about the "normal" operators, which exclude at least &&, ||, ?:, ++, --, ., or ===. For each of the normal operators, we insert the following tests at the beginning of their definition after all operands are evaluated and GetValue()d but before they are ToPrimitive()d. For example, for infix "+" at 11.6.1 The Addition Operator in the ES3.1 spec, after step 4, taking a few self-hosting notational shorthands:

Let Left = Result(2)
Let Right = Result(4)
If isOperable(Left) and isOperable(Right) then
    harmony code: return Left['+'](Right)
// else fall through to the current behavior.

To make this work, for each of the operators that operate on numbers, we'd add a corresponding built-in method to the original Number.prototype. In self-hosting style:

Number.prototype['+'] = function(arg) { if (isNumber(arg)) { return primAdd(this, arg); } else { return arg'reverse+'; } };

Number.prototype['reverse+'] = function(receiver) { if (isNumber(receiver)) { return primAdd(receiver, this); } else { throw ...; } };

This trick is adapted from Smalltalk, which did something similar. Newer numeric abstraction understand the build in ones and some number of widely adopted older ones. But the older ones don't understand the new ones. If X is a primitive Number and Y and Z are user-defined Complex numbers, then

Y + Z // fine, since complex Y knows what to do with complex Z Y + X // fine, since complex Y knows what to do with number X X + Y // number X doesn't know what to do with Y, so it asks Y to handle it by // Y'reverse+', which Y can successfully handle.

What if we compose together two new numeric types, say Rational and Complex, where each was defined in ignorance of the other. Say R is a Rational.

Y + R // Y is clueless, so it asks R: R'reverse+' // R is clueless, so it gives up rather than re-reversing

This distinction between "+" and "reverse+" also handles the non-commutativity of some operators.

An elaboration suggested by Mike Samuel:

Rather than have Operable.prototype be empty, since one expects various operators to have algebratic relationships to each other, we can install default methods defining some operators in terms of a smaller set than new numerics will still need to override. For example

Operable.prototype['+'] = function(arg) { return this - -arg; };

A related issue: Should various internal methods have similar tests for Operable, and invoke the operable to handle this. Waldemar raised coercion to boolean. Perhaps ToBoolean(foo), if foo is an Operable, should invoke foo['!!'] or something. We'd then install default methods for these in Operable.prototype.

Mike also raised the issue of whether we should split Operable (+,-,*,/,...) vs Comparable (<,<=,==,>=,>). I find that plausible, given that we have a nominal type system that can handle non-tree subtype relationships, i.e., something other than instanceof.

So, is there some reason why this won't work?

# Michael Daumling (16 years ago)

OK, the example would most likely create a bad recursion; it should better be:

MyClass.prototype['+'] = function(arg, reverse) { return reverse ? String(arg) + " plus " + this.toString() : this.toString() + " plus " + String(arg); }

I hope, though, that the meaning is clear enough :)

Michael

From: es-discuss-bounces at mozilla.org [mailto:es-discuss-bounces at mozilla.org] On Behalf Of Michael Daumling Sent: Friday, January 09, 2009 2:35 AM To: Mark S. Miller; es-discuss Subject: RE: Operator Overloading Strawman for Harmony

Hi Mark,

This is very close to the pattern that Adobe's ExtendScript already offers, and that has been rejected before, unfortunately. The differences are:

  1.  There is no "reverse+" (or other construct). Instead, ExtendScript has a second argument "reverse" that is a Boolean, and is true if the operator is reverse.

  2.  For unary operators, the argument is undefined.

  3.  If the function returns undefined, it invokes the default behavior of the operator.

Example (very brief), assuming that toString() returns "5": ... MyClass.prototype['+'] = function(arg, reverse) { return reverse ? arg + " plus " + this : this + " plus " + arg; }

var obj = new MyClass (5); +obj; // prints "5 plus undefined" obj + 10; // prints "5 plus 10" 10 + obj; // prints "10 plus 5"

Michael

From: es-discuss-bounces at mozilla.org [mailto:es-discuss-bounces at mozilla.org] On Behalf Of Mark S. Miller Sent: Thursday, January 08, 2009 9:13 PM To: es-discuss Subject: Operator Overloading Strawman for Harmony

First, let me acknowledge that there's been a long history of attempts to add operator overloading to EcmaScript, that these have all died for various reasons leaving much documentation in its wake, and that I have not read this documentation at the present time. If I am simply rehashing ideas which are already well refuted, my apologies for wasting everyone's time. But I did verbally discuss these ideas with Waldemar, who has lived through this history, and he thought this was plausible enough to be worth proposing.

Why raise this difficult issue again? We some may remember from the decimal wars, my motivation in this is to enable library designers to create new numeric types -- like rational, complex, quaternions, vector, matrix, surreal numbers, whatever -- so that not every numeric type need come from the EcmaScript committee. If we do our job as language designers -- of providing adequate abstraction mechanisms -- then library authors can use these to build a diversity of abstractions. For efficiency reasons, some scalar numeric types, like integers and decimal, may still be provided primitively. But once we have a framework such that these could be provided as a library, then adopting them as directly supported primitives becomes merely an optimization. (Indeed, efficiency considerations in the presence of multicore may demand that vectors and matricies be primitives, but let's not worry about that yet.)

At the request of the rest of the committee, one of Sam's early decimal attempts did exactly this. It defined a decimal API equivalent to one that could be provided as a library on ES3.1. ES3.1 has no operator overloading whatsoever, so we thought we could do this and be downwards compatible from a future language 9ES4 at the time) which would allow the operators to be used directly on decimal values. These plans ran aground on "+", which clarifies what problems need to be solved.

              Strawman proposal

Define a new nominal type named, let's say, "Operable". (I'm not stuck on this name. Perhaps "Numeric"?) We don't yet know what nominal type system Harmony will have, so for concreteness, and to separate issues, let's use "instanceof" as our nominal type test, where only functions whose "prototype" property is frozen can be treated as types. (Otherwise, instanceof is not monotonic.) Let's say that Operable is ES3.1-like function acting as an purely abstract class (or in Java, a marker interface). Operable throws if called (hey, it's abstract), it's prototype property is frozen (non-writable, non-enumerable, non-configurable), and points at an empty object that inherits from Object.prototype. We redefine the original Number.prototype so that it inherits from Operable.prototype, so all numbers are Operable. Crucially, strings remain non-Operable. (I'm dangerously ignoring here the distinction between primitives and wrappers. In fact, in ES3 and ES3.1, "e instanceof Number" is false. Fixing the strawman accordingly presents no problems but makes it tedious. I will continue ignoring this for now.)

We are concerned only about the "normal" operators, which exclude at least &&, ||, ?:, ++, --, ., or ===. For each of the normal operators, we insert the following tests at the beginning of their definition after all operands are evaluated and GetValue()d but before they are ToPrimitive()d. For example, for infix "+" at 11.6.1 The Addition Operator in the ES3.1 spec, after step 4, taking a few self-hosting notational shorthands:

Let Left = Result(2)
Let Right = Result(4)
If isOperable(Left) and isOperable(Right) then
    harmony code: return Left['+'](Right)
// else fall through to the current behavior.

To make this work, for each of the operators that operate on numbers, we'd add a corresponding built-in method to the original Number.prototype. In self-hosting style:

Number.prototype['+'] = function(arg) { if (isNumber(arg)) { return primAdd(this, arg); } else { return arg'reverse+'; } };

Number.prototype['reverse+'] = function(receiver) { if (isNumber(receiver)) { return primAdd(receiver, this); } else { throw ...; } };

This trick is adapted from Smalltalk, which did something similar. Newer numeric abstraction understand the build in ones and some number of widely adopted older ones. But the older ones don't understand the new ones. If X is a primitive Number and Y and Z are user-defined Complex numbers, then

Y + Z // fine, since complex Y knows what to do with complex Z Y + X // fine, since complex Y knows what to do with number X X + Y // number X doesn't know what to do with Y, so it asks Y to handle it by // Y'reverse+', which Y can successfully handle.

What if we compose together two new numeric types, say Rational and Complex, where each was defined in ignorance of the other. Say R is a Rational.

Y + R // Y is clueless, so it asks R: R'reverse+' // R is clueless, so it gives up rather than re-reversing

This distinction between "+" and "reverse+" also handles the non-commutativity of some operators.

An elaboration suggested by Mike Samuel:

Rather than have Operable.prototype be empty, since one expects various operators to have algebratic relationships to each other, we can install default methods defining some operators in terms of a smaller set than new numerics will still need to override. For example

Operable.prototype['+'] = function(arg) { return this - -arg; };

A related issue: Should various internal methods have similar tests for Operable, and invoke the operable to handle this. Waldemar raised coercion to boolean. Perhaps ToBoolean(foo), if foo is an Operable, should invoke foo['!!'] or something. We'd then install default methods for these in Operable.prototype.

Mike also raised the issue of whether we should split Operable (+,-,*,/,...) vs Comparable (<,<=,==,>=,>). I find that plausible, given that we have a nominal type system that can handle non-tree subtype relationships, i.e., something other than instanceof.

So, is there some reason why this won't work?

# Mark S. Miller (16 years ago)

On Fri, Jan 9, 2009 at 2:35 AM, Michael Daumling <mdaeumli at adobe.com> wrote:

Hi Mark,

This is very close to the pattern that Adobe's ExtendScript already offers, and that has been rejected before, unfortunately. The differences are:

Hi Michael,

Yes, this is indeed quite similar. Why was it rejected? Do the reasons for rejection apply equally well to the present proposal?

# Michael Daumling (16 years ago)

The discussion about operator overloading quickly went away from the JavaScript'ish approach that ExtendScript and your proposal used towards generic functions. At some time, the discussion stranded in areas too exotic for me. There is a rationale here: discussion:operators

Michael

From: Mark S. Miller [mailto:erights at google.com] Sent: Friday, January 09, 2009 7:48 AM To: Michael Daumling Cc: es-discuss Subject: Re: Operator Overloading Strawman for Harmony

On Fri, Jan 9, 2009 at 2:35 AM, Michael Daumling <mdaeumli at adobe.com<mailto:mdaeumli at adobe.com>> wrote:

Hi Mark,

This is very close to the pattern that Adobe's ExtendScript already offers, and that has been rejected before, unfortunately. The differences are:

Hi Michael,

Yes, this is indeed quite similar. Why was it rejected? Do the reasons for rejection apply equally well to the present proposal?