Adobe position paper on the ECMAScript 4 proposal space -- decimal
taking your questions in order:
Regarding "Premature" / "Knowledgable users"
When we say decimal is "premature" we mean it in the sense outlined in the paper only: The TG had an intuition about what we wanted for the language (The Right Thing - essentially a type of big red switch) but then we came to the conclusion that what we wanted is simply not workable on the web (see below), and therefore we now no longer know exactly what it is that we want. Therefore decimal is premature in the sense we use here: more experience may be necessary before the TG members know what we want. As authors of the position paper, Jeff and I did not and do not mean to imply that decimal arithmetic, considered in isolation, is immature, far from it.
About the big red switch. There are two variants: One is global to an execution context, were a program sets the global floating point mode to be decimal and all arithmetic is carried out in decimal (and all literals are decimal); the other is lexically scoped, where a "use decimal" pragma affects operators and literal interpretation in its lexical scope.
(In either case the implementation will use a polymorphic number representation and perform most arithmetic on integers, of course, but on overflow the representation becomes decimal floating point rather than binary floating point if the switch has been turned on.)
To take the global case first:
Two of the themes that have been very prominent in the work of the TG have been that ES4 must support existing ES3 content exceptionally well, and that we must expect that most web sites that migrate to ES4 will not do so all at once, but will upgrade some of their code to ES4, leaving other code as unmodified ES3. For example, one could imagine a page that has exception handlers and some infrastructure code that are all ES3 code but which loads ES4 code dynamically if the browser supports ES4. Mashups are another example; in this case, the page author has no control over the code loaded into the page (and for a long time most of it will be ES3 code). As a third case, code on one page can call code on another page (or in a frame or iframe). In summary, the TG has considered it to be unacceptable to require a program to consist of all ES3 code or all ES4 code, mixing is going to be the reality.
As a consequence of those compatibility concerns, it is unacceptable for one piece of code to alter the floating point environment of the other code by flipping the big red switch to "decimal", because the consequences of doing so are unknown. Of course decimal has a larger exponent range and higher precision than double-precision floating point; but my experience with ES on the web is that people write code with all sorts of dependencies, and many TG members believe that there is important code out there that depends on the number representation being "double".
As a compromise it would be possible to make the big red switch apply only to ES4 code that runs in an all-ES4 environment, but my sense is that the browser vendors do not believe that is a particularly compelling use case (nor does Adobe).
The lexically scoped case has other problems. On the one hand, the programmer controls when "1" means "decimal(1)" and when "a+b" means "decimal(a)+decimal(b)". On the other hand, the question is really how far this control extends. What happens when code in the context of "use decimal" reads a number from a data structure defined outside that scope, or receives a number from a function defined outside that scope -- is the number converted to decimal? What if the code stores a decimal value in an external data structure or passes it to an external function that expects a double? The issues of compatibility with existing ES3 code reappear in this context.
Furthermore, the intent was also that the "use decimal" pragma should control the precision and rounding settings in a lexical fashion, but that precludes those settings being passed off implicitly to separately defined functions or library functions that (maybe!) should take them into account.
Overall, the lexically scoped case has usability issues, and TG members (myself as much as anyone) were not convinced that they can be overcome at this time. More experience is needed.
Maciej has already discussed performance concerns surrounding a general switch to decimal. Since a polymorphic number representation ensures that decimal is used only for floating-point numbers, any concerns should apply mainly to programs that perform a significant amount of floating-point arithmetic (programs that have a significant number of overflows from machine integer but which then chop values back to integer would be impacted too). I believe strongly that there are web applications out there that will see a significant slowdown, but I do not know how important the problem is in practice. Speaking for Adobe, current ActionScript code tends to use strong static typing and very specific number types, so most existing ActionScript code would not be impacted much. Once ActionScript aligns with ES4 the issue is more open.
None of the above speaks to the possibility that "decimal" might be a distinct data type in the language, of course, along with "double" and "int" and "uint". Such a data type would in our opinion not be a poor fit for ES4, and as your own writings demonstrate, there are good use cases for it. We think a few of our customers, and probably a few users on the web, would find the data type useful. In fact, Adobe has implemented prototypical support for decimal in the Tamarin virtual machine, we have polled some of our users about the need for a distinct decimal type, and we are evaluating whether to support the data type in our compilers. So when we now advocate cutting the type from the language it is in part because we also advocate cutting the int and uint types, which also seem premature (less well thought out than they probably ought to be). The simplification of going back to the ES3 numerics model without any of these three types is substantial in our opinion. If there was a huge demand for a separate decimal type then the situation might be different; as it is, the need is there but probably marginal, and it is in our estimation not worth the complexity cost to the language (also taking into consideration the next two issues).
"Hardship on small systems"
A 50KB piece of code (your decimal code + various levels of support in libraries and the on-line compiler) would increase the size of at least one commercial, compliant, high-performance ES engine I know of by more than 20%, and it would increase the size of another by more than that still. Either way 50KB is quite a bit of code on current mid-range mobile platforms. (Percentage-wise an increase will be less than 20% once those engines is upgraded to ES4, granted.)
This becomes a matter of opinion, but the TG has in fact accepted other restrictions in the proposed language in order to accomodate very small systems, such as using a constructor function prefix syntax (a la C++) to initialize nullable and constant instance properties instead of requiring definite assignment analysis (which would require building trees for the program in the compiler -- something that is not necessary for ES3, and which we hope to avoid requiring for ES4).
"Hardship to implement"
Speaking as somebody who used to work for an OEM browser vendor: Some OEM buyers are extremely reluctant to take software that includes open source, public domain, or other "free" software. This situation may change over time, obviously, but at one of my previous employers there was always pressure to write code from scratch rather than incorporate source code authored elsewhere, for this very reason. It just seems to be a fact of life. (By itself it would not be a reason to reject any sort of decimal type, of course. It's just a contributing fractor.)
On Feb 27, 2008, at 5:33 AM, Lars Hansen wrote:
None of the above speaks to the possibility that "decimal" might be
a distinct data type in the language, of course, along with
"double" and "int" and "uint". Such a data type would in our
opinion not be a poor fit for ES4, and as your own writings
demonstrate, there are good use cases for it. We think a few of
our customers, and probably a few users on the web, would find the
data type useful.
I agree with what Lars wrote, except here I think the value of a
hypothetical (possibly mythical) "big red switch" is understated.
That most dup'ed JS bug in buzilla.mozilla.org,
bugzilla.mozilla.org/show_bug.cgi?id=5856
still collects dups at a good clip. There was even a recent variation
complaining about mantissa binary precision limits, but that's not
the common problem. The common problem is that you can't do "dollars
and cents" or "pounds and pennies" arithmetic and get the "right
answer":
js> 74.96-39.96
34.99999999999999
This is not a problem for only a "few users on the web". I wish we
could fix it. But the big red switch can't be on the side of the
whole browser, or even the current tab or window (and all its
subsidiary frames or iframes or popups).
Mike, have you any experience with mixed mode (due to mixed code)
environments?
Was decimal ruled out as its own type?
Peter
On Feb 27, 2008, at 10:25 AM, Peter Hall wrote:
Was decimal ruled out as its own type?
First, nothing's "ruled out" -- you're asking the wrong guy if you
want Adobe's position, but see Lars's reply to Mike Cowlishaw:
decimal as a type without any implicit literal/operators mode is
still possible, although the Adobe position paper defers it. As Lars
noted, it even has a trial implementation in Tamarin.
Second, decimal is in the RI as proposed, more or less.
Finally, whatever we do, we won't make certain BigMistakes. I wrote a
long time ago in
citing the amusing Cameron Purdy blog post at
jroller.com/cpurdy/entry/the_seven_habits_of_highly1
The first comment:
At a client gig, they were doing business/financial coding, so were
using BigDecimal.Of course, .add() and friends is too difficult, so they ended up
with roughly:BigDecimal subA = ... BigDecimal subB = ...
BigDecimal total = new BigDecimal( subA.doubleValue() +
subB.doubleValue() );It was beautiful.
Posted by Bob McWhirter on October 31, 2005 at 08:17 AM EST #
is horrifying testimony to the need for operator and literal syntax,
if not implicit modal defaulting. If we keep decimal in ES4, it will
have operators and literal support.
On Feb 27, 2008, at 10:40 AM, Brendan Eich wrote:
First, nothing's "ruled out" -- you're asking the wrong guy if you want Adobe's position, but see Lars's reply to Mike Cowlishaw: decimal as a type without any implicit literal/operators mode is still possible,
I should have written "without generic operator methods" -- ES4 could
still have a decimal type and built-in operators and literal support,
but no modal defaulting (no "big red switch").
My first attempt to reply bounced from es4-discuss since I wasn't subscribed.
OK. Decimal type just makes sense to me. And I think this is one case where I think you can break "the rule" that says correct type annotations do not affect the program.
Peter
There's no such rule in ES4. There are implicit conversions among primitive number types, and between primitive types and wrapper types, and those kick in when storing something in an annotated location. So these two programs are different, and both are correct:
var s = "foo" var t = 10.5
and
var s : String = "foo" var t : int = 10.5
(Decimal makes sense without any type annotations, but annotations probably cause coercions to decimal to happen more often and thus reduce the scope for error. I assume this is what Dick meant. But 1m would be a decimal value 1, and 1m/10 would convert 10 to decimal before dividing -- no annotations involved.)
Lars is correct. If you can declare decimal literals, that is enough to get you into decimal arithmetic. The various automatic coercions will do the rest, though as he said, type annotations could reduce the scope of errors. You would also need some static methods on decimal like exp and log, since the ones on Number expect doubles.
Dick
Lars, thanks ...
Regarding "Premature" / "Knowledgable users"
When we say decimal is "premature" we mean it in the sense outlined in the paper only: The TG had an intuition about what we wanted for the language (The Right Thing - essentially a type of big red switch) but then we came to the conclusion that what we wanted is simply not workable on the web (see below), and therefore we now no longer know exactly what it is that we want. Therefore decimal is premature in the sense we use here: more experience may be necessary before the TG members know what we want. As authors of the position paper, Jeff and I did not and do not mean to imply that decimal arithmetic, considered in isolation, is immature, far from it.
OK, thanks for the explanation.
About the big red switch. There are two variants: One is global to an execution context, were a program sets the global floating point mode to be decimal and all arithmetic is carried out in decimal (and all literals are decimal); the other is lexically scoped, where a "use decimal" pragma affects operators and literal interpretation in its lexical scope.
[snip detailed walkthrough of the issues]
Many thanks for the excellent summing up of the complexity of the problem, and indeed I sense there is not yet a consensus on the 'best' solution for ECMAScript. Much of your analysis applies to ES4 as a whole of course
-----Original Message----- From: Dick Sweet Sent: 27. februar 2008 23:27 To: Lars Hansen; 'Peter Hall' Cc: 'Brendan Eich'; 'es4-discuss Discuss'; 'TC39'; 'Mike Cowlishaw' Subject: RE: Adobe position paper on the ECMAScript 4 proposal space -- decimal
Lars is correct. If you can declare decimal literals, that is enough to get you into decimal arithmetic. The various automatic coercions will do the rest, though as he said, type annotations could reduce the scope of errors. You would also need some static methods on decimal like exp and log, since the ones on Number expect doubles.
The current thinking for ES4 is that all the Math methods are polymorphic and would return decimal results for decimal inputs (and int results for int inputs when that makes sense). There could be corresponding monomorphic methods on double and decimal, too, of course, but that's not been brought up.
But do you really want to have to check the type at runtime? That could hurt performance even in cases where the type was never a decimal. Or do you intend to use early binding to select the correct method? (ie. bind to Math.decimal::cos() whenever the argument can be proven to be decimal, and bind to the normal method otherwise).
Peter
-----Original Message----- From: peterjoel at gmail.com [mailto:peterjoel at gmail.com] On Behalf Of Peter Hall Sent: 28. februar 2008 16:08 To: Lars Hansen Cc: Dick Sweet; Brendan Eich; es4-discuss Discuss; TC39; Mike Cowlishaw Subject: Re: Adobe position paper on the ECMAScript 4 proposal space -- decimal
But do you really want to have to check the type at runtime? That could hurt performance even in cases where the type was never a decimal. Or do you intend to use early binding to select the correct method? (ie. bind to Math.decimal::cos() whenever the argument can be proven to be decimal, and bind to the normal method otherwise).
Assume that the Math object is bound immutably and that its methods are bound immutably too. Then if the type of the argument to eg Math.cos is statically known to the compiler then a good implementation is clearly no worse off than it is now, it can avoid all type checking. [*]
For the other case, assume there's a variable x whose type is unknown, and then there's the call Math.cos(x). In present code that call requires a check on entry to cos() that x is a primitive number (a double), with a fallback to conversion and then a retry. The common case is that the argument is a double.
In ES4 with decimal, the common case (absent a Big Red Switch for decimal) is still that the argument is a double, so the code would be written to handle that case well (check that it's double and perform the operation; other valid types are handled on the slow branch), and there would be no slowdown compared to the existing case.
--lars
[*] In E262-3, the Math object is mutable and all its properties are mutable too, so there is pretty much nothing a traditional compiler can assume for portable code. A tracing compiler (taking into account run-time information) might do better but only if it gets to grow the context large enough to collapse multiple checks into one. In ES4, we're making the Math binding immutable (or we're providing an immutable intrinsic::Math, I forget the exact resolution), and it will have immutable intrinsic bindings. So in practice, both traditional and tracing compilers will be able to do better than in ES3.
On 27/02/2008, Brendan Eich <brendan at mozilla.org> wrote: ...
I agree with what Lars wrote, except here I think the value of a hypothetical (possibly mythical) "big red switch" is understated. ... The common problem is that you can't do "dollars and cents" or "pounds and pennies" arithmetic and get the "right answer":
js> 74.96-39.96 34.99999999999999
The problem here is not the binary float arithmetics but rather the conversion of numbers into strings. If a global switch would exist that would make a default toString conversion to use a particular version of toFixed, the problem would disappear for most user cases.
Moreover, decimal floats does not help much when calculating a VAT or similar taxes. In many countries that requires explicit rounding to cents, øre etc. of the final result with a particular minimal precision in the intermediate calculations. Such requirements can be meet both with decimal and binary floats.
For example, in Norway shops almost always show a price that includes VAT. Typically the prices are round numbers or in those x9(9) formats. Then the price excluding VAT is calculated using devisions and included into the receipt using toFixed(2). Thus with either decimal or binary floats the rounding is inevitable.
, Igor
"Igor Bukanov" <igor at mir2.org> wrote on 03/03/2008 14:39:32:
On 27/02/2008, Brendan Eich <brendan at mozilla.org> wrote: ...
I agree with what Lars wrote, except here I think the value of a hypothetical (possibly mythical) "big red switch" is understated. ... The common problem is that you can't do "dollars and cents" or "pounds and pennies" arithmetic and get the "right answer":
js> 74.96-39.96 34.99999999999999
The problem here is not the binary float arithmetics but rather the conversion of numbers into strings. If a global switch would exist that would make a default toString conversion to use a particular version of toFixed, the problem would disappear for most user cases.
That's unfortunately not true. Not only is the required toFixed number unpredictable in many applications, but it may give the wrong answer even when known. See item 2 at:
www2.hursley.ibm.com/decimal/decifaq1.html#inexact
(This particular problem is systematic, and it adds up to around $2M for a medium-sized telco. That misplaced $2M is cheating either the shareholders, the customers, or the taxman. None of those options look good in a newspaper report :-).)
Hiding a wrong answer by rounding makes problems worse, because the errors accumulate unseen.
Moreover, decimal floats does not help much when calculating a VAT or similar taxes. In many countries that requires explicit rounding to cents, øre etc. of the final result with a particular minimal precision in the intermediate calculations. Such requirements can be meet both with decimal and binary floats.
The IEEE 754r description of decimal floating-point allows for this (as do the implementations). If a requirement for an intermediate or final precision is defined as a number of decimal digits (as in Euro conversions, for example), this cannot in general be done accurately in binary floating-point.
For example, in Norway shops almost always show a price that includes VAT. Typically the prices are round numbers or in those x9(9) formats. Then the price excluding VAT is calculated using devisions and included into the receipt using toFixed(2). Thus with either decimal or binary floats the rounding is inevitable.
The rounding is inevitable, but the rounding you get using binary floating-point is different to decimal because there is no base-5 component available. (See the item referred to at the URL above.)
Mike
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
On Mar 3, 2008, at 7:25 AM, Mike Cowlishaw wrote:
"Igor Bukanov" <igor at mir2.org> wrote on 03/03/2008 14:39:32:
On 27/02/2008, Brendan Eich <brendan at mozilla.org> wrote: ...
I agree with what Lars wrote, except here I think the value of a hypothetical (possibly mythical) "big red switch" is understated. ... The common problem is that you can't do "dollars and cents" or "pounds and pennies" arithmetic and get the "right answer":
js> 74.96-39.96 34.99999999999999
The problem here is not the binary float arithmetics but rather the conversion of numbers into strings. If a global switch would exist that would make a default toString conversion to use a particular version of toFixed, the problem would disappear for most user cases.
ES3 added Number.prototype.toFixed and a few others. They are not
enough as Mike says, and practically no one knows to use them.
That's unfortunately not true. Not only is the required toFixed
number unpredictable in many applications, but it may give the wrong
answer even when known. See item 2 at:www2.hursley.ibm.com/decimal/decifaq1.html#inexact
(This particular problem is systematic, and it adds up to around
$2M for a medium-sized telco. That misplaced $2M is cheating either the shareholders, the customers, or the taxman. None of those options
look good in a newspaper report :-).)
Office Space ;-)
Lars, many thanks for posting the interesting Adobe position paper. Could you explain a little more about your position on decimal support? In particular:
Your description of the standardization of decimal arithmetic as 'premature':
The requirement for adding decimal arithmetic to ECMAScript was raised in TC39 in October 1998. The notes of TC39 WG meetings show this was discussed in the meetings of 1998/11 and also of 1999/11. The final TC39 'futures' list shows decimal support as being on the very short list of 'provisionally agreed' items for ES4, as of 1999/03.
In the meantime, the decimal arithmetic in the IEEE 754 revision (754r, now in ballot) has been widely adopted, with extensions being made to many languages, including ECMA C# and CLI, Java, ISO C, Python, and SAP ABAP, to better support it. A number of C compilers ? including GCC ? already have the new decimal support, and software libraries are available from several sources. Hardware support is now available in two server architectures: power.org's PowerPC (since June 2007) and IBM's z10 mainframes (announced yesterday).
In short, decimal support in ECMAScript is overdue, not premature.
The 'use decimal' notation means that only knowledgeable users will get to use it:
This is a valid criticism. This approach is not ideal, but at least means that where decimal arithmetic is essential it is available easily. In contrast, it is currently not available at all; it is very hard to replicate server-side calculations on the client using binary floating-point.
However, the 'big red switch' approach that you advocate would probably be better ? in essence saying that external environments can override the arithmetic base used within scripting if the scripting engine supports that. What approach would Adobe favor for this? There have been a number of suggestions (for example, in an HTML context a meta tag in the <head> of a page could be used, which page builders could add by default
for new pages). It is not necessary to define the mechanism for all hosting environments (and that is probably outside the scope of TG1 in any case).
Your assertion that implementing decimal may be a hardship for implementations on smaller systems:
Our current fixed-size decimal library, including all the 754r operations on both basic decimal formats (including FMA and many other functions and modes not needed for ECMAScript) are less than 80kBytes when compiled using GCC for Pentium. An ECMAScript implementation, using just one format, would probably need at most a third of this.
Your assertion that implementing decimal may be a hardship for implementations that cannot use existing open-source libraries:
There are now a number of open-source libraries implementing the 754r decimal types. At least two of these are essentially 'public domain' licenses. For example, ours is a part of the International Components for Unicode (ICU) library, whose license allows unrestricted use of source and derivative works in any project or product. The only requirement is acknowledgement of copyright. (For the full text of the license, which is just three paragraphs, see source.icu-project.org/repos/icu/icu/trunk/license.html .)
We have had no requests or suggestions that would imply that the ICU license would prevent any implementation from using the library. The ICU libraries are widely used in many projects and products (including a number of Adobe products).
Thanks ? Mike
Mike Cowlishaw, IBM Fellow IBM UK (MP8), PO Box 31, Birmingham Road, Warwick, CV34 5JL mailto:mfc at uk.ibm.com -- www2.hursley.ibm.com/mfcsumm.html
Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU