Re precision, etc.

# Mike Cowlishaw (17 years ago)

Re:

----- Message from Waldemar Horwat <waldemar at google.com> on Fri, 22 Aug 2008 11:39:57 -0700 -----

Contrary to some beliefs, the different cohort members in IEEE P754 are not a valid indicator of precision. To prove this, assume that the different cohort members are an indicator of precision. Then evaluate:

3.00 + 1.00000

IEEE P754 returns 4.00000, while the result is actually precise to only two digits after the decimal point. Hence a contradiction. QED.

The preservation of exponents in IEEE 754 is indeed not an indication of 'precision' in the significance arithmetic sense, and it was never intended to be. See:

speleotrove.com/decimal/decifaq2.html#precision

for some of the uses of the word 'precision' -- it is best avoided -- and see:

speleotrove.com/decimal/decifaq4.html#signif

for why 'significance arithmetic' makes no sense at all, beyond single operations and rules-of-thumb.

The rules in 754 codify current 'best' practice (the rule for division, for example, came from the .Net and C# division operator in its decimal class; it has the very desirable property that 2.40/2 gives 1.20, etc.). The result of 1/.1 is a necessary consequence of that .. and a lesser evil than not preserving the information that's available. There might well be a better rule for division, but that was the best proposal (and that has been in the public drafts since 2003, so there has been plenty of time to make alternative proposals).

On the Array indexes, I have some homework to do, obviously. I had thought ES arrays were indexed by integers (0 though 2^n-1) rather than by strings (as in Rexx stems). If an index of "0.00" is indeed different from one of "0" than it would seem to be appropriate that the same distinction apply if the index is a decimal -- because the whole idea of decimal numbers is that they are the same as the string representation, from the user's point of view.

Mike

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

# Waldemar Horwat (17 years ago)

Mike Cowlishaw wrote:

Re:

----- Message from Waldemar Horwat <waldemar at google.com> on Fri, 22 Aug 2008 11:39:57 -0700 -----

Contrary to some beliefs, the different cohort members in IEEE P754 are not a valid indicator of precision. To prove this, assume that the different cohort members are an indicator of precision. Then evaluate:

3.00 + 1.00000

IEEE P754 returns 4.00000, while the result is actually precise to only two digits after the decimal point. Hence a contradiction. QED.

The preservation of exponents in IEEE 754 is indeed not an indication of 'precision' in the significance arithmetic sense, and it was never intended to be. See:

speleotrove.com/decimal/decifaq2.html#precision

for some of the uses of the word 'precision' -- it is best avoided -- and see:

speleotrove.com/decimal/decifaq4.html#signif

for why 'significance arithmetic' makes no sense at all, beyond single operations and rules-of-thumb.

The rules in 754 codify current 'best' practice (the rule for division, for example, came from the .Net and C# division operator in its decimal class; it has the very desirable property that 2.40/2 gives 1.20, etc.). The result of 1/.1 is a necessary consequence of that .. and a lesser evil than not preserving the information that's available. There might well be a better rule for division, but that was the best proposal (and that has been in the public drafts since 2003, so there has been plenty of time to make alternative proposals).

You're still making the unjustified assumption that the choice of cohort member is productive information. The examples here have shown that it is not, and it leads to bad results. Things are much simpler by not trying to make questionable distinctions between 1000000000000 and 1e12.

Waldemar
# Mike Cowlishaw (17 years ago)

You're still making the unjustified assumption that the choice of cohort member is productive information. The examples here have shown that it is not, and it leads to bad results. Things are much simpler by not trying to make questionable distinctions between 1000000000000 and 1e12.

You are quite right to question that last case (I think there is plenty of evidence that the choice of exponent is important and productive 'in real life' -- for example, the most common reason for calculators being returned as 'broken' is when a calculation such as 1.23 + 1.27 does not give 2.50).

In that last case, indeed, (and perhaps naively) when I designed the Rexx language I decided that any 'positive exponent' case should be treated as though it were an integer: 1E+9 in the underlying representation was (and is, in classic Rexx) displayed/printed as 1000000000.

I now think that was a mistake -- it's not simpler; it is a 'special rule' that often confuses. If one sees a number such as 1.23E+9 (as in 1.23 GHz for example), it's very misleading to convert that for display as 1230000000 (units Hz). It gets even worse in computing contexts, with all the usual arguments as to whether a GB is 1000 MB or 1024 MB (etc.).

So the general idea is: the string representation should provide a reversible and exact mirror of the underlying decimal number. Your worry about the divide case is valid, but that is a concern about the division rules, not about the string representation (the division rules could have been written to always return an integer if the exponent would be positive and fits -- but that introduces a different set of problems).

[Separately .. I am having problems with my subscription to this discussion; I am getting digests but cannot convert to individual e-mails as I don't have a password. So please keep cc: me on replies!]

Mike

Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU