parseInt and implicit octal constants
I seem to recall that the brave (ie, risky) decisions to remove octal was taken over 11 years ago. Except for Opera, no-one has dared to implement the decision. That probably says something.
I also seem to recall that implementing octal in JScript .NET was far easier than updating all of the test cases that failed if I did not. ;-)
Herman
From: es-discuss-bounces at mozilla.org [mailto:es-discuss-bounces at mozilla.org] On Behalf Of Allen Wirfs-Brock Sent: Friday, February 20, 2009 12:15 PM To: es-discuss at mozilla.org Subject: parseInt and implicit octal constants
15.1.2.2 says: When radix is 0 or undefined and the string's number begins with a 0 digit not followed by an x or X, then the implementation may, at its discretion, interpret the number either as being octal or as being decimal. Implementations are encouraged to interpret numbers in this case as being decimal.
This is item 3.10 in the JScript Deviations document and my notes from when we reviewed the deviations document for ES3.1 changes says we decided to disallow the octal interpretation in ES3.1. However that change has never been made in the ES3.1 draft.
Does anyone remember if there was a subsequent explicit decision to keep the ES3 status quo or is this just an editing oversight?
Note that IE, FF, and Safari all interpret leading 0 strings as octal while Opera does not. So, arguably ,the de facto standard is to support such octal strings.
The argument for removing the option and mandating either one of the alternatives is improved interoperability across all browsers over the long term. The argument for disallowing such implicit octal strings is that they are an error hazard. The argument of requiring support of implicit octal strings is that it is the de facto standard of the web and that disallowing them may break existing webapps.
The cop-out is to just leave it as it is. The safe decision is to mandate the current de facto standard. The brave (ie, risky) decision for a better long term language is to disallow octal.
Thoughts?
2009/2/20 Allen Wirfs-Brock <Allen.Wirfs-Brock at microsoft.com>:
The cop-out is to just leave it as it is.
The safe decision is to mandate the current de facto standard.
The brave (ie, risky) decision for a better long term language is to disallow octal.
Given that Opera has survived the decision not to use octal, doesn't this establish that this decision is adequately compatible with the web? Opera folk, do you have any data (anecdotal would be fine) of how much breakage you encounter because of this decision?
If the Opera experience says the risk is acceptably low, I vote to disallow octal.
On Feb 20, 2009, at 3:26 PM, Mark S. Miller wrote:
2009/2/20 Allen Wirfs-Brock <Allen.Wirfs-Brock at microsoft.com>:
The cop-out is to just leave it as it is.
The safe decision is to mandate the current de facto standard.
The brave (ie, risky) decision for a better long term language is to disallow octal.
Given that Opera has survived the decision not to use octal, doesn't this establish that this decision is adequately compatible with the web? Opera folk, do you have any data (anecdotal would be fine) of how much breakage you encounter because of this decision?
If the Opera experience says the risk is acceptably low, I vote to disallow octal.
I wouldn't assume that an Opera-only behavior is safe for the Web;
there are a number of Opera-specific behaviors that for instance
Firefox or Safari would not accept as meeting our threshold of Web
compatibility. I would be interested in hearing what, if any, bugs
they have run into.
I think the wise thing to do here is specify a requirement for octal
support. The potential improvement in overall usability of the
language seems small and not worth taking a compatibility risk.
, Maciej
On Feb 20, 2009, at 3:26 PM, Mark S. Miller wrote:
2009/2/20 Allen Wirfs-Brock <Allen.Wirfs-Brock at microsoft.com>:
The cop-out is to just leave it as it is.
The safe decision is to mandate the current de facto standard.
The brave (ie, risky) decision for a better long term language is to disallow octal.
Given that Opera has survived the decision not to use octal, doesn't this establish that this decision is adequately compatible with the web? Opera folk, do you have any data (anecdotal would be fine) of how much breakage you encounter because of this decision?
If the Opera experience says the risk is acceptably low, I vote to disallow octal.
I wouldn't assume that an Opera-only behavior is safe for the Web;
there are a number of Opera-specific behaviors that for instance
Firefox or Safari would not accept as meeting our threshold of Web
compatibility. I would be interested in hearing what, if any, bugs
they have run into.I think the wise thing to do here is specify a requirement for octal
support. The potential improvement in overall usability of the
language seems small and not worth taking a compatibility risk.
I agree. The octal thing is problematic, but I don't think Opera's example is compelling. If we didn't have the radix parameter, which nicely corrects the problem, then I would be willing to accept the breakage that fixing this would cause. But we do, so we should play it safe.
The smart kids are using the radix parameter. The other kids have already written code which may be depending on the bad behavior.
On Fri, Feb 20, 2009 at 3:45 PM, Maciej Stachowiak <mjs at apple.com> wrote:
[...] Opera folk, do you have any data (anecdotal would be fine) of how much breakage you encounter because of this decision?
If the Opera experience says the risk is acceptably low, I vote to disallow octal.
I wouldn't assume that an Opera-only behavior is safe for the Web; there are a number of Opera-specific behaviors that for instance Firefox or Safari would not accept as meeting our threshold of Web compatibility. I would be interested in hearing what, if any, bugs they have run into.
I think the wise thing to do here is specify a requirement for octal support. The potential improvement in overall usability of the language seems small and not worth taking a compatibility risk.
Absent further information from the Opera folks, I agree. But if we can, we should postpone committing to a decision until we hear about their experience with this.
Opera folks?
On Sat, 21 Feb 2009 01:47:45 +0100, Mark S. Miller <erights at google.com>
wrote:
Absent further information from the Opera folks, I agree. But if we can, we should postpone committing to a decision until we hear about their experience with this.
Opera folks?
We have no compatiblity bugs filed regarding parseInt as far as I can tell.
On Sat, 21 Feb 2009 12:34:17 +0100, Anne van Kesteren <annevk at opera.com>
wrote:
Absent further information from the Opera folks, I agree. But if we can, we should postpone committing to a decision until we hear about their experience with this.
Opera folks?
We have no compatiblity bugs filed regarding parseInt as far as I can
tell.
Indeed. I found 6 bugs filed on parseInt octal behaviour from 2002 to
2009. Half of them are filed against test suites or JS documentation
pages, the rest appear to be authors just trying things out. As far as I
can tell we've never seen a bug on an actual production website using
parseInt() for octal.
Herman Venter wrote:
I appreciate that this proposal does not try to go all the way on octal. I am not so sure this is a good thing or that it makes the proposal more likely to succeed.
I wouldn't be opposed to removing octal entirely from the spec, but bearing in mind the section 16 wording on syntactic extensions, even that would not prevent implementors from conformantly supporting it.
For the record, I'm personally all for the proposed change, would also like to all other forms of octal go away and most of all would like to have a standard that defines just one language, not a powerset.
But even if the standard does change, I'm not going to bet on the implementations following suit any time soon. I'm not so sure that having a standard that is ignored is a good thing either.
If the change in the standard is agreed to by the representatives of the implementers, they should first be sure that the change will in fact be made in their implementations (and sooner rather than later, as in their next release).
If there is a thorough test suite for specification changes in ES3.1: bugs.ecmascript.org/ticket/449,
then I would expect there to be considerable pressure from developers for implementations to pass that test suite, as there has been in similar cases such as the ACID tests.
(Of course, a test suite cannot guarantee conformance, but it can test whether implementors have tried to address spec changes and known bugs.)
I have submitted a bug for this change to parseInt, with a test case: bugs.ecmascript.org/ticket/449.
David-Sarah Hopwood wrote:
I have submitted a bug for this change to parseInt, with a test case: bugs.ecmascript.org/ticket/449
That URL should have been bugs.ecmascript.org/ticket/459.
David-Sarah Hopwood wrote:
Herman Venter wrote:
I appreciate that this proposal does not try to go all the way on octal. I am not so sure this is a good thing or that it makes the proposal more likely to succeed.
I wouldn't be opposed to removing octal entirely from the spec, but bearing in mind the section 16 wording on syntactic extensions, even that would not prevent implementors from conformantly supporting it.
Actually, I don't think section 16 applies. Without the explicit allowance for implementation variance parseInt("010") would be explicitly required to produce decimal 10. It would be a violation of the specification for it to produce decimal 8. I don't believe that section 16 permits extensions that violates the rest of the specification in this manner.
Bottom line, I think we need to eliminate the waffling and decide to either explicitly require or forbid octal treatment of a leading zero string in parseInt. Ultimately, I think the decision will come down to whether we believe that disallowing octal will fix more undetected bugs (unwittingly applying parseInt (without preconditioning) to user input that happens to contain leading zeros) than it creates new ES3.1 breaking change bugs (intentional uses of parseInt that expect such octal constants). I expect the former but it is going to be hard to prove because most such bugs would be conditional upon actual user input.
Finally, there is another approach to resolving this issue. Define a new global function, parseInteger, that does the "right thing" and relegate parseInt to Annex B.
Allen Wirfs-Brock wrote:
David-Sarah Hopwood wrote:
Herman Venter wrote:
I appreciate that this proposal does not try to go all the way on octal. I am not so sure this is a good thing or that it makes the proposal more likely to succeed.
I wouldn't be opposed to removing octal entirely from the spec, but bearing in mind the section 16 wording on syntactic extensions, even that would not prevent implementors from conformantly supporting it.
Actually, I don't think section 16 applies.
It doesn't apply to parseInt; it does to octal numeric literals and string/regexp octal escapes (which "going all the way on octal" would remove from the spec).
[...]
Finally, there is another approach to resolving this issue. Define a new global function, parseInteger, that does the "right thing" and relegate parseInt to Annex B.
That's not a bad idea, given that parseInt has the additional flaw of silently stopping at the first invalid character, which this change will not fix. But it should be a "static" method of Number, not a global, to avoid further polluting the global namespace and potentially clashing with existing user functions.
On Sun, Feb 22, 2009 at 9:30 AM, Allen Wirfs-Brock <Allen.Wirfs-Brock at microsoft.com> wrote:
David-Sarah Hopwood wrote:
Herman Venter wrote:
Finally, there is another approach to resolving this issue. Define a new global function, parseInteger, that does the "right thing" and relegate parseInt to Annex B.
That is a good idea, but I wonder: Would programs use that?
a program could use:
i = parseInt(x)
-
which can result in undesirable behavior. That is fixed by:-
i = parseInt(x, 10)
-
or the proposal:-
if(typeof parseInteger == "function") { i = parseInteger(x); } else { i = parseInt(x, 10); }
-which seems a bit clunky. How about just:-
i = parseInteger(x)
but that last one may result in an error, should the implementation not support the new function. Programs that use parseInt(x, 10) will work in more implementations.
Garrett
On Mon, Feb 23, 2009 at 10:13 AM, Garrett Smith <dhtmlkitchen at gmail.com> wrote:
On Sun, Feb 22, 2009 at 9:30 AM, Allen Wirfs-Brock <Allen.Wirfs-Brock at microsoft.com> wrote:
David-Sarah Hopwood wrote:
Herman Venter wrote:
Finally, there is another approach to resolving this issue. Define a new global function, parseInteger, that does the "right thing" and relegate parseInt to Annex B.
That is a good idea, but I wonder: Would programs use that?
a program could use:
i = parseInt(x)
- which can result in undesirable behavior. That is fixed by:-
i = parseInt(x, 10)
- or the proposal:-
if(typeof parseInteger == "function") { i = parseInteger(x); } else { i = parseInt(x, 10); }
-which seems a bit clunky.
How about this:
parseInt(x, 10) is already the compatible version "parseInteger".
You could define the function parseInt to simply throw a typeError if the second argument is missing. Or perhaps to break less existing programs- to throw some kind of error should it recieve a string with a leading zero in the first argument, and there is no supplied second argument. This essentially turns a silent bug into a noisy bug, which may be desirable.
David-Sarah Hopwood wrote:
Allen Wirfs-Brock wrote:
Finally, there is another approach to resolving this issue. Define a new global function, parseInteger, that does the "right thing" and relegate parseInt to Annex B.
That's not a bad idea, given that parseInt has the additional flaw of silently stopping at the first invalid character, which this change will not fix. But it should be a "static" method of Number, not a global, to avoid further polluting the global namespace and potentially clashing with existing user functions.
It seems like having two functions with the same usecase but subtly different semantics contributes significantly to the cognitive burden of learning the language, and of reading other people's code, without — at least in this case — providing a huge number of clear cut benefits.
It seems like standardizing something that is theoretically nicer without any clear promise that implementations will converge on that behavior is not doing users any favours since they will still not be able to predict the behaviour of implementations they have not tested, rendering the specification pointless. That suggests to me that specifying the de-facto standard set by Microsoft/Mozilla/Apple here is the right way forward.
James Graham wrote:
David-Sarah Hopwood wrote:
Allen Wirfs-Brock wrote:
Finally, there is another approach to resolving this issue. Define a new global function, parseInteger, that does the "right thing" and relegate parseInt to Annex B.
That's not a bad idea, given that parseInt has the additional flaw of silently stopping at the first invalid character, which this change will not fix. But it should be a "static" method of Number, not a global, to avoid further polluting the global namespace and potentially clashing with existing user functions.
It seems like having two functions with the same usecase but subtly different semantics contributes significantly to the cognitive burden of learning the language, and of reading other people's code, without — at least in this case — providing a huge number of clear cut benefits.
Reporting an error when there are invalid characters in the string, rather than silently returning the wrong value in that case, is a clear-cut benefit.
parseInt is not particularly useful to programmers who actually care about input validation. (If you validate that a string represents a decimal integer first, then you could just as well use 'Number(s)' or '+s' to convert it to a number.)
15.1.2.2 says: When radix is 0 or undefined and the string's number begins with a 0 digit not followed by an x or X, then the implementation may, at its discretion, interpret the number either as being octal or as being decimal. Implementations are encouraged to interpret numbers in this case as being decimal.
This is item 3.10 in the JScript Deviations document and my notes from when we reviewed the deviations document for ES3.1 changes says we decided to disallow the octal interpretation in ES3.1. However that change has never been made in the ES3.1 draft.
Does anyone remember if there was a subsequent explicit decision to keep the ES3 status quo or is this just an editing oversight?
Note that IE, FF, and Safari all interpret leading 0 strings as octal while Opera does not. So, arguably ,the de facto standard is to support such octal strings.
The argument for removing the option and mandating either one of the alternatives is improved interoperability across all browsers over the long term. The argument for disallowing such implicit octal strings is that they are an error hazard. The argument of requiring support of implicit octal strings is that it is the de facto standard of the web and that disallowing them may break existing webapps.
The cop-out is to just leave it as it is. The safe decision is to mandate the current de facto standard. The brave (ie, risky) decision for a better long term language is to disallow octal.
Thoughts?