Close review of Language Overview whitepaper
On Nov 14, 2007 2:03 PM, Maciej Stachowiak <mjs at apple.com> wrote:
Nullability: Are non-nullable types really worth it? I am not sure. Does any other explicit type system for a dynamic OO language have such a concept? The whitepaper says that "the ability to store null is occasionally the source of run-time errors" but will not dynamic- checking result in runtime errors anyway when assigning null to a non- nullable variable (except in strict mode)?
That was one of the features I most liked. Working in Flex, I have problems with nullable variables all the time.
Map: Long overdue to have a real hashtable type.
Yes! Parameterized types in general are probably my favorite addition. (Now if only there were some sort of solution comparable to abstract classes....)
Maciej Stachowiak wrote:
Hello ES4 fans,
I have now read the recently posted whitepaper. I marked up my printed
copy with many comments in the margins, and I am sharing them with the
list now.
Excellent comments, thanks!
Some points in response, both places where I agree with you and places I might offer some explanation.
Classes: If any of the new type system is worthwhile, surely this is.
The impedance mismatch between the class model used by most OO
languages and by specifications like the DOM, and ES3's prototype
model, is needlessly confusing to authors. So I approve of adding
classes in a reasonable and tasteful way.Dynamic properties: the fact that the "dynamic" behavior is not
inherited makes class inheritence violate the Liskov Substitution
Principle. I think this is a problem. Subclassing should be subtyping
in the LSP sense. I am not sure offhand how to fix this.
LSP is not preserved by most forms of OO inheritence, alas; as soon as you have overriding of methods, it's gone.
Virtual Properties: I wish the keyword for catchall getters and
setters was something other than "meta", which is a vague word that
doesn't mean much. Why not "catchall" or "fallback" or something along
similarly concrete lines? (I realize now upon re-reading my margin
comments that this is supposed to match meta invoke, but there too I
am not sure the relationship is worth the vagueness.)
Plausible. We reserved the namespace for "stuff that you would use a fully-formed MOP to fiddle with", but other names would work.
Literals:
- I am surprised to see a decimal type (a type that is not directly
supported in current mainstream hardware) even though generally
popular types like single-precision IEEE floating point and 64 bit
integers are not present.
This is based on bug-report / user-request frequency. ES is used by a lot of people who do not do numerical computation for a living, notably in casual yet meaningful financial circumstances. Most financial math in ES3 is presently wrong.
Whether or not it shows up in the hardware (IBM and Intel have both made varying degrees of commitment) is mostly irrelevant: the key is actually arriving at correct answers in contexts like tax law and shopping carts, where it really matters.
- Since ints/uints overflow to doubles, then either all int math must
be performed in double space (requiring constant conversions when
working with int variables), or every operation must check for
overflow and possibly fall back to double space. Even when the final
result cannot overflow, certainly in many expressions the difference
between int and double intermediates can be observed. It seems likely,
then, that math on variables declared int will be slower than math on
variables declared double, which will surely be confusing to
developers. This seems pretty bogus. Is there any case where int math
using the normal operators can actually be efficient? Would it be
plausible to make ints not overflow to double unless there is an
actual double operand involved (in which case int constants would
always need a special suffix, or perhaps can somehow be determined
contextually).
Possible. It would require unadorned literals to default to double (with optimized-but-overflowing int as an invisible subcase, as in ES3), and users to request specific instances of int and uint types with 'i' and 'u' suffixes. Fits with the treatment of decimal. Might be preferable to the uint32ops namespace.
Record and array types: Structural types are confusingly similar to
yet different from classes. Mostly they offer a subset of class
functionality (though reading ahead I did see a few features limited
to them). Also, already having prototype-based objects and class-based
objects it seems excessive to add yet a third way. I recommend
removing them and adding any features that are sorely missed as a
result to classes.
I concur that this step would greatly simplify the language; I have no clear feeling about whether the structural types are more or less crucial for interoperability.
The arrival at "structural types" as a general term followed from the consensus that we wanted to capture anonymous function and union types. Arrays and records without class-hierarchy membership seemed to polish off the needs of modeling existing ES3 style latent types, but perhaps merging them both back into classes and using a sufficiently weak "like" operator will do. We've been around this block many times.
"Any": The spec explains vaguely that the "any" type is not identical
to the union (null, undefined, Object). How is it different? Is the
difference observable to ES4 programs or is it purely a matter
internal to the spec (in which case the difference is not relevant)?
I do not think that it is observably different. I may be incorrect; best for the type theorists to respond.
Nullability: Are non-nullable types really worth it? I am not sure.
Does any other explicit type system for a dynamic OO language have
such a concept? The whitepaper says that "the ability to store null is
occasionally the source of run-time errors" but will not dynamic- checking result in runtime errors anyway when assigning null to a non- nullable variable (except in strict mode)?
Well, I argued for this initially, so I guess I'll defend it :)
It is precisely this ability to "push the null pointer error earlier" that we seek. If you push it as far back as a function signature or interface barrier, it is easier to detect, comprehend and recover from. If you push it back to strict-check time, it is easier still. This is all well-demonstrated in other OO languages with static types: C# ("structs"), nice, haxe, felix, scala ... it's not terribly new.
For dynamic languages that bother providing useful and fine-grained type predicates, well ... that's not a huge number of languages! But consider common lisp, it does what we do, as far as I know: there's an empty type that holds no values (our void type, their nil type) and there's a value that holds a unique sentinel (their nil value, our null value) which lives in its own unique type (their null type, our null type). Like what we're doing, you need to union this with a class type to get a nullable class type.
(As far as I can tell -- not being a dylan hacker -- dylan doesn't even go as far as having a global sentinel type like nil)
package: Now that I have learned more about them, I think that
exposing packages and namespaces as separate user-level concepts is
confusing. Let's get this down to a single concept that developers
have to learn. Namespaces can just have a paired internal namespace
implicitly, I do not think it is helpful to give the public/internal
pair a special different name.
I too am a bit uneasy about the proliferation of concepts here (package, unit, and namespace). We've tried to fuse some of them in the past, but with little luck. Your suggestion is good. Let's flesh it out: make a bug. It has to support the notion of a public and private "half", one of which can be named from outside the syntactic form that defines the namespace, one of which can only be used from inside the syntactic form that defines it.
Program units:
- Is there any need for the concept of "unit" to be exposed in the
syntax? Why not just allow "use unit" at top level, and implicitly
make each file (or in the browser context each inline script) a unit?
We've heard input that a server-side / ahead-of-time transformation of inlining nested units is desirable, in the sense that it makes for a single chunk of text you can send in a single HTTP response. The syntactic form was designed to permit this possibility: you can replace a use with its definition textually, without altering the meaning.
- I think the difference between using units and importing packages is
going to be confusing to authors. Seriously, can anyone explain in one
sentence of 12 words or less how Joe Random Developers will decide
whether to use a namespace, import a package, or use a unit? Can we
get this down to only one kind of thing that needs to be mentioned in
the syntax? This would be a big win in reducing conceptual footprint.
True, though I suspect at least 2 of them need to remain. Namespaces are quite disjoint from loading and definition order. My eye is on packages.
Type annotations and type checking: This section implies that type
annotations are not at all being added for performance reasons and may
indeed be harmful to performance. Wow! Seriously? I think runtime
assertions are interesting when debugging but I do would not want them
happening for every assignment statement in a release build of my C++
code.
You may not be the sole voice of opinion on that matter :)
Types are the meeting point between two different -- and equally important! -- pressures felt throughout programming: correctness and speed. It is fair to discuss both; you're right that it is unfair to deny the motivation of one. Typed ES4 programs may run faster due to a variety of optimizations enabled by types. Whether those are performed is up to implementations. Specifically:
- Dense allocation of the fixed portion of an object
- Early-binding of fixture references to fixed properties
- Inlining after early binding
- Specialized selection of primitive operators
However, note that an implementation of ES4 clever enough to perform a large set of these optimizations may not be terribly different from an implementation of ES3 clever enough to synthesize types optimistically on the fly and check its guesses against runtime facts, correcting any that rest on violated assumptions. These sorts of clever runtimes may actually notice no speed difference whatsoever.
What a clever runtime cannot do is synthesize the programmer's intentions wrt. type-based correctness conditions of their code. The programmer needs to say "this variable being an int is part of what it means to be correct". That is why there is a focus on the correctness motivation of types here, not the speed motivation.
Pragmas: The "use decimal" pragma highlights how much complexity there
is to the decimal type. Seriously, is it worth it? Is the problems it
solves really that common?
Yes. Users have been reporting their displeasure with binary floating point arithmetic on the web since way back in the 20th century.
See also: www2.hursley.ibm.com/decimal/decifaq1.html#dbstats
Generators: Do ordinary programmers really understand coroutine
control flow? Is this really a significantly better paradigm than
passing a visitor function? Not really convinced in this one yet.
The key purpose is to make for loops look like for loops. Agreed that it might be a heavy way of achieving it, but it divides the labor asymmetrically: the library author does the hard work, the users don't. With visitor functions, the users all have to get cozy with higher order functions.
"switch type" statement: I guess this beats switching on typeof, but
is it really significantly better than a series of "if" statements
using the "is" operator?
It is better from a static-type perspective, in an implementation that wants to optimize union type representations. Maybe this is not sufficiently compelling. I suggested it, and I concur that it's trimmable.
Expression closures: I actually find the examples hard to follow given
my expectation of ES3-like syntax. I think this may actually be
syntactic salt.
No opinion. I know how to write either way.
"type": Are runtime meta-objects representing types ruly necessary?
What are they good for?
Tooling, runtime code synthesis and analysis, runtime adaptation to code added at a later date, etc. Completeness of the dynamic nature of the language ... ask a smalltalk or lisp person :)
Slicing: This one I mildly object to. Array/String slicing is not, to
my knowledge, particularly common in ECMAScript code of today. I am
dubious that it merits its own operator syntax.
No opinion. Similar to expression closures: willing to trim.
Early binding, static type checking, and predictable behavior with
"intrinsic": Perhaps it should be highlighted more that this is a
potential significant performance improvement.
Agreed! If it took until this late in the document to understand that angle, it ought to move up.
Reflection: This feature seems like it could be complex to implement
and potentially unnecessary for small implementations. I note that
J2ME omits reflection, which we can perhaps take as a sign that it is
not suitable for small implementations.
Not clear. The language has to keep quite a lot of machinery around for its dynamic aspects anyways; the idea is to given an interface through which the runtime can be asked to lazily manufacture reflective bits that cover only the things it already has to carry. If you see things that you think could be optimized away while still executing the rest of the language, by all means suggest cutting.
ControlInspector: I think an interface that's meant for debuggers and
similar tools, and not implementable in all interesting contexts, does
not need to be standardized. Better than having an optional feature.
Not clear. Debuggers are one perspective, sure; but general dynamic scope is a useful (if sharp and dangerous) feature for advanced library authors (security contexts, continuations, dynamically-scoped resources), and they can't get it otherwise without interposing wrapper functions everywhere, and that defeats tail calls.
uint-specific operations: This is syntactically ugly. Why can't
integer math just always work this way? Also, why only uint versions?
Surely it is desirable to do efficient math on signed integers as
well. Also, bitops already happen in integer math space, thus type- specific versions should not be necessary since no floating point
conversion will need to occur if both operands of ^ or & are
statically typed as int or uint.
As said above, your ideas about how better to reform int/uint/double arithmetic and promotion are compelling. Can we move it to a bug and hash it out there? I don't want to lose it.
Again, thanks so much for giving it the fine-tooth comb treatment. This is valuable feedback.
Hey Maciej, thanks for the detailed comments. As many detailed
responses as I can muster below -- all opinions mine unless noted
(e.g. where I cite a group opinion).
On Nov 14, 2007, at 2:03 PM, Maciej Stachowiak wrote:
Goals: I strongly agree with the stated goals of compatibility and enabling large software development. I wonder if perhaps performance should be added as a goal. At the very least we want it to be possible to achieve performance on par with ES3 engines, and ideally we want to enable better performance.
I know of proofs that much better performance (order of magnitude
over latest interpreters) is coming to JS1 implementations, so I
think this is a red herring. It would be good to avoid anti-
performance mandatory changes, of course -- that might not be obvious.
Programming in the small: "... make the writing and reading of fragments of code simpler and more effortless." That is somewhat dubious gramatically,
English crit, watch out! ;-) I did not write this, but I'll leap to
the author's defense. The grammar's fine and usage manuals
(Partridge, Fowler -- if memory serves) allow "more effortless". See
also
www.google.com/search?hl=en&client=firefox-a&rls=org.mozilla% 3Aen-US%3Aofficial&hs=CGS&q=%22more+effortless%22+English +usage&btnG=Search
First hit:
www.bartleby.com/227/0206.html
I suggest (with additional style fixes) "make the reading and writing of code fragments easier."
I like your suggestion, though.
Portability: This section first it says that the full language must be supported - subset profiles are not desirable. Then it says that, to allow ES4 to be practically implementable on small devices and in hosted environments, certain features, like extensive compile-time analysis and stack marks cannot be part of the language. Then it says those features are part of the language, but optional.
Clearly the overview should distinguish between mandatory standard
mode features and optional strict mode and reflection features.
But you have a point about optional reflection features being useless
on the web, so what's the point? One answer is to have a normative
spec, so where support exists, the implementations can interoperate.
This may not be enough for reflection -- it depends on likely uptake.
We think it is enough to justify strict mode. More below.
Syntax: The new non-contextual keywords, and the resulting need to specify dialect out of band, are a problem. I'll have more to say about compatibility under separate cover.
We've talked about this before, and I'm looking forward to your
thoughts.
I would hope we can avoid having to do what we believe should be post-
ES4 standardized AST, reader, and even macro work, just to allay
concerns about ES4->ES5. We can't do all that work before ES4, but of
course we want to make next time better. I noted on IRC that adding
the 'x' flag to regexps, and codifying how IE handles / in a
character class in a regexp (which browsers have had to follow), both
break by-the-book ES3 scanners.
Behavior:
- This section has says that "variation among ES3 implementations entails a license to specify behavior more precisely for ES4". However, the example given is a case where behavior among two implementations was already the same, due to compatibility considerations. I actually think both convergence on a single behavior where variation is allowed, and variation that leads to practical compatibility issues are license to spec more precisely,
We do not want to overspecify, however. The majority of those who've
expressed an opinion in TG1 do not want, e.g., to specify Date.parse
as it is implemented in any given browser, never mind finding the
intersection among all browsers.
- The RegExp change - is this really a bug fix? It's likely that this is not a big compatibility issue (Safari's ES3 implementation had things the proposed ES4 way for some time) but I think ES3's approach may be more performance and generating a new object every time does not seem especially helpful.
This bug is the second most duplicated among bugs filed with
mozilla.org's bug system since 1998:
bugzilla.mozilla.org/show_bug.cgi?id=98409
Full dup-count available at
bugzilla.mozilla.org/duplicates.cgi? sortby=component&maxrows=1000&changedsince=3600&product=Core
sort by Component for best results.
This singleton pigeon-hole problem hurts users all the time (I get
mail from individual developers confused by it, at least once a year).
The performance worry for the proposed ES4 fix of evaluating a regexp
literal to a new object, just as is done for other mutable objects
expressed literally, is not an issue in our experience. Just as the
compiler precomputes invariant parts of function objects, so it can
memoize regexp constant parts, and wrap a mutable object around the
shared immutable innards on each evaluation.
Quality implementations do this for function objects already, and
some do it for regexps too, in order to support executing a
precompiled script in several execution contexts (possibly
concurrently).
Is there any significant implementation that anyone would claim is 100% free of ECMAScript 3 compliance bugs?
No, and that's a good thing!
I doubt it, and so I think we should make this section less judgmental in tone.
Agreed, thanks for pointing this out. I certainly missed it (everyone
has bugs; some bugs are just problems in an implementation to fix,
and we need not ascribe them to inadequate engineering practices, or
blame darker motives for that matter :-/).
Wrappers: The whitepaper implies that providing catchall getters and setters for primitive types and skipping boxing isn't a compatibility issue. However, it is possible in ES3 to capture an implicit wrapper:
var x; String.prototype.myFunc = function() { this.foo = "foo"; x = this; }; "bar".myFunc();
Prototype hacking allows you to observe identity of the temporary wrappers, save them for later, and store properties. Perhaps there is evidence that practices relying on techniques like this are exceedingly uncommon (I'd certainly believe it), if so it should be cited.
This was a bug to fix, which I brought up too late to be included in
the overview. See
bugs.ecmascript.org/ticket/281
We believe this is fixed now, but please comment there if you see a
problem still.
Literals:
- I am surprised to see a decimal type (a type that is not directly supported in current mainstream hardware) even though generally popular types like single-precision IEEE floating point and 64 bit integers are not present.
See the third most duplicated bug in bugzilla.mozilla.org:
bugzilla.mozilla.org/show_bug.cgi?id=5856
See also the "Apple Flunks First Grade Math" blog post, I kid you
not, here:
www.mikeindustries.com/blog/archive/2004/08/apple-calculator
I am sure on this basis you will agree with our inclusion of decimal
in ES4 :-].
Even when the final result cannot overflow, certainly in many expressions the difference between int and double intermediates can be observed. It seems likely, then, that math on variables declared int will be slower than math on variables declared double, which will surely be confusing to developers.
Are you sure integer ops with well-predicted overflow guards are
slower than FP ops? I have contrary evidence and know of some papers
in submission on this that suggest it is not a problem. But some of
these papers also show the advice in favor of annotating :int to
"improve performance" to be bad advice.
In other words, I'm disputing your contention that it's likely let
i:int loop control variables will be slower than let i:double -- but
I'm also saying that I think let i may be as fast as the faster of
the annotated forms in ES4-as-proposed, on some of the forthcoming JS
VMs. Performance is a red herring, again. To be demonstrated!
Record and array types: Structural types are confusingly similar to yet different from classes. Mostly they offer a subset of class functionality (though reading ahead I did see a few features limited to them). Also, already having prototype-based objects and class-based objects it seems excessive to add yet a third way. I recommend removing them and adding any features that are sorely missed as a result to classes.
You miss the main difference: structural types are not equated by
name, so they avoid the inheritance trap of classes, which consists
of building hierarchies deriving from non-final nominal types
(classes and interfaces) that constrain the evolution of the base
types; or else slapping final too much on base classes (especially in
the standard library).
Structural types formalize the "duck typing" done in ES3 code on the
web. Most such code will never be retrofitted to use nominal types,
but with structural types for API parameters, along with 'like' and
'wrap', duck-typed data from such code can flow into typesafe libraries.
This is a crucial use-case, and perhaps the overview needs to spend
more time on it. It's not something I (at least; also the majority of
the group working on ES4, I think) will give up easily.
"Any": The spec explains vaguely that the "any" type is not identical to the union (null, undefined, Object). How is it different? Is the difference observable to ES4 programs or is it purely a matter internal to the spec (in which case the difference is not relevant)?
Did you see footnote 17? It's a formality, perhaps not worth
burdening the overview reader with, though.
"wrap": Seems like a version of this feature and/or "like" founded on classes would work just as well.
Not so -- consider retrofitting existing code that passes objects and
arrays, in JSON-like trees, to Ajax library APIs. We do not want to
require all the library-client code to be rewritten to use nominal
types. We should not prefer nominal types anyway, since they do not
scale to the web due to the base-class pigeon hole problem I
mentioned above, which leads to overconstrained class hierarchies
over time, or else the final keyword prohibiting derived classes in
the first place.
Conversions: "In addition, any value in the language converts to a member of AnyBoolean", but the conversions specified are all to the more specific "boolean" type, so perhaps it should be expressed that way to avoid confusion.
I thought this too, when reviewing this section. I think this is an
open issue, but I can't find a ticket for it. I'll follow up.
Binding objects and scopes: It seems like introducing lexical block scopes makes things more challenging for online implementations. Creating a dynamic scope object per block scope is clearly unacceptable, but more work may be needed to build a per-function symbol table that can properly accomodate block scope.
Shipped in Firefox 2. It was Not Hard ;-).
Is block scope worth it? Yes, "var" is a little weird, but having both "var" and "let" increases conceptual footprint and may overall lead to more author confusion.
Block scope is important to those larger-scale programs you favored
early on. We have experience from Firefox 2 on supporting this, as do
others who've extended ES3.
package: Now that I have learned more about them, I think that exposing packages and namespaces as separate user-level concepts is confusing. Let's get this down to a single concept that developers have to learn.
As I noted in a previous thread, I'm sympathetic...
Namespaces can just have a paired internal namespace implicitly,
... but not to this proposal. A namespace should not be two
namespaces in disguise. I'd rather defer packages altogether from ES4
than double namespace costs and remove their primitive utility (which
ES4 relies on heavily).
I do not think it is helpful to give the public/internal pair a special different name.
It is definitely not helpful to double the cost of namespaces just to
cover package's use-cases.
Program units:
- Is there any need for the concept of "unit" to be exposed in the syntax? Why not just allow "use unit" at top level, and implicitly make each file (or in the browser context each inline script) a unit?
See
for examples including server-side expansion and client-side caching
of units, independent of how they are stored in files or transported
in HTTP responses.
Versioning: I am suspicious of versioning mechanisms, especially big giant switch versioning. Is there any use of ECMASCRIPT_VERSION that is not better handled by feature testing? (Maybe there is and I am not thinking of it.)
Two points:
-
Feature-testing in multi-version languages and libraries, in my
experience at least, has produced more implicit version combinations
against which to measure and support compatibility, than a total
version order, ideally a number line. -
We expect users to write in the new language and transcode to the
old using offline or just-in-time source-to-source translators. Such
systems want whole-language versioning, not feature tests.
arguments: It seems strange to both deprecate a feature and improve it at the same time.
Deprecation via strong language in specs is worth the paper it's
printed on, IMHO. You need carrots as well as sticks. Rest params are
the sweetest carrot, but arguments will endure, and already Opera (at
least) makes arguments delegate to Array. People want it, and it is
easy and free of downside. Give the people what they want here :-).
Strict:
- I would strongly prefer if strict mode did not alter behavior of programs at all, except to reject those that do not pass the checks. Otherwise, since strict mode is optional, this risks interop issues.
The interop issue would be that standard mode runs a program that
would fail at runtime with strict mode's eval change. Such a program
must therefore make bindings in eval's dynamic scope. We can't have a
sound strict mode without this change, and there may be others --
Mark Miller has me convinced that a short list of runtime semantic
changes (restrictions, exception-throwing) made by Google Caja are
all necessary. More on this in due course.
See two responses below for more on why it's probably helpful to
interop to specify an optional strict mode, once for all
implementations that choose to support 'use strict'.
So I'm curious what the eval detail is. Perhaps strict mode could remove the eval operator and allow only the eval function, with some suitably named version made available ahead of time, if the difference is just removing local eval behavior.
It's not that simple The problem is the contents of the eval
argument, which are not visible to strict mode or the compiler.
- I am somewhat concerned about having strict mode at all. It seems like it could create the same kinds of problems we see today with content that is served as application/xhtml+xml to some browsers and text/html to others. It's not infrequent to see such content break only in the browsers that really support XML, due to sloppy testing of changes and the fact that the 78% browser doesn't support XHTML.
We see this problem from the other side. There are numerous competing
"lint-like" tools, and some optional type checkers in JS-derived
languages. These will tend to produce interop bugs with greater
frequency in the absence of a normative strict mode spec (optional at
each implementation's discretion) than if there were such a spec for
strict mode. So we are specifying. If an ES4 implementation supports
'use strict', it must follow the spec.
Verification:
- Does strict mode actually allow for any optimizations that couldn't be done to the exact same program in standard mode?
IMHO strict mode has nothing to do with optimizations.
Section IX.
"switch type" statement: I guess this beats switching on typeof, but is it really significantly better than a series of "if" statements using the "is" operator?
And lexical bindings for the types cracked by the cases? Yes.
Consider typed exception handling vs. a single catch clause with an
if/else chain or switch (and the obligation to re-throw in the else
or default -- which users forget).
You're right that switch type, like destructuring assignment, is
sugar. We are providing sugar. It reduces code size, improves
readability, and eliminates bug habitat. Its cost in practical JS
parsers, based on Firefox 2 and other experience, is in the noise.
Expression closures: I actually find the examples hard to follow given my expectation of ES3-like syntax. I think this may actually be syntactic salt.
Implemented in the RI, and in Firefox 3 beta. Based on the experience
of people using these, it's sweet, sweet sugar -- but tastes vary. My
only thought is to urge you to use expression closures in your own
code, and give them a chance to grow on you, before rendering final
judgment.
Semicolon insertion: I'd like more detail on the compatibility of the return change.
The return change was withdrawn. See
bugs.ecmascript.org/ticket/263
Reflection: This feature seems like it could be complex to implement and potentially unnecessary for small implementations. I note that J2ME omits reflection, which we can perhaps take as a sign that it is not suitable for small implementations.
Or blow bronx cheers. :-P Sorry, but citing J2ME is fighting words to
me, and proof of nothing in particular.
JSON: Sounds good.
This proposal is withdrawn and another API is being considered for
reinclusion later. See http:;//json.org/json2.js.
Things I didn't see:
What about standardizing the de facto <!-- comment syntax that is necessary for web compatibility?
Please write the spec ;-). Don't forget the --> mess.
JSON: Sounds good. This proposal is withdrawn and another API is being considered for reinclusion later. See http:;//json.org/json2.js.
toJSONString and parseJSON are going away? I was actually wanting to write and suggest the removal of these, with Douglas's recent change in his JSON API. I am glad to see these will be going away. Will ES4 include the other API (JSON.parseJSON / JSON.stringify)?
You miss the main difference: structural types are not equated by name, so they avoid the inheritance trap of classes, which consists
I would love to understand the purpose of structural types better. I don't understand how base class evolution is constrained in ways that super record types aren't. I also don't understand how the goal of applying types to existing ES3 objects can not be achieved with wrap operator and nominal types. I know there something I am missing here. Thanks, Kris
On Nov 14, 2007, at 9:51 PM, Kris Zyp wrote:
JSON: Sounds good. This proposal is withdrawn and another API is being considered for reinclusion later. See http:;//json.org/json2.js. toJSONString and parseJSON are going away? I was actually wanting
to write and suggest the removal of these, with Douglas's recent change in
his JSON API. I am glad to see these will be going away. Will ES4 include
the other API (JSON.parseJSON / JSON.stringify)?
(It's JSON.parse, not JSON.parseJSON.)
I thought I answered that above, sorry if I was unclear. We like
Doug's new API and hope to include it in due course into ES4. JSON
was already an accepted proposal, so it has a foot in the door.
Various interested parties favored something like the json2.js API
already, and I think everyone will rally round it and beat on it, to
make sure it has the right usability and knobs. I'm hopeful.
You miss the main difference: structural types are not equated by
name, so they avoid the inheritance trap of classes, which consists I would love to understand the purpose of structural types better.
I don't understand how base class evolution is constrained in ways that
super record types aren't.
You can have an object {p: 42, q: "hi"} match an annotation of the
form 'like {p:int, q:string}', but there is no class with which that
object is compatible in any well-defined sense except for Object itself.
While you can make a subclass of Object, say class C {p:int,
q:string}, be a structural subtype of a record type, say {p:int} or
{q:string} or {p:int, q:string}, you cannot do the reverse. {p:int,
q:string} is not a subtype of C.
Evolution is constrained when you want to inherit from two classes,
which ES4 does not allow. Assume no name conflicts, since those can
arise with nominal and structural types, and namespaces stand ready
to help.
The problem in general is that Bob's classes and Alice's classes were
written without anticipating Carol's combination of the two, but
Carol cannot use MI. Nor can she provide objects that match
structural types. She has to inherit from both Bob's and Alice's
classes.
This inevitably means duplication, peered objects delegating to each
other, even if Bob and Alice used interfaces well (interfaces help,
but bring in other problems and can't be used everywhere).
This also means conflicts are hard to resolve without writing more
(and more verbose) nominally typed code, instead of writing some
untyped code that satisfies like-types or that can be wrapped with
tolerable performance, or some structurally typed code (see below for
an iterator example).
I also don't understand how the goal of applying types to existing ES3 objects can not be achieved with wrap operator and nominal types. I
know there something I am missing here.
You do not want to let any old object masquerade as a nominal type
(class, in this case). Imagine if you had class C { var p:int,
q:string ; final function must_not_override() {...} }. Would {p:42,
q:"hi", must_not_override: function() "rm -rf /"} be safe to pass off
as 'like C'? How about as 'wrap C'?
Remember, the users of C as a type annotation are not interested in
the details of C's implementation (even if final; or if not final, of
a subtype of C coming in as-a C). They care about behavior, in
general. But at the limit, and in essence (i.e., what the type
means), they really want a type named C, and nothing but that type
(or of course a subtype, if C is not final).
Ok, ignore final classes and methods, say you are willing to special
case (I'm kidding, but play along ;-). If you let untyped objects
wrap into nominal types, you've killed another important use-case.
Say you want to return instances of a type that no one can construct
except your code. You don't want anyone wrapping some alien object.
You really need your own instances and only those instances. Think of
those capability objects created by private constructors that David
Teller was asking about the other week (thread head).
There are other use-cases that want nominal types to have this
"branded type" integrity property, e.g. hybrid information flow,
which must see all the subtypes in the system in order to analyze all
effects along implicit flows.
But just for the sake of argument, who cares? Let's say you are hell-
bent on minimizing even if it undermines your system's security (not
that this has ever happened before :-P). If you thus reduce the
integrity of classes until they are named record types, and subtype
judgments are record-width subtype judgments decoupled from the
class's extends clause, then you've reinvented structural types. But
look at the costs.
While like and wrap can be handy for dealing with untyped objects,
and efficient when the actual value passing through the annotation is
well-typed, if you want to avoid the potential costs of like (a deep
shape-test) and wrap (a deep read and write barrier), you really do
want to allow a type with C's structure, not C's name -- you want
structural types for the actual arguments flowing in, not like or
wrap types.
The IteratorType is an example. You don't have to extend a built-in
class, or implement a built-in interface, to make an iterator. All
you need is a next method of the right type:
type IteratorType.<T> = { next: function (): T };
You can implement iterators using object initialisers. From the RI's
builtins/Map.es (with a bug-fix not yet in the monotone source):
helper function iterate.<T>(f: function(*,*,*):*) {
let a = [] : [T];
informative::allElements( tbl, limit, function (k,v) { f
(a,k,v) } ); let i = 0; return { next: function () : T { if (i === a.length) throw iterator::StopIteration; return a[i++]; } } : IteratorType.<T>; }
You don't need classes, nor should you. You can retrofit and migrate,
starting with wrap (or like if in the same trust domain), and moving
to annotate object initialisers over time.
To restate the problems above as tersely as possible:
- Class integrity means that no one can forge an instance using an
untyped object, even with wrap. - You shouldn't have to use wrap, it costs enough that it must be
explicit, and not mandatory for all cases where untyped meets typed. - Other cases where untyped meets typed can use like.
- Structurally typing untyped code is lightweight compared to
subclassing and implementing nominal types.
Modula 3 had branding for making nominal types from structural types,
but going the other way, "unbranding" a nominal type to get a
structural type, has no precedent I know of, probably because it
undermines the integrity property that motivates branding or nominal
typing in the first place. A class C can mean many things, depending
on its members, finality, etc. But its meaning is bound to its name,
not its structure.
Does this clear things up?
Various interested parties favored something like the json2.js API already, and I think everyone will rally round it and beat on it, to make sure it has the right usability and knobs. I'm hopeful.
+1 from me. One request: When a filter function is provided to JSON.parse, I would like the filter to be called with |this| defined to be the root object that is being created by the parsed JSON text. Having a reference to the created root object can be useful for some forms of filters such as reference resolvers.
The problem in general is that Bob's classes and Alice's classes were written without anticipating Carol's combination of the two, but Carol cannot use MI. Nor can she provide objects that match structural types. She has to inherit from both Bob's and Alice's classes.
Was multiple inheritance discussed for inclusion in ES4? I am aware of the general arguments against it, but I was wondering if had been considered or if there are technical aspects of ES4 that preclude it.
Does this clear things up?
Yes, that certainly helps me to understand the rationale. Thanks for being so willing to answer questions about ES4 issues.
Kris
On Nov 14, 2007, at 5:34 PM, Brendan Eich wrote:
Things I didn't see:
What about standardizing the de facto <!-- comment syntax that is necessary for web compatibility?
Please write the spec ;-). Don't forget the --> mess.
We did talk about this at some point. I'm not sure everyone is aware
of this web compatibility constraint, so I'll describe it briefly and
incompletely.
When I embedded JS in HTML in 1995, I had a backward compatibility
problem: Netscape 1.x and every other browser would see the inline
script tag's content, as marked up text. So I took advantage of a
content model that HTML-as-it-is inherited from SGML, CDATA, and made
the script tag's content model be CDATA. This means that < does not
need to be spelled < -- thank goodness -- but it also allows SGML
comments (or the approximation of them supported by HTML browsers) to
wrap the entire inline content of <script>:
<script> <!-- hide from old browsers for (i = 0; i < N; i++) do_stuff(i); // --> </script>
This two-way comment hiding hack works because JS in browsers (not
necessarily in other embeddings, see below) treats <!-- as the
starting delimiter of a comment till end of line. I required
programmers to use a JS one-line comment to hide the closing -->.
Unfortunately, and I'm going from memory so this may be off slightly,
IE at least does not require --> to be commented in JS. This is bad
in general since --> looks like post-decrement immediately followed
by greater-than. There's a heuristic that insists on only leading
horizontal whitespace on the line before the -->, and (I may have
this part wrong in particular) nothing after the --> but whitespace.
Postel's Law bites back, so of course by the Web's law of copy and
paste programming (cut and paste, actually), you find <!-- in the
middle of .js files included via <script src="foo.js">. These must be
treated by JS as comment to end of line initiators.
This ends my tale of woe, except that I think embeddings on DVDs and
in firmware that do not see web script should not have to deal with
this nonsense. So I'm not sure a spec for this de-facto web standard
is appropriate in ES4 or any normative core language spec. Of course,
the copy/paste argument applies across embedding domains, but not so
much. What do you think?
On Nov 15, 2007, at 9:17 AM, Kris Zyp wrote:
+1 from me. One request: When a filter function is provided to
JSON.parse, I would like the filter to be called with |this|
defined to be the root object that is being created by the parsed
JSON text. Having a reference to the created root object can be
useful for some forms of filters such as reference resolvers.
Interesting -- a short example would help sell this, I bet.
The problem in general is that Bob's classes and Alice's classes
were written without anticipating Carol's combination of the two,
but Carol cannot use MI. Nor can she provide objects that match
structural types. She has to inherit from both Bob's and Alice's
classes. Was multiple inheritance discussed for inclusion in ES4? I am aware
of the general arguments against it, but I was wondering if had
been considered or if there are technical aspects of ES4 that
preclude it.
We passed over MI without any regrets, for the general reasons you
give. Also, even with MI, classes are not as flexible as structural
types, as I've pointed out. They're different beasts, with different
as well as some overlapping use-cases from structural types.
Does this clear things up?
Yes, that certainly helps me to understand the rationale. Thanks
for being so willing to answer questions about ES4 issues.
No problem.
On Nov 14, 2007, at 11:56 PM, Brendan Eich wrote:
Modula 3 had branding for making nominal types from structural
types, but going the other way, "unbranding" a nominal type to get
a structural type, has no precedent I know of,
Shaver pointed to generic metaprogramming using C++ templates, which
is close -- but of course the C++ static type system must make sense
of everything before runtime, and you can't forge an instance of a C+
- class. Important safety tip!
On Nov 15, 2007 9:58 AM, Brendan Eich <brendan at mozilla.org> wrote:
On Nov 15, 2007, at 9:17 AM, Kris Zyp wrote:
+1 from me. One request: When a filter function is provided to JSON.parse, I would like the filter to be called with |this| defined to be the root object that is being created by the parsed Interesting -- a short example would help sell this, I bet.
Example: one could write a reference resolving filter that could handle JSON referencing schemes for circular references and such. obj = JSON.parse('{child:{$ref:"this"}}',function(k,v) { return k=='$ref' ? eval(v) : v; // non-eval resolvers could be used, but this is succint }); obj.child==obj -> true
This is really not a very big deal, it is easy to walk the tree after parse to accomplish the same thing, I just figured that when a filter is called, |this| must be something, why not something of value?
My expanded ideas on JSON referencing (not really relevant to ES4, just FYI): www.json.com/2007/10/19/json-referencing-proposal-and-library although there are plenty of other ways to do it. Kris
On Nov 14, 2007, at 5:34 PM, Brendan Eich wrote:
On Nov 14, 2007, at 2:03 PM, Maciej Stachowiak wrote:
Conversions: "In addition, any value in the language converts to a member of AnyBoolean", but the conversions specified are all to the more specific "boolean" type, so perhaps it should be expressed that way to avoid confusion.
I thought this too, when reviewing this section. I think this is an open issue, but I can't find a ticket for it.
Found thanks to Lars:
On 14/11/2007, Maciej Stachowiak <mjs at apple.com> wrote:
Section III.
Syntax: The new non-contextual keywords, and the resulting need to specify dialect out of band, are a problem. I'll have more to say about compatibility under separate cover.
The model with version and e4x arguments in the Content-Type for changing JS parsing has been used by moz already. Can we hear their experience with regard to this compatibility problem?
Generally I think opt-in versioning such as this is the best you can get for compatibility. Perhaps it would be wise to have a method that is not external to the script as well, though, but I fail to see how that could work compatibly in current ES3 only implementations.
- The RegExp change - is this really a bug fix? It's likely that this is not a big compatibility issue (Safari's ES3 implementation had things the proposed ES4 way for some time) but I think ES3's approach may be more performance and generating a new object every time does not seem especially helpful.
It's a SpiderMonkey+ES3 fix, as I recall. The main problem with the ES3 spec is that developers don't expect lastIndex to persist when they evaluate the literal a second time, but also other mutable properties. Real world code breaks because of this.
Is there any significant implementation that anyone would claim is 100% free of ECMAScript 3 compliance bugs?
I don't even think you would get 100% compliance if you counted all engines taken together - I think there's issues where no engine really "gets it right", or for that matter can afford to "get it right". At least, no browser hosted engine.
Section IV.
Classes: If any of the new type system is worthwhile, surely this is. The impedance mismatch between the class model used by most OO languages and by specifications like the DOM, and ES3's prototype model, is needlessly confusing to authors. So I approve of adding classes in a reasonable and tasteful way.
<somewhat offtopic>
For compatibility reasons with the ES3 bindings for the DOM, I think the train has already left with crafting the DOM bindings fully into the ES4 model. The way the ES bindings work is incompatible with both ES3 prototype object hierarchies and ES4 classes/interfaces. In particular we have multiple parallel interfaces, each of which developers expect to be an object inherited from using the prototype scheme. The bindings are incompatible with to ES4 interfaces and classes in that ES4 interfaces from what I understand don't carry implementation, aren't runtime objects nor can they present prototype objects (obviously, since they aren't runtime objects).
I can only see two ways to solve this problem:
- Add a multiple inheritance scheme to ES4 that works on the prototype delegation system as well as the nominal type system, solving the diamond problem*. --> Severely complicating the object system.
- Remove these multiple parallel interfaces being exposed as run time objects from the ES bindings, allowing a single inheritance hierarchy to be formed from implementing them using ES4 interfaces. --> Making a
conformant DOM.next implementation not be a conformant DOM.current implementation.
- A good resolution mechanism solving the diamond problem can be found in Python and was borrowed by Perl 6, IIRC. </somewhat offtopic>
Literals:
- I am surprised to see a decimal type (a type that is not directly supported in current mainstream hardware) even though generally popular types like single-precision IEEE floating point and 64 bit integers are not present.
I guess it directly addresses one large real world problem - that fifths are inexactly represented in doubles, and there is a large demand for reliable number handling of decimal values for amongst other things money and measurements. I doubt the demand for single floats or 64-bit integers is even close to as large as the demand for accurate handling of common real world values like money.
Section V. Record and array types: Structural types are confusingly similar to yet different from classes. Mostly they offer a subset of class functionality (though reading ahead I did see a few features limited to them).
They do allow for orthogonal type ideas, and I think many ES3 developers will be more comfortable with structural types than with classes and interfaces, because they can add contracts without changing their coding style. Replacing code written for the ES3 system of using closures for privacy and prototypes for inheritance with code written for the ES4 classical inheritance system will require considerably more rethinking one's implementation.
Also, already having prototype-based objects and class-based objects it seems excessive to add yet a third way. I recommend removing them and adding any features that are sorely missed as a result to classes.
Well, structural types doesn't really affect the object types, do they? AIUI structural types are part of the contract system, not the inheritance model. The object is still just a plain object, it just has the given constraints.
Type definitions: Seeing the example of a type definition for a record makes this feature seem even more redundant with classes.
Data Types: If structural types cannot be recursive, then one of the canonical applications of record-like types, the linked list, cannot be implemented this way. I assume it can be with classes. Yet another reason to fold any interesting record features into classes.
Again, the difference is one of contract versus implementation. Structural types cannot provide implementation, they can only provide constraints.
Nullability: Are non-nullable types really worth it? I am not sure. Does any other explicit type system for a dynamic OO language have such a concept? The whitepaper says that "the ability to store null is occasionally the source of run-time errors" but will not dynamic- checking result in runtime errors anyway when assigning null to a non- nullable variable (except in strict mode)?
It's a very desired distinction for at least library writers. Getting early detection of this is very good for both code correctness and due to possible performance improvements if the engine optimises it.
Section VII.
Type annotations and type checking: This section implies that type annotations are not at all being added for performance reasons and may indeed be harmful to performance. Wow! Seriously? I think runtime assertions are interesting when debugging but I do would not want them happening for every assignment statement in a release build of my C++ code. I am not sure why ECMAScript programmers would want that. Later this section says "it is plausible" that typed programs will run faster and not slower with enough analysis, but this issue seems far too crucial to take such a blase attitude. Unless we can show that type annotations won't cause a performance hit in practice, and in particular give a convincing argument that the relevant analysis can be done with reasonable speed and without introducing an ahead-of-time compile phase, then it is irresponsible to include type annotations as currently designed. I am willing to believe that this is the case, but I cannot sign on to an attitude that we don't care if typed programs get faster or slower. Nor am I willing to take experience based on ahead-of-time compilers as definitive.
Most of the benefit from type annotations and type checking can be gotten through a good enough compiler even for ES3 code, so I think the performance side of the issue, while still important, is not at all as important as being able to put guarantees for correctness.
Section IX.
Expression closures: I actually find the examples hard to follow given my expectation of ES3-like syntax. I think this may actually be syntactic salt.
Mostly it's a question of the code being perceived as slightly off if you're used to the ES3 function expression syntax only. I still think the syntax is a bit heavy, but it's pretty neat to have if you're from FP background.
Destructuring assignment and binding: I grudgingly accept that this sort of construct has been proven in the context of Python and Perl.
It's sugar and honey to me:)
Slicing: This one I mildly object to. Array/String slicing is not, to my knowledge, particularly common in ECMAScript code of today. I am dubious that it merits its own operator syntax.
Seems like an innocent enough extension to me, and for some of the uses, it should definitely allow engine to perform it faster than if the developers had to code the equivalent functionality using only ES3.
Thanks for your feedback! Two additional comments...
"Any": The spec explains vaguely that the "any" type is not identical to the union (null, undefined, Object). How is it different? Is the difference observable to ES4 programs or is it purely a matter internal to the spec (in which case the difference is not relevant)?
The essential difference is in strict mode, where a variable of type "Any" can be passed to a context expecting a more specific type (say int), and "Any" gets implicitly downcast. In contrast, passing "(null, undefined, Object)" or even "Object" to somewhere expecting an int would be a verify-time error.
"switch type" statement: I guess this beats switching on typeof, but is it really significantly better than a series of "if" statements using the "is" operator?
Well, you'd need a bunch of cast operations too, at least in strict mode, and it gets a little ugly/verbose.
Hello ES4 fans,
I have now read the recently posted whitepaper. I marked up my printed
copy with many comments in the margins, and I am sharing them with the
list now.
Please note that this does not constitute an official Apple position,
just some personal off-the-cuff opinions. I have discussed the
proposal with some of my colleagues, including Geoff Garen who
attended the recent f2f, but we have not figured out a consensus
overall position or anything. With the disclaimers out of the way,
here are my review comments:
Section I.
Goals: I strongly agree with the stated goals of compatibility and
enabling large software development. I wonder if perhaps performance
should be added as a goal. At the very least we want it to be possible
to achieve performance on par with ES3 engines, and ideally we want to
enable better performance.
Section II.
Programming in the small: "... make the writing and reading of
fragments of code simpler and more effortless." That is somewhat
dubious gramatically, I suggest (with additional style fixes) "make
the reading and writing of code fragments easier."
Portability: This section first it says that the full language must be
supported - subset profiles are not desirable. Then it says that, to
allow ES4 to be practically implementable on small devices and in
hosted environments, certain features, like extensive compile-time
analysis and stack marks cannot be part of the language. Then it says
those features are part of the language, but optional.
I hope the problems here are clear: first, the section plainly
contradicts itself. It argues against subsets and certain classes of
features, and then says the spec includes such features as optional,
thus defining a subset. So that needs to be fixed in the whitepaper.
More significantly, I think this may be an indication that the
language has failed to meet its design goals. My suggestion would be
to remove all optional features (though I could be convinced that
strict mode is a special case).
Section III.
Syntax: The new non-contextual keywords, and the resulting need to
specify dialect out of band, are a problem. I'll have more to say
about compatibility under separate cover.
Behavior:
This section has says that "variation among ES3 implementations
entails a license to specify behavior more precisely for ES4".
However, the example given is a case where behavior among two
implementations was already the same, due to compatibility
considerations. I actually think both convergence on a single behavior
where variation is allowed, and variation that leads to practical
compatibility issues are license to spec more precisely,
The RegExp change - is this really a bug fix? It's likely that this
is not a big compatibility issue (Safari's ES3 implementation had
things the proposed ES4 way for some time) but I think ES3's approach
may be more performance and generating a new object every time does
not seem especially helpful.
Impact: This section talks a lot about incompatibilities between ES4
and ES3, however I think incompatibilities with ES3 as specced are in
themselves almost irrelevant. What matters is incompatibilities with
existing implementations and the content that depends on them. This
section also appears to talk disparagingly about some implementations
prioritizing compatibility over ES3 compliance, implies that any
deviations may be due to "inadequate engineering practices", and
implies that only "some" implementations are not compatible with ES3.
Is there any significant implementation that anyone would claim is
100% free of ECMAScript 3 compliance bugs? I doubt it, and so I think
we should make this section less judgmental in tone.
The web: Here especially, the actual concern is real-world
compatibility, not compatibility with the ES4 spec. Furthermore, it
completely ignores forward compatibility (the ability to serve ES4 to
older browsers that do not support it). It implies that this is just
an issue of aligning the timing of implementations. Ignoring for the
moment how impractical it is to expect multiple implementations to
roll out major new features in tandem, I note that there were similar
theories behind XHTML, XSL, XHTML 2, and many other technologies that
have largely failed to replace their predecessors. Again, I'll say
more about compatibility (and in particular how the WHATWG approach to
compatibility can be applied to ES4) under separate cover.
Section IV.
Classes: If any of the new type system is worthwhile, surely this is.
The impedance mismatch between the class model used by most OO
languages and by specifications like the DOM, and ES3's prototype
model, is needlessly confusing to authors. So I approve of adding
classes in a reasonable and tasteful way.
Dynamic properties: the fact that the "dynamic" behavior is not
inherited makes class inheritence violate the Liskov Substitution
Principle. I think this is a problem. Subclassing should be subtyping
in the LSP sense. I am not sure offhand how to fix this.
Virtual Properties: I wish the keyword for catchall getters and
setters was something other than "meta", which is a vague word that
doesn't mean much. Why not "catchall" or "fallback" or something along
similarly concrete lines? (I realize now upon re-reading my margin
comments that this is supposed to match meta invoke, but there too I
am not sure the relationship is worth the vagueness.)
Wrappers: The whitepaper implies that providing catchall getters and
setters for primitive types and skipping boxing isn't a compatibility
issue. However, it is possible in ES3 to capture an implicit wrapper:
var x; String.prototype.myFunc = function() { this.foo = "foo"; x = this; }; "bar".myFunc();
Prototype hacking allows you to observe identity of the temporary
wrappers, save them for later, and store properties. Perhaps there is
evidence that practices relying on techniques like this are
exceedingly uncommon (I'd certainly believe it), if so it should be
cited.
Literals:
supported in current mainstream hardware) even though generally
popular types like single-precision IEEE floating point and 64 bit
integers are not present.
be performed in double space (requiring constant conversions when
working with int variables), or every operation must check for
overflow and possibly fall back to double space. Even when the final
result cannot overflow, certainly in many expressions the difference
between int and double intermediates can be observed. It seems likely,
then, that math on variables declared int will be slower than math on
variables declared double, which will surely be confusing to
developers. This seems pretty bogus. Is there any case where int math
using the normal operators can actually be efficient? Would it be
plausible to make ints not overflow to double unless there is an
actual double operand involved (in which case int constants would
always need a special suffix, or perhaps can somehow be determined
contextually).
Section V.
Record and array types: Structural types are confusingly similar to
yet different from classes. Mostly they offer a subset of class
functionality (though reading ahead I did see a few features limited
to them). Also, already having prototype-based objects and class-based
objects it seems excessive to add yet a third way. I recommend
removing them and adding any features that are sorely missed as a
result to classes.
"Any": The spec explains vaguely that the "any" type is not identical
to the union (null, undefined, Object). How is it different? Is the
difference observable to ES4 programs or is it purely a matter
internal to the spec (in which case the difference is not relevant)?
Type definitions: Seeing the example of a type definition for a record
makes this feature seem even more redundant with classes.
Data Types: If structural types cannot be recursive, then one of the
canonical applications of record-like types, the linked list, cannot
be implemented this way. I assume it can be with classes. Yet another
reason to fold any interesting record features into classes.
Nullability: Are non-nullable types really worth it? I am not sure.
Does any other explicit type system for a dynamic OO language have
such a concept? The whitepaper says that "the ability to store null is
occasionally the source of run-time errors" but will not dynamic- checking result in runtime errors anyway when assigning null to a non- nullable variable (except in strict mode)?
"wrap": Seems like a version of this feature and/or "like" founded on
classes would work just as well.
Conversions: "In addition, any value in the language converts to a
member of AnyBoolean", but the conversions specified are all to the
more specific "boolean" type, so perhaps it should be expressed that
way to avoid confusion.
Section VI.
Predefined namespaces: ES4 predefines and automatically opens the
ES4 namespace. What will happen in ES5 (or ES4.1 or whatever)?
Will they still name the primary namespace ES4? Will it have
ES5 instead? Will it have both? I don't care that much about the
specifics as long as this has been thought through.
Bindings: The sheer number of constructs that bind names is a little
scary. I count 16 in the list. I don't think anyone has raised the
paucity of binding constructs as a critical flaw in ES3. Are all these
different constructs really necessary?
Bonding objects and scopes: It seems like introducing lexical block
scopes makes things more challenging for online implementations.
Creating a dynamic scope object per block scope is clearly
unacceptable, but more work may be needed to build a per-function
symbol table that can properly accomodate block scope. Is block scope
worth it? Yes, "var" is a little weird, but having both "var" and
"let" increases conceptual footprint and may overall lead to more
author confusion.
package: Now that I have learned more about them, I think that
exposing packages and namespaces as separate user-level concepts is
confusing. Let's get this down to a single concept that developers
have to learn. Namespaces can just have a paired internal namespace
implicitly, I do not think it is helpful to give the public/internal
pair a special different name.
let, let const: Are expression let and block let really that useful,
other than to make old-school Lisp/Scheme hackers smile? To
programmers mainly used to imperative paradigms I think these will
come off as syntactic salt. See also my previous comments about
whether lexical block scope is worth adding to the language at all.
Program units:
syntax? Why not just allow "use unit" at top level, and implicitly
make each file (or in the browser context each inline script) a unit?
going to be confusing to authors. Seriously, can anyone explain in one
sentence of 12 words or less how Joe Random Developers will decide
whether to use a namespace, import a package, or use a unit? Can we
get this down to only one kind of thing that needs to be mentioned in
the syntax? This would be a big win in reducing conceptual footprint.
Section VII.
Versioning: I am suspicious of versioning mechanisms, especially big
giant switch versioning. Is there any use of ECMASCRIPT_VERSION
that is not better handled by feature testing? (Maybe there is and I
am not thinking of it.)
Type annotations and type checking: This section implies that type
annotations are not at all being added for performance reasons and may
indeed be harmful to performance. Wow! Seriously? I think runtime
assertions are interesting when debugging but I do would not want them
happening for every assignment statement in a release build of my C++
code. I am not sure why ECMAScript programmers would want that. Later
this section says "it is plausible" that typed programs will run
faster and not slower with enough analysis, but this issue seems far
too crucial to take such a blase attitude. Unless we can show that
type annotations won't cause a performance hit in practice, and in
particular give a convincing argument that the relevant analysis can
be done with reasonable speed and without introducing an ahead-of-time
compile phase, then it is irresponsible to include type annotations as
currently designed. I am willing to believe that this is the case, but
I cannot sign on to an attitude that we don't care if typed programs
get faster or slower. Nor am I willing to take experience based on
ahead-of-time compilers as definitive.
Pragmas: The "use decimal" pragma highlights how much complexity there
is to the decimal type. Seriously, is it worth it? Is the problems it
solves really that common?
"for each" statement: This seems like a convenient piece of syntactic
sugar.
Generators: Do ordinary programmers really understand coroutine
control flow? Is this really a significantly better paradigm than
passing a visitor function? Not really convinced in this one yet.
Operator overloading through global multimethods: Overloading? Yikes.
Seems complicated. Aren't we worried that this could make the common
case of existing untyped code slower than it is already?
Tail calls:
control stack" means. Are recursive calls allowed to accumulate other
kinds of space (in which case the usefulness of the requirement is
dubious)? Do functions that may be implemented in native code count
(so for instance if you eval an expression that calls your function in
tail position repeatedly, does the requirement apply?)
un-abstract control structures to consumption of control stack space,
among other things." This sentence seems to be buggy and has triggered
a parse error in my brain.
language.
"this": The most common reason that I know of for trying to copy this
into a variable is for lexically nested functions that are set as
event listeners or similar, and not called immediately by name. So I
don't think the this-passing feature actually addresses the common
likely use-case for such a thing, and so may be more confusing than
helpful.
"eval" operator and the "eval" function: This seems like a good
approach to sanitizing eval. Perhaps it should be highlighted that
splitting the eval function and eval operator is a potential
performance benefit through opening significant new optimization
opportunities.
arguments: It seems strange to both deprecate a feature and improve it
at the same time.
"typeof" operator: I think it's been decided to back out the typeof
"null" change so this may as well be dropped from the whitepaper.
Section VIII.
Strict:
programs at all, except to reject those that do not pass the checks.
Otherwise, since strict mode is optional, this risks interop issues.
So I'm curious what the eval detail is. Perhaps strict mode could
remove the eval operator and allow only the eval function, with some
suitably named version made available ahead of time, if the difference
is just removing local eval behavior.
like it could create the same kinds of problems we see today with
content that is served as application/xhtml+xml to some browsers and
text/html to others. It's not infrequent to see such content break
only in the browsers that really support XML, due to sloppy testing of
changes and the fact that the 78% browser doesn't support XHTML.
Verification:
be done to the exact same program in standard mode?
Section IX.
"switch type" statement: I guess this beats switching on typeof, but
is it really significantly better than a series of "if" statements
using the "is" operator?
Expression closures: I actually find the examples hard to follow given
my expectation of ES3-like syntax. I think this may actually be
syntactic salt.
Array comprehensions: This seems pretty sugary to me but this kind of
syntax has proven useful for typical developers using Python.
Destructuring assignment and binding: I grudgingly accept that this
sort of construct has been proven in the context of Python and Perl.
"type": Are runtime meta-objects representing types ruly necessary?
What are they good for?
Slicing: This one I mildly object to. Array/String slicing is not, to
my knowledge, particularly common in ECMAScript code of today. I am
dubious that it merits its own operator syntax.
Semicolon insertion: I'd like more detail on the compatibility of the
return change. The do-while change adopts de facto reality and so is
good.
Trailing commas: Good to standardize this de facto extension.
Section X.
Map: Long overdue to have a real hashtable type.
Early binding, static type checking, and predictable behavior with
"intrinsic": Perhaps it should be highlighted more that this is a
potential significant performance improvement.
Reflection: This feature seems like it could be complex to implement
and potentially unnecessary for small implementations. I note that
J2ME omits reflection, which we can perhaps take as a sign that it is
not suitable for small implementations.
ControlInspector: I think an interface that's meant for debuggers and
similar tools, and not implementable in all interesting contexts, does
not need to be standardized. Better than having an optional feature.
JSON: Sounds good.
DontEnum: Overloading a getter to sometimes also be a setter seems to
be in poor taste. (1) It's confusing. (2) It makes it impossible to
separately feature-test for existence of the setter. I suggest adding
setPropertyIsEnumerable instead. Why this design choice? Also: can
built-in properties that are naturally DontEnum be made enumerable?
That seems like annoying additional complexity?
Math: I'm surprised to learn that the numeric value 4 is distinct in
int and double types, and yet int math still must (effectively) be
done in double space. This seems bad for performance all around. If
ints are to be a distinct type, then integer math should always be
done in int space.
uint-specific operations: This is syntactically ugly. Why can't
integer math just always work this way? Also, why only uint versions?
Surely it is desirable to do efficient math on signed integers as
well. Also, bitops already happen in integer math space, thus type- specific versions should not be necessary since no floating point
conversion will need to occur if both operands of ^ or & are
statically typed as int or uint.
Things I didn't see:
What about standardizing the de facto <!-- comment syntax that is
necessary for web compatibility?
Hello ES4 fans, I have now read the recently posted whitepaper. I marked up my printed copy with many comments in the margins, and I am sharing them with the list now. Please note that this does not constitute an official Apple position, just some personal off-the-cuff opinions. I have discussed the proposal with some of my colleagues, including Geoff Garen who attended the recent f2f, but we have not figured out a consensus overall position or anything. With the disclaimers out of the way, here are my review comments: Section I. Goals: I strongly agree with the stated goals of compatibility and enabling large software development. I wonder if perhaps performance should be added as a goal. At the very least we want it to be possible to achieve performance on par with ES3 engines, and ideally we want to enable better performance. Section II. Programming in the small: "... make the writing and reading of fragments of code simpler and more effortless." That is somewhat dubious gramatically, I suggest (with additional style fixes) "make the reading and writing of code fragments easier." Portability: This section first it says that the full language must be supported - subset profiles are not desirable. Then it says that, to allow ES4 to be practically implementable on small devices and in hosted environments, certain features, like extensive compile-time analysis and stack marks cannot be part of the language. Then it says those features are part of the language, but optional. I hope the problems here are clear: first, the section plainly contradicts itself. It argues against subsets and certain classes of features, and then says the spec includes such features as optional, thus defining a subset. So that needs to be fixed in the whitepaper. More significantly, I think this may be an indication that the language has failed to meet its design goals. My suggestion would be to remove all optional features (though I could be convinced that strict mode is a special case). Section III. Syntax: The new non-contextual keywords, and the resulting need to specify dialect out of band, are a problem. I'll have more to say about compatibility under separate cover. Behavior: - This section has says that "variation among ES3 implementations entails a license to specify behavior more precisely for ES4". However, the example given is a case where behavior among two implementations was already the same, due to compatibility considerations. I actually think both convergence on a single behavior where variation is allowed, and variation that leads to practical compatibility issues are license to spec more precisely, - The RegExp change - is this really a bug fix? It's likely that this is not a big compatibility issue (Safari's ES3 implementation had things the proposed ES4 way for some time) but I think ES3's approach may be more performance and generating a new object every time does not seem especially helpful. Impact: This section talks a lot about incompatibilities between ES4 and ES3, however I think incompatibilities with ES3 as specced are in themselves almost irrelevant. What matters is incompatibilities with existing implementations and the content that depends on them. This section also appears to talk disparagingly about some implementations prioritizing compatibility over ES3 compliance, implies that any deviations may be due to "inadequate engineering practices", and implies that only "some" implementations are not compatible with ES3. Is there any significant implementation that anyone would claim is 100% free of ECMAScript 3 compliance bugs? I doubt it, and so I think we should make this section less judgmental in tone. The web: Here especially, the actual concern is real-world compatibility, not compatibility with the ES4 spec. Furthermore, it completely ignores forward compatibility (the ability to serve ES4 to older browsers that do not support it). It implies that this is just an issue of aligning the timing of implementations. Ignoring for the moment how impractical it is to expect multiple implementations to roll out major new features in tandem, I note that there were similar theories behind XHTML, XSL, XHTML 2, and many other technologies that have largely failed to replace their predecessors. Again, I'll say more about compatibility (and in particular how the WHATWG approach to compatibility can be applied to ES4) under separate cover. Section IV. Classes: If any of the new type system is worthwhile, surely this is. The impedance mismatch between the class model used by most OO languages and by specifications like the DOM, and ES3's prototype model, is needlessly confusing to authors. So I approve of adding classes in a reasonable and tasteful way. Dynamic properties: the fact that the "dynamic" behavior is not inherited makes class inheritence violate the Liskov Substitution Principle. I think this is a problem. Subclassing should be subtyping in the LSP sense. I am not sure offhand how to fix this. Virtual Properties: I wish the keyword for catchall getters and setters was something other than "meta", which is a vague word that doesn't mean much. Why not "catchall" or "fallback" or something along similarly concrete lines? (I realize now upon re-reading my margin comments that this is supposed to match meta invoke, but there too I am not sure the relationship is worth the vagueness.) Wrappers: The whitepaper implies that providing catchall getters and setters for primitive types and skipping boxing isn't a compatibility issue. However, it is possible in ES3 to capture an implicit wrapper: var x; String.prototype.myFunc = function() { this.foo = "foo"; x = this; }; "bar".myFunc(); Prototype hacking allows you to observe identity of the temporary wrappers, save them for later, and store properties. Perhaps there is evidence that practices relying on techniques like this are exceedingly uncommon (I'd certainly believe it), if so it should be cited. Literals: - I am surprised to see a decimal type (a type that is not directly supported in current mainstream hardware) even though generally popular types like single-precision IEEE floating point and 64 bit integers are not present. - Since ints/uints overflow to doubles, then either all int math must be performed in double space (requiring constant conversions when working with int variables), or every operation must check for overflow and possibly fall back to double space. Even when the final result cannot overflow, certainly in many expressions the difference between int and double intermediates can be observed. It seems likely, then, that math on variables declared int will be slower than math on variables declared double, which will surely be confusing to developers. This seems pretty bogus. Is there any case where int math using the normal operators can actually be efficient? Would it be plausible to make ints *not* overflow to double unless there is an actual double operand involved (in which case int constants would always need a special suffix, or perhaps can somehow be determined contextually). Section V. Record and array types: Structural types are confusingly similar to yet different from classes. Mostly they offer a subset of class functionality (though reading ahead I did see a few features limited to them). Also, already having prototype-based objects and class-based objects it seems excessive to add yet a third way. I recommend removing them and adding any features that are sorely missed as a result to classes. "Any": The spec explains vaguely that the "any" type is not identical to the union (null, undefined, Object). How is it different? Is the difference observable to ES4 programs or is it purely a matter internal to the spec (in which case the difference is not relevant)? Type definitions: Seeing the example of a type definition for a record makes this feature seem even more redundant with classes. Data Types: If structural types cannot be recursive, then one of the canonical applications of record-like types, the linked list, cannot be implemented this way. I assume it can be with classes. Yet another reason to fold any interesting record features into classes. Nullability: Are non-nullable types really worth it? I am not sure. Does any other explicit type system for a dynamic OO language have such a concept? The whitepaper says that "the ability to store null is occasionally the source of run-time errors" but will not dynamic- checking result in runtime errors anyway when assigning null to a non- nullable variable (except in strict mode)? "wrap": Seems like a version of this feature and/or "like" founded on classes would work just as well. Conversions: "In addition, any value in the language converts to a member of AnyBoolean", but the conversions specified are all to the more specific "boolean" type, so perhaps it should be expressed that way to avoid confusion. Section VI. Predefined namespaces: ES4 predefines and automatically opens the __ES4__ namespace. What will happen in ES5 (or ES4.1 or whatever)? Will they still name the primary namespace __ES4__? Will it have __ES5__ instead? Will it have both? I don't care that much about the specifics as long as this has been thought through. Bindings: The sheer number of constructs that bind names is a little scary. I count 16 in the list. I don't think anyone has raised the paucity of binding constructs as a critical flaw in ES3. Are all these different constructs really necessary? Bonding objects and scopes: It seems like introducing lexical block scopes makes things more challenging for online implementations. Creating a dynamic scope object per block scope is clearly unacceptable, but more work may be needed to build a per-function symbol table that can properly accomodate block scope. Is block scope worth it? Yes, "var" is a little weird, but having both "var" and "let" increases conceptual footprint and may overall lead to more author confusion. package: Now that I have learned more about them, I think that exposing packages and namespaces as separate user-level concepts is confusing. Let's get this down to a single concept that developers have to learn. Namespaces can just have a paired internal namespace implicitly, I do not think it is helpful to give the public/internal pair a special different name. let, let const: Are expression let and block let really that useful, other than to make old-school Lisp/Scheme hackers smile? To programmers mainly used to imperative paradigms I think these will come off as syntactic salt. See also my previous comments about whether lexical block scope is worth adding to the language at all. Program units: - Is there any need for the concept of "unit" to be exposed in the syntax? Why not just allow "use unit" at top level, and implicitly make each file (or in the browser context each inline script) a unit? - I think the difference between using units and importing packages is going to be confusing to authors. Seriously, can anyone explain in one sentence of 12 words or less how Joe Random Developers will decide whether to use a namespace, import a package, or use a unit? Can we get this down to only one kind of thing that needs to be mentioned in the syntax? This would be a big win in reducing conceptual footprint. Section VII. Versioning: I am suspicious of versioning mechanisms, especially big giant switch versioning. Is there any use of __ECMASCRIPT_VERSION__ that is not better handled by feature testing? (Maybe there is and I am not thinking of it.) Type annotations and type checking: This section implies that type annotations are not at all being added for performance reasons and may indeed be harmful to performance. Wow! Seriously? I think runtime assertions are interesting when debugging but I do would not want them happening for every assignment statement in a release build of my C++ code. I am not sure why ECMAScript programmers would want that. Later this section says "it is plausible" that typed programs will run faster and not slower with enough analysis, but this issue seems far too crucial to take such a blase attitude. Unless we can show that type annotations won't cause a performance hit in practice, and in particular give a convincing argument that the relevant analysis can be done with reasonable speed and without introducing an ahead-of-time compile phase, then it is irresponsible to include type annotations as currently designed. I am willing to believe that this is the case, but I cannot sign on to an attitude that we don't care if typed programs get faster or slower. Nor am I willing to take experience based on ahead-of-time compilers as definitive. Pragmas: The "use decimal" pragma highlights how much complexity there is to the decimal type. Seriously, is it worth it? Is the problems it solves really that common? "for each" statement: This seems like a convenient piece of syntactic sugar. Generators: Do ordinary programmers really understand coroutine control flow? Is this really a significantly better paradigm than passing a visitor function? Not really convinced in this one yet. Operator overloading through global multimethods: Overloading? Yikes. Seems complicated. Aren't we worried that this could make the common case of existing untyped code slower than it is already? Tail calls: - The whitepaper doesn't define very precisely what "accumulate control stack" means. Are recursive calls allowed to accumulate other kinds of space (in which case the usefulness of the requirement is dubious)? Do functions that may be implemented in native code count (so for instance if you eval an expression that calls your function in tail position repeatedly, does the requirement apply?) - "The use of procedural abstraction for iteration requires the use of un-abstract control structures to consumption of control stack space, among other things." This sentence seems to be buggy and has triggered a parse error in my brain. - It seems odd to mention goto here, since it is not a feature of the language. "this": The most common reason that I know of for trying to copy this into a variable is for lexically nested functions that are set as event listeners or similar, and not called immediately by name. So I don't think the this-passing feature actually addresses the common likely use-case for such a thing, and so may be more confusing than helpful. "eval" operator and the "eval" function: This seems like a good approach to sanitizing eval. Perhaps it should be highlighted that splitting the eval function and eval operator is a potential performance benefit through opening significant new optimization opportunities. arguments: It seems strange to both deprecate a feature and improve it at the same time. "typeof" operator: I think it's been decided to back out the typeof "null" change so this may as well be dropped from the whitepaper. Section VIII. Strict: - I would strongly prefer if strict mode did not alter behavior of programs at all, except to reject those that do not pass the checks. Otherwise, since strict mode is optional, this risks interop issues. So I'm curious what the eval detail is. Perhaps strict mode could remove the eval operator and allow only the eval function, with some suitably named version made available ahead of time, if the difference is just removing local eval behavior. - I am somewhat concerned about having strict mode at all. It seems like it could create the same kinds of problems we see today with content that is served as application/xhtml+xml to some browsers and text/html to others. It's not infrequent to see such content break only in the browsers that really support XML, due to sloppy testing of changes and the fact that the 78% browser doesn't support XHTML. Verification: - Does strict mode actually allow for any optimizations that couldn't be done to the exact same program in standard mode? Section IX. "switch type" statement: I guess this beats switching on typeof, but is it really significantly better than a series of "if" statements using the "is" operator? Expression closures: I actually find the examples hard to follow given my expectation of ES3-like syntax. I think this may actually be syntactic salt. Array comprehensions: This seems pretty sugary to me but this kind of syntax has proven useful for typical developers using Python. Destructuring assignment and binding: I grudgingly accept that this sort of construct has been proven in the context of Python and Perl. "type": Are runtime meta-objects representing types ruly necessary? What are they good for? Slicing: This one I mildly object to. Array/String slicing is not, to my knowledge, particularly common in ECMAScript code of today. I am dubious that it merits its own operator syntax. Semicolon insertion: I'd like more detail on the compatibility of the return change. The do-while change adopts de facto reality and so is good. Trailing commas: Good to standardize this de facto extension. Section X. Map: Long overdue to have a real hashtable type. Early binding, static type checking, and predictable behavior with "intrinsic": Perhaps it should be highlighted more that this is a potential significant performance improvement. Reflection: This feature seems like it could be complex to implement and potentially unnecessary for small implementations. I note that J2ME omits reflection, which we can perhaps take as a sign that it is not suitable for small implementations. ControlInspector: I think an interface that's meant for debuggers and similar tools, and not implementable in all interesting contexts, does not need to be standardized. Better than having an optional feature. JSON: Sounds good. DontEnum: Overloading a getter to sometimes also be a setter seems to be in poor taste. (1) It's confusing. (2) It makes it impossible to separately feature-test for existence of the setter. I suggest adding setPropertyIsEnumerable instead. Why this design choice? Also: can built-in properties that are naturally DontEnum be made enumerable? That seems like annoying additional complexity? Math: I'm surprised to learn that the numeric value 4 is distinct in int and double types, and yet int math still must (effectively) be done in double space. This seems bad for performance all around. If ints are to be a distinct type, then integer math should always be done in int space. uint-specific operations: This is syntactically ugly. Why can't integer math just always work this way? Also, why only uint versions? Surely it is desirable to do efficient math on signed integers as well. Also, bitops already happen in integer math space, thus type- specific versions should not be necessary since no floating point conversion will need to occur if both operands of ^ or & are statically typed as int or uint. Things I didn't see: What about standardizing the de facto <!-- comment syntax that is necessary for web compatibility?