I'm confused about the design constraints on ES4
On Nov 10, 2007, at 8:00 PM, Mark Miller wrote:
- To prevent these new keywords breaking ES3 compatibility, these keywords are only recognized when a version attribute on the script tag says the script is in ES4.
This has happened before, FYI -- from ES1-3 under various vendor
experimental MIME types or script language attribute values -- and it
continued after ES3 with (at least) JS1.6 and 1.7 (application/
javascript;version=1.7).
- This attribute does create two languages in effect, but not two language implementations, and therefore does not violate the size constraint.
Writing "this attribute does create two languages in effect" seems to
conflate compatible successive versions with one a subset of the
other, and two completely separate programming languages. I'm not
sure what it means, so it's hard to tell what its significance is in
the bulleted list. I'm probably missing something obvious -- little
help?
- The intersection of ES3 and ES4 must be approximately as pleasant a programming language as ES3 currently is.
No, the intersection is "ES3 + reality" + two (known) true
incompatibilities. "ES3 + reality" means the ES3 spec was not
followed (for variously better and worse reasons) and the de-jure
standard needs to be adjusted based on what browsers actually do that
web developers actually count on.
"ES3 + reality" does not mean every last JScript bug, of course,
especially since many have not been emulated by other browsers'
implementations. It does not mean over-specifying what should remain
implementation-dependent (e.g., Date.parse as implemented in various
browsers). It does mean some case-by-case greater specification where
browsers differ in ways that hurt interoperation.
If all this makes sense, does the following follow:
- The ES4 language (as recognized with the version attribute) and the ES3 language must be able to smoothly share an implementation.
Yes.
- Whenever the goals of ES4 can be achieved by removing unneeded features from ES3 (e.g. "with"), rather than adding features, the
shared implementation can smaller, and therefore even better for small devices.
This does not follow if the goals of ES4 include backward
compatibility (they do), since the web depends all over the place on
"with". So the (common) implementation must support ES3 "with", and
adding runtime versioning to make "with" an error in ES4 just adds
code (a tiny amount for the version check) and makes migration harder
(more important that the miniscule version check overhead). See also
reformed with.
Does this make sense so far? Am I missing something?
You wrote about design constraints, but also talked about goals of
ES4 without listing them. The two are not the same. Many design
constraints are imposed by backward compatibility with existing web
content. Goals such as optional static type checking and programming
in the large support on the other hand transcend compatibility
concerns by allowing migration away from old forms toward new ones.
There are no guarantees, but the best way to move the web is to
provide better forms and short (low-cost) migration paths to those
forms, in addition to deprecating the worse forms.
I expect "with" will be with us forever, BTW. My apologies to all!
If backward compatibility requires "with", and the new version does
not remove old forms even when you opt into it explicitly, then the
footprint is minimized. Since removing "with" is a non-starter given
the compatibility constraint, there's no gain in code size by adding
code to remove "with" when the new version is selected -- and there's
no gain in migration costs, since "with" costs are long-ago sunk and
not trivial to unsink. Bad sunk costs, i.e. bugs due to "with"
ambiguities, could be unearthed, but the existence of these
hypothetical bugs in quantity enough to offset the unsinking costs
need to be demonstrated.
Backward compatibility is not simple or static -- it is fuzzy and it
changes as the web changes. There's hope, but the time scales are
long and you can't force change by removing things from browsers. You
have to see competitors take the plunge, and even then you must face
down the rear guard (e.g., removing gopher from Firefox).
Hope this helps,
On Nov 10, 2007 9:10 PM, Brendan Eich <brendan at mozilla.org> wrote:
On Nov 10, 2007, at 8:00 PM, Mark Miller wrote:
[out of order]
- The intersection of ES3 and ES4 must be approximately as pleasant a programming language as ES3 currently is.
No, the intersection is "ES3 + reality" + two (known) true incompatibilities. "ES3 + reality" means the ES3 spec was not followed (for variously better and worse reasons) and the de-jure standard needs to be adjusted based on what browsers actually do that web developers actually count on.
"ES3 + reality" does not mean every last JScript bug, of course, especially since many have not been emulated by other browsers' implementations. It does not mean over-specifying what should remain implementation-dependent (e.g., Date.parse as implemented in various browsers). It does mean some case-by-case greater specification where browsers differ in ways that hurt interoperation.
Yes, I appreciate the point about reality. And well put!
I will use "ES3R" below to mean "ES3 + reality".
- To prevent these new keywords breaking ES3 compatibility, these keywords are only recognized when a version attribute on the script tag says the script is in ES4.
This has happened before, FYI -- from ES1-3 under various vendor experimental MIME types or script language attribute values -- and it continued after ES3 with (at least) JS1.6 and 1.7 (application/javascript;version=1.7).
Yes, good. Specifically the overview doc states:
The mechanism that supplies the dialect information will depend on the
environment. In a web browser the information comes from the MIME
type of the
script or from a version parameter on the SCRIPT tag in the document.14 New
web pages that choose to use ES4 will have to specify the dialect.
It seems we are agreeing on what this text means so far.
- This attribute does create two languages in effect, but not two language implementations, and therefore does not violate the size constraint.
Writing "this attribute does create two languages in effect" seems to conflate compatible successive versions with one a subset of the other, and two completely separate programming languages. I'm not sure what it means, so it's hard to tell what its significance is in the bulleted list. I'm probably missing something obvious -- little help?
I'm simply observing that the version switch explained above changes what language is being recognized. With the switch off (or set to ES3 or ES3R), the new keywords aren't recognized as keywords. With the switch set to ES4, the new keywords are recognized as keywords. What I don't understand is: Why can't this switch change other aspects of the language being recognized, so long as this doesn't result in a larger implementation? For example, if one of the purposes of ES4 can be served by removing a keyword such as "with", can we consider having this same switch (set to ES4) remove it?
I'm using "with" here only as an example, to try to understand the nature of the compatibility constraints on the ES4 process.
- The ES4 language (as recognized with the version attribute) and the ES3 language must be able to smoothly share an implementation.
Yes.
Good.
- Whenever the goals of ES4 can be achieved by removing unneeded features from ES3 (e.g. "with"), rather than adding features, the shared implementation can smaller, and therefore even better for small devices.
This does not follow if the goals of ES4 include backward compatibility (they do), since the web depends all over the place on "with". So the (common) implementation must support ES3 "with", and adding runtime versioning to make "with" an error in ES4 just adds code (a tiny amount for the version check) and makes migration harder (more important that the miniscule version check overhead). See also reformed with.
Let's say, hypothetically, that by removing "with" from ES4 (with the version switch) we could avoid adding something else to ES4. Let's say that the feature we could avoid adding is larger than the code needed to switch "with" on and off. Would the removal of "with" from ES4 then be an option? If not, what compatibility issue does it raise that isn't raised by the addition of the other keywords?
Does this make sense so far? Am I missing something?
You wrote about design constraints, but also talked about goals of ES4 without listing them. The two are not the same. [...]
Indeed. That's exactly why I didn't list them. I want to understand the design constraints first before making proposals regarding goals.
Backward compatibility is not simple or static -- it is fuzzy and it changes as the web changes. There's hope, but the time scales are long and you can't force change by removing things from browsers. You have to see competitors take the plunge, and even then you must face down the rear guard (e.g., removing gopher from Firefox).
Also well put.
On Nov 10, 2007, at 9:52 PM, Mark Miller wrote:
What I don't understand is: Why can't this switch change other aspects of the language being recognized, so long as this doesn't result in a larger implementation?
It could. We considered removing "with" from ES4. But as I noted,
migration costs (on order of unsinking sunk "with" costs, whether
good or bad -- sunk cost fallacy acknowledged!) loomed. Same thinking
about eval, but we do want to make it better under strict mode, and
possibly even under explicit version=4. So we could revisit "with"
too. Thanks for pointing this out.
I'm using "with" here only as an example, to try to understand the nature of the compatibility constraints on the ES4 process.
Compatibility imposes footprint costs, but it also creates a
migration tax proportional to usage, roughly. We're trying not to
blow either footprint budget or migration budget. These are fuzzy,
hard to quantify and reason about; nice judgment is required.
Let's say, hypothetically, that by removing "with" from ES4 (with the version switch) we could avoid adding something else to ES4. Let's say that the feature we could avoid adding is larger than the code needed to switch "with" on and off. Would the removal of "with" from ES4 then be an option? If not, what compatibility issue does it raise that isn't raised by the addition of the other keywords?
We went through this in considering reformed with. The reformed with
proposal is small in parser overhead, but we haven't evaluated
practical implementations' code costs for the runtime semantics
(ignore strict mode; any implementation supporting it has room for
lots of stuff). Our belief (mine anyway) was that you would hurt ES4
adoption-with-migration too much by removing with from version=4 and
you wouldn't save footprint enough; compared to adding reformed-with
support. We should check this belief now that we are getting on to
practical implementations.
Good stuff, keep it coming.
On Nov 10, 2007, at 10:08 PM, Brendan Eich wrote:
On Nov 10, 2007, at 9:52 PM, Mark Miller wrote:
I'm using "with" here only as an example, to try to understand the nature of the compatibility constraints on the ES4 process.
Compatibility imposes footprint costs, but it also creates a migration tax proportional to usage, roughly. We're trying not to blow either footprint budget or migration budget. These are fuzzy, hard to quantify and reason about; nice judgment is required.
I meant to add: we're on a slippery slope once opt-in versioning
means removing stuff if the trade against adding compensating/better
stuff seems to win. Slide a little way down, fix or remove with and
bad-eval (keep good-eval, TBD ;-). Slide further, you are starting to
talk "two languages" and that has brainprint and creeping
compatibility/shared-code hazards we don't like.
But let's target the bad boys first: with, bad-eval. We've tried to
reform both. Recidivism is a risk if we misjudge. ;-)
There are many individual aspects of the the ES4 proposal that I like, many things I don't. All of which I hope to enumerate for this list over time. But overall my main distress is still the size of the language.
At weblogs.mozillazine.org/roadmap/archives/2007/11/es4_news_and_opinion.html you state:
[...] the proposed ECMAScript 4th edition (ES4) grammar is a bit
more than twice
as big as ES3's, counting several ways [...]
This is not out of line given the eight years since ES3, which came less than
three years after ES1.
It is one thing to sigh with despair at other people's tendencies to add features to languages over time. It is another to see this as a virtue. Perhaps EcmaScript 15, eighty years from now, will be 10 times as large as ES4. I would agree that this may happen, as it seems to be happening to Java. But should we look forward to this as a good thing?
Please don't dismiss such "mood of the language" issues. The "fuzzy" points you made in your previous reply to me were quite valuable, even though they are at least as non-objectively "mood of the language"-like as many of the arguments you've dismissed on these grounds. Language size has a real cost. The "brainprint" you refer to, at least for my brain, goes up more than linearly in the size of language.
Fortunately, later in that same paragraph, you say:
And we're being generous with syntactic conveniences,
which desugar to a smaller core language.
From my perspective, this may be very good news! Is this smaller core
language documented anywhere? E, Scheme, Mozart/Oz, and many languages I like are organized as small core/kernel language + syntactic sugar. Bottom-up learners like myself can more easily understand such languages by first absorbing the core language, and then understanding the rest in terms of their expansion to the constructs they've already understood.
From a security perspective, a small core/kernel language is
especially important. Secure computing is best understood as a multi-player game. In order to plan one's moves, one must understand not only what one is able to do, one must also understand the limits on what the other players are able to do. Such arguments, whether formal or informal, are effectively inductions over all the operations available to the other players. Fortunately, with the core + sugar approach, such arguments only need to enumerate the elements of the core.
On Nov 11, 2007, at 10:44 AM, Mark Miller wrote:
Hi Brendan,
There are many individual aspects of the the ES4 proposal that I like, many things I don't. All of which I hope to enumerate for this list over time.
That would be great.
It is one thing to sigh with despair at other people's tendencies to add features to languages over time.
(People tend to do that with all languages, but why despair? It's
either inevitable or good. I'm assuming the philosophy of despair is
something we both try to avoid.)
It is another to see this as a virtue. Perhaps EcmaScript 15, eighty years from now, will be 10 times as large as ES4. I would agree that this may happen, as it seems to be happening to Java. But should we look forward to this as a good thing?
There's not going to be an ES15, and eighty years is way over the
event horizon. The problems JS users face now won't be solved that
slowly in the language. Instead, at the margins at first and
increasing non-linearly, some new runtime (probably only one) will
eat away at all of the open standards implemented in browsers --
unless those standards can evolve quickly.
This is a real threat. Right now major web sites are being paid tons
of money to retool using Silverlight, while others are being paid to
develop Adobe AIR applications. I'm not saying one bad word about the
merits of those closed-standard runtimes, except that they are simply
not open by the same definition as browsers based on web standards
that co-evolve with the browsers are. Even when those other runtimes
embrace and extend web standards, the whole result is proprietary,
single-vendor.
With ES4 and other "post Firefox" web standards (the WHAT-WG Web Apps
1 and Web Forms 2 work, which is supposedly feeding into the re-
chartered W3C HTML Working Group), a combination of browser vendors
(minority market share, the game theory explains everything here --
there's no secret alternate-runtime agenda) and web developers (who
have been traditionally unrepresented in pay-to-play standards
consortia) have tried to evolve open web standards quickly.
This "open standards common" approach is not easy, and it doesn't
follow the traditional model for language design. At best it involves
a high-quality peer group folding the results of experiments into the
emerging standard, factoring to ensure compositionality and a smaller
core than syntax indicates. And these efforts tend to be rolling
standards parties -- nothing is pushed to de-jure status prematurely.
Again, not ideal for interop, but much better than stagnation.
This model for evolving the web is new since 2004, but something like
it was already happening behind the scenes, without being formalized,
when XMLHttpRequest was implemented in other browsers in the early
2000s. There, an informal peer group (most of the people knew their
peers at other browser vendors via standards work) was following the
lead of Microsoft (which gave Java the boot and then had to add XHR
for Outlook's webmail!), but the general principles of incremental,
backward-compatible change prefigured the more explicit goals and
methods of the WHAT-WG. Similar work happened with JavaScript
variants, including Mozilla's and Macromedia's dialects, which fed
forward into Ecma TG1.
You can call this process a bad thing because it's far removed from
what I'll call the Steele/Sussman or Niklaus Wirth language design
model. I don't care to moralize about it at this critical point in
web evolution. I think it's at worst the lesser evil, and it is in
any event inevitable for any of us among the minority-share browser
vendors, and the Ajax web developers who stick to the browser
runtime, who together are trying to make progress without commanding
dominant market power that includes OS auto-updated distribution
channels.
What I've written here seems far removed from language design
principles. It is close to a description of fitness characteristics
and survival threats in the web ecosystem. The principles and harsh
realities have to meet somehow. Speaking for myself (since you
addressed me; I'm not sole or lead ES4 designer), I don't see a
better way to proceed than shared effort in groups such as the Ecma
TG1 group, the WHAT-WG, etc.
Please don't dismiss such "mood of the language" issues. The "fuzzy" points you made in your previous reply to me were quite valuable, even though they are at least as non-objectively "mood of the language"-like as many of the arguments you've dismissed on these grounds.
First, I don't dismiss "mood" arguments, I object to them being
substituted as evidence for categorically distinct claims about
compatibility and other technical properties. Those "break the web"
claims need to be substantiated or withdrawn.
Second, my point about compatibility being fuzzy is not a subjective
argument. It could be quantified, at great expense, and pretty much
only in hindsight. The plain fact, and Firefox and Safari rising in
market share are proof of this, is that web compatibility does not
mean emulating the last (monopoly or near-monopoly) implementation
bug-for-bug. It never has, not when Netscape took over from Mosaic,
not when IE took over Netscape's market share.
Language size has a real cost. The "brainprint" you refer to, at least for my brain, goes up more than linearly in the size of language.
People invoke Scheme when talking about JavaScript. I was originally
drawn to Scheme when Netscape recruited me in early 1995, but there
was less than zero time to use it, and anyway, it was and remains a
teaching and research language. People using it industry have to
procure or write a lot of code to make it usable, and interoperation
is not great (as with C) because the reports intentionally
underspecify. At least with JS we specified more for interop. But I
over-minimized the original design, and ES2 and 3 were growth-
restricted. The result has been a big complexity tax on users of the
language, especially those trying to build robust libraries.
The situation with Scheme today seems relevant to ES4/JS2 in another
respect. R6RS has grown from R5, a lot, and it has sometimes
overspecified capriciously, as Will Clinger and others have pointed
out. This has led to an ERR5RS counter-action. From 100,000 feet, you
might compare what's going on to ES4 vs. "ES3.1" or whatever it might
be called. Closer up, the comparison breaks down badly in my opinion.
Notwithstanding all the differences that prevent easy comparisons, I
think it's fair to say that those of us in TG1 working on ES4 take
the traditional first paragraph of the Scheme reports to heart:
"Programming languages should be designed not by piling feature on
top of feature, but by removing the weaknesses and restrictions that
make additional features appear necessary. Scheme demonstrates that a
very small number of rules for forming expressions, with no
restrictions on how they are composed, suffice to form a practical
and efficient programming language that is flexible enough to support
most of the major programming paradigms in use today."
Compositional, orthogonal features really do mean (if the language
designers got it right) that you don't have to think about order N^2
or N! special case combinations as N grows. ES4 is not being monkey-
patched or otherwise hacked simplistically or carelessly on top of
ES3. I note that kind of language-hacking, when it happens in
standards bodies, is more a hazard of growing by too-small increments
forced through a standards process that dumbs down the technical
thinking and stamps the standard before the dominant vendors have
implemented it fully (this happened with ES3; less so with ES2).
Fortunately, later in that same paragraph, you say:
And we're being generous with syntactic conveniences,
which desugar to a smaller core language.
From my perspective, this may be very good news! Is this smaller core language documented anywhere? E, Scheme, Mozart/Oz, and many languages I like are organized as small core/kernel language + syntactic sugar. Bottom-up learners like myself can more easily understand such languages by first absorbing the core language, and then understanding the rest in terms of their expansion to the constructs they've already understood.
The desugarings I'm thinking of are documented in the wiki (e.g.
destructuring assignment), not so much in the overview. I agree they
should be documented so as to separate the core language -- we're
just getting to spec-writing now, but this is something we'll work
on. The core is still a significantly bigger language than ES3 by a
good bit, however. This is intentional and we had better get it right
before it becomes a standard.
From a security perspective, a small core/kernel language is especially important. Secure computing is best understood as a multi-player game. In order to plan one's moves, one must understand not only what one is able to do, one must also understand the limits on what the other players are able to do. Such arguments, whether formal or informal, are effectively inductions over all the operations available to the other players. Fortunately, with the core + sugar approach, such arguments only need to enumerate the elements of the core.
Yeah, we are hip to this. It's a crucial point. We haven't led with
it, but it's evident in some of the voluminous materials on the wiki.
I've been reading the overview doc on the ES4 proposal. From it, and from the threads that I've been reading, I'm confused about the design constraints on the ES4 process. Are the following statements all accurate:
If all this makes sense, does the following follow:
Does this make sense so far? Am I missing something?
Hi all, I've been reading the overview doc on the ES4 proposal. From it, and from the threads that I've been reading, I'm confused about the design constraints on the ES4 process. Are the following statements all accurate: * We want ES4 to work well on small devices, so size of implementation is a concern. * Therefore, we'd rather require only one language *implementation* rather than two. * The ES4 proposal includes new keywords. * To prevent these new keywords breaking ES3 compatibility, these keywords are only recognized when a version attribute on the script tag says the script is in ES4. * This attribute does create two languages in effect, but not two language implementations, and therefore does not violate the size constraint. * The intersection of ES3 and ES4 must be approximately as pleasant a programming language as ES3 currently is. If all this makes sense, does the following follow: * The ES4 language (as recognized with the version attribute) and the ES3 language must be able to smoothly share an implementation. * Whenever the goals of ES4 can be achieved by removing unneeded features from ES3 (e.g. "with"), rather than adding features, the shared implementation can smaller, and therefore even better for small devices. Does this make sense so far? Am I missing something? -- Text by me above is hereby placed in the public domain Cheers, --MarkM