complexity tax
On 3/26/2008, "Douglas Crockford" <douglas at crockford.com> wrote:
I think this is the specific point of disagreement. Complexity in a language does not necessarily reduce the complexity of programs. I think the opposite may be truer. The difficulties we have had in the development community since
I think this is THE point of prose rather than technique in that I can argue either side. An example is Microsofts LINQ - it is already implemented in .Net 3.5. I can do the same effects using delegates to static IEnumerable in older versions of .Net but it would take me several (dozen) more lines of code. When compiled, the result may be the same, but my implementation may not be standard. (Doing it my own way may allow me to discover something better, but that is another discussion)
I respect both sides: I trust Brendan as the language designer, but I can not get over Dougs fear of The Ghost of Netscape.
Let us take action instead of throwing opinions around: Brendan: What new features that can not be implemented via code constructs now? Doug: Please give concrete example of YUI code that accomplishes something proposed
Then we, as computer SCIENTISTS can calculate the complexity ratio of code vs. language implementation. We could keep on arguing about speed and best specific algorithms after that if you want.
Ric
"Douglas Crockford" wrote:
Brendan Eich wrote:
- competition from other languages in runtimes vying to displace the browser.
I can't make sense of your point about competition.
From my perspective, this is about: . Flash; Silverlight; Java (client side).
Three major approaches requiring large plug-ins to provide a rich environment. Notably all three approaches have languages substantially beyond Javascript. As a toolmaker, I would love to see convergence between Javascript and Flash and a Silverlight language that was powerful enough to also use server side a la Java.
Does such a language need to be as extensive as ES4? Probably not.
Is this more urgent than the topics Douglas mentions? That seems worthy of more discussion.
Best , ~TMSteve
On Mar 26, 2008, at 10:01 AM, Ric Johnson wrote:
Let us take action instead of throwing opinions around: Brendan: What new features that can not be implemented via code constructs now?
This is reductionism, therefore between silly and wrong, but I will
list a few things:
- you can't make read-only properties in ES3;
*you can't make don't-delete properties of plain old objects (only
vars in closures);
-
you can't make objects that cannot be extended;
-
you can't make don't-enum properties in plain old objects;
-
you can't yield from a function to its caller, saving state
automatically in local variables, and send a value back to the
suspended activation of that function.
The last is generators, but let's suppose that we add those. I'm
intentionally leaving out things that are hard to do, but can be
done, with closures (e.g., guaranteeing that |this| binds to a
particular object when a method is called; or guaranteeing that a
particular method will be called by a given name on an object).
Suppose further that you chose the absolute minimal set of additions
to support some semblance of a solution for the first four bullets.
My response is that adding a few meta-methods to flip property
attributes is not thereby "enough", since it is still incredibly
verbose, error-prone, inefficient, and subject to "in the middle of
construction" integrity attacks to build up "classes" or other ES4
features using objects with const/permanent/non-enumerable additions.
Almost no one will write such "classes" out by hand.
I noted years ago to Doug (after a talk at Yahoo! I gave) how a macro
system would help programmers use some primitives better without
adding more surface syntax. He wondered whether we shouldn't do
macros first. Problem is macros are not theoretically worked out yet
-- and that's in the context of Scheme. So rather than starve the
patient further, I argue we should be willing to add syntactic
conveniences as well as syntax for new substance. Macros may come,
and much can be recast in terms of them, and the ecosystem can take
over the business of syntactic extension, to a great extent.
Anyway, I don't think my answer here helps decide what to do.
Opinions and tastes vary about things like syntax, dynamic meta-
programming using fewer primitives vs. declarative new forms that can
be (optionally) statically checked, etc. Some people will always
prefer to use fewer primitives. It seems better to me to grow
JavaScript based on existing "design patterns" used (at some cost) on
top of ES3, and based on solutions from "nearby" languages such as
Python and ActionScript 3 (to take just two examples).
Even this does not mean anything particular about complexity budgets.
I will say this: ES4 has overspent its complexity budget in order to
explore a large design space. It is now shrinking to meet practical
budgets involving Ecma member consensus, prime mover commitments,
fatigue effects, community expectations, and the like. No one working
on ES4 wants it to be like Perl 6. We aim to finish this year. So
you'll see proposals like multimethods getting deferred.
On Wed, Mar 26, 2008 at 8:23 AM, Douglas Crockford <douglas at crockford.com> wrote:
Brendan Eich wrote:
And over-minimizing a language imposes a complexity tax on programmers using it.
That is true.
To decide whether to evolve JS or shrink it, you need only look at two things: 1) problems JS hackers run into every day, which are not solved by more idiomatic functional programming hacks that add cycle and space bloat to their libraries and applications; 2) competition from other languages in runtimes vying to displace the browser.
I think this is the specific point of disagreement. Complexity in a language does not necessarily reduce the complexity of programs.
Not necessarily, but to be specific, simply adding 'private' access modifier would avoid a closure. This is good because its a lot more convenient and a lot clearer.
I think the opposite may
be truer. The difficulties we have had in the development community since 1999 were not due to over-minimization. They were due to features that did not work as expected or reliably over the various brands and versions.
That is true. Implementation differences are significant difficulties. Other difficulties had to do with browser differences/competing technology platforms (<layer>, document.all).
There are also limitations to the language itself that cause problems (no access modifiers, limited typechecking, enumeration). I disagree that limitations in the language did not cause difficulties. I think the opposite is true. I think that developers ran into problems when trying to add to Object.prototype. I think developers have struggled trying to figure out if [some_object] was a function or array. I think that developers exploited closures for all that closures had to offer. Closures are the most useful thing we have because there simply aren't other alternatives available.
Garrett
Brendan,
I admit to being a lurker and a bit of a troll, except the discussions I inspire are actually help me understand. Inso much that the community also learned, mayhaps I am an 'ethical' troll.
On 3/26/2008, "Brendan Eich" <brendan at mozilla.org> wrote:
This is reductionism, therefore between silly and wrong,
Can you explain the above fragment?
- you can't make read-only properties in ES3;
I think I can! Maybe a bad developer could re-define parts of my objects, but is the intention sugar or hacker proof?
*you can't make don't-delete properties of plain old objects (only vars in closures);
Again, I can make it pretty darn hard to remove privledged fields without killing the tree, and I ususally use plain old objects (poo) as data only.
- you can't make objects that cannot be extended;
This may be true. But I think the crazy guy that invented livescript did this to allow the power of prototype and redefine the Object/ Function constructors actually had a good idea: JS is not a CLASSical language.
- you can't yield from a function to its caller, saving state automatically in local variables, and send a value back to the suspended activation of that function.
I actually have a hack that used recursion, a global hash, timers, and closures that effectively did YIELD (and it worked most of the time too!)
It is now shrinking to meet practical budgets involving Ecma member consensus, prime mover commitments,
This is good news that we are coming to consensus, even if it is only for timelines (for now).
Thank you for your considered response!
- Ric
P.S. I would also love to hear from Doug on this, if you are out there!
On Mar 26, 2008, at 4:47 PM, Ric Johnson wrote:
On 3/26/2008, "Brendan Eich" <brendan at mozilla.org> wrote:
This is reductionism, therefore between silly and wrong,
Can you explain the above fragment?
The whole of a language is more than the sum of its parts. The lambda
calculus is enough for computation but no one wants to use only it.
Danny Hillis built a computer from tinkertoys once...
- you can't make read-only properties in ES3;
I think I can!
How?
but is the intention sugar or hacker proof?
Hacker.
*you can't make don't-delete properties of plain old objects (only vars in closures);
Again, I can make it pretty darn hard to remove privledged fields
without killing the tree,
How?
- you can't make objects that cannot be extended;
This may be true. But I think the crazy guy that invented
livescript did this to allow the power of prototype and redefine the Object/ Function constructors actually had a good idea: JS is not a CLASSical language.
That does not solve problems where integrity matters more than it
does for the obvious use-cases I had in mind in favoring mutability.
- you can't yield from a function to its caller, saving state automatically in local variables, and send a value back to the suspended activation of that function.
I actually have a hack that used recursion, a global hash, timers, and closures that effectively did YIELD (and it worked most of the time
too!)
I don't think you read what I wrote: save local variables, resume the
same function activation. Sure, you can implement state machines with
other techniques, which are ugly and hard to use by comparison :-P.
On Wed, 2008-03-26 at 08:23 -0700, Douglas Crockford wrote:
The difficulties we have had in the development community since 1999 were not due to over-minimization. They were due to features that did not work as expected or reliably over the various brands and versions.
In my experience, the main problem with JS was the impossibility of extending it. That is, no notion of libraries and no built-in pre-processor (although reflexivity could be used for similar purposes). A consequence was that any extension deemed important by the developers of a browser had to be bolted-on in non-specified manners.
Now, all the features I see en ES4 are nice (my favorite being the hybrid type system, although type inference is going to be a heck to design and implement) and I can see myself using most of them, but I would have been content with just the addition of libraries and pre-processor.
On Thu, Mar 27, 2008 at 7:06 AM, David Teller <David.Teller at univ-orleans.fr> wrote:
In my experience, the main problem with JS was the impossibility of extending it. That is, no notion of libraries and no built-in pre-processor (although reflexivity could be used for similar purposes). A consequence was that any extension deemed important by the developers of a browser had to be bolted-on in non-specified manners.
I don't quite understand this -- could you give an example of a language that has better support for libraries as part of its language specification? I don't think the C specification includes linking (or even the ABI, though C++ grew an ABI specification later), so it seems to be about at the same point of "get your code into scope somehow". Browsers use <script> for that, and many an AJAX toolkit has added new
capabilities to the environment through just that means.
Mike
I think this is the specific point of disagreement. Complexity in a
language does not necessarily reduce the complexity of programs. I think the
opposite may be truer. The difficulties we have had in the development community
since 1999 were not due to over-minimization. They were due to features that
did not work as expected or reliably over the various brands and versions. I
think that with minimal changes we can significantly improve this language. Most of
the changes I would make would make it simpler, not more complex. And of course,
our overwhelmingly most important problem, insecurity, is not addressed
by this proposal. The proposal is trying to solve a problem from a 2000
viewpoint. We have moved on since then. We have new problems now, and the proposal
does not match them.
The community of JS developers is wide and deep, and the applications
of JS differ dramatically in scope. I, for one, am a member of "we"
who couldn't disagree with you more. So who exactly is "we"? What
evidence do you have that simplification would meet the needs of the
vast majority of "us"?
I say this not to be argumentative, but to point out that the
"development community" can't be generalized so easily as you
suggest. You represent an important voice in the community to be
sure, but it's by far not the only one.
I believe that it will be easier to improve the performance of the
language by simplifying it. That performance improvement will be critical as we
move toward mobile.
Do you have evidence to support this claim? Which simplifications can
you point to where objective evidence demonstrates that performance
will improve?
It's all well and good to invoke the "simple is better" mantra (no
disagreement here), but it needs to be backed up with evidence, detail
and definition.
Well, most languages have better support for libraries than C or C++. But there are things that even C has and JS<2 doesn't:
- #include
- a linker
- a somewhat standard manner of distributing these libraries (.a or .so)
- arguably, a [not necessarily hacker-proof] mechanism for sharing symbols between several components without exposing them to the client.
Note that I'm not attempting to advocate the merits of #include, which I find rather ugly. I also have no idea whether the linker is part of the spec or not -- but then, I wasn't discussing specs, more developer experience, and linkers have been part of the standard package for as long as I've been programming.
Now, with JS<2, to get something linke #include + linking (or Java's import), you need to use somewhat more convoluted methods. In addition, all the techniques I have seen require some hard-wiring inside the document, which may be good for small webpages, but looks like a rather bad practice when you're talking about whole applications: essentially, this is equivalent to putting all your #includes inside the UI code. Finally, while the methods may work in a webpage, not all of them apply well to non-web JS -- I'm thinking about off-line extensions, XPCOM or server-side JS.
This strikes me as a good reason to improve library management in JS. Which was the case last time I looked at ES4, although I may have missed a few developments on that front.
Brendan Eich wrote:
On Mar 26, 2008, at 10:01 AM, Ric Johnson wrote:
Let us take action instead of throwing opinions around: Brendan: What new features that can not be implemented via code constructs now?
This is reductionism, therefore between silly and wrong, but I will list a few things:
- you can't make read-only properties in ES3;
*you can't make don't-delete properties of plain old objects (only vars in closures);
you can't make objects that cannot be extended;
you can't make don't-enum properties in plain old objects;
It looks like these omissions will be corrected in ES3.1 by the Object.defineProperties function. These essential features will be added without resorting to new syntax.
On May 19, 2008, at 4:32 PM, Douglas Crockford wrote:
Brendan Eich wrote:
On Mar 26, 2008, at 10:01 AM, Ric Johnson wrote:
Let us take action instead of throwing opinions around: Brendan: What new features that can not be implemented via code constructs now?
This is reductionism, therefore between silly and wrong, but I
will list a few things:
- you can't make read-only properties in ES3;
*you can't make don't-delete properties of plain old objects (only
vars in closures);
you can't make objects that cannot be extended;
you can't make don't-enum properties in plain old objects;
It looks like these omissions will be corrected in ES3.1 by the Object.defineProperties function.
Of course, Ric asked "What new features [...] can not be implemented
via code constructs now?" Code constructs added in 3.1, with or
without new syntax, don't count.
These essential features will be added without resorting to new syntax.
New syntax is what's needed to make these usable. Who wants to use
Object.defineProperties on a closure to bind a read-only property
when you could use 'const'?
The problem with Object.defineProperties, apart from standardizing it
in two committees, is the verbosity and (at the limit) overhead.
There's really no reason not to have better UI for the underlying
semantics.
On Mon, May 19, 2008 at 4:46 PM, Brendan Eich <brendan at mozilla.org> wrote:
These essential features will be added without resorting to new syntax.
New syntax is what's needed to make these usable. Who wants to use Object.defineProperties on a closure to bind a read-only property when you could use 'const'?
The problem with Object.defineProperties, apart from standardizing it in two committees, is the verbosity and (at the limit) overhead. There's really no reason not to have better UI for the underlying semantics.
I agree that it would be nice to have better syntactic conveniences for some of these features. I also agree that the ES4 syntax has some decent conveniences. Given the inescapable legacy compatibility constraints, it's amazing how well they turned out; kudos! If I could have a language with some of the syntactic conveniences of ES4 but without ES4's semantics, I'd be quite happy. My "Classes as Sugar" proposal mail.mozilla.org/pipermail/es4-discuss/2008-March/002496.html
suggests how some of ES4's syntactic conveniences could be applied to a language with only ES3.1's semantics.
However, so long as ES4 ties these syntactic conveniences to ES4's semantics, ES3.1's successors cannot grow in the direction of the "Classes as Sugar" proposal without violating the subset agreement. Given our current situation, I think maintaining the subset agreement is more important. For some, ES3.1 is too little. For others, ES4 is too much. From prior discussions, I don't believe it's possible to find something in the middle that we can all agree on -- though I would still like to!
On May 19, 2008, at 6:54 PM, Mark Miller wrote:
If I could have a language with some of the syntactic conveniences of ES4 but without ES4's semantics, I'd be quite happy.
What semantics in particular, can you pick on something specific
that's not in your classes as sugar proposal (rather than have me
guess)? BTW, since you missed the cuts in the spreadsheet, you may
have missed the optional type checker being cut too: 'use strict' is
good-taste mode, a la Perl and in accord with discussions we've had
at the last two TC39 meetings.
Thanks for the kind words, although since neither 3.1 nor 4 is done
yet, specific constructive criticism is even better.
2008/5/19 Brendan Eich <brendan at mozilla.org>:
On May 19, 2008, at 6:54 PM, Mark Miller wrote:
If I could have a language with some of the syntactic conveniences of ES4 but without ES4's semantics, I'd be quite happy.
What semantics in particular, can you pick on something specific that's not in your classes as sugar proposal (rather than have me guess)?
Good suggestion. Will do. Top of my list will be first-class Name objects.
BTW, since you missed the cuts in the spreadsheet [spreadsheets.google.com/pub?key=pFIHldY_CkszsFxMkQOReAQ&gid=2], you may have missed the optional type checker being cut too: 'use strict' is good-taste mode, a la Perl and in accord with discussions we've had at the last two TC39 meetings.
Got those, and am quite happy about them. As for the spreadsheet, I'm very impressed by the amount of red in the Apple column. Apple folks, can you say more about this?
Brendan, how should one derive from the spreadsheet what the current ES4 decisions are? Should we have a column for that?
Thanks for the kind words, although since neither 3.1 nor 4 is done yet, specific constructive criticism is even better.
Understood. Will do.
On Tue, May 20, 2008 at 7:02 AM, Mark S. Miller <erights at google.com> wrote:
2008/5/19 Brendan Eich <brendan at mozilla.org>:
On May 19, 2008, at 6:54 PM, Mark Miller wrote:
If I could have a language with some of the syntactic conveniences of ES4 but without ES4's semantics, I'd be quite happy.
What semantics in particular, can you pick on something specific that's not in your classes as sugar proposal (rather than have me guess)?
Good suggestion. Will do. Top of my list will be first-class Name objects.
I have now read the three draft ES4 spec docs Lars sent out. Yes, the tops of my list are names and namespaces. The complexity it adds pushes the property lookup resolution algorithm way beyond what programmers, including myself, can reliably predict. (Grain of salt alert: I had the same reaction to C++'s overload resolution rules and their interaction with C++'s implicit coercion rules. Nevertheless, C++ is widely used. Draw from this what conclusions you will.)
AFAICT, there are three use-cases for namespaces in ES4:
a) At the package-ish level, in order to have some discipline over inter-module-ish linkage and who gets to import/export what global-ish variable names to whom.
b) At the property-name level, in order to be able to add additional properties to existing objects without breaking existing code that is already using those names for other reasons.
c) As a secure encapsulation mechanism, so that the impossibility of mentioning some property names prevents access to the properties associated with those names.
I don't find any of these use cases compelling. Out of order:
a) The YUI and Dojo libraries (and perhaps others) already adopts a convention of dotted path names for namespace discipline of global-ish variables and inter-module-ish linkage. This has the familiar look of Java-ish package names and fully qualified class names, and is built as a pattern, with no appreciable inconvenience, out of existing ES3 concepts.
c) As an encapsulation mechanism, this provides implicit sibling access, as in Java. Java "private" is per-class. Joe-E inherits this design from Java, and uses it successfully for security. Any instance of a Joe-E class can implicitly see the private members of any other instance of that class. By contrast, Smalltalk's instance variables are encapsulated per-instance. E and Caja follow Smalltalk, and use this design successfully for security.
We have discussed on cap-talk and/or e-lang the confused deputy hazards of Joe-E's per-class encapsulation as contrasted with E's per-instance encapsulation. The problem is that the greater access siblings have to each other -- beyond what clients have to each -- provides an implicit form of rights amplification. (See section 10.2.1 and Figure 10.1 of www.evoluware.eu/fsp_thesis.pdf for how
rights-amplification raises confused deputy hazards.) Our conclusion was that this is indeed a hazard, but a lint-like tool for Joe-E could mitigate the danger since such amplification is always statically detectable. (Access to a non-public member of something other than "this".)
ES4 has no such reliable ability to detect amplification statically. Therefore, the confused deputy hazards inherit in using rights amplifications will be non-obvious and difficult to detect. Better to have an attribute-based encapsulation mechanism, that explicitly distinguishes public vs private properties, and to use per-instance encapsulation of private properties. (Adding a "private" attribute to properties would be much more elegant that Caja's current "does it end with an underbar?" test.) When rights amplification is actually needed, better to express it explicitly by other means, such as the sealer/unsealer pairs shown in the Caja spec.
b) Having dispensed with the utility of ES4 namespaces as a secure encapsulation mechanism, there remains the issue of using them by convention/discipline to avoid accidental collisions. Experience with Smalltalk and the Prototype library for JS shows that there really is a genuine problem here. In Smalltalk, hundreds of separately written extensions add methods to Object. Early JavaScript libraries like Prototype did likewise. Existing Smalltalk virtual machines, going back at least to Smalltalk-80, already support method dispatch on non-string selectors, so an ES4-like solution was already possible with only a change to the compiler. I know of at least three designs (the best is home.cc.gatech.edu/tony/uploads/32/SharedSmalltalk.htm)
to bring this power to Smalltalk. The T language (an object-oriented variant of Scheme) and Joule (E's immediate ancestor) all had some form of first-class selector www.erights.org/history/joule/MANUAL.B17.pdf. In all of
these, as in ES4, a source identifier is first looked up in a lexical context to find a first-class selector object. Then, this selector object is used for method-dispatch (i.e., property lookup).
None of these caught on. Adoption is a complex process, so lack of adoption is not strong evidence of anything. However, some interesting data:
-
Java classes are not extensible the way Smalltalk's classes are. In Java, I have to put up with the pain that my module is unable to extend Object. In exchange, I gain the benefit that your module is also unable to extend Object. Talking to several programmers who made the shift from Smalltalk to Java (not a scientific survey, and I'm sure I asked leading questions), the reports I got were that the gain was worth the pain.
-
More recent JavaScript libraries have learned to avoid Prototype's practice of extending primordial objects. Rather, by convention and discipline, they largely program as if subject to Java's non-extensibility constraints. Extending primordial objects is now widely regarded as bad practice among JavaScript programmers.
-
E descends directly from Joule but dropped this aspect of Joule. Although we keep expecting to miss it, we haven't actually missed it in practice. E objects are not extensible by their clients -- not actually, and not virtually.
So, although I don't find any of these use-cases compelling, I'd agree that #b is at least a real problem, and is the most plausible of the arguments for some additional support in a future JavaScript. My next message, if I get to it, will be "Namespaces as Sugar". If you agree with me that #b is better avoided than solved, feel free to skip it.
Brendan Eich wrote:
I think this is the specific point of disagreement. Complexity in a language does not necessarily reduce the complexity of programs. I think the opposite may be truer. The difficulties we have had in the development community since 1999 were not due to over-minimization. They were due to features that did not work as expected or reliably over the various brands and versions. I think that with minimal changes we can significantly improve this language. Most of the changes I would make would make it simpler, not more complex. And of course, our overwhelmingly most important problem, insecurity, is not addressed by this proposal. The proposal is trying to solve a problem from a 2000 viewpoint. We have moved on since then. We have new problems now, and the proposal does not match them.
I believe that it will be easier to improve the performance of the language by simplifying it. That performance improvement will be critical as we move toward mobile.
I can't make sense of your point about competition. The web's competitors all provide access to multiple languages, including JavaScript. In the unlikely case that the proposed language is successful, I would expect them to support it too. It would seem that the web would be more competitive if it also offered multiple languages. I don't believe that a single, all-inclusive monster language will be competitive.