implementation dependencies (was Re: ES4 work)
Comments below:
Going further, I have mentally considered the language as providing 3 "big" categories of enhancement: fixtures, types, and namespaces. I think that within -- and possibly between -- these groups there are dependencies. For example, we can consider these "levels" of type-implementation:
TY-VAL: a runtime representation of types exists, and values have a pointer to their type
TY-PROP: properties can be annotated with types, and a dynamic check is made when an assignment is made
TY-DEF: type-aliases ("type T = ...") can be defined
TY-STRUCT: the full structural type grammar exists (object types, array types, function types)
TY-NULL: the nullability extension to the type grammar exists
TY-NOMINAL: class and interface types exist, with a hard-coded subtype lattice
TY-PARAM: the parametric type system exists
TY-LIKE: the 'like' types exist
TY-STRICT: approximate static checking of types
TY-REFLECT: meta-objects exist and can be acquired with typeof
This is a partial dependency list. You need at least TY-VAL to do TY-PROP, but it's probably possible to implement any combination of the remainder once you're at TY-PROP. You could also stop at TY-PROP, only permitting users to denote the ES3 types (prims and objs). Though IMO this would be silly.
We began and have a fairly complete TY_VAL and TY_PROP. Next we did TY_NOMINAL and TY_REFLECT. We are missing DEF, STRUCT, NULL, PARAM and LIKE. We have partial STRICT.
We can also consider "levels" of namespace-implementation:
NS-VAL: a runtime type "namespace" exists, and has some nonempty population
NS-PROP: every property has a namespace and namespace references can be used in reference expressions like obj.ns::prop and ns::lexref
NS-USE: the "use namespace" and "use default namespace" pragmas automatically qualify definitions or references
NS-DEF: "namespace" declarations are accepted and new namespaces can be defined through them, either anonymous or with strings
NS-CLS: classes (and interfaces?) define their own namespaces for conventional OO visibility control
NS-PKG: the "package" construct exists for automatically defining namespaces
Similarly, this list is more linear at the top than the bottom: NS-PROP and NS-USE require NS-VAL, though one could stop there with (perhaps) a fixed population of namespaces. The remaining 3 are mostly orthogonal: you could for example stop implementing with NS-DEF and ignore classes and packages, and still have a useful system. Or do NS-CLS alone and ignore NS-DEF, using namespaces only to model class-visibility issues.
I think this is very much an all or nothing. It is hard to separate out these from each other. We started with NS_DEF, NS_VAL and NS_PROP. NS_USE was easy NS_USE is pretty easy once you have NS_VAL and NS_PROP. NS_PKG is essential if you are going to handle package qualified variables and avoid name collisions. So I'd imagine it would be hard to have a cohesive whole without doing all these items.
All the type and namespace issues depend, however, on fixtures. We are some ways towards "proving" that fixtures-in-absence-of-namespaces are equivalent to the "dontdelete" property attribute (see ticket #233, bugs.ecmascript.org/ticket/233) but if you have namespaces there appears to be a requirement to be modeling fixtures, to run the multiname algorithm properly. Fixtures are sort of "super-dontdelete" properties -- those that can safely be early bound, in addition to not being deletable -- and it's hard to make much use of the namespace or type systems without them.
Agree. We had fixtures first and retrofitted namespace (which I would not recommend). Namespaces are so foundational, you need to design them in at the start. Otherwise, there is a lot of rework.
Re-encoding the ES3 primitives as classes, and the new classes like map, vector and the meta objects, all require a fair amount of the TY and NS work: at least TY-NOMINAL, TY-PARAM and NS-CLS (I think).
Looking at the builtins, I can't imagine how you could do them without namespaces. They are a vital solution to various name lookup and collision problems.
Finally there is a category I left off the above elaboration, mostly because it is under-developed in the RI: control mechanisms. There are dependencies between tail calls, generators and stack inspection, and I can't say I fully understand the dependencies nor the impact they have on the rest of the implementation.
There are a whole raft of implementation "toughies" that will vary a bit from implementation to implementation. We have spent a lot of time trying to get ES4 to be small and fast. But there is a long, long way to go.
Michael
On Feb 19, 2008, at 6:39 PM, Graydon Hoare wrote:
Finally there is a category I left off the above elaboration, mostly because it is under-developed in the RI: control mechanisms. There are dependencies between tail calls, generators and stack inspection,
and I can't say I fully understand the dependencies nor the impact they have on the rest of the implementation.
This reminds me of something that Maciej already questioned in his
initial response to the language overview: if the control inspector
is optional, it's either useless on the web because not supported by
all major browsers, or else de-facto non-optional: mandatory due to
support in one or two top-market-share browsers. Therefore it should
not be included as an optional part out of ES4. We already agreed not
to make it mandatory for the sake of small-device implementations.
I like the control inspector proposal (
doku.php?id=proposals:stack_inspection), but Maciej's argument seems
decisive to me. Those of us likely to implement it in the near term
could work on developing an ad-hoc spec from the proposal, to be
considered for inclusion in a later edition.
Generators have an implementation plan in the RI based on delimited
continuations in SML, and we seem to have reached consensus recently
on proper tail calls. To lighten the load and focus on these two high-
value control abstractions proposed for ES4, could we think about
deferring control inspector? This is discussion fodder, I'm not yet
filing a trac ticket.
An HTML attachment was scrubbed... URL: esdiscuss/attachments/20080221/12f8103f/attachment-0002
On Feb 21, 2008, at 11:07 PM, Michael O'Brien wrote:
Seems to me we may have some emerging agreement on the following
items. Please be kind if I'm overstating the consensus, but I
believe the following items start us in the right direction without
being too onerous. Triage the existing proposals into those that are current and
correct and those that aren't. Publish that list.
Agreed.
Bring the out-of-date proposal pages up to date
This sounds good, but if we've accepted proposals and need detailed
specs, why not write specs? This is not just a matter of wiki
namespace (proposal: vs. spec:). Proposals have emphasized
precedents, use-cases, and anti-use-cases, and considered
alternatives. Discussion uncovered further detail, but still not
enough for a spec in all cases.
Also, the spec can reference the RI (not just SML but, for the
standard library, the self-hosted ES4!) in a systematic way.
Proposals preceded the RI.
Suggest we evaluate Lars's forthcoming library spec and see what
proposals pages it effectively updates. Agree we should mark those
pages somehow as superseded by specs, with links to the specs. Not
thrilled about leaving out-of-date proposals around, but there's
clearly a conflict between the wiki, which is great for content
creation, and the trac and spec, which are better for disposition and
finalizable specification.
Implementers & designers pitch in and write up design notes for
proposals. This is then both input to the spec and immediate
guidance for implementers. Some doc/comments for the RI
Again I am leery of reinventing the RI in prose, duplicating its
meaning with added bugs and no testability. Better to refer to the RI
directly as I think you proposed earlier, or even excerpt it as was
planned for the spec. The ES4 excerpts should not need lowering;
Graydon's script can be used if people find SML hard to read.
Create a common place to store resolutions and clarifications on
issues. The mailing list isn't great for this gems get lost in the
volume. Perhaps the author for each design note could maintain the
document and append questions on the end as clarifications in wiki
style.
Tracking issues is a job for the trac, although there's always room
for on-the-side summaries linking to tickets, if you keep editing to
keep up with the primary source of truth in the trac.
I'll start the ball rolling with writing up some notes on Program
Units, use unit and unit dependencies. Brendan/Jeff: what format
would you like these notes in?
Graydon and Lars should weigh in too.
Comments below:
This sounds good, but if we've accepted proposals and need detailed specs, why not write specs? This is not just a matter of wiki namespace (proposal: vs. spec:). Proposals have emphasized precedents, use-cases, and anti-use-cases, and considered alternatives. Discussion uncovered further detail, but still not enough for a spec in all cases.
The reason is we could remove a few road blocks with some design notes. These won't be as complete as the spec but could from the basis of writing some of the spec prose.
Also, the spec can reference the RI (not just SML but, for the standard library, the self-hosted ES4!) in a systematic way. Proposals preceded the RI.
The problem here is time. I think doing the spec with the required level of rigour will take much longer than would be ideal to get implementations started.
Suggest we evaluate Lars's forthcoming library spec and see what proposals pages it effectively updates. Agree we should mark those pages somehow as superseded by specs, with links to the specs. Not thrilled about leaving out-of-date proposals around, but there's clearly a conflict between the wiki, which is great for content creation, and the trac and spec, which are better for disposition and finalizable specification.
I've seen an early cut of the Library spec. Is an update coming?
Again I am leery of reinventing the RI in prose, duplicating its meaning with added bugs and no testability. Better to refer to the RI directly as I think you proposed earlier, or even excerpt it as was planned for the spec. The ES4 excerpts should not need lowering; Graydon's script can be used if people find SML hard to read.
Fine to refer to the RI, but I think we're talking notes here.
Graydon's emails contain a lot of good "notes" but are certainly not a
spec (yet). I think we just need more of those "notes".
I would agree that if this can't be done simply and easily, we should just push on as forward momentum will eventually deliver the goods. Time is never a friend if we are slow or meandering.
Create a common place to store resolutions and clarifications on issues. The mailing list isn't great for this gems get lost in the volume. Perhaps the author for each design note could maintain the document and append questions on the end as clarifications in wiki style.
Tracking issues is a job for the trac, although there's always room for on-the-side summaries linking to tickets, if you keep editing to keep up with the primary source of truth in the trac.
I think the summaries are where the gold is. That is the piece we are missing. We actually have a lot of information, but it is scattered and hard to put together in a coherent manner.
I'll start the ball rolling with writing up some notes on Program Units, use unit and unit dependencies. Brendan/Jeff: what format would you like these notes in?
You missed this question above.
On Feb 21, 2008, at 11:40 PM, Michael O'Brien wrote:
The reason is we could remove a few road blocks with some design
notes. These won't be as complete as the spec but could from the basis of writing some of the spec prose.
Ok.
Also, the spec can reference the RI (not just SML but, for the standard library, the self-hosted ES4!) in a systematic way.
Proposals preceded the RI.The problem here is time. I think doing the spec with the required
level of rigour will take much longer than would be ideal to get implementations started.
Yes, I've agreed with this loudly recently (no waterfall).
Tracking issues is a job for the trac, although there's always room for on-the-side summaries linking to tickets, if you keep editing to keep up with the primary source of truth in the trac.
I think the summaries are where the gold is. That is the piece we are missing. We actually have a lot of information, but it is scattered
and hard to put together in a coherent manner.
In this light the overview and tutorial were more than the sum of
their parts from the wiki and trac.
I'll start the ball rolling with writing up some notes on Program Units, use unit and unit dependencies. Brendan/Jeff: what format would you like these notes in?
You missed this question above.
No, I ducked :-). Lars is editor with Jeff assisting and (as always)
maintaining the grammar; I would appreciate Graydon's thoughts too.
On Thu, 2008-02-21 at 23:07 -0800, Michael O'Brien wrote:
Triage the existing proposals into those that are current and correct and those that aren't. Publish that list.
All current proposals are marked as "accepted :)", but perhaps there should be more shades of grey?
I've made the assumption that those who have higher contribution percentages to proposal pages are probably the best people to ask about the current state of said proposals. As such, I've generated a list of proposals and user/percentage contributions which might be useful in pinpointing those who have the necessary context to bring the pages up to speed (see attached).
It would be naïve to think that all the contributors to the proposals are going to be willing to trudge through the required thoughts, list discussions, tickets & RI to update the pages, but ideally they could point someone else in the right direction, or perhaps make sure they're kept on the right track.
Cheers,
-- Nathan de Vries
Brendan Eich wrote:
I'll start the ball rolling with writing up some notes on Program Units, use unit and unit dependencies. Brendan/Jeff: what format would you like these notes in?
You missed this question above.
No, I ducked :-). Lars is editor with Jeff assisting and (as always) maintaining the grammar; I would appreciate Graydon's thoughts too.
Shrug I'm not picky. I think Lars has been working more in HTML, and that's what I hope most of the spec-writing will happen in, just because it's real easy to render, edit, version control and such. But plaintext or some flavour of wikitext is also convenient. Word docs less so, though I think they're the final target form ECMA wants. And of course there are our more academic members who feel more at home in LaTeX. Whatever floats your boat. It's relatively easy to interconvert, either way.
-----Original Message----- From: es4-discuss-bounces at mozilla.org [mailto:es4-discuss-bounces at mozilla.org] On Behalf Of Graydon Hoare Sent: 22. februar 2008 09:43 To: Brendan Eich Cc: Michael O'Brien; es4-discuss Discuss; Dave Herman Subject: Re: implementation dependencies (was Re: ES4 work)
Brendan Eich wrote:
I'll start the ball rolling with writing up some notes on Program
Units, use unit and unit dependencies. Brendan/Jeff: what format would you like these notes in?
You missed this question above.
No, I ducked :-). Lars is editor with Jeff assisting and (as always) maintaining the grammar; I would appreciate Graydon's thoughts too.
Shrug I'm not picky. I think Lars has been working more in HTML, and that's what I hope most of the spec-writing will happen in, just because it's real easy to render, edit, version control and such. But plaintext or some flavour of wikitext is also convenient. Word docs less so, though I think they're the final target form ECMA wants. And of course there are our more academic members who feel more at home in LaTeX. Whatever floats your boat. It's relatively easy to interconvert, either way.
The spec sources will be written using the mixed HTML/Wiki notation that is already used by the draft library spec, see what's in the monontone repository if you're curious (spec/library).
That should however not be a concern to most people who want to contribute draft spec(let)s, since the actual markup is mainly a problem for the editors. On the other hand, the more magic software that is used to produce speclets, the more pain it is for us to extract it into the actual spec. So be kind. Text and/or wiki notation (whatever flavor) is best.
(I have written some speclets myself that I will post here if the present discussion ever subsides, and I have used plain text.)
What about the actual content sections?
We want the same look and feel over the text -- what perspectives do you need covered in describing a feature. Do you have a sample?
Michael
(I have written some speclets myself that I will post here if the present discussion ever subsides, and I have used plain text.)
Yes please, that will help as a sample -- and I'm interested to read all available material ;-)
Michael
Jeff Dyer wrote:
One thing that springs to mind when looking at this is that the proposals -- that is, really, the "features" to be implemented -- have implementation dependencies that need to be reflected in any work plan.
I pointed this out in the phone meeting today and was asked to outline some of the implementation dependencies I've noted while working on the RI.
I think a certain number of features are mostly-independent of all others, so they can be cleared off first: enhancements to unicode and regexps, slice syntax, string.prototype.trim, the global object, triple quotes, line terminator normalization, improvements to Date, the set of "pure bug fixes" in ES3, expression closures, let-binding, fixes to the arguments object, control of enumerability, destructuring assignment. These are all essentially stand-alone and several have already shipped as vendor extensions to ES3.
Going further, I have mentally considered the language as providing 3 "big" categories of enhancement: fixtures, types, and namespaces. I think that within -- and possibly between -- these groups there are dependencies. For example, we can consider these "levels" of type-implementation:
TY-VAL: a runtime representation of types exists, and values have a pointer to their type
TY-PROP: properties can be annotated with types, and a dynamic check is made when an assignment is made
TY-DEF: type-aliases ("type T = ...") can be defined
TY-STRUCT: the full structural type grammar exists (object types, array types, function types)
TY-NULL: the nullability extension to the type grammar exists
TY-NOMINAL: class and interface types exist, with a hard-coded subtype lattice
TY-PARAM: the parametric type system exists
TY-LIKE: the 'like' types exist
TY-STRICT: approximate static checking of types
TY-REFLECT: meta-objects exist and can be acquired with typeof
This is a partial dependency list. You need at least TY-VAL to do TY-PROP, but it's probably possible to implement any combination of the remainder once you're at TY-PROP. You could also stop at TY-PROP, only permitting users to denote the ES3 types (prims and objs). Though IMO this would be silly.
We can also consider "levels" of namespace-implementation:
NS-VAL: a runtime type "namespace" exists, and has some nonempty population
NS-PROP: every property has a namespace and namespace references can be used in reference expressions like obj.ns::prop and ns::lexref
NS-USE: the "use namespace" and "use default namespace" pragmas automatically qualify definitions or references
NS-DEF: "namespace" declarations are accepted and new namespaces can be defined through them, either anonymous or with strings
NS-CLS: classes (and interfaces?) define their own namespaces for conventional OO visibility control
NS-PKG: the "package" construct exists for automatically defining namespaces
Similarly, this list is more linear at the top than the bottom: NS-PROP and NS-USE require NS-VAL, though one could stop there with (perhaps) a fixed population of namespaces. The remaining 3 are mostly orthogonal: you could for example stop implementing with NS-DEF and ignore classes and packages, and still have a useful system. Or do NS-CLS alone and ignore NS-DEF, using namespaces only to model class-visibility issues.
All the type and namespace issues depend, however, on fixtures. We are some ways towards "proving" that fixtures-in-absence-of-namespaces are equivalent to the "dontdelete" property attribute (see ticket #233, bugs.ecmascript.org/ticket/233) but if you have namespaces there appears to be a requirement to be modeling fixtures, to run the multiname algorithm properly. Fixtures are sort of "super-dontdelete" properties -- those that can safely be early bound, in addition to not being deletable -- and it's hard to make much use of the namespace or type systems without them.
Re-encoding the ES3 primitives as classes, and the new classes like map, vector and the meta objects, all require a fair amount of the TY and NS work: at least TY-NOMINAL, TY-PARAM and NS-CLS (I think).
Finally there is a category I left off the above elaboration, mostly because it is under-developed in the RI: control mechanisms. There are dependencies between tail calls, generators and stack inspection, and I can't say I fully understand the dependencies nor the impact they have on the rest of the implementation.
Jeff Dyer wrote: > The basic workflow: > > Proposal --> > Implementation --> > Feature spec --> > Feature review --> > ES4-RI --> > ES4 spec --> > One thing that springs to mind when looking at this is that the proposals -- that is, really, the "features" to be implemented -- have implementation dependencies that need to be reflected in any work plan. I pointed this out in the phone meeting today and was asked to outline some of the implementation dependencies I've noted while working on the RI. I think a certain number of features are mostly-independent of all others, so they can be cleared off first: enhancements to unicode and regexps, slice syntax, string.prototype.trim, the global object, triple quotes, line terminator normalization, improvements to Date, the set of "pure bug fixes" in ES3, expression closures, let-binding, fixes to the arguments object, control of enumerability, destructuring assignment. These are all essentially stand-alone and several have already shipped as vendor extensions to ES3. Going further, I have mentally considered the language as providing 3 "big" categories of enhancement: fixtures, types, and namespaces. I think that within -- and possibly between -- these groups there are dependencies. For example, we can consider these "levels" of type-implementation: TY-VAL: a runtime representation of types exists, and values have a pointer to their type TY-PROP: properties can be annotated with types, and a dynamic check is made when an assignment is made TY-DEF: type-aliases ("type T = ...") can be defined TY-STRUCT: the full structural type grammar exists (object types, array types, function types) TY-NULL: the nullability extension to the type grammar exists TY-NOMINAL: class and interface types exist, with a hard-coded subtype lattice TY-PARAM: the parametric type system exists TY-LIKE: the 'like' types exist TY-STRICT: approximate static checking of types TY-REFLECT: meta-objects exist and can be acquired with typeof This is a partial dependency list. You need at least TY-VAL to do TY-PROP, but it's probably possible to implement any combination of the remainder once you're at TY-PROP. You could also stop *at* TY-PROP, only permitting users to denote the ES3 types (prims and objs). Though IMO this would be silly. We can also consider "levels" of namespace-implementation: NS-VAL: a runtime type "namespace" exists, and has some nonempty population NS-PROP: every property has a namespace and namespace references can be used in reference expressions like obj.ns::prop and ns::lexref NS-USE: the "use namespace" and "use default namespace" pragmas automatically qualify definitions or references NS-DEF: "namespace" declarations are accepted and new namespaces can be defined through them, either anonymous or with strings NS-CLS: classes (and interfaces?) define their own namespaces for conventional OO visibility control NS-PKG: the "package" construct exists for automatically defining namespaces Similarly, this list is more linear at the top than the bottom: NS-PROP and NS-USE require NS-VAL, though one could stop there with (perhaps) a fixed population of namespaces. The remaining 3 are mostly orthogonal: you could for example stop implementing with NS-DEF and ignore classes and packages, and still have a useful system. Or do NS-CLS alone and ignore NS-DEF, using namespaces only to model class-visibility issues. All the type and namespace issues depend, however, on fixtures. We are some ways towards "proving" that fixtures-in-absence-of-namespaces are equivalent to the "dontdelete" property attribute (see ticket #233, http://bugs.ecmascript.org/ticket/233) but if you have namespaces there appears to be a requirement to be modeling fixtures, to run the multiname algorithm properly. Fixtures are sort of "super-dontdelete" properties -- those that can safely be early bound, in addition to not being deletable -- and it's hard to make much use of the namespace or type systems without them. Re-encoding the ES3 primitives as classes, and the new classes like map, vector and the meta objects, all require a fair amount of the TY and NS work: at least TY-NOMINAL, TY-PARAM and NS-CLS (I think). Finally there is a category I left off the above elaboration, mostly because it is under-developed in the RI: control mechanisms. There are dependencies between tail calls, generators and stack inspection, and I can't say I fully understand the dependencies nor the impact they have on the rest of the implementation. -Graydon