System.import()?
From: es-discuss [mailto:es-discuss-bounces at mozilla.org] On Behalf Of
There's a planned feature that addresses all these use cases:
System.import(moduleSpec, referrer).It's possible to make minor changes to HostResolveImportedModule and then specify
System.importin terms of that. It could ship in the existing compilation-plus-polyfill module system implementations (like webpack) immediately. And it'd be fully compatible with the coming JS Loader Standard.
I don't understand. If it's in terms of HostResolveImportedModule, and HostResolveImportedModule is not implemented or specified in any way yet, introducing a new thing on top of that does not help the situation at all.
In other words, it seems necessary to make import "moduleSpec" work before even thinking about how to make System.import("moduleSpec") work.
The timing and extensibility is too complex to easily fit into ECMA-262, see some things mentioned in whatwg/loader#54 . I vote no for a few years at least.
I agree that it could stand to wait. Also, for what it's worth, the WHATWG loader spec is still a huge work in progress AFAIK.
Having a simple general purpose import seems a good enough goal to me. Use if or switch for conditional. Use try/catch/finally for error handling. Make everything load on first call and its lazy enough.
I'd extend it to loading other content-types though. If you want to import Coffee or JSON or XML it should be the same. If it's not JavaScript expose it as a stream and throw an error so alternate means can process it. Maybe implement it as an object that could be modified or inherited to leave it easily extensible.
Thanks, Michael McGlothlin Sent from my iPhone
I think we all want to find a good solution to creating a Loader for ES6 modules. I would follow WHATWG's Loader if you want to participate. There are a surprising number of considerations, particularly due to existing code bases.
Node and WHATWG's Loader hook will be different, but are the only form of contention, and even then would only be visible to module loader authors; module inspection, reflections, runtime loading etc. will be compatible.
On Mon, Aug 17, 2015 at 5:02 PM, Domenic Denicola <d at domenic.me> wrote:
It's possible to make minor changes to HostResolveImportedModule and then specify
System.importin terms of that. [...]I don't understand. If it's in terms of HostResolveImportedModule, and HostResolveImportedModule is not implemented or specified in any way yet, introducing a new thing on top of that does not help the situation at all.
HostResolveImportedModule is widely implemented by compilers, though not by browsers. People are writing, using, and sharing ES6 modules today.
In other words, it seems necessary to make
import "moduleSpec"work before even thinking about how to makeSystem.import("moduleSpec")work.
It is working, and the people using it have a need for the dynamic API to go along with the static syntax, for reasons I pointed out in the initial post.
On Mon, Aug 17, 2015 at 5:02 PM, Bradley Meck <bradley.meck at gmail.com> wrote:
The timing and extensibility is too complex to easily fit into ECMA-262, see some things mentioned in whatwg/loader#54 . I vote no for a few years at least.
I am not calling for importing the entire loader spec into ECMA-262.
From: Jason Orendorff [mailto:jason.orendorff at gmail.com]
HostResolveImportedModule is widely implemented by compilers, though not by browsers. People are writing, using, and sharing ES6 modules today.
So what is your proposal then? Just standardize the node module resolution algorithm, since that is what transpilers are doing?
It is working, and the people using it have a need for the dynamic API to go along with the static syntax, for reasons I pointed out in the initial post.
It is not working outside of transpilers, which implement nonstandard resolution algorithms. The same transpilers implement JSX, but I don't think we'd take that as a reason to say that JSX is "working" and now let's start discussing how to standardize a reflective JSX layer.
It seems to me that what is needed is something like what used to be System.get().
In an async environment this should return nothing if it's not already loaded but in sync environments it could be treated as the imperative API for sync module loading and go through the same loader steps as System.import but perhaps with no async wrappers?
- Matthew Robb
On Mon, Aug 17, 2015 at 9:50 PM, Bradley Meck <bradley.meck at gmail.com> wrote:
I think we all want to find a good solution to creating a Loader for ES6 modules. I would follow WHATWG's Loader if you want to participate. There are a surprising number of considerations, particularly due to existing code bases.
I'm aware of the considerations. I helped Dave Herman hash out the Loader design.
The opportunity here is that we can specify System.import() now without having to solve all those considerations overnight.
There is nothing stopping us. We have the primitive. We have standard syntax that uses it. All we have to do is say "...and here is a standard API that exposes the same primitive".
Then JS programmers will have a way to call that primitive procedurally, something they now lack.
This proposal is not for the immediate benefit of browsers, which indeed need to wait for Loaders and/or <script type="module">. But
implementations that already support ES6 modules, such as Babel+webpack, could implement this tonight. They've already got code to load modules; they expose it via nonstandard APIs; the additional effort to provide System.import() would be minimal. Then people writing ES6 code today would have a complete standard module system to code to and use.
This would assume they can support the transformation hooks to do things like load coffeescript etc. right now, which is the main contention point.
So what is your proposal then? Just standardize the node module resolution algorithm, since that is what transpilers are doing?
Could do a lot worse. It's pretty simple and it works. To many languages overcomplicate imports. I especially like how node just uses normal variables instead of giving special meaning to the names or needing special syntax to rename.
The System.loader.import feature isn't ready for stabilization yet, and
is blocked on a few other pieces of the Loader design. The concerns you
mention are definitely important and we're working on them in the Loader
spec, but no need to rush something through before it's ready. Over time,
we can definitely uplift aspects of the Loader spec into Ecma-262.
I'm working on a roadmap for the Loader spec to clarify the bigger picture. Right now I've just got a skeletal dependency graph but I'll be fleshing it out with some text to explain it better:
On Tue, Aug 18, 2015 at 12:11 PM, Domenic Denicola <d at domenic.me> wrote:
HostResolveImportedModule is widely implemented by compilers, though not by browsers. People are writing, using, and sharing ES6 modules today.
So what is your proposal then? Just standardize the node module resolution algorithm, since that is what transpilers are doing?
Here's what I had in mind:
-
Do not standardize the Node algorithm or any other module loading algorithm in ECMA-262. That responsibility belongs to the Loader standard.
-
Change the signature of HostResolveImportedModule to match
System.import(), which takes a this-value and two strings (name and referrer). -
Clarify that HostResolveImportedModule (and its existing callers in the spec) operate asynchronously, e.g. by adding a PromiseCapability argument, like PerformPromiseAll. (The async functions proposal takes a different approach to specifying async algorithms, but I'm not sure if that's final wording or what exactly.)
-
Specify a
Systembuiltin object. -
Specify a builtin method System.import(name[, referrer]) that performs these steps:
-
Let promiseCapability be NewPromiseCapability(%Promise%).
-
Return HostResolveImportedModule(this value, name, referrer, promiseCapability).
-
Nothing terribly exciting here. Some vaguely interesting refactoring in the second and third bullet points.
The intent with System.import() all along was to provide a dynamic
equivalent to the static import syntax. Why not go one step further
and make it official that they share an implementation? That's all
this proposal does, and that's enough to make a useful standard API
for people who are writing ES6 modules right now.
From: Jason Orendorff [mailto:jason.orendorff at gmail.com]
Here's what I had in mind:
I have re-read this a few times and still don't understand how you expect this to be implemented in a standards-compliant way given that HostResolveImportedModule is not standardized. It doesn't seem to change the equilibrium at all.
If this is just to help transpilers, well, they can already do whatever they want, with no guarantees of future compatibility, like they are doing now. They already invented nonstandard semantics for HostResolveImportedModule; they might as well add a nonstandard property like System to the global and give it a nonstandard method named import with nonstandard behavior.
On Tue, Aug 18, 2015 at 12:45 PM, Bradley Meck <bradley.meck at gmail.com> wrote:
This would assume they can support the transformation hooks to do things like load coffeescript etc. right now, which is the main contention point.
It is a perfectly ordinary occurrence in software to ship some capability at one point, and ways of customizing that capability later.
The problem is timing; WHATWG uses promises, which Node cannot use due to
existing code bases on npm + potential mixing w/ require. This causes
time discrepancies and would mean different actions depending on how
exactly hooks are specced out. Node does not want to prevent async loading
on the browser, and the WHATWG loader does not want to cause breakage in
Node.
There are ways for syncing promises for Node compatibility, although they all lie in C++ land. Similar has already been done with Node callbacks.
(You could also spawn a thread, block for a message, and join it with the promise result from a C++ callback from a thenable.)
It's great to see more interest in dynamic loading here.
The contextual loader may be worth considering as a focus here - this is
the special contextual loading syntax to effectively provide a scoped
System.import call whose referrer is locked to the current module (import module from this as Dave has put in the roadmap, but I'm not sure what the
exact syntax proposal is).
This is the equivalent of the dynamic require in Node.
As syntax it seems that would be in ECMA-262, and also might avoid the async Promise issues to share with Node as it can be entirely implementation-specific.
Having a spec for this syntax would certainly be beneficial as it would immediately enable transpilers and module loading environments to then provide the dynamic loading feature, without other specification effort necessary.
I'd like to see a general horn-clause style dependency resolution system built into JavaScript functions and see that applied to dynamic loading. Then the same system could be used to executing async operations smoothly, logic programming, etc. Something smart enough to resolve all the relationships from simple rules and lazy load the required bits as soon as possible. I've done that with running async code and it makes more sense to me than Promises.
📱 Michael McGlothlin
I think a nice syntax would be allowing you to CALL import as a function from within function bodies to get contextual dynamic loading that is still SORT OF analyzable. It would also be nice if these import calls could SOMEHOW work like async function calls and not require direct interaction with a promise or callback. But I just work up so I may still be dreaming.
- Matthew Robb
We have discussed the issue somewhat at length with the WHATWG loader spec with calls. We agree that having diverging async/sync properties is problematic if a feature is present in both Node and the browser.
-
the consensus is
System.importreturns aPromise, which means the execution of the load can always be done during the promise job queue. -
Loader hooks for fetch/transformation/execution are left unspecified in terms of async/sync
-
- this delegates the branching logic from the Loader spec to the Loader implementation and creators of Loader hooks, metadata can be provided to feature detect what kind of Loader it is
-
using co-routines/non-typical JS can be avoided and any potential compatibility problems with VMs can be avoided as well.
-
- also avoids some VM limitations about only being able to materialize objects on the VM thread
-
new syntax is the branching for breaking changes
-
- opt-in of ES6 modules changes the loading order for modules for Node, but execution order can be kept consistent with what an async loader would perform
-
- circular bindings are an error (not circular dependencies)
-
System.importis still able to userequirein node and preserve blocking nature some modules expect/require. -
there is some inconsistency currently about when user
Promisesresolve during code loading, these should be looked into more
A decent amount of information is not fleshed out in the spec / is discussed via email, so I would suggest moving to issues on the WHATWG Loader's github: whatwg/loader
Until Loader is in a decent enough shape that both Node and WHATWG give it a thumbs up, I would wait.
On Tue, Aug 18, 2015 at 1:36 PM, Dave Herman <dherman at mozilla.com> wrote:
The
System.loader.importfeature isn't ready for stabilization yet, and is blocked on a few other pieces of the Loader design.
Well, if it isn't ready, it isn't ready. But can you tell a little bit more? What sort of changes are anticipated?
Apart from "System" apparently being renamed, the API doesn't seem to have changed in the past few years.
Cool. Is the intent to release a standard as each stage of work is finished? Or will everything remain unstable until stage 3 is completed?
Jason,
I agree with you that there needs to be making rapid progress with the dynamic module API. I also think, that TC39 need to be full engaged with the design of that API.
However, I have some issues with the specifics you defined below. My over-riding concern is that we don't loose from the specification the ability to do static, ahead-of-time module linking for implementations and situations where all modules are know and available prior to program execution. Ahead-of-time lining was an important consideration for the structuring of ES6 clause 15.2.1 and I'd hate to see us go backwards in that regard (instead I think that we should refractor ModuleDeclarationInstantiation (and Module Record) to clarify which parts of it can be performed during ahead-of-time linking).
The key enabler for ahead-of-time linking is that the abstract operations and data structures required to specify module linking must not depend upon ES runtime values or runtime data structures.
More specific concerns below
On Aug 18, 2015, at 2:37 PM, Jason Orendorff wrote:
On Tue, Aug 18, 2015 at 12:11 PM, Domenic Denicola <d at domenic.me> wrote:
HostResolveImportedModule is widely implemented by compilers, though not by browsers. People are writing, using, and sharing ES6 modules today.
So what is your proposal then? Just standardize the node module resolution algorithm, since that is what transpilers are doing?
Here's what I had in mind:
- Do not standardize the Node algorithm or any other module loading algorithm in ECMA-262. That responsibility belongs to the Loader standard.
agreed
- Change the signature of HostResolveImportedModule to match
System.import(), which takes a this-value and two strings (name and referrer).
HostResovleImportedModule must be an ahead-of-time friendly abstract operation, hence it can't depend upon a this-value or any runtime value. It isn't clear to me why you think that parameter is needed. Changing the refer from a Module Record to a string would not create an ahead-of-time link issue, but I'm not sure why you think that is needed.
- Clarify that HostResolveImportedModule (and its existing callers in the spec) operate asynchronously, e.g. by adding a PromiseCapability argument, like PerformPromiseAll. (The async functions proposal takes a different approach to specifying async algorithms, but I'm not sure if that's final wording or what exactly.)
The linking algorithm (which is threaded through ModuleDeclarationInstantiation, that's what I think needs refactoring) calls HostResolveImportedModule and so it can't have any dependencies upon ES promises or any other ES runtime artifacts.
The easy way to make module loading asynchronous (without adding runtime dependencies or complicating linking with async steps algorithm) is to define a Job (all ES Jobs are async) that performs the module loading steps. That's essentially what TopLevelModuleEvaluationJob does. A SystemLoad job would be similar, but could have additional steps that are specific to the semantics of dynamic module loading.
Specify a
Systembuiltin object.Specify a builtin method System.import(name[, referrer]) that performs these steps:
Let promiseCapability be NewPromiseCapability(%Promise%).
Return HostResolveImportedModule(this value, name, referrer, promiseCapability).
Nothing terribly exciting here. Some vaguely interesting refactoring in the second and third bullet points.
I would do the equivalent via the DynamicModuleEvaluation Job:
System.import (or whatever it is called)would:
- Create a new PromiseCapability
- Start a DynamicModuleEvaluation job, passing it the promise capability along with the user supplied arguments
- Return the Promise object of the PromiseCapability
The intent with
System.import()all along was to provide a dynamic equivalent to the staticimportsyntax. Why not go one step further and make it official that they share an implementation? That's all this proposal does, and that's enough to make a useful standard API for people who are writing ES6 modules right now.
Fine, as long as we don't further muddle-up those parts of module linking/loading that don't have runtime dependencies with those that do.
On Tue, Aug 18, 2015 at 11:36 AM, Dave Herman <dherman at mozilla.com> wrote:
From a loader/tool perspective, it seems like working out more of the
Stage 1 items before many of the Stage 0 items will lead to higher leverage payoffs: the dynamic loading and module meta help existing transpiling efforts, better inform userland module concatenation efforts.
In the spirit of the extensible web, defining these lower level APIs and more of the loader would make it possible to use custom elements to help prototype a <module> tag. The custom element mechanism can be
used in a similar way to how JS transpilers are used for the existing module syntax.
If the Stage 0 normalization is about normalizing IDs to URLs as the internal normalized storage IDs, I suggest reaching out to talk with more people in the AMD loader community about the reasons behind it, how it sits with AMD loader plugin IDs, seeing IDs more like scoped, nested property identifiers, and module concatenation.
James
From: es-discuss [mailto:es-discuss-bounces at mozilla.org] On Behalf Of James Burke
In the spirit of the extensible web, defining these lower level APIs and more of the loader would make it possible to use custom elements to help prototype a <module> tag. The custom element mechanism can be used in a similar way to how JS transpilers are used for the existing module syntax.
No, custom elements cannot prototype the module tag, because custom elements will always be parsed as unknown elements. So e.g.
<custom-module>
const stuff = "<p>Stuff</p>";
</custom-module>
will be parsed as
custom-module
|- text node: const stuff = "
|- p
|---- text node: Stuff
| - text node: ";
and of course it gets much worse if you use any elements that do something visual or interesting, like <img> or <custom-module> or <script>.
This is why we need to use an existing parsed-as-is element (namely <script>, although I guess we could maybe make <style> work in a pinch), and why custom elements won't work.
On Fri, Aug 21, 2015 at 8:12 AM, Domenic Denicola <d at domenic.me> wrote:
No, custom elements cannot prototype the module tag, because custom elements will always be parsed as unknown elements. So e.g.
Ah, thanks for pointing that out. I was thinking more of the script src style of tag. The need for module bodies directly inside HTML tags, vs loaded as separate files, is of lower importance, less leveraged impact, than the other module/loader APIs.
James
and this is why XHTML...
The ES6 module system is taking a real beating in the comments section here: hacks.mozilla.org/2015/08/es6-in-depth-modules
People are concerned about things like:
There's a planned feature that addresses all these use cases:
System.import(moduleSpec, referrer).It's possible to make minor changes to HostResolveImportedModule and then specify
System.importin terms of that. It could ship in the existing compilation-plus-polyfill module system implementations (like webpack) immediately. And it'd be fully compatible with the coming JS Loader Standard.Arguably something this fundamental to module usage belongs in ECMA-262 anyway.
What do you think?