Understanding the 'normalize' Loader hook
See the implemention in es6-module-loader: ModuleLoader/es6-module-loader/blob/master/lib/system.js#L117
In traceur we have different code that passes the same tests.
The tests: google/traceur-compiler/blob/master/test/unit/runtime/System.js
We would map your examples to:
On Tue, Aug 12, 2014 at 2:05 PM, Ian Hickson <ian at hixie.ch> wrote:
Assume my script is example.com/test.js, and assume that there is no extra information (nobody has registered any module names or anything like that). If I do:
import "a";
import "./a";
import "/a";
/a.js
import "//example.com/a";
import "example.com/a"
On Aug 12, 2014, at 2:05 PM, Ian Hickson <ian at hixie.ch> wrote:
Assume my script is example.com/test.js, and assume that there is no extra information (nobody has registered any module names or anything like that). If I do:
import "a"; import "./a"; import "/a"; import "//example.com/a"; import "example.com/a"
...am I supposed to end up with the same normalized name for all five?
How many entries in [[Modules]] should this result in?
(I understand that the answer depends on the default loader; what I'm asking is, what should the default loader's behaviour be.)
This is part of the design work we're actively working on. I'd ask you to hold off until we have some more information for you. I'll keep you posted -- we should have some more details for you soon.
On Tue, 12 Aug 2014, John Barton wrote:
See the implemention in es6-module-loader: ModuleLoader/es6-module-loader/blob/master/lib/system.js#L117
Ah, fascinating. So basically:
-
"normalize" does relative URL logic based on the current base URL and the referrer (i.e. the URL of the script doing the importing).
-
"locate" resolves that URL and adds ".js" (only if the last segment is part of the "path" part of the URL and doesn't contain a "."?).
Is that more or less right? (There's also the loader.paths stuff. Why is that in "locate" rather than "normalize"?)
(BTW, I noticed that both in that code above and in the spec, "referrer" is consistently misspelt as "referer", the HTTP way.)
I'm exploring some ways that the module system could be extended on the Web platform. One thing that some people have asked for is ways to declaratively import stylesheets and the like from inside script, as in:
import sprites from "images.png";
...where sprites gives you access to an Image element (<img>).
For some cases, e.g. style sheets, you really want to provide more information than just a name, so we'd presumably have the import statement use an element's ID to do the lookup, as in:
<link rel=stylesheet href="foo.css" id=foo-styles ...>
import styles from "foo-styles"; // gets us a reference to the <link> // element somehow
Presumably, the way this works is the "locate" hook returns the actual HTMLLinkElement. But what does the "normalize" hook return? It can't return a URL, since then "locate" and "fetch" won't know to not fetch anything. Is there any way to return out-of-band data with the normalized name? (Really I think "normalize" and "locate" should probably be one hook, which returns either a URL or an object. Having them split, especially with the caching happening off a key in the middle, is quite difficult to map to the Web platform.)
One option that jorendorff suggested in #whatwg is that if you want to refer to a local ID, you use "#foo", as in:
import styles from "#foo-styles";
This kind of approach has historically been problematic (see the history of usemap="" for example), but I don't see a better solution.
On Tue, 12 Aug 2014, David Herman wrote:
This is part of the design work we're actively working on. I'd ask you to hold off until we have some more information for you. I'll keep you posted -- we should have some more details for you soon.
Where is the design work happening? I'd love to be part of it, since this directly impacts the rest of the work I'm doing on the HTML spec right now.
On Tue, Aug 12, 2014 at 2:45 PM, Ian Hickson <ian at hixie.ch> wrote:
On Tue, 12 Aug 2014, John Barton wrote:
See the implemention in es6-module-loader:
ModuleLoader/es6-module-loader/blob/master/lib/system.js#L117
Ah, fascinating. So basically:
- "normalize" does relative URL logic based on the current base URL and the referrer (i.e. the URL of the script doing the importing).
I would expect that normalized names to be platform independent.
- "locate" resolves that URL and adds ".js"
This step is platform dependent, it produces an address from a name.
(only if the last segment is part of the "path" part of the URL and doesn't contain a "."?).
No such restriction is applied in our code.
Is that more or less right? (There's also the loader.paths stuff. Why is that in "locate" rather than "normalize"?)
Because the paths processing can produce arbitrary addresses (to cache, CDN, I guess)
(BTW, I noticed that both in that code above and in the spec, "referrer" is consistently misspelt as "referer", the HTTP way.)
I've already reported this bug.
I'm exploring some ways that the module system could be extended on the Web platform.
You might look at Guy Bedford's experiment on loader plugins systemjs/systemjs that follow the path laid down by Jame Burke in requirejs requirejs.org/docs/api.html#plugins
jjb
Ah, fascinating. So basically:
"normalize" does relative URL logic based on the current base URL and the referrer (i.e. the URL of the script doing the importing).
"locate" resolves that URL and adds ".js" (only if the last segment is part of the "path" part of the URL and doesn't contain a "."?).
In my opinion, we should consider the possibility that we have one too many hooks here. Why not one hook to resolve a module specifier to an absolute URL, given a base URL? What does the added indirection of an ad hoc global namespace which maps into the URL namespace buy us?
On Tue, 12 Aug 2014, John Barton wrote:
On Tue, Aug 12, 2014 at 2:45 PM, Ian Hickson <ian at hixie.ch> wrote:
On Tue, 12 Aug 2014, John Barton wrote:
See the implemention in es6-module-loader: ModuleLoader/es6-module-loader/blob/master/lib/system.js#L117
Ah, fascinating. So basically:
- "normalize" does relative URL logic based on the current base URL and the referrer (i.e. the URL of the script doing the importing).
I would expect that normalized names to be platform independent.
Sure. I'm talking specifically about the Web platform here.
- "locate" resolves that URL and adds ".js"
This step is platform dependent, it produces an address from a name.
Right.
(only if the last segment is part of the "path" part of the URL and doesn't contain a "."?).
No such restriction is applied in our code.
Sure. What do we want for the default Web loader though?
Is that more or less right? (There's also the loader.paths stuff. Why is that in "locate" rather than "normalize"?)
Because the paths processing can produce arbitrary addresses (to cache, CDN, I guess)
Sure but presumably if someone explicitly puts in the long URL, they still don't want it to cause a second load, right?
I mean, if you do:
loader.paths = { 'foo': 'scripts/foo', };
// in a different module
import "foo";
// in a different module
import "scripts/foo";
...surely you want just one module loaded. No?
Come to think of it, maybe you actually just want one loaded even if someone does:
import "scripts/foo.js";
...or even:
// assuming the base URL the whole time has been // www.example.com import "www.example.com/scripts/foo.js";
Is there any time we'd want all those imports to result in separate module loads?
I'm exploring some ways that the module system could be extended on the Web platform.
You might look at Guy Bedford's experiment on loader plugins systemjs/systemjs that follow the path laid down by Jame Burke in requirejs requirejs.org/docs/api.html#plugins
Same basic idea, yeah. It's unfortunate, IMHO, that we have to have special syntax to get something from the "normalize" hook to the "locate" hook. It means it gets exposed in the "name" of the Module, and it means that if your syntax has any redundancy (e.g. the way the loader plugins SystemJS can infer the plugin type before the "!" or have it explicitly given after the "!") you can end up with duplicates you didn't intend.
On Tue, Aug 12, 2014 at 3:26 PM, Ian Hickson <ian at hixie.ch> wrote:
On Tue, 12 Aug 2014, John Barton wrote:...
(only if the last segment is part of the "path" part of the URL and doesn't contain a "."?).
No such restriction is applied in our code.
Sure. What do we want for the default Web loader though?
Be consistent with URLs.which.allow.dots
Is that more or less right? (There's also the loader.paths stuff. Why is that in "locate" rather than "normalize"?)
Because the paths processing can produce arbitrary addresses (to cache, CDN, I guess)
Sure but presumably if someone explicitly puts in the long URL, they still don't want it to cause a second load, right?
I mean, if you do:
loader.paths = { 'foo': 'scripts/foo', };
// in a different module
import "foo";
// in a different module
import "scripts/foo";
...surely you want just one module loaded. No?
One module, but my point was more that the right hand side can be an absolute path.
Come to think of it, maybe you actually just want one loaded even if someone does:
import "scripts/foo.js";
In my opinion this should be an error. Either we have .js or we don't, not half-baked.
...or even:
// assuming the base URL the whole time has been // www.example.com import "www.example.com/scripts/foo.js";
Also an error.
Is there any time we'd want all those imports to result in separate module loads?
FWIW, most of the time I expect to write relative paths in imports: import "./foo/bar";
If I see import "foo/bar"; I expect foo to be remapped to a package.
Or to put it another way, we need a story for packages.
jjb
On Aug 12, 2014, at 2:46 PM, Ian Hickson <ian at hixie.ch> wrote:
On Tue, 12 Aug 2014, David Herman wrote:
This is part of the design work we're actively working on. I'd ask you to hold off until we have some more information for you. I'll keep you posted -- we should have some more details for you soon.
Where is the design work happening? I'd love to be part of it, since this directly impacts the rest of the work I'm doing on the HTML spec right now.
Right now what I've been doing is working with various stakeholders gathering requirements and constraints and beginning to sketch out what the solution space looks like. This includes people working on userland loader implementations, people in the Node community and browser development communities, people working on the JS modules design and spec, and people working on WC and HTML imports. Most of that has been in one-off discussions e.g. at the last TC39 face-to-face. I'll be filling out
https://github.com/dherman/web-modules/tree/master/browser-loader
with our progress. I don't want to block you and I'm happy to include you in these discussions, but there are enough stakeholders that it's not going to work to make everyone come to #whatwg at this stage.
On Tue, 12 Aug 2014, David Herman wrote:
On Aug 12, 2014, at 2:46 PM, Ian Hickson <ian at hixie.ch> wrote:
On Tue, 12 Aug 2014, David Herman wrote:
This is part of the design work we're actively working on. I'd ask you to hold off until we have some more information for you. I'll keep you posted -- we should have some more details for you soon.
Where is the design work happening? I'd love to be part of it, since this directly impacts the rest of the work I'm doing on the HTML spec right now.
Right now what I've been doing is working with various stakeholders gathering requirements and constraints and beginning to sketch out what the solution space looks like. This includes people working on userland loader implementations, people in the Node community and browser development communities, people working on the JS modules design and spec, and people working on WC and HTML imports. Most of that has been in one-off discussions e.g. at the last TC39 face-to-face. I'll be filling out
https://github.com/dherman/web-modules/tree/master/browser-loaderwith our progress. I don't want to block you and I'm happy to include you in these discussions, but there are enough stakeholders that it's not going to work to make everyone come to #whatwg at this stage.
Cool, ok. (I wouldn't expect to do any real development work in an IRC channel, FWIW. That's just where I've been bouncing ideas off people. The real work for HTML happens on the WHATWG mailing list.)
What's the expected output here? Are you hoping to write the spec for the Web's default loader? Or the spec for the spec?
Assume my script is example.com/test.js, and assume that there is no extra information (nobody has registered any module names or anything like that). If I do:
import "a"; import "./a"; import "/a"; import "//example.com/a"; import "example.com/a"
...am I supposed to end up with the same normalized name for all five?
How many entries in [[Modules]] should this result in?
(I understand that the answer depends on the default loader; what I'm asking is, what should the default loader's behaviour be.)