Modules, Concatenation, and Better Solutions
On Mon, Oct 15, 2012 at 9:31 AM, Kevin Smith <khs4473 at gmail.com> wrote:
With modules, we're going to see code broken up into more and smaller files, which directly opposes the desire to minimize the number of HTTP round-trips. On solution is concatenation, but if I'm not mistaken there are potential order-of-execution issues with that approach.
I believe that concatenation of modules ought to work fine as an approach for minimization of requests, because modules are executed when imported.
2012/10/15 Kevin Smith <khs4473 at gmail.com>
Hi all,
With modules, we're going to see code broken up into more and smaller files, which directly opposes the desire to minimize the number of HTTP round-trips.
Arguably, by the time we can use ES6 modules, browsers will support HTTP/2.0 with which minimizing the number of HTTP requests stops being a goal [1] (see slides 34-39 in particular)
David
I believe that concatenation of modules ought to work fine as an approach for minimization of requests, because modules are executed when imported.
OK, so:
module A {
console.log("a");
export var x;
}
console.log("$");
import x from A;
Does this print:
$
a
or
a
$
?
On Oct 15, 2012, at 6:38 AM, Sam Tobin-Hochstadt wrote:
On Mon, Oct 15, 2012 at 9:31 AM, Kevin Smith <khs4473 at gmail.com> wrote:
With modules, we're going to see code broken up into more and smaller files, which directly opposes the desire to minimize the number of HTTP round-trips. On solution is concatenation, but if I'm not mistaken there are potential order-of-execution issues with that approach.
I believe that concatenation of modules ought to work fine as an approach for minimization of requests, because modules are executed when imported.
should that be "when first imported". Presumably a module is executed only once per Realm
On Mon, Oct 15, 2012 at 9:45 AM, Kevin Smith <khs4473 at gmail.com> wrote:
OK, so:
module A { console.log("a"); export var x; } console.log("$"); import x from A;Does this print: $ a or a $
The first - "$", then "a".
At least, that's how most module systems I've played with seem to work - CommonJS-ish ones I've written and used, and mostly AMD; there was an issue with the AMD almond loader that it would execute a "module factory" when the module was define()'d, not when it was first require()'d. Can't remember the final stand on that one. I think most of the AMD loaders will ensure that factories are not run until needed.
This has worked out quite well, as it means ordering of modules doesn't matter - Browserify is a good example of this. It means you don't HAVE to arrange your modules in any particular order, just make sure they are all defined before you do your first require(). MUCH, MUCH easier to build concatenators if you don't care what the order is.
Just to be sure... Does a get printed only the first time the module A is imported somewhere, or every time?
Just to be sure... Does a get printed only the first time the module A is imported somewhere, or every time?
Only the first time. But the question here is about nested/inline modules.
Patrick, it must be the other way. Here's why:
module A {
export function f() { console.log("A"); }
}
A.f();
No import required before usage of an inline module. There is a concatenation strategy which will preserve order-of-execution, but but without some scope artifacts:
I'm not saying this is a problem with the current design - just that it complicates the concatenation story.
Yehuda Katz (ph) 718.877.1325
On Tue, Oct 16, 2012 at 9:21 AM, Kevin Smith <khs4473 at gmail.com> wrote:
Just to be sure... Does a get printed only the first time the module A is
imported somewhere, or every time?
Only the first time. But the question here is about nested/inline modules.
Patrick, it must be the other way. Here's why:
module A { export function f() { console.log("A"); } } A.f();No import required before usage of an inline module.
Is that actually true of the current proposal?
On Tue, Oct 16, 2012 at 9:21 AM, Kevin Smith <khs4473 at gmail.com> wrote:
Patrick, it must be the other way. Here's why:
module A { export function f() { console.log("A"); } } A.f();No import required before usage of an inline module. There is a concatenation strategy which will preserve order-of-execution, but but without some scope artifacts:
So, first, I've been barely following the ES Modules bits - I wanted to chime in on how I "expected" it would work based on module stuff I've been playing with.
The scoped module definition in your gist is horrifying.
I wouldn't expect to see this pattern from a concatenator though. I'd expect that they would do an "import" of A before referencing anything it. This would typically be the "main" referenced by a run of the concatenator, which would spit out the import for the "main" module. But this couldn't change the dynamics here, right? It's not like if you changed your snippet above to first "import f from A" before invoking it, that the module factory would NOT be run first, right?
Unfortunate. Perhaps there's a story around wrapping each concatenated module in another module, deferring the run of the actual factory until first "really" used, via proxies, or something. Ick.
I suppose I should dig in a bit more on this ES module stuff ...
I guess I'm mostly interested in seeing if there's a module story we can transpile in, that would also allow concatenation in the typical sense used today, so people could start playing with this stuff. Eg, I could see taking something Browserify as we know it today, allowing you to use ES module syntax, and have it "just work".
Maybe that's too much to ask for, or that we'll have the real thing in short order, or for some other reason that doesn't make sense. One of those reasons would be that those "ES modules that Browserify can understand" wouldn't be understandable by node.js, directly, which could be a practical issue.
On Oct 15, 2012, at 6:45 AM, Kevin Smith <khs4473 at gmail.com> wrote:
OK, so:
module A { console.log("a"); export var x; } console.log("$"); import x from A;Does this print:
Good question. The way we have it currently specified isn't ideal for concatenation, I think. The code executes eagerly, from top to bottom, but if it encounters an external load, it executes that external module on demand (i.e., on first import). This means that this:
// a.js
import b from "b.js";
console.log("a");
export let a = "a";
// b.js
console.log("b");
export let b = "b";
// main.js
import a from "a.js";
console.log("main");
prints "b" then "a" then "main", whereas this:
module a {
import b from b;
console.log("a");
export let a = "a";
}
module b {
console.log("b");
export let b = "b";
}
import a from "a".js";
console.log("main");
prints "a" then "b" then "main". That's clearly a problem for simple concatenation. On the one hand, the point of eager execution was to make the execution model simple and consistent with corresponding IIFE code. On the other hand, executing external modules by need is good for usually (except in some cases with cyclic dependencies) ensuring that the module you're importing from is fully initialized by the time you import from it.
I see two coherent alternatives:
(a) execute inline modules by need (i.e., on first import) rather than eagerly (b) execute external modules "transactionally", trying to order them by dependency so that imported modules have fully initialized before the code that depends on them runs
The weird thing about (a) is that code that appears to be straight-line actually executes in somewhat more unpredictable (although deterministic) order. The downside of (b) is that when you have cyclic dependencies, one module won't even be partly initialized before the other module starts running. (Although usually, when you have cyclic dependencies, you probably don't want to be evaluating exports from each other in top-level code -- you want all those mutual references to be happening under functions.)
On Tue, Oct 16, 2012 at 2:58 PM, David Herman <dherman at mozilla.com> wrote:
prints "a" then "b" then "main". That's clearly a problem for simple concatenation. On the one hand, the point of eager execution was to make the execution model simple and consistent with corresponding IIFE code. On the other hand, executing external modules by need is good for usually (except in some cases with cyclic dependencies) ensuring that the module you're importing from is fully initialized by the time you import from it.
In earlier versions of requirejs, I used to eagerly evaluate define() calls as they were encountered, trying to duplicate the IIFE feel.
This caused a problem for concatenation: some build scenarios build in all the modules used for a page into one JS script. However, only half the modules may be used for the first "screen render", with the second half of the modules for a second "screen render" that is triggered by a user action. The secondary set of modules can have global state changes, like CSS/style changes.
By eagerly evaluating the modules as they were encountered in the built script, the page would have unwanted style changes applied during the first screen render when they should have been held until the second set of module use for the second render.
By switching to "evaluate module factory functions by need" in requirejs, it gained the following benefits:
- concat code executes closer to the order in non-concat form.
- delaying work that does not need to be done up front. If optimizations like delayed function parsing (like v8 does?) extended to modules, even parse time could be avoided.
- modules can be concatenated in an order that does not strictly match the linearized dependency chain (the benefit Patrick Mueller mentions earlier in the thread).
James
On Tue, Oct 16, 2012 at 2:58 PM, David Herman <dherman at mozilla.com> wrote:
On Oct 15, 2012, at 6:45 AM, Kevin Smith <khs4473 at gmail.com> wrote:
OK, so:
module A { console.log("a"); export var x; } console.log("$"); import x from A;Does this print:
Good question. The way we have it currently specified isn't ideal for concatenation, I think. The code executes eagerly, from top to bottom, but if it encounters an external load, it executes that external module on demand (i.e., on first import). This means that this:
// a.js import b from "b.js"; console.log("a"); export let a = "a"; // b.js console.log("b"); export let b = "b"; // main.js import a from "a.js"; console.log("main");prints "b" then "a" then "main", whereas this:
module a { import b from b; console.log("a"); export let a = "a"; } module b { console.log("b"); export let b = "b"; } import a from "a".js"; console.log("main");prints "a" then "b" then "main". That's clearly a problem for simple concatenation. On the one hand, the point of eager execution was to make the execution model simple and consistent with corresponding IIFE code.
(I know this is obvious, but the eager execution model is simple and IIFE-like only if you imagine that "module execute eagerly".)
On the other hand, executing external modules by need is good for usually (except in some cases with cyclic dependencies) ensuring that the module you're importing from is fully initialized by the time you import from it.
I see two coherent alternatives:
(a) execute inline modules by need (i.e., on first import) rather than eagerly (b) execute external modules "transactionally", trying to order them by dependency so that imported modules have fully initialized before the code that depends on them runs
The weird thing about (a) is that code that appears to be straight-line actually executes in somewhat more unpredictable (although deterministic) order.
I guess if we can handle deeply nested callbacks, event handlers, and promises, then this is a pretty tame as weird things go. Are you are saying is that the body of the module does not run until we need its content? So dependents pull in their dependencies and modules that are not needed are not executed. That's what we want right?
The downside of (b) is that when you have cyclic dependencies, one module won't even be partly initialized before the other module starts running. (Although usually, when you have cyclic dependencies, you probably don't want to be evaluating exports from each other in top-level code -- you want all those mutual references to be happening under functions.)
I don't understand b, but dependency on internal vs external sounds bad.
jjb ... eager to see modules happen!
On Oct 16, 2012, at 4:51 PM, John J Barton <johnjbarton at johnjbarton.com> wrote:
On Tue, Oct 16, 2012 at 2:58 PM, David Herman <dherman at mozilla.com> wrote:
The weird thing about (a) is that code that appears to be straight-line actually executes in somewhat more unpredictable (although deterministic) order.
I guess if we can handle deeply nested callbacks, event handlers, and promises, then this is a pretty tame as weird things go. Are you are saying is that the body of the module does not run until we need its content? So dependents pull in their dependencies and modules that are not needed are not executed. That's what we want right?
My main concern comes from the fact that module initialization code can have arbitrary side effects: modify variables or shared objects, update the DOM, change some CSS, send an XHR request to fire a nuclear missile (HTTP UPDATE please -- I like my global thermonuclear war RESTful), ... You don't generally want languages to cause side effects to happen in unpredictable orders. If modules are executed on demand, you can't as easily predict when they are going to execute. (This is an instance of the general rule of PL design: lazy execution and side effects don't mix.)
But, that said, the execution of the modules isn't triggered by arbitrary control flow of the program, just the syntactically restricted top-level imports. So that probably makes it pretty tame.
Still, I'm interested in hearing Sam's and Andreas Rossberg's input on this.
The downside of (b) is that when you have cyclic dependencies, one module won't even be partly initialized before the other module starts running. (Although usually, when you have cyclic dependencies, you probably don't want to be evaluating exports from each other in top-level code -- you want all those mutual references to be happening under functions.)
I don't understand b, but dependency on internal vs external sounds bad.
Both (a) and (b) are meant to avoid dependency on internal vs external (unlike the system as currently described on the wiki). I didn't explain (b) very well. The idea would be that you would execute modules one at a time, rather than interleaved, but their order of execution would be topologically sorted by dependency. So if you don't have cyclic dependencies, you know that by the time you refer to an export of a module you depend on, it will have already been initialized. However, with cyclic dependencies, it's harder to understand what order they'll execute in, and you need to make sure not to refer to exports of a module until it's been fully initialized. By contrast, with (a), there's still a chance that you won't blow up.
Concrete example: Even and Odd modules refer to each other, but the import statements occur after some initialization:
module Odd {
export let odd = function(x) {
return x === 0 ? false : !even(x - 1);
}
import even from Even; // force execution of Even here, if it hasn't already
export let b = odd(17);
}
module Even {
export let even = function(x) {
return x === 0 || !odd(x - 1);
}
import odd from Odd; // force execution of Odd here, if it hasn't already
export let b = even(17);
}
console.log(Odd.b);
With semantics (a), this executes like so:
- start executing Odd
- initialize Odd.odd
- start executing Even
- initialize Even.even
- try to start executing Odd (but it's already started, so don't)
- initialize Even.b by calling Even.even
- initialize Odd.b by calling Odd.odd
And everything succeeds. With semantics (b), regardless of how we break the tie in a cycle, this will fail, because it will try to call either Odd.odd or Even.even before it has initialized the function. Say it starts executing Odd first. It would do this:
- start executing Odd
- initialize Odd.odd
- initialize Odd.b by calling Even.even
- error: Even.even is not yet initialized
So semantics (a) makes it more possible to make a cyclic dependency work, but it's still subtle if you're trying to let them refer to each other in top-level initialization code: you have to make sure to carefully place the imports late enough that the relevant pieces of each modules will have adequately initialized.
jjb ... eager to see modules happen!
Me too! :)
On Tue, Oct 16, 2012 at 7:10 PM, David Herman <dherman at mozilla.com> wrote:
On Oct 16, 2012, at 4:51 PM, John J Barton <johnjbarton at johnjbarton.com> wrote: ...
Concrete example: Even and Odd modules refer to each other, but the import statements occur after some initialization:
module Odd { export let odd = function(x) { return x === 0 ? false : !even(x - 1); } import even from Even; // force execution of Even here, if it hasn't already
Ah, I see the import has three possible duties here, 1) declaration of |even| from Even 2) definition of |even| 3) execution of module code. Its that last category that cause the issues in this example: the circular value isn't used until the next stmt.
export let b = odd(17); } module Even { export let even = function(x) { return x === 0 || !odd(x - 1); } import odd from Odd; // force execution of Odd here, if it hasn't already export let b = even(17); } console.log(Odd.b);With semantics (a), this executes like so:
- start executing Odd
- initialize Odd.odd
- start executing Even
- initialize Even.even
- try to start executing Odd (but it's already started, so don't)
- initialize Even.b by calling Even.even
- initialize Odd.b by calling Odd.odd
And everything succeeds. With semantics (b), regardless of how we break the tie in a cycle, this will fail, because it will try to call either Odd.odd or Even.even before it has initialized the function. Say it starts executing Odd first. It would do this:
- start executing Odd
- initialize Odd.odd
- initialize Odd.b by calling Even.even
- error: Even.even is not yet initialized
So semantics (a) makes it more possible to make a cyclic dependency work, but it's still subtle if you're trying to let them refer to each other in top-level initialization code: you have to make sure to carefully place the imports late enough that the relevant pieces of each modules will have adequately initialized.
I'm surprised by (b). A behavior I would expect from a module loader (c)
- start executing Odd
- initialize Odd.odd
- start executing Even
- initialize Even.even
- try to start executing Odd
- error: Odd has not completed (circular). Unlike (b) we load and execute Even until we discover the cycle. (b) seems to declare Even.even but rely on some other mechanism to define it.
I guess that all three allow Even/Odd cycles without the execution of b.
I guess (a) excludes later standardization of (b) or (c) but not the reverse.
jjb
On the one hand, the point of eager execution was to make the execution model simple and consistent with corresponding IIFE code.
When we switch from scripts to modules, it makes sense to switch from execute-as-encountered-for-side-effect to resolve-dependencies- and-extract-exports. The former may use IIFEs to limit the scope of some side-effects, the latter uses functions and results to avoid side-effects (*).
(a) execute inline modules by need (i.e., on first import) rather than eagerly
The weird thing about (a) is that code that appears to be straight-line actually executes in somewhat more unpredictable (although deterministic) order.
Think of a module as a function and import declarations as function calls. Only difference is the sharing of evaluation (each module body is only run once), with caching of results (exports).
Claus
(*) libraryinstitute.wordpress.com/2010/12/01/loading-javascript-modules
I see two coherent alternatives:
(a) execute inline modules by need (i.e., on first import) rather than eagerly (b) execute external modules "transactionally", trying to order them by dependency so that imported modules have fully initialized before the code that depends on them runs
I think we should also consider :
(c) remove easy and transparent concatability from the list of design goals.
Why do we want to concat modules? I can think of two reasons:
- To reduce the number of HTTP requests.
- To distribute a program as a single file.
Now, transpilers targeting ES<6 will have to wrap each "separate file" module in a factory function anyway, so order-of-execution is no problem for them. Ergo, the current module design presents no concat problems for ES<6 targets.
What about ES6 targets? Do we really need to concat for ES6 targets?
Concating source files has always been a suboptimal solution to the multiple request problem anyway. It's not as cringe-inducing as image sprites, but it's up there. As David Bruant mentioned, a design goal for HTTP 2 is to make such spriting unnecessary. Will HTTP 2 permeate before ES6..?
The obvious solution to (2) from above is to distribute programs as zip files. A crazy idea: what if browsers natively supported loading modules from zip files?
Kevin Smith wrote:
Concating source files has always been a suboptimal solution to the multiple request problem anyway. It's not as cringe-inducing as image sprites, but it's up there. As David Bruant mentioned, a design goal for HTTP 2 is to make such spriting unnecessary. Will HTTP 2 permeate before ES6..?
Probably not, my hunch.
The multiplication principle is not your friend. We have risk on both of those, they're independent, if we couple them the odds of success get smaller.
The obvious solution to (2) from above is to distribute programs as zip files. A crazy idea: what if browsers natively supported loading modules from zip files?
Browsers can load from ZIP files, some even supported the .jar variant and some.site.com/big.jar!member notation (not sure what's still working there). But this is again multiplying risks. Will developers all ZIP this way, instead of using TE (Transfer Encoding) or a different compression format?
I think Claus hit it: don't try to match IIFEs exactly, match internal to external module loading via by-need.
Browsers can load from ZIP files, some even supported the .jar variant and some.site.com/big.jar!**membersome.site.com/big.jar!membernotation (not sure what's still working there). But this is again multiplying risks. Will developers all ZIP this way, instead of using TE (Transfer Encoding) or a different compression format?
I see your point about risks, but FWIW I think they would. OS's make it effortless to zip, and zip supports all kinds of extensions...
I think Claus hit it: don't try to match IIFEs exactly, match internal to external module loading via by-need.
How do we tell when an inline module is needed, though?
module A {
export var x = "you need me";
}
export function f() {
console.log(A.x);
}
Do we log "you need me" only when f is called for the first time?
Do we log "you need me" only when f is called for the first time?
Sorry - that makes no sense. What I meant was:
module A {
console.log("you need me");
export var x = "x";
}
export function f() {
console.log(A.x);
}
Do we log "you need me" only when f is called for the first time?
Kevin Smith wrote:
Do we log "you need me" only when f is called for the first time?Sorry - that makes no sense. What I meant was:
module A { console.log("you need me"); export var x = "x"; }
export function f() { console.log(A.x); }Do we log "you need me" only when f is called for the first time?
Yes, that's what I took from Dave's description of (b). Laziness + side effects, FTW?
On Tue, Oct 16, 2012 at 10:10 PM, David Herman <dherman at mozilla.com> wrote:
On Oct 16, 2012, at 4:51 PM, John J Barton <johnjbarton at johnjbarton.com> wrote:
On Tue, Oct 16, 2012 at 2:58 PM, David Herman <dherman at mozilla.com> wrote:
The weird thing about (a) is that code that appears to be straight-line actually executes in somewhat more unpredictable (although deterministic) order.
I guess if we can handle deeply nested callbacks, event handlers, and promises, then this is a pretty tame as weird things go. Are you are saying is that the body of the module does not run until we need its content? So dependents pull in their dependencies and modules that are not needed are not executed. That's what we want right?
My main concern comes from the fact that module initialization code can have arbitrary side effects: modify variables or shared objects, update the DOM, change some CSS, send an XHR request to fire a nuclear missile (HTTP UPDATE please -- I like my global thermonuclear war RESTful), ... You don't generally want languages to cause side effects to happen in unpredictable orders. If modules are executed on demand, you can't as easily predict when they are going to execute. (This is an instance of the general rule of PL design: lazy execution and side effects don't mix.)
But, that said, the execution of the modules isn't triggered by arbitrary control flow of the program, just the syntactically restricted top-level imports. So that probably makes it pretty tame.
Still, I'm interested in hearing Sam's and Andreas Rossberg's input on this.
I think (a) is the better choice. It means that reordering source files with only modules in them doesn't change your program, and it supports concatenation/minification better, which I think are likely to be with us for a long time. Also, I think Claus' analogy to functions is the right one, with the proviso that the modules are run as soon as as we start running the module that imports them.
On Wed, Oct 17, 2012 at 9:51 AM, Kevin Smith <khs4473 at gmail.com> wrote:
Do we log "you need me" only when f is called for the first time?
Sorry - that makes no sense. What I meant was:
module A { console.log("you need me"); export var x = "x"; } export function f() { console.log(A.x); }Do we log "you need me" only when f is called for the first time?
I think that would be far to hard to understand. Instead, we should
treat a module with a reference like A.x to an unimported module as
if it implicitly had an import of A, so "you need me" would be
logged as soon as the surrounding module was imported, even if f is
never called.
On Oct 17, 2012, at 9:09 AM, Sam Tobin-Hochstadt wrote:
On Wed, Oct 17, 2012 at 9:51 AM, Kevin Smith <khs4473 at gmail.com> wrote:
Do we log "you need me" only when f is called for the first time?
Sorry - that makes no sense. What I meant was:
module A { console.log("you need me"); export var x = "x"; }
export function f() { console.log(A.x); }
Do we log "you need me" only when f is called for the first time?
I think that would be far to hard to understand. Instead, we should treat a module with a reference like
A.xto an unimported module as if it implicitly had an import ofA, so "you need me" would be logged as soon as the surrounding module was imported, even iffis never called.
Is the reference to A required to get the implicit import and hence implicit initialization? For example in:
module Outer {
/* no export */ module A {
console.log("you need me");
export var x = "x";
}
export function f() { console.log("no reference to A"); } }
I would expect inner modules to always be initialized when their outer module is initialized
On Wed, Oct 17, 2012 at 12:40 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Oct 17, 2012, at 9:09 AM, Sam Tobin-Hochstadt wrote:
On Wed, Oct 17, 2012 at 9:51 AM, Kevin Smith <khs4473 at gmail.com> wrote:
Do we log "you need me" only when f is called for the first time?
Sorry - that makes no sense. What I meant was:
module A { console.log("you need me"); export var x = "x"; }
export function f() { console.log(A.x); }
Do we log "you need me" only when f is called for the first time?
I think that would be far to hard to understand. Instead, we should treat a module with a reference like
A.xto an unimported module as if it implicitly had an import ofA, so "you need me" would be logged as soon as the surrounding module was imported, even iffis never called.Is the reference to A required to get the implicit import and hence implicit initialization? For example in:
module Outer { /* no export */ module A { console.log("you need me"); export var x = "x"; }
export function f() { console.log("no reference to A"); } }
I would expect inner modules to always be initialized when their outer module is initialized
That would make the semantics of wrapping things in a module, and then importing that module, substantially different than just having a top-level module. I think regularity suggests that "you need me" not be logged here.
On Oct 17, 2012, at 9:43 AM, Sam Tobin-Hochstadt wrote:
On Wed, Oct 17, 2012 at 12:40 PM, Allen Wirfs-Brock <allen at wirfs-brock.com> wrote:
On Oct 17, 2012, at 9:09 AM, Sam Tobin-Hochstadt wrote:
On Wed, Oct 17, 2012 at 9:51 AM, Kevin Smith <khs4473 at gmail.com> wrote:
Do we log "you need me" only when f is called for the first time?
Sorry - that makes no sense. What I meant was:
module A { console.log("you need me"); export var x = "x"; }
export function f() { console.log(A.x); }
Do we log "you need me" only when f is called for the first time?
I think that would be far to hard to understand. Instead, we should treat a module with a reference like
A.xto an unimported module as if it implicitly had an import ofA, so "you need me" would be logged as soon as the surrounding module was imported, even iffis never called.Is the reference to A required to get the implicit import and hence implicit initialization? For example in:
module Outer { /* no export */ module A { console.log("you need me"); export var x = "x"; }
export function f() { console.log("no reference to A"); } }
I would expect inner modules to always be initialized when their outer module is initialized
That would make the semantics of wrapping things in a module, and then importing that module, substantially different than just having a top-level module. I think regularity suggests that "you need me" not be logged here.
So any reference to A (not necessarily dotted) in the outer module triggers hoisted initialization of A? And if no such reference exists module A is essentially deal code?
So any reference to A (not necessarily dotted) in the outer module triggers hoisted initialization of A? And if no such reference exists module A is essentially deal code?
So side-effects would be allowed by the language, but would be incredibly confounding. Hmmm....
On Wed, Oct 17, 2012 at 1:59 PM, Kevin Smith <khs4473 at gmail.com> wrote:
So any reference to A (not necessarily dotted) in the outer module
triggers hoisted initialization of A? And if no such reference exists module A is essentially deal code?
So side-effects would be allowed by the language, but would be incredibly confounding. Hmmm....
I'm not sure if I like how this is turning out. I think modules should either not allow side effects, or should be evaluated immediately like an IIFE. While I appreciate lazy evaluation when it is really utilized like Haskell, I think it would be very surprising here. If we need to make it easier for concatenation, maybe what we really want is an easy way to effectively prime the module loader with a sort of cache so that it only "loads" the module when needed, in the same way it would work as multiple http requests, but it never needs to go to the server.
// a.js
import b from "b.js";
console.log("a");
export let a = "a";
// b.js
console.log("b");
export let b = "b";
// main.js
import a from "a.js";
console.log("main");
becomes
//lib.js
module "a.js" {
import b from "b.js";
console.log("a");
export let a = "a";
}
module "b.js" {
console.log("b");
export let b = "b";
}
//main.js
import a from "a.js";
console.log("main");
The idea here, is to allow a new bit of syntax where the module identifier is a string literal. If that is the case, it assumes the module is loaded in place of a file, where the url for the file is the value of the string literal. Just to be clear, though, it doesn't actually execute the module at all - it just caches it in the loader.
Of course, the content in main.js could still just go in lib.js, but I wanted to point out something important, namely that the import of "a.js" can be retained as is which is another potential problem with module concat.
On Oct 17, 2012, at 2:04 PM, Russell Leggett <russell.leggett at gmail.com> wrote:
I'm not sure if I like how this is turning out. I think modules should either not allow side effects, or should be evaluated immediately like an IIFE. While I appreciate lazy evaluation when it is really utilized like Haskell, I think it would be very surprising here. If we need to make it easier for concatenation, maybe what we really want is an easy way to effectively prime the module loader with a sort of cache so that it only "loads" the module when needed, in the same way it would work as multiple http requests, but it never needs to go to the server.
I agree the on-demand execution is disconcerting (as I said: side-effects + laziness = trouble). And carving out a side-effect-free sub-language of JS is not going to happen, of course.
If there's a simple way to accommodate simple concatenation, possibly something along the lines of what you're talking about, it's worth considering. But it also intersects with some other possibilities that have come up both on-list and off-list. So I need to spend some time off-list thinking through the space.
Maybe this is already obvious, but if modules are tightly coupled to files then the concatenation really has to be a container of files (eg zip) and it's not part of the language. If multiple modules can be declared in a single file, then concatenation is part of the language and has to work whether its done by the developer in a file or a build processor or browser cache etc.
jjb
I think we all agree that concatenation is not going away. In principle, it's the concatenator's fault if they do a non-semantics-preserving transformation. In practice, if getting the semantics-preserving transformation right is too hard, people will get it wrong, and it's our fault for making it too hard. So... I agree, don't worry. I just have stuff to work through that can't be done in a rapid-fire email conversation.
On Wed, Oct 17, 2012 at 5:04 PM, Russell Leggett <russell.leggett at gmail.com>wrote:
module "a.js" { import b from "b.js"; console.log("a"); export let a = "a"; } module "b.js" { console.log("b"); export let b = "b"; } //main.js import a from "a.js"; console.log("main");The idea here, is to allow a new bit of syntax where the module identifier is a string literal. If that is the case, it assumes the module is loaded in place of a file, where the url for the file is the value of the string literal. Just to be clear, though, it doesn't actually execute the module at all - it just caches it in the loader.
Ya, I kinda like that. Basically let's me mark an "internal" module as an "external one" - I even get to assign a "file name", which maybe a debugger can make sense of. Win!
OTOH, this is yet another feature; the feature set for modules seems pretty rich already - compared to what you have with CommonJS.
If there's any interest in simplifying ... might be worth noting that, in node, there is no notion of "internal modules". No one seems to mind. Maybe just get rid of them? Only top-level modules, and they're either from a "file" or you've poofed one up using new Module(nameString, codeString). Concatenators would of course use the Module constructor. Just like we use eval() today (with //@sourceURL!) in concat'd files to provide per-embedded-file debugging.
I think we all agree that concatenation is not going away. In principle, it's the concatenator's fault if they do a non-semantics-preserving transformation. In practice, if getting the semantics-preserving transformation right is too hard, people will get it wrong, and it's our fault for making it too hard. So... I agree, don't worry. I just have stuff to work through that can't be done in a rapid-fire email conversation.
Cool - look forward to seeing what you come up with!
Just to bring the thread back around to where I started it, I would like to just point out that there seems to be considerable consensus in the community (*) that some kind of "packaging" scheme is desirable. (I use the word "packaging" very loosely here.) The idea is that components are more than just code sprites - they encompass stylesheets and images and json configuration files and more. All of these have to be delivered somehow, and it would be nice if we didn't have to use separate spriting techniques for each. This probably goes beyond TC39, but if we could address the problem at that level, then we wouldn't have to bend over backwards to support these funky code sprites.
Kevin
- Twitter's Bower: twitter/bower
- TJ's Component: component/component
From: Patrick Mueller
On Wed, Oct 17, 2012 at 5:04 PM, Russell Leggett <russell.leggett at gmail.com> wrote:
module "a.js" { import b from "b.js"; console.log("a"); export let a = "a"; } module "b.js" { console.log("b"); export let b = "b"; }
//main.js import a from "a.js"; console.log("main");
The idea here, is to allow a new bit of syntax where the module identifier is a string literal. If that is the case, it assumes the module is loaded in place of a file, where the url for the file is the value of the string literal. Just to be clear, though, it doesn't actually execute the module at all - it just caches it in the loader.
Ya, I kinda like that. Basically let's me mark an "internal" module as an "external one" - I even get to assign a "file name", which maybe a debugger can make sense of. Win!
Of note, TypeScript currently uses a similar syntax to solve a variant of this problem. To describe the surface area of an API like the core node.js modules, we wanted a way for a single file to describe all of the modules that are made available within a bare node.js installation. This includes the ability to declare:
declare module "http" { export function get(options, callback); //... }
Indicating that a module named "http" exists and has the listed members. TypeScript does not yet support a syntax for actually implementing a module using this form, but Russell's syntax is almost exactly what would make sense as a logical extension of the declaration syntax.
The key difference between internal module definitions and Russell's inline external module definition syntax is that the later creates no lexical binding at the site of its declaration, and just puts an entry in the loader table. This feels to me like a clean way to support the concatenation scenarios discussed in this thread, while still supporting the IIFE module patterns that motivate internal modules.
Luke
On Wed, Oct 17, 2012 at 10:10 AM, David Herman <dherman at mozilla.com> wrote:
Concrete example: Even and Odd modules refer to each other, but the import statements occur after some initialization:
module Odd { export let odd = function(x) { return x === 0 ? false : !even(x - 1); } import even from Even; // force execution of Even here, if it hasn't already export let b = odd(17); } module Even { export let even = function(x) { return x === 0 || !odd(x - 1); } import odd from Odd; // force execution of Odd here, if it hasn't already export let b = even(17); } console.log(Odd.b);
What about this?
module Odd {
import even from Even; // force execution of Even here, if it
hasn't already export let odd = function(x) { return x === 0 ? false : !even(x - 1); } export let b = odd(17); } module Even { import odd from Odd; // force execution of Odd here, if it hasn't already export let even = function(x) { return x === 0 || !odd(x - 1); } export let b = even(17); } console.log(Odd.b);
I guess this doesn't work...
With modules, we're going to see code broken up into more and smaller files, which directly opposes the desire to minimize the number of HTTP round-trips. On solution is concatenation, but if I'm not mistaken there are potential order-of-execution issues with that approach. Another (possibly better?) approach would be to bundle groups of modules (and resources) into some compressed container format (like zip) and serve that directly to the client.
This goes beyond es-discuss, I imagine, but is there a list where such things are discussed?