modules proposal
On Thu, May 13, 2010 at 16:41, David Herman <dherman at mozilla.com> wrote:
This looks really good. I hope we can move this from strawman into proposals at the next face to face meeting.
On May 13, 2010, at 11:39 PM, Erik Arvidsson wrote:
On Thu, May 13, 2010 at 16:41, David Herman <dherman at mozilla.com> wrote:
This looks really good. I hope we can move this from strawman into proposals at the next face to face meeting.
Agreed, this looks really strong. I particularly like the local renaming ability. I hope this can be discussed and adopted very quickly.
-- Alex Russell slightlyoff at google.com alex at dojotoolkit.org BE03 E88D EABB 2116 CC49 8259 CF78 E242 59C3 9723
On 2010-05-13, at 19:41, David Herman wrote:
I've updated the strawman proposals for static modules and dynamic module loaders and would love to get feedback from the es-discuss community.
Static modules:
Looks great!
I wonder if you considered having an export list, rather than tagging the individual exports? I think that makes it easier to see/document the public API of a module. You could at the same time allow renaming on export.
module Cowboy { export {draw: d, shoot: s}; function d () { ... }; function s () { d(); }; }
FWIW, the rename on import looked "backwards" to me at first glance, but I think I can learn.
Do we really need to say .* for all? We couldn't just say import Math?
On 15 May 2010, at 12:53, P T Withington wrote:
On 2010-05-13, at 19:41, David Herman wrote:
I've updated the strawman proposals for static modules and dynamic module loaders and would love to get feedback from the es-discuss community.
Static modules:
Looks great!
I wonder if you considered having an export list, rather than tagging the individual exports? I think that makes it easier to see/document the public API of a module. You could at the same time allow renaming on export.
module Cowboy { export {draw: d, shoot: s}; function d () { ... }; function s () { d(); }; }
FWIW, the rename on import looked "backwards" to me at first glance, but I think I can learn.
Do we really need to say
.*for all? We couldn't just sayimport Math?
If you wan't the exported symbols from the Math module to be available as (for example) just pi and not Math.pi, then yes, you need the .*. IIRC this is very similar to how python and java behave w.r.t imports and I think its the right way.
Thought looking at the syntax section it seems that import Math; isn't valid and instead you do it as module Math = Math. Why not use import here as well? My first instinct here is module is for defining a module and import is importing it - instead module serves a dual function. Is there any reasoning for this behaviour that I might have missed?
On 15.05.2010 15:53, P T Withington wrote:
On 2010-05-13, at 19:41, David Herman wrote:
I've updated the strawman proposals for static modules and dynamic module loaders and would love to get feedback from the es-discuss community.
Static modules:
Looks great!
I wonder if you considered having an export list, rather than tagging the individual exports? I think that makes it easier to see/document the public API of a module. You could at the same time allow renaming on export.
Yep, I also thought. Cons: we should repeat method name twice (and in the worth case -- three times) and can forget to export something (as e.g. Erlang programmers sometimes where should be -export([...]) statement(s) at the top of a source file). On the other hand (without common "export"), we should repeat the "export" keyword every time. As a variation, naming convention, thus the system can understand automatically what should be exported (as an example on Rhino: gist.github.com/363056). But, because of backward compatibility this is not acceptable (although, is convenient).
module Cowboy { export {draw: d, shoot: s}; function d () { ... }; function s () { d(); }; }
FWIW, the rename on import looked "backwards" to me at first glance, but I think I can learn.
It can be done for shortness e.g. or naming preferences of an author (maybe he doesn't like "Math" name ;)). But usually, yes, it is useful "backwards". If some module has been rewritten, then the old one can be named with "old" prefix. Thus, a user shouldn't rename it in all files but just (example from Python's django): "import oldforms as forms" and after that continue to use this "forms" name.
Do we really need to say
.*for all? We couldn't just sayimport Math?
Yep, ".*" seems obsolete, although, the reason is to be consistent I guess.
Another proposal:
import Geometry.{draw: drawShape};
How about:
import Geometry:{draw: drawShape};
colon apt more for "from" word. Another variations:
from Geometry import: {draw: drawShape};
or
from Geometry import: {draw as drawShape, ...};
from Geometry import {draw as drawShape, ...}; // without colon at all
Cons: additional/obsolete "from" and "as" keywords.
P.S.: small off-topic/proposal (just again saw this usage):
module JSON = load('JSON' ||'json.org/modules/json2.js');
How about a small syntactic sugar for useful and widespread construction:
x = x || 10
to remove this repetition from the right side:
x ||= 10.
module JSON ||= load(...);
Dmitry.
How about a small syntactic sugar for useful and widespread construction:
x = x || 10
to remove this repetition from the right side:
x ||= 10.
module JSON ||= load(...);
I do not mean to hijack this thread about what seems a very good modules proposal. But this operator is something I have wished for many times when writing ES code. So it would be a very welcome addition, and the construction is indeed in very widespread use.
Jürg
I wonder if you considered having an export list, rather than tagging the individual exports? I think that makes it easier to see/document the public API of a module. You could at the same time allow renaming on export.
Yes, this is a good point. We chose inline-export for convenience, but I don't see any reason not to allow both.
FWIW, the rename on import looked "backwards" to me at first glance, but I think I can learn.
Yeah, I'm not thrilled about how hard it is to remember which way it goes. I meant for it to be consistent with the syntax of destructuring:
let { draw: d } = obj;
import M.{ draw: d };
Do we really need to say
.*for all? We couldn't just sayimport Math?
The proposal originally left out the '.*' and others didn't like it. But more importantly, this has subtler implications for nested modules, e.g. what the meaning of the following example is:
module Outer { module Inner { ... } }
import Outer.Inner;
The way we've written the proposal, any time you import a path without the '.{...}' syntax, you're importing a single binding. So import Outer.Inner is the same as
import Outer.{Inner};
That is, it binds Inner locally to the Outer.Inner module instance.
Alternatively, we could a) disallow leaving off the '.{...}' for importing a single binding and 'import x1.---.xn' would only be allowed to specify a module-binding and would import all its exports, or b) allow leaving off the '.{...}' but specify that it imports just the single binding when it's a value-binding and imports-all when the path indicates a module-binding. I am a little concerned that the former is too restrictive and the latter too subtle. IMO. the extra '.*' is only a two-character hardship and EIBTI.
Thought looking at the syntax section it seems that
import Math;isn't valid and instead you do it asmodule Math = Math. Why not use import here as well? My first instinct here ismoduleis for defining a module andimportis importing it - insteadmoduleserves a dual function.
That's not quite right; 'module' is for creating a module binding. I.e., all the forms start with 'module m' and create a lexical binding of 'm' to a statically known module.
Is there any reasoning for this behaviour that I might have missed?
The idea is that 'module' creates module bindings and 'import' creates value bindings. So you can create a statically bound module binding via:
module M = MyLib.Math; // M is now a module binding that aliases MyLib.Math
import M.*; // import all bindings from MyLib.Math
whereas 'import' always creates a value binding:
import MyLib.Math; // Math is now a value binding
import Math.*; // error: Math is not a module binding
Since we are allowing dynamic reflection of modules, a goal of this proposal is to keep the distinction between static module bindings and dynamic module values as firm as possible, while still making it as convenient as possible to reflect modules as values.
On May 15, 2010, at 7:53 AM, David Herman wrote:
I wonder if you considered having an export list, rather than
tagging the individual exports? I think that makes it easier to
see/document the public API of a module. You could at the same
time allow renaming on export.Yes, this is a good point. We chose inline-export for convenience,
but I don't see any reason not to allow both.
+1 on an export form that takes a list of already-declared names.
FWIW, the rename on import looked "backwards" to me at first
glance, but I think I can learn.Yeah, I'm not thrilled about how hard it is to remember which way it
goes. I meant for it to be consistent with the syntax of
destructuring:let { draw: d } = obj; import M.{ draw: d };
One has to grok destructuring, but once past that, this is the only
sane way. The shorthand applies.
Alternatively, we could a) disallow leaving off the '.{...}' for
importing a single binding and 'import x1.---.xn' would only be
allowed to specify a module-binding and would import all its
exports, or b) allow leaving off the '.{...}' but specify that it
imports just the single binding when it's a value-binding and
imports-all when the path indicates a module-binding. I am a little
concerned that the former is too restrictive and the latter too
subtle. IMO. the extra '.*' is only a two-character hardship and
EIBTI.
+1, or more -- agree on always requiring .{x} for lone x being too
restrictive, and the subtlety of .x meaning import-all sometimes,
import just x from left-context module other times, is even worse!
EIBTI FTW ;-)
On 2010-05-15, at 11:22, Brendan Eich wrote:
On May 15, 2010, at 7:53 AM, David Herman wrote: [...]
FWIW, the rename on import looked "backwards" to me at first glance, but I think I can learn.
Yeah, I'm not thrilled about how hard it is to remember which way it goes. I meant for it to be consistent with the syntax of destructuring:
let { draw: d } = obj; import M.{ draw: d };
One has to grok destructuring, but once past that, this is the only sane way. The shorthand applies.
Ah, destructing ., I missed that. Perhaps I would have gotten it with a closer parallel like
let { draw: d } = import M;
or
import { draw: d } from M;
Alternatively, we could a) disallow leaving off the '.{...}' for importing a single binding and 'import x1.---.xn' would only be allowed to specify a module-binding and would import all its exports, or b) allow leaving off the '.{...}' but specify that it imports just the single binding when it's a value-binding and imports-all when the path indicates a module-binding. I am a little concerned that the former is too restrictive and the latter too subtle. IMO. the extra '.*' is only a two-character hardship and EIBTI.
+1, or more -- agree on always requiring .{x} for lone x being too restrictive, and the subtlety of .x meaning import-all sometimes, import just x from left-context module other times, is even worse! EIBTI FTW ;-)
I guess .* is there for a purpose: to remind me that it's very likely I don't want to say that. :)
Along these lines of imports, a regex would allow partial imports.
On Sat, May 15, 2010 at 05:20, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com> wrote:
How about a small syntactic sugar for useful and widespread construction:
x = x || 10
Off topic but see harmony:parameter_default_values
On 16.05.2010 1:33, Erik Arvidsson wrote:
On Sat, May 15, 2010 at 05:20, Dmitry A. Soshnikov <dmitry.soshnikov at gmail.com> wrote:
How about a small syntactic sugar for useful and widespread construction:
x = x || 10
Off topic but see harmony:parameter_default_values
Ah, thanks for info, didn't see it (I have to find the time to check all proposal specs).
Dmitry.
I'm happy this is finally receiving some comments. This is at the top of my wish list for ESH.
I do agree that the current syntax seemed a bit off at first glance. Now that Brendan pointed out that it is the destructuring pattern it seems slightly better but still not perfect.
I also agree with others that having an explicit export list would be useful. It is good practice to declare your imports and exports at the top of the file.
On Sat, May 15, 2010 at 09:13, P T Withington <ptw at pobox.com> wrote:
Ah, destructing
., I missed that. Perhaps I would have gotten it with a closer parallel likelet { draw: d } = import M;
I don't like this since it makes it look like (import M) is an expression that returns an object that is later destructured.
or
import { draw: d } from M;
import {drawShape: draw} from Geometry; import {drawGun: draw} from Cowboy; import * from Math;
I like it. I think this one is clearer than the proposed "import Geometry.{drawShape: draw}".
On May 15, 2010, at 2:49 PM, Erik Arvidsson wrote:
On Sat, May 15, 2010 at 09:13, P T Withington <ptw at pobox.com> wrote:
Ah, destructing
., I missed that. Perhaps I would have gotten it
with a closer parallel likelet { draw: d } = import M;
I don't like this since it makes it look like (import M) is an expression that returns an object that is later destructured.
Agreed.
import { draw: d } from M;
import {drawShape: draw} from Geometry; import {drawGun: draw} from Cowboy; import * from Math;
I like it.
The exported name is draw, the rename target is drawShape, so you want
import {draw: drawShape} from Geometry.
Compare destructuring:
let {draw: drawGun} = cowboy; drawGun();
On Sat, May 15, 2010 at 16:47, Brendan Eich <brendan at mozilla.com> wrote:
The exported name is draw, the rename target is drawShape, so you want
import {draw: drawShape} from Geometry.
Compare destructuring:
let {draw: drawGun} = cowboy; drawGun();
See, even after I knew the rules it was too confusing. Can we go back to using "as"?
import draw as drawGun from Cowboy;
On 15.05.2010 19:22, Brendan Eich wrote:
On May 15, 2010, at 7:53 AM, David Herman wrote:
I wonder if you considered having an export list, rather than tagging the individual exports? I think that makes it easier to see/document the public API of a module. You could at the same time allow renaming on export.
Yes, this is a good point. We chose inline-export for convenience, but I don't see any reason not to allow both.
+1 on an export form that takes a list of already-declared names.
Besides, the variation with
export all; // or export "all";
can be considered. It can be useful for debug.
Or, simply -- to omit export statement. Thus, if there will be no local export statement for some function, it means that all functions/properties are exported. Although, it breaks the first design where by default methods are private for a module.
Excluding:
export all except [intrinsic, builder]; // if we don't want to write 20 of 22 methods to be exported
Dmitry.
On May 16, 2010, at 9:32 AM, Dmitry A. Soshnikov wrote:
On 15.05.2010 19:22, Brendan Eich wrote:
On May 15, 2010, at 7:53 AM, David Herman wrote:
I wonder if you considered having an export list, rather than
tagging the individual exports? I think that makes it easier to
see/document the public API of a module. You could at the same
time allow renaming on export.Yes, this is a good point. We chose inline-export for convenience,
but I don't see any reason not to allow both.+1 on an export form that takes a list of already-declared names.
Besides, the variation with
export all; // or export "all";
can be considered. It can be useful for debug.
Debugging is good.
This is a minor point, but rather than all, we'd use * instead, to
mirror import M.* or M.{*} or import * from M; whatever it ends up
being.
Or, simply -- to omit export statement. Thus, if there will be no
local export statement for some function, it means that all
functions/properties are exported. Although, it breaks the first
design where by default methods are private for a module.
Sorry, the idea of implicit everything-exported is a footgun. Just say
no.
Excluding:
export all except [intrinsic, builder]; // if we don't want to write
20 of 22 methods to be exported
This is overdesign. By far the most common case is explicit export of
a select list of API functions and consts.
I am unclear from this proposal. What would happen if I declared the
same module twice? Would it reopen the module and add the extra
declarations?
On 16.05.2010 22:11, Brendan Eich wrote:
On May 16, 2010, at 9:32 AM, Dmitry A. Soshnikov wrote:
On 15.05.2010 19:22, Brendan Eich wrote:
On May 15, 2010, at 7:53 AM, David Herman wrote:
I wonder if you considered having an export list, rather than tagging the individual exports? I think that makes it easier to see/document the public API of a module. You could at the same time allow renaming on export.
Yes, this is a good point. We chose inline-export for convenience, but I don't see any reason not to allow both.
+1 on an export form that takes a list of already-declared names.
Besides, the variation with
export all; // or export "all";
can be considered. It can be useful for debug.
Debugging is good.
This is a minor point, but rather than all, we'd use * instead, to mirror import M.* or M.{*} or import * from M; whatever it ends up being.
Ah, sorry, yes. * on import covers this need, so "all" is obsolete on export.
Excluding:
export all except [intrinsic, builder]; // if we don't want to write 20 of 22 methods to be exported
This is overdesign. By far the most common case is explicit export of a select list of API functions and consts.
Yep, I thought so too after that, so forget about this. The main purpose of a general export at the top of a file is a quick look on what are being exported (and convenience to write the only export statement) and "except" concept from this viewpoint is less informative. Moreover, again * on import eliminates this need.
Dmitry.
On May 16, 2010, at 11:15 AM, Charles Jolley wrote:
I am unclear from this proposal. What would happen if I declared
the same module twice? Would it reopen the module and add the extra
declarations?
The simple modules proposal makes it an error to export twice. This is
a bit implicit in
strawman:simple_modules#export_declarations
That you get an error on duplicate export is a good point to make
explicitly.
Not sure if you meant Dmitry's "export *;" idea. That would export
everything, but again if something was already exported, then there
was already a const binding in the module's exports and you'd get an
error.
I think Charles means examples like:
module Foo {
export var x = 1;
}
module Foo {
export var y = 2;
}
The answer is that this is a static error. Modules are not open and extensible. You can't declare the same module name twice in the same scope. (This should be made clearer in the subsection "Module binding and resolution," thanks for asking about it.)
You can redeclare a new module with the same name at a different level of scope:
module Foo {
export var x = 1;
}
module Bar {
module Foo {
export var y = 2;
}
import Foo.x; // error: no such binding
}
Since modules are bound in the scope chain like everything else, the semantics is just the same as all other lexical scope: the innermost binding of Foo wins.
This is a minor point, but rather than all, we'd use * instead, to mirror import M.* or M.{*} or import * from M; whatever it ends up being.
Agreed.
As for the meaning: from my experience with PLT Scheme it works well for "export *;" to export everything that's defined in the module, but not re-export any imports. For that you'd have to explicitly re-export them. I think either semantics (import "all defined here" vs. "all in scope here") works fine, but the common case of importing is for private use, and re-exporting is the less common case.
On a related note, I forgot to stipulate in the proposal what happens with diamond imports. For example:
module Shared {
export var x = ...
}
module Lib1 {
import Shared.x;
export x;
}
module Lib2 {
import Shared.x;
export x;
}
module Client {
import Lib1.*;
import Lib2.*;
... x ... // error or no error?
}
Because module bindings are static, it's straightforward to determine that both Lib1.x and Lib2.x point to the same binding (Shared.x). So I claim that we should loosen the error condition to the following:
It is a static error to import the same name from two different modules unless both bindings are the same.
Along these lines of imports, a regex would allow partial imports.
I'm a little hesitant on this idea; on the one hand it's nice that we already have literal support for regexps, so it's not a huge conceptual or syntactic overhead. OTOH, like Brendan suggested for "export all except" it smacks of over-design.
See, even after I knew the rules it was too confusing. Can we go back to using "as"?
import draw as drawGun from Cowboy;
This is an incomplete suggestion. What do you want the full syntax to be for multiple imports? Force the user to write a separate one on each line? Bracket them? Comma-separate without bracketing?
Whatever syntax you pick, "as" starts upping the syntactic overhead. The destructuring notation is pleasantly terse and consistent with destructuring.
[FWIW, here's how I would conceptually distinguish the two sides of the ":" token. The LHS is a fixed label. When you write an object literal, you're creating a property with that fixed label. When you are destructuring an object, you are selecting out that fixed label. When you are importing, you are requesting the import with that fixed label. The RHS is the varying part.]
On May 16, 2010, at 1:36 PM, David Herman wrote:
Along these lines of imports, a regex would allow partial imports.
I'm a little hesitant on this idea; on the one hand it's nice that
we already have literal support for regexps, so it's not a huge
conceptual or syntactic overhead.
RegExp literal expressions evaluate to mutable object values, per ES5
and reality (ES3 got this wrong, making an object per literal lexeme
-- not per evaluated expression).
Modules are static. Do we really want to drag in some of the runtime
(RegExps) just to filter imports? This seems not only like over-
design, but also the wrong design.
Stupid question - is the following form of (Python-ish) import possible (from the grammar it doesn't look like it):
<script type="harmony">
import Math; // Math module declared above
alert("2π = " + Math.sum(pi, pi)); </script>
Or is it the case that with a module {...} declaration the module is immediately accessible in the scope it was declared in - and no import required if module.function syntax is used:
<script type="harmony"> // Math module declared above - no import needed
alert("2π = " + Math.sum(pi, pi)); </script>
RegExp literal expressions evaluate to mutable object values, per ES5 and reality (ES3 got this wrong, making an object per literal lexeme -- not per evaluated expression).
Yes, but there's no reason to assume a RegExp token would be a RegExp literal expression.
Modules are static. Do we really want to drag in some of the runtime (RegExps) just to filter imports? This seems not only like over-design, but also the wrong design.
Import/export declarations are distinguished contexts; there's no need to introduce runtime evaluation. A sensible design for import/export declaration with a RegExp would simply be run a regular expression at compile time, with no ES object creation involved.
I'm not advocating the feature, but if we did include it, ISTM the benefit of using regexps is precisely their limited power.
Stupid question - is the following form of (Python-ish) import possible (from the grammar it doesn't look like it):
[snip]
Or is it the case that with a module {...} declaration the module is immediately accessible in the scope it was declared in - and no import required if module.function syntax is used:
<script type="harmony"> // Math module declared above - no import needed
alert("2π = " + Math.sum(pi, pi)); </script>
The latter, yes. Since modules are bound in the scope chain, they're already available for use as values.
This makes it very convenient to reflect them as first-class values:
var x = Math;
alert(x["sum"]); // function(x,y) { ... }
alert(x["thisIsNotDefined"]); // undefined
But since module instance objects are immutable, it's also easy for implementations to make static uses, such as your example above, as efficient as if they had been explicitly imported.
On May 16, 2010, at 7:18 PM, David Herman wrote:
RegExp literal expressions evaluate to mutable object values, per
ES5 and reality (ES3 got this wrong, making an object per literal
lexeme -- not per evaluated expression).Yes, but there's no reason to assume a RegExp token would be a
RegExp literal expression.
The reader of the source might assume exactly that. One regexp looks
like another, and might be expected to quack like any other.
Modules are static. Do we really want to drag in some of the
runtime (RegExps) just to filter imports? This seems not only like
over-design, but also the wrong design.Import/export declarations are distinguished contexts; there's no
need to introduce runtime evaluation. A sensible design for import/ export declaration with a RegExp would simply be run a regular
expression at compile time, with no ES object creation involved.
That's possible, and desirable in practice: regexps have effects in
real-world implementations (the unspecified but still used RegExp.$_,
RegExp.$1, etc. properties). If written with /y or /g they mutate
there own lastIndex property and use it for subsequent matches (how
many times is /.*.js$/g evaluated in that import directive?).
I'm not advocating the feature, but if we did include it, ISTM the
benefit of using regexps is precisely their limited power.
You'd have to limit that power from what it is today in terms of
effects (including those pesky de-facto standard effects).
Another unwanted power is that these NFAs misnamed "regular
expressions" can blow up with O(exp(n)) complexity due to
backtracking. This is enough of a problem that we're about to try
throwing an exception when JS regexp exec complexity measured in terms
of active backtrack states compared to target string length seems to
be worse than O(n^3) (see bugzilla.mozilla.org/show_bug.cgi?id=452451
and bugs it cites).
Making some tamed, quasi-regexp-just-for-import seemed not what Kam
proposed, but anyway undesirable on its face. I mean, the taming
attempt is desirable ignoring the big picture, but the whole idea
seems misbegotten. My two cents.
On Mon, May 17, 2010 at 3:27 AM, David Herman <dherman at mozilla.com> wrote:
This makes it very convenient to reflect them as first-class values:
var x = Math; alert(x["sum"]); // function(x,y) { ... } alert(x["thisIsNotDefined"]); // undefined
Whats the difference between: var x = Math; // or: const x = Math;
And: module x = Math;
Thanks.
Thanks Brendan, David. A few comments below.
This was the point I was explaining here:
https://mail.mozilla.org/pipermail/es-discuss/2010-May/011162.html
Modules are static entities, whose structure is known at compile-time. The "module" form creates these static bindings. But they can be reflected as first-class values. (Similar to e.g. classes in Java and proposed ES4.)
The "import" form, like "var" and "const" and "function" creates a binding to a first-class value, whereas the "module" form creates a static module binding.
So when you write:
var x = Math;
you create a variable binding to the reflected first-class module value. Whereas when you write:
module x = Math;
you are creating a static module binding x which is bound to the static module bound to Math.
On Mon, May 17, 2010 at 6:02 AM, David Herman <dherman at mozilla.com> wrote:
This was the point I was explaining here:
Modules are static entities, whose structure is known at compile-time. The "module" form creates these static bindings. But they can be reflected as first-class values. (Similar to e.g. classes in Java and proposed ES4.)
The "import" form, like "var" and "const" and "function" creates a binding to a first-class value, whereas the "module" form creates a static module binding.
So when you write:
var x = Math;
you create a variable binding to the reflected first-class module value. Whereas when you write:
module x = Math;
you are creating a static module binding x which is bound to the static module bound to Math.
OK. On the simple_modules_examples wiki page:
Reflecting module instances as first-class objects
<script type="harmony"> // a static module reference module M = Math;
// reify M as an immutable "module instance object" alert("2π = " + M.sum(M.pi, M.pi)); </script>
Q: Why does referencing the module M's functions/constants - M.sum and M.pi - 'reify' M as an object.(Is this the same process as generating a 'reflected first-class module value'?)
I get the difference between lexical/source vs first class - but this comment 'reify' genuinely confuses me. Why isn't the call to M.sum() static - known at compile time.
Yeah, sorry, that example went through several iterations as the modules proposal evolved, and now it's kind of muddled. I'll update it tomorrow.
Since the reference to the module is used in an expression, it could technically be thought of as reflecting the module as an object and pulling out its property. But the system is designed so that such uses can be guaranteed to have the same performance as if "sum" and "pi" were explicitly imported. I'll clarify that example and add another example that's truly dynamic, such as:
function inspect(m) {
for (key in m) {
if (m.hasOwnProperty(key))
alert("found export: " + key);
}
}
inspect(Math);No worries - the examples page is very useful.
In your original email: module Even = "example.com/even.js"; module Odd = "example.com/odd.js";
Is the load keyword missing: module Even = load "example.com/even.js";
A couple more questions:
In strawman:simple_modules_examples - Remote modules on the web (1):
<script type=”harmony”> // loading from a URL module JSON = load 'json.org/modules/json2.js';
alert(JSON.stringify({'hi': 'world'})); </script>
- Does - or must - the file json2.js contain only a single top level module declaration:
module JSON { ...source... }
(Can js source files contain multiple top level modules?)
- If so, is the load statement equivalent to:
<script type="harmony" src="json.org/modules/json2.js"></script>
Apart from:
- The script tag will make the module globally available to subsequent script tags. Whereas, the load form makes the module available only within the script tag load is called from.
- the load form can parallelize the fetching and compilation of source more efficiently.
David Herman wrote:
Hello!
I've updated the strawman proposals for static modules and dynamic module loaders and would love to get feedback from the es-discuss community.
Static modules:
strawman:simple_modules_examples
Hi David, and thanks for putting in this work. I have a few questions below.
Cyclic dependencies
On Mon, May 17, 2010 at 12:37 AM, Kam Kasravi <kamkasravi at yahoo.com> wrote:
[kam] An example might be something like SVG.Filter where the importer was interested in retrieving only filter related features within a SVG module.
For this, I would rather let the exporter define named export lists, so that there's better future proofing. I would hate to be the person adding something to the SVG module knowing that there were a bunch of regexps floating around the web that might cause me to clobber something unexpected.
Given Foo.* meaning what it will, I suspect we would see people writing exactly what you did above, and being surprised that they didn't get what they wanted: they want globs, not regexps, for that. ("Filter" isn't even a legal regexp, so they'd get a compilation error, but when you have to write SVG..Filter., I think it gets to be pretty unwieldy.)
Mike
No worries - the examples page is very useful.
In your original email: module Even = "example.com/even.js"; module Odd = "example.com/odd.js";
Is the load keyword missing: module Even = load "example.com/even.js";
Yes, sorry for another inconsistency there. FWIW, I'm not married to that particular syntax. I put the "load" keyword into the proposal just to emphasize that these strings correspond to a compile-time action (loading the bits). For example, consider:
module M = load "http://example.com/foo.js";
module N = load "http://example.com/foo.js";
These two modules are loaded from the exact same MRL, but the module system treats them as two completely separate, independent modules. (Even custom module loaders shouldn't be able to change this fact, since all they can do is deliver the bits of a module resource.)
The reason for this is that on the web, there's just no way to know that two URL's point to the "same" resource-- fetching the bit-for-bit-same URL even microseconds apart can result in completely different data, since web servers are free to deliver whatever they want. So the module loading semantics is resolutely non-clever about interpreting MRL's.
module JSON = load 'json.org/modules/json2.js';
- Does - or must - the file json2.js contain only a single top level module declaration:
module JSON { ...source... }
No. The file json2.js contains the body of the JSON module. It can declare as many modules, variables, and functions as it likes.
This is less brittle (doesn't require careful coordination between client and library), and crucially it gives the client control over module naming, so that if two vendors produce a module called SuperWidgets, then the client can do:
module Acme = "http://acme.com/superwidgets.js";
module Haxx0rz = "http://haxx0rz.com/superwidgets.js";
import Acme.SuperWidgets.foo;
import Haxx0rz.SuperWidgets.bar;
// ...
It's also more convenient for library writers, who don't have to wrap their entire top-level source file with an annoying boilerplate module declaration.
- If so, is the load statement equivalent to:
<script type="harmony" src="json.org/modules/json2.js"></script>
Apart from:
- The script tag will make the module globally available to subsequent script tags. Whereas, the load form makes the module available only within the script tag load is called from.
- the load form can parallelize the fetching and compilation of source more efficiently.
That's mostly true, except for the fact that the <script> tag doesn't wrap the contents in a module. Thanks for bringing this up, it's a good point. It might suggest an extra HTML attribute to go that last step:
<script type="harmony" module="JSON" src="http://json.org/modules/json2.js"></script>
(Which is of course, out of our jurisdiction to specify here, but coordination with other web standards will at some point be necessary, and it's helpful to have at least plausible recommendations.)
However, I believe it should also still be possible for browsers to prefetch and parallelize the compilation of source from within module declarations with MRL's, too. This is something we need to experiment with in real implementations, though.
For this, I would rather let the exporter define named export lists, so that there's better future proofing. I would hate to be the person adding something to the SVG module knowing that there were a bunch of regexps floating around the web that might cause me to clobber something unexpected.
This gets to my main issue with regexps (aside from the YAGNI aspect) -- it puts too much significance into variable names, and becomes a refactoring hazard.
("Filter" isn't even a legal regexp, so they'd get a compilation error, but when you have to write SVG..Filter., I think it gets to be pretty unwieldy.)
FWIW, you could distinguish this with the regexp syntax, for example:
import SVG./.*Filter.*/;
import /.*Filter.*/ from SVG;
But ... um, well, yeah.
Thanks Mike, David.
I agree with the conclusion that regex probably isn't the best way to tackle these problems. I'll summarize the use cases since it may apply in later discussions...
- Is it possible to import parts of a module to avoid potentially large network payloads?
- Is there syntax that would allow the server to group or concatenate delivery of multiple modules within one request?
thanks Kam
From: David Herman <dherman at mozilla.com>
To: Mike Shaver <mike.shaver at gmail.com>
Cc: Kam Kasravi <kamkasravi at yahoo.com>; Brendan Eich <brendan at mozilla.com>; P T Withington <ptw at pobox.com>; es-discuss Steen <es-discuss at mozilla.org>
Sent: Mon, May 17, 2010 8:22:52 AM Subject: Re: modules proposal
For this, I would rather let the exporter define named export lists, so that there's better future proofing. I would hate to be the person adding something to the SVG module knowing that there were a bunch of regexps floating around the web that might cause me to clobber something unexpected.
This gets to my main issue with regexps (aside from the YAGNI aspect) -- it puts too much significance into variable names, and becomes a refactoring hazard.
("Filter" isn't even a legal regexp, so they'd get a compilation error, but when you have to write SVG..Filter., I think it gets to be pretty unwieldy.)
FWIW, you could distinguish this with the regexp syntax, for example:
import SVG./.*Filter.*/;
import /.*Filter.*/ from SVG;
But ... um, well, yeah.
On May 17, 2010, at 10:30 AM, Kam Kasravi wrote:
Thanks Mike, David.
I agree with the conclusion that regex probably isn't the best way
to tackle these problems. I'll summarize the use cases since it may apply in later
discussions...
- Is it possible to import parts of a module to avoid potentially
large network payloads?
It's modules all the way down. You only import what you specify from
the module, but that doesn't affect downloading or indeed express the
module dependency -- the dependency comes from module declarations
creating bindings first, before any import from the module via its
bound name.
Network payloads are induced by URL requests that miss the HTTP-rules-
based cache. MRLs are not URLs, however, so there's an opportunity
here to coalesce load requests. See below.
- Is there syntax that would allow the server to group or
concatenate delivery of multiple modules within one request?
This is a good question. The grouping of modules for download should
be decoupled from how one declares modules, to avoid mixing concerns
and requiring refactoring or rewriting just on account of repackaging.
Module declarations allow ahead of runtime loading, but they don't
help coalesce requests.
Coalescing requests (for some value of request) could be pushed down a
layer, not specified as part of the ECMA-262 language. This serves the
decoupling requirement. Is it enough? A while ago Allen wrote about
separating "configuration management":
Part of this was about version selection, but part of it was about
"assembly" structuring. It seems worth working through this, both with
a real implementation of simple modules, and with some thought
experiments.
- Is it possible to import parts of a module to avoid potentially large network payloads?
Not parts of a module, no. (The semantics of this would be incredibly hairy and hard to define.) But you can dynamically load modules, so you can write code that decides dynamically when to load modules.
- Is there syntax that would allow the server to group or concatenate delivery of multiple modules within one request?
Since modules can be nested, you can certainly put multiple modules in a file; grouping them in a single module is nothing more than namespace management. So you can write:
// file1.js
module A { ... }
module B { ... }
...
module Z { ... }
// client.html
<script type="harmony">
module MyLib = load "file1.js";
</script>
or with HTML support, something like:
// client.html
<script type="harmony" module="MyLib" src="file1.js"></script>Coalescing requests (for some value of request) could be pushed down a layer, not specified as part of the ECMA-262 language. This serves the decoupling requirement. Is it enough? A while ago Allen wrote about separating "configuration management":
Part of this was about version selection, but part of it was about "assembly" structuring. It seems worth working through this, both with a real implementation of simple modules, and with some thought experiments.
Yes, certainly. To some extent the module declaration syntax provides you enough to do this already:
<!-- my configuration table -->
<script type="harmony">
module M1 = "m1.js";
module M2 = "m2.js";
...
</script>
But it might be nice to crystallize this in a restricted special form so browsers can reliably prefetch:
<script type="harmony">
module table {
M1: "m1.js",
M2: "m2.js",
// ...
}
</script>On May 17, 2010, at 12:12 PM, David Herman wrote:
Coalescing requests (for some value of request) could be pushed
down a layer, not specified as part of the ECMA-262 language. This
serves the decoupling requirement. Is it enough? A while ago Allen
wrote about separating "configuration management":esdiscuss/2010-January 010686.html
Part of this was about version selection, but part of it was about
"assembly" structuring. It seems worth working through this, both
with a real implementation of simple modules, and with some thought
experiments.Yes, certainly. To some extent the module declaration syntax
provides you enough to do this already:<!-- my configuration table --> <script type="harmony"> module M1 = "m1.js"; module M2 = "m2.js"; ... </script>But it might be nice to crystallize this in a restricted special
form so browsers can reliably prefetch:<script type="harmony"> module table { M1: "m1.js", M2: "m2.js", // ... } </script>
Either way allows prefetching once the type="harmony" script content
has been parsed. But the browser still would have to stall rendering
while that "module table" script tag was being processed, as far as I
can tell. Any later scripts would block too, to avoid using a not-yet-
loaded module binding.
Oh, but you probably meant that the module table form, besides being
sugar, is written in a restricted language that cannot have effects
other than to create module bindings -- cannot do document.write or
document.createElementNS("script") or whatever. In that case we'd want
type="harmony-module-table" or some such, and then such a script
indeed would allow layout to proceed immediately, and not block
rendering.
Thinking about it more, simple modules let authors bundle things
in .js files, and src them with scripts. That's almost enough.
Anything more, we do not want to standardize prematurely. Simple
modules are really about lexical scope all the way up, and guaranteed
errors (early errors, even), and static code partitioning with
information hiding, and of course the lexical-only module-binding
namespace management.
Which is all good. Prefetching and packaging are separable issues, we
should keep thinking about them but I'd hate to try to "solve" them
before considering simple modules as a proposal.
Oh, but you probably meant that the module table form, besides being sugar, is written in a restricted language that cannot have effects other than to create module bindings -- cannot do document.write or document.createElementNS("script") or whatever. In that case we'd want type="harmony-module-table" or some such, and then such a script indeed would allow layout to proceed immediately, and not block rendering.
Yes, sorry for the mixup. I should've written something like:
<script type="harmony-module-configuration-table">
{ M1: "m1.js",
M2: "m2.js",
// ...
}
</script>
Thinking about it more, simple modules let authors bundle things in .js files, and src them with scripts. That's almost enough. Anything more, we do not want to standardize prematurely.
Agreed.
Simple modules are really about lexical scope all the way up, and guaranteed errors (early errors, even), and static code partitioning with information hiding, and of course the lexical-only module-binding namespace management.
Well put.
On Mon, May 17, 2010 at 4:07 PM, David Herman <dherman at mozilla.com> wrote:
[snip] For example, consider:
module M = load "example.com/foo.js"; module N = load "example.com/foo.js";
These two modules are loaded from the exact same MRL, but the module system treats them as two completely separate, independent modules. (Even custom module loaders shouldn't be able to change this fact, since all they can do is deliver the bits of a module resource.)
The reason for this is that on the web, there's just no way to know that two URL's point to the "same" resource-- fetching the bit-for-bit-same URL even microseconds apart can result in completely different data, since web servers are free to deliver whatever they want. So the module loading semantics is resolutely non-clever about interpreting MRL's.
Dave
That's surprising. Within a moduleloader I would have thought that same url meant the same static module. Across moduleloaders maybe not.
For the following scenario: <script>
module ModA = "acme.com/moda.js"; module ModB = "acme.com/modb.js"; ... source </script>
Both ModA and ModB use a utility module - that is the moda.js and modb.js files both contain: module ModUtils = "widgets.com/modutils.js";
I would have guessed that ModA and ModB are using the exact same static utility module. Firstly for efficiency - "hey this source has been fetched and compiled already".
But also, simple modules seem like 'traditional' modules. That is, if a top level variable is declared in a C source file, that variable occurs once in the resulting compiled image/program. Thus the source files (modules) author can reason about the (shared state) of the variable.
However, if an ES module source can be declared multiple times (within a moduleloader) - doesn't it become harder for the author of the module to reason about a top level variable. If multiple instance of a module variable are required - then maybe an object should be used.
Also, it seems unlikely that a new module source would be generated by a server within the context of a moduleloader fetching/compiling source. Maybe the rule should be - first source fetched for a url is the source used for subsequent module references.
On Wed, May 19, 2010 at 2:59 AM, Kevin Curtis <kevinc1846 at gmail.com> wrote:
For the following scenario: <script> module ModA = "acme.com/moda.js"; module ModB = "acme.com/modb.js"; ... source </script>
Both ModA and ModB use a utility module - that is the moda.js and modb.js files both contain: module ModUtils = "widgets.com/modutils.js";
I would have guessed that ModA and ModB are using the exact same static utility module. Firstly for efficiency - "hey this source has been fetched and compiled already".
The problem with this is that the web is not stable enough. First, is it based on the string in the URL? If so, then is "Widgets.com" the same as "widgets.com"? What about "www.widgets.com"? What if the url resolves to a different IP each time? Or if the source changes over time?
We feel that it's a bad idea to try to use addresses on the web to determine identity of modules. Instead, we based the naming system on lexical scope, and let the programmer manage resources on the web. If you want to share a common library with your context, then you refer to it by name, and the context can provide it the module under that name.
That's surprising. Within a moduleloader I would have thought that same url meant the same static module. Across moduleloaders maybe not.
The problem is defining "same url." One option is that "same" means "identical string" but then when example.com/foo.html says:
<script>
module A = "http://example.com/lib.js";
module B = "./lib.js";
</script>
the programmer would be tempted to think those are the same URL. And yet a server has access to its request URL and can deliver entirely different bits depending on exactly what was requested.
So you could propose some sort of canonicalization that happens on the client side, and say "same" means "same canonicalized string." But then the semantics depends on some non-trivial algorithm that happens away from view of the programmer.
For the following scenario: <script> module ModA = "acme.com/moda.js"; module ModB = "acme.com/modb.js"; ... source </script>
Both ModA and ModB use a utility module - that is the moda.js and modb.js files both contain: module ModUtils = "widgets.com/modutils.js";
If they want to share a utility module, you pull it out into a place that's in scope for both ModA and ModB:
<script>
module ModUtils = load "http://acme.com/modutils.js";
module ModA = load "http://acme.com/moda.js";
module ModB = load "http://acme.com/modb.js";
...
</script>
But also, simple modules seem like 'traditional' modules. That is, if a top level variable is declared in a C source file, that variable occurs once in the resulting compiled image/program. Thus the source files (modules) author can reason about the (shared state) of the variable.
I'm surprised to here you cite C as a precedent for modules, since C has no module system. All it has is #include, which is much harder to work with. The "load" semantics does still have the hazard of re-loading the same source (just as <script> does, BTW), but the scope is more tightly controlled; a loaded module is only sensitive to the modules in scope (no var-, let-, const- or function-bindings).
On Wed, May 19, 2010 at 3:04 PM, David Herman <dherman at mozilla.com> wrote:
If they want to share a utility module, you pull it out into a place that's in scope for both ModA and ModB:
<script> module ModUtils = load "acme.com/modutils.js"; module ModA = load "acme.com/moda.js"; module ModB = load "acme.com/modb.js"; ... </script>
Ahh - thanks for this.
So both the moda.js and modb.js source files can contain (for example): ModUtils.myfunc();
And can import the exports of ModUtils: import ModUtils.myfunc; myfunc();
Also, a script author can configure which modules they will use: For example use modules ModA and ModB - but not say - ModC and ModD. <script>
module Acme { module ModUtils = load "acme.com/modutils.js"; module ModA = load "acme.com/moda.js"; module ModB = load "acme.com/modb.js"; // don't load the source files modc.js and modd.js - not used } Acme.ModA.myfunc(); Acme.ModB.myfunc2(); ... </script>
Is it correct that a module declaration within a script tag only has scope within that script tag?
I'm surprised to here you cite C as a precedent for modules, since C has no module system.
That would be a surprise! No just the lexical concept.
So both the moda.js and modb.js source files can contain (for example): ModUtils.myfunc();
And can import the exports of ModUtils: import ModUtils.myfunc; myfunc();
Yes.
Is it correct that a module declaration within a script tag only has scope within that script tag?
No, each script is in scope for subsequent scripts. (The section "Top Level" in the strawman is the relevant part.) One of the enhancements to the current proposal, which we've considered but not worked out in detail, would be the ability to create private modules.
sorry I overlooked this message.
| This allows cyclic dependencies between any two modules | in a single scope
- Is this to say that cycles are allowed, or not allowed, in other scenarios? (f ex remote or filesystem-loaded modules)
It's an automatic consequence of lexical scope. It's no different from functions. If you write:
function f() {
function even(n) { ... odd(n-1) ... }
function g() {
function odd(n) { ... even(n-1) ... }
}
}
you get a scope error, because |even| refers to |odd| which is not in scope. But if you put them both at the same level of scope, they can refer to each other. Same deal with modules.
Since MRL's and module names are distinct, there's no problem letting remote modules refer to one another, they just have to do so through agreed-upon names. For example:
module Even = "http://zombo.com/even.js";
module Odd = "http://realultimatepower.net/odd.js";
By binding them both at the same level of scope, both even.js and odd.js can refer to Even and Odd.
- Am I understanding this correctly as the module loader fetching and registering Lexer.js "on demand" if not already present in its registry mapping?
The offline-JS examples are mostly just suggestive; this question is left to the (host-dependent) module loader to determine. Some offline-JS engines might prefer to have a module loader that can register top-level names implicitly. It might also want to load these modules on demand (which is somewhat orthogonal), but this would likely only be desirable behavior for built-in libraries that have no observable side effects. (Laziness with side-effects is pain.)
But all of this would require some host-dependent, built-in support; there's nothing in the spec per se that provides this functionality.
- Is Lexer.js required to contain exactly one module declaration that must match the filename, or otherwise an exception is thrown?
The short answer is no: the contents are the body of the module.
The long answer is that it depends on the module loader. Module loaders have hooks that can arbitrarily transform the contents of a module. So a module loader can decide on whatever format it wants.
Note that when you say "an exception is thrown," this is a compile/load-time exception. IOW, in the code being loaded, there's no observable exception. But of course, with dynamic loading, one program's compile-time error is another program's run-time error. :)
- Would this on demand loading functionality be present also on the web platform?
No. We avoided lazy loading in the semantics, because it'd be a hornets' nest on the web. Modules are eagerly evaluated in deterministic, top-to-bottom order. As I say above, offline-JS has different needs than web-JS, so in that context a JS engine may wish to provide a built-in custom module loader that can deal with the filesystem in more clever ways.
As I replied to Kam Kasravi earlier in this thread, on-demand loading is achievable via explicit dynamic loading. We tried to be non-clever about doing this stuff behind programmers' backs.
- If so, would then the uses in main1 and main2 be equivalent wrt binding MyLib names in the example below?
// lib/MyLib.js module MyLib { export doStuff() ... }
// main1.js import lib.MyLib.*; doStuff();
// main2.js module wrapper = load "lib/MyLib.js"; import wrapper.MyLib.*; doStuff();
No. There'd be no implicit loading on the web.
- If I want to change the previous example to instead expose MyLib's names inside a MyLib module name, would I then do the following? :
// main1.js module MyLib = lib.MyLib; MyLib.doStuff();
// main2.js module wrapper = load "lib/MyLib.js"; module MyLib = wrapper.MyLib; MyLib.doStuff();
I'm not sure I see what you're getting at, but IIUC, both of those examples are fine. You can always declare a new module binding that points to/aliases an existing module binding.
In the "Remote modules on the web (1)" example we have:
| module JSON = load 'json.org/modules/json2.js'; | alert(JSON.stringify({'hi': 'world'}));
- Am I understanding correctly that this is pointing to a plain script (without module decls) which is wrapped inside the JSON module we specify?
It's a module body, which may contain nested module declarations, variable declarations, function declarations, statements, etc.
- Imagine a json2mod.js with an embedded:
module JSON { ... }
which would result in:
module JSON = load 'json.org/modules/json2mod.js'; alert(JSON.JSON.stringify({'hi': 'world'}));
Is this assumption correct?
Yes, but you wouldn't wanna do that. ;)
- If so, what syntax could be used to avoid the extra wrapper module at load?
The creator of the JSON library doesn't name it; the client names it. The JSON module just contains the body of the module. This is crucial for the web: you can have millions of libraries without worrying about library name collisions.
Could there f ex be a standalone load syntax that evaluates straight into the current scope:
load 'json.org/modules/json2mod.js'; alert(JSON.stringify({'hi': 'world'}));
You can get partway there via:
module JSON = load 'http://json.org/modules/json2mod.js';
import JSON.*;
But yeah, it might be nice to have a shorthand that avoids the extra module binding. (I'd probably want the syntax to start with "import" or "module").
(disclaimer: haven't thought much about implications of this)
Me neither... I'll have to think about it.
In the "Static module detection" example we have:
| // compiler tries each in order | module JSON = load ('JSON' || 'json.org/modules/json2.js');
- Does this syntax mix module identifiers and MRLs? (the 'JSON' item looks very much like a pre-registered module in the module registry?)
No, I was just positing some syntax for built-in MRL's. Maybe I should've written something like
module JSON = load ('browser://JSON' || 'http://json.org/modules/json2.js');
but I didn't want anyone to think I was making some specific proposal for a new URL scheme or anything. This is one of those things that stands outside the jurisdiction of TC39 but would still want standardization. I think we're going to need some amount of (hopefully light) collaboration with a W3C committee.
- [Excuse my poor BNF] What is the +(',') for in:
| ModuleDeclaration ::= 'module' Identifier '=' 'load' MRL+(',') ';'
Heh-- excuse my bogus BNF! That's just a made-up extension that allows 1 or more instances of MRL, separated by the ',' token. Spelled out:
MRL+(',') ::= MRL
| MRL+(',') ',' MRL
Thanks for your feedback,
Heh-- excuse my bogus BNF! That's just a made-up extension that allows 1 or more instances of MRL, separated by the ',' token. Spelled out:
MRL+(',') ::= MRL | MRL+(',') ',' MRL
PS If this is still unclear, just replace MRL+(',') with MRLList, and add the production:
MRLList ::= MRL
| MRLList ',' MRL
I'll do that in the spec to avoid this confusion in the future.
Thanks for the detailed explanations, David. Here's a few follow- up questions (with "done" questions snipped out):
David Herman wrote:
| This allows cyclic dependencies between any two modules | in a single scope
- Is this to say that cycles are allowed, or not allowed, in other scenarios? (f ex remote or filesystem-loaded modules)
<snip>
Since MRL's and module names are distinct, there's no problem letting remote modules refer to one another, they just have to do so through agreed-upon names. For example:
module Even = "http://zombo.com/even.js"; module Odd = "http://realultimatepower.net/odd.js";By binding them both at the same level of scope, both even.js and odd.js can refer to Even and Odd.
Agreed, but I was rather wondering about cases like:
even.js module Odd = "odd.js"; ... odd.js module Even = "even.js"; ...
On Thu, May 20, 2010 at 3:47 PM, Mike Wilson <mikewse at hotmail.com> wrote:
Thanks for the detailed explanations, David. Here's a few follow- up questions (with "done" questions snipped out):
Thanks for all your detailed feedback!
David Herman wrote:
| This allows cyclic dependencies between any two modules | in a single scope
- Is this to say that cycles are allowed, or not allowed, in other scenarios? (f ex remote or filesystem-loaded modules)
<snip>
Since MRL's and module names are distinct, there's no problem letting remote modules refer to one another, they just have to do so through agreed-upon names. For example:
module Even = "zombo.com/even.js"; module Odd = "realultimatepower.net/odd.js";
By binding them both at the same level of scope, both even.js and odd.js can refer to Even and Odd.
Agreed, but I was rather wondering about cases like:
even.js module Odd = "odd.js"; ... odd.js module Even = "even.js"; ...
From other discussion I'm guessing this kind of circular referencing will not be supported?
Right. If this was allowed, then we'd have the infinite chain of modules Even.Odd.Even.Odd.Even and so on.
- Is Lexer.js required to contain exactly one module declaration that must match the filename, or otherwise an exception is thrown?
The short answer is no: the contents are the body of the module.
The long answer is that it depends on the module loader. <snip>
I'm still a little confused by the rules here, but assuming a naive loader implementation, what would the following result in? (bonus points for describing how you envision a made-up setup process):
// compiler/Lexer.js module Lexer { ... } module Tokenizer { ... } ... // Main.js import compiler.Tokenizer.*;
First, I want to emphasize again that our ideas about offline JS are preliminary, and not intended for standardization. Different host environments can make different choices here. But on this example, we're envisioning that Main.js would import compiler.Lexer.Tokenizer. Or perhaps there would only be one module per file when implicitly loading modules from the file system, but that would probably be needlessly restrictive.
In the "Remote modules on the web (1)" example we have:
| module JSON = load 'json.org/modules/json2.js'; | alert(JSON.stringify({'hi': 'world'}));
- Am I understanding correctly that this is pointing to a plain script (without module decls) which is wrapped inside the JSON module we specify?
It's a module body, which may contain nested module declarations, variable declarations, function declarations, statements, etc.
I don't find a definition for "module body" in your strawman. Are you meaning that json.org/modules/json2.js above is organized as:
function stringify... function parse...
or as
export function stringify... export function parse... (although this is invalid according to the grammar)
The latter. We should think of the contents of json2.js as an anonymous module, which the client names using 'load'.
In the "Static module detection" example we have:
| // compiler tries each in order | module JSON = load ('JSON' || 'json.org/modules/json2.js');
- Does this syntax mix module identifiers and MRLs? (the 'JSON' item looks very much like a pre-registered module in the module registry?)
No, I was just positing some syntax for built-in MRL's. Maybe I should've written something like
module JSON = load ('browser://JSON' || 'json.org/modules/json2.js');
but I didn't want anyone to think I was making some specific proposal for a new URL scheme or anything.
Two random thoughts on this:
- I think it would be suitable with some special marker when addressing built-ins, as module JSON = load "JSON"; might as well be used for relative addressing to the ./JSON application URL, which should issue an external request.
I agree, it's probably confusing for "JSON" and "./JSON" to mean radically different things. But staking out URL space for this seems heavyweight and ugly. Is there a good middle ground?
- For the web it is probably not interesting to do this resolution for anything else than built-ins and one external URL. Providing a search path with multiple external URLs would mean depending on hitting 404s as part of the load phase, which is usually frowned upon.
I think this is right in terms of how people should program on the web, but I'm not sure that it should be standardized. Only allowing one external URL requires making a determination of what's external and what's built-in, and any one decision might not necessarily be right for every host environment.
I've been following the modules strawman proposals from the sidelines and I like many properties of this one. Can you clarify the following use case (inspired in part by the discussion of programming in the small on the earlier thread):
Suppose a.js and b.js use jquery and they import it from the containing modules context.
module Foo { module JQ = load 'http://....'; module Drawing = load 'http://.../gun.js'; // gun.js exports gun shooting method "draw" module A = load 'http://.../a.js'; // a.js imports JQ and Drawing module B = load 'http://.../b.js'; // b.js imports JQ }
Each of A and B get the shared frozen (but not transitively frozen) JQ and all is well. Module A also needs the gun drawing module Drawing which it uses to draw and shoot guns. Unbeknownst to Foo (since the contract between Foo and B isn't explicit) and the internet is in flux, B also starts importing Drawing expecting a GUI drawing method.
module Foo { module JQ = load 'http://....'; module Drawing = load 'http://.../gun.js'; module A = load 'http://.../a.js'; module B = load 'http://.../b.js'; // b.js imports JQ AND Drawing expecting something that draws graphics }
Since there happens to be a Drawing module floating around, it gets handed that module and sadness happens instead of an error message.
The problem occurs because although the scopes are lexical, the textual content of the load call - the content that is pulled into the current scope is not under the control of the author of Foo. The modules that A or B uses and has access to are not apparent to the author of Foo. As a result, the problems are similar to the problems you suffer from with dynamic rather than lexical scopes.
I can see at least two ways the simple modules proposal can handle the problem. One is to recognize that a contract between two modules has two parts - what the module requests (in the case of A, JQ and Drawing) and what the container module provides (in the case of A, JQ and Drawing and in the case of B, JQ only). Currently the imports of a module are explicit but the objects that the container provides are implicit. One solution is for the syntax of load to be explicit about what it is providing to the module being loaded (crappy syntax aside):
module Foo { module JQ = load 'http://....'; module Drawing = load 'http://.../gun.js'; module GUI = load 'http://.../gui.js';
module A = load 'http://.../a.js', {JQ: JQ, Drawing: Drawing}; // in module A, uttering Drawing gets you Drawing module B = load 'http://.../b.js', {JQ: JQ, Drawing: GUI }; // in module B, uttering Drawing gets you GUI }
The other is to limit the scope from which the imported modules imports get selected to the current scope:
module Fribble {}
module Foo { module JQ = load 'http://....'; module Drawing = load 'http://.../gun.js'; module A = load 'http://.../a.js'; }
module Bar { module JQ = load 'http://....'; module Drawing = load 'http://.../gui.js'; module B = load 'http://.../b.js'; // module B gets JQ and Drawing but it can't utter something to get it the shared Fribble }
This problem can probably be ignored for systems designed for programming-in-the-small.
Jasvir
2010/5/20 ๏̯͡๏ Jasvir Nagra <jasvir at gmail.com>:
The other is to limit the scope from which the imported modules imports get selected to the current scope: module Fribble {} module Foo { module JQ = load 'http://....'; module Drawing = load 'http://.../gun.js'; module A = load 'http://.../a.js'; } module Bar { module JQ = load 'http://....'; module Drawing = load 'http://.../gui.js'; module B = load 'http://.../b.js'; // module B gets JQ and Drawing but it can't utter something to get it the shared Fribble }
Two things about this. This is the recommended solution to this problem in our proposal, since it eliminates accidental access to the Drawing module. But note that all modules have access to Fribble, since it's in the scope chain. Also, what you've written loads the JQ module twice - probably you want to only load that once, and have it shared.
2010/5/20 Sam Tobin-Hochstadt <samth at ccs.neu.edu>
2010/5/20 ๏̯͡๏ Jasvir Nagra <jasvir at gmail.com>:
The other is to limit the scope from which the imported modules imports get selected to the current scope: module Fribble {} module Foo { module JQ = load 'http://....'; module Drawing = load 'http://.../gun.js'; module A = load 'http://.../a.js'; } module Bar { module JQ = load 'http://....'; module Drawing = load 'http://.../gui.js'; module B = load 'http://.../b.js'; // module B gets JQ and Drawing but it can't utter something to get it the shared Fribble }
Two things about this. This is the recommended solution to this problem in our proposal, since it eliminates accidental access to the Drawing module.
Cool - we agree that the problem is one which the modules proposal system needs to solve. I am not sure your recommended solution addresses the accidental access problem. Here's a slightly clearer example of the issue:
zero.js: module One = load 'one.js'; module Drawing = load 'gun.js'; module JQ = load 'jquery.js';
one.js: import JQ; module Two = load 'two.js';
two.js: import JQ; import Drawing; Drawing.draw();
The module of concern to us is one.js. According to your proposal, one.js is doing in essence a text inclusion of the contents of two.js and as a result is susceptible to the problems unhygienic macro expansion suffers. As a result, JQ in two.js gets bound to jquery which zero.js setup and one.js imported as intended however Drawing also gets bound to the Drawing that zero.js setup. The author of one.js - the importee that is responsible setting up two.js - had no way of knowing that two.js needed Drawing and was going to find one in the scope chain. Nor could it take any action to withholding bindings to other imports deeper in modules that two.js may request.
But note that all modules have access to Fribble,
Jasvir Nagra wrote:
According to your proposal, one.js is doing in essence a text inclusion of the contents of two.js and as a result is susceptible to the problems unhygienic macro expansion suffers.
I have been thinking about similar concerns, but, as the proposal mentions separate compilation of modules, I'm currently looking upon it as dynamic scoping rather than macro expansion. Ie, a separately compiled external module (compiled at "top-level" lexical scope) is instantiated an arbitrary number of levels down in another module hierarchy and is getting references to the modules at this location through dynamic scoping. (Maybe my usage of the term dynamic scoping is not fully on the point but I'm hoping it is getting the message through)
I'd like to express my full respect for that you, David and Sam, have probably thought a lot on this, but it would still be great if you could add some discussion around this on the proposal page. Comparing with Java classes and classloaders, where classloaders can affect the referential environment for subsequently loaded classes, it looks to me like every module has the capability to affect the referential environment of its sub-modules. Ie, each time you create a module you are also creating an environment. What is quite an explicit operation in Java (setting up a separate classloader) is implicit in the proposal, and I'm thinking that there might be risks associated with this. I'm thinking that both Jasvir and I are thinking about possible advantages if instead using a dedicated syntax just for setting up the environment to use for sub-modules?
Btw, Jasvir, I see that you are using the syntax:
import JQ;
in your examples, but as far as I can see the grammar doesn't allow this form, as all import statements are about aliasing names inside a module to local names. I, too, think it would be great to have a simple form that expresses an intent to use a certain module without binding any of its names. At the moment this seems to be done like this:
module JQ = JQ;
but I'm not sure that is what is intended?
Best Mike Wilson
zero.js: module One = load 'one.js'; module Drawing = load 'gun.js'; module JQ = load 'jquery.js';
one.js: import JQ; module Two = load 'two.js';
two.js: import JQ; import Drawing; Drawing.draw();
The module of concern to us is one.js. According to your proposal, one.js is doing in essence a text inclusion of the contents of two.js
It's not a textual inclusion. The only context-sensitivity within 'two.js' is its free references to other modules -- at the outset of the body of a module, only static module bindings are in scope.
and as a result is susceptible to the problems unhygienic macro expansion suffers.
That's an exaggeration and only very tangentially related.
The point of module references is to strike a balance between clean separation and convenience. The ability to refer to other modules implicitly makes direct references possible (including cyclic references) while still providing the client with ultimate control over naming.
As a result, JQ in two.js gets bound to jquery which zero.js setup and one.js imported as intended however Drawing also gets bound to the Drawing that zero.js setup. The author of one.js - the importee that is responsible setting up two.js - had no way of knowing that two.js needed Drawing and was going to find one in the scope chain.
I'm not sure what you mean by "no way of knowing." In this setup, a module's free references to existing modules are part of its API. This information would be part of the documentation. It's similar to the way things work now, in that libraries collaborate by sharing things through documented top-level bindings in the global object, but done in a more static way to allow for true lexical scope.
Nor could it take any action to withholding bindings to other imports deeper in modules that two.js may request.
On the contrary-- it can, via module loaders. If you want to load, say, an untrusted 3rd party module, you can use module loaders to do so in a separate environment.
But the purpose of the core module system is not to provide isolation between untrusted parties. It's important to distinguish modularity and isolation. The module system provides cleaner code separation while preserving convenient collaboration between modules. The module loaders API, by contrast, provides stronger controls for isolation.
Jasvir Nagra www.cs.auckland.ac.nz/~jas
On Mon, May 24, 2010 at 6:21 AM, Mike Wilson <mikewse at hotmail.com> wrote:
Jasvir Nagra wrote:
According to your proposal, one.js is doing in essence a text inclusion of the contents of two.js and as a result is susceptible to the problems unhygienic macro expansion suffers.
I have been thinking about similar concerns, but, as the proposal mentions separate compilation of modules, I'm currently looking upon it as dynamic scoping rather than macro expansion. Ie, a separately compiled external module (compiled at "top-level" lexical scope) is instantiated an arbitrary number of levels down in another module hierarchy and is getting references to the modules at this location through dynamic scoping. (Maybe my usage of the term dynamic scoping is not fully on the point but I'm hoping it is getting the message through)
I've tended to compare it to both macro expansion and dynamic scoping not because those comparisons are necessarily accurate but because the extent to which the comparison holds helps me identify the strengths of the proposal and the pitfalls it could avoid. Another way that I've found very helpful in comparing module proposals (and language constructs in general) is the amount of additional code that I'd have to read as a programmer in order to understand a small changelist (CL) in a language that contains the proposed language feature. So for example if a language contains arguments.caller then given a CL that contains function calls, in order to understand the change, I'd have to look all functions transitively called by functions in the CL to be sure the invariants of the function being changed are maintained.
I'd like to express my full respect for that you, David and Sam, have
probably thought a lot on this, but it would still be great if you could add some discussion around this on the proposal page. Comparing with Java classes and classloaders, where classloaders can affect the referential environment for subsequently loaded classes, it looks to me like every module has the capability to affect the referential environment of its sub-modules. Ie, each time you create a module you are also creating an environment. What is quite an explicit operation in Java (setting up a separate classloader) is implicit in the proposal, and I'm thinking that there might be risks associated with this. I'm thinking that both Jasvir and I are thinking about possible advantages if instead using a dedicated syntax just for setting up the environment to use for sub-modules?
Btw, Jasvir, I see that you are using the syntax:
import JQ;
in your examples, but as far as I can see the grammar doesn't allow this form, as all import statements are
Oops. You're right. I should say "import JQ.*".
2010/5/24 David Herman <dherman at mozilla.com>
zero.js: module One = load 'one.js'; module Drawing = load 'gun.js'; module JQ = load 'jquery.js';
one.js: import JQ; module Two = load 'two.js';
two.js: import JQ; import Drawing; Drawing.draw();
The module of concern to us is one.js. According to your proposal, one.js is doing in essence a text inclusion of the contents of two.js
It's not a textual inclusion. The only context-sensitivity within 'two.js' is its free references to other modules -- at the outset of the body of a module, only static module bindings are in scope.
Ok, got it.
and as a result is susceptible to the problems unhygienic macro expansion suffers.
That's an exaggeration and only very tangentially related.
The point of module references is to strike a balance between clean separation and convenience. The ability to refer to other modules implicitly makes direct references possible (including cyclic references) while still providing the client with ultimate control over naming.
As a result, JQ in two.js gets bound to jquery which zero.js setup and one.js imported as intended however Drawing also gets bound to the Drawing that zero.js setup. The author of one.js - the importee that is responsible setting up two.js - had no way of knowing that two.js needed Drawing and was going to find one in the scope chain.
I'm not sure what you mean by "no way of knowing." In this setup, a module's free references to existing modules are part of its API. This information would be part of the documentation. It's similar to the way things work now, in that libraries collaborate by sharing things through documented top-level bindings in the global object, but done in a more static way to allow for true lexical scope.
Nor could it take any action to withholding bindings to other imports deeper in modules that two.js may request.
On the contrary-- it can, via module loaders. If you want to load, say, an untrusted 3rd party module, you can use module loaders to do so in a separate environment.
It isn't a concern of trusted vs untrusted modules as much as it is of change maintenance. If modules are to handle programming in the small, then this is a clear and sufficient improvement over modifying top-level bindings of the global object. But if (as I suspect) modules are meant to handle programming in the large, then module documentation is insufficient even for trusted parties or the very same author. The addition of "import Drawing" in my example even if it was accompanied by a documentation change in the Drawing module wouldn't cause existing users of the Drawing module to realize something had changed in the api.
But the purpose of the core module system is not to provide isolation
between untrusted parties. It's important to distinguish modularity and isolation. The module system provides cleaner code separation while preserving convenient collaboration between modules. The module loaders API, by contrast, provides stronger controls for isolation.
Ok cool. Let me refresh myself on the module loaders API. I re-iterate - the question isn't one of trusted vs untrusted parties, its a question of programming in the large vs small. In a sufficiently large system, even trusted parties are equipped with sufficiently advanced cluelessness about other modules invariants that the line between malice and ignorance is blurred. Rather than create two different module mechanisms to deal with the two cases, if its possible for the same mechanism to deal with both cases, it ought to. I think your modules proposal is close to achieving this. If the module loader API handles this case, would you suggest the direct use of modules in small programs and insist on the module loader api for large programs?
Hello!
I've updated the strawman proposals for static modules and dynamic module loaders and would love to get feedback from the es-discuss community.
Static modules:
strawman:simple_modules, strawman:simple_modules_examples
Briefly, the proposal provides a static module form:
The declaration binds the module name lexically. Modules declared at the same scope can be mutually recursive. They can also be loaded from external URL's:
The body of a module contains nothing in scope except for outer module bindings. Modules are truly lexically scoped, with no global object pollution. Modules can easily import bindings from other modules, either by dereferencing the modules that are in scope directly:
or by importing them explicitly:
Although modules are static entities, they can be dynamically reflected as objects.
There's plenty more details on the wiki page.
Dynamic module loaders:
strawman:module_loaders
These provide an API for dynamically loading and evaluating modules, and creating separate isolated contexts in which modules can be evaluated. You can create a separate module loader that shares no module instances with the current module loader; this could be used e.g. by an IDE (such as Bespin) that wants to run client code without letting the client code affect the running environment of the IDE itself. You can also share some module instances, to have finer-grained control over the sharing between module loaders.
There are some API notes on the wiki page. I expect it'll be revised and refined over time. Feedback welcome!
Thanks,
Hello! I've updated the strawman proposals for static modules and dynamic module loaders and would love to get feedback from the es-discuss community. * Static modules: http://wiki.ecmascript.org/doku.php?id=strawman:simple_modules http://wiki.ecmascript.org/doku.php?id=strawman:simple_modules_examples Briefly, the proposal provides a static module form: module Widgets { ... } The declaration binds the module name lexically. Modules declared at the same scope can be mutually recursive. They can also be loaded from external URL's: module Even = "http://example.com/even.js"; module Odd = "http://example.com/odd.js"; The body of a module contains nothing in scope except for outer module bindings. Modules are truly lexically scoped, with no global object pollution. Modules can easily import bindings from other modules, either by dereferencing the modules that are in scope directly: module Client { ... Widgets.messageBox("hello!"); ... } or by importing them explicitly: module Client { import Widgets.messageBox; ... messageBox("hello!"); ... } Although modules are static entities, they can be dynamically reflected as objects. There's plenty more details on the wiki page. * Dynamic module loaders: http://wiki.ecmascript.org/doku.php?id=strawman:module_loaders These provide an API for dynamically loading and evaluating modules, and creating separate isolated contexts in which modules can be evaluated. You can create a separate module loader that shares no module instances with the current module loader; this could be used e.g. by an IDE (such as Bespin) that wants to run client code without letting the client code affect the running environment of the IDE itself. You can also share some module instances, to have finer-grained control over the sharing between module loaders. There are some API notes on the wiki page. I expect it'll be revised and refined over time. Feedback welcome! Thanks, Dave