Discuss
  • Feed back and proposal for modules: allow importing ES5 files

Feed back and proposal for modules: allow importing ES5 files

# 程劭非 (13 years ago)

everyone,

I noticed that current importing grammar will not work for ES5 files, I mean there is no way to import one or more es5/es3 files as a module and import variables from it.

The problem is:

  1. There are no "export" keywords in a es5/es3 file, no variables are exported.
  2. They might be using more than one files but current mudule grammar allows only one file as one module.

When the new standard(es6) comes out, I guess a big number of libraries need updating. And before they finish their work, I think we need a cheap solution for the new code(es6) to use old libraries(es5/es3) without modifying their code

For the two points, I was considering two solution:

  1. Export all global variable by default.

Anonymous function could be used to protect the global namespace and most libraries are already using it. So I don't think "export" is needed.

Exporting all global variable will not pollute the module's user's namespace for we still need "import" phase.

  1. Allow importing multiple files as one module.

This will completely decouple file and module. We need to modify the ModuleSpecifier grammar:

ModuleSpecifier ::= StringLiteral | Path ===>

ModuleSpecifier ::= ( StringLiteral | Path ) ( "," StringLiteral | Path )*

With this, for example, if I'm using a RSA library like www-cs-students.stanford.edu/~tjw/jsbn, I can do the

following:

import "jsbn.js","prng4.js","rng.js","rsa.js" as RSA;

If I want to use a module depend on jQuery1.3.2, I can do:

import "jQuery1.3.2.js","MyModule.js" as MyModule;

I've mentioned this idea in a reply, post it as a separate thread to get more feed back :-)

/Shaofei Cheng

# Alex Russell (13 years ago)

On Sep 22, 2012, at 6:29 PM, 程劭非 <csf178 at gmail.com> wrote:

Hi, everyone,

I noticed that current importing grammar will not work for ES5 files, I mean there is no way to import one or more es5/es3 files as a module and import variables from it.

The problem is:

  1. There are no "export" keywords in a es5/es3 file, no variables are exported.
  2. They might be using more than one files but current mudule grammar allows only one file as one module.

When the new standard(es6) comes out, I guess a big number of libraries need updating. And before they finish their work, I think we need a cheap solution for the new code(es6) to use old libraries(es5/es3) without modifying their code

One option here is for you to create an export "wrapper" for them, in essence a separate file that requires the old library be loaded which then exports the library's symbols through an export. Not pretty, but it requires no code changes for well-designed libraries.

For the two points, I was considering two solution:

  1. Export all global variable by default.

Anonymous function could be used to protect the global namespace and most libraries are already using it. So I don't think "export" is needed.

You inadvertently identified why we need "exports": it's very difficult what is "global". In particular, what should be exported in the case of a file that includes:

var f = function(param) { thinger = param; };

// ...

if (someGlobalConfigurationSetting) { f("howdy!"); }

?

Exporting all global variable will not pollute the module's user's namespace for we still need "import" phase.

Perhaps instead of "global" you mean "the top-level IFFE"?

In that case, it's easier, but then it depends on the IFFE both being executed (it's not a static form here) and there being only one. Or is the suggestion that a file like this should export both a and b? And what will the value of c be?

// example.js

(function(p){ var a = p; })("hello");

(function(p){ var b = p; })("world");

var setValue = function(p){ var c = p; };

setValue("it's a sunny day!"); setValue("it's raining in London");

// end example.js

As you can see, at the limit, this creates some serious hazards for violating the data-hiding provided by closure-bound variables. Suddenly importers can not only see these internal names, but watch their values change!

  1. Allow importing multiple files as one module.

This will completely decouple file and module. We need to modify the ModuleSpecifier grammar:

ModuleSpecifier ::= StringLiteral | Path ===> ModuleSpecifier ::= ( StringLiteral | Path ) ( "," StringLiteral | Path )*

With this, for example, if I'm using a RSA library like www-cs-students.stanford.edu/~tjw/jsbn, I can do the following:

import "jsbn.js","prng4.js","rng.js","rsa.js" as RSA;

If I want to use a module depend on jQuery1.3.2, I can do:

import "jQuery1.3.2.js","MyModule.js" as MyModule;

I've mentioned this idea in a reply, post it as a separate thread to get more feed back :-)

This is interesting. Do you have a proposed resolution mechanism for conflicts? We had such a thing in the old traits system for classes, but I don't think it has survived anywhere.

-- Alex Russell slightlyoff at google.com slightlyoff at chromium.org alex at dojotoolkit.org BE03 E88D EABB 2116 CC49 8259 CF78 E242 59C3 9723

# 程劭非 (13 years ago)

Thank you for replying, Alex.

Replied inline.

2012/9/24 Alex Russell <alex at dojotoolkit.org>:

Hi Shaofei:

On Sep 22, 2012, at 6:29 PM, 程劭非 <csf178 at gmail.com> wrote:

Hi, everyone,

I noticed that current importing grammar will not work for ES5 files, I mean there is no way to import one or more es5/es3 files as a module and import variables from it.

The problem is:

  1. There are no "export" keywords in a es5/es3 file, no variables are exported.
  2. They might be using more than one files but current mudule grammar allows only one file as one module.

When the new standard(es6) comes out, I guess a big number of libraries need updating. And before they finish their work, I think we need a cheap solution for the new code(es6) to use old libraries(es5/es3) without modifying their code

One option here is for you to create an export "wrapper" for them, in essence a separate file that requires the old library be loaded which then exports the library's symbols through an export. Not pretty, but it requires no code changes for well-designed libraries.

I tried to find a way to make a wrapper but I failed.

Consider I have a file // mylib.js

//depends on jQuery $(......).attr(......)

When importing it as a library

//There is no way to add $ to it's scope //except polluting the global object import "mylib.js" as myLib;

For the two points, I was considering two solution:

  1. Export all global variable by default.

Anonymous function could be used to protect the global namespace and most libraries are already using it. So I don't think "export" is needed.

You inadvertently identified why we need "exports": it's very difficult what is "global". In particular, what should be exported in the case of a file that includes:

var f = function(param) { thinger = param; };

// ...

if (someGlobalConfigurationSetting) { f("howdy!"); }

?

Exporting all global variable will not pollute the module's user's namespace for we still need "import" phase.

Perhaps instead of "global" you mean "the top-level IFFE"?

In that case, it's easier, but then it depends on the IFFE both being executed (it's not a static form here) and there being only one. Or is the suggestion that a file like this should export both a and b? And what will the value of c be?

// example.js

(function(p){ var a = p; })("hello");

(function(p){ var b = p; })("world");

var setValue = function(p){ var c = p; };

setValue("it's a sunny day!"); setValue("it's raining in London");

// end example.js

As you can see, at the limit, this creates some serious hazards for violating the data-hiding provided by closure-bound variables. Suddenly importers can not only see these internal names, but watch their values change!

Perhaps I'm not expressing clearly.(Sorry I'm not a native speaker.)

I'm not saying that we should export variables inside a IFFE.

I mean that :

//example.js

var $; // for ES5 $ should be a property of global object, //I mean even without “export”, Module instance object //should have a binding for "$" when example.js imported //as a Module

(function(){

$ = function(){
}
// $ could be changed

var $2 = function(){
}
// $2 will not be imported
// because it's in a IFFE.

})();

// somewhere // Variant A: import "example.js" as myQuery; import $ from myQuery;

// Variant B: module myQuery = "bar.js"; import $ from myQuery;

//$ should be able to be used here

  1. Allow importing multiple files as one module.

This will completely decouple file and module. We need to modify the ModuleSpecifier grammar:

ModuleSpecifier ::= StringLiteral | Path ===> ModuleSpecifier ::= ( StringLiteral | Path ) ( "," StringLiteral | Path )*

With this, for example, if I'm using a RSA library like www-cs-students.stanford.edu/~tjw/jsbn, I can do the following:

import "jsbn.js","prng4.js","rng.js","rsa.js" as RSA;

If I want to use a module depend on jQuery1.3.2, I can do:

import "jQuery1.3.2.js","MyModule.js" as MyModule;

I've mentioned this idea in a reply, post it as a separate thread to get more feed back :-)

This is interesting. Do you have a proposed resolution mechanism for conflicts? We had such a thing in the old traits system for classes, but I don't think it has survived anywhere.

Do you mean name conflict?

If "jQuery1.3.2.js" and "MyModule.js" are all ES5 style (no module system used). There should not be any conficts or they will not work in ES5.

This is different from import "jQuery1.3.2.js" as jQuery; import "MyModule.js" as MyModule;

The declaration import "jQuery1.3.2.js","MyModule.js" as MyModule; should behave like importing a file just like "jQuery1.3.2.js" and "MyModule.js" are copied together.

# Alex Russell (13 years ago)

Inline:

On Mon, Sep 24, 2012 at 7:22 AM, 程劭非 <csf178 at gmail.com> wrote:

Thank you for replying, Alex.

Replied inline.

2012/9/24 Alex Russell <alex at dojotoolkit.org>:

Hi Shaofei:

On Sep 22, 2012, at 6:29 PM, 程劭非 <csf178 at gmail.com> wrote:

Hi, everyone,

I noticed that current importing grammar will not work for ES5 files, I mean there is no way to import one or more es5/es3 files as a module and import variables from it.

The problem is:

  1. There are no "export" keywords in a es5/es3 file, no variables are exported.
  1. They might be using more than one files but current mudule grammar allows only one file as one module.

When the new standard(es6) comes out, I guess a big number of libraries need updating. And before they finish their work, I think we need a cheap solution for the new code(es6) to use old libraries(es5/es3) without modifying their code

One option here is for you to create an export "wrapper" for them, in essence a separate file that requires the old library be loaded which then exports the library's symbols through an export. Not pretty, but it requires no code changes for well-designed libraries.

I tried to find a way to make a wrapper but I failed.

Consider I have a file // mylib.js

//depends on jQuery $(......).attr(......)

When importing it as a library

//There is no way to add $ to it's scope //except polluting the global object import "mylib.js" as myLib;

Right. What I'm suggesting isn't that you'll be able to prevent the global from being augmented, rather that if your goal is to take an already well behaved library and wrap it with modules, that's possible.

For the two points, I was considering two solution:

  1. Export all global variable by default.

Anonymous function could be used to protect the global namespace and most libraries are already using it. So I don't think "export" is needed.

You inadvertently identified why we need "exports": it's very difficult what is "global". In particular, what should be exported in the case of a file that includes:

var f = function(param) { thinger = param; };

// ...

if (someGlobalConfigurationSetting) { f("howdy!"); }

?

Exporting all global variable will not pollute the module's user's namespace for we still need "import" phase.

Perhaps instead of "global" you mean "the top-level IFFE"?

In that case, it's easier, but then it depends on the IFFE both being executed (it's not a static form here) and there being only one. Or is the suggestion that a file like this should export both a and b? And what will the value of c be?

// example.js

(function(p){ var a = p; })("hello");

(function(p){ var b = p; })("world");

var setValue = function(p){ var c = p; };

setValue("it's a sunny day!"); setValue("it's raining in London");

// end example.js

As you can see, at the limit, this creates some serious hazards for violating the data-hiding provided by closure-bound variables. Suddenly importers can not only see these internal names, but watch their values change!

Perhaps I'm not expressing clearly.(Sorry I'm not a native speaker.)

I'm not saying that we should export variables inside a IFFE.

I mean that :

//example.js

var $; // for ES5 $ should be a property of global object, //I mean even without “export”, Module instance object //should have a binding for "$" when example.js imported //as a Module

(function(){

$ = function(){
}
// $ could be changed

var $2 = function(){
}
// $2 will not be imported
// because it's in a IFFE.

})();

// somewhere // Variant A: import "example.js" as myQuery; import $ from myQuery;

// Variant B: module myQuery = "bar.js"; import $ from myQuery;

//$ should be able to be used here

I see. It's unclear, then, how this sort of thing works with conditionals and "this" assignment. E.g.:

// conditional.js if (false) { var thinger = "whatevs"; }

print(thinger); // undefined

this.thinger = "whatevs";

print(thinger); // "whatevs" // end conditional.js

I think you can make a strong argument that since it's the user of the module that is deciding how to use it, "this" assignment not having the normal side-effects is fine...although I expect legacy scripts to point to a "this" which isn't module scope but rather is a global. The "var" thing points to a deep vein of difficulty -- notably the inability to reason about the exports in a reliable way.

I'm curious what others thing about this.

  1. Allow importing multiple files as one module.

This will completely decouple file and module. We need to modify the ModuleSpecifier grammar:

ModuleSpecifier ::= StringLiteral | Path ===> ModuleSpecifier ::= ( StringLiteral | Path ) ( "," StringLiteral | Path )*

With this, for example, if I'm using a RSA library like www-cs-students.stanford.edu/~tjw/jsbn, I can do the following:

import "jsbn.js","prng4.js","rng.js","rsa.js" as RSA;

If I want to use a module depend on jQuery1.3.2, I can do:

import "jQuery1.3.2.js","MyModule.js" as MyModule;

I've mentioned this idea in a reply, post it as a separate thread to get more feed back :-)

This is interesting. Do you have a proposed resolution mechanism for conflicts? We had such a thing in the old traits system for classes, but I don't think it has survived anywhere.

Do you mean name conflict?

If "jQuery1.3.2.js" and "MyModule.js" are all ES5 style (no module system used). There should not be any conficts or they will not work in ES5.

That's not true. One will simply overwrite the other. There's a default resolution mechanism there (last wins). Let bindings and module forms create a new static early-error scenario that we didn't have before.

# Claus Reinke (13 years ago)

Right. What I'm suggesting isn't that you'll be able to prevent the global from being augmented, rather that if your goal is to take an already well behaved library and wrap it with modules, that's possible.

Aren't Module Loader Translation hooks meant to help with this?

harmony:module_loaders#translation_semantics

Naïve idea: fetch the code, then add exports before evaluating the code as a local module scope.

Though this would require parameterizing the Loader with some configuration data (export $ from jquery, export {f,g,x} from anotherES5utils, ..). Which seems to call for interaction between fetch and translate hooks, and possibly even between import declarations and loader hooks (how does the loader hook get to know which exports to add to an ES5 script, for a given import declaration?).

RequireJS has some experience with wrapping non-modules via configuration options.

jrburke/requirejs/wiki/Upgrading-to-RequireJS-2.0#wiki-shim

Without a prototype implementation of modules as an executable spec, it will be difficult to find such edge cases and check how well they are covered in the existing semi-formal spec.

Claus

# Alex Russell (13 years ago)

I think module loaders would, indeed help, although as you point out, you're gonna need some sort of transformation step and we don't provide an AST to work against. You might need something like Traceur's front-end or esprima to get started on a reliable transform pass. But yeah, doable.

# 程劭非 (13 years ago)

Thanks, Claus and Alex,

I've looked at the Loader but it seems to be async?

I know there are many ways to load multiple files together(iframe, XHR, script element and datauri for browser host) but what I'm suggesting is to make es6 module more friendly to es5 libraries.

I mean currently the es6 module grammar is using

export var a; var b; var c;

But why not export all top-level scope variables and let developers use IFFE to hide variables?

var a; // export a automatically void function(){ var b; var c; }();

This will make libraries like old versions of jQuery work with es6 module without any code change. These libraries might be on CDN and changing their code will be a big cost in some scenarios.

2012/9/24 Alex Russell <alex at dojotoolkit.org>

# Claus Reinke (13 years ago)

I've looked at the Loader but it seems to be async?

So far, yes, but previous discussions here led to the conclusion that making the Loader functionality available for module syntax imports would be useful and unproblematic. See, for instance, the final paragraph of

esdiscuss/2012-July/024184

Perhaps that hasn't been worked into the spec yet. I hope it doesn't get lost.

I know there are many ways to load multiple files together(iframe, XHR, script element and datauri for browser host) but what I'm suggesting is to make es6 module more friendly to es5 libraries.

At first, a standard Loader plugin for treating ES5 scripts as modules sounds useful, instead of everyone writing their own. Especially if a generic script-as-modules-wrapper requires code analysis to figure out global assignments.

That, however, is exactly the issue: ES5 scripts are not friendly to ES6 modularity, as all export information is encoded dynamically - one cannot figure out the exports without running the code.

So a general ES5-scripts-as-modules seems unlikely, and even if one wanted to write converters for each currently encoded module system, one would run into the runtime-vs-static issue, unless all modules happen to use a static subset of current module libraries (most module interface code will be static in intention, if not in expression, but there are no guarantees).

Which is why I was referring to the RequireJS approach of encoding the expected exports in the wrapper configuration.

Translating that to ES6 should be possible: we'd not even try to extract the global assignments from the code automatically; instead we'd tell the module wrapper manually how to extract the expected exports from whatever the ES5 module code does to its environment, by appending export declarations to the ES5 code. At least, that is how I understand this approach;-)

Claus

# Russell Leggett (13 years ago)

On Tue, Sep 25, 2012 at 12:08 AM, 程劭非 <csf178 at gmail.com> wrote:

Thanks, Claus and Alex,

I've looked at the Loader but it seems to be async?

I know there are many ways to load multiple files together(iframe, XHR, script element and datauri for browser host) but what I'm suggesting is to make es6 module more friendly to es5 libraries.

I mean currently the es6 module grammar is using

export var a; var b; var c;

But why not export all top-level scope variables and let developers use IFFE to hide variables?

var a; // export a automatically void function(){ var b; var c; }();

This will make libraries like old versions of jQuery work with es6 module without any code change. These libraries might be on CDN and changing their code will be a big cost in some scenarios.

Why not simply the proposal a little. It would be nice to be able to export top level etc. but I think there is a clear danger there. What if instead of trying to make arbitrary scripts into modules, we just did the loading, making sure the script was loaded before running the code that depended on it. ES5 relies on global scope, and so would this. Instead of allowing:

import "jQuery1.3.2.js","MyModule.js" as MyModule;

Maybe just allow something like:

import "jQuery1.3.2.js"
import "MyModule.js"

$(......).attr(......)

Those scripts would still be executed in the global scope, and so this module would be able to access $. It would be extremely easy to wrap this way as well. Otherwise, you'd still need to worry about actually loading the dependency, no?

# Sam Tobin-Hochstadt (13 years ago)

On Tue, Sep 25, 2012 at 4:27 PM, Claus Reinke <claus.reinke at talk21.com> wrote:

I've looked at the Loader but it seems to be async?

So far, yes, but previous discussions here led to the conclusion that making the Loader functionality available for module syntax imports would be useful and unproblematic. See, for instance, the final paragraph of

esdiscuss/2012-July/024184

Perhaps that hasn't been worked into the spec yet. I hope it doesn't get lost.

There seems to be some misunderstanding about what that message says. As I said, we may add more configurability to the system loader that's present by default. However, the async portions of the Loader API is not something that will change -- we can't add synchronous IO to JS.

# 程劭非 (13 years ago)

Import is currently used by Module. This way of using "import" looks like a loader feature.

2012/9/26 Russell Leggett <russell.leggett at gmail.com>

# 程劭非 (13 years ago)

So what I've got is we need static meta data for Module exports. For this reason "export" keyword is needed.

I was considering if these meta data could be given by the user of Module (instead of author of Module).

If module file could only be used in ES6 (on need lexical convertion), I thinking most guys are not going to use this feature with in a very long time after ES6 released.

These days I'm thinking about simply use proposal 2.(Allow importing multiple files as one module)

We could easily make a wrapper for ES5 libs。

// jQueryWrapper.js export var $;

//in ***.html import "jQueryWrapper.js","jQuery.js" as jQuery; jQuery.$

What do you think?

2012/9/26 Claus Reinke <claus.reinke at talk21.com>