Discuss
  • Written Proposal for "package" Keyword for ES20XX Imports

Written Proposal for "package" Keyword for ES20XX Imports

# Greg McLeod (8 years ago)

Back in 2015 I was working on a proposal for the "package" keyword, but stopped prematurely as ES6/ES2015 had already been shipped. Since then I've been occasionally iterating on it anyway in order to flesh out my ideas with how it could fit with the new import syntax. I wanted to share with the mailing list to get some feedback on it, as well as help facilitate some of the discussion regarding future import syntax.

Proposal can be found here: gist.github.com/Cleod9/7f73017905d124210ec8

I should note that a lot of this is derived from an ActionScript to JavaScript proof-of-concept transpiler I wrote a few years back. (Live demo: as3js.org/demo ). I believe this type of package syntax/behavior has already been possible since ES5, although I have yet to find any proposals for the keyword that resemble that of battle-tested languages like Java/C#.

I think there are a lot of time-saving qualities that packages could provide for bundled JS applications we have yet to explore, and it could even open the door to a JS standard library. Even if this doesn't go anywhere I certainly hope by making this proposal public it will be a useful for reference for what is possible in the current state of affairs.

Any feedback would be greatly appreciated!

Greg

P.S. Screen-cap of a potential IDE future with packages: i.imgur.com/D5EGNUN.gifv

# Ryan Birmingham (8 years ago)

I like the idea you're working from here, it seems to partially resemble header files. I'm not sure if a package keyword is needed though; couldn't you accomplish the same thing with just adding dictionary support and an optional alias parameter to import/export?

-Ryan Birmingham

# Greg McLeod (8 years ago)

Thanks Ryan. I think there would be a bit of an issue with introducing dictionary support alone in that it does not provide a new way for the language to define where modules actually live in the application itself as opposed to the file system. This misses out on the benefits the package keyword could provide. "package" should impose some limitations on the top-most scope of a module to facilitate the ability to define the packages before actually executing a module's body.

To elaborate, one of the biggest benefits I'd like to emphasize is the complete isolation of dependency resolution and actual module body execution.

The package keyword in my proposal would allow you to arbitrarily say "here's the complete structure of my app, all dependencies already resolved" and subsequently take action on it after it has been fully loaded into memory in its entirety. Augmenting existing import syntax cannot allow this since the core body of each module is being executed amidst the dependency graph being resolved as each "import" is encountered by the interpreter.

# Logan Smyth (8 years ago)

I don't have tons of comments on the proposal as a whole, but I do think there may be confusion about ES6 imports. Imports are resolved and processed and modules linked before any execution of the module body takes place. I can't tell from your description if you are reading that differently.

# Greg McLeod (8 years ago)

@Logan: Oh I see what you mean, I suppose I was thinking about it in a different way. I do know in ES6 the import keywords are always hoisted to the top, so it should be a given that the modules will be "resolved" already as a consequence of that. But there is no guarantee that all of the modules are linked successfully, namely in the edge cases of inheritance.

Consider the following

// app.js
import A from "./a.js";
import B from "./b.js";

var a = new A();
var b = new B();

// a.js
import B from "./b.js";

console.log("A's reference to B", B);

export default class A {
  constructor() {
  }
}

// b.js
import A from "./a.js";

console.log("B's reference to A", A);

export default class B extends A {
  constructor() {
    super();
  }
}

In the latest Babel the above code fails to run on the first line of app.js (at the time it attempts to execute _inherits(B, _A);.). This is because "A" is undefined in module b.js due to the circular dependency. In this case Babel basically executed code in the incorrect order for this to be able to work. The package keyword avoids this by creating a hoist-like behavior for classes that are inherited, which is safe to do given the proposed limitations in the top-level scope

I should note that circular dependencies like this are (hopefully) not too common, but one coming from another language might expect this to work based on how ES6 imports are described. I think the behavior of the package keyword could be much more transparent about what it's capable of.

# Logan Smyth (8 years ago)

Fair enough. Is the idea that class declarations would be lazy-initialized somehow then? It seems like the runtime behavior of class declarations would make things difficult. What is responsible for knowing that the class A needs to be initialized before class B extends A {}? What if the declaration is using runtime behavior like class B extends doThing(A) {}?

# Greg McLeod (8 years ago)

@Logan

Yes that's correct more or less. This is demonstrated with the module.injects concept showcased in the Gist. Any inheritance that occurs at the top of level of a module can be statically detected by parsing the AST for extends within that scope. Keywords found after an extends could be given execution priority so that they are imported inline immediately. All other exported class declarations that don't have an extends can be deferred via the injection mechanism shown in the example code.

In the case of dynamic extension with extends doThing(A), both doThing and A would need to treated with priority. Though I imagine for a first iteration of something like this there could be a restriction on that type of syntax while those details are fleshed out to keep things maximally minimal.

# Isiah Meadows (8 years ago)

I'll point out that Babel doesn't follow spec perfectly because it can't transpile the bindings correctly in the case of synchronously executed code with circular dependencies. That's a CommonJS limitation that Babel can't work around any more than it already does.

# T.J. Crowder (8 years ago)

On Wed, Feb 22, 2017 at 5:50 PM, Isiah Meadows <isiahmeadows at gmail.com>

wrote:

I'll point out that Babel doesn't follow spec perfectly because it can't transpile the bindings correctly in the case of synchronously executed code with circular dependencies. That's a CommonJS limitation that Babel can't work around any more than it already does.

I was wondering about that. Greg, re your example where you said:

In the latest Babel the above code fails to run on the first line of app.js

(at the time it attempts to execute _inherits(B, _A);.). This is because "A" is undefined in module b.js due to the circular dependency. In this case Babel basically executed code in the incorrect order for this to be able to work.

What does the spec say should happen in that situation? (You've clearly been into it in some detail.) As opposed to what Babel does, since of course Babel while excellent is working with some limitations...?

-- T.J. Crowder

# Logan Smyth (8 years ago)

What does the spec say should happen in that situation?

It would be a TDZ error, rather than undefined, in a real implementation of block scoping + ES6 modules. The ordering still means that A won't exist when class B extends A runs.

# Greg McLeod (8 years ago)

I'll point out that Babel doesn't follow spec perfectly because it can't transpile the bindings correctly in the case of synchronously executed code with circular dependencies. That's a CommonJS limitation that Babel can't work around any more than it already does.

@Isiah Is that so? Babel is leading the charge in terms of transpilers right now, so that's unfortunate. Anyway I did some quick Googling and found a couple of things related to the subject:

webpack/webpack#1788, esdiscuss.org/topic/how-to-solve-this-basic-es6- module-circular-dependency-problem

While it's evident people have been encountering this for quite some time, it seems like the solutions up until now have been mostly workarounds. My hope is that a "package" keyword could tackle this kind of thing at more of a holistic level for bundled applications.

What does the spec say should happen in that situation? (You've clearly

been into it in some detail.) As opposed to what Babel does, since of course Babel while excellent is working with some limitations...?

@T.J. Good question, see below my response to Logan

It would be a TDZ error, rather than undefined, in a real implementation

of block scoping + ES6 modules. The ordering still means that A won't exist when class B extends A runs.

@Logan So it sounds like this TDZ error would be at runtime as opposed to compile-time, correct? Do you have a link to where this is referenced? Both cases would imply the ES6 spec does not truly support circular dependencies, and that Babel is not entirely incorrect

# Logan Smyth (8 years ago)

So it sounds like this TDZ error would be at runtime as opposed to

compile-time

Correct, since extends, computed properties, and decorators all require runtime behavior, class declarations are initialized a runtime when execution reaches them, unlike function declarations, where initialization can be hoisted.

would imply the ES6 spec does not truly support circular dependencies

ES6 modules have no explicit handling of circular dependencies. What they do to approach this problem is have imported values reference the current value of the exported variable, rather than the value at the time the import was evaluated. The common problem with circular dependencies in CommonJS is that when you call require, the exported value might not have been assigned yet. Because ES6 modules expose live access to the current value, you don't need to worry about that edge case.

This does not however free you from ensuring that your code initializes all values in the proper order. In your example, the code is executed like

// app.js
console.log("B's reference to A", A);

class B extends A {
  constructor() {
    super();
  }
}

console.log("A's reference to B", B);

class A {
  constructor() {
  }
}

var a = new A();
var b = new B();

because of the way the imports in your snippet are ordered. As is clear, you are trying to extend class A before the class declaration has executed. Since initialization of block-scoped variables is not hoisted, class B extends A will throw because A is uninitialized.

Do you have a link to where this is referenced?

The TDZ behavior is the standard behavior of block-scoped values. There is no special logic around this for imports.

Babel is not entirely incorrect

Babel's behavior essentially treats class A {} like var A = class A {}, hence the undefined value. That is not entirely correct, but the ordering of the imports is correct. There are cases where Babel is limited in implementing ES6's semantics 100%, but they are all around the behavior of live re-exports like export * from 'foo';.

# Greg McLeod (8 years ago)

because of the way the imports in your snippet are ordered. As is clear, you are trying to extend class A before the class declaration has executed. Since initialization of block-scoped variables is not hoisted, class B extends A will throw because A is uninitialized.

Ahh, I see. Importing B before A in app.js does take care of the issue in Babel. And by placing additional console logs in the constructor I can see that A goes from being initially undefined to being valid by the time an instance of B is created. Definitely the utility of live exports.

Babel's behavior essentially treats class A {} like var A = class A {},

hence the undefined value. That is not entirely correct, but the ordering of the imports is correct. There are cases where Babel is limited in implementing ES6's semantics 100%, but they are all around the behavior of live re-exports like export * from 'foo';.

Thanks for the additional insight. When I first started thinking about a package operator a couple of years ago, it was definitely behaviors like these that I wanted to remove from the list of daily JS development nuances. I've found that this type of dependency issue just simply doesn't occur in languages like C#/Java/AS3 because of how restricted those languages are at the top level scope. The proposal presents the package keyword in JS as an opt-in way to impose a similar restriction, but with the goals of improving static analysis potential of code and making circular dependency bugs a thing of the past.

I should also note that the code-snippet in the Gist demonstrating the implementation of "package" is actual working ES5. The code allows the circular imports example I wrote without the need for even a live exports object.