Sam Tobin-Hochstadt (2013-04-25T22:27:54.000Z)
github at esdiscuss.org (2013-07-12T02:27:00.318Z)
First, I appreciate you setting your thoughts down in detail. I think this will help us move forward in the discussion. You write in a later message: > Having said that, interoperability with existing module systems was > not the main motivation for the change in the proposal, as far as I > can tell. It rather was simplicity (which is debatable, as I hope I've > made clear), and convenient support for common configuration and > concatenation use cases. I don't think this is right, and I think this is the heart of the issue. Let me back up a bit to explain this. Module names play a role in three processes, in general: 1. As a way to identify local components. 2. As a way to find the physical resource that is the source code (or object code) of the module. 3. As a way for two separately developed components to coordinate about which module they mean. In the current design, the internal names (eg, "jquery") serve role 1, and URLs (as generated by the loader hooks) serve role 2. The coordination role is played by internal names in a shared registry. To pick another example (at random :), take SML with CM [1]. Here, lexically-bound module name serve role 1, and CM serves role 2. Coordination is managed by the ML toplevel, and files are typically open in the sense that they refer to modules that will ultimately be bound at the top level during the compilation process. The system you outline at the end of your message, and the similar system that Dave and I originally proposed, doesn't have anything that really fulfills role 3. That's why we changed the design. This is perhaps easiest to understand with an example. Imagine you're developing a library, which will depend on jQuery, but you're not shipping jQuery with your library (this is how Ember works, as do a lot of other libraries). Then you have to write something like this: ```js // in ember.js import "jquery" as $; ``` That's what you'd write in the current design. In the design you're proposing, there are a few choices. You can give up on allowing people choose where they load jQuery from, and hardcode a dependency on a specific URL in the Ember source code. This is obviously not ok. You could decide that instead there's a single well-known URL that everyone is going to identify with jQuery. This is basically using the URL namespace as keys in the module registry of the current design, except without separating logical names from URLs. Then your program looks like this: ```js // in ember.js module jquery = "http://code.jquery.com/jquery-1.9.1.js"; import jquery as $; ``` Of course, even though that looks like a URL, it isn't being used to fetch anything, and everyone has to use some form of loader hook to map it to where they actually keep jQuery. Moreover, not everyone will have the resources or want to take the trouble to register a domain to publish their library. Or you could settle on a local URL to use in the same way, again as basically a registry key. ```js // in ember.js module jquery = "scripts/jquery.js"; import jquery as $; ``` Again, this thing that looks like a URL isn't being fetched, instead it will have to be rewritten to some other URL. And again, we have the same division into internal names that are in a registry, and external URLs that are actually for fetching the resource, except now because your proposal tries to abolish this distinction, it's much harder to manage. In other words, we are now manually simulating the registry, and it's more painful and problematic for exactly the reasons that manually simulating things on top of abstractions that don't support them always is. In contrast, the current design factors these two concerns, so that this configuration is about the mapping from internal names (like "jquery") to external names (like "http://...."). #### The key takeaway The key takeaway here is this: in the current design, different modules from different developers can coordinate based on the *internal* names of their dependencies. In your lexical proposal, it's not possible to coordinate globally on internal names, because they're lexical. So instead developers would have to coordinate on *external* names. This is fundamentally flawed, because external names are about where to get bits, not which abstraction a name represents. Supporting this use case properly is what led us to realize that the earlier lexically-named proposal was flawed. Note that none of this is about concatenation. We've made some particular design decisions where concatenation played a role, but it wasn't a part of the reason we moved away from lexical modules. #### Some smaller points > * As an external naming mechanisms, it violates standard relative > path/URL semantics. This isn't done particularly to be compatible, and node or AMD could have required "/" in front of absolute paths. But since that's an important use case, we don't think it's a good idea to tax it with extra typing. This convention has become popular in JS for a reason. > * The shared name space between internal and external modules can lead > to incoherent programs. I've already pointed out above how these namespaces aren't shared, and in fact their separation is an important reason for the current design. Also, the incoherent programs you refer to are ruled out by a suggestion (of yours!) that we adopted at the last meeting: declarative module forms that define a module that already exists are a static error. > * Likewise, a single global name space for all internally defined > modules can lead to incoherent programs. I would be more worried about this if (a) we didn't provide convenient ways to structure this name space and (b) it wasn't already an existing successful approach in real-world JS systems. > * "Local" references are globally overridable. This is only true in the sense that executing first lets you define the meaning of particular modules. But this is a *feature*. This is just another way to describing configuration. It's not reasonable to think that we can decide for everyone what will need configuration and what won't. > * Internal module definition is coupled with external module > registration, and modules cannot be renamed or re-registered. These are not "external" names, but it is true that declaring a module registers it. It's possible to take it out of the table afterward, or give it a new name, or have it share multiple names, all by using the loader API in very easy ways. I don't think these will be common operations, and this seems like a reasonable tradeoff. > * Language-level naming semantics interferes with file system/URL > naming semantics. While it's sad that everyone doesn't use the same filesystem ;), this is inevitable unless we force everyone to write *both* internal and external names explicitly for every module; that's clearly more user-hostile than the alternative. Lots of languages (Java, Node, ...) manage a default mapping to the file system while making Windows and Mac filesystems work sensibly. > * Bundling ("concatenation") generally would require embedding > arbitrary string resources, not syntactic modules. The reason to use concatenation is to avoid consuming excessive client resources -- in this setting of course you won't want to run translation on the client side. Translation hooks are important (a) in less perf-sensitive settings like development and (b) for isolating remotely-loaded code, neither of which require concatenation. > * Module "declarations" are not a declarative mechanism, but an operational one. This comes back to my original point. Registration of module names is about coordination, and thus this is an import feature, not a problem. This is again long, but I hope it's clarified why the current design is the way it is, and why the alternative you propose doesn't solve the use cases we need to address. Sam [1] http://www.smlnj.org/doc/CM/ for those following along at home.