Native modules
Could a ECMAScript module proposal cover 'native modules' - which are built into the platform (ECMAScript engine and browser) - and can be imported if required and are not a binding on the global object. Could be useful for vendor specific innovation.
I have implemented (and know of two other implementations) native modules which meet the CommonJS module specification. There was no requirement in that specification to explicitly allow native modules.
The key is that the modules specification does not describe the behaviour of the loader, only the effect that loading the module has on the exports object. Provided that the native module sticks with this paradigm, there is no conflict and very little (if any) specification language would be required in an ES-next description of modules to allow for either native or hybrid (native + JS) modules.
Maybe 'native modules' is the wrong term. The issue is maybe more to do with finding a namespace home for new native functionality. Namespaces have some overlap with modules (i think).
Whenever a new bit of native platform functionality needs to be exposed to ECMAScript there is a debate about where to bind it. Global? Object? Could the import syntax assist with this. e.g. import #math2 // special import syntax for native math functionality OR const math2 = services.get("math2"); // request a native service
Python has 'native modules' built in to the runtime (not dynamically loaded extension modules) which are not in the global top-level namespace. They must be explicitly imported via the import command. So the 'import' command covers importing both pure python modules and built-in 'native modules'.
Note: i am definitely not talking about dynamically loading native code into the browser over the web. Just a way of exposing additional native functionality - without dumping it in the global namespace or Object.
On Tue, Jan 19, 2010 at 12:25 PM, Kevin Curtis <kevinc1846 at googlemail.com>wrote:
Maybe 'native modules' is the wrong term. The issue is maybe more to do with finding a namespace home for new native functionality. Namespaces have some overlap with modules (i think).
Whenever a new bit of native platform functionality needs to be exposed to ECMAScript there is a debate about where to bind it. Global? Object? Could the import syntax assist with this. e.g. import #math2 // special import syntax for native math functionality OR const math2 = services.get("math2"); // request a native service
I think this is an excellent suggestion. Weak pointers, Ephemeron tables, and catchalls are all on the table for the january mtg (< strawman:weak_references> and <
strawman:proxies>). Pending a modules
proposal, these purposely postpone the question of how their new start objects -- WeakPtr, EphemeronTable, and Proxy -- come into scope. For EphemeronTable and Proxy, since they provide neither new source of authority nor of non-determinism,
const EphemeronTable = import 'sys/EphemeronTable';
const Proxy = import 'sys/Proxy';
or something seems reasonable, modulo what the package names should be. For WeakPtr, since it provides a new source of non-determinism, it needs not to be available by default to SES code, but rather, only at the discretion of its sandboxing environment. I think this distinction is an interesting use case for the module proposals to examine.
On Tue, Jan 19, 2010 at 1:07 PM, Mark S. Miller <erights at google.com> wrote:
I think this is an excellent suggestion. Weak pointers, Ephemeron tables, and catchalls are all on the table for the january mtg. Pending a modules proposal, these purposely postpone the question of how their new start objects -- WeakPtr, EphemeronTable, and Proxy -- come into scope. For EphemeronTable and Proxy, since they provide neither new source of authority nor of non-determinism, const EphemeronTable = import 'sys/EphemeronTable'; const Proxy = import 'sys/Proxy'; or something seems reasonable, modulo what the package names should be. For WeakPtr, since it provides a new source of non-determinism, it needs not to be available by default to SES code, but rather, only at the discretion of its sandboxing environment. I think this distinction is an interesting use case for the module proposals to examine.
My response assumes the further constraint that some other functionality (e.g., Parser) may be provided as a pure-ES module at some point, then migrated into the runtime later on, and clients of Parser might want to be insulated from this change.
In my proposal, I mention a "platform import-module" object, "pim". This is the root of the importing chain and its API is unspecified by my proposal (though it might be later on).
This object may allow its creator to specify "overlays" of part of the module namespace. In other words:
myNewPim = pim.withOverlay({ 'sys/foo': import 'fooImpl' from { /* some package / }, 'sys/bar': import 'barEmulator' from { / some package */ } });
This should fail immediately if any of the overlaid objects is not an (authority-free) module function. All module functions loaded by "import" are stamped as such, as are native pieces of authority-free functionality provided by the interpreter.
The "pim" provided by an interpreter will come pre-populated with overlays for the appropriate functionality the interpreter provides.
As for where to supply that "pim" and other, authority-bearing objects: that is a "service registry" question and is out of scope for the module system.
Ihab
I had approached this concept in a slightly different way where resources required were specified using a derived MessageEvent type. There is precedence in building from the event system since its the only place I'm aware of where distributed semantics are being formally broached -eg postMessage, MessageChannels, etc. So motivations could be:
- Already a rich hierarchy of DOM events
- MessageEvent, postMessage, onMessage has good support across browsers
- Security is factored into postMessage with remote or cross-document messaging.
- MessagePorts are queued, providing well-defined dependency ordering
I had also used WebSockets (supported in Chrome and Webkit). Also supported in Jetty and a few other servers.WebSockets can be emulated in other browsers via flash. Although this is delving too deep into implementation details, describing resources required within a derived MessageEvent like ResourcesRequest and ResourcesResponse with the ability to send related info per request like if-modified-since would allow a server to respond with a ResourcesResponse that included aggregated content (images,css,js) specified in the ResourcesRequest. Done over a websocket, it would satisfy Ihab's suggestion to do this over a devoted bidirectional channel which is the WebSockets protocol. Performance characteristics especially if you use the gmail technique of delaying eval of javascript sources until needed were very convincing.Is this forum appropriate for discussion of eventing, WebIDL, etc? I'm not sure, though I imagine this committee has spent uncountable hours discussing events related to the DOM etc.
kam
Perhaps I should also clarify what I mean here --
On Tue, Jan 19, 2010 at 11:46 AM, <ihab.awad at gmail.com> wrote:
Are you really asking for a "service registry"?
A "module" is a piece of code that can be instantiated in multiple isolated contexts. The ability to import a module provides no authority. A module may be implemented in pure ES or as "native" code, but its API must still be the standard module API.
A "service" is an ES object that exposes some arbitrary API. The ability to reach a service typically provides authority. A service may (of course...) be implemented in pure ES or as "native" code.
Ihab
On Wed, Jan 20, 2010 at 8:21 AM, <ihab.awad at gmail.com> wrote:
Perhaps I should also clarify what I mean here --
On Tue, Jan 19, 2010 at 11:46 AM, <ihab.awad at gmail.com> wrote:
Are you really asking for a "service registry"?
A "module" is a piece of code that can be instantiated in multiple isolated contexts. The ability to import a module provides no authority. A module may be implemented in pure ES or as "native" code, but its API must still be the standard module API.
A "service" is an ES object that exposes some arbitrary API. The ability to reach a service typically provides authority. A service may (of course...) be implemented in pure ES or as "native" code.
Yes, this is clarifying. In these terms, for my previous example, WeakPtr would be a registered native service. EphemeronTable and Proxy would native modules.
Native functionality/services or captabilities can be either part of the ES enviroment (JSON) or the platform browser (XMLHttpRequest).
At the moment the global object acts as the "service registry". All native services get dumped there.
The global object seems to have a couple of functions:
- exposing native functionality (or capabilities) in a "dependency injection container" fashion.
- passing 'global vars' between scripts tags.
So, a lot depends on what the role of the global object is in the future. Will it continue as container/provider of services/capabilities? With top-level ES code then parceling out the capabilities to modules. Or is there something else in the pipeline?
It's interesting to ask (as Ihab alluded to) if the module system was in place how would the evolution of JSON object happened from pure ES to a 'native module'.
A crude idea: "use strict sys" const pim = sys.pim; // sys acts as the root for exposing additional core native functionality. // import is just for pure ES
This dicussion on how to expose native platform functionality - securely - to ES in a HTML5 context is interesting: arstechnica.com/business/news/2010/01/chrome-os-interview-1.ars/3 See section 'Programming Nuts and Blots'. A bit OT - but related to namespaces and maybe modules.
On Wed, Jan 20, 2010 at 8:25 AM, Mark S. Miller <erights at google.com> wrote:
In these terms, for my previous example ... EphemeronTable and Proxy would be native modules.
By the same token, perhaps something like code.google.com/p/nativeclient modules could be loaded via the module system too.
Ihab
I do think we could use some suggestions from the committee on how resources could be fetched ahead of import. Given the myriad of ways that exist now to get resources - mostly based on script injection or xhr, I think its going to be difficult to define a common service registry to fetch resources and eval them.
kam
It's sometimes desirable to expose native platform functionality to the Ecmascript engine. This is typically done by adding a binding to the global object e.g. JSON and ByteVector.
This leads to the issues of namespace collisions and if the new native object should be universally accessible by all scripts.
Could a ECMAScript module proposal cover 'native modules' - which are built into the platform (ECMAScript engine and browser) - and can be imported if required and are not a binding on the global object. Could be useful for vendor specific innovation.