Modules and dependencies found during the load (before instantiation)
On Mon, Aug 18, 2014 at 10:55 AM, Ian Hickson <ian at hixie.ch> wrote:
On Wed, 13 Aug 2014, Ian Hickson wrote:
One of the problems I'm running into when it comes to trying to integrate ES6 modules with HTML and new HTML-based dependency features is the way that I can't tell ES about dependencies I know about before the data is actually fetched and instantiated.
Another example of where we have something like this is HTML imports. The "fetch" hook for HTML imports needs to actually be the hook that does all the parsing, since HTML loads incrementally. (For HTML imports, the "translate" and "instantiate" hooks are essentially no-ops.) This means that the in-band dependencies for HTML imports are found during the "fetch" hook, and need to be set up right away. For example, if an HTML import contains a <script type=module> block, that inline module needs to be added as dependency of the import itself (so that the import's 'load' event doesn't fire until all internal modules are loaded). It it contains an <img src=""> element, that needs to be added as a dependency as its loading. This is similar to the "instantiate" hook adding dependencies, except that it has to happen earlier due to the incremental parsing.
Can we explore the opposite question: how much does the HTML dependency problem really overlap the ES dependency problem?
To the first approximation these are the same problem: given the root of graph, load all of the nodes of the graph. The solution to this generic problem is well known and the code, while not trivial, is not very complex.
As soon as we go beyond this first level, the problems diverge. In fact I think your posts make the case that the character of these problems differ on so many points that code reuse is unlikely and algorithm reuse unwise.
In particular the reason we -- 'we' being you and me -- can't understand the ES spec is (evidently) that it supports loading a graph of mixed nodes (ES and legacy) with different assumptions about circular references. HTML is unlikely to have the identical requirements on circular references and it's legacy (script tags, document.write, mutable declarations, HTML imports) has quite different issues.
It seems to me that a better design for HTML dependency loading would integrate with the ES Loader rather than attempt to mutate it.
jjb
On Mon, 18 Aug 2014, John Barton wrote:
On Mon, Aug 18, 2014 at 10:55 AM, Ian Hickson <ian at hixie.ch> wrote:
On Wed, 13 Aug 2014, Ian Hickson wrote:
One of the problems I'm running into when it comes to trying to integrate ES6 modules with HTML and new HTML-based dependency features is the way that I can't tell ES about dependencies I know about before the data is actually fetched and instantiated.
Another example of where we have something like this is HTML imports. The "fetch" hook for HTML imports needs to actually be the hook that does all the parsing, since HTML loads incrementally. (For HTML imports, the "translate" and "instantiate" hooks are essentially no-ops.) This means that the in-band dependencies for HTML imports are found during the "fetch" hook, and need to be set up right away. For example, if an HTML import contains a <script type=module> block, that inline module needs to be added as dependency of the import itself (so that the import's 'load' event doesn't fire until all internal modules are loaded). It it contains an <img src=""> element, that needs to be added as a dependency as its loading. This is similar to the "instantiate" hook adding dependencies, except that it has to happen earlier due to the incremental parsing.
Can we explore the opposite question: how much does the HTML dependency problem really overlap the ES dependency problem?
As far as I can tell they're the same problem. At the simplest level, if we want it to be possible for one inline module to reference another inline module, then they're in fact exactly the same. Similarly if an HTML import contains a module and another HTML import contains a module, and they need to reference each other, then they're the same problem. HTML imports can also import style sheets and images and so on. Images can be imported by HTML documents that aren't imports. The transitive closure of the problem that includes ES6 Modules being downloaded is the entire Web platform's loading mechanism. Plus, if we want to make sure that ES6 module loading participates in the Web platform's Service Workers and Fetch infrastructures, they need to be implemented in a manner consistent with other loads.
The alternative is a lot of redundant code. I could be wrong, maybe browser vendors really do want to write dependency systems twice. I haven't heard one way or the other from browser vendors in this thread, so it's hard to tell.
To the first approximation these are the same problem: given the root of graph, load all of the nodes of the graph. The solution to this generic problem is well known and the code, while not trivial, is not very complex.
As soon as we go beyond this first level, the problems diverge. In fact I think your posts make the case that the character of these problems differ on so many points that code reuse is unlikely and algorithm reuse unwise.
I disagree. The problems seem very similar and more than just reuse code, they can just be the same code. There's actually very few problems that need changes to the module loading mechanism, and all those problems would benefit ES6 modules alone, as far as I can tell.
In particular the reason we -- 'we' being you and me -- can't understand the ES spec is (evidently) that it supports loading a graph of mixed nodes (ES and legacy) with different assumptions about circular references.
I think I have a pretty clear grip on the ES6 module spec at this point, FWIW. I just think a few things need tweaking to support the Web.
HTML is unlikely to have the identical requirements on circular references and it's legacy (script tags, document.write, mutable declarations, HTML imports) has quite different issues.
The ES6 module system is mostly already generic enough to handle most of these cases. The problems are mostly around needing more control over metadata (to distinguish CSS loads from ES loads, e.g. -- or indeed, to distinguish ES Module loads from legacy ES async script loads), needing to be able to manage dependencies for a load before its instantiate hook (useful for legacy JS module loaders, and for making it possible to preload ES6 modules), and needing to be able to delay execution (e.g. so that ES6 modules can be preparsed but not executed until needed).
It seems to me that a better design for HTML dependency loading would integrate with the ES Loader rather than attempt to mutate it.
I don't understand the difference. How do I "integrate" without duplicating requirements (e.g. having to manage the dependency tree twice) if the ES6 module loader doesn't support the requirements?
On Wed, 13 Aug 2014, Ian Hickson wrote:
Another example of where we have something like this is HTML imports. The "fetch" hook for HTML imports needs to actually be the hook that does all the parsing, since HTML loads incrementally. (For HTML imports, the "translate" and "instantiate" hooks are essentially no-ops.) This means that the in-band dependencies for HTML imports are found during the "fetch" hook, and need to be set up right away. For example, if an HTML import contains a <script type=module> block, that inline module needs to
be added as dependency of the import itself (so that the import's 'load' event doesn't fire until all internal modules are loaded). It it contains an <img src=""> element, that needs to be added as a dependency as its
loading. This is similar to the "instantiate" hook adding dependencies, except that it has to happen earlier due to the incremental parsing.