Where'd Promise#done go?

# Chad Austin (11 years ago)

Hoping to proactively polyfill likely parts of upcoming standards, I am adding an implementation of DOM promises to IMVU's base JavaScript library.

Why was done() removed from the draft spec? " Rename Futures per TC39 discussion. Drop done() per same." whatwg/dom/commit/7a741b953990a89c6d27532928fe7817c3b25528

I read that there is an expectation that promise implementations will log when an error goes unhandled, but how is that possible?

To log when an error goes unhandled, an implementation would have to log when a promise has state=rejected and no reject callbacks, AND that the promise has been dropped on the floor. Without weak reference callbacks, how could you know that a promise has been GC'd?

I found done() to be a useful concept and in the absence of relevant notes from the below minutes, an explanation for why it was removed.

Thanks, Chad Austin

# Domenic Denicola (11 years ago)

The logging solution proposed is not polyfillable with today's tools, at least not when logging to the browser console.

The idea would be that rejection reasons are logged when nobody has handled them, but then "un-logged" when they are handled. Since there is no console.unlog, you see our problem.

You could "polyfill" this by creating a secondary "console" overlay on top of the browser window. I believe when.js has something like this in an experimental branch:

cujojs/when/commit/65f89e9eeb397186aa1508bc58e49653aca2fcb4

Theirs is based off of a proposed Promises/A+ API, console.unhandledRejection and console.rejectionHandled, which would allow cross-promise-library sharing of such an interface:

promises-aplus/unhandled-rejections-spec#2

I know RSVP.js has expressed interest in this as well, and I speak for Q in saying we would love that. At this point it's just a matter of someone, perhaps myself, putting in the time to create a generic bookmarklet that pops up a unhandled-rejections console and intercepts console.unhandledRejection/console.rejectionHandled.

# David Bruant (11 years ago)

Le 18/06/2013 23:49, Chad Austin a écrit :

Hi all,

Hoping to proactively polyfill likely parts of upcoming standards, I am adding an implementation of DOM promises to IMVU's base JavaScript library.

Why was done() removed from the draft spec? " Rename Futures per TC39 discussion. Drop done() per same." whatwg/dom/commit/7a741b953990a89c6d27532928fe7817c3b25528

I read that there is an expectation that promise implementations will log when an error goes unhandled, but how is that possible?

To log when an error goes unhandled, an implementation would have to log when a promise has state=rejected and no reject callbacks, AND that the promise has been dropped on the floor. Without weak reference callbacks, how could you know that a promise has been GC'd?

You can't. That's a part that can't be polyfilled today indeed. This feature requires native promises.

# Kris Kowal (11 years ago)

As Domenic mentions, there will be no place for "done" in our bright promise debugger future.

It will however be necessary for promise users to keep on ending their chains with "done()" until promise debuggers are ubiquitously available. This is a simple problem. If you are writing code that targets both old and new engines, the promise polyfill will simply have to patch a no-op "done" onto the engine’s Promise.prototype.

Kris Kowal

# Mark S. Miller (11 years ago)

I don't understand this. I am onboard with console.unhandledRejection/console.rejectionHandled and all that for better logging, and with using WeakRef notification to improve the logging yet further. But I don't see how any of this can substitute for the need that .done() serves. I think we will still need .done() in ES7 promises.

Note: I didn't have .done() in E, but that's only because I didn't think of it. I did have the problem that .done() addresses and I would have found it useful.

# Kevin Smith (11 years ago)

I am of the opinion that program errors should fail loudly by default, which the current proposal does not support.

To this end, in my own promise implementation I use a form of "fail soon", like so:

Define a promise tree. As a base case we have a promise created with the Promise constructor. Leaves are added to the tree by using the "then" method. The "promise forest" is the collection of all such trees for a running program. An "invalid promise tree" is a tree which contains a rejected leaf node which is not also a root. During a "callback flush", the callback queue is repeatedly emptied until no more callbacks remain. At the end of the callback flush, the promise forest is checked for any invalid promise trees. If there exists an invalid promise tree, then an unhandled error is thrown (which will of course propagate to window.onerror).

It's worked well for me so far, and has obviated the need for bolt-ons like "done" or "console.unlog", etc. YMMV.

zenparsing/zen-bits/blob/master/src/Promise.js

# Domenic Denicola (11 years ago)

From: es-discuss-bounces at mozilla.org [mailto:es-discuss-bounces at mozilla.org] On Behalf Of Kevin Smith

To this end, in my own promise implementation I use a form of "fail soon", like so:

It sounds like this does not support handling rejections in an event loop turn after they are generated, e.g. it would disallow

var rejected = Promise.reject(new Error("bad news!"));
var fulfilled = Promise.resolve(5);

fulfilled.then(() => {
    rejected.catch(err => console.error(err));
});

Is that correct?

# Domenic Denicola (11 years ago)

From: Mark S. Miller [mailto:erights at google.com]

I don't understand this. I am onboard with console.unhandledRejection/console.rejectionHandled and all that for better logging, and with using WeakRef notification to improve the logging yet further. But I don't see how any of this can substitute for the need that .done() serves. I think we will still need .done() in ES7 promises.

While I think I see what you're getting at, let me play devil's advocate for a bit to draw this out more clearly. Using the sample code from promises-aplus/unhandled-rejections-spec/issues/1:

var rejectPromise;
var promise = new Promise((resolve, reject) => rejectPromise = reject);

promise.then(() => console.log("I only attached a handler for fulfillment"));
// All is OK (A)

rejectPromise(promise, new Error("who handles me?"));
// Nobody sees the error! Oh no, maybe we should crash here? (B)

setTimeout(function () {
    promise.then(undefined, (err) =>console.error("I got it!", err));
    // But if we crashed there, then how would this code ever get run? (C)
}, 5000);

Using a done-less promise implementation with a unhandled rejections console, we have the flow that:

  1. At line (A), all is fine, and the unhandled rejections console is empty.
  2. At line (B), the unhandled rejections console contains Errror: "who handles me?". This remains true for the next five seconds.
  3. At line (C), after five seconds have passed, the unhandled rejections console becomes yet again empty.

This seems to neatly solve the problem without done, at least in my devil's-advocate world. Where's the problem? :)

# Mark S. Miller (11 years ago)

On Tue, Jun 18, 2013 at 8:11 PM, Domenic Denicola < domenic at domenicdenicola.com> wrote:

From: Mark S. Miller [mailto:erights at google.com]

I don't understand this. I am onboard with console.unhandledRejection/console.rejectionHandled and all that for better logging, and with using WeakRef notification to improve the logging yet further. But I don't see how any of this can substitute for the need that .done() serves. I think we will still need .done() in ES7 promises.

While I think I see what you're getting at,

What do you think I'm getting at? ;)

# Ron Buckton (11 years ago)

I've often looked at Promise#then() as sugar over Promise#done() for something like:

Promise.prototype.then = function(resolve, reject) {
  return new Promise(resolver => {
    this.done(
      value => {
        try {
          resolver.resolve(resolve ? resolve(value) : value);
        }
        catch (e) {
          resolver.reject(e);
        }
      },
      err => {
        try {
          resolver.resolve(reject ? reject(value) : value);
        }
        catch (e) {
          resolver.reject(e);
        }
      });
  });
}

Promise#done() doesn't have the overhead that Promie#then does (allocating a new chained Promise), so it is more efficient if you don't need to chain. It feels easier to be more explicit with done then without, since to polyfill done requires calling something like setImmediate to raise the error to the engine/window.onerror, since throwing it in the reject handler would just result in a new rejected Promise.

If we had an 'await' keyword I might find the need for done to be less important, as it would be easy to write "await p" to bubble the exception to user code. Although, 'await' would also be more efficient with 'done' rather than 'then'.

If Promise#done is out, a polyfill could just have an array of unhandled exceptions that could be analyzed programmatically in user code or via the console.

# Domenic Denicola (11 years ago)

From: Mark S. Miller [mailto:erights at google.com]

What do you think I'm getting at? ;)

Heh. In short, non-browser environments.

# Kevin Smith (11 years ago)

var rejected = Promise.reject(new Error("bad news!")); var fulfilled = Promise.resolve(5);

fulfilled.then(() => { rejected.catch(err => console.error(err)); });

In my implementation, this works for two reasons. First, "root" promises (i.e. not created via then) such as rejected above are not "throwable".

Second, errors are not thrown until after the callback queue has been repeatedly flushed, until empty. Both arrow functions above would execute before the next checkpoint.

This, however, would crash the program:

Promise.reject(new Error("bad news!")).then(x => x);

As would this:

var rejected = Promise.reject(new Error("bad news!")).then(x => x);
setTimeout($=> rejected.catch(err => console.log(err)), 0);
# medikoo (11 years ago)

I use promises a lot, and (for all valid points stated above) I see done as a must have for implementation to be practically usable.

Also it needs to be provided natively, as it's not possible to shim it just with then.

Proposed log/un-log in background mechanism doesn't solve the issue, as you're still left with overhead of then and it complicates error handling which with done is straightforward. I hope it won't land in any spec.

-- View this message in context: mozilla.6506.n7.nabble.com/Where-d-Promise-done-go-tp281461p281493.html Sent from the Mozilla - ECMAScript 4 discussion mailing list archive at Nabble.com.

# Mark S. Miller (11 years ago)

On Wed, Jun 19, 2013 at 3:28 AM, Alex Russell <slightlyoff at gmail.com> wrote:

On Wednesday, June 19, 2013, Ron Buckton wrote:

I’ve often looked at Promise#then() as sugar over Promise#done() for something like:

Promise.prototype.then = function(resolve, reject) {
  return new Promise(resolver => {
    this.done(
      value => {
        try {
          resolver.resolve(resolve ? resolve(value) : value);
        }
        catch (e) {
          resolver.reject(e);
        }
      },
      err => {
         try {
          resolver.resolve(reject ? reject(value) : value);
        }
        catch (e) {
          resolver.reject(e);
        }
      });
  });
}

Promise#done() doesn’t have the overhead that Promie#then does (allocating a new chained Promise), so it is more efficient if you don’t need to chain.

That is my only latent argument for #done(), and one that I think I agree with Luke to re-visit at some later date. We can always add.

What was less clear is that there's any real problem with error handling: polyfills can (should?) keep lists of unhandled promises and make them available to tools while we wait for devtools to surface the list. The concerns here with direct logging seem, to me, to be premature.

Regarding prematurity, I said "I think we will still need .done() in ES7 promises." I do not think this is something that DOMPromises need to address prior to the careful ES7 promise work still in front of us. Since, as you and Luke say, "We can always add", DOMPromises only needs be approximately the minimum we need quick agreement on, so that we can add the rest of what's needed in the ES7 process. This they seem to be, which is great.

# Domenic Denicola (11 years ago)

From: Alex Russell [slightlyoff at gmail.com]

On Wednesday, June 19, 2013, Ron Buckton wrote:

Promise#done() doesn’t have the overhead that Promie#then does (allocating a new chained Promise), so it is more efficient if you don’t need to chain.

That is my only latent argument for #done(), and one that I think I agree with Luke to re-visit at some later date. We can always add.

Given the heroics browsers pull for optimizing unused return values already, this argument never made much sense to me. E.g. Brendan has earlier mentioned SpiderMonkey taking a different code path if it sees if (regExp.exec(...)), which only returns a truthy result instead of computing the entire array. It seems like a trivial optimization to notice that nobody's using the return value of then and not create a promise to return.

# Chad Austin (11 years ago)

On Wed, Jun 19, 2013 at 7:01 AM, Mark S. Miller <erights at google.com> wrote:

On Wed, Jun 19, 2013 at 3:28 AM, Alex Russell <slightlyoff at gmail.com>wrote:

What was less clear is that there's any real problem with error handling: polyfills can (should?) keep lists of unhandled promises and make them available to tools while we wait for devtools to surface the list. The concerns here with direct logging seem, to me, to be premature.

Last night I converted a bunch of our continuation-passing code to use promises, so now I have both implementation and usage experience with the current promise proposal. Inadvertent error swallowing and incomplete error bubbling were the primary pain points. I frequently found myself with typos and mistakes that became opaque TypeErrors by the time they reached the browser debugger or error log.

A DOM promise polyfill, in current web browsers*, cannot provide the same level of error handling and reporting as we've come to expect in traditional onload/onerror asynchronous code. Consider:

Promise.fulfill({}).then(function(x) { //... x.oops_not_a_function(); //... });

A naive polyfill would swallow ReferenceError or TypeError and hide it completely. So let us suppose that the polyfill keeps lists of unhandled rejections. How does the promise debugger know whether promises are still live and simply don't have reject callbacks configured yet? In addition, by the time the error in inspected, the stack trace is gone as well as any activation records, so even though Chrome and Firefox provide Error#stack, there's no way to inspect into 'x'. This is reproducible by bubbling the error with setTimeout:

function bubbleToBrowser(e) { setTimeout(function() { throw e; }, 0); }

Promise.reject(new Error("will_bubble")).catch(function(e) { bubbleToBrowser(e); });

Or its fundamental equivalent:

try {

({}).not_a_function();

}

catch (e) {

setTimeout(function() {

    throw e;

}, 0);

}

It's very hard in practice to trace the error back to its original cause. There are two issues here:

  1. Something like Promise#done allows idiomatically expressing "No really, that's it, don't chain anymore, and bubble errors into the event loop"
  2. Bubbling errors to the main loop with useful amounts of information is not possible in current browsers and JavaScript.

In the meantime, I will likely add an option to our Promise implementation, and perhaps default it on, that makes no attempt to catch errors thrown in Promise callbacks, guaranteeing they reach the event loop at the time they're thrown.

Thanks, Chad

  • I am happy to live with limitations of current web browsers as long as there is a clear plan to solve them.
# Forbes Lindesay (11 years ago)

I've been answering quite a few questions about promises on stack overflow lately. One of the key things people seem to struggle to get their head around is the idea of .then as being something that transforms the promise and returns a new promise. They either expect it to mutate the existing promise or they expect it to behave like .done() does.

I think .done() could be extremely useful purely as a teaching device. If we started everyone off by learning to use .done() they would have a much shallower learning curve. Initially they'd get all their errors thrown immediately which would be easier to see. It would be much more similar to typical (but terrible) DOM APIs and jQuery APIs that are event based or have a callback and an errback. Having learnt to use .done() we could teach .then() as a more advanced feature that let you compose asynchronous operations by returning a promise that has been transformed by the callbacks.

# medikoo (11 years ago)

Stating then() is something more advanced than done() doesn't make much sense to me. They're different methods, that serve different purpose, I think they should be valued on a similar level.

Key thing is that then() is conceptually a map(), and there's something wrong if just to access the value we have to use map(). Second issue, mentioned already numerous times is not desired (in case of "just access") error suppression.

# David Bruant (11 years ago)

Le 20/06/2013 14:55, Forbes Lindesay a écrit :

I've been answering quite a few questions about promises on stack overflow lately.

Do you have a link to a list to these questions (and/or your answers) off-top your browser history by any chance?

One of the key things people seem to struggle to get their head around is the idea of .then as being something that transforms the promise and returns a new promise. They either expect it to mutate the existing promise or they expect it to behave like .done() does.

I wasn't there when that started, but it feels like "then" organically grew out of the experience of using promises a lot which naturally leads to promise pipelining. It doesn't feel like the most "fundamental brick" to understand what promises are (what .done looks like though), because it isn't. At a first approximation, people can use .then the way the expect (without caring for the return value)

I think .done() could be extremely useful purely as a teaching device. If we started everyone off by learning to use .done() they would have a much shallower learning curve. Initially they'd get all their errors thrown immediately which would be easier to see.

That's how Q behaves out of necessity, but native promises can have better integration with debugging tools and don't need to reflect the error at runtime.

It would be much more similar to typical (but terrible) DOM APIs and jQuery APIs that are event based or have a callback and an errback.
Having learnt to use .done() we could teach .then() as a more advanced feature that let you compose asynchronous operations by returning a promise that has been transformed by the callbacks.

.then can be taught without telling it returns something ;-)

# Mark S. Miller (11 years ago)

On Thu, Jun 20, 2013 at 7:50 AM, David Bruant <bruant.d at gmail.com> wrote:

Le 20/06/2013 14:55, Forbes Lindesay a écrit :

I’ve been answering quite a few questions about promises on stack overflow lately.

Do you have a link to a list to these questions (and/or your answers) off-top your browser history by any chance?

One of the key things people seem to struggle to get their head around is the idea of .then as being something that transforms the promise and returns a new promise. They either expect it to mutate the existing promise or they expect it to behave like .done() does.

I wasn't there when that started,

I was ;)

but it feels like "then" organically grew out of the experience of using promises a lot which naturally leads to promise pipelining. It doesn't feel like the most "fundamental brick" to understand what promises are (what .done looks like though), because it isn't. At a first approximation, people can use .then the way the expect (without caring for the return value)

.then comes directly from E's "when", which comes from Original-E's "when", which comes from experience with Joule. See Concurrency Among Strangers < www.erights.org/talks/promises/paper/tgc05.pdf> or the expanded and

updated version in Part 3 and Chapter 23 of < erights.org/talks/thesis/markm-thesis.pdf>. For the history

especially, see that Chapter 23 -- "From Objects to Actors and Back Again".

I'm always surprised when people think that any of these callback forms are the fundamental starting point. The fundamental starting point is the asynchronous message send, expressed in E as infix "<-", in Q as .send, and proposed for ES7 as infix "!". In both E and Joule, .then-like callbacks were built as a pattern on top of message sending.

On the particular issue in question, the return value of .then, historically you are correct. Original-E and early E had a "when" that did not return a promise for the result. That idea was contributed by Mark Seaborn on the e-lang list, I believe sometime in the late '90s, but I haven't yet found the crucial message in the archive < www.eros-os.org/pipermail/e-lang>.

This change made a fundamental difference in the style of using "when". Without the return result, the only purpose of "when" is the side-effects performed by the callbacks, which necessarily takes you out of functional thinking into imperative thinking. With the return result, "when" is still potentially functional. As with lazy evaluation, when used functionally, both asynchronous message sending and "when" are just ways of rearranging the time of functional computation.

So the question about the order of teaching of .done vs .then depends on the overall teaching strategy. Some teaching strategies, such as SICP and CTM, teach functional first and only get to imperative constructs rather late. Prior to imperative constructs, .done makes sense only for its error reporting, when capping a promise sequence -- which, after all, was the reason for introducing .done in the first place. For this purpose, I recommend teaching only the parameterless form: p.done(); But I do recommend teaching it, as dropping errors silently is especially harmful to someone first learning a new programming paradigm.

# Mark Miller (11 years ago)

On Thu, Jun 20, 2013 at 7:50 AM, David Bruant <bruant.d at gmail.com> wrote:

Le 20/06/2013 14:55, Forbes Lindesay a écrit :

I’ve been answering quite a few questions about promises on stack overflow lately.

Do you have a link to a list to these questions (and/or your answers) off-top your browser history by any chance?

One of the key things people seem to struggle to get their head around is the idea of .then as being something that transforms the promise and returns a new promise. They either expect it to mutate the existing promise or they expect it to behave like .done() does.

I wasn't there when that started, but it feels like "then" organically grew out of the experience of using promises a lot which naturally leads to promise pipelining.

I'm worried that you may be suffering from and spreading a terminology confusion. "Promise pipelining" is an important latency reduction optimization when using promises over a network. See Chapter 16 of < erights.org/talks/thesis/markm-thesis.pdf>. Using .then, either with

or without the return result, prevents promise pipelining, which is another reason to emphasize asynchronous message sending and deemphasize .then.

# Forbes Lindesay (11 years ago)

Do you have a link to a list to these questions?

Here are a few examples:

You can see many more questions by selecting the promises tag on stack overflow.

native promises can have better integration with debugging tools

In deed they can, there are issues though. We will never be able to have the application crash in the event that an unhandled rejection occurs. That's no big deal for browsers, where the idea of carrying on with a best effort attempt is so ingrained anyway, but it's a significant issue for node.js and WinJS apps. In both those cases I don't really look at the logs unless I know something's gone wrong, which might lead to bugs going undiscovered for a long time.

I think it's important that there be a way to say, "If this rejection is not handled, crash my application".

I'm always surprised when people think that any of these callback forms are the fundamental starting point.

They aren't the fundamental starting point from either a conceptual point of view or an implementation point of view. However, most people in the short term will start out with knowledge of either callbacks (if they come from node.js) or the weird event style onSuccess and onFail handlers (if they come from the browser). This means that most of the time when we teach promises we're going to have to teach it with that as the starting point of knowledge, not with message sending as the starting point.

# Claus Reinke (11 years ago)

I'm worried that you may be suffering from and spreading a terminology confusion. "Promise pipelining" is an important latency reduction optimization when using promises over a network. See Chapter 16 of erights.org/talks/thesis/markm-thesis.pdf. Using .then, either with or without the return result, prevents promise pipelining, which is another reason to emphasize asynchronous message sending and deemphasize .then.

I couldn't imagine why you would think that using .then would prevent promise pipelining. A properly designed, monadic .then, is nothing but a more powerful let. Perhaps you could elaborate?

Naively translating the standard pipeline example gives

x.a().then( t1=>
y.b().then( t2=>
t1.c(t2).then( t3=>
... ) ) )

where x, y, t1, and t2 are meant to live on the same remote machine, computation is meant to proceed without delay into the ... part, and the implementation is meant to take care of avoiding unnecessary round-trips for the intermediate results.

This is naïve because the synchronous method calls should really be asynchronous message sends. If we assume local proxies that forward local method calls to remote objects and remote results to local callbacks, then y.b() will not start until t1 comes back.

But if t1 is itself a promise, then it can come back immediately, and the same applies to t2 and t3. So computation can proceed directly to the ... part, with delays happening only when required by data dependencies.

A user-level promise will not have the same opportunities for network traffic optimization as an implementation-level future (an obvious one would be moving the callback code to where the data is), but the .then itself does not prevent optimizations.

Unless, that is, one insists on flattening promises (no promises passed to .then-callbacks), which would sequentialize the chain...

What am I missing here? Claus

# Tom Van Cutsem (11 years ago)

2013/6/20 David Bruant <bruant.d at gmail.com>

I wasn't there when that started, but it feels like "then" organically grew out of the experience of using promises a lot which naturally leads to promise pipelining.

Terminology nit: if by "promise pipelining" you mean the fact that p.then() returns a new promise dependent on p, I would call that "promise chaining". In the two languages that established promises, i.e. Argus and E, the term "promise pipelining" specifically refers to minimizing network round-trips to reduce latency when using promises combined with RPCs. See < www.erights.org/elib/distrib/pipeline.html>.

# Tom Van Cutsem (11 years ago)

Sorry for the noise. I didn't see Mark's reply while offline.

# Mark S. Miller (11 years ago)

On Thu, Jun 20, 2013 at 9:29 AM, Forbes Lindesay <forbes at lindesay.co.uk>wrote: [...]

I'm always surprised when people think that any of these callback forms are the fundamental starting point.

They aren't the fundamental starting point from either a conceptual point of view or an implementation point of view. However, most people in the short term will start out with knowledge of either callbacks (if they come from node.js) or the weird event style onSuccess and onFail handlers (if they come from the browser). This means that most of the time when we teach promises we're going to have to teach it with that as the starting point of knowledge, not with message sending as the starting point.

Any beginning JavaScript programmer learns the meaning of "a.foo(b,c)", i.e., synchronous message send, long before they learn about callbacks.

# Mark S. Miller (11 years ago)

On Thu, Jun 20, 2013 at 10:34 AM, Claus Reinke <claus.reinke at talk21.com>wrote:

I'm worried that you may be suffering from and spreading a terminology

confusion. "Promise pipelining" is an important latency reduction optimization when using promises over a network. See Chapter 16 of <erights.org/talks/**thesis/markm-thesis.pdferights.org/talks/thesis/markm-thesis.pdf>. Using .then, either with or without the return result, prevents promise pipelining, which

is another reason to emphasize asynchronous message sending and deemphasize .then.

I couldn't imagine why you would think that using .then would prevent promise pipelining. A properly designed, monadic .then, is nothing but a more powerful let. Perhaps you could elaborate?

Naively translating the standard pipeline example gives

x.a().then( t1=> y.b().then( t2=> t1.c(t2).then( t3=> ... ) ) )

where x, y, t1, and t2 are meant to live on the same remote machine, computation is meant to proceed without delay into the ... part, and the implementation is meant to take care of avoiding unnecessary round-trips for the intermediate results.

This is naïve because the synchronous method calls should really be asynchronous message sends. If we assume local proxies that forward local method calls to remote objects and remote results to local callbacks, then y.b() will not start until t1 comes back.

But if t1 is itself a promise, then it can come back immediately,

I think this is what you are missing. If x.a() returns, for example, an int, then x!a() returns a promise that will turn out to be a promise-for-int. In that case, x!a().then(t1 => ...t1...), the callback

will only be invoked with t1 bound to the int itself. This can't happen prior to the completion of the round trip.

and the same applies to t2 and t3. So computation can proceed directly to the ... part, with delays happening only when required by data dependencies.

A user-level promise will not have the same opportunities for network traffic optimization as an implementation-level future (an obvious one would be moving the callback code to where the data is)

Moving the callback code to where the data is is indeed an important idea. Back when .then was called "when", this was called "where". With the renaming of "when" to "then", "where" is renamed "there" < strawman:concurrency#there>. .there

and .then must be kept distinct because they have very different security properties, as well as very different failure and progress properties under partition.

, but the .then itself does not prevent optimizations.

Unless, that is, one insists on flattening promises (no promises passed to .then-callbacks), which would sequentialize the chain...

I don't see how this is relevant, as the round-trip above is still forced on flatMap. But yes, we do insist on this flattening. That's the difference between .then and .flatMap.

# David Bruant (11 years ago)

From context, I suspect you're talking about "promise chaining".

Yes, I meant "chaining". Sorry for the confusion.

# Claus Reinke (11 years ago)

Naively translating the standard pipeline example gives

x.a().then( t1=> y.b().then( t2=> t1.c(t2).then( t3=> ... ) ) ) .. This is naïve because the synchronous method calls should really be asynchronous message sends. If we assume local proxies that forward local method calls to remote objects and remote results to local callbacks, then y.b() will not start until t1 comes back.

But if t1 is itself a promise, then it can come back immediately,

I think this is what you are missing. If x.a() returns, for example, an int, then x!a() returns a promise that will turn out to be a promise-for-int. In that case, x!a().then(t1 => ...t1...), the callback will only be invoked with t1 bound to the int itself. This can't happen prior to the completion of the round trip.

As I was saying, that restriction is not necessary - it is a consequence of the flatten-nested-promises-before-then-callback philosophy. Instead, the local proxy can send the remote message and locally pass a receiver promise to its callback. That way, the callback can start to run until it actually needs to query the receiver promise for a value.

If we did this for the .a and .b calls, the translation would change to

x.a().then( t1p=>
y.b().then( t2p=>

t1p.then( t1=>
t2p.then( t2=>
t1.c(t2).then( t3=>

... ) ) ) ) )

and the .b call could be triggered before the .a call roundtrip completes. If we want to push the "lazy-evalutation" into the ... part, things get more interesting, as one would need to model the data-dependencies and delay looking at t1p/t2p further. One could define an inline then-able to capture this:

x.a().then( t1p=>
y.b().then( t2p=>
let t3p = { then(cb): { t1p.then( t1=> t2p.then( t2=>
                                    t1.c(t2).then( t3=> cb(t3) ) ) ) };
...' ) )

(where ...' is ..., transformed to work with a promise t3p instead of t3)

Now, waiting for the .a and .b roundtrips would be delayed until some code in ...' actually needs to look at t3. One could further delay looking at t2 if t1.c() could deal with a promise t2p.

This additional flexibility is not available in a flat-promise design, which is why I think such a design is a mistake. Of course, even if one wants to accept the restrictions of a flat-promise design, the flattening should happen in promise construction, not in .then.

Claus

# Forbes Lindesay (11 years ago)

Returning to Mark Miller's comment:

Any beginning JavaScript programmer learns the meaning of a.foo(b,c), i.e., synchronous message send, long before they learn about callbacks.

I don't see any simple and obvious path from understanding the synchronous message send to understanding how promises work. How does that explanation look?

It seems on the other hand I can see a natural progression from callbacks to .done and from .done to .then because if you write any code using .done it's infinitely obvious that you need .then.

I think it's interesting to also reflect on Tasks in C# which have been around for a while. They don't have any methods for automatically tracking unhandled rejections even though the language supports handling object destruction. All the methods that are built in to the Task object match up with .done rather than .then. This becomes much less of a problem once the language has await or similar.

# Mark S. Miller (11 years ago)

On Thu, Jun 20, 2013 at 3:19 PM, Claus Reinke <claus.reinke at talk21.com>wrote:

Naively translating the standard pipeline example gives

x.a().then( t1=> y.b().then( t2=> t1.c(t2).then( t3=> ... ) ) ) .. This is naïve because the synchronous method calls should really be asynchronous message sends. If we assume local proxies that forward local method calls to remote objects and remote results to local callbacks, then y.b() will not start until t1 comes back.

But if t1 is itself a promise, then it can come back immediately,

I think this is what you are missing. If x.a() returns, for example, an int, then x!a() returns a promise that will turn out to be a promise-for-int. In that case, x!a().then(t1 => ...t1...), the callback will only be invoked with t1 bound to the int itself. This can't happen prior to the completion of the round trip.

As I was saying, that restriction is not necessary - it is a consequence of the flatten-nested-promises-**before-then-callback philosophy. Instead, the local proxy can send the remote message and locally pass a receiver promise to its callback. That way, the callback can start to run until it actually needs to query the receiver promise for a value.

If we did this for the .a and .b calls, the translation would change to

x.a().then( t1p=> y.b().then( t2p=>

t1p.then( t1=> t2p.then( t2=> t1.c(t2).then( t3=>

This corresponds only to running .a and .b overlapped. Since .c isn't run until both the .a and .b round trips complete, this isn't promise pipelining. This isn't just a matter of definitions. You lose most of the important optimization if all you're doing is overlapping independent requests.

... ) ) ) ) )

and the .b call could be triggered before the .a call roundtrip completes. If we want to push the "lazy-evalutation" into the ... part, things get more interesting, as one would need to model the data-dependencies and delay looking at t1p/t2p further. One could define an inline then-able to capture this:

x.a().then( t1p=> y.b().then( t2p=> let t3p = { then(cb): { t1p.then( t1=> t2p.then( t2=> t1.c(t2).then( t3=> cb(t3) ) ) ) }; ...' ) )

I can't yet respond to this because I don't understand your notation. What does "{ then(cb): " mean?

# Tab Atkins Jr. (11 years ago)

On Thu, Jun 20, 2013 at 7:03 PM, Mark S. Miller <erights at google.com> wrote:

x.a().then( t1p=> y.b().then( t2p=> let t3p = { then(cb): { t1p.then( t1=> t2p.then( t2=> t1.c(t2).then( t3=> cb(t3) ) ) ) }; ...' ) )

I can't yet respond to this because I don't understand your notation. What does "{ then(cb): " mean?

It seems pretty clear that's intended to be a concise method definition. Claus accidentally added a colon, likely through force of habit.

# Mark S. Miller (11 years ago)

Wasn't clear to me. But now that you mention it, it does fit. Unless I hear to the contrary from Claus, I will respond on that basis.

Thanks for the clarification.

# Mark S. Miller (11 years ago)

On Thu, Jun 20, 2013 at 3:19 PM, Claus Reinke <claus.reinke at talk21.com>wrote:

If we want to push the "lazy-evalutation" into the ... part, things get

more interesting, as one would need to model the data-dependencies and delay looking at t1p/t2p further. One could define an inline then-able to capture this:

x.a().then( t1p=> y.b().then( t2p=> let t3p = { then(cb) { t1p.then( t1=> t2p.then( t2=> t1.c(t2).then( t3=> cb(t3) ) ) ) }; ...' ) )

(where ...' is ..., transformed to work with a promise t3p instead of t3)

Now, waiting for the .a and .b roundtrips would be delayed until some code in ...' actually needs to look at t3. One could further delay looking at t2 if t1.c() could deal with a promise t2p.

[colon removed from above quote per Tab's suggestion]

Ok, you've delayed sending the .c until you needed t3p. But that's the opposite of the promise pipelining optimization! The point is not to delay sending .c till even later. The point is to send it before the round trips from .a or .b complete. This code still cannot do that.

# Mark S. Miller (11 years ago)

I don't see any simple and obvious path from understanding the synchronous message send to understanding how promises work. How does that explanation look?

Take it in three steps.

Step 1

var d = a.foo(b,c);

This does synchronous message sending. It delivers the message foo(b,c) to the object designated by a immediately, transferring control to that object -- the callee -- now. The caller blocks waiting for the callee to respond. The callee runs to completion, finally returning a value. At this point the caller receives that value into d and continues.

In order for the callee to be invoked synchronously, it must be local, since we don't want to wait on a round trip to a remote object.

In a communicating event-loop system, such synchronous call/return has a strong side effect contract with pros and cons.

  • pros: no local time passes between the calling and the being called, so the callee gets control in the state in which the caller made the request.
  • cons: the callee runs while the caller, and the caller's caller, etc, are suspended in the midst of some operations. They might have suspended invariants, or otherwise be unprepared for recursive entry in their current state. This threatens both caller and callee.

Step 2

var dP = aP ! foo(b,c);

This does asynchronous message sending. It delivers the message foo(b,c) to the object designated by aP eventually, eventually causing that object -- the callee -- to gain control. But the caller proceeds now without any interleaving of control by others. Since the caller proceeds immediately, dP cannot yet provide access to what the callee will return. But it still designates that value, whatever it will be. A designator whose designation is not yet determined is a promise. If the callee eventually returns an int, then dP is already a promise for that int, though neither it nor we know that yet.

Since we're only sending a message to the callee eventually and not waiting for it to respond, we don't much care whether it is local or remote.

In a communicating event loop system, such asynchronous message sending has a strong side effect contract with the opposite pros and cons.

  • pros: The caller executes to completion without possibility of interference from the callee. Any delicate state the caller was in the midst of manipulating is unperturbed, and the caller can complete its manipulation, confident that its invariants were not disrupted. Likewise, the callee receives the message in an empty stack state, in which all previous turns have presumably restored all heap invariants. This is a robust situation from which to start running.
  • cons: Between the caller requesting and the callee receiving, and arbitrary number of previously queued turns may run in the meantime, changing the world from the one in which the caller decided to send the message. By the time it arrives, it may no longer be relevant or appropriate.

Step 3

Ok, so we can . on local objects like a, and we can ! on local or remote objects like aP. What about dP? It also designates something, but that something is separated from us in time, not (necessarily) in space. No matter! Asynchrony handles that too. If aP is remote, the message gets queued on the event-loop hosting the object aP designates. If dP is pending, the message gets queued in dP itself. Once dP knows what it designates, it forwards all these queued messages to that target using !.

# Claus Reinke (11 years ago)

x.a().then( t1p=> y.b().then( t2p=> let t3p = { then(cb) { t1p.then( t1=> t2p.then( t2=> t1.c(t2).then( t3=> cb(t3) ) ) ) }; ...' ) )

(where ...' is ..., transformed to work with a promise t3p instead of t3)

Now, waiting for the .a and .b roundtrips would be delayed until some code in ...' actually needs to look at t3. One could further delay looking at t2 if t1.c() could deal with a promise t2p.

[colon removed from above quote per Tab's suggestion]

oops, force of habit, as Tab guessed correctly.

Ok, you've delayed sending the .c until you needed t3p. But that's the opposite of the promise pipelining optimization! The point is not to delay sending .c till even later. The point is to send it before the round trips from .a or .b complete. This code still cannot do that.

I do not expect to be able to emulate pipelining fully in user-land (at least not in JavaScript).

My aims were to demonstrate that .then does not need to stand in the way of such an optimization, and that the additional flexibility/ expressiveness provided by non-flattening .then is relevant here.

Back to your objection: there is nowhere to send the .c until you have t1 at hand. You could, however, move waiting on the t2 dependency to later, by passing the t2 receiver promise t2p to .c

x.a().then( t1p=>
y.b().then( t2p=>
let t3p = { then(cb) { t1p.then( t1=>
                                    t1.c'(t2p).then( t3=> cb(t3) ) ) };
...' ) )

(where t1.c' is t1.c, modified to work with a promise)

Now, the call to t1.c' can go out after the .a roundtrip yet before the .b roundtrip completes. If you have also moved the callback code to the remote site, then the call to t1.c' could happen even without the .a roundtrip completing (from the perspective of the local site that triggered the chain) because t1 would be on the same site as the callback code and the remaining data.

This latter aspect of pipelining is simpler to do in the language implementation (unless the language itself supports sending of instantiated code, aka closures) - my point was merely to question the statement that .then would be in the way of such optimization.

Claus

# Mark S. Miller (11 years ago)

On Fri, Jun 21, 2013 at 1:30 AM, Claus Reinke <claus.reinke at talk21.com>wrote:

x.a().then( t1p=>

y.b().then( t2p=> let t3p = { then(cb) { t1p.then( t1=> t2p.then( t2=> t1.c(t2).then( t3=> cb(t3) ) ) ) }; ...' ) )

(where ...' is ..., transformed to work with a promise t3p instead of t3)

Now, waiting for the .a and .b roundtrips would be delayed until some code in ...' actually needs to look at t3. One could further delay looking at t2 if t1.c() could deal with a promise t2p.

[colon removed from above quote per Tab's suggestion]

oops, force of habit, as Tab guessed correctly.

Ok, you've delayed sending the .c until you needed t3p. But that's the

opposite of the promise pipelining optimization! The point is not to delay sending .c till even later. The point is to send it before the round trips from .a or .b complete. This code still cannot do that.

I do not expect to be able to emulate pipelining fully in user-land (at least not in JavaScript).

< code.google.com/p/google-caja/source/browse/trunk/src/com/google/caja/ses/makeQ.js>

supports promise pipelining in user land, using the makeRemote and makeFar extension points.

My aims were to demonstrate that .then does not need to stand in the way of such an optimization, and that the additional flexibility/ expressiveness provided by non-flattening .then is relevant here.

Back to your objection: there is nowhere to send the .c until you have t1 at hand. You could, however, move waiting on the t2 dependency to later, by passing the t2 receiver promise t2p to .c

x.a().then( t1p=> y.b().then( t2p=> let t3p = { then(cb) { t1p.then( t1=> t1.c'(t2p).then( t3=> cb(t3) ) ) }; ...' ) )

(where t1.c' is t1.c, modified to work with a promise)

Now, the call to t1.c' can go out after the .a roundtrip yet before the .b roundtrip completes. If you have also moved the callback code to the remote site, then the call to t1.c' could happen even without the .a roundtrip completing (from the perspective of the local site that triggered the chain) because t1 would be on the same site as the callback code and the remaining data. This latter aspect of pipelining is simpler to do in the language implementation (unless the language itself supports sending of instantiated code, aka closures) - my point was merely to question the statement that .then would be in the way of such optimization.

If we distinguish .then vs .there, you are describing .there above. With this distinction, do you agree that .then prevents this optimization?

# Claus Reinke (11 years ago)

< code.google.com/p/google-caja/source/browse/trunk/src/com/google/caja/ses/makeQ.js> supports promise pipelining in user land, using the makeRemote and makeFar extension points.

Hmm. If you are moving JS code (from the callbacks) to another site, some local references (including some that aren't even behind promises) become remote and vice versa. How do you manage this? And could you please link to an application that shows how makeRemote would be used in context?

If we distinguish .then vs .there, you are describing .there above. With this distinction, do you agree that .then prevents this optimization?

No. I described how a specific variant of .then, passing promises to callbacks, could account for more flexibility in resolution, time-wise, than a flattening .then could. Providing an interface that fits with the protocol of a remote-executing .there is just one application of this additional flexibility (and my code left the remote-executing aspects implicit).

For language-level futures, the lack of explicit nesting gives the implementation the freedom to rearrange resolution as needed. For JS promises, ruling out nesting robs programmers of the freedom to rearrange resolution explicitly.

Claus