Discuss
  • A Challenge Problem for Promise Designers (was: Re: Futures)

A Challenge Problem for Promise Designers (was: Re: Futures)

# Mark Miller (11 years ago)

I think we see a correlation -- not a 1.0 correlation, but something. Those who've actually used promise libraries with this flattening property find it pleasant. Those who come from either a statically typed or monadic perspective, or have had no experience with flattening promises, generally think they shouldn't flatten. Even if this correlation is there and even if it indicates what it seems to indicate, by itself it is hard to use as a basis for argument. It would merely shut out of the argument people without that experience, which is not what I intend and would not help us move forward.

At research.google.com/pubs/pub40673.html, Tom, Bill Tulloh, and I

wrote a paper where we demonstrate with a very small example, how beautiful programming with promises can be. This example is small enough that those who advocate different rules can try rewriting it using promises-as-they-wish-them-to-be and compare.

Even this exercise though would leave out the history of the code in the paper, and so not illuminate the refactoring point David just made. A very concrete example of the effect David is talking about is seen in Figure 1: The Mint Maker, on page 13. In the current paper, its deposit method reads:

deposit: (amount, srcP) =>
  Q(srcP).then(src => {
    Nat(balance + amount);
    m.get(src)(Nat(amount));
    balance += amount;
  })

where the body of the deposit method doesn't fire until the promise for the source purse, srcP, resolves to a source purse. Previously, it read

deposit: (amount, src) => {
  Nat(balance + amount);
  m.get(src)(Nat(amount));
  balance += amount;
}

requiring its clients to pass in a resolved source purse, rather than a promise for one. Some of the clients already had a resolved source purse to pass, so for these it was no problem, they just passed it. On page 12:

var ackP = paymentP ! deposit(10, myPurse);

Others had received a promise for a payment purse from elsewhere, that they intended to use as a source purse. They had to do a .then on this payment purse and then send .deposit only from inside the body of the then. The buy method of page 13 used to be:

buy: (desc, paymentP) =>
  Q(paymentP).then(payment => {
    // do whatever with desc, look up $10 price
    return (myPurse ! deposit(10, payment)).then(_ => good);
  })

This also came up in several places in the escrow exchange agent in Figure 2. The deposit method itself, having had no explicit return, would terminate either by returning undefined, indicating success, or throwing, indicating failure. The promise for the eventual result of the deposit method, such as ackP above, would thereby be a promise-for-undefined. It would eventually either fulfill successfully with undefined or be rejected with the thrown error as the reason.

The refactoring of putting the "Q(srcP).then" in the deposit method unburdened all clients such as the buy method above from doing this postponement themselves. The new buy method on page 13 now reads:

buy: (desc, paymentP) => {
  // do whatever with desc, look up $10 price
  return (myPurse ! deposit(10, paymentP)).then(_ => good);
}

The old deposit method returned undefined or threw. The new deposit method itself returns a promise-for-undefined that either fulfills to undefined or rejects. However, in both cases, the promise for what the deposit method will return remains the same, and so the buy method above did not become burdened with having to do a doubly nested then in order to find out whether the deposit succeeded. In addition, our first example line of client code

var ackP = paymentP ! deposit(10, myPurse);

did not have to change at all. The Q(srcP).then at the beginning of the deposit method will turn a purse into a promise for a purse, but will not turn a promise for a purse into a promise for a promise for a purse. The ackP remains a one-level promise whose fulfillment or rejection indicates whether the deposit succeeds or fails.

Call this refactoring "shifting the burden of postponement".

I hope this gives some sense about why those who've experienced such patterns like them. And I hope this provides a concrete and meaningful challenge to those who think promises should work otherwise.

# Claus Reinke (11 years ago)

I think we see a correlation -- not a 1.0 correlation, but something. Those who've actually used promise libraries with this flattening property find it pleasant. Those who come from either a statically typed or monadic perspective, or have had no experience with flattening promises, generally think they shouldn't flatten.

I think the dispute could be settled easily:

  • flattening 'then' is a convenience
  • non-flattening 'then' is necessary for promises being thenables (in the monad-inspired JS patterns sense)

Why not have both? Non-flattening 'then' for generic thenable coding, and a convenience method 'then_' for 'then'+flattening.

That way, coders can document whether they expect convenience or standard thenable behavior. And we can have convenience for Promise coding without ruining Promises for more general thenable coding patterns.

Claus

# Kevin Smith (11 years ago)

I think flattening is also tied inextricably to the fact that promises are a featureless wrapper for values. Nobody cares about promises-as-values because of this featureless-ness. And because they are completely uninteresting as values, programmers can think "straight through" to the eventual value.

This is highly simplifying for the programmer, especially in the context of complex asynchronous data flows. It is a valuable property and in a sense resembles a pure, universal currency.

This suggests a warning: if we admit promise subclassing (in which subclasses have extra features, such as cancel-ability), then this useful property goes away, and with it flattening.

# Mark Miller (11 years ago)

That's a good point. Neither the E language nor the Q library allow subclassing of promises. The motivating reason in both cases is the security properties that promises must provide. But you're right -- this is an additional benefit. Promises/A+, being a minimalistic codification of broader agreement, neither prohibits nor demands subclassing.

Both the E language and the Q library instead provide extension mechanisms other than subclassing[1] which are carefully designed so that such extended promises cannot violate the promise security properties. These extension mechanisms are sufficient to create lazy promises -- promises that compute their resolution of demand, and of course remote objects. This is how we keep the distributed object system out of the promise foundations itself, and instead build it as a layer on top.

[1] wiki.erights.org/wiki/Proxy#makeProxy

the explanation of E's makeProxy on page 10 of < research.google.com/pubs/pub36574.html> code.google.com/p/es-lab/source/browse/trunk/src/ses/makeQ.js#410

# Claus Reinke (11 years ago)

I'm still wading through the various issue tracker threads, but only two concrete rationales for flattening nested Promises have emerged so far:

1 "library author doesn't want nested Promises." 2 crossing Promise library boundaries can create unwanted nesting

There is little to be said about 1, only that those library authors still have a choice: add a separate recursive flattening operation and keep the thenable operations unharmed, or give up on Promises being thenables in the monad-inspired JS patterns sense (and hence give up on profiting from generic thenable library code).

The second point is somewhat more interesting, as it stems from yet another convenience-driven thenable deviation: if a then-callback does not return a Promise, its result is implicitly lifted into a Promise; the unwanted nesting apparently comes from different libs not recognizing each others promises, mistaking foreign promises for values, and lifting them into their own promises. Recursive flattening (assimilation) is then intended as a countermeasure to recursive lifting of foreign promises.

It will come as no surprise that I think implicit lifting is just as mistaken and recursive flattening;-) Both should be moved to explicit convenience methods, leaving the generic 'then'/'of' interface with the properties needed for generic thenable library code.

Claus

# Domenic Denicola (11 years ago)

Can you point to any code in wide use that makes use of this "thenables = monads" idea you seem to be implicitly assuming? Perhaps some of this "generic thenable library code"? I have never seen such code, whereas the use of "thenable" to mean "object with a then method, which we will try to treat as a promise" as in Promises/A+ seems widely deployed throughout libraries that are used by thousands of people judging by GitHub stars alone.

Thus I would say it's not promise libraries that are "harming the thenable operations," but perhaps some minority libraries who have misinterpreted what it means to be a thenable.

# Dean Tribble (11 years ago)

I've built multiple large systems using promises. A fundamental distinction that must be clear to the client of a function is whether the function "goes async": does it return a result that can be used synchronously or will the result only be available in a later turn. The .Net async libraries require the async keyword precisely to surface that in the signature of the function; i.e., it is a breaking change to a function to go from returning a ground result vs. a promise for a result. The same is basically isn't true for returning a promise that will only be resolved after several turns.

For example: I have a helper function to get the size of contents at the other end of a URL. Since that requires IO, it must return a Promise<int>.

size: (url) => { return url.read().then(contents => contents.length) }

This is obviously an expensive way to do it, and later when I get wired into some nice web caching abstraction, I discover that the cache has a similar operation, but with a smarter implementation; e.g., that can get the answer back by looking at content-length in the header or file length in the cache. That operation of course may require IO so it returns a promise as well. Should the client type be different just because the implementation uses any of several perfectly reasonable approaches for the implementation.

size: (url) => { return _cacheService.getLength(url) }

If in order to not change the signature, I have to "then" the result, it leads to

size: (url) => { return _cacheService.getLength(url).then(length => length) }

This just adds allocation and scheduling overhead for the useless then block, precludes (huge) tail return optimization, and clutters the code. This also leads to a depth of nesting types which is comparable to the function nesting depth (i.e., if x calls y calls z do I have promise<promise<promise<Z>>>?), which is overwhelming both to the type

checkers and to the programmers trying to reason about the code. the client invoked an operation that will eventually produce the integer they need.

There is also a relation between flattening and error propagation: consider that returning a broken promise is analogous to throwing an exception in languages with exceptions. In the above code, if the cache service fails (e..g, the URL is bogus), the result from the cache service will (eventually) be a rejected promise. Should the answer from the size operation be a fulfilled promise for a failed result? That would extremely painful in practice. Adding a layer of promise at each level is equivalent in sequential to requiring that every call site catch exceptions at that site (and perhaps deliberately propagate them). While various systems have attempted that, they generally have failed the usability test. It certainly seems not well-suited to the JS environment.

There are a few cases that may require "promise<promise<T>>". Most can be

more clearly expresses with an intermediate type. For example, in an enterprise security management system, the service manager returned a promise for a (remote) authorization service, but the authorization service might have been broken. Instead of returning a Promise<Promise<AuthorizationService>>, it returned

Promise<AuthorizationConnection> where AuthorizationConnection had a member "service" that returned a Promise<AuthorizationService>. When you deal

with higher level abstractions in a parameterized type system like C#s, however, you may end up with APIs that want to work across any T, including promises. If the abstractions internally use promises, then they may well end up with Promise<T> where T : Promise<U> or some such. Those are very

rare in practice, and can typically make use of operators (e.g., like Q) to limit their type nesting depth.

# Tab Atkins Jr. (11 years ago)

On Thu, Apr 25, 2013 at 4:30 PM, Dean Tribble <tribble at e-dean.com> wrote:

I've built multiple large systems using promises. A fundamental distinction that must be clear to the client of a function is whether the function "goes async": does it return a result that can be used synchronously or will the result only be available in a later turn. The .Net async libraries require the async keyword precisely to surface that in the signature of the function; i.e., it is a breaking change to a function to go from returning a ground result vs. a promise for a result. The same is basically isn't true for returning a promise that will only be resolved after several turns.

For example: I have a helper function to get the size of contents at the other end of a URL. Since that requires IO, it must return a Promise<int>.

size: (url) => { return url.read().then(contents => contents.length) }

This is obviously an expensive way to do it, and later when I get wired into some nice web caching abstraction, I discover that the cache has a similar operation, but with a smarter implementation; e.g., that can get the answer back by looking at content-length in the header or file length in the cache. That operation of course may require IO so it returns a promise as well. Should the client type be different just because the implementation uses any of several perfectly reasonable approaches for the implementation.

size: (url) => { return _cacheService.getLength(url) }

If in order to not change the signature, I have to "then" the result, it leads to

size: (url) => { return _cacheService.getLength(url).then(length => length) }

This just adds allocation and scheduling overhead for the useless then block, precludes (huge) tail return optimization, and clutters the code.

I don't understand this example. In the last one, if the return value of _cacheService.getLength(url) is a future already, why do you need to call .then() on it? Are you unsure of whether getLength() returns a Future<length> or a Future<Future<length>>? If so, getLength() is

terribly broken, and should be flattening stuff by itself to return a consistent type. We don't need to engineer around that kind of application design mistake at the language level.

This also leads to a depth of nesting types which is comparable to the function nesting depth (i.e., if x calls y calls z do I have promise<promise<promise<Z>>>?), which is overwhelming both to the type checkers and to the programmers trying to reason about the code. the client invoked an operation that will eventually produce the integer they need.

There is also a relation between flattening and error propagation: consider that returning a broken promise is analogous to throwing an exception in languages with exceptions. In the above code, if the cache service fails (e..g, the URL is bogus), the result from the cache service will (eventually) be a rejected promise. Should the answer from the size operation be a fulfilled promise for a failed result? That would extremely painful in practice. Adding a layer of promise at each level is equivalent in sequential to requiring that every call site catch exceptions at that site (and perhaps deliberately propagate them). While various systems have attempted that, they generally have failed the usability test. It certainly seems not well-suited to the JS environment.

I don't understand this either, probably because I don't understand the reason for the .then() in the earlier example. If cacheService.getLength() returns a future, then you don't need to do anything special in the size() function - just return the future that it returns. It sounds like you're nesting values in futures for the hell of it, which of course is problematic. Hiding the application's mistakes by auto-flattening isn't a good idea.

If you don't know whether a given function will return a bare value or a future, but you need to return a future, there's always Future.resolve(), which has the same semantics "unwrap 0 or 1 layers" semantics as Future#then.

I can't quite understand the point of the examples given here, so I may be misinterpreting them uncharitably. Could you elaborate on them, particularly on the cacheService example? What does cacheService.getLength() return, and what does size() need to return?

# Dean Tribble (11 years ago)

Hmm. I agree that the example code isn't relevant to JavaScript. For background, the last time issues this came up for me was in the context of a language keyword (which had other interesting but unrelated trade offs), where it really did impose that interaction (call sites had to declare that the type was a promise, and handle that, even though they were then returning promises). I'm glad we agree that needing to "then" in the tail-call case would be silly for a promise library. So what's an example that motivates you to want to build a tower of promise types? The main one I know of is the implementation (not use of) higher-order collection constructs that use promises internally (e.g., the implementation of map and reduce for an async, batching, flow-controlled stream of Promise<T>).

That kind of rare example can have more advanced hooks (like Q).

# Tab Atkins Jr. (11 years ago)

On Thu, Apr 25, 2013 at 6:03 PM, Dean Tribble <tribble at e-dean.com> wrote:

So what's an example that motivates you to want to build a tower of promise types? The main one I know of is the implementation (not use of) higher-order collection constructs that use promises internally (e.g., the implementation of map and reduce for an async, batching, flow-controlled stream of Promise<T>). That kind of rare example can have more advanced hooks (like Q).

I think it's more important to see examples of why you want to flatten them automatically. Having "towers of promises" keeps us consistent with Futures-as-monads, which is useful from a theoreticaly standpoint. This is similar, from an algebraic standpoint, to allowing arrays of arrays, or sets of sets, or any other monadic context doubled up.

# Tab Atkins Jr. (11 years ago)

On Thu, Apr 25, 2013 at 8:57 AM, Mark Miller <erights at gmail.com> wrote:

The refactoring of putting the "Q(srcP).then" in the deposit method unburdened all clients such as the buy method above from doing this postponement themselves. The new buy method on page 13 now reads:

buy: (desc, paymentP) => {
  // do whatever with desc, look up $10 price
  return (myPurse ! deposit(10, paymentP)).then(_ => good);
}

The old deposit method returned undefined or threw. The new deposit method itself returns a promise-for-undefined that either fulfills to undefined or rejects. However, in both cases, the promise for what the deposit method will return remains the same, and so the buy method above did not become burdened with having to do a doubly nested then in order to find out whether the deposit succeeded. In addition, our first example line of client code

var ackP = paymentP ! deposit(10, myPurse);

did not have to change at all. The Q(srcP).then at the beginning of the deposit method will turn a purse into a promise for a purse, but will not turn a promise for a purse into a promise for a promise for a purse. The ackP remains a one-level promise whose fulfillment or rejection indicates whether the deposit succeeds or fails.

Call this refactoring "shifting the burden of postponement".

I hope this gives some sense about why those who've experienced such patterns like them. And I hope this provides a concrete and meaningful challenge to those who think promises should work otherwise.

This same thing is handled by Future.resolve(), no? If you expect your function to receive either a value or a Future<value>, you can

just pass it through Future.resolve() to get a guaranteed Future<value>. It looks like it would result in roughly identical

code to your refactoring - rather than this (taken from your first code example):

deposit: (amount, srcP) =>
  Q(srcP).then(src => {
    Nat(balance + amount);
    m.get(src)(Nat(amount));
    balance += amount;
  })

You'd just do this:

deposit: (amount, srcP) =>
  Future.resolve(srcP).then(src => {
    Nat(balance + amount);
    m.get(src)(Nat(amount));
    balance += amount;
  })

Right?

Based on this example, it looks like the problem you're solving with recursive-unwrapping is that you speculatively wrap values in a promise, then rely on .then() to double-unwrap if necessary. If this is an accurate summary of the problem, then Future.resolve() solves it better - same code, but better theoretical semantics.

# Mark S. Miller (11 years ago)

What is the semantics of Future.resolve?

# Tab Atkins Jr. (11 years ago)

On Thu, Apr 25, 2013 at 6:49 PM, Mark S. Miller <erights at google.com> wrote:

What is the semantics of Future.resolve?

Creates an already-accepted future using the "resolve" algorithm, which is the same magic that happens to the return value of a .then() callback (if it's a future, it adopts the state; otherwise, it accepts with the value).

In other words, "If this is a future, use it; otherwise, make me a future for it".

# Mark S. Miller (11 years ago)

So how does the semantics of Q(x) differ from the semantics of Future.resolve(x) ?

# Ron Buckton (11 years ago)

I’m not sure I fully grok the use cases for FutureResolver#accept and having Future<Future<value>>. Having to call an Unwrap extension method on a Task<Task<T>> in .NET is an unfortunate necessity. Also, since Future#then implicitly resolves a future it is difficult to return a Future<Future<value>> from a then.

In every case where I've used something like a Future it has always seemed more convenient to have it implicitly unwrap.

For native Futures, I don’t think it makes sense to try and unwrap just any object with a callable “then”. Its a necessity today for Promise libraries in ES5 as there’s no ideal way to brand an object as a Future.

This is what seems to make sense to me from a practical standpoint when using native Futures:

  • If you resolve a Future (A) with a Future (B), the result of Future (A) should be B.
  • This implicit unwrap should only work out of the box for Future subclasses or branded Futures.
  • To coerce a “thenable” should be an explicit opt-in (e.g. Q(), or Future.of).
  • There should be a well-defined mechanism for chaining futures from subclasses to preserve capabilities (ProgressFuture, etc.). One option might be a FutureFactory, another is to have subclasses override the .then method.
  • An alternative to FutureResolver#accept that would allow for an explicit Future for a Future, might be to box the future, either explicitly (e.g. resolver.resolve({ future: f })) or with something like a Future.box() that encapsulates the future. In this way, if you need a future for a future you can support it on both FutureResolver and “then”.

A Future for a Future seems like a corner case compared to the broader simplicity of an implicit unwrap.

If we had Future.box() instead of FutureResolver#accept, we might be able to do things like:

function someFutureFutureV() { return new Future(function (resolver) { var F = someFutureV(); var Fboxed = Future.box(F); // some special instance with a .value property? // F === Fboxed.value; resolver.resolve(Fboxed); // i.e. resolver.resolve(Future.box(F)) instead of resolver.accept(F) }) }

someFutureFutureV().then(function (F) { // “then” unboxes Fboxed, just as it might have unwrapped F were it not boxed. // ... return Future.box(F); // another Fboxed }).then(function (F) { // F is again preserved, this time from a call to then // ... return F; // no boxing this time }).then(function (V) { // F is now unwrapped to V });

While slightly more complicated for the FutureFuture case, the expectations are simpler for the broader usage scenarios, and the API surface is simpler (no FutureResolver#accept) for the most common use cases. If you really do intend to have a FutureFuture, Future.box would let you have a single way to opt in for both FutureResolver#resolve as well as Future#then.

Ron

Sent from Windows Mail

From: Tab Atkins Jr. Sent: ‎Thursday‎, ‎April‎ ‎25‎, ‎2013 ‎8‎:‎38‎ ‎PM To: Mark S. Miller Cc: Mark Miller, es-discuss

On Thu, Apr 25, 2013 at 6:49 PM, Mark S. Miller <erights at google.com> wrote:

What is the semantics of Future.resolve?

Creates an already-accepted future using the "resolve" algorithm, which is the same magic that happens to the return value of a .then() callback (if it's a future, it adopts the state; otherwise, it accepts with the value).

In other words, "If this is a future, use it; otherwise, make me a future for it".

# Tab Atkins Jr. (11 years ago)

On Thu, Apr 25, 2013 at 8:52 PM, Mark S. Miller <erights at google.com> wrote:

So how does the semantics of Q(x) differ from the semantics of Future.resolve(x) ?

I suppose you tell me?

You offered, as an example of why recursive unwrapping was useful, some example code that used Q(val).then(). The surrounding explanatory text suggested that this helped with the case where "val" could be either a plain value or a promise.

I assumed this meant that Q() simple wraps its argument in a promise (like Future.accept() does), resulting in either a promise or a promise-for-a-promise, and .then() recursively unwrapped, so you ended up with the plain value at the end.

If that's not the case, and Q() does the same "conditional wrapping" that Future.resolve() does, then I don't understand the point of your example, or how your OP in this thread supports the assertion that recursive unwrapping is useful.

# Kevin Gadd (11 years ago)

Something that wasn't clear to me personally until reading the last few posts: I suspect that some of the negative reaction to unwrapping/wrapping, and the suggestion that Future<Future<T>> is a meaningful construct, comes

from the mindset of static typing - not in the sense that static types themselves are important, but that it feels implicitly 'wrong' or 'incorrect' to write a function that is uncertain as to whether a value is a future or not - being so incredibly haphazard about asynchronicity. This is probably the root of my personal opinion that unwrapping isn't a good thing. I've never seen a scenario like this in the .NET world, perhaps just because I've never encountered code that was designed with this sort of fundamental confusion about where asynchrony is necessary.

On the other hand, that kind of haphazardness/laziness is probably a pillar of successful JS applications and architectures, because it lets you get things done without arcane, perhaps inexplicable nuances of type systems and libraries getting in your way. That definitely enables more applications to be built faster.

Consider this my 2c regardless - the idea of functions being unsure about whether they need to be asynchronous, because a value might be a future or might not, and then passing that conceptual ambiguity on down the chain and on outward into the rest of the application, does bug me. My instincts suggest that it would lead to less maintainable code and more problems in the wild for users, even if unwrapping itself is an incredibly useful feature for these use cases. That is, unwrapping and wrapping are great, but maybe the fact that they are so essential indicates a real problem that should be addressed in the design of any standard Future primitive?

# David Bruant (11 years ago)

Le 26/04/2013 00:21, Claus Reinke a écrit :

I'm still wading through the various issue tracker threads, but only two concrete rationales for flattening nested Promises have emerged so far:

1 "library author doesn't want nested Promises." 2 crossing Promise library boundaries can create unwanted nesting

Perhaps you didn't read my post then? esdiscuss/2013-April/030192 I've shared experience on why flattening promises are convenient (easier refactoring, easier to reason about) and why non-flattening would be annoying (impose some sort of boilerplate somewhere to get to the actual value you're interested in).

Kevin Smith made a point that I believe is underestimated by non-flattening advocates:

I think flattening is also tied inextricably to the fact that promises are a featureless wrapper for values. Nobody cares about promises-as-values because of this featureless-ness. And because they are completely uninteresting as values, programmers can think "straight through" to the eventual value.

This is highly simplifying for the programmer, especially in the context of complex asynchronous data flows.

From experience, I couldn't care less for a wrapper for a wrapper for a value. I just want wrappers and values. Promises are just async boxes.

Beyond rationale, I'd like non-flattening advocates to show use cases where a Future<Future<T>> can be useful; more useful than just Future<T>

and T.

# Claus Reinke (11 years ago)

A Future for a Future seems like a corner case compared to the broader simplicity of an implicit unwrap.

The argument is not about whether Future<Future<...>> is a common

case. The Argument is that Future<...> and Array<...> and Optional<...>

and things that may raise catchable errors and other types have enough structure in common that it makes sense to write common library code for them.

One example is a map method, other examples may need more structure - eg, filter would need a way to represent empty structures, so not all wrapper types can support filter.

The goal is to have types/classes with common structure implement common interfaces that represent their commonalities. On top of those common interfaces, each type/class will have functionality that is not shared with all others. As long as the common and non-common interfaces are clearly separated, that is not a problem.

It is only when non-common functionality is mixed into what could be common interface methods, for convenience in the non-common case, that a type/class loses the ability to participate in code written to the common interface. That is why recursive flattening and implicit lifting of Promises is okay, as long as it isn't mixed into 'then'.

Claus

# David Bruant (11 years ago)

Le 26/04/2013 03:39, Tab Atkins Jr. a écrit :

On Thu, Apr 25, 2013 at 6:03 PM, Dean Tribble <tribble at e-dean.com> wrote:

So what's an example that motivates you to want to build a tower of promise types? The main one I know of is the implementation (not use of) higher-order collection constructs that use promises internally (e.g., the implementation of map and reduce for an async, batching, flow-controlled stream of Promise<T>). That kind of rare example can have more advanced hooks (like Q). I think it's more important to see examples of why you want to flatten them automatically. Having "towers of promises" keeps us consistent with Futures-as-monads, which is useful from a theoreticaly standpoint.

The Priority of Constituencies [1] asks us to be remain careful about theoretical standpoints. How does the theoretical part translates into helping users? authors (more than what I described at [2] which is derived from my own experience)? implementors? specifiers? I'm not saying the theoretical benefits don't exist, but I'd like to see how they translate in concretely improving my life as a developer using promises. I've explained the benefits I see for flattening from the dev point of view, I'd like to see the equivalent.

This is similar, from an algebraic standpoint, to allowing arrays of arrays, or sets of sets, or any other monadic context doubled up.

Arrays of arrays make sense as a data structure. Promise of promise does in theory, but as Kevin Smith said, in practice, promises are featureless values; the only thing you care about is that an operation is async and the actual non-promise eventual value that you'll get. Given this practical use of promises, why bother with Future<Future<T>>?

I don't mind if they exist, but as a developer, I'd like Future<Future<T>> to be hidden so that I don't have to deal with them. I

want all APIs I interact with to make me interact with promises and non-promises values and that's it. I don't care about the rest. Hide Future<Future<T>> under the carpet, please.

If super-experts want to play with Future<Future<T>>, give them, but

don't make that the default way to interact with promises. (as a side note, Mark Miller and Domenic Denicola are some of the most experts I know and they're also asking for flattening. It sounds strong enough of a signal for me to give on non-flattening; if they had any use, they would have expressed it).

David

[1] www.w3.org/TR/html-design-principles/#priority-of-constituencies [2] esdiscuss/2013-April/030192

# Andreas Rossberg (11 years ago)

On 26 April 2013 10:54, David Bruant <bruant.d at gmail.com> wrote:

The Priority of Constituencies [1] asks us to be remain careful about theoretical standpoints. How does the theoretical part translates into helping users? authors (more than what I described at [2] which is derived from my own experience)? implementors? specifiers? I'm not saying the theoretical benefits don't exist, but I'd like to see how they translate in concretely improving my life as a developer using promises. I've explained the benefits I see for flattening from the dev point of view, I'd like to see the equivalent.

The argument is for regularity. On a global scale, regularity helps the user, while exceptions, at best, are useful only locally -- an observation that, unfortunately, is difficult to demonstrate with toy examples. In particular, irregularity and exceptions become a pain when you start building abstractions, or plug together abstractions. In other words, regularity is a prerequisite for what some people (including me) like to call "compositionality".

Flattening for futures/promises is irregular because it means that certain behaviour reliably exists in all cases except when the input is itself a future/promise. This may not seem like a big deal if you use them directly. But now imagine somebody built a bigger generic abstraction that uses futures/promises only internally. You find that abstraction useful, and for whatever reason, need to funnel in a value that happens to be a future. And for no reason apparent to you the nice abstraction misbehaves in weird ways.

The nasty thing about lack of compositionality is that it typically only starts to harm after a while, when people start building more and more complex systems on top of a feature. But at that point, it's already too late, and you cannot fix the mistake anymore. Instead what happens is that more and more exceptions get introduced on top, to work around the problems created by the first one.

The functional programming community has a lot of experience with building compositional abstractions. And they have learned to value regularity very high. (And not entirely coincidentally, that also is the reason why they value type systems, because those force language and library designers to stay honest about regularity.)

# David Bruant (11 years ago)

[adding public-script-coord and Anne]

Le ven. 26 avril 2013 11:43:35 CEST, Andreas Rossberg a écrit :

On 26 April 2013 10:54, David Bruant <bruant.d at gmail.com> wrote:

The Priority of Constituencies [1] asks us to be remain careful about theoretical standpoints. How does the theoretical part translates into helping users? authors (more than what I described at [2] which is derived from my own experience)? implementors? specifiers? I'm not saying the theoretical benefits don't exist, but I'd like to see how they translate in concretely improving my life as a developer using promises. I've explained the benefits I see for flattening from the dev point of view, I'd like to see the equivalent.

The argument is for regularity. On a global scale, regularity helps the user, while exceptions, at best, are useful only locally -- an observation that, unfortunately, is difficult to demonstrate with toy examples.

I see. To a large extent, it's also very hard to explain benefits to refactoring. I gave an abstract example, Mark gave a more concrete small example, but admittedly, none really capture the benefit I have experience in a medium-sized Node.js application.

In particular, irregularity and exceptions become a pain when you start building abstractions, or plug together abstractions. In other words, regularity is a prerequisite for what some people (including me) like to call "compositionality".

Flattening for futures/promises is irregular because it means that certain behaviour reliably exists in all cases except when the input is itself a future/promise. This may not seem like a big deal if you use them directly. But now imagine somebody built a bigger generic abstraction that uses futures/promises only internally. You find that abstraction useful, and for whatever reason, need to funnel in a value that happens to be a future.

This last sentence and especially the ambiguous "for whatever reason" is the heart of the debate I believe. The idea expressed by Kevin Smith of promises as featureless values is that there should be no reason for you to funnel a value that happens to be a promise unless it's expected to be a promise and not a non-promise value. I have read somewhere (I can't remember where, hopefully MarkM will confirm or say if I imagined it) that in E, if a variable contains a promise and this promise is resolved, then the variable unwraps its value and referencing to the variable suddenly means referencing to the value. Promises are so deeply embedded in the language (syntax included) that this unwrapping is seamless. If I didn't imagine this, it'd be interesting if MarkM could expand on that as it seems to imply that promises shouldn't be values but lower-level than that.

Even if he confirms, it probably won't be possible to have something that embedded in JavaScript, but I think it's an interesting perspective.

More on the idea of "there should be no reason for you to funnel a value that happens to be a promise", that's obviously unless you're building a promise library or tools around promises.

One thing I have been convinced by your message is that it seems necessary to provide the lowest-level non-flattening primitives so that people build abstractions can that consider promises as values and will want to know about nested promise types. It should be possible, but building such abstractions doesn't seem like the 80% use case. Based on my experience, I remain pretty strong on the fact that flattening is a sensible default and really has practical advantages to large-scale development.

So, what about the following:

  1. Providing lowest-level non-flattening primitives so that library authors have fine-grained control and can build their own abstraction where they can consider promises as values (what is missing in the current API?)
  2. The API to be used by the average devs (then/catch/done/every/any/etc.) has flattening semantics as this is what people with the most experience have been advocating for (people with a different experience are still free to speak up). This API would just happen to be one possible library that can be built on top of 1), but one library which semantics has proven to be useful and popular among devs.

Would that satisfy everyone?

# Andreas Rossberg (11 years ago)

On 26 April 2013 12:19, David Bruant <bruant.d at gmail.com> wrote:

In particular, irregularity and exceptions become a pain when you start building abstractions, or plug together abstractions. In other words, regularity is a prerequisite for what some people (including me) like to call "compositionality".

Flattening for futures/promises is irregular because it means that certain behaviour reliably exists in all cases except when the input is itself a future/promise. This may not seem like a big deal if you use them directly. But now imagine somebody built a bigger generic abstraction that uses futures/promises only internally. You find that abstraction useful, and for whatever reason, need to funnel in a value that happens to be a future.

This last sentence and especially the ambiguous "for whatever reason" is the heart of the debate I believe. The idea expressed by Kevin Smith of promises as featureless values is that there should be no reason for you to funnel a value that happens to be a promise unless it's expected to be a promise and not a non-promise value.

I have read somewhere (I can't remember where, hopefully MarkM will confirm or say if I imagined it) that in E, if a variable contains a promise and this promise is resolved, then the variable unwraps its value and referencing to the variable suddenly means referencing to the value. Promises are so deeply embedded in the language (syntax included) that this unwrapping is seamless. If I didn't imagine this, it'd be interesting if MarkM could expand on that as it seems to imply that promises shouldn't be values but lower-level than that.

Even if he confirms, it probably won't be possible to have something that embedded in JavaScript, but I think it's an interesting perspective.

I'm not sure if your description of E is accurate -- I'd find that surprising. It is a perfectly sensible design to have transparent futures that you can just use in place of the value they eventually get resolved to (let's call that value their 'content'). In fact, that is how futures/promises where originally proposed (back in the 70s) and implemented (e.g. in MultiLisp in 85, Oz in 95, and others later). However, there are really only two consistent points in the design space:

  1. Either, futures are objects in their own right, with methods and everything. Then they should be fully compositional, and never be equated with their content, even after resolved (because that would change the meaning of a reference at locally unpredictable points in time).

  2. Or, futures are entirely transparent, i.e. can be used everywhere as a placeholder for their content. In particular, that means that 'f.then' always denotes the 'then' method of the content, not of the future itself (even when the content value is not yet available).

Unfortunately, option (2) requires that accessing a future that is not yet resolved has to (implicitly) block until it is (which is what happens in MultiLisp and friends). That only makes sense with threads, so I don't see how it can be reconciled with JavaScript.

One thing I have been convinced by your message is that it seems necessary to provide the lowest-level non-flattening primitives so that people build abstractions can that consider promises as values and will want to know about nested promise types. It should be possible, but building such abstractions doesn't seem like the 80% use case. Based on my experience, I remain pretty strong on the fact that flattening is a sensible default and really has practical advantages to large-scale development.

So, what about the following:

  1. Providing lowest-level non-flattening primitives so that library authors have fine-grained control and can build their own abstraction where they can consider promises as values (what is missing in the current API?)
  2. The API to be used by the average devs (then/catch/done/every/any/etc.) has flattening semantics as this is what people with the most experience have been advocating for (people with a different experience are still free to speak up). This API would just happen to be one possible library that can be built on top of 1), but one library which semantics has proven to be useful and popular among devs.

Would that satisfy everyone?

Maybe, although I fear that the distinction isn't all that clear-cut, and library writers will still feel encouraged to build non-compositional abstractions on top of the high-level API. After all, one programmer's high-level API is the next programmer's low-level API.

I have a strong feeling that the request for implicit flattening stems from the desire to somehow have your cake and eat it too regarding the choice between future semantics (1) and (2) above. Even if you want flattening, isn't an explicit flattening method good enough?

# Kevin Smith (11 years ago)

What exactly is the controversy here?

I think we all agree with the semantics of "then" as specified in Promises/A+. (If not, then we have a really big problem!)

If so, then the only real controversy is whether or not the API allows one to create a promise whose eventual value is itself a promise. Q does not: it provides only "resolve" and "reject". DOM Futures do by way of "Future.accept". As far as I know, there's nothing about Q's implementation that would make such a function impossible, it just does not provide one.

Do I have that right so far?

# Alex Russell (11 years ago)

Yes, you do.

# Kevin Smith (11 years ago)

On Fri, Apr 26, 2013 at 9:28 AM, Alex Russell <slightlyoff at google.com>wrote:

Yes, you do.

Mark or Domenic, is the point about Q true as well? (That it could, in principle, provide something like Future.accept, but it chooses not to.)

Just wanted to check before I say somethin' foolish : )

# Juan Ignacio Dopazo (11 years ago)

2013/4/26 Kevin Smith <zenparsing at gmail.com>

What exactly is the controversy here?

I think we all agree with the semantics of "then" as specified in Promises/A+. (If not, then we have a really big problem!)

Promise/A+ does not prohibit promises for promises. But in practice the problem is recognizing what is a promise. There are two options:

  1. Recognize all thenables as promises
  2. Recognize only the promises from your own library

Many implementations go with (1), including ODMFuture. Since it doesn't distinguish between promises and thenables, then() flattens all of them.

Juan

# Claus Reinke (11 years ago)

I'm still wading through the various issue tracker threads, but only two concrete rationales for flattening nested Promises have emerged so far:

1 "library author doesn't want nested Promises." 2 crossing Promise library boundaries can create unwanted nesting Perhaps you didn't read my post then? esdiscuss/2013-April/030192 I've shared experience on why flattening promises are convenient (easier refactoring, easier to reason about) and why non-flattening would be annoying (impose some sort of boilerplate somewhere to get to the actual value you're interested in).

Yes, I had seen that, but it doesn't explain where those nested Promises are supposed to come from. For a normal thenable thread (without implicit flattening or lifting), the nesting level should remain constant - .of(value) wraps value in a Promise, .then(cb) unwraps the intermediate result before passing it to cb, and cb constructs a new Promise.

In a later message, you suspect the reason for implicit flattening is fear of buggy code that may or may not wrap results in Promises. You say that such code may result from refactoring but, in JS, Promise<value>

is different from value, so trying to hide the level of promise nesting is likely to hide a bug. Yes, it is more difficult to spot such bugs in JS, but they need to be fixed nevertheless. Wrapping them in more duct-tape isn't helping.

Beyond rationale, I'd like non-flattening advocates to show use cases where a Future<Future<T>> can be useful; more useful than just Future<T> and T.

My own argument is not for nested futures themselves, but (1) for futures to offer the same interface (.of, .then) as other thenables, which (2) implies that there is to be no implicit lifting or flattening in .then.

In other words, you are worried about others giving you arbitrary nested promises and want to protect against that by implicit flattening, whereas I want to have control over the level of nesting and keep that level to one. For promises, I don't expect to use nested promises much, but I do expect to define and use thenable methods that should work for promises, too.

Claus

# Claus Reinke (11 years ago)

Can you point to any code in wide use that makes use of this "thenables = monads" idea you seem to be implicitly assuming? Perhaps some of this "generic thenable library code"? I have never seen such code, whereas the use of "thenable" to mean "object with a then method, which we will try to treat as a promise" as in Promises/A+ seems widely deployed throughout libraries that are used by thousands of people judging by GitHub stars alone.

Thus I would say it's not promise libraries that are "harming the thenable operations," but perhaps some minority libraries who have misinterpreted what it means to be a thenable.

Instead of rehashing the arguments from the various issue tracker threads, where examples have been presented even in the half (or less) I've read so far, let me try a different track: consider the case of Roman vs Arabic numerals.

As a user of Roman numerals, you might point to centuries of real world use in the great Roman empire, complain that Arabic numerals don't have explicit numbers for things like IX or LM etc, ask why anyone would need an explicit symbol representing nothing at all, or ask for examples of real world use of Arabic numerals in Roman markets, or say that Roman numerals don't need to follow the same rules as Arabic numerals, and that instead users of Arabic numerals have misinterpreted what it means to work with numbers.

All those arguments are beside the point, though. The point is that Arabic numerals (with 0) are slightly better than Roman numerals at representing the structure behind the things they represent, making it slightly easier to work with those things. And that is why Arabic numerals have won and Roman numerals are obsolete, after centuries of real-world use in a great empire.

Thenables in the JS-monadic sense represent common structure behind a variety of data types and computations, including Promises, they represent that structure well, and they give JS an equivalent to vastly successful computational structures in other languages.

And that isn't because I or someone famous says so, but because lots of people have worked hard for lots of years to figure out what those common structures are and how they might be represented in programming languages, refining the ideas against practice until we have reached a state where the only question is how and when to translate those ideas to another language, in this case JS.

Promises differ from other thenables, but there is no reason to burden the common interface with those differences.

Claus

# Domenic Denicola (11 years ago)

So there are no such libraries, and you are just wishing that they existed and that they took over the meaning of then from promises?

# David Bruant (11 years ago)

Le 26/04/2013 15:47, Claus Reinke a écrit :

My own argument is not for nested futures themselves, but (1) for futures to offer the same interface (.of, .then) as other thenables, which (2) implies that there is to be no implicit lifting or flattening in .then. For promises, I don't expect to use nested promises much, but I do expect to define and use thenable methods that should work for promises, too.

I see. So the underlying problem is the assimilation problem. If any object with a 'then' method is considered as a promise, then non-promise values with a 'then' method (that are not supposed to exist according to Promise/A+, but do anyway in real life [1]) will be mistakenly unwrapped if flattening semantics is the default...

I was in the naive belief that assimilation and flattening semantics were disjoint problems :-) Sorry about that.

If I suggest giving up on thenable, Domenic and others are going to starting throwing rocks at me, so I won't do that :-) Instead: (messed up) idea: a unique symbol to denote that a value is not a promise. Useful for the flattening semantics to know when to stop even if there is a 'then' method (which is admittedly a rare case anyway and can justify the use of a specific symbol).

Thoughts?

David

[1] casperjs.org/api.html#casper.then

# Domenic Denicola (11 years ago)

From: David Bruant [bruant.d at gmail.com]

Thoughts?

Since this entire problem seems predicated on Claus's misunderstanding of the term "thenable," which apparently has no basis in real libraries but instead entirely in wishful thinking, it might be more prudent for him to use the term "monad" instead of "thenable" and perhaps choose a different name for the method he wants, e.g. "flatMap" or "bind" instead of "then." That seems more aligned with reality than trying to change promise semantics based on some desire to co-opt the word "then" from promise libraries.

# David Sheets (11 years ago)

2013/4/26 Kevin Smith <zenparsing at gmail.com>

What exactly is the controversy here?

I believe the controversy is over the number of resolution iterations for a given wrapped value.

I think we all agree with the semantics of "then" as specified in Promises/A+. (If not, then we have a really big problem!)

I believe the semantics of "then" refer to the semantics of [[Resolve]] in Promises/A+. The controversy, as far as I can tell, is over whether [[Resolve]].2.3.1 at promises-aplus/promises-spec should read:

If/when resolvePromise is called with a value y, run [[Resolve]](promise, y).

OR

If/when resolvePromise is called with a value y, fulfill promise with y.

On Fri, Apr 26, 2013 at 2:43 PM, Juan Ignacio Dopazo <dopazo.juan at gmail.com> wrote:

Promise/A+ does not prohibit promises for promises. But in practice the problem is recognizing what is a promise. There are two options:

  1. Recognize all thenables as promises
  2. Recognize only the promises from your own library

Many implementations go with (1), including ODMFuture. Since it doesn't distinguish between promises and thenables, then() flattens all of them.

This does not appear to be the behavior specified in Promises/A+ as far as I can tell.

If onFulfilled is called with a "promise", that value is returned. If onFulfilled is called with a normal value, that value is lifted into a promise and returned. If onFulfilled is called with a non-promise "thenable", the "thenable" is chained and then re-resolved per 2.3.1.

It is this recursive re-resolution which is up for debate.

PRO:

Re-resolution prevents confusing nesting

CON:

Re-resolution causes confusing recursive behavior

In the common case (no nesting), both above versions of 2.3.1 behave identically. In the exceptional case, the non-recursive version is easier to reason about because the resolution doesn't strip away all your thenables.

From my reading, DOM Futures doesn't state anything about resolution

semantics, to its detriment, but abstracts those semantics behind FutureResolver.

Have I presented this correctly? Why is it a problem to specify a single level of assimilation instead of sometimes-flattening "thenables" but never flattening promises?

Have I missed something?

Thanks,

# David Sheets (11 years ago)

On Fri, Apr 26, 2013 at 3:20 PM, David Sheets <kosmo.zb at gmail.com> wrote:

2013/4/26 Kevin Smith <zenparsing at gmail.com>

What exactly is the controversy here?

I believe the controversy is over the number of resolution iterations for a given wrapped value.

I think we all agree with the semantics of "then" as specified in Promises/A+. (If not, then we have a really big problem!)

I believe the semantics of "then" refer to the semantics of [[Resolve]] in Promises/A+. The controversy, as far as I can tell, is over whether [[Resolve]].2.3.1 at promises-aplus/promises-spec should read:

If/when resolvePromise is called with a value y, run [[Resolve]](promise, y).

OR

If/when resolvePromise is called with a value y, fulfill promise with y.

On Fri, Apr 26, 2013 at 2:43 PM, Juan Ignacio Dopazo <dopazo.juan at gmail.com> wrote:

Promise/A+ does not prohibit promises for promises. But in practice the problem is recognizing what is a promise. There are two options:

  1. Recognize all thenables as promises
  2. Recognize only the promises from your own library

Many implementations go with (1), including ODMFuture. Since it doesn't distinguish between promises and thenables, then() flattens all of them.

This does not appear to be the behavior specified in Promises/A+ as far as I can tell.

If onFulfilled is called with a "promise", that value is returned. If onFulfilled is called with a normal value, that value is lifted into a promise and returned. If onFulfilled is called with a non-promise "thenable", the "thenable" is chained and then re-resolved per 2.3.1.

Gah! This, of course, should read:

If onFulfilled returns a "promise", that value is returned. If onFulfilled returns a normal value, that value is lifted into a promise and returned. If onFulfilled returns a non-promise "thenable", the "thenable" is chained and then re-resolved per 2.3.1.

# Domenic Denicola (11 years ago)

From: David Sheets [kosmo.zb at gmail.com]

From my reading, DOM Futures doesn't state anything about resolution semantics, to its detriment, but abstracts those semantics behind FutureResolver.

This is not correct. See "Let resolve be a future callback for the context object and its resolve algorithm." inside the resolve algorithm itself. DOM Futures are recursive, just like Promises/A+.

Have I presented this correctly?

Yes.

Why is it a problem to specify a single level of assimilation instead of sometimes-flattening "thenables" but never flattening promises?

The idea is that conformant libraries may want to prohibit promises-for-thenables (since, as discussed many times already, they are nonsensical, unless you care more about monads than you do about promises---which all promise libraries I know of do not). To do so, two things must occur:

  1. The library must never allow creation of promises-for-thenables. That is, it must not provide a fulfilledPromise(value) method, only a resolvedPromise(value) method. DOM Future violates this by providing accept(value), but presumably TC39-sanctioned promises will not provide such a method.

  2. The library must prevent thenables-for-thenables from turning into promises-for-thenables via assimilation. Instead, it must do recursive unwrapping.

In this way, Promises/A+ allows promises-for-promises within a library, if that library allows creation of such things in the first place (like DOM Future does). But it does not allow promises-for-thenables, i.e. it does not allow foreign promises-for-promises to infect a library with multiple layers of wrapping. Multi-layered wrapping must stay within a single library.

(In the above I am using "thenable" in the sense it is used today, i.e. "object with a then method," not Claus's version.)

# Kevin Smith (11 years ago)

Promise/A+ does not prohibit promises for promises. But in practice the problem is recognizing what is a promise.

I would say rather that we have two orthogonal, but highly interfering issues:

  1. Do we allow promises-(for-promises)+?
  2. How do we recognize a promise type within the "resolve" operation?

I was going to wait for confirmation on Q, but I'll go ahead and say somethin' foolish anyway : )

Proposal

  1. I don't see a problem with allowing nested promises, although I do see it as an anti-pattern which ought to be avoided if at all possible. I also see promise-subclassing as an anti-pattern for the same reasons.

  2. We test for promise-ness using an ES6 symbol, if available, and otherwise the string "then".

  • Let thenName be "then"
  • If the Symbol function is provided, then let thenName be Symbol()

Within the resolve operation, replace step 3 with:

  • If value is a JavaScript Object, set then to the result of calling the JavaScript [[Get]] internal method of value with property name thenName.

Add a Future.thenName property which returns thenName.

And finally, in the Future constructor, add something to this effect:

  • If thenName is not the string "then", then set future[thenName] = future.then

It's a little hacky but I'm sure it could be cleaned up by you W3C chaps. : )

Userland promise libraries would perform essentially the same operation in their initialization routines:

if (typeof Future === "function" && typeof Future.thenName === "symbol")
    this[Future.thenName] = this.then;
# David Sheets (11 years ago)

On Fri, Apr 26, 2013 at 3:27 PM, Domenic Denicola <domenic at domenicdenicola.com> wrote:

From: David Sheets [kosmo.zb at gmail.com]

From my reading, DOM Futures doesn't state anything about resolution semantics, to its detriment, but abstracts those semantics behind FutureResolver.

This is not correct. See "Let resolve be a future callback for the context object and its resolve algorithm." inside the resolve algorithm itself. DOM Futures are recursive, just like Promises/A+.

Ah, you are correct and this would appear to unnecessarily break expected identities. Though, it's at least consistent instead of special casing its own promises.

Have I presented this correctly?

Yes.

Why is it a problem to specify a single level of assimilation instead of sometimes-flattening "thenables" but never flattening promises?

The idea is that conformant libraries may want to prohibit promises-for-thenables (since, as discussed many times already, they are nonsensical, unless you care more about monads than you do about promises---which all promise libraries I know of do not). To do so, two things must occur:

  1. The library must never allow creation of promises-for-thenables. That is, it must not provide a fulfilledPromise(value) method, only a resolvedPromise(value) method. DOM Future violates this by providing accept(value), but presumably TC39-sanctioned promises will not provide such a method.

  2. The library must prevent thenables-for-thenables from turning into promises-for-thenables via assimilation. Instead, it must do recursive unwrapping.

In this way, Promises/A+ allows promises-for-promises within a library, if that library allows creation of such things in the first place (like DOM Future does). But it does not allow promises-for-thenables, i.e. it does not allow foreign promises-for-promises to infect a library with multiple layers of wrapping. Multi-layered wrapping must stay within a single library.

Why is there a semantic distinction between my thenables and your thenables?

If someone is using nested thenables, presumably they have a good reason. Promises/A+ acknowledges this possibility by allowing own-promises to nest. If we are interesting in constructing the "most standard" promises system, surely this system must grant other, foreign systems the same possibility of nesting own-promises without interference? Of course, these systems also control their own resolution semantics and could opt to magically flatten all results.

Could you point me to some code that needs dynamic flattening?

I understand the need for automatic lifting and 1-level assimilation. I use these a lot. I'm still fuzzy on the utility of flattening k levels for dynamic k.

Thanks,

# Domenic Denicola (11 years ago)

From: David Sheets [kosmo.zb at gmail.com]

Why is there a semantic distinction between my thenables and your thenables?

Because your thenables are not to be trusted! They could do pathological things like jQuery, or conceptually incoherent things like thenables-for-thenables. Sanitation at the boundary is the idea behind the resolve algorithm.

If someone is using nested thenables, presumably they have a good reason. Promises/A+ acknowledges this possibility by allowing own-promises to nest.

Yes, but more importantly, it preserves the guarantees within a single library---whether they be allowing promises-for-thenables, or disallowing them. Q, when, RSVP, and others guarantee no promises-for-thenables. That is a great feature for consumers of those libraries, as has been emphasized many times in this thread (especially elegantly, I think, by David Bruant). If there were no recursive foreign thenable assimilation, then promises-for-thenables could sneak into Q/when/RSVP promise systems, breaking the assumptions of consumers of those promises.

If we are interesting in constructing the "most standard" promises system, surely this system must grant other, foreign systems the same possibility of nesting own-promises without interference?

No. Generally, foreign systems must be normalized, for security concerns if nothing else. Trying to accommodate foreign system semantics into your own promise system is a recipe for disaster. Mark can expand upon this more in detail, if you think it's an important point.

Could you point me to some code that needs dynamic flattening?

From promises-aplus/promises-spec#101

var promise = getDataFromServerUsingQ().then(function (data) {
   return $('.foo').animate('opacity', data.opacityLevel).promise().then(function () {
       return updateBackboneModelViaSomeThirdPartyLibraryUsingUnderscoreDeferred().then(function () {
           return tellServerThatTheUpdateSucceededUsingQ();
       });
   });
});

If Q, as a proper Promises/A+ library, does recursive [[Resolve]], this is a promise for undefined that will be rejected with the appropriate error if any of the operations failed. But if it did one-level unwrapping, this would be a QPFAUDPFAQPFU, and it would always fulfill---failure information would have to be manually extracted.

Hope that helps!

# Tab Atkins Jr. (11 years ago)

On Fri, Apr 26, 2013 at 3:19 AM, David Bruant <bruant.d at gmail.com> wrote:

Le ven. 26 avril 2013 11:43:35 CEST, Andreas Rossberg a écrit :

On 26 April 2013 10:54, David Bruant <bruant.d at gmail.com> wrote:

The Priority of Constituencies [1] asks us to be remain careful about theoretical standpoints. How does the theoretical part translates into helping users? authors (more than what I described at [2] which is derived from my own experience)? implementors? specifiers? I'm not saying the theoretical benefits don't exist, but I'd like to see how they translate in concretely improving my life as a developer using promises. I've explained the benefits I see for flattening from the dev point of view, I'd like to see the equivalent.

The argument is for regularity. On a global scale, regularity helps the user, while exceptions, at best, are useful only locally -- an observation that, unfortunately, is difficult to demonstrate with toy examples.

I see. To a large extent, it's also very hard to explain benefits to refactoring. I gave an abstract example, Mark gave a more concrete small example, but admittedly, none really capture the benefit I have experience in a medium-sized Node.js application.

I disagree with all those examples, though. Not from a theoretical level, but from a "those examples don't show what you want" level.

Unless I'm reading Mark's concrete example wrong, he didn't actually show any nested promises at all - he showed a function that could take a plain value or a promise, and wants to always treat it as a promise (and return a promise). That has nothing to do with nested promises, and is already addressed by Futures anyway through Future.resolve(). (However, Mark hasn't been able to respond to this thread yet since my last message, so he may point out something that I missed.)

Your abstract example was:

If Future<Future<x>> can exist, then you'll have to write this boilerplate code in a lot of places: f.then(function res(v){ if(Future.isFuture(v)){ v.then(res); } else{ // actual work with the resolved value. } })

I don't understand why this boilerplate code has to exist, or why it does anything useful for you. Fundamentally, your example seems to show you having to react to a badly-authored outside environment, where somebody fucked up and double-wrapped a value by accident. This is exactly like arguing that Array#map needs to fully flatten, in case people accidentally pass in [[1, 2, 3]]. Your scare-boilerplate isn't even complete - if you're scared that the environment might be broken and doing extra wrapping by accident, it might be doing three or more levels of wrapping, too, so you've really got a problem on your hands.

It's impossible for this to happen by accident unless someone's code is broken. I don't think this is an easy mistake to make, in the first place, and if it is made, it should be fixed, not papered over by the language.

The main reason people have given for this potentially happening is authors mixing promise libraries, and the libraries not recognizing each other's promises, so .then() doesn't properly unwrap the return value of its callback. This seems rare on its face (most authors don't mix major libraries; authors on this list may be an exception), will be much less common as the standard Futures crowd out non-standard promises (not only will bespoke library promises be less common, but whatever recognition mechanism Futures uses will be copied by libraries, so the non-recognition problem will go away), and should be fixed by an explicit bandage, such as Future.flatten() or something, that can be applied by the author as necessary when crossing library boundaries. Breaking the regularity of the entire feature for a rare, temporary, author-predictable library-interop failure seems like a bad trade-off.

If you get stacked promises on purpose, then obviously you don't want to break the feature by having .then() recursively unwrap.

I would love to see a concrete example of a nested promise problem that's not a result of an authoring error (which seems hard to do on its face - at least with Futures, it seems difficult to accidentally double-wrap), a weird non-JS environment (like Dean Tribble's example, which turned out to be invalid for JS promises and DOM Futures), or a mixing of major libraries with bespoke promises that don't mutually recognize each other (because this will go away reasonably fast now that we have standardized Futures).

# David Sheets (11 years ago)

On Fri, Apr 26, 2013 at 4:03 PM, Domenic Denicola <domenic at domenicdenicola.com> wrote:

From: David Sheets [kosmo.zb at gmail.com]

Why is there a semantic distinction between my thenables and your thenables?

Because your thenables are not to be trusted! They could do pathological things like jQuery, or conceptually incoherent things like thenables-for-thenables. Sanitation at the boundary is the idea behind the resolve algorithm.

But I am the programmer! If my own tools distrust me, all hope is lost. With the present special casing of own-promises, there can only ever be a single semantics (own-promises) even if I have an alternative semantics that I would like to use.

If someone is using nested thenables, presumably they have a good reason. Promises/A+ acknowledges this possibility by allowing own-promises to nest.

Yes, but more importantly, it preserves the guarantees within a single library---whether they be allowing promises-for-thenables, or disallowing them.

It destroys the guarantee that foreign libraries are in charge of their own resolution. Preserving foreign libraries' guarantees would require meddling with their values the least (1-level assimilation).

Q, when, RSVP, and others guarantee no promises-for-thenables. That is a great feature for consumers of those libraries, as has been emphasized many times in this thread (especially elegantly, I think, by David Bruant).

That's a great feature for those programmers! If Q, when, and RSVP make that guarantee, they should enforce it by recursively resolving in their bind (or assimilation) operation, not relying on Promises/A+ or DOM Future to enforce it for them.

If there were no recursive foreign thenable assimilation, then promises-for-thenables could sneak into Q/when/RSVP promise systems, breaking the assumptions of consumers of those promises.

After their recursively resolving bind? How?

If they have a thenable and use their libraries' bind, their invariant is maintained.

If we are interesting in constructing the "most standard" promises system, surely this system must grant other, foreign systems the same possibility of nesting own-promises without interference?

No. Generally, foreign systems must be normalized, for security concerns if nothing else.

What, specifically, are these security concerns?

Trying to accommodate foreign system semantics into your own promise system is a recipe for disaster. Mark can expand upon this more in detail, if you think it's an important point.

It depends on what invariants are maintained, I believe. I've read quite a number of threads about this topic (including far too many pages of GitHub issues) and I've not, yet, come across something that indicates that delegating resolution to each own implementation breaks things.

Could you point me to some code that needs dynamic flattening?

From promises-aplus/promises-spec#101

var promise = getDataFromServerUsingQ().then(function (data) {
   return $('.foo').animate('opacity', data.opacityLevel).promise().then(function () {
       return updateBackboneModelViaSomeThirdPartyLibraryUsingUnderscoreDeferred().then(function () {
           return tellServerThatTheUpdateSucceededUsingQ();
       });
   });
});

This looks like a case for static resolution to me. Like this:

var promise = getDataFromServerUsingQ().then(function (data) {
   return Q($('.foo').animate('opacity',
data.opacityLevel).promise()).then(function () {
       return Q(updateBackboneModelViaSomeThirdPartyLibraryUsingUnderscoreDeferred()).then(function
() {
           return tellServerThatTheUpdateSucceededUsingQ();
       });
   });
});

I favor this expression because it explicitly invokes Q's behavior early and every use of 'then' is Q's 'then'. It costs 6 more bytes. Do you have any examples where you don't know until runtime how many thenables are wrapped?

If Q, as a proper Promises/A+ library, does recursive [[Resolve]], this is a promise for undefined that will be rejected with the appropriate error if any of the operations failed. But if it did one-level unwrapping, this would be a QPFAUDPFAQPFU, and it would always fulfill---failure information would have to be manually extracted.

If each of the wrapped promises also did one-level unwrapping, Q would not have to do dynamic unwrapping. If you use Q's assimilating constructor, you don't need dynamic unwrapping. If one-level unwrapping were the standard, all these libraries could interoperate without recursive resolution.

If we're looking for the most flexible, straightforward, easiest, and future-proofed semantics, it seems to me that decreasing the amount of magic inside the abstraction is nearly always better.

Does that make sense?

# Tab Atkins Jr. (11 years ago)

On Fri, Apr 26, 2013 at 8:45 AM, David Sheets <kosmo.zb at gmail.com> wrote:

Could you point me to some code that needs dynamic flattening?

From promises-aplus/promises-spec#101

var promise = getDataFromServerUsingQ().then(function (data) {
   return $('.foo').animate('opacity', data.opacityLevel).promise().then(function () {
       return updateBackboneModelViaSomeThirdPartyLibraryUsingUnderscoreDeferred().then(function () {
           return tellServerThatTheUpdateSucceededUsingQ();
       });
   });
});

This looks like a case for static resolution to me. Like this:

var promise = getDataFromServerUsingQ().then(function (data) {
   return Q($('.foo').animate('opacity',
data.opacityLevel).promise()).then(function () {
       return Q(updateBackboneModelViaSomeThirdPartyLibraryUsingUnderscoreDeferred()).then(function
() {
           return tellServerThatTheUpdateSucceededUsingQ();
       });
   });
});

I favor this expression because it explicitly invokes Q's behavior early and every use of 'then' is Q's 'then'. It costs 6 more bytes. Do you have any examples where you don't know until runtime how many thenables are wrapped?

Agreed. This is just an example of the libraries-with-mutually-unrecognizable-promises problem, which is a very specific issue with only a weak connection to the wider recursive-resolve problem. As I've argued before, and David is arguing here, the correct response to this problem is to have an explicit assimilation operation for converting foreign promises into your preferred version, and just sprinkling that around as necessary when crossing library boundaries.

The need for this will decrease now that DOM Futures exist, and libraries switch to using those (or a subclass of them) rather than rolling bespoke promises.

# Domenic Denicola (11 years ago)

From: Tab Atkins Jr. [jackalmage at gmail.com]

The need for this will decrease now that DOM Futures exist, and libraries switch to using those (or a subclass of them) rather than rolling bespoke promises.

Last I heard, jQuery has committed to never switching their promises implementation to one that works, for backward compatibility reasons. Rick might know more about if thinking has changed recently, though.

Even then, it's very naive to expect all code will be upgraded to subclass a DOM API.

# Tab Atkins Jr. (11 years ago)

On Fri, Apr 26, 2013 at 10:03 AM, Domenic Denicola <domenic at domenicdenicola.com> wrote:

From: Tab Atkins Jr. [jackalmage at gmail.com]

The need for this will decrease now that DOM Futures exist, and libraries switch to using those (or a subclass of them) rather than rolling bespoke promises.

Last I heard, jQuery has committed to never switching their promises implementation to one that works, for backward compatibility reasons. Rick might know more about if thinking has changed recently, though.

Yeah, that's fine. jQuery's promises are super-broken anyway, so it was unlikely they'd be able to change over to being compliant with the rest of the ecosystem. That's why we have explicit assimilation functions (or, in jQuery's case, I expect them to provide a function that converts their promise into a Future).

Even then, it's very naive to expect all code will be upgraded to subclass a DOM API.

I hear some disdain in your text here. Why, because it's a "DOM API"? Futures are only DOM because we needed it faster than TC39 could have provided it, but the intention is that they'll migrate into the ES spec in ES7 or so.

I do expect that a lot of smaller libraries which currently use bespoke promises to upgrade into Futures or Future subclasses, simple because they have less back-compat pressure and being auto-compatible with the standard version is useful. I also expect that new libraries will just use Futures or Future subclasses. (All of this will happen after browsers actually implement Futures, of course.)

# Rick Waldron (11 years ago)

On Apr 26, 2013 1:03 PM, "Domenic Denicola" <domenic at domenicdenicola.com> wrote:

From: Tab Atkins Jr. [jackalmage at gmail.com]

The need for this will decrease now that DOM Futures exist, and libraries switch to using those (or a subclass of them) rather than rolling bespoke promises.

Last I heard, jQuery has committed to never switching their promises implementation to one that works, for backward compatibility reasons. Rick might know more about if thinking has changed recently, though.

Before I respond, let me make it clear that I have no intention of arguing with anyone who chooses to follow up my comments. I don't agree with every decision made by every committer to jQuery, and I'm not going to defend decisions that I disagree with...

The libraries discussed in this and similar threads have the benefit of very limited adoption, where breaking changes incur minimal costs. jQuery doesn't have that luxury ;) and therefore won't break backward compatibility. I can assure you that we won't press for adoption of our implementation as a standard—despite its more than adequate qualification as a de facto standard (like it or not).

# Tom Van Cutsem (11 years ago)

(catching up on old threads, sorry for the asynchrony [no puns intended ;-)])

On 26 April 2013 12:19, David Bruant <bruant.d at gmail.com> wrote:

I have read somewhere (I can't remember where, hopefully MarkM will confirm or say if I imagined it) that in E, if a variable contains a promise and this promise is resolved, then the variable unwraps its value and referencing to the variable suddenly means referencing to the value. Promises are so deeply embedded in the language (syntax included) that this unwrapping is seamless.

I can confirm that this is correct: E promises, once fulfilled, are indistinguishable from their fulfilled value.

2013/4/26 Andreas Rossberg <rossberg at google.com>

I'm not sure if your description of E is accurate -- I'd find that surprising. It is a perfectly sensible design to have transparent futures that you can just use in place of the value they eventually get resolved to (let's call that value their 'content'). In fact, that is how futures/promises where originally proposed (back in the 70s) and implemented (e.g. in MultiLisp in 85, Oz in 95, and others later). However, there are really only two consistent points in the design space:

  1. Either, futures are objects in their own right, with methods and everything. Then they should be fully compositional, and never be equated with their content, even after resolved (because that would change the meaning of a reference at locally unpredictable points in time).

  2. Or, futures are entirely transparent, i.e. can be used everywhere as a placeholder for their content. In particular, that means that 'f.then' always denotes the 'then' method of the content, not of the future itself (even when the content value is not yet available).

Unfortunately, option (2) requires that accessing a future that is not yet resolved has to (implicitly) block until it is (which is what happens in MultiLisp and friends). That only makes sense with threads, so I don't see how it can be reconciled with JavaScript.

Your conclusion is based on the premise that the operation used to synchronize on the promise is expressed as a method.

E reconciles entirely transparent promises with non-blocking semantics by introducing a language construct, called "when", to await the value of a promise:

when ( promise ) -> { // code here executed in later turn, when promise is fulfilled // within this block of code, the promise is its fulfilled value }

Semantically, the "when" operation acts on the promise itself, not on its fulfilled value. This distinction is blurred in JavaScript as "when" is expressed as a ".then" method on an explicit promise object.

# Andreas Rossberg (11 years ago)

On 20 May 2013 14:15, Tom Van Cutsem <tomvc.be at gmail.com> wrote:

2013/4/26 Andreas Rossberg <rossberg at google.com>

I'm not sure if your description of E is accurate -- I'd find that surprising. It is a perfectly sensible design to have transparent futures that you can just use in place of the value they eventually get resolved to (let's call that value their 'content'). In fact, that is how futures/promises where originally proposed (back in the 70s) and implemented (e.g. in MultiLisp in 85, Oz in 95, and others later). However, there are really only two consistent points in the design space:

  1. Either, futures are objects in their own right, with methods and everything. Then they should be fully compositional, and never be equated with their content, even after resolved (because that would change the meaning of a reference at locally unpredictable points in time).

  2. Or, futures are entirely transparent, i.e. can be used everywhere as a placeholder for their content. In particular, that means that 'f.then' always denotes the 'then' method of the content, not of the future itself (even when the content value is not yet available).

Unfortunately, option (2) requires that accessing a future that is not yet resolved has to (implicitly) block until it is (which is what happens in MultiLisp and friends). That only makes sense with threads, so I don't see how it can be reconciled with JavaScript.

Your conclusion is based on the premise that the operation used to synchronize on the promise is expressed as a method.

E reconciles entirely transparent promises with non-blocking semantics by introducing a language construct, called "when", to await the value of a promise:

when ( promise ) -> { // code here executed in later turn, when promise is fulfilled // within this block of code, the promise is its fulfilled value }

What's your definition of "entirely transparent" then? Or in other words, what if I use 'promise' outside a when?

# Tom Van Cutsem (11 years ago)

2013/5/21 Andreas Rossberg <rossberg at google.com>

What's your definition of "entirely transparent" then? Or in other words, what if I use 'promise' outside a when?

I clarified this with Andreas in person, but FTR: "entirely transparent" is indeed the wrong word to describe E promises.

For context, E has two message passing operators, obj.m() indicates an immediate call (as in JS), obj<-m() indicates an eventual send, aka asynchronous message send, which returns a promise (this is the obj ! m() syntax proposed in < strawman:concurrency>).

Outside of "when"-blocks, promises are only transparent w.r.t. "<-", not w.r.t. "."

IOW: obj.m() will fail if obj is a promise, while obj<-m() will work "transparently", regardless of whether obj is a promise or non-promise.

I hope this clarifies things.