Discuss
  • Enable async/await to work on functions that don't just return promises.

Enable async/await to work on functions that don't just return promises.

# Codefined (8 years ago)

It strikes me as an interesting development to see that the current definition of Async/Await (as I see it), is just simple syntactic sugar for .then().  While I, to an extent, see the point of such a command as being useful, I remain unaware of the exact reasoning why we need to include promises in the first place.  Wouldn't it be so much more powerful to be able to use completely normal syntax, as you would in synchronous code as well as the option of promise chains? For example, take the following code snippet:

async function asyncFunction() {
  return new Promise((resolve, reject) => {
    someAsync('data', (err, data) => {
      if (err) {
        reject(err); return;
      }
      resolve(data);
    });
  }); 
}

This seems to be so very confusing for anybody new studying this language, almost everyone I talk to gets stuck up on some part of it.  Wouldn't it be so very beautiful if we could just do:

async function asyncFunction() {
	someAsync('data', (err, data) => {
		async return [err, data]
	})
}

When we call this with the await keyword, we simply await a return.  No special promises or extra keywords needed, everything works as you would expect it to.  This also has the benefit of shortening our required code from 10 lines to 5 lines, removing a significant proportion of boilerplate code.

async function asyncFunction() {
	let [err, data] = await asyncFunction()
}

Some interesting counterpoints to consider from the #node.js channel on freenode
 - "Doesn't this just add complexity to the specification?"
   + Yes, it does mean that people who wish to implement Javascript will have to spend longer implementing await/async.  However, in my opinion it's a better solution that forcing everyone who wishes to use async/await to also learn how to use new Promise() and what the reject() and resolve() do.  Due to how often you have to deal with asynchronous code in Javascript, often people come across this issue very early on in learning the language, making is as easy as adding an extra async before your return statement seems like an acceptable exchange.
 - "Why not just automatically promisify functions using a library like bluebird?"
   + Similar to the previous question, I honestly feel that forcing people to learn the API for yet another library in order to be able to do such a simple task is much more taxing than in this method.
 - "I just don't like the async return"
   + Originally it was just a return, but a friend pointed out this ran into issues that it would simply return from the last function.  What I thought would be much easier was some sort of keyword that makes it return from the async function, instead of any other functions.  To me, async came naturally, but this is just my first suggestion on the future of Javascript, and I'd be interested to know if you have any better suggestions.

# Florian Bösch (8 years ago)

On Sat, Feb 25, 2017 at 11:55 PM, Codefined <codefined at debenclipper.com>

wrote:

This seems to be so very confusing for anybody new studying this language, almost everyone I talk to gets stuck up on some part of it.

Promises are bad, and mixing them with async/await is worse. Should never have been added to any kind of standard.

async function asyncFunction() {let [err, data] = await asyncFunction() }

function asyncFunction(){ return otherAsyncFunction(); }

Even simpler, you'd just need co-routines.

# Isiah Meadows (8 years ago)

Here's my thoughts:

  1. There's minimal benefit to be gained with your example, since it can already be simulated very similarly with the current API:

function asyncFunction() { return new Promise(resolve => { someAsync('data', (...args) => resolve(args)) }) }

  1. This won't alleviate you of the need to learn promises anyways, and in general, when you're working with async code, you should learn the Promise APIs. Otherwise, async functions won't make much sense to you, and you're liable to forget an await when you needed one.

  2. The concepts aren't really that complicated. It's already easy to explain as an asynchronous return or throw, just they aren't syntactic at the start because you can't return from a function in a callback.[1] You don't have to explain monadic join or bind just to explain how a promise works.

[1] Kotlin is an exception here, in that it allows explicitly non-local jumps, provided they're correctly typed.

Isiah Meadows me at isiahmeadows.com

# Codefined (8 years ago)

Hello Isiah,

I'm not sure you understand my point on why this is important.  Although if you understand all the concepts used here it's logical, it's very intimidating to a person newer to Javascript.  This could be especially true in a framework like Node.JS, where you can run into a situation like this in your first program (for example, reading/writing to a file using promises).  Take: function asyncFunction() { return new Promise(resolve => { someAsync('data', (...args) => resolve(args)) }) }

Here you have to explain the concept of returning a promise, anonymous functions, array decomposition and what resolving does.  Remember that this could be required to just write to a file. However, if one was to explain: function asyncFunction() { someAsync('data', data => { async return data }) }

Then one only has to talk about anonymous functions and how async return works.  You say in your second point you'll need to learn promises anyway, but you can delay learning about them for a very long time.  I'm struggling to think of a simple program that requires the use of promises over this syntax.  People are able to use things without needing to know how the internal components.  For example, I don't know even vaguely how bcrypt works under the surface but I still use it in almost every project I work on.  I'm sure you can think of similar examples where you don't know the exact implementation of something but still use it often without forgetting the necessary syntax (like await)..

And I'm afraid I have to completely disagree with your third point.  The concepts are very complicated to a beginner, especially considering how many different concepts you have to learn at once within just your first couple of programs.  We're talking about learning 3-4 new concepts things in just 5 lines of a program that wouldn't be out of place as the third or fourth program you wrote. On 26/02/2017 00:15:14, Isiah Meadows <isiahmeadows at gmail.com> wrote:

Here's my thoughts:

  1. There's minimal benefit to be gained with your example, since it can already be simulated very similarly with the current API:

function asyncFunction() { return new Promise(resolve => {

someAsync('data', (...args) => resolve(args)) }) }

  1. This won't alleviate you of the need to learn promises anyways, and in general, when you're working with async code, you should learn the Promise APIs. Otherwise, async functions won't make much sense to you, and you're liable to forget an await when you needed one.

  2. The concepts aren't really that complicated. It's already easy to explain as an asynchronous return or throw, just they aren't syntactic at the start because you can't return from a function in a callback.[1] You don't have to explain monadic join or bind just to explain how a promise works.

[1] Kotlin is an exception here, in that it allows explicitly non-local jumps, provided they're correctly typed.

Isiah Meadows me at isiahmeadows.com

# Alexander Jones (8 years ago)

Florian, you shouldn't pass the argument of explicit vs implicit coroutines off as being so simple. There are many compelling arguments for both! Please Google them!

# Florian Bösch (8 years ago)

It would be nice if there even was an argument, but there isn't. There isn't because async/await naturally devolves into implicit coroutines, so any argument would be moot.

To illustrate, suppose you have these 4 functions:

let a = function(){ return b(); }

let b = function(){ return c(); }

let c = function(){ return d(); }

let d = function(){ return whatever(); }

Call chains like this are typical. It's the staple of software engineering (for reasons of proper separation of concerns, reuse of utility code, etc.). If you believe that there is an argument about this being exemplary, it would be impossible to have an argument with you about software engineering at all. Of course real-world examples are more complex and don't just return whatever the underlying function produced, but as a control flow example it suffices. These chains are often much deeper than 4 levels, it's not uncommon to encounter call chains 10, 15, 20 or 30 layers deep.

Now let's suppose you figure that function d wants to do something asynchronous.

So you go and do:

let d = function(){ return xhr(); }

But of course that doesn't work, because d is not async. So you go and do:

let d = async function(){ return await xhr(); }

Of course that doesn't work because c is not async, and so forth, so eventually your code looks like that.

let a = async function(){ return await b(); }

let b = async function(){ return await c(); }

let c = async function(){ return await d(); }

let d = async function(){ return await xhr(); }

In essence, you've applied to following two regular expression: s/function/await function/g and s/.+?()/await ()/ . Of course that'd be horrid to do, so in reality you'd please use a proper JS parser. How did your code look before you applied these regular expressions? Well, it looks exactly like at the start.

let a = function(){ return b(); }

let b = function(){ return c(); }

let c = function(){ return d(); }

let d = function(){ return xhr(); }

But it isn't like at the start, because now it can trigger race conditions and is asynchronous. It is in fact now idempotent with true co-routines, except some unnecessary code transmoglification.

This conclusively proves that await/async is an inconvenient clutch that naturally devolves into true co-routines. Now you might try to argue, that real-world code isn't just going to prefix every function call with await and every function body with async and stay that way.

However, this would be in invalid argument for actual real-world code, because. People don't just constantly switch back and forth and re-engineer their code just because they want something async to happen underneath. You don't go and bicycle repair every call and function definition if you should decide to toggle synchronous or asynchronous. Therefore, since prefixing everything works no matter if it is asynchronous or synchronous, you will stay with the prefixes once you've added them. Which not only guarantees that async/await devolves into true co-routines, but it also gurantees that they proliferate everything and once they're in, they're never going out.

And that's why it isn't an argument.

# Jerald Cohen (8 years ago)

I actually really like this idea. It reminds me of the C# syntax to try to accomplish the same thing. Although under the bonnet C# uses "tasks", which appear to be their form of promises, the surface code does look very similar, take for example:

public async Task MyMethodAsync(){
    Task<int> longRunningTask = LongRunningOperationAsync();
    // independent work which doesn't need the result of
LongRunningOperationAsync can be done here

    //and now we call await on the task
    int result = await longRunningTask;
    //use the result
    Console.WriteLine(result);}
public async Task<int> LongRunningOperationAsync() // assume we return

an int from this long running operation {
    await Task.Delay(1000); //1 seconds delay
    return 1;
}

Taken from this stackoverflow question stackoverflow.com/questions/14455293/how-and-when-to-use-async-and-await.

I would be completely in support of shifting async/await from requiring promises and instead allowing just the ability to use semi-normal "async returns" (although, at the moment, I fail to see why one can't just use a normal "return").

# Florian Bösch (8 years ago)

To illustrate that, I went to Github search?l=JavaScript&q=await&ref=simplesearch&type=Repositories&utf8=✓,

I looked for repositories that match "await" and are JS (471 repositories). The first few hits where microframeworks with minimal code or language parser, but then comes:

Starhackit FredericHeem/starhackit/tree/master/client/src (3rd

search result hit) some kind of full-stack application framework):

And so on. I could go through most of these non minimal code repositories, and show you how every single one of them has async/await proliferated through their entire call chain top to bottom.

I hope you can understand that my "theory" on actual use is therefore not just idle speculation, it's actual use reality, and you should consider that.

# Florian Bösch (8 years ago)

On Sun, Feb 26, 2017 at 2:08 PM, Jerald Cohen <cohenjerald7 at gmail.com>

wrote:

(although, at the moment, I fail to see why one can't just use a normal "return").

You're probably unaware of this, but there is a fixation in this community, and many adjacent ones, that if you sprinkle syntax magic dust atop co-routines, it fixes issues of concurrency (it doesn't). But in a nutshell, that's why you can't just use a normal "return".

# Codefined (8 years ago)

Hey Florian,

Why do we even need to do that? async function someAsync() { something(() => { return "hello"; }); } let d = function() { return await someAsync(); } let c = function() { return d(); }

Because d() is no longer an asynchronous function, you can call it like normal, surely?

On 26/02/2017 12:44:01, Florian Bösch <pyalot at gmail.com> wrote:

It would be nice if there even was an argument, but there isn't. There isn't because async/await naturally devolves into implicit coroutines, so any argument would be moot.

To illustrate, suppose you have these 4 functions:

let a = function(){   return b(); }

let b = function(){   return c(); }

let c = function(){    return d(); }

let d = function(){   return whatever(); }

Call chains like this are typical. It's the staple of software engineering (for reasons of proper separation of concerns, reuse of utility code, etc.). If you believe that there is an argument about this being exemplary, it would be impossible to have an argument with you about software engineering at all. Of course real-world examples are more complex and don't just return whatever the underlying function produced, but as a control flow example it suffices. These chains are often much deeper than 4 levels, it's not uncommon to encounter call chains 10, 15, 20 or 30 layers deep.

Now let's suppose you figure that function d wants to do something asynchronous.

So you go and do:

let d = function(){   return xhr(); }

But of course that doesn't work, because d is not async. So you go and do:

let d = async function(){   return await xhr(); }

Of course that doesn't work because c is not async, and so forth, so eventually your code looks like that.

let a = async function(){   return await b(); }

let b = async function(){   return await c(); }

let c = async function(){    return await d(); }

let d = async function(){   return await xhr(); }

In essence, you've applied to following two regular expression: s/function/await function/g and s/.+?()/await ()/ . Of course that'd be horrid to do, so in reality you'd please use a proper JS parser. How did your code look before you applied these regular expressions? Well, it looks exactly like at the start.

let a = function(){   return b(); }

let b = function(){   return c(); }

let c = function(){    return d(); }

let d = function(){   return xhr(); }

But it isn't like at the start, because now it can trigger race conditions and is asynchronous. It is in fact now idempotent with true co-routines, except some unnecessary code transmoglification.

This conclusively proves that await/async is an inconvenient clutch that naturally devolves into true co-routines. Now you might try to argue, that real-world code isn't just going to prefix every function call with await and every function body with async and stay that way.

However, this would be in invalid argument for actual real-world code, because. People don't just constantly switch back and forth and re-engineer their code just because they want something async to happen underneath. You don't go and bicycle repair every call and function definition if you should decide to toggle synchronous or asynchronous. Therefore, since prefixing everything works no matter if it is asynchronous or synchronous, you will stay with the prefixes once you've added them. Which not only guarantees that async/await devolves into true co-routines, but it also gurantees that they proliferate everything and once they're in, they're never going out.

And that's why it isn't an argument.

On Sun, Feb 26, 2017 at 12:05 PM, Alexander Jones <alex at weej.com [mailto:alex at weej.com]> wrote:

Florian, you shouldn't pass the argument of explicit vs implicit coroutines off as being so simple. There are many compelling arguments for both! Please Google them!

On Sun, 26 Feb 2017 at 00:01, Florian Bösch <pyalot at gmail.com [mailto:pyalot at gmail.com]> wrote:

On Sat, Feb 25, 2017 at 11:55 PM, Codefined <codefined at debenclipper.com [mailto:codefined at debenclipper.com]> wrote:

This seems to be so very confusing for anybody new studying this language, almost everyone I talk to gets stuck up on some part of it. Promises are bad, and mixing them with async/await is worse. Should never have been added to any kind of standard. async function asyncFunction() { let [err, data] = await asyncFunction() } function asyncFunction(){   return otherAsyncFunction(); }

Even simpler, you'd just need co-routines.

# Florian Bösch (8 years ago)

On Sun, Feb 26, 2017 at 2:17 PM, Codefined <codefined at debenclipper.com>

wrote:

Because d() is no longer an asynchronous function, you can call it like normal, surely?

Only async functions can await. Only await pauses execution to wait on async.

# Codefined (8 years ago)

Ah, that would indeed make sense.  What I feel we need is some way of making an asynchronous function "appear" to be synchronous.  Actually that's sort of what I expected async/await to do, but that only seems to affect where it's called, as opposed to the function itself.

# Florian Bösch (8 years ago)

On Sun, Feb 26, 2017 at 2:51 PM, Codefined <codefined at debenclipper.com>

wrote:

What I feel we need is some way of making an asynchronous function "appear" to be synchronous.

That's what co-routines are. Best practices for co-routines is usually to have some sort of API to spawn them (like new Routine(somefunction)), to be able to switch to them (like someroutine.switch(funargs)) and to throw exceptions into them (like someroutine.throw(error)) as well as a way to obtain the currently used routine (like Routine.current).

# Jerald Cohen (8 years ago)

Florian,

You sure you're not just adding more complexities to a language with features that were meant to remove such complexity?

Codefined's solution to me seems to be the one with the least amount of added techniques in order to learn. Although I understand how co-rountines are awesome, they can quickly get very confusing when you switch contexts.

# Florian Bösch (8 years ago)

await/async are de-facto co-routines. await/async will infect all code by necessity of software engineering. At which point they're actual co-routines, implemented badly, without a proper API.

# Codefined (8 years ago)

Thank-you guys for the excellent problems with the previous idea, to attempt to fix as many of them as I can, here is an alteration to the proposal:

You have two types of functions, asynchronous and synchronous functions.  To distinguish between them, an asynchronous function has the keyword of async prepended before it.  Also, unlike the return statement in a synchronous function, the asynchronous code has async return.  Examples:

function someSync() {
	return "Sync";
}

async function someAsync() {
	setTimeout(() => {
		async return "Async";
	}, 1000);
}

The synchronous code is called as one would expect, however the asynchronous code can be called in two ways: function calleeOne() { let x = await someAsync(); } function calleeTwo() { let x = someAsync(); } You'll notice the main difference between this latest iteration and the first proposal is that you do not need to put async around your callee function.  In the first case, the code waits until x has a value, here the value will be "Async" after one second.  In the second case, the code continues on instantly, with x being an automatically wrapped promise.  This has many advantages, including the most obvious example given by Florian: // Assumes a function someAsync() which is an asychronous function. function c() { return d() } function d() { return await someAsync() } You'll notice we've simply made someAsync() as a function, synchronous.  No code changes are needed above the c() function, unlike in the previous proposal iteration.

I'll be interested to see what you guys consider the advantages/disadvantages of this method, which I hope to be "the middle between two ends" on whether to go fully into promises or fully into co-routines.  Neither of which are in my opinion the optimal stance. On 26/02/2017 14:31:30, Florian Bösch <pyalot at gmail.com> wrote:

await/async are de-facto co-routines. await/async will infect all code by necessity of software engineering. At which point they're actual co-routines, implemented badly, without a proper API.

On Sun, Feb 26, 2017 at 3:26 PM, Jerald Cohen <cohenjerald7 at gmail.com [mailto:cohenjerald7 at gmail.com]> wrote:

Florian,

You sure you're not just adding more complexities to a language with features that were meant to remove such complexity?

Codefined's solution to me seems to be the one with the least amount of added techniques in order to learn.  Although I understand how co-rountines are awesome, they can quickly get very confusing when you switch contexts.

# Florian Bösch (8 years ago)

On Sun, Feb 26, 2017 at 5:30 PM, Codefined <codefined at debenclipper.com>

wrote:

I'll be interested to see what you guys consider the advantages/disadvantages of this method, which I hope to be "the middle between two ends" on whether to go fully into promises or fully into co-routines. Neither of which are in my opinion the optimal stance.

Hereabouts nobody's going to either full co-routines or even semi-implicit co-routines. But JS doesn't matter, just compile to asm.js/WebAssembly or write a bytecode engine atop JS etc. from a language that actually solves concurrent programming well and isn't a hodgepodge of missed opportunities.

# Codefined (8 years ago)

You seem to be feel incredibly jaded about nearly everything posted here.  Perhaps if you suggested your own proposal that showed the clear advantages of co-routines as you see it, then you might solve some of the issues instead of just whining about it.

I assume that every single Javascript developer out there just wants the best for the language as a whole, so maybe you should look at the counter-arguments to co-routines to see where they're coming from and attempt to fix them? On 26/02/2017 16:35:16, Florian Bösch <pyalot at gmail.com> wrote:

On Sun, Feb 26, 2017 at 5:30 PM, Codefined <codefined at debenclipper.com [mailto:codefined at debenclipper.com]> wrote:

I'll be interested to see what you guys consider the advantages/disadvantages of this method, which I hope to be "the middle between two ends" on whether to go fully into promises or fully into co-routines.  Neither of which are in my opinion the optimal stance.   Hereabouts nobody's going to either full co-routines or even semi-implicit co-routines. But JS doesn't matter, just compile to asm.js/WebAssembly or write a bytecode engine atop JS etc. from a language that actually solves concurrent programming well and isn't a hodgepodge of missed opportunities.

# Alexander Jones (8 years ago)

Required reading for anyone who wants to be so... opinionated ;)

joeduffyblog.com/2015/11/19/asynchronous-everything

# Codefined (8 years ago)

A very interesting read indeed Alexander!  Gave me a new example to give when people ask what the worst code I'd ever seen was:

Promise<void> DoSomething(Promise<string> cmd) {
    return cmd.WhenResolved(
        s => {
            if (s == "...") {
                return DoSomethingElse(...).WhenResolved(
                    v => {
                        return ...;
                    },
                    e => {
                        Log(e);
                        throw e;
                    }
                );
            }
            else {
                return ...;
            }
        },
        e => {
            Log(e);
            throw e;
        }
    );
}

My question is however, that this article defines that an await keyword must be within an async function, which seems an interesting choice to me.  For example, the following code snippet:

function callee() {
    let x = await someAsync(); 
}

Should callee() be asynchronous here?  To my mind, no, it shouldn't.  Every single line here is synchronous, so the function itself should surely be synchronous.  Shouldn't functions that may not have await in them, but instead that are actually asynchronous and hence use the async return keyword be the ones we define with async?

Although, saying that, I think I may steal their use of async Bar() when calling functions and add it to the proposal.  It makes sense to me that let x = someAsync() should error out to let developers know they're doing something wrong.  let x = async someAsync() clearly does show that they know the function is async and will return a promise..

# Dean Tribble (8 years ago)

Should callee() be asynchronous here? To my mind, no, it shouldn't. Every single line here is synchronous, so the function itself should surely be synchronous. Shouldn't functions that may not have await in them, but instead that are actually asynchronous and hence use the async return keyword be the ones we define with async?

In the Javascript (and Midori) model, concurrent execution of multiple activities is achieved by breaking those activities up into coarse-grained, application-defined "turns" (or "jobs") and interleaving those. An async boundary is where the current turn could end, and the turns for other concurrent activities might run, changing the state before the current activity proceeds.

Therefore, callee must be async, because that declares that there could be a turn boundary within it, and thus, the rest of the state of the program could change as a result of the call. The caller of callee *must *ensure that it's invariants are correct before allowing other code to interleave with it.

# Mark (8 years ago)

Codefined, just out of curiousity, do you have anything to do with this proposal that got announced today tc39/proposals#41? Or is it just a coincidence? :) ​

# Isiah Meadows (8 years ago)

May I add one more thing: the main topic this was about is adapting non-standard async APIs (like Node's error-first callback idiom) to the land of promises. Async functions and iterators are incredibly useful when you're dealing with just promises, especially consuming them, but this is about creating promise adapters, not consuming promises.

# Andy Earnshaw (8 years ago)

I had a similar thought a while ago for adapting non-promise functions, by way of async.resolve() and async.reject():

async function asyncFunction() {
    someAsync('data', (err, data) => {
        if (err) async.reject(err);
        async.resolve(data);
    })
}

This differs from your proposal in that it's more explicit. Neither really give much benefit beyond saving a few bytes of code, probably not worth the extra complexity. It might be ok as an API, e.g. Promise.adapt(someAsync), but the downside there is that it can't be very generic (an argument could be made for the very common Node callback pattern where the last argument to the function is the callback and the params are always err, result).

There are also existing NPM libraries to "promisify" module exports.

# Tab Atkins Jr. (8 years ago)

On Mon, Feb 27, 2017 at 12:41 AM, Isiah Meadows <isiahmeadows at gmail.com> wrote:

May I add one more thing: the main topic this was about is adapting non-standard async APIs (like Node's error-first callback idiom) to the land of promises. Async functions and iterators are incredibly useful when you're dealing with just promises, especially consuming them, but this is about creating promise adapters, not consuming promises.

You don't need to change the behavior of core syntax to make Node-style error-first callbacks work. That's easily done by libraries, which have existed in Node-land for quite a while, and can automatically convert functions that take Node-style callbacks into functions that return promises.

# Isiah Meadows (8 years ago)

I was speaking objectively about the proposal itself, and the scope of it. I'm personally strongly against it for reasons I stated earlier in the thread (the status quo is better IMHO). I was just trying to direct people back to the actual scope of the proposal instead of basically reinventing async functions using async functions, and also simultaneously attempting to assist the OP in better understanding what he's really trying to propose (which he didn't appear to grasp well).

# Michał Wadas (8 years ago)

Actually, this proposal would be a revolution and I can think of too many edge cases to make it viable.

Consider:

async function foo() {

async function bar() {

    [1,2,3].forEach(async function() {                 async return

3; });

} return (await bar()) + 39;

}

What does happen here? For me it's absolutely counterintuitive. What about functions coming from other scopes, like?

async function foo() {

function bar() {

    [1,2,3].forEach(async function() {                 async return

3; });

} return (await bar()) + 39;

}

Or multiple returns?

async function foo() {

setTimeout(()=>{ async return 42; }, 0); return null;

}

Promisification should be done by userland libraries, not by introducing new syntax.

# Isiah Meadows (8 years ago)

I'm guessing you missed the second sentence in my reply about the fact I'm strongly against the original proposal...