Proposal: `await.all {...}` for parallelism

# Jacob Bloom (a month ago)

To simplify the problem of working with promises in parallel, I propose this new syntax:

async function initialize() {
  let foo, bar, baz;
  await.all {
    foo = (await request('foo.json')).data;
    bar = (await request('bar.json')).data;
    baz = (await request('baz.json')).data;
  }
  render(foo, bar, baz);
}

Each child statement of the curly braces is evaluated in parallel and execution resumes when they've all resolved.

The Problem: with current syntax, the above function would probably look something like this:

async function initialize() {
  const [
    { data: foo }, // renaming response.data => foo
    { data: bar },
    { data: baz },
  ] = await Promise.all([
    request('foo.json'),
    request('bar.json'),
    request('baz.json'),
  ]);
  render(foo, bar, baz);
}

For this kind of use case, Promise.all leads to "parallel lists" of promises and their return values, which must be kept in sync. Using those values either requires (sometimes deep) destructuring or temporary variables.

This structure is also just fundamentally different from working serially in async/await and it forces you to reason about the problem in a specific way. This doesn't appear to be a conscious decision to force good code practices, it's just a limitation that falls naturally out of the current syntax. Thus, we have an opportunity to shift some of the burden back to the language with this new syntax.

Here's the full proposal: mrjacobbloom/proposal-await-all -- let me know what you think!

# Jordan Harband (a month ago)

If we have await.all, what about await.race, await.allSettled, await.any?

# Guy Bedford (a month ago)

Typically I find I want to loop over an iterator of items and apply a function body of work on them in parallel.

So it would be nice to support full blocks of statements that can do this work in parallel, instead of relying on just expressions or functions to achieve this.

Extending for loops to have a parallel form is another option (I seem to recall something similar brought up before here):

for await.all (const entry of entries) {
  await doWork(entry);
}

Whether async iteration should be supported or treated as sync or parallel is another question though, and possibly a confusion of the form.

# Jacob Bloom (a month ago)

Regarding the other Promise methods, this syntax could certainly extend to all of them (that's part of the reason I chose this syntax, to leave that option open).

await.any and await.race would work analogously to await.all, but since we're no longer dealing with return values there's no good way to get the value that won the race. This is fine as long as you're not dependent on that value and only care about executing some code as soon as the race is won. The best way I can think of to get the winning value is something like this:

async function myFunc() {
  let value1, value2;
  await.any {
    value1 = await foo;
    value2 = await bar;
  }
  const firstResult = value1 || value2; // could also use ??
}

...which isn't any easier than using Promise.any.

await.allSettled would, in many cases, be the same as Promise.all except it would also swallow errors. I'd have to think more about its use cases but it could be implemented the same way.

# Jacob Bloom (a month ago)

Regarding parallel for-loops: I'd consider that a separate problem. You'd want parallel for-loops for things like requests to the same API, where each promise is handled the same way. await.all is more for handling disparate tasks in parallel without having to resort to thinking in terms of arrays and iteration.

# Cyril Auburtin (25 days ago)

There's for await loops since recently, there could be await for loops for wrapping the whole execution of a loop

console.time(1);
await for (const fn of [()=>delay(50).then(()=>'a'),
()=>delay(80).then(()=>'b')]) {
  console.timeLog(1, await fn());
}
console.timeEnd(1);

This would log the same than:

console.time(1);
await Promise.all(
  [()=>delay(50).then(()=>'a'), ()=>delay(80).then(()=>'b')]
    .map(async fn => { console.timeLog(1, await fn()) })
);
console.timeEnd(1);
/*
1: 51.066162109375ms a
1: 80.998291015625ms b
1: 81.315185546875ms
*/

without await for, things are serial:

console.time(1);
for (const fn of [()=>delay(50).then(()=>'a'),
()=>delay(80).then(()=>'b')]) {
  console.timeLog(1, await fn());
}
console.timeEnd(1);
/*
1: 50.68212890625ms a
1: 130.9951171875ms b
1: 131.1162109375ms
*/

(var delay = t => new Promise(r => setTimeout(r, t));)

# Naveen Chawla (25 days ago)

I don't like the idea of await behaving differently inside vs outside of the "await.all" block, and I think is a source of bugs:

await.all { const x = await doSomethingAsync(); //x is still undefined here! Not the case outside of an await.all block }

Maybe if you drop the "await" in your example:

await.all { const x = doSomethingAsync(); //x is just the promise here, but at least is the same whether inside or outside of the await.all block }

...but that still waits for the async functions to complete, I think it would cause fewer bugs and would seem to still satisfy the motivation?

# Bergi (25 days ago)

Hello!

This [current] structure is also just fundamentally different from working serially in async/await and it forces you to reason about the problem in a specific way. This doesn't appear to be a conscious decision to force good code practices

Actually I'd argue that it is. Doing stuff concurrently is fundamentally different from doing it serially, and should be reasoned about every time you use it.

kind , Bergi

# Jacob Bloom (25 days ago)

Maybe if you drop the "await" in your example:

await.all {
   const x = doSomethingAsync();
   //x is just the promise here
}

...but that still waits for the async functions to complete, I think it would cause fewer bugs and would seem to still satisfy the motivation?

It doesn't seem like the await.all block is doing anything in that case. That code seems equivalent to this:

const x = doSomethingAsync();
myFunction(await x)
await.all {
 const x = await doSomethingAsync();
 //x is still undefined here!
}

You bring up a good point about scoping and race conditions. It's a little tricky since the curly braces create a block scope but none of the parallel statements should be allowed to access each-other's variables, it's almost like each statement should have its own scope. Maybe it'd be better to have a syntax that ensures a set of curly braces for each parallel task? Async do-expressions could be a good solution (assuming they'd work kind of like an async IIFE):

async function initialize() {
  let foo, bar, baz;
  await Promise.all([
    async do { foo = (await request('foo.json')).data },
    async do { bar = (await request('bar.json')).data },
    async do { baz = (await request('baz.json')).data },
  ]);
  render(foo, bar, baz);
}

(this is also a less drastic syntax change that piggybacks on an existing proposal)

# Jacob Bloom (25 days ago)

...strike that, I misread the "but that still waits for the async functions to complete" part. So what you're proposing is that everything functions normally inside the curly braces, but execution doesn't continue until all promises have resolved? So your example would work essentially like this:

const x = doSomethingAsync();
const y = doSomethingElseAsync();
await x, await y;
// all promises are resolved by now, but
// still need to use await to unbox the values
someFunction(await x, await y);
# Isiah Meadows (25 days ago)

Just FYI, I previously suggested a couple things substantially more flexible than this 1, 2 (originated from this 3), and it mostly fell flat due to being highly premature. Anything exclusive to promises is unlikely to win as library methods exist for basically all use cases and from my experience, committee members are in general very hesitant to add syntax for anything that doesn't pay for itself well. Similar questions have come up a few times in the past, too, and I've commented on two of them. 4, 5

If anything, I don't feel we know the problem space well enough, and the language lacks the primitives needed to really dig into it. (This is why I came up with my generator forking strawman. 6)


Isiah Meadows contact at isiahmeadows.com, www.isiahmeadows.com

# manuelbarzi (24 days ago)

why not just await as already is, but supporting an iterable / array of promises, as Promise.all already does, automatically discerning single promises vs multiple ones:

const promises = [...]

// in parallel (`await` automatically acts as `Promise.all` here)
const results = await promises

results.forEach(result => ...)
# Jacob Bloom (24 days ago)

Just FYI, I previously suggested a couple things substantially more flexible than this

Ah, thank you for bringing those proposals to my attention. I looked through the archives for relevant discussions but I must've missed them.

It seems like we converged on a similar syntax for what you called "merging," and the idea that there ought to be a separate syntax for iteration. I don't know whether that means that this is the right solution or just the most obvious one, but either way it's encouraging to know that other people have the same difficulties with the current syntax and are thinking about the problem.

from my experience, committee members are in general very hesitant to add syntax for anything that doesn't pay for itself well

Yeah, I figured the bar would be high for new syntax. I've run into the awkwardness of dealing with distinct parallel tasks several times, and a few of the people I discussed it with were in the same boat, so I wrote up this proposal thinking it might have a wide appeal. The proposed syntax desugars via a relatively simple transformation but encourages developers to reason about the problem in a completely different way that I'd argue is more intuitive. Whether the committee agrees and thinks it justifies a new syntax remains to be seen, but either way I'm excited to see where this discussion goes (whether it leads to the proposed syntax, to some other syntax, or to somewhere else entirely).

As a side note: thank you to everyone for the thoughtful questions and responses, I had no idea what to expect from this thread and it's gone better than I could've hoped for. Thank you for not immediately shooting down a proposal that looks similar to other proposals before it.

# Jacob Bloom (24 days ago)

why not just await as already is, but supporting an iterable / array of promises, as Promise.all already does

await can already accept a non-promise, so I believe that'd be breaking syntax if Array.prototype.then is set. It also requires collecting the promises in an array, which is what the proposed syntax is trying to avoid.

# manuelbarzi (24 days ago)

AFAIK await can only accept an expression as a Promise, not other thing: developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await

# Naveen Chawla (24 days ago)

I have a solution for that:

const promises = [...] await.all promises //returns an array of results await.race promises //returns a single result

etc.

# Matthew Morgan (24 days ago)

Why not just use a combination of async/await and .then?

async function initialize() {
    const [foo, bar, baz] = await Promise.all([
        request('foo.json').then(t => t.data),
        request('bar.json').then(t => t.data),
        request('baz.json').then(t => t.data),
    ]);
    render(foo, bar, baz);
}
# Jacob Bloom (24 days ago)
const promises = [...]
await.all promises //returns an array of results
await.race promises //returns a single result

One of the goals of this proposal is to simplify the process of collecting the promises in an array and then having to get them out of another array.

Why not just use a combination of async/await and .then?

That's an okay solution, though we still end up having to maintain "parallel lists" of promises and values, which I'm hoping to avoid. But you can sidestep them with something like this, which isn't too bad:

async function initialize() {
    let foo, bar, baz;
    await Promise.all([
        request('foo.json').then(t => foo = t.data),
        request('bar.json').then(t => bar = t.data),
        request('baz.json').then(t => baz = t.data),
    ]);
    render(foo, bar, baz);
}
# manuelbarzi (24 days ago)

I have a solution for that:

const promises = [...] await.all promises //returns an array of results await.race promises //returns a single result

well, my proposal is exactly that, but doing await.all by default with just await.

# Naveen Chawla (24 days ago)

Yes of course, I was responding to your proposal and the subsequent email about it being incompatible with existing JavaScript because "await" on its own accepts non-promises, so wouldn't return an array of results from an array of promises, hence why I proposed await.all etc.

# Jacob Bloom (24 days ago)

This [current] structure is also just fundamentally different from working serially in async/await and it forces you to reason about the problem in a specific way. This doesn't appear to be a conscious decision to force good code practices

Actually I'd argue that it is. Doing stuff concurrently is fundamentally different from doing it serially, and should be reasoned about every time you use it.

I agree that parallelism is different and should be handled with care, but I don't think it follows that the best way to reason about parallelism is the way that Promise.all encourages. Making something more complicated doesn't necessarily mean you'll do a better job of reasoning about it.

If you think the proposed syntax encourages poorly-reasoned-about code, I'm open to iterating on it to find a syntax that works with the developer to handle parallelism in a safe way, and also doesn't require them to write too much boilerplate code.

# Naveen Chawla (24 days ago)

I do find the pattern of promise "all" combined with destructuring the easiest way to handle parallelism. I think it's the only "deterministic" parallel pattern code wise.

I think non determinism in code increases the probability of bugs.

# Tom Boutell (23 days ago)

I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) { await doTheThing(item); }

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) within the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.

# Naveen Chawla (22 days ago)

However, if await.all { ... } were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}

Notice I have omitted await inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by await.all. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to Promise.all, which I think is a little more difficult to conceptualize than the await.all { ... } way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

# Tom Boutell (22 days ago)

This is very interesting, but this code:

await.all { x = getXAsync(); y = getYAsync(); }

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general. If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.

# Naveen Chawla (21 days ago)

It does not change the meaning of the ";" at all. As you may already know, omitting await already invokes multiple async function calls in parallel, in current JavaScript, so absolutely no change in that respect.

The only thing this await.all suggestion does, is ensure that all non-awaited async function calls are completed before proceeding beyond the end of the block.

i.e. it adds fairly straightforward and terse deterministic control to otherwise non-deterministic code, without requiring knowledge of destructuring or Promise.all.

# Tom Boutell (20 days ago)

Hey, you're absolutely right! It's OK because it just means things are more deterministic before the block exits. It doesn't impact any reasonable expectations during the block.

I am convinced that your syntax is useful and does not introduce any new confusion.

I wonder, then, if it is also possible to implement concurrency limits for this properly?

await.all({ concurrency: 5 }) { for (const item of items) { // returns promise item.process(); } }

This is more challenging because, in our transpilation, we can't just bottle up all the promises and call Promise.all at the end. It would be too late to manage how many are in process at once, bashing on various API limits (:

# Tom Boutell (20 days ago)

There is however a performance concern with your code that we should talk about.

If I write this:

await.all { returnsAPromise(); for (let i = 0; (i < 100000); i++) { doesSimpleFastSynchronousMath(); } returnsAnotherPromise(); }

Then Babel will have no choice but to compile this to:

{ const promises = []; { const maybeThenable = returnsAPromise(); if (maybeThenable && maybeThenable.then) { promises.push(maybeThenable); } } promises.push(returnsAPromise()); for (let i = 0; (i < 100000); i++) { const maybeThenable = doesSimpleFastSynchronousMath(); if (maybeThenable && maybeThenable.then) { promises.push(maybeThenable); } } const maybeThenable = returnsAnotherPromise(); if (maybeThenable && maybeThenable.then) { promises.push(maybeThenable); } } await Promise.all(promises);

Which could have a significant performance impact on that synchronous inner loop.

# manuelbarzi (20 days ago)

why not making it work with the addition of a new keyword suffix for parallel awaits (for example, ||), grouping the mid blocks that require parallelism. playing with your example and going a bit further:

async {
  const v0 = await|| returnsAPromise(); // to be grouped in parallel
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  const v1 = await|| returnsAnotherPromise(); // to be grouped in parallel
  async {
      await returnsAnotherPromise1();
      const v2 = await|| returnsAnotherPromise2(); // to be grouped in
parallel
      const v3 = await|| returnsAnotherPromise3(); // to be grouped in
parallel
      await returnsAnotherPromise4(v2, v3);
      const v4 = await returnsAnotherPromise5();
  }
  await returnsAnotherPromiseX(v0, v1);
}
# manuelbarzi (20 days ago)

another example, that may "normalize" it a bit more:

async { // ... = returns a promise
    const x1 = await|| ...
    const x2 = await ... (x1)
    const x3 = await|| ...
    const x10 = await ... (x2, x3)

    let x4, x5, x6

    async {
        x4 = await|| ... (x1, x2)
        x5 = await|| ... (x2, x3)
        x6 = await ... (x4, x5, x10)
    }

    let x7, x8, x9

    async {
        x7 = await|| ... (x4, x6)
        x8 = await ... (x6, x7)
        x9 = await|| ... (x5, x6)
    }

    await ... (x8, x9)
}
# Naveen Chawla (20 days ago)

Would you mind clarifying the performance hit you are referring to? I'm seeing that the "synchronous" calls wouldn't be added to the array you used in your example, so it's not clear to me to which performance hit you are referring.

Hi Manuel! Would you mind explaining the added value of your proposal? I am only seeing it being more verbose, but I've not found any added functionality or benefit from it

# manuelbarzi (20 days ago)

Hi Manuel! Would you mind explaining the added value of your proposal? I am only seeing it being more verbose, but I've not found any added functionality or benefit from it

the proposal combines parallel and series async / await and could resolve complex trees, like the following example:

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await|| p1()
    const x2 = await p2(x1)
    const x3 = await|| p3()
    const x10 = await p10(x2, x3)

    let x4, x5, x6

    async {
        x4 = await|| p4(x1, x2)
        x5 = await|| p5(x2, x3)
        x6 = await p6(x4, x5, x10)
    }

    let x7, x8, x9

    async {
        x7 = await|| p7(x4, x6)
        x8 = await p8(x6, x7)
        x9 = await|| p9(x5, x6)
    }

    await p11(x8, x9)
}

// it would resolve a tree of parallel and series like following with
traditional promises

Promise.resolve()
    .then(() => Promise.all([p1, p3]))
    .then((x1, x3) =>
        p2(x1)
            .then(x2 =>
                p10(x2, x3)
                     .then(x10 => {
                        let x4, x5, x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1, x2), p5(x2, x3
)]))
                            .then(results => [x4, x5] = results)
                            .then(() => p6(x4, x5, x10))
                            .then(x6 => {
                                let x7, x8, x9

                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4, x6), p9(

x5, x6)]))
                                    .then(results => [x7, x9] = results)
                                    .then(() => p8(x6, x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8, x9))
                            })
                     })
            )
    )
# manuelbarzi (20 days ago)

a few little corrections...

// NOTE async {} = asynchronous scope (it groups parallel awaits => await||) // NOTE p? = function call that returns a promise (? = just and index)

async { const x1 = await || p1() const x2 = await p2(x1) const x3 = await || p3() const x10 = await p10(x2, x3)

let x4, x5, x6

async {
    x4 = await || p4(x1, x2)
    x5 = await || p5(x2, x3)
    x6 = await p6(x4, x5, x10)
}

let x7, x8, x9

async {
    x7 = await || p7(x4, x6)
    x8 = await p8(x6, x7)
    x9 = await || p9(x5, x6)
}

await p11(x8, x9)

}

Promise.resolve() .then(() => Promise.all([p1(), p3()])) .then((x1, x3) => p2(x1) .then(x2 => p10(x2, x3) .then(x10 => { let x4, x5, x6

                    return Promise.resolve()
                        .then(() => Promise.all([p4(x1, x2), p5(x2, x3

)])) .then(results => [x4, x5] = results) .then(() => p6(x4, x5, x10)) .then(_x6 => x6 = _x6) .then(() => { let x7, x8, x9

                            return Promise.resolve()
                                .then(() => Promise.all([p7(x4, x6), p9(

x5, x6)])) .then(results => [x7, x9] = results) .then(() => p8(x6, x7)) .then(_x8 => x8 = _x8) .then(() => p11(x8, x9)) }) }) ) )

# manuelbarzi (20 days ago)

yet smaller, as Promise.resolve() was totally redundant here...

// NOTE async {} = asynchronous scope (it groups parallel awaits => await||) // NOTE p? = function call that returns a promise (? = just and index)

async { const x1 = await || p1() const x2 = await p2(x1) const x3 = await || p3() const x10 = await p10(x2, x3)

let x4, x5, x6

async {
    x4 = await || p4(x1, x2)
    x5 = await || p5(x2, x3)
    x6 = await p6(x4, x5, x10)
}

let x7, x8, x9

async {
    x7 = await || p7(x4, x6)
    x8 = await p8(x6, x7)
    x9 = await || p9(x5, x6)
}

await p11(x8, x9)

}

Promise.all([p1(), p3()]) .then((x1, x3) => p2(x1) .then(x2 => p10(x2, x3) .then(x10 => { let x4, x5, x6

                    return Promise.all([p4(x1, x2), p5(x2, x3)])
                        .then(results => [x4, x5] = results)
                        .then(() => p6(x4, x5, x10))
                        .then(_x6 => x6 = _x6)
                        .then(() => {
                            let x7, x8, x9

                            return Promise.all([p7(x4, x6), p9(x5, x6)])
                                .then(results => [x7, x9] = results)
                                .then(() => p8(x6, x7))
                                .then(_x8 => x8 = _x8)
                                .then(() => p11(x8, x9))
                        })
                })
        )
)
# Tom Boutell (20 days ago)

I don't think I like "await ||" as a syntax, because it doesn't have anything to do with the "OR" operator.

It does avoid adding a bunch of potentially optimization-breaking if statements to the transpiled output for synchronous code like in my example, though, because you only get the behavior for the promises you actually choose to collect with it.

Manuel, I am not sure I understand your examples. You are consuming the values x1 and x2 in p4 right in the middle of the same async block that contains the "await ||" statements that produce them. They are not guaranteed to resolve until after that block is over, no?

# manuelbarzi (20 days ago)

I don't think I like "await ||" as a syntax, because it doesn't have anything to do with the "OR" operator.

it is just a quick proposal to symbolise parallelism easily (||), but just that, of course it could be any other more convenient symbol

Manuel, I am not sure I understand your examples. You are consuming the values x1 and x2 in p4 right in the middle of the same async block that contains the "await ||" statements that produce them. They are not guaranteed to resolve until after that block is over, no?

the async {} block would group only the parallel awaits (await||) into a Promise.all and would chain it in order with the other series awaits inside (normal await).

# manuelbarzi (19 days ago)

So the parallel awaits would execute before the series awaits?

not exactly. it would group the parallel awaits at the position the first parallel await|| is sentenced.

following the demo i sent:

// NOTE async {} = asynchronous scope (it groups parallel awaits => await||) // NOTE p? = function call that returns a promise (? = just and index)

async { const x1 = await || p1() const x2 = await p2(x1) const x3 = await || p3() const x10 = await p10(x2, x3)

let x4, x5, x6

async {
    x4 = await || p4(x1, x2)
    x5 = await || p5(x2, x3)
    x6 = await p6(x4, x5, x10)
}

let x7, x8, x9

async {
    x7 = await || p7(x4, x6)
    x8 = await p8(x6, x7)
    x9 = await || p9(x5, x6)
}

await p11(x8, x9)

}

in the outer block, the first sentence is already parallel, ok, then all parallel awaits in this block will group at first position, siblings of this first one (that is p1 and p3).

in the inner blocks too (just a casualty i created the demos this way, but could be different and still work). in the first inner block, p4 and p5, and in the second inner block, p7 and p9.

"transpiling" it to promises, it would resolve a complex scenario like following (the demo could be improved, of course):

// it would resolve a tree of parallel and series like following with traditional promises

Promise.all([p1(), p3()]) .then((x1, x3) => p2(x1) .then(x2 => p10(x2, x3) .then(x10 => { let x4, x5, x6

                    return Promise.all([p4(x1, x2), p5(x2, x3)])
                        .then(results => [x4, x5] = results)
                        .then(() => p6(x4, x5, x10))
                        .then(_x6 => x6 = _x6)
                        .then(() => {
                            let x7, x8, x9

                            return Promise.all([p7(x4, x6), p9(x5, x6)])
                                .then(results => [x7, x9] = results)
                                .then(() => p8(x6, x7))
                                .then(_x8 => x8 = _x8)
                                .then(() => p11(x8, x9))
                        })
                })
        )
)

What if there are series awaits before the first parallel await?

no problem, if there was one before, then in it would be chained first in the outer promise, and following it would be the promise.all for the next parallel group.

What if the parallel awaits depend on the results of those series awaits? It reads very strangely.

that's totally valid as far as the parallel await is located after the series awaits.

# manuelbarzi (19 days ago)

by the way, i think it can also be improved without the need of async {} blocks, just marking together the parallel awaits (await||) and write the code like in series, but grouping the parallel awaits when transpiling:

// NOTE async {} = asynchronous scope (it groups parallel awaits => await||) // NOTE p? = function call that returns a promise (? = just and index)

const x0 = await p0()

const x11 = f11() // sync code in-the-middle

const x1 = await || p1(x0) const x3 = await || p3(x11) const x2 = await p2(x1) const x10 = await p10(x2, x3)

const x4 = await || p4(x1, x2) const x5 = await || p5(x2, x3) const x6 = await p6(x4, x5, x10)

const x7 = await || p7(x4, x6) const x9 = await || p9(x5, x6) const x8 = await p8(x6, x7)

await p11(x8, x9)

// it would resolve a tree of parallel and series like following with traditional promises

p0 .then(x0 => { const x11 = f11()

    return Promise.all([p1(x0), p3(x11)])
        .then((x1, x3) =>
            p2(x1)
                .then(x2 =>
                    p10(x2, x3)
                        .then(x10 =>
                            Promise.all([p4(x1, x2), p5(x2, x3)])
                                .then((x4, x5) =>
                                    p6(x4, x5, x10)
                                        .then(x6 => Promise.all([p7(x4,

x6), p9(x5, x6)]) .then((x7, x9) => p8(x6, x7) .then(x8 => p11(x8, x9)) ) ) ) ) ) ) })

# Naveen Chawla (19 days ago)

I think your idea is too much magic, in the sense of not being straightforward to understand. It actually executes subsequent code first, e.g.

async { x1 = await || doStuffAsync1(); x2 = await doStuffAsync2(); x3 = await || doStuffAsync3(); }

In your idea, doStuffAsync3() is executed and completed before doStuffAsync2() even begins! I think this would add bugs due to the confusion, more particularly if outside variables were modified and relied upon.

Why not just maximally preserve current JavaScript for parallel execution, just by omitting await in multiple async calls, simply wrapping it in an await.all block to ensure completion before code continues past the block. This surely is the more straightforward way to satisfy the same goals?

# manuelbarzi (19 days ago)

Why not just maximally preserve current JavaScript for parallel execution, just by omitting await in multiple async calls, simply wrapping it in an await.all block to ensure completion before code continues past the block. This surely is the more straightforward way to satisfy the same goals?

because wrapping it an await.all on the one hand explicitly groups promises, but brings complexity on returning values assignment, specially when is about constants (const). so, if you avoid blocks, just marking parallel awaits with, for example, a suffix await||, or whatever other more convenient way, you can just write the code in series as normally, and avoid that complexity. the transpiler would just require to group the consecutive marked parallel awaits (await||) into a Promise.all() and that's it. following i reproduce the demo before:

// NOTE p? = function call that returns a promise (? = just and index)
// NOTE s? = function call that runs synchronously and returns a value
(? = just and index)

const x0 = await p0()

const x11 = s11() // sync code in-the-middle

const x1 = await || p1(x0)
const x3 = await || p3(x11)
const x2 = await p2(x1)
const x10 = await p10(x2, x3)

const x12 = s12() // sync code in-the-middle

const x4 = await || p4(x1, x2)
const x5 = await || p5(x2, x3, x12)
const x6 = await p6(x4, x5, x10)

const x7 = await || p7(x4, x6)
const x9 = await || p9(x5, x6)
const x8 = await p8(x6, x7)

await p11(x8, x9)

// it would resolve a tree of parallel and series like following with
traditional promises

p0
    .then(x0 => {
        const x11 = f11()

        return Promise.all([p1(x0), p3(x11)])
            .then((x1, x3) =>
                p2(x1)
                    .then(x2 =>
                        p10(x2, x3)
                            .then(x10 => {
                                const x12 = s12()

                                return Promise.all([p4(x1, x2), p5(x2, x3,
x12)])
                                    .then((x4, x5) =>
                                        p6(x4, x5, x10)
                                            .then(x6 => Promise.all([p7(x4,

x6), p9(x5, x6)])
                                                .then((x7, x9) => p8(x6, x7)
                                                    .then(x8 => p11(x8, x9))
                                                )
                                            )
                                    )
                            })
                    )
            )
    })
# Naveen Chawla (19 days ago)

OK I'm even more confused now. x1 is surely not a resolved value until all the next "non parallel await" so is it "undefined" until then?

Could you give an example of what you mean by the await.all { ... } block syntax bringing "complexity on returning values assignment, specially when" "about constants (const)", as I'm unclear what you are referring to.

# manuelbarzi (19 days ago)

OK I'm even more confused now. x1 is surely not a resolved value until all the next "non parallel await" so is it "undefined" until then?

as a const x1 does not exist until those parallel awaits await|| (for p1 and p3) are resolved (same for x3). then p2 is resolved after that.

what it tries to bring is a simplification of syntax.

Could you give an example of what you mean by the await.all { ... } block

syntax bringing "complexity on returning values assignment, specially when" "about constants (const)", as I'm unclear what you are referring to

const and let are block-scoped, so this is referring to avoid that block notation, and then keep coherent with the possibility to directly infer in current scope references without extra segments / blocks.

# manuelbarzi (19 days ago)

however, i find yet things to solve with it, like how to face two consecutive and depending parallel blocks (no series awaits neither sync statements in the middle). for that, the initial proposal using arrays would fit, but i understand is what you trying to avoid.

const [x1, x2] = await|| [p1(), p2()]

const [x3, x4] = await|| [p3(x1), p4(x2)]

... do whatever with x3 and x4

# Naveen Chawla (19 days ago)

If I have, as per your examples,

x1 = await||actionAsync1() x2 = await||actionAsync2(x1) //x1 is undefined here, only resolved on the next "non-parallel-await"

vs adding a line between the two calls:

x1 = await||actionAsync1() let c; x2 = await||actionAsync2(x1)

...does the let c automatically break the parallel grouping since it's a non-parallel operation (thereby making x1 defined)?

It seems to me like you are doing block logic without blocks, which I think increases the chances of bugs. Also you're not leveraging the existing parallel execution pattern for async functions (which is just to omit the await), so it would seem you would be increasing learning overhead. And, you're not really allowing for submitting more sophisticated mixtures of serial async, parallel async and synchronous code for "parallel completion" guarantee, by requiring that parallel calls be "grouped" together in terms of lines of code, almost allowing for nothing beyond the current "Promise.all" pattern, logically. I don't think this is satisfying the original motivation.

For a awail.all { ... } block, maybe allowing a "return"/"yield" value could neaten up the block scope separation, but maybe that could be left to the "do" expression if that were adopted. But I don't think it's a big sacrifice if neither exist.

# Isiah Meadows (19 days ago)

Just wanted to drop in and remind people of this by me earlier in the thread: esdiscuss.org/topic/proposal-await-all-for-parallelism#content-10

The way things are shaping up, it's starting to look like an ad-hoc version of this proposal of mine: isiahmeadows/non-linear-proposal

As I stated earlier, I feel it's premature, especially before we figure out how observables fit into it all.

# Naveen Chawla (18 days ago)

I don't know that an await.all { ... } block would be premature, especially since it's straightforward, so I can't see it clashing with anything in the future e.g. on the "observables" front, if that were to become a thing. If the semantics of await were to be extended somehow, then identically and naturally would the semantics of await.all { ... }.

# Jacob Bloom (3 days ago)

It seems to me like you are doing block logic without blocks, which I think increases the chances of bugs.

I agree. Without curly braces, it's not always clear when the parallel code is guaranteed to have executed by. The first version of my proposal did something similar:

const a = await|| doSomethingAsync();
const b = await|| doSomethingElseAsync();
const [aValue, bValue] = await async.all;

...where async.all represents something like Promise.all(promises). The problem is, if you forget the await async.all then your promises never execute (within the function), so accidentally using await|| instead of await would have the opposite of the intended effect.

async { await|| ... } sidesteps both of these issues: it makes it clear when the promises have all settled by, and if await isn't allowed in the curly brackets then it avoids the "wrong operator" confusion issue as well.

you're not leveraging the existing parallel execution pattern for async functions (which is just to omit the await), so it would seem you would be increasing learning overhead.

If await|| (or whatever) is learned in conjunction with async {} blocks, and is only allowed within them, then it just becomes "this is the syntax for parallelism." And as already stated, you'd need an explicit marker for which expressions are being awaited in parallel for any reasonable transpilation.

const and let are block-scoped, so this is referring to avoid that block notation, and then keep coherent with the possibility to directly infer in current scope references without extra segments / blocks.

Yeah, it'd be nice to have a syntax that doesn't create a block scope. But that strikes me as less important than making it obvious where the nondeterminism is.

I assume making these braces not create a block scope is unacceptable. And using an alternative bracket (maybe async [ await|| foo ]) would be too different from everything else in the language and might read as an array, which discourages using non-expression statements inside it