Discuss
  • Improved syntax for observable mapping and subscribing

Improved syntax for observable mapping and subscribing

# Bob Myers (7 years ago)

Could someone jog my memory about proposals for better syntax for observable mapping and subscribing, if any?

I'm getting really tired of writing

foo$.pipe(map(bar => mapper(bar)))

I would much prefer to write something along the lines of

stream function fooMapper(foo$) {
  while async (const bar = foo$()) {
    emit mapper(bar);
  }
}

Yes, I'm aware of all the potential issues here and this is just an example, not an actual syntax proposal. I'm just wondering about any prior art.

Bob

# Thomas Grainger (7 years ago)

You can convert an observable into an async iterator. You have to choose between discarding or buffering uniterated items

# Isiah Meadows (7 years ago)

I've already looked into this kind of thing myself privately. I came up with this last year 1, and I more recently came up with this 2 (the second is much better IMHO). Both of those offer solutions to this problem, and they do in fact offer ways for 1. iteration, and 2. mapping/filtering/etc.

Isiah Meadows me at isiahmeadows.com

Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com

# Isiah Meadows (7 years ago)

I probably sent that last message prematurely.

Here's some of the things I learned while making that, among dabbling with other things:

The problem domain of working with collections is hard. Generically modifying them is also hard. It wasn't until we got OO until we learned how to iterate an immediate collection (sets, arrays, maps). With interfaces, synchronous iteration* becomes easy, since you can only iterate a collection all at once, and there's a predetermined order that's easy to configure.

  • Arrays can be easily iterated in either direction, by simply moving a pointer where you want it.
  • Sets and maps are a bit more complex, but they still have all the values immediately ready.
  • Generators are also a little more complex, but this is where the interface makes things easy.

Really, all you need for sync iteration is:

  • A .next() method that returns either "done" (with optional value) or "not done" (with required value).
  • A method to create a collection of that type from a list (if you plan to allow mapping).

Now, with async stuff, it's all nice to support iteration more generally, but there's a few other things that need to be included:

  • A way to initialize lazy values (lazy properties), collections (Lodash wrappers, observables), etc.
  • A way to break iteration.

Both of these are required for sync iterators already (think: .next() + .return(), where .next() also forcibly initializes the collection), but with async collections, it's not always as simple as that. With traditional .next()-based iterators, you can just stop requesting subsequent values (.return() is really just best-effort notification of "not needed" for resource cleanup), but with observables, you have to have an explicit notification mechanism in place, so it knows to stop sending you things and so you know to start ignoring what you receive.

There's another issue, too: sync iterators can't do things between iterations, but async ones could. The obvious result would be to allow the user to specify whether to handle the results in parallel, but not everything supports that (like async coroutines). Also, the user can't always handle results as quickly as the "iterator" produces them. This frequently is the case with observables listening to spammy event emitters, and is why RxJS has .debounce(ms).

Now that we've gotten that out of the way, what now? Well...what about mapping? How do we go from a bunch of As to a bunch of Bs? This is where procedural OO starts to falter:

  • We find ourselves building a set of Bs as we iterate the As, and we return that. That's a bunch of boilerplate.
  • We can't generically create that set of Bs without knowing the type of As we have. Interfaces only type values, not types.

We instead need to look to the functional and type-level stuff, where this once again becomes easy.

  • A .map(f) method, which for each foo in it, calls f with it, and returns a new collection with all the results.

That is magical. It simplifies a lot. We don't have syntax to capture it yet in JS, but it's a natural extension of what we already have for promises. In fact, promise .then isn't far off, except there's a catch: we also need a way to flatten a structure. This is where we need a new method, to go from a collection of As to a collection of Bs while transforming a single A to a collection of Bs.

  • A .chain(f) method, which for each foo in it, calls f with it, and returns a new collection with all the values within the results.

Now, this seems very convenient, and it's very well typed. But this isn't very incredibly convenient: we might only want to flatten it sometimes, and promises are a good example of this (it's why .then implicitly unwraps promises). A common solution in the functional world is to always flatten (giving rise to a .constructor.of), and it simplifies it on paper, but it doesn't simplify it for the user. Instead, we could use two methods to do this, that are a little looser in their constraints:

  • A .lift(f) method, which for each foo in it, calls f with it, and returns a new collection with all the results, optionally flattened.
  • A .chain(f) method, which does the same, but f instead returns either a wrapped instance or an array of 0 or more values to be wrapped.

This allows types to more easily be mapped over, but there's a critical thing this version of .chain enables us to do: it allows us to filter and otherwise manipulate the collection, without being dependent on the caller's type (it doesn't even need to have a .constructor property). Filtering would equate to .chain(x => f(x) ? [x] : []). If you read the proposal, I wrote a basic implementation of the .distinctBy(func) operator commonly found in observable implementations.

This does not address the issue of executing lazily allocated pipelines, but that's required for iteration. If there's nothing to lift over, there's conceptually no need to iterate to fulfill the required interface contract. This necessitates a new operation:

  • An .each(f) method, which initializes the pipeline and calls f with each result, breaks if f returns falsy, and optionally returns a promise resolved when done.

This enables iteration as well as breaking early. This doesn't cover the scenario when you want to break from outside the loop (think: external consumer closed abruptly. .return()/.throw() on iterators), but it does cover the scenario of breaking within the loop.

Or to put it another way, iterables are hard, mapping is harder, and chaining, it's complicated. There's just so many edge cases I could write a book about it (and I'm pretty sure there already has been one written already, just from the common nature of the topic).

Or to put it another way, pipelines are hard. Looping is hard. Language design is hard.

Or to put it another way, I have no clue what the hell I'm doing, and I'm just trying to figure it out as I go along. Diving head-first into fully opaque water is always fun. :-)

On Fri, Mar 23, 2018 at 7:55 PM, Isiah Meadows <isiahmeadows at gmail.com> wrote:

I've already looked into this kind of thing myself privately. I came up with this last year 1, and I more recently came up with this 2 (the second is much better IMHO). Both of those offer solutions to this problem, and they do in fact offer ways for 1. iteration, and 2. mapping/filtering/etc.

Isiah Meadows me at isiahmeadows.com

Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com

On Fri, Mar 23, 2018 at 11:06 AM, Thomas Grainger <tagrain at gmail.com> wrote:

You can convert an observable into an async iterator. You have to choose between discarding or buffering uniterated items

On 23 Mar 2018 14:39, "Bob Myers" <rtm at gol.com> wrote:

Could someone jog my memory about proposals for better syntax for observable mapping and subscribing, if any?

I'm getting really tired of writing

foo$.pipe(map(bar => mapper(bar)))

I would much prefer to write something along the lines of

stream function fooMapper(foo$) {
  while async (const bar = foo$()) {
    emit mapper(bar);
  }
}

Yes, I'm aware of all the potential issues here and this is just an example, not an actual syntax proposal. I'm just wondering about any prior art.

Bob

es-discuss mailing list es-discuss at mozilla.org, mail.mozilla.org/listinfo/es-discuss

es-discuss mailing list es-discuss at mozilla.org, mail.mozilla.org/listinfo/es-discuss

Isiah Meadows me at isiahmeadows.com

Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com