Improved syntax for observable mapping and subscribing
You can convert an observable into an async iterator. You have to choose between discarding or buffering uniterated items
I've already looked into this kind of thing myself privately. I came up with this last year 1, and I more recently came up with this 2 (the second is much better IMHO). Both of those offer solutions to this problem, and they do in fact offer ways for 1. iteration, and 2. mapping/filtering/etc.
Isiah Meadows me at isiahmeadows.com
Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com
I probably sent that last message prematurely.
Here's some of the things I learned while making that, among dabbling with other things:
The problem domain of working with collections is hard. Generically modifying them is also hard. It wasn't until we got OO until we learned how to iterate an immediate collection (sets, arrays, maps). With interfaces, synchronous iteration* becomes easy, since you can only iterate a collection all at once, and there's a predetermined order that's easy to configure.
- Arrays can be easily iterated in either direction, by simply moving a pointer where you want it.
- Sets and maps are a bit more complex, but they still have all the values immediately ready.
- Generators are also a little more complex, but this is where the interface makes things easy.
Really, all you need for sync iteration is:
- A
.next()
method that returns either "done" (with optional value) or "not done" (with required value). - A method to create a collection of that type from a list (if you plan to allow mapping).
Now, with async stuff, it's all nice to support iteration more generally, but there's a few other things that need to be included:
- A way to initialize lazy values (lazy properties), collections (Lodash wrappers, observables), etc.
- A way to break iteration.
Both of these are required for sync iterators already (think:
.next()
+ .return()
, where .next()
also forcibly initializes the
collection), but with async collections, it's not always as simple as
that. With traditional .next()
-based iterators, you can just stop
requesting subsequent values (.return()
is really just best-effort
notification of "not needed" for resource cleanup), but with
observables, you have to have an explicit notification mechanism in
place, so it knows to stop sending you things and so you know to start
ignoring what you receive.
There's another issue, too: sync iterators can't do things between
iterations, but async ones could. The obvious result would be to
allow the user to specify whether to handle the results in parallel,
but not everything supports that (like async coroutines). Also, the
user can't always handle results as quickly as the "iterator" produces
them. This frequently is the case with observables listening to spammy
event emitters, and is why RxJS has .debounce(ms)
.
Now that we've gotten that out of the way, what now? Well...what about mapping? How do we go from a bunch of As to a bunch of Bs? This is where procedural OO starts to falter:
- We find ourselves building a set of Bs as we iterate the As, and we return that. That's a bunch of boilerplate.
- We can't generically create that set of Bs without knowing the type of As we have. Interfaces only type values, not types.
We instead need to look to the functional and type-level stuff, where this once again becomes easy.
- A
.map(f)
method, which for eachfoo
in it, callsf
with it, and returns a new collection with all the results.
That is magical. It simplifies a lot. We don't have syntax to capture
it yet in JS, but it's a natural extension of what we already have for
promises. In fact, promise .then
isn't far off, except there's a
catch: we also need a way to flatten a structure. This is where we
need a new method, to go from a collection of As to a collection of Bs
while transforming a single A to a collection of Bs.
- A
.chain(f)
method, which for eachfoo
in it, callsf
with it, and returns a new collection with all the values within the results.
Now, this seems very convenient, and it's very well typed. But this
isn't very incredibly convenient: we might only want to flatten it
sometimes, and promises are a good example of this (it's why .then
implicitly unwraps promises). A common solution in the functional
world is to always flatten (giving rise to a .constructor.of
), and
it simplifies it on paper, but it doesn't simplify it for the user.
Instead, we could use two methods to do this, that are a little looser
in their constraints:
- A
.lift(f)
method, which for eachfoo
in it, callsf
with it, and returns a new collection with all the results, optionally flattened. - A
.chain(f)
method, which does the same, butf
instead returns either a wrapped instance or an array of 0 or more values to be wrapped.
This allows types to more easily be mapped over, but there's a
critical thing this version of .chain
enables us to do: it allows us
to filter and otherwise manipulate the collection, without being
dependent on the caller's type (it doesn't even need to have a
.constructor
property). Filtering would equate to .chain(x => f(x) ? [x] : [])
. If you read the proposal, I wrote a basic implementation
of the .distinctBy(func)
operator commonly found in observable
implementations.
This does not address the issue of executing lazily allocated pipelines, but that's required for iteration. If there's nothing to lift over, there's conceptually no need to iterate to fulfill the required interface contract. This necessitates a new operation:
- An
.each(f)
method, which initializes the pipeline and callsf
with each result, breaks iff
returns falsy, and optionally returns a promise resolved when done.
This enables iteration as well as breaking early. This doesn't cover
the scenario when you want to break from outside the loop (think:
external consumer closed abruptly. .return()
/.throw()
on
iterators), but it does cover the scenario of breaking within the
loop.
Or to put it another way, iterables are hard, mapping is harder, and chaining, it's complicated. There's just so many edge cases I could write a book about it (and I'm pretty sure there already has been one written already, just from the common nature of the topic).
Or to put it another way, pipelines are hard. Looping is hard. Language design is hard.
Or to put it another way, I have no clue what the hell I'm doing, and I'm just trying to figure it out as I go along. Diving head-first into fully opaque water is always fun. :-)
On Fri, Mar 23, 2018 at 7:55 PM, Isiah Meadows <isiahmeadows at gmail.com> wrote:
I've already looked into this kind of thing myself privately. I came up with this last year 1, and I more recently came up with this 2 (the second is much better IMHO). Both of those offer solutions to this problem, and they do in fact offer ways for 1. iteration, and 2. mapping/filtering/etc.
Isiah Meadows me at isiahmeadows.com
Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com
On Fri, Mar 23, 2018 at 11:06 AM, Thomas Grainger <tagrain at gmail.com> wrote:
You can convert an observable into an async iterator. You have to choose between discarding or buffering uniterated items
On 23 Mar 2018 14:39, "Bob Myers" <rtm at gol.com> wrote:
Could someone jog my memory about proposals for better syntax for observable mapping and subscribing, if any?
I'm getting really tired of writing
foo$.pipe(map(bar => mapper(bar)))
I would much prefer to write something along the lines of
stream function fooMapper(foo$) { while async (const bar = foo$()) { emit mapper(bar); } }
Yes, I'm aware of all the potential issues here and this is just an example, not an actual syntax proposal. I'm just wondering about any prior art.
Bob
es-discuss mailing list es-discuss at mozilla.org, mail.mozilla.org/listinfo/es-discuss
es-discuss mailing list es-discuss at mozilla.org, mail.mozilla.org/listinfo/es-discuss
Isiah Meadows me at isiahmeadows.com
Looking for web consulting? Or a new website? Send me an email and we can get started. www.isiahmeadows.com
Could someone jog my memory about proposals for better syntax for observable mapping and subscribing, if any?
I'm getting really tired of writing
I would much prefer to write something along the lines of
Yes, I'm aware of all the potential issues here and this is just an example, not an actual syntax proposal. I'm just wondering about any prior art.
Bob