Fwd: Proposed major simplification of the Akka Streams FlowGraph APIs

[Viktor Klang]

Hi everyone,

I'm proposing the following (breaking) changes to the FlowGraph-related APIs, in order to make it consistent, with a smaller surface area and with more obvious demarcation of power-level for "dropping down to"/"tapping into" FlowGraph-mode.

Most, if not all, of the migration can be mechanically applied (I know because I did it in the codebase).

See the PR for details: https://github.com/akka/akka/pull/18700

--

Cheers,

√

[Adam]

Hi,

 

I like most of the described changes.

Here are my personal reservations:

 

1. I prefer a single method name with many overloads over many method names that say what are the arguments.

 

2. I think the usage of from in two places (e.g. BidiFlow.fromXYZ and Builder.from(...), might be confusing.

The latter makes a lot of sense, so the former can be renamed to something else. Being more used to RxJava\RxScala, I kind of like “of” which is very succinct and intuitive.

Of course BidiFlow is just an example, in the list of renames there are lots of fromXYZ methodz. I would replace them all with “of”.

This way when you need to create something, you always know to look for an “of” method, or whatever consistent name you pick (this sort of works in conjunction with my first comment).

 

3. I think loosing addEdge will make it harder for new comers to learn the DSL, which is already not trivial (at least in my eyes as  compare it to RxJava\RxScala).

Usually having less options can make things simpler, but in this case I know I’ve used the addEdge method in order to try and make sense of how to use the Akka streams DSL.

Adam

[Viktor Klang]

Hi Adam,

Thanks for your feedback!
Responses inline

On 18 Oct 2015 09:36, "Adam" <adam...@gmail.com> wrote:
>
> Hi,
>
>  
>
> I like most of the described changes.
>
> Here are my personal reservations:
>
>  
>
> 1. I prefer a single method name with many overloads over many method names that say what are the arguments.

In general I do too, but with erasure and type inference it creates what I've dubbed "Overloading overload" where the compiler (both Java and Scala) will start spitting out nasty error messages with N signatures. Hardly helpful for newcomers, I'd say!

>  
>
> 2. I think the usage of from in two places (e.g. BidiFlow.fromXYZ and Builder.from(...), might be confusing.
>
> The latter makes a lot of sense, so the former can be renamed to something else. Being more used to RxJava\RxScala, I kind of like “of” which is very succinct and intuitive.
>
> Of course BidiFlow is just an example, in the list of renames there are lots of fromXYZ methodz. I would replace them all with “of”.
>
> This way when you need to create something, you always know to look for an “of” method, or whatever consistent name you pick (this sort of works in conjunction with my first comment).

That's an interesting proposal, I'll try it out locally and see how it reads.

>
>  
>
> 3. I think loosing addEdge will make it harder for new comers to learn the DSL, which is already not trivial (at least in my eyes as  compare it to RxJava\RxScala).

The reason for removing it is to not have multiple APIs for the same thing, choice for the sake of choice is something I think creates more harm than value.
As for RxJava, does it even have a graph dsl? link?

>
> Usually having less options can make things simpler, but in this case I know I’ve used the addEdge method in order to try and make sense of how to use the Akka streams DSL.

Besides the anti-bifurcation reason, what makes addEdge "easier" than the DSL?
In general I am not for symbolic dsls but in the Scala DSL case it actually reads very cleanly. My main concern with the symbolic DSL is that it almost *requires* reading docs to even use it, which means also knowing that one has to import the FlowGraph.Implicits._, I think moving the FlowGraph.Implicits to the builder might help increase readability. Or moving the arrows to the builder?

implicit builder => import builder.dsl._

Any other thoughts on improving the accessibility?

>
> Adam
>
> On Saturday, October 17, 2015 at 11:57:25 PM UTC+3, √ wrote:
>>
>> Hi everyone,
>>
>> I'm proposing the following (breaking) changes to the FlowGraph-related APIs, in order to make it consistent, with a smaller surface area and with more obvious demarcation of power-level for "dropping down to"/"tapping into" FlowGraph-mode.
>>
>> Most, if not all, of the migration can be mechanically applied (I know because I did it in the codebase).
>>
>> See the PR for details: https://github.com/akka/akka/pull/18700
>>
>> --
>> Cheers,
>> √
>

> --
> >>>>>>>>>> Read the docs: http://akka.io/docs/
> >>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
> >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
> ---
> You received this message because you are subscribed to the Google Groups "Akka User List" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
> To post to this group, send email to akka...@googlegroups.com.
> Visit this group at http://groups.google.com/group/akka-user.
>
> For more options, visit https://groups.google.com/d/optout.

[אדם חונן‎]

Ah.. I didn't think of that type erasure issue. Makes sense.

RxJava doesn't have a graph DSL as far as I know.

I'm guessing you'd have to use Subjects to implement something similar to graphs there, but so far I've never had the need for that.

I guess what I usually do with RxJava is more similar to only ever using Source from Akka Streams, although I'm pretty sure I'd have to use both Source and Flow.

In my case, we're using RxJava because we're using Couchbase and the java client is based on RxJava.

I guess our use case is much more similar to using Futures - we don't need complex graphs for most of our system.

Methods are simply easier to find inside an IDE than symbols.

You start with "builder.", hit ctrl+space and there it is, with properly named arguments and all.

But thinking of it further, it actually only matters if the code samples in the documentation use it as well.

I mean, my starting point was from a code sample, so if that uses "~>" and I fail to copy all parts (or the sample skips some parts under the assumption that I already know them), it won't matter if addEdge is available or not - my copied code won't work. It might make it clearer if my mistake was to not have "builder =>".

Anyway, none of the above was what was most difficult to me.

I think the difficulties I had are sort of out of scope for this discussion.

I think it's mostly knowing what to use where.

With RxJava you have a huge amount of operators, which is sometimes annoying, but it's a one stop shop, so when you're "stumbling in the dark" (although the documentation also makes a difference), you're more likely to bump into the right thing.

With Akka Streams things are cleaner, but are also found in different places, so you have to look for them in more places.

I guess it's really a design choice. Akka Streams pushes you more towards writing new composable components, which may be nice for the long run, but less intuitive initially.

I'll try to be specific, although I find it hard to remember precisely -

1. I often found myself *thinking* that I probably need to call builder.add(something) and I found myself unsure what that "something" should be.
2. It's easy to miss stuff like that instead of flatMap, you often need to use join. Initially it takes browsing through lots of documentation pages. After a while you get used to where to look for it. I guess I'd say it's not as "search friendly" as Rx (for now).
   

You received this message because you are subscribed to a topic in the Google Groups "Akka User List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/akka-user/XcqoDW3csPk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to akka-user+...@googlegroups.com.

[Viktor Klang]

On Sun, Oct 18, 2015 at 5:57 PM, אדם חונן <adam...@gmail.com> wrote:

Ah.. I didn't think of that type erasure issue. Makes sense.

Yeah, that one is somewhat of a silent killer. Everything works until you suddenly need both Graph[SourceShape] and Graph[FlowShape] and now it's all for nothing…

 

RxJava doesn't have a graph DSL as far as I know.

I'm guessing you'd have to use Subjects to implement something similar to graphs there, but so far I've never had the need for that.

I guess what I usually do with RxJava is more similar to only ever using Source from Akka Streams, although I'm pretty sure I'd have to use both Source and Flow.

That's interesting that you mention because we're currently adding a lot more surface-area to Source/Flow/Sink to try to make that the 80%-solution, so you only have to "drop down" to FlowGraph for the 20% case (more power but also more risk).

 

In my case, we're using RxJava because we're using Couchbase and the java client is based on RxJava.

I guess our use case is much more similar to using Futures - we don't need complex graphs for most of our system.

Makes sense, but once you do—is it a no-go or is there a means of doing it?

 

Methods are simply easier to find inside an IDE than symbols.

You start with "builder.", hit ctrl+space and there it is, with properly named arguments and all.

But thinking of it further, it actually only matters if the code samples in the documentation use it as well.

I mean, my starting point was from a code sample, so if that uses "~>" and I fail to copy all parts (or the sample skips some parts under the assumption that I already know them), it won't matter if addEdge is available or not - my copied code won't work. It might make it clearer if my mistake was to not have "builder =>".

True.

My main beef with the scaladsl for FlowGraph.Builder is that it both requires the builder to be implicit (as in { implicit builder => }) but also requires import FlowGraph.Implicits._, both of which are hard to discover if all you do is to work with the code (and not reading docs).

The question is how this could be improved, one solution would be to make the implicits available from the builder itself, so when you write `builder.` you get `dsl`/`ops` and the scaladoc thereof explains that you import it and need to make the builder reference an implicit and then you can use this awesome DSL to wire your graph.

 

Anyway, none of the above was what was most difficult to me.

I think the difficulties I had are sort of out of scope for this discussion.

I think it's mostly knowing what to use where.

I 100% agree. This is why I am trying to cut down the surface area and different locations of functionality, with my proposal there is *one way* of creating FlowGraphs => FlowGraph.create

 

With RxJava you have a huge amount of operators, which is sometimes annoying, but it's a one stop shop, so when you're "stumbling in the dark" (although the documentation also makes a difference), you're more likely to bump into the right thing.

With Akka Streams things are cleaner, but are also found in different places, so you have to look for them in more places.

I guess it's really a design choice. Akka Streams pushes you more towards writing new composable components, which may be nice for the long run, but less intuitive initially.

Agreed. The question is if there's a magic balance to strike—I am betting on that.

 

I'll try to be specific, although I find it hard to remember precisely -

1. I often found myself *thinking* that I probably need to call builder.add(something) and I found myself unsure what that "something" should be.

Yeah, I raised that topic last week, I think the consensus is to always require it, or never require it, because not knowing when to do it means that it creates a feeling of unease.

 

1. It's easy to miss stuff like that instead of flatMap, you often need to use join. Initially it takes browsing through lots of documentation pages. After a while you get used to where to look for it. I guess I'd say it's not as "search friendly" as Rx (for now).
   

True. Adding it via an implicit is easy enough if one feels that it is missing.

--

Cheers,

√

[Konrad Malawski]

My main beef with the scaladsl for FlowGraph.Builder is that it both requires the builder to be implicit (as in { implicit builder => }) but also requires import FlowGraph.Implicits._, both of which are hard to discover if all you do is to work with the code (and not reading docs).

The question is how this could be improved, one solution would be to make the implicits available from the builder itself, so when you write `builder.` you get `dsl`/`ops` and the scaladoc thereof explains that you import it and need to make the builder reference an implicit and then you can use this awesome DSL to wire your graph.

Yes, I'd want to remove that implicits import a lot.

We show the ~> syntax in most places, so it's basically "the" DSL and really "oh and by the way there's another one".

I'd definitely lobby for and merge such change as part of the cleanups if you had a bit of time to try doing it.

-- 

Cheers,

Konrad 'ktoso’ Malawski
Akka @ Typesafe

[אדם חונן‎]

"Makes sense, but once you do—is it a no-go or is there a means of doing it?"

Well, RxJava implements reactive streams, so obviously it's possible to do the same things.

For example:

For Broadcast, I'd use PublishSubject.

For Balance, I'd create an Observable, that holds multiple subscriptions and push an item into each according to a balancing strategy.

For UnzipWith, again you'd use an Observable of <X,Y,Z,...> and subscriptions of X, Y, Z, ... and for each item push the unzipped items.

etc.

For the fan in operations, you'd do the reverse by having one subscription pointed to by many observables, with the only trick being that for that one subscription, you'd probably use a SerializedSubject (for thread safety).

[Viktor Klang]

If one puts `dsl` or `api` or whatnot as a member of Builder, then it get dot-completion, if then also we annotate Builder with @ImplicitNotFound, then we can guide the user to using `implicit builder =>` iso `builder =>`.

So:

val s = Sink.fromGraph( FlowGraph.create() { implicit b =>

  import b.api._

  … ~> …

  } )

Another option would be to move the arrow-dsl into the scaladsl package object to get it by default, then you'd only have to make the builder implicit (which we can assist with the @ImplicitNotFound). That would hamper discoverability of the api though, as there'd no longer be a dsl/api member of Builder.

 

-- 

Cheers,

Konrad 'ktoso’ Malawski
Akka @ Typesafe

--

Cheers,

√

[Konrad Malawski]

I really like the DSL member approach, +1!

-- 

Cheers,

Konrad 'ktoso’ Malawski
Akka @ Typesafe

[Viktor Klang]

On Mon, Oct 19, 2015 at 12:51 PM, אדם חונן <adam...@gmail.com> wrote:

"Makes sense, but once you do—is it a no-go or is there a means of doing it?"

Well, RxJava implements reactive streams, so obviously it's possible to do the same things.

For example:

For Broadcast, I'd use PublishSubject.

For Balance, I'd create an Observable, that holds multiple subscriptions and push an item into each according to a balancing strategy.

For UnzipWith, again you'd use an Observable of <X,Y,Z,...> and subscriptions of X, Y, Z, ... and for each item push the unzipped items.

etc.

For the fan in operations, you'd do the reverse by having one subscription pointed to by many observables, with the only trick being that for that one subscription, you'd probably use a SerializedSubject (for thread safety).

Right. It would be nice to keep a comparison between RxScala and Akka Streams for doing more "interesting" stream processing.

--

Cheers,

√