akka streams with http - downstream error propagation without collapsing the stream
236 views
Skip to first unread message
leslie...@googlemail.com
Jul 13, 2015, 9:50:10 AM7/13/15
to akka...@googlegroups.com
I've tried to implement a straight forward http flow using akka streams 1.0-RC4. It's just about getting some http/json input which is transformed/evaluated, gets some more info via akka-http, transformed evaluated and finally outputs to json http response.
For the optimistic case - where no errors occur - its quite simple and straight forward.
Thinking about the error handling propagation makes it quite complicated to me. Especially when keeping the stream running while don't swallow errors silently. They should appear in the final Sink stage wrapped into a Try or Either object. To make it worse nearly every stage could run into some kind of error/exception case.
I thought about some options which seam not really promising to me like:
1. For each stage just propagate a Try or Either object to the output, like HttpExt.superPool flow does.
-would need to pass data through all stages of the stream even if the first stage failed already.
-each flow stage would need a Try/Either input than in order to handle the upstream success/error case.
-at least HttpExt.superPool flow stage would not accept Try/Either input so the flow needs to be forked here
-it seems quite complicated to me to make every flow stage a graph with two outputs (one for success flowing into the next stage and one for error flowing into the final stage)
2. Exception handling:
-just throwing exceptions collapses the stream which is definitely not what I want
- (misuse?) the supervision decider for error handling, but I have no access to the final Sink stage here
-when using decider exception handler, I would need to add it either to each stage separately or globally to the materializer (can I have on per stream or is it to heavy weight?, than this would be ok. but still no access to the Sink)
-when using decider exception handler, I still would need to add exception handling code to all stages due to the request context Promise which I need to pass through all flow stages in order to fulfill the incoming request in the sink. There needs to be some way in order to pass this Promise request context to the decider.
Currently I use the supervision decider solution. I intercept all flow stage operations. In case of exceptions I put the original exception together with the request context Promise into a special exception and let it throw to the decider where I can complete the request context Promise with failure and resume the flow.
It looks somehow not right to me but I run out of ideas so far.
Regards,
Leslie
+------------+
|http/json in+-----------------+
+-----+------+ |
| |
v |
+-----+------------------+ |
|several processing steps+-----+
+-----+------------------+ |
| |
v |
+-----+---+ |
|akka http+--------------------+
+-----+---+ |
| |
v |
+-----+------------------+ |
|several processing steps+-----+
+-----+------------------+ |
| |error / exception
v |propagation?
+-----+---+ |
|akka http+--------------------+
+-----+---+ |
| |
v |
+-----+------------------+ |
|several processing steps+-----+
+-----+------------------+ |
| |
v v
+-----+------------------------+------+
|final http/json out (succes or error)|
+-------------------------------------+
For the optimistic case - where no errors occur - its quite simple and straight forward.
Thinking about the error handling propagation makes it quite complicated to me. Especially when keeping the stream running while don't swallow errors silently. They should appear in the final Sink stage wrapped into a Try or Either object. To make it worse nearly every stage could run into some kind of error/exception case.
I thought about some options which seam not really promising to me like:
1. For each stage just propagate a Try or Either object to the output, like HttpExt.superPool flow does.
-would need to pass data through all stages of the stream even if the first stage failed already.
-each flow stage would need a Try/Either input than in order to handle the upstream success/error case.
-at least HttpExt.superPool flow stage would not accept Try/Either input so the flow needs to be forked here
-it seems quite complicated to me to make every flow stage a graph with two outputs (one for success flowing into the next stage and one for error flowing into the final stage)
2. Exception handling:
-just throwing exceptions collapses the stream which is definitely not what I want
- (misuse?) the supervision decider for error handling, but I have no access to the final Sink stage here
-when using decider exception handler, I would need to add it either to each stage separately or globally to the materializer (can I have on per stream or is it to heavy weight?, than this would be ok. but still no access to the Sink)
-when using decider exception handler, I still would need to add exception handling code to all stages due to the request context Promise which I need to pass through all flow stages in order to fulfill the incoming request in the sink. There needs to be some way in order to pass this Promise request context to the decider.
Currently I use the supervision decider solution. I intercept all flow stage operations. In case of exceptions I put the original exception together with the request context Promise into a special exception and let it throw to the decider where I can complete the request context Promise with failure and resume the flow.
It looks somehow not right to me but I run out of ideas so far.
Regards,
Leslie
+------------+
|http/json in+-----------------+
+-----+------+ |
| |
v |
+-----+------------------+ |
|several processing steps+-----+
+-----+------------------+ |
| |
v |
+-----+---+ |
|akka http+--------------------+
+-----+---+ |
| |
v |
+-----+------------------+ |
|several processing steps+-----+
+-----+------------------+ |
| |error / exception
v |propagation?
+-----+---+ |
|akka http+--------------------+
+-----+---+ |
| |
v |
+-----+------------------+ |
|several processing steps+-----+
+-----+------------------+ |
| |
v v
+-----+------------------------+------+
|final http/json out (succes or error)|
+-------------------------------------+
Matthew
May 30, 2016, 5:03:55 PM5/30/16
to Akka User List
Hi Leslie,
Did you ever find a nice pattern for this? I ask because I'm doing something very similar and am currently unhappy with the error handling capabilities of Akka streams...
Jeff
Jun 21, 2017, 6:28:01 PM6/21/17
to Akka User List
I am also running into a similar situation and have not found a solution that is satisfactory
Reply all
Reply to author
Forward
0 new messages