[HTTP] stream a subpart of another stream

[Guillaume]

Hi folks,

I'm stuck with an HttpRequest query processing. I try to return a stream who contains a subpart of a a REST stream response.  eg:

GET /foo
=>
{
 bar: {},
 ....
 results: [
  {foo: 1},
  {foo: 2},
  {foo: 3},
  {foo: 4},
  {foo: 5},
   ....
  {foo: 1000000}
 ]
}

What I want to stream :
[
  {foo: 1},
  {foo: 2},
  {foo: 3},
  {foo: 4},
  {foo: 5},
   ....
  {foo: 1000000}
 ]

My current code use akka-stream-circe and akka-http-circe and looks like :

val request = HttpRequest(uri =
  Uri(cfg.get[String]("my.endpoint"))
    .withPath(Path("/foo"))
    .withQuery(Query("search" -> "foo"))
)

Http()
  .singleRequest(request)
  .map(response => response.entity.dataBytes.via( CirceStreamSupport.decode[???????????????????????] )) pipeTo sender

Unfortunately, I don't understand  how to configure Jawn with Circe to "ignore" all json content except this array... Maybe Akka HTTP module allow me to do that out of the box ?

Regards,

[Martynas Mickevičius]

Hi,

judging by the fact that the example of streaming large files using circe removes the top level object definition (https://github.com/circe/circe/blob/75061316f210aaf90e12faed5bdda48e105723f4/examples/sf-city-lots/README.md) I think it is not quite feasible to do what you want currently with circe (streaming arbitrary nested json structure).

You will have to take a look at event based json parsers. I have played before with one such marshaller that uses event based parser: ts-reaktive (https://github.com/Tradeshift/ts-reaktive).

Here is an code example that uses that library to parse responses from github: https://github.com/2m/authors/blob/master/core/src/main/scala/GithubProtocol.scala

--
>>>>>>>>>> Read the docs: http://akka.io/docs/
>>>>>>>>>> Check the FAQ: http://doc.akka.io/docs/akka/current/additional/faq.html
>>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user
---
You received this message because you are subscribed to the Google Groups "Akka User List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to akka-user+...@googlegroups.com.
To post to this group, send email to akka...@googlegroups.com.
Visit this group at https://groups.google.com/group/akka-user.
For more options, visit https://groups.google.com/d/optout.