I working on a Akka Streams project that reads gzipped files from S3 and parses json. The issue I'm running into the stream stalls at about 24523530 bytes and then times-out after a 1 minute (java.util.concurrent.TimeoutException: No elements passed in the last 1 minute), but there is no error otherwise. If I remove the
JsonFraming.objectScanner from the flow, it does not exhibit this behavior. The stream looks like:
val s3FileSource: Source[ByteString, NotUsed] = s3FileNamesSource.via(Flows.downloadObjectsFlow(s3Client, bucket))
val result = s3FileSource
.via(Compression.gunzip(100))
.via(JsonFraming.objectScanner(Int.MaxValue))
.runWith(FileIO.toPath(Paths.get("stream.out"), Set(CREATE, WRITE)))
The json that is does write to the file looks just fine. I'm on Akka 2.4.14. Any ideas on what might be going on or how to troubleshoot? Thanks,
Andrew