Streaming with bytestring chunks delimited by newlines

71 views
Skip to first unread message

Andrew Martin

unread,
Jan 19, 2015, 6:47:39 PM1/19/15
to haskel...@googlegroups.com
I have a log file where the entries are delimited by newlines. So, I want a `Producer` of strict bytestrings, where each bytestring is one line of the file (and the carriage returns should end up omitted). If I have my original producer `p`, then I thought that `PG.concats (PB.view lines p)` (wih PG = Pipes.Group and PB = Pipes.ByteString) would be the stream I wanted, but it does not appear to divide them properly. Could anyone shed any insight on this?

Gabriel Gonzalez

unread,
Jan 19, 2015, 8:49:36 PM1/19/15
to haskel...@googlegroups.com, andrew....@gmail.com
Using `L = Control.Foldl`, you can do this using:

    L.purely PG.folds L.mconcat . view PB.lines :: Monad m => Producer ByteString m r -> Producer ByteString m r


On 01/19/2015 03:47 PM, Andrew Martin wrote:
I have a log file where the entries are delimited by newlines. So, I want a `Producer` of strict bytestrings, where each bytestring is one line of the file (and the carriage returns should end up omitted). If I have my original producer `p`, then I thought that `PG.concats (PB.view lines p)` (wih PG = Pipes.Group and PB = Pipes.ByteString) would be the stream I wanted, but it does not appear to divide them properly. Could anyone shed any insight on this?
--
You received this message because you are subscribed to the Google Groups "Haskell Pipes" group.
To unsubscribe from this group and stop receiving emails from it, send an email to haskell-pipe...@googlegroups.com.
To post to this group, send email to haskel...@googlegroups.com.

Reply all
Reply to author
Forward
0 new messages