Learning how to use pipes

[bret...@gmail.com]

I'm trying to learn how to use pipes and I have two files with one line each that I'd like to read into a tuple. I'd like to get something with a type signature like Producer (String, String) IO (), where the tuple contains the first lines from the files. So far I have (with the wrong type signature):

buildAuth :: MonadIO m => FilePath -> FilePath -> IO (Producer String m (), Producer String m ())

buildAuth keypath secretpath =

    withFile keypath ReadMode (\keyh ->

        withFile secretpath ReadMode (\secreth ->

            return (getLine keyh, getLine secreth)))

  where

    getLine h = P.fromHandle h >-> P.take 1

It feels like I'm going at it backwards or something. How can I modify this to get the type signature I want? Or if I can't get that signature, what's the proper way to get the tuple?ks!

Thanks!

[Ben Gamari]

Why not just start with the two `Producer String IO ()`s and pass them
to `Pipes.Prelude.zip`?

import Control.Monad.IO.Class
import System.IO
import Pipes
import qualified Pipes.Prelude as PP

buildAuth :: MonadIO m => Handle -> Handle -> Producer (String, String) m ()
buildAuth keyH secretH = PP.zip (PP.fromHandle keyH) (PP.fromHandle secretH)

doThings :: FilePath -> FilePath -> IO ...
doThings keyPath secretPath = do
withFile keyPath ReadMode $ \keyH -> do
withFile secretPath ReadMode $ \secretH -> do
let prod = buildAuth keyH secretH
...

Cheers,

- Ben

[Gabriel Gonzalez]

Yeah, `Pipes.Prelude.zip` is what you want. It has this type:

zip :: Monad m => Producer a m () -> Producer b m () -> Producer
(a, b) m ()

[bret...@gmail.com]

Got it. Thanks guys! I think I was confusing myself because I wasn't separating the IO stuff from the pipes stuff.

[Michael Thompson]

What is best here might of course depend on *where you are getting the filenames from*, and *what you are going to do with the pairs of first lines*.  I was wondering if some of the trouble you felt was the constriction you're under with `System.IO.withfile`. Its complexity is sort of replicated by the type of bgamari's `doThings`:

    doThings :: FilePath -> FilePath -> (Producer (String,String) IO () -> IO ()) -> IO ()

    doThings keyPath secretPath f = do 

        withFile keyPath ReadMode $ \keyH -> do 

        withFile secretPath ReadMode $ \secretH -> do 

        let prod = buildAuth' keyH secretH 

        f prod

If you use the `withFile` from Pipes.Safe, you have a few more options, but are put under a MonadSafe constraint. In fact the `readFile` from `Pipes.Safe.Prelude` has the same simple line-by-line behavior that `Pipes.Prelude.fromHandle` has, so you could as well write

    buildAuthP :: MonadSafe m => FilePath -> FilePath -> Producer (String, String) m ()

    buildAuthP keypath secretpath  = 

      P.zip (S.readFile keypath >-> P.take 1)

            (S.readFile secretpath >-> P.take 1)

in the confident security that files will be properly closed.  If you keep in mind the ways of managing the constraint that `m` must be a MonadSafe, this gives you a little more freedom of operation.  So here I reuse `buildAuthP` repeatedly to produce pairs of strings .  It is just a question of finessing the SafeT machinery appropriately. 

    import Pipes

    import Pipes.Safe 

    import qualified Pipes.Safe.Prelude as S

    import qualified Pipes.Prelude as P

    import System.IO

    import Control.Monad.IO.Class 

    main = runEffect effect1 where 

      effect1 = for filePairs (\(a,b) -> hoist runSafeT (buildAuthP a b) >-> P.print) 

      effect2 = runSafeP $ for filePairs (\(a,b) -> buildAuthP a b >-> P.print)  

      filePairs = each (replicate 2000 ("/usr/share/dict/words", "/usr/share/dict/words"))

    buildAuth :: MonadIO m => Handle -> Handle -> Producer (String, String) m () 

    buildAuth keyH secretH = P.zip (P.fromHandle keyH) (P.fromHandle secretH) 

    doThings :: FilePath -> FilePath -> (Producer (String,String) IO () -> IO ()) -> IO ()

    doThings keyPath secretPath f = do 

        withFile keyPath ReadMode $ \keyH -> do 

        withFile secretPath ReadMode $ \secretH -> do 

        let prod = buildAuth' keyH secretH 

        f prod

    buildAuthP :: MonadSafe m 

               => FilePath -> FilePath -> Producer (String, String) m ()

    buildAuthP keypath secretpath  = 

      P.zip (S.readFile keypath >-> P.take 1)

            (S.readFile secretpath >-> P.take 1)

 

                           

[bret...@gmail.com]

OK this is awesome. I'm actually just importing a config module that has the two paths defined. I'm tinkering around with Riak and I'm going to build a tool that can OAuth into the Twitter API and dump some data into the database. I wanted to be able to put the project on Github eventually without putting my API keys in the repo, hence this roundabout way. Anyway, thanks for the help. It'll get me far.