Re: Problem with Pipes.Text and processing two files

67 views
Skip to first unread message

Лев Никитин

unread,
Feb 5, 2015, 1:06:14 PM2/5/15
to haskel...@googlegroups.com

I'm trying to generate pairs of words from two files. Here is a Main.hs:
 
 
{-# LANGUAGE OverloadedStrings, FlexibleContexts #-}
 
import qualified Data.Text as T
 
import System.IO
 
import Control.Monad.Managed
 
import Pipes
import qualified Pipes.Prelude as P hiding (fromHandle)
import qualified Pipes.Text.IO as P
import qualified Pipes.Text as PT
import qualified Pipes.Group as PG
import Lens.Family (view)
 
inFile ⦂ FilePath → Managed Handle
inFile filePath = managed (withFile filePath ReadMode)
 
badTest ⦂ MonadIO m => Handle → Handle → Producer T.Text m ()
badTest names1 names2 = 
  for (PG.concats ∘ view PT.lines $ P.fromHandle names1) $ λn1 → 
    for (PG.concats ∘ view PT.lines $ P.fromHandle names2) $ λn2 → yield $ n1 `T.append` " ||| " `T.append` n2
 
goodTest ⦂ MonadIO m => Producer T.Text m ()
goodTest = 
  for (each ["A", "B", "C", "D"]) $ λn1 →
    for (each ["1", "2", "3", "4"]) $ λn2 → yield $ n1 `T.append` " ||| " `T.append` n2
 
main ⦂ IO ()
main = runManaged $ do
  names1 ← inFile "names1.txt"
  names2 ← inFile "names2.txt"
  liftIO $ runEffect $ badTest names1 names2 >-> P.map ((`T.append` "\n")) >-> P.stdout
  liftIO $ putStrLn "-------------------------"
  liftIO $ runEffect $ goodTest >-> P.map ((`T.append` "\n")) >-> P.stdout
 
 
where names1.txt containts:
A
B
C
D
 
names2.txt:
1
2
3
 
I've got result:
 
A ||| 1
A ||| 2
A ||| 3
-------------------------
A ||| 1
A ||| 2
A ||| 3
A ||| 4
B ||| 1
B ||| 2
B ||| 3
B ||| 4
C ||| 1
C ||| 2
C ||| 3
C ||| 4
D ||| 1
D ||| 2
D ||| 3
D ||| 4
 
It means that badTest produces only first line of names1.txt file. But similar goodTest does work right. What I am missed?

names1.txt
Main.hs
names2.txt
res.txt
Setup.hs
test-pipes.cabal

Daniel Díaz

unread,
Feb 6, 2015, 2:25:55 AM2/6/15
to haskel...@googlegroups.com
I suspect the problem is that in "badTest", the "names2" handle is exhausted after the first execution of the inner loop, and subsequent reads do not produce anything. Perhaps you could invoke hSeek at the beginning of the inner loop.

Лев Никитин

unread,
Feb 6, 2015, 7:23:37 AM2/6/15
to haskel...@googlegroups.com
Opps... Yes. You are right. Thanks.
As I understand if names2.txt is a "big" file this is a best way to process it.
Another way is to load all data of the file into memory (to [], for example).
Or there is another better way to generate all combinations of data?


пятница, 6 февраля 2015 г., 0:06:14 UTC+6 пользователь Лев Никитин написал:

Daniel Díaz

unread,
Feb 6, 2015, 11:59:41 AM2/6/15
to haskel...@googlegroups.com
By the way, instead of using Producers directly, I think your use case is more elegantly described using the ListT that comes with pipes. Something like this:

badTest names1 names2 = enumerate $ do
  n1 <- Select (PG.concats . view PT.lines (P.fromHandle names1))
  n2 <- Select (PG.concats . view PT.lines (P.fromHandle names2)) 
  return (n1 `T.append` " ||| " `T.append` n2)

(This is a rough sketch, I haven't typechecked it.)

(>>=) in ListT is like "for" in the Producer, and "return" in ListT is like "yield" in the Producer.
Reply all
Reply to author
Forward
0 new messages