Custom input output Format

20 views
Skip to first unread message

Sarvesh Jain

unread,
Feb 1, 2015, 1:10:01 PM2/1/15
to rha...@googlegroups.com

Hello Everyone,
I am using Rhadoop. Details
Operating System : Mint 17 64 bit
JAVA -1.6 64 bit
Hadoop- version 2.5.2
R version 3.1.2
rmr2-3.3.0
rhdfs-1.0.8
Project requirement is to break input file per line in order to pass it to map function. and generate output in same sequence of lines.
So for the same reason i tried to write custom input format and output format.
Result of attached R program is (Null,Null ).

It would be great help if someone could help me to rectify problem.
Note: Reduce is purposefully set to NULL.
###################################
inputformatperline<-function(fil,totallines)
{
data<-readLines(con = fil,n=-1)
for(i in 1 :length(data))
keyval(i,data[i])

}
outputformatperline<-function(kv, con, keyval.size) {

out = values(kv)
writeLines(out, con = con)
}

map <-function(k,l1){
#‪#‎SomeOperation‬
return (keyval(k,l2))
}

from.dfs(mapreduce(input ='/Work/stop words testing/testhindi.txt',map =map,reduce=NULL,input.format = make.input.format(format=inputformatperline,mode="text"),output.format = make.output.format(format=outputformatperline,mode="text")))

Antonio Piccolboni

unread,
Feb 1, 2015, 4:14:44 PM2/1/15
to RHadoop Google Group
Your program seems to refer to a slightly older way of doing these things. What document did you access? Maybe you run into the wrong docs, maybe we let some of them get a little outdated. If you give me some pointers I'll try to fix them and then we'll see how that applies to your program. Thanks


Antonio

--
post: rha...@googlegroups.com ||
unsubscribe: rhadoop+u...@googlegroups.com ||
web: https://groups.google.com/d/forum/rhadoop?hl=en-US
---
You received this message because you are subscribed to the Google Groups "RHadoop" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rhadoop+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply all
Reply to author
Forward
0 new messages