Accessing content in hdfs in other data node from master node

18 views
Skip to first unread message

Rajasri

unread,
Jan 20, 2013, 9:28:48 AM1/20/13
to chenn...@googlegroups.com
hi all..Is it possible to access the hdfs of one data node from another node using Java code.? If yes can anyone tell me which "class" should be used ? and another doubt is , will one be able to access master node from data node ? Thanks in advance for ur time n effort :)
Regards,
Rajasri.J

Chandra Reddy

unread,
Jan 20, 2013, 11:01:47 AM1/20/13
to chenn...@googlegroups.com
hi,
we can get all blocks of the file using the commnad:
-hadoop fsck / -files -blocks

if you want all the splits if the file you can get from the FileInputFormat (InputFormat) class with method getSplits() method.

By knowing all splits/blocks location you can access the data.

thanks
chandra



--
 
 



--
Thanks,
-Chandra.

Arun Kumar Dharuman

unread,
Jan 20, 2013, 11:41:43 AM1/20/13
to chenn...@googlegroups.com

client need not bother on handling datanode.
all data access pass through namenode. namenode resolves which datanode has the content being read, and passes the client to appropriate datanode.

--
 
 

Rajasri janakiraman

unread,
Jan 20, 2013, 11:57:51 AM1/20/13
to chenn...@googlegroups.com
If a replica of the file (which was stored from master) is in another datanode, then we can access it. But is it possible to access any file in another datanode which wasn't stored from masternode i.e a file which was stored directly into hdfs at the data node ?


--
 
 

Reply all
Reply to author
Forward
0 new messages