Intro to MapReduce

[krishna chaitanya]

Hi,

 

I am a hadoop beginner. I am aware of setting up the hadoop environment and have basic idea of map-reduce functionality.

Can you elaborate on Map-Reduce functionality and how to start with map-reduce alogorithms.

 

 

 

[Prasad]

Hi Krishna,

Just like hello world example in any java application, word count is the basic exercise for learning map reduce. I would suggest you to go through the tutorial from apache.

http://hadoop.apache.org/docs/r0.17.1/mapred_tutorial.html 

Ask us any doubts if you face any difficulties in understanding the logic.

Regards,

Prasad

[Prasad]

Probably you can go through this example from latest packages http://wiki.apache.org/hadoop/WordCount . The previous post is with deprecated ones.

[Chintan Jain]

If you are a fan pf hadoop and big database, then you should been attending the amazon web services AWS today.

I guess you missed out.. Lot of potential learning..

--
 
 

--

CHINTAN R JAIN

Facebook Fan Page - http://www.facebook.com/meetchintanjain
Twitter Profile - http://twitter.com/meetchintanjain
Facebook Profile - http://www.facebook.com/chintanjain25

[Pavan Kulkarni]

Please check the book Hadoop "A Definitive Guide " from Tom White . It is written very comprehensively and gives you an overall idea of how Hadoop works and how to write MapReduce programs.

--
 
 

--

--With Regards
Pavan Kulkarni

[siva]

  Hi experienced hadoop professionals,

     Can you share some real time use cases where and how to apply MapReduce?
     This will really add more interest to this topic...

[Prasad]

Hi Siva,

The primary constraint in using hadoop and map reduce is this can be used for scenarios where there is a huge volume of data and batch processing is applicable.

It is definitely not suited for real time processing.

 

To quote a few examples,

1. Consider deriving various analytics such as audit log reports from monthly system log report of all employees in organization

2. An example to analyse the trend in customers buying capacity of various products in an organisation. (Behavioural Marketing)

3.  Credit cards fraud detection by analysing users' history of banking records where sampling will not be best possible solution to analyse the entire data set

4. Call Records Processing in telecommunication industries.

5. Image Processing areas etc..

[kumar vaibhav]

Hi,

I have mentioned this earlier too but I think what beginners need are not books like Hadoop:The Definitive Guide or Hadoop In Action. Although they are good books but there is no fun at all in reading huge volumes like these. What beginners need are simple step-by-step POC ideas that someone can guide them through and helpl building them. 

@Prasad - Hadoop is being used by Stumbleupon recommendation engine and companies like Explorys. I agree it cannot be real time since its a batch processing system but frameworks like Storm are a big help.

Thanks,

Kumar

[Senthil Kumar]

@Vaibhav

Sorry for the delayed reply. We will discuss the same in presence with other members during next meetup.

Senthil