Hadoop Datalake

[Prakhyat Mallikarjun]

Hi Team,

We are working on a POC to build a data lake in Hadoop. In this regard we are evaluating ETL tools(Talend,Sqoop), Pig scripts,HIVE,Ooozie,Spark etc.. on top of Hadoop.

The clarity I am looking into is,

 1. How to plan efforts/code release from development environment to production? How to plan release of hadoop based applications from development environment to        production environment.   

    

 2. How to move pig scripts,HIVE QL,Oozie configurations from development environment to production?   

    I am looking into hello world examples using pig,hive, oozie etc on Hortonworks\cloudera site. Pig scripts/HIVEQL are deployed/created from UI.

    

Also most important, are there any Java components required apart from HIVE/PIG/OOzie? If yes how to move these java components into hadoop? 

Is there any possibility we can write some custom utilities in java and make them available in hadoop context? Consider we want write these java components in eclipse/local and want these components to be available in hadoop context.

Any replies and guidance will be appreciated.

-Prakhyat M M

[Charly CLAIRMONT]

Hello Prakhyat !

Firts ! This is a French speaking mailing list. So your question must be in French !

To quickly answer you, take a look of Apache Falcon !

Cheers.

--
Hadoop User Group France
http://hugfrance.fr | https://twitter.com/hugfrance
---
Vous recevez ce message, car vous êtes abonné au groupe Google Groupes "Hadoop User Group France".
Pour vous désabonner de ce groupe et ne plus recevoir d'e-mails le concernant, envoyez un e-mail à l'adresse hadoop-user-group-...@googlegroups.com.
Pour obtenir davantage d'options, consultez la page https://groups.google.com/d/optout.

--

-- -- -- --
Charly CLAIRMONT

ALTIC

une ALTernative à votre système d'Information et de Communication

Tel : 01 55 28 89 40

www : http://www.altic.org
twitter.com/egwada

charly+