How to Write MapReduce Transformation to populate into HBASE

52 views
Skip to first unread message

Ramasubramanian

unread,
Sep 16, 2013, 3:16:03 AM9/16/13
to kettle-d...@googlegroups.com
Dear All,

I have a requirement to read from text file input and write into multiple HBASE output using MapReduce. I am using Pentaho 4.4 version. Please help how to do this using MapReduce. Really appreciate if you could attach a sample for the same.

regards,
Rams

高清元

unread,
Sep 16, 2013, 5:25:10 AM9/16/13
to kettle-d...@googlegroups.com
写个mapreduce的程序,去处理文本文件,在reduce以后将结果写到hbase,你可以使用hbase api 也可以采用hfile的方式将数据装载到hbase
将上述的程序打个jar包,然后写个shell调用这个jar包,最后kettle调用这个shell脚本。


2013/9/16 Ramasubramanian <ramasubraman...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "kettle-developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kettle-develop...@googlegroups.com.
To post to this group, send email to kettle-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/kettle-developers.
For more options, visit https://groups.google.com/groups/opt_out.

Gaetano Fabiano

unread,
Sep 16, 2013, 5:46:57 AM9/16/13
to kettle-d...@googlegroups.com

2013/9/16 高清元 <hord...@gmail.com>

写个mapreduce的程序,去处理文本文件,在reduce以后将结果写到hbase,你可以使用hbase api 也可以采用hfile的方式将数据装载到hbase
将上述的程序打个jar包,然后写个shell调用这个jar包,最后kettle调用这个shell脚本。

I don't think that posting a reply in chinese is is ethically correct,
G.


Gaetano Fabiano
via Timpone n° 79
87055 San  Giovanni in Fiore (Cs) ITALY
phone: +39 0984 991 980

skype: deepyoudeep
msn: Dee...@hotmail.it
gtalk: fg.p...@gmail.com

Matt Casters

unread,
Sep 16, 2013, 5:54:15 AM9/16/13
to Kettle Developers mailing list
I don't mind it as the advice was on topic (even though I wouldn't advocate scripting if you have visual map/reduce available) but I would still advice folks to use the Big Data forum as it's read more by the members of the Pentaho Big Data team.

--
Matt Casters <mcas...@pentaho.org>
Chief Data Integration, Kettle founder, Author of Pentaho Kettle Solutions (Wiley)
Fonteinstraat 70 - 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
Pentaho  -  Powerful Analytics Made Easy


2013/9/16 Gaetano Fabiano <fg.p...@gmail.com>

--

高清元

unread,
Sep 16, 2013, 5:59:00 AM9/16/13
to kettle-d...@googlegroups.com
why ?



2013/9/16 Gaetano Fabiano <fg.p...@gmail.com>

--

高清元

unread,
Sep 16, 2013, 6:04:54 AM9/16/13
to kettle-d...@googlegroups.com
我只能说可视化的设计方式把你们宠坏了



2013/9/16 高清元 <hord...@gmail.com>

高清元

unread,
Sep 16, 2013, 6:08:09 AM9/16/13
to kettle-d...@googlegroups.com

Ramasubramanian

unread,
Sep 16, 2013, 6:11:21 AM9/16/13
to kettle-d...@googlegroups.com
Hi, 

Thanks for the URL… 

I am looking for writing a MapReduce Job with the components available in pentaho without going to Java…

I am aware of MapReduce program writing to HDFS or HIVE.. but I need for HBASE.. really appreciate if you could suggest something on this..

regards,
Rams

Matt Casters

unread,
Sep 16, 2013, 6:16:54 AM9/16/13
to Kettle Developers mailing list
If you need some sort of dedicated support, consider purchasing a Pentaho support contract.
This mailing list really is not the appropriate medium for your questions so please stop mailing now or I'll take appropriate action.

Thanks!

Matt
--
Matt Casters <mcas...@pentaho.org>
Chief Data Integration, Kettle founder, Author of Pentaho Kettle Solutions (Wiley)
Fonteinstraat 70 - 9400 OKEGEM - Belgium - Cell : +32 486 97 29 37
Pentaho  -  Powerful Analytics Made Easy


2013/9/16 Ramasubramanian <ramasubraman...@gmail.com>

高清元

unread,
Sep 16, 2013, 6:21:49 AM9/16/13
to kettle-d...@googlegroups.com
thrift
avro


2013/9/16 Ramasubramanian <ramasubraman...@gmail.com>

高清元

unread,
Sep 16, 2013, 6:25:33 AM9/16/13
to kettle-d...@googlegroups.com
《hbase 权威指南》
Chapter 6. AvailableClients 


2013/9/16 高清元 <hord...@gmail.com>

高清元

unread,
Sep 16, 2013, 6:29:29 AM9/16/13
to kettle-d...@googlegroups.com
Chapter 6. Available Clients
HBase comes with a variety of clients that can be used from various
programming languages. This chapter is going to give you an
overview of what is available.




2013/9/16 高清元 <hord...@gmail.com>
Reply all
Reply to author
Forward
0 new messages