Any Read/Write support for MongoDB ?

[Ebot Tabi]

Hi guys

i wish to find out if there any mongodb read/write support for pangool out there ?

thanks

[Iván de Prado]

Hi Ebot, 

There is not out-of-the-box mongodb read/write connector for Pangool. But shouldn't be difficult to create a Pangool Map Only job that writes documents to Mongo in pararell. 

Iván

--
Has recibido este mensaje porque estás suscrito al grupo "pangool-user" de Grupos de Google.
Para anular la suscripción a este grupo y dejar de recibir sus correos electrónicos, envía un correo electrónico a pangool-user...@googlegroups.com.
Para obtener más opciones, visita https://groups.google.com/groups/opt_out.
 
 

--
Iván de Prado

CEO & Co-founder

www.datasalt.com

[Pere Ferrera Bertran]

Actually you can use any Hadoop OutputFormat in Pangool, and I think there is a MongoOutputFormat in mongo-hadoop.

2013/6/3 Iván de Prado <iv...@datasalt.com>

--
Pere Ferrera

CTO & Co-founder

www.datasalt.com

[Ebot Tabi]

Thanks for the suggestion.

[Ebot Tabi]

Nice thoughts there, i think i will give a try a on getting MongoInputFormat & MongoOutputFormat. Already Pangool is doing well for our log analysis, i have over 150Million docs sitting in mongodb, now need to crunch those as well. i will try out this suggestion and see how it goes.

[Pere Ferrera]

Hello Ebot,

Find this new blog post: http://www.datasalt.com/2013/06/scraping-qype-reviews-with-pangool-and-saving-results-into-mongodb/

We have implemented a poor-man's review parser for Qype... and it saves the result into MongoDB.

As you will see, there is one two things you need to do for persisting the output of a Pangool job to Mongo:

1) Defining the mongo URI:

MongoConfigUtil.setOutputURI(conf, "mongodb://localhost/test.qype");

2) Setting the appropriate OutputFormat in the Pangool job builder (be it TupleMRBuilder or MapOnlyJobBuilder):

builder.setOutput(new Path(outPath), new HadoopOutputFormat(MongoOutputFormat.class), Text.class, BSONObject.class);

Give it a try and let us know,

Thanks,

[Ebot Tabi]

Hey bro you rock, thanks for this awesome piece.

--

Has recibido este mensaje porque estás suscrito a un tema del grupo "pangool-user" de Grupos de Google.

Para anular la suscripción a este tema, visita https://groups.google.com/d/topic/pangool-user/iGb2xghltyo/unsubscribe. Para anular la suscripción a este grupo y todos sus temas, envía un correo electrónico a pangool-user...@googlegroups.com.

Para obtener más opciones, visita https://groups.google.com/groups/opt_out.
 
 

--
Ebot T.