do we have mongodb connector for prestodb?

[ngoct...@gmail.com]

Hello all,

Do anyone have plan to build the mongodb connector for prestodb? Our company using mongodb but we want to query with join purpose from prestodb.

Thanks.

[Min Yu]

I was working on the mongodb connector for fun but unfortunately didn't finish the first version yet.

Thanks

Min

[Min Yu]

Hello,

I've sent a pull request of the first draft of mongodb plugin.

https://github.com/facebook/presto/pull/3337

It would require a lot of optimizations and bug fixes for various situations, but I hope it would be a good start.

Thanks

Min

[Thanh Truong]

Thanks so much Min Yu.

Thanks. 

[baska...@gmail.com]

Hi,

Does it support the Mongoldb version 3.2?

Can i have a download link for the latest presto mongo connector?

Thanks,
Baskar.S

[Dongmin Yu]

Hello,

Here’s the source repo of the PR.

https://github.com/miniway/presto/tree/mongodb

I tested with 3.0.5 but it expect it would work with 3.2 as long as mongo-java driver 3.1.0 is compatible with mongo 3.2.

Thanks
Min

[Baskar Sikkayan]

Thanks for the link.

I see many connectors here. 

Could you please share the catalog properties for mongo? Or, is it same as oracle with irk change.

--
Sent from Gmail Mobile

[Dongmin Yu]

https://github.com/miniway/presto/blob/mongodb/presto-docs/src/main/sphinx/connector/mongodb.rst

Has the configuration information. If you build the package, you can see complete html document from this raw file.

Thanks
Min

[Baskar Sikkayan]

Hi,

  I am unable to build it. Getting the following error. Any help pls ....

ERROR] Failed to execute goal pl.project13.maven:git-commit-id-plugin:2.1.13:revision (default) on project presto-spi: .git directory could not be found! Please specify a valid [dotGitDirectory] in your pom.xml -> [Help 1]

[ERROR] 

[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.

[ERROR] 

[ERROR] For more information about the errors and possible solutions, please read the following articles:

[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

[ERROR] 

[ERROR] After correcting the problems, you can resume the build with the command

[ERROR]   mvn <goals> -rf :presto-spi

[Baskar Sikkayan]

Hi,

   I just tried with the below link.

https://github.com/miniway/presto/tree/mongodb

When I do git clone of this project, I am not getting mongoldb connector. But mvn clean install works fine without any issues.

If I download the source, I am able to see it, but I am not able to build it successfully. 

Is there anything else I am missing?

Help will be appreciated.

Thanks,

Baskar.S

[Min]

After you clone the project, you need to change the working branch to mongodb.

The downloaded zip will not be built with maven.

Thanks

Min

[Baskar Sikkayan]

Will git checkout mongodb work?

[Baskar Sikkayan]

Hi,

  

  I am able to build now. Getting some error and just wanted to confirm if it is compatible with Presto-server 0.133 or any other version with bug fix.

https://prestodb.io/docs/current/installation/deployment.html

Thanks,

Baskar.S

[Dongmin Yu]

Hi,

what error did you get?

If you able to build the project by “mvn clean package”, you will have a tarball under presto-server/target/presto-server-VERSION.tar.gz
You should use the the tarball to avoid any conflict between version.

Basic configuration would be almost same in the document. And here’s my test configuration after I launch `mongod` in my local laptop.

## etc/config.properties
datasources=mongodb
####

## etc/catalog/mongodb.properties
connector.name=mongodb
mongodb.seeds=localhost
####

After you launch the presto,

./bin/launcher.py run | start

You can use mongodb connector through the client
./presto-cli/target/presto-cli-VERSION-executable.jar —catalog mongodb


Thanks
Min

[Baskar Sikkayan]

Thanks for the detail info.

I get the following error.  I replaced the real credential and the server from the logs.

cluster-ClusterId{value='3241234134234', description='null'}-<host>:<port> org.mongodb.driver.cluster Exception in monitor thread while connecting to server <host>:<port>

com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=null, userName='<username>', source='<db>', password=<hidden>, mechanismProperties={}}

at com.mongodb.connection.SaslAuthenticator.authenticate(SaslAuthenticator.java:61)

at com.mongodb.connection.DefaultAuthenticator.authenticate(DefaultAuthenticator.java:32)

at com.mongodb.connection.InternalStreamConnectionInitializer.authenticateAll(InternalStreamConnectionInitializer.java:99)

at com.mongodb.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:44)

at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:115)

at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:128)

at java.lang.Thread.run(Thread.java:745)

Caused by: com.mongodb.MongoCommandException: Command failed with error 18: 'Authentication failed.' on server <host>:<port>. The full response is { "ok" : 0.0, "code" : 18, "errmsg" : "Authentication failed." }

at com.mongodb.connection.CommandHelper.createCommandFailureException(CommandHelper.java:170)

at com.mongodb.connection.CommandHelper.receiveCommandResult(CommandHelper.java:123)

at com.mongodb.connection.CommandHelper.executeCommand(CommandHelper.java:32)

at com.mongodb.connection.SaslAuthenticator.sendSaslStart(SaslAuthenticator.java:95)

at com.mongodb.connection.SaslAuthenticator.authenticate(SaslAuthenticator.java:45)

... 6 more

## etc/catalog/mongodb.properties

connector.name=mongodb

mongodb.seeds=<host>:<port>

mongodb.credentials=<username>:<password>@<db>

I am able to connect to mongo from a java program using the above credential.

We use mongo java driver 3.2.0.

Thanks,

Baskar.S

[Baskar Sikkayan]

HI,

  Am I missing any other configuration?

[Dongmin Yu]

Hmm,

It’s very strange. I get the same message only when the credential is wrong on any of password or collection.

Thanks
Min

[Baskar Sikkayan]

Hi,

  I am extremely sorry. It was a typo error.

I am not getting the error now on starting the server.

Thanks a lot for your support.

Now, I connect to the cli using,

 ./presto --server localhost:8080 --catalog mongodb --schema <schemaName>

presto:<schemaName>>  

When i try sql like queries here, it is throwing error.

May I know the query syntax to be used here?

Thanks,

Baskar.S

[Min]

What error message or log did you get at your Presto client, sever and mongod?

You can use most of standard SQL. Try simple 'select * from <table>' If your table/collection contains complex data type like nested array/dictionary it might not be supported well. We need more test cases for it. Please test with a simple collection first.

[Baskar Sikkayan]

I connect using the presto client as follows.

./presto --server localhost:8080 --catalog mongodb --schema <DB_Name>

Client Error,

presto:esdbd1> select * from mongodb.<schemaName>.<collectionName>;

Query 20160124_215502_00007_2y3wq failed: line 1:8: SELECT * not allowed in queries without FROM clause

select * from mongodb.esdbd1.businessGroupReference

I tired the below also.

select * from <collectionName>

Still the error is same.

Server Error :-

2016-01-24T14:01:49.296-0800 ERROR Query-20160124_220149_00008_2y3wq-204 com.facebook.presto.execution.QueryStateMachine Query 20160124_220149_00008_2y3wq failed

com.facebook.presto.sql.analyzer.SemanticException: line 1:8: SELECT * not allowed in queries without FROM clause

at com.facebook.presto.sql.analyzer.StatementAnalyzer.analyzeSelect(StatementAnalyzer.java:1548)

at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuerySpecification(StatementAnalyzer.java:939)

at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuerySpecification(StatementAnalyzer.java:209)

at com.facebook.presto.sql.tree.QuerySpecification.accept(QuerySpecification.java:125)

at com.facebook.presto.sql.tree.AstVisitor.process(AstVisitor.java:22)

at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuery(StatementAnalyzer.java:735)

at com.facebook.presto.sql.analyzer.StatementAnalyzer.visitQuery(StatementAnalyzer.java:209)

at com.facebook.presto.sql.tree.Query.accept(Query.java:103)

at com.facebook.presto.sql.tree.AstVisitor.process(AstVisitor.java:22)

at com.facebook.presto.sql.analyzer.Analyzer.analyze(Analyzer.java:60)

at com.facebook.presto.execution.SqlQueryExecution.doAnalyzeQuery(SqlQueryExecution.java:254)

at com.facebook.presto.execution.SqlQueryExecution.analyzeQuery(SqlQueryExecution.java:240)

at com.facebook.presto.execution.SqlQueryExecution.start(SqlQueryExecution.java:204)

at com.facebook.presto.execution.QueuedExecution.lambda$start$335(QueuedExecution.java:68)

at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

at java.lang.Thread.run(Thread.java:745)

Do let me know if you need further details in this regards.

Thanks

[gonzalo...@mercadolibre.com]

Hello Min... Do you know when will to add this connector for MongoDB to the master build of prestodb.io?

[konfu...@gmail.com]

Hi Baskar,

Did you solve your issue?
Because I am facing the same issue right now and cannot find any elaborate documentation on the connector.

Can you please explain what you did to solve the issue?

Thanks.