change a mongo-spark connection configuration from a databricks python notebook

[Matteo Moci]

Hello Everyone, 

Just posted this question also on stack overflow [0], re posting it here in case anyone can help. 

I succeeded at connecting to mongodb from spark, using the mongo-spark connector [1] from a databricks notebook in python.

Right now I am configuring the mongodb uri in an environment variable, but it is not flexible, since I want to be able to change the connection parameter right in my notebook.

I read in the connector documentation [2] that it is possible to override any values set in the SparkConf.

How can I override the values from python?

Best, 

Matteo

[0] https://stackoverflow.com/questions/38011631/how-to-change-a-mongo-spark-connection-configuration-from-a-databricks-python-no

[1] https://github.com/mongodb/mongo-spark

[2] https://github.com/mongodb/mongo-spark/blob/master/doc/2-configuring.md#mongo-spark-connector-configuration

--

Matteo Moci
http://mox.fm

[Ross Lawley]

Hi,

I posted a full on Stackoverflow, but the short answer is you can supply any configuration option to the DataFrame reader or writer using the option method.

All the best,

Ross

[Ross Lawley]

Hi,

I posted a full on Stackoverflow, but the short answer is you can supply any configuration option to the DataFrame reader or writer using the option method.

All the best,

Ross

On Saturday, June 25, 2016 at 3:57:42 AM UTC+1, Matteo Moci wrote: