MongoDB Spark Unable to Insert data to existing collection.

[Ammar Shadiq]

I'm using Mongo Spark Connector, and I couldn't do insert operation to an existing collection.

with error message:

Exception in thread "main" java.lang.UnsupportedOperationException: MongoCollection already exists

at com.mongodb.spark.sql.DefaultSource.createRelation(DefaultSource.scala:99)

at com.mongodb.spark.sql.DefaultSource.createRelation(DefaultSource.scala:38)

at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:222)

at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:148)

...

My code of insert:

DataFrame centenarians = SparkSingletonUtils.sqlContext.sql("SELECT * FROM temp_table_name");

MongoSpark.write(centenarians).option("collection", "hundredClub").save(); // insert is not working

The usage page of that repo: https://github.com/mongodb/mongo-spark/wiki/Usage

said that I should use MongoWriter class. But I couldn't found it.

Is the class exist? or should i use different method to insert new data?

[Ross Lawley]

Hi Ammar,

This is expected behaviour as Spark's default save mode is to error if the "table" (collection) exists. For more information on save modes see the Spark SQL Documentation.  

In this example instead of erroring if the collection exists we append the results to the collection:

MongoSpark.write(centenarians).option("collection", "hundredClub").option("mode", "append").save(); 

All the best,

Ross

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/45102107-c009-4ccb-8855-20cd5c2a8525%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

{ name     : "Ross Lawley",
  title    : "Senior Software Engineer",
  location : "London, UK",
  twitter  : ["@RossC0", "@MongoDB"],
  facebook :"MongoDB"}

[Ammar Shadiq]

Hi Ross,

Thank you very much for the reply and explanation, It worked beautifully. 

Kind Regards,

Ammar

To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.

To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/45102107-c009-4ccb-8855-20cd5c2a8525%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.