Spark Mongodb: Error - java.lang.NoClassDefFoundError: com/mongodb/MongoDriverInformation - mongodb

Question

I am working with Spark-shell using Mongo-spark-connector to read/write data into MongoDB, while I am facing the below error, besides placing the required JARS as follows, can someone find what the problem is and help me out!
Thank you in advance
Jars:
mongodb-driver-3.4.2.jar;
mongodb-driver-sync-3.11.0.jar;
mongodb-driver-core-3.4.2.jar;
mongo-java-driver-3.4.2.jar;
mongo-spark-connector_2.11-2.2.0.jar;
mongo-spark-connector_2.11-2.2.7.jar
Error:
scala> MongoSpark.save(dfRestaurants.write.option("spark.mongodb.output.uri", "mongodb://username:password#server_name").option("spark.mongodb.output.database", "admin").option("spark.mongodb.output.collection", "myCollection").mode("overwrite"));
**java.lang.NoClassDefFoundError: com/mongodb/MongoDriverInformation**
at com.mongodb.spark.connection.DefaultMongoClientFactory.mongoDriverInformation$lzycompute(DefaultMongoClientFactory.scala:40)
at com.mongodb.spark.connection.DefaultMongoClientFactory.mongoDriverInformation(DefaultMongoClientFactory.scala:40)
at com.mongodb.spark.connection.DefaultMongoClientFactory.create(DefaultMongoClientFactory.scala:49)
at com.mongodb.spark.connection.MongoClientCache.acquire(MongoClientCache.scala:55)
at com.mongodb.spark.MongoConnector.acquireClient(MongoConnector.scala:242)
at com.mongodb.spark.MongoConnector.withMongoClientDo(MongoConnector.scala:155)
at com.mongodb.spark.MongoConnector.withDatabaseDo(MongoConnector.scala:174)
at com.mongodb.spark.MongoConnector.withCollectionDo(MongoConnector.scala:187)
at com.mongodb.spark.sql.DefaultSource.createRelation(DefaultSource.scala:72)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:654)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:654)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:654)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:273)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:267)
at com.mongodb.spark.MongoSpark$.save(MongoSpark.scala:192)
... 59 elided

Answer 1

This is a typical problem when you have incorrect dependencies. In your case:
Mongo Spark Connector 2.2.7 was built with driver 3.10/3.11, so it could be incompatible with driver 3.4
you have 2 different versions of Mongo Spark Connector - 2.2.0 & 2.2.7 - this also could lead to problems
The better solution is to pass Maven coordinates in --packages option when starting spark shell, and allow Spark pull the package with all necessary & correct dependencies:
spark-shell --packages org.mongodb.spark:mongo-spark-connector_2.11:<version>
please make sure that you're using the Scala version that is matching your Spark version (2.12 for Spark 3.0, 2.11 for previous versions). See documentation for more details.