无法在认证模式下使用Mongo与Mongo-Spark连接器连接

我试图通过Mongo-Spark连接器在远程机器上使用MongoDB实例运行Spark提交作业。无法在认证模式下使用Mongo与Mongo-Spark连接器连接

当我发起的mongod服务没有的--auth标志，像这样运行命令：

./bin/spark-submit --master spark://10.0.3.155:7077 \ 
        --conf "spark.mongodb.input.uri=mongodb://10.0.3.156/test.coll?readPreference=primaryPreferred" \ 
        --conf "spark.mongodb.output.uri=mongodb://10.0.3.156/test.coll" \ 
        --packages org.mongodb.spark:mongo-spark-connector_2.11:2.0.0 \ 
        app1.py

一切工作就像一个魅力。

但是，当我与运行的mongod服务的--auth标志，并运行这样的：

./bin/spark-submit --master spark://10.0.3.155:7077 \ --conf "spark.mongodb.input.uri=mongodb://admin:[email protected]/test.coll?readPreference=primaryPreferred" \ --conf "spark.mongodb.output.uri=mongodb://admin:[email protected]/test.coll" \ --packages org.mongodb.spark:mongo-spark-connector_2.11:2.0.0 \ app1.py

我得到这些错误：

py4j.protocol.Py4JJavaError: An error occurred while calling o47.save. : com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches WritableServerSelector. Client view of cluster state is {type=UNKNOWN, servers=[{address=10.0.3.156:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=null, userName='admin', source='test', password=<hidden>, mechanismProperties={}}}, caused by {com.mongodb.MongoCommandException: Command failed with error 18: 'Authentication failed.' on server 10.0.3.156:27017. The full response is { "ok" : 0.0, "code" : 18, "errmsg" : "Authentication failed." }}}] at com.mongodb.connection.BaseCluster.createTimeoutException(BaseCluster.java:369) at com.mongodb.connection.BaseCluster.selectServer(BaseCluster.java:101) at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:75) at com.mongodb.binding.ClusterBinding$ClusterBindingConnectionSource.<init>(ClusterBinding.java:71) at com.mongodb.binding.ClusterBinding.getWriteConnectionSource(ClusterBinding.java:68) at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:158) at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:133) at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:128) at com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:118) at com.mongodb.operation.DropCollectionOperation.execute(DropCollectionOperation.java:54) at com.mongodb.operation.DropCollectionOperation.execute(DropCollectionOperation.java:39) at com.mongodb.Mongo.execute(Mongo.java:781) at com.mongodb.Mongo$2.execute(Mongo.java:764) at com.mongodb.MongoCollectionImpl.drop(MongoCollectionImpl.java:419) at com.mongodb.spark.sql.DefaultSource$$anonfun$createRelation$1.apply(DefaultSource.scala:89) at com.mongodb.spark.sql.DefaultSource$$anonfun$createRelation$1.apply(DefaultSource.scala:89) at com.mongodb.spark.MongoConnector$$anonfun$withCollectionDo$1.apply(MongoConnector.scala:186) at com.mongodb.spark.MongoConnector$$anonfun$withCollectionDo$1.apply(MongoConnector.scala:184) at com.mongodb.spark.MongoConnector$$anonfun$withDatabaseDo$1.apply(MongoConnector.scala:171) at com.mongodb.spark.MongoConnector$$anonfun$withDatabaseDo$1.apply(MongoConnector.scala:171) at com.mongodb.spark.MongoConnector.withMongoClientDo(MongoConnector.scala:154) at com.mongodb.spark.MongoConnector.withDatabaseDo(MongoConnector.scala:171) at com.mongodb.spark.MongoConnector.withCollectionDo(MongoConnector.scala:184) at com.mongodb.spark.sql.DefaultSource.createRelation(DefaultSource.scala:89) at org.apache.spark.sql.execution.datasources.DataSource.write(DataSource.scala:518) at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:215) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:280) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:214) at java.lang.Thread.run(Thread.java:748)

我检查的凭据和角色，一切都很好。不知道我在做什么错...

来源

2017-07-30 Ran P

您需要配置authSource查询参数在连接字符串，否则提供的数据库（测试）将用于：

spark.mongodb.input.uri=mongodb://10.0.3.156/test.coll?authSource=admin&readPreference=primaryPreferred

来源

2017-07-31 11:29:10 Ross

准确。谢谢 –

因此，这毕竟是一个凭证问题。问题是我试图连接到test DB，而我输入的凭据是admin DB。

就是这样。

来源

2017-07-31 11:09:16

无法在认证模式下使用Mongo与Mongo-Spark连接器连接

回答

相关问题