远程mysql使用spark的数据库访问错误

从远程mysql数据库访问表，该数据库在eclipse中本地很好地工作，但在尝试从尝试时抛出错误。远程mysql使用spark的数据库访问错误

ERROR SparkContext: Error initializing SparkContext.java.lang.IllegalArgumentException: java.net.UnknownHostException: namenode 
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUtil.java:378)

低于火花提交：

spark-submit \ 
--jars /home/hduser/.ivy2/cache/com.typesafe/config/bundles/config-1.3.1.jar \ 
--class "retail.stack" --master local[2] \ 
--driver-memory 2g --executor-memory 2g --executor-cores 2 \ 
sample-spark-180417_2.11-1.0.jar \ 
/home/hduser/spark-training/workspace/demos/output/extra_in_src \ 
/home/hduser/spark-training/workspace/demos/output/extradest \ 
/home/hduser/spark-training/workspace/demos/output/bestMatch

下面的代码：

val spark = SparkSession.builder().master("local").appName("Database") 
    .getOrCreate() 

val jdbcUsername = "root" 
val jdbcPassword = "root" 
val url = "jdbc:mysql://192.168.175.35:3306/sample?useSSL=false" 

val connectionProperties = new java.util.Properties() 
connectionProperties.put("user", jdbcUsername) 
connectionProperties.put("password", jdbcPassword) 

val querySource = "(select * from source1) as src" 
val queryDestination = "(select * from destination1) as dest" 

val source = spark.read.jdbc(url, querySource, connectionProperties).rdd.map(_.mkString(",")) 
val destination = spark.read.jdbc(url, queryDestination, connectionProperties).rdd.map(_.mkString(","))

谢谢

来源

2017-07-06 Vignesh

可以请你分享一下你试过的代码吗？ –

@senthilkumarp代码更新..！ – Vignesh

https://stackoverflow.com/questions/33174386/accessing-hdfs-ha-from-spark-job-unknownhostexception-error请看这 –

我解决它通过重新配置连接字符串。它工作得很好。

来源

2017-07-07 10:52:42 Vignesh

远程mysql使用spark的数据库访问错误

回答

相关问题