我已经创建了一个azure hdinsight火花集群,我试图在headnode上安装Spark Job Server。我已经ssh'ed到headnode,这里是我下面火花作业服务器安装在hdisght
其HDI 3.5星火1.6.3
- 的步骤安装SBT:
回声 “的deb https://dl.bintray.com/sbt/debian /” |须藤发球-a /etc/apt/sources.list.d/sbt.list
sudo易于关键进阶--keyserver HKP://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
须藤易于得到更新
须藤易于得到安装SBT
- GIT中克隆https://github.com/spark-jobserver/spark-jobserver.git
移动到火花观察者目录
SBT组件 此时我正在错误这样
主是用于火花1.6.3,因此我不改变b牧场。
*
[warn] Merging 'reference.conf' with strategy 'concat'
[error] 1 error was encountered during merge
java.lang.RuntimeException: deduplicate: different file contents found in the following:
/home/kmk/.ivy2/cache/io.netty/netty-all/jars/netty-all-4.0.37.Final.jar:META-INF/io.netty.versions.properties
/home/kmk/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.0.37.Final.jar:META-INF/io.netty.versions.properties
/home/kmk/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.0.37.Final.jar:META-INF/io.netty.versions.properties
/home/kmk/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.0.37.Final.jar:META-INF/io.netty.versions.properties
/home/kmk/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.0.37.Final.jar:META-INF/io.netty.versions.properties
/home/kmk/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.0.37.Final.jar:META-INF/io.netty.versions.properties
at sbtassembly.Assembly$.applyStrategies(Assembly.scala:140)
at sbtassembly.Assembly$.x$1$lzycompute$1(Assembly.scala:25)
at sbtassembly.Assembly$.x$1$1(Assembly.scala:23)
at sbtassembly.Assembly$.stratMapping$lzycompute$1(Assembly.scala:23)
at sbtassembly.Assembly$.stratMapping$1(Assembly.scala:23)
at sbtassembly.Assembly$.inputs$lzycompute$1(Assembly.scala:67)
at sbtassembly.Assembly$.inputs$1(Assembly.scala:57)
at sbtassembly.Assembly$.apply(Assembly.scala:83)
at sbtassembly.Assembly$$anonfun$assemblyTask$1.apply(Assembly.scala:240)
at sbtassembly.Assembly$$anonfun$assemblyTask$1.apply(Assembly.scala:237)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
java.lang.RuntimeException: Building python API received non-zero exit code 1
at scala.sys.package$.error(package.scala:27)
at PythonTasks$.buildPythonTask(PythonTasks.scala:25)
at $b9e935155022d705b7b0$$anonfun$jobServerPythonSettings$5.apply(build.sbt:110)
at $b9e935155022d705b7b0$$anonfun$jobServerPythonSettings$5.apply(build.sbt:110)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[error] (root/*:assembly) deduplicate: different file contents found in the following:
[error] /home/kmk/.ivy2/cache/io.netty/netty-all/jars/netty-all-4.0.37.Final.jar:META-INF/io.netty.versions.properties
[error] /home/kmk/.ivy2/cache/io.netty/netty-handler/jars/netty-handler-4.0.37.Final.jar:META-INF/io.netty.versions.properties
[error] /home/kmk/.ivy2/cache/io.netty/netty-buffer/jars/netty-buffer-4.0.37.Final.jar:META-INF/io.netty.versions.properties
[error] /home/kmk/.ivy2/cache/io.netty/netty-common/jars/netty-common-4.0.37.Final.jar:META-INF/io.netty.versions.properties
[error] /home/kmk/.ivy2/cache/io.netty/netty-transport/jars/netty-transport-4.0.37.Final.jar:META-INF/io.netty.versions.properties
[error] /home/kmk/.ivy2/cache/io.netty/netty-codec/jars/netty-codec-4.0.37.Final.jar:META-INF/io.netty.versions.properties
[error] (job-server-python/*:buildPython) Building python API received non-zero exit code 1
[error] Total time: 151 s, completed Apr 1, 2017 3:41:54 PM
*
我在这里做错了什么?