我想寻求与Anaconda Jupyter笔记本相关的帮助。我想在Jupyter笔记本中编写PySpark和SparkR,然后我按照在线教程教授如何将Apache Toree与Jupyter笔记本一起安装。Apache Toree与蟒蛇Jupyter笔记本
我使用Cloudera Manager包来管理我的kerberized Hadoop集群。
但是,我无法在服务器日志中使用以下错误打开Apache Toree PySpark的内核。
[I 15:24:50.529 NotebookApp] Creating new notebook in
[I 15:24:52.079 NotebookApp] Kernel started: 8cb4838c-2171-4672-96a4-b21ef191ffc6
Starting Spark Kernel with SPARK_HOME=/opt/cloudera/parcels/CDH/lib/spark
WARNING: User-defined SPARK_HOME (/opt/cloudera/parcels/CDH-5.9.0-1.cdh5.9.0.p2024.2115/lib/spark) overrides detected (/opt/cloudera/parcels/CDH/lib/spark).
WARNING: Running spark-class from user-defined location.
Exception in thread "main" java.lang.NoSuchMethodError: joptsimple.OptionParser.acceptsAll(Ljava/util/Collection;Ljava/lang/String;)Ljoptsimple/OptionSpecBuilder;
at org.apache.toree.boot.CommandLineOptions.<init>(CommandLineOptions.scala:37)
at org.apache.toree.Main$delayedInit$body.apply(Main.scala:25)
at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.App$$anonfun$main$1.apply(App.scala:71)
at scala.collection.immutable.List.foreach(List.scala:318)
at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
at scala.App$class.main(App.scala:71)
at org.apache.toree.Main$.main(Main.scala:24)
at org.apache.toree.Main.main(Main.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
我已经把jopt-simple-4.5.jar放在了Toree lib和Spark home中。是否有任何地方需要将该罐放在那里,以便在试图创建新笔记本时能够找到它?谢谢。
最好的问候, 拉赫蒂
我有同样的问题。链接https://issues.apache.org/jira/browse/TOREE-324说要用jopt-simple 4.9重新编译。我已经完成了重新编译,但需要部署IT - 所以我不能说它解决了这个问题。我有一个关于https://gitter.im/apache/toree的最新话题 - 敬请关注。 – codeaperature