GraphFrame：丢失或同时加载类文件

这里创建一个图形检测无效的依赖是代码：

import org.graphframes._ 

// Node DataFrames 
val v = sqlContext.createDataFrame(List(
    ("a", "Alice", 34), 
    ("b", "Bob", 36), 
    ("c", "Charlie", 30), 
    ("d", "David", 29), 
    ("e", "Esther", 32), 
    ("f", "Fanny", 36), 
    ("g", "Gabby", 60) 
)).toDF("id", "name", "age") 

// Edge DataFrame 
val e = sqlContext.createDataFrame(List(
    ("a", "b", "friend"), 
    ("b", "c", "follow"), 
    ("c", "b", "follow"), 
    ("f", "c", "follow"), 
    ("e", "f", "follow"), 
    ("e", "d", "friend"), 
    ("d", "a", "friend"), 
    ("a", "e", "friend") 
)).toDF("src", "dst", "relationship") 

// Create a GraphFrame 
val g = GraphFrame(v, e)

但是，这是我收到的错误：

error: missing or invalid dependency detected while loading class file 'GraphFrame.class'. Could not access type Logging in package org.apache.spark, because it (or its dependencies) are missing. Check your build definition for missing or conflicting dependencies. (Re-run with -Ylog-classpath to see the problematic classpath.) A full rebuild may help if 'GraphFrame.class' was compiled against an incompatible version of org.apache.spark.

我正在使用Apache Spark 2.1和Scala 2.11。任何建议可能是什么问题？

来源

2017-06-13 sjishan

下载以下从中央的maven回购

com.typesafe.scala-logging_scala-logging-api_2.11-2.1.2.jar  
graphframes_graphframes-0.5.0-spark2.1-s_2.11.jar 
org.slf4j_slf4j-api-1.7.7.jar 
com.typesafe.scala-logging_scala-logging-slf4j_2.11-2.1.2.jar 
org.scala-lang_scala-reflect-2.11.0.jar

包添加以下到您的火花，default.conf文件（逗号分隔在上面下载jar文件所在的绝对路径列表）

spark.jars   path_2_jar/org.slf4j_slf4j-api-1.7.7.jar, path_2_jar/org.scala-lang_scala-reflect-2.11.0.jar, path_2_jar/graphframes_graphframes-0.5.0-spark2.1-s_2.11.jar, path_2_jar/com.typesafe.scala-logging_scala-logging-slf4j_2.11-2.1.2.jar, path_2_jar/com.typesafe.scala-logging_scala-logging-api_2.11-2.1.2.jar

来源

2017-09-11 12:02:19 ashwinids

GraphFrame：丢失或同时加载类文件

回答

相关问题