Spark - ClassNotFoundException：org.eclipse.jetty.server.bio.SocketConnector

我试图将本地文件读入Spark RDD对象。我使用Spark的Gradle使用Java的API。我的操作系统是Win8.1 64bit。Spark - ClassNotFoundException：org.eclipse.jetty.server.bio.SocketConnector

代码：

import org.apache.spark.SparkConf; 
import org.apache.spark.api.java.JavaRDD; 
import org.apache.spark.api.java.JavaSparkContext; 

public class Resampling { 
    public static void main(String[] args) { 

     String folderName ="C:/Users/data/"; 
     String inputfilename = "file://" + folderName + "trainingset.csv"; 

     SparkConf conf = new SparkConf().setAppName("Simple Application").setMaster("local[2]").set("spark.executor.memory","1g"); 
     JavaSparkContext sc = new JavaSparkContext(conf); 
     JavaRDD<String> dataSet = sc.textFile(inputfilename).cache(); 

    } 
}

堆栈跟踪：

SLF4J: Class path contains multiple SLF4J bindings. 
SLF4J: Found binding in [jar:file:/C:/Users/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.5/6edffc576ce104ec769d954618764f39f0f0f10d/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: Found binding in [jar:file:/C:/Users/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-simple/1.7.7/8095d0b9f7e0a9cd79a663c740e0f8fb31d0e2c8/slf4j-simple-1.7.7.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 
16/02/07 18:01:03 INFO spark.SecurityManager: Changing view acls to: usern 
16/02/07 18:01:03 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(usern) 
16/02/07 18:01:04 INFO slf4j.Slf4jLogger: Slf4jLogger started 
16/02/07 18:01:04 INFO Remoting: Starting remoting 
16/02/07 18:01:04 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://[email protected]:65518] 
16/02/07 18:01:04 INFO Remoting: Remoting now listens on addresses: [akka.tcp://[email protected]:65518] 
16/02/07 18:01:04 INFO spark.SparkEnv: Registering MapOutputTracker 
16/02/07 18:01:04 INFO spark.SparkEnv: Registering BlockManagerMaster 
16/02/07 18:01:04 INFO storage.DiskBlockManager: Created local directory at C:\Users\AppData\Local\Temp\spark-local-20160207180104-3802 
16/02/07 18:01:04 INFO storage.MemoryStore: MemoryStore started with capacity 2.1 GB. 
16/02/07 18:01:04 INFO network.ConnectionManager: Bound socket to port 65521 with id = ConnectionManagerId(bla.corp,65521) 
16/02/07 18:01:04 INFO storage.BlockManagerMaster: Trying to register BlockManager 
16/02/07 18:01:04 INFO storage.BlockManagerInfo: Registering block manager bla.corp:65521 with 2.1 GB RAM 
16/02/07 18:01:04 INFO storage.BlockManagerMaster: Registered BlockManager 
16/02/07 18:01:04 INFO spark.HttpServer: Starting HTTP Server 
Exception in thread "main" java.lang.NoClassDefFoundError: org/eclipse/jetty/server/bio/SocketConnector 
    at org.apache.spark.HttpServer.start(HttpServer.scala:55) 
    at org.apache.spark.broadcast.HttpBroadcast$.createServer(HttpBroadcast.scala:156) 
    at org.apache.spark.broadcast.HttpBroadcast$.initialize(HttpBroadcast.scala:127) 
    at org.apache.spark.broadcast.HttpBroadcastFactory.initialize(HttpBroadcastFactory.scala:31) 
    at org.apache.spark.broadcast.BroadcastManager.initialize(BroadcastManager.scala:48) 
    at org.apache.spark.broadcast.BroadcastManager.<init>(BroadcastManager.scala:35) 
    at org.apache.spark.SparkEnv$.create(SparkEnv.scala:218) 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:202) 
    at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:53) 
    at org.gradle.Resampling.main(Resampling.java:26) 
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.server.bio.SocketConnector 
    at java.net.URLClassLoader.findClass(Unknown Source) 
    at java.lang.ClassLoader.loadClass(Unknown Source) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) 
    at java.lang.ClassLoader.loadClass(Unknown Source) 
    ... 10 more

我是第一次使用码头 9.0.0后来我试图用码头的旧版本中类似的问题建议 - 但仍然如此成功。

我的build.gradle依赖是：

dependencies { 
    compile 'com.sparkjava:spark-core:2.2' 
    compile 'org.apache.spark:spark-mllib_2.10:1.0.0' 
    compile 'org.eclipse.jetty:jetty-server:7.0.0.RC0' 
    compile 'org.eclipse.jetty:jetty-servlet:7.0.0.RC0' 

    testCompile group: 'junit', name: 'junit', version: '4.+' 
}

来源

2016-02-07 Serendipity