2016-08-16 17 views
4

我的代码工作很好火花外壳:IDEA为什么找不到toDS()和toDF()函数?

scala> case class Person(name:String,age:Int) 
defined class Person 

scala> val person = Seq(Person("ppopo",23)).toDS() 
person: org.apache.spark.sql.Dataset[Person] = [name: string, age: int] 

scala> person.show() 
+-----+---+ 
| name|age| 
+-----+---+ 
|ppopo| 23| 
+-----+---+ 

,但错在IDEA:

enter image description here
我已经进口的所有罐子在“火花2.0.0彬hadoop2.7 /罐子/“,但仍然无法找到这个功能。

回答

7

我发现这个问题,usering TODS()之前添加一个依赖:

val ss = SparkSession.builder().appName("DataSet Test") 
    .master("local[*]").getOrCreate() 

// This import is needed 
import ss.implicits._ 
val simpleDS = Seq(Person("po",12)).toDS() 
simpleDS.show()