Spark shell命令行

Question

我是Spark新手，想弄清楚如何使用Spark shell。

看着Spark的站点文档，它不显示如何创建目录或如何在Spark壳中查看我的所有文件。如果有人能帮助我，我将不胜感激。

Answer 1

在这种情况下，您可以假定Spark shell只是一个正常的Scala REPL，所以适用相同的规则。您可以使用:help获取可用命令的列表。

Welcome to 
     ____    __ 
    /__/__ ___ _____/ /__ 
    _\ \/ _ \/ _ `/ __/ '_/ 
    /___/ .__/\_,_/_/ /_/\_\ version 2.3.0 
     /_/ 

Using Scala version 2.11.8 (OpenJDK 64-Bit Server VM, Java 1.8.0_151) 
Type in expressions to have them evaluated. 
Type :help for more information. 

scala> :help 
All commands can be abbreviated, e.g., :he instead of :help. 
:edit <id>|<line>  edit history 
:help [command]   print this summary or command-specific help 
:history [num]   show the history (optional num is commands to show) 
:h? <string>    search the history 
:imports [name name ...] show import history, identifying sources of names 
:implicits [-v]   show the implicits in scope 
:javap <path|class>  disassemble a file or class name 
:line <id>|<line>  place line(s) at the end of history 
:load <path>    interpret lines in a file 
:paste [-raw] [path]  enter paste mode or paste a file 
:power     enable power user mode 
:quit     exit the interpreter 
:replay [options]  reset the repl and replay all previous commands 
:require <path>   add a jar to the classpath 
:reset [options]   reset the repl to its initial state, forgetting all session entries 
:save <path>    save replayable session to a file 
:sh <command line>  run a shell command (result is implicitly => List[String]) 
:settings <options>  update compiler options, if possible; see reset 
:silent     disable/enable automatic printing of results 
:type [-v] <expr>  display the type of an expression without evaluating it 
:kind [-v] <expr>  display the kind of expression's type 
:warnings    show the suppressed warnings from the most recent line which had any 

正如你可以看到上面你可以使用:sh调用shell命令。例如：

scala> :sh mkdir foobar 
res0: scala.tools.nsc.interpreter.ProcessResult = `mkdir foobar` (0 lines, exit 0) 

scala> :sh touch foobar/foo 
res1: scala.tools.nsc.interpreter.ProcessResult = `touch foobar/foo` (0 lines, exit 0) 

scala> :sh touch foobar/bar 
res2: scala.tools.nsc.interpreter.ProcessResult = `touch foobar/bar` (0 lines, exit 0) 

scala> :sh ls foobar 
res3: scala.tools.nsc.interpreter.ProcessResult = `ls foobar` (2 lines, exit 0) 

scala> res3.line foreach println 
line lines 

scala> res3.lines foreach println 
bar 
foo 

Answer 2

Click for image description here

：q或：退出命令用于从阶波纹退出。