删除列表中的火花

Question

格式化我有一些火花像这样的代码：

val combPrdGrp = custPrdGrp3.join(cmpgnPrdGrp3) 

val combPrdGrp2 = combPrdGrp.groupByKey 


val combPrdGrp3 = combPrdGrp2.map{case (k3, vals3) => { 
    val valsString3 = vals3.map{case (id3, m3) => { 
    s"$id3 $m3" 
    } 
    } 
    s"$k3 $valsString3" 
}} 

当我做了combPrdGrp3.first我得到下面的结果。

res1: String = 110| List({'CNSMR_DIRCT_SAVG': {PRVCY_CALL: 1, PRVCY_SWP: 1, PRVCY_MAIL: 1, PRVCY_AFIL: 1, PRVCY_FCRA: 1, PRVCY_PIPE: 1, PRVCY_GLBA: 4}}| {'CARDXSL1503L': {contacted: '3/25/2015', channel: 'CARD-XSL', hit_home_date: 'ASPEN - Reminder', campaign: 'XSELL TO 360', creative: 'EM', refcode: 'Y'}}) 

我想List(和结束)被删除，但似乎无法弄清楚如何做到这一点。我试图用.pipe但似乎并没有这样的工作：

val combPrdGrp4 = combPrdGrp3.pipe("sed s/List((//g").pipe("sed s/)//g") 

会崩溃SC出于某种原因。尝试处理结果时，我会得到一个sc shutdown错误。

做combPrdGrp2.first会产生这样的结果：

res2: (String, Iterable[(String, String)]) = (110|,CompactBuffer(({'CNSMR_DIRCT_SAVG': {PRVCY_CALL: 1, PRVCY_SWP: 1, PRVCY_MAIL: 1, PRVCY_AFIL: 1, PRVCY_FCRA: 1, PRVCY_PIPE: 1, PRVCY_GLBA: 4}}|, {'CARDXSL1503L': {contacted: '3/25/2015', channel: 'CARD-XSL', hit_home_date: 'ASPEN - Reminder', campaign: 'XSELL TO 360', creative: 'EM', refcode: 'Y'}}))) 

Answer 1

你可以使用mkString格式化CompactBuffer/List自己：

List(1, 2, 3).toString 
// String = List(1, 2, 3) 

List(1, 2, 3).mkString 
// String = 123 

List(1, 2, 3).mkString(", ") 
// String = 1, 2, 3 

在你combPrdGrp2的情况下，这可能看起来像：

val elem = (
    "110|", 
    Iterable((
    "{'CNSMR_DIRCT_SAVG': {PRVCY_CALL: 1, PRVCY_SWP: 1, PRVCY_MAIL: 1, PRVCY_AFIL: 1, PRVCY_FCRA: 1, PRVCY_PIPE: 1, PRVCY_GLBA: 4}}|", 
    "{'CARDXSL1503L': {contacted: '3/25/2015', channel: 'CARD-XSL', hit_home_date: 'ASPEN - Reminder', campaign: 'XSELL TO 360', creative: 'EM', refcode: 'Y'}}" 
)) 
) 
val combPrdGrp2 = List(elem) 

combPrdGrp2.map { case (n, list) => 
    val formattedPairs = list.map { case (a, b) => s"$a $b" } 
    s"$n ${formattedPairs.mkString}" 
}