根据与另一个DataFrame的列值匹配的列值对子集SparkR DataFrame

我有两个SparkR DataFrame，newHiresDF和salesTeamDF。我想根据newHiresDF$name的值salesTeamDF$name获得newHiresDF的子集，但我无法找到一种方法来实现此目的。以下是我尝试的代码。根据与另一个DataFrame的列值匹配的列值对子集SparkR DataFrame

#Create DataFrames 
newHires <- data.frame(name = c("Thomas", "George", "Bill", "John"), 
    surname = c("Smith", "Williams", "Brown", "Taylor")) 
salesTeam <- data.frame(name = c("Thomas", "Bill", "George"), 
    surname = c("Martin", "Clark", "Williams")) 
newHiresDF <- createDataFrame(newHires) 
salesTeamDF <- createDataFrame(salesTeam) 
display(newHiresDF) 

#Try to subset newHiresDF based on name values in salesTeamDF 
#All of the below result in errors 
NHsubset1 <- filter(newHiresDF, newHiresDF$name %in% salesTeamDF$name) 
NHsubset2 <- filter(newHiresDF, intersect(select(newHiresDF, 'name'), 
    select(salesTeamDF, 'name'))) 
NHsubset3 <- newHiresDF[newHiresDF$name %in% salesTeamDF$name,] #This is how it would be done in R 

#What I'd like NHsubset to look like: 
    name surname 
1 Thomas Smith 
2 George Williams 
3 Bill Brown

如果您愿意，PySpark代码也可以工作。

来源

2017-06-13 Gaurav Bansal

刚刚意识到这是https://stackoverflow.com/questions/43095208/subset-dataframe-based-on-matching-values-in-another-dataframe-pyspark-1-6-的近重复1但这个问题还没有得到解答。 –

找到了一个看似简单的解决方案：只需使用merge。

NHsubset <- merge(newHiresDF, select(salesTeamDF, 'name'))

来源

2017-06-13 20:42:28

根据与另一个DataFrame的列值匹配的列值对子集SparkR DataFrame

回答

相关问题