在R中绘制xgboost模型的AUC

Question

我目前正在关注以下link的幻灯片。我在幻灯片121/128上，我想知道如何复制AUC。作者没有解释如何这样做（幻灯片124中也一样）。其次，在幻灯片125上生成以下代码;

bestRound = which.max(as.matrix(cv.res)[,3]-as.matrix(cv.res)[,4]) 
bestRound 

我收到以下错误;

错误as.matrix（cv.res），2]：下标出界

以下代码中的数据可以从here被下载和我已经产生下面的代码供你参考。

问题：如何生成作为作者的AUC以及为什么下标越界？

-----代码------

# Kaggle Winning Solutions 

train <- read.csv('train.csv', header = TRUE) 
test <- read.csv('test.csv', header = TRUE) 
y <- train[, 1] 
train <- as.matrix(train[, -1]) 
test <- as.matrix(test) 

train[1, ] 

#We want to determin who is more influencial than the other 

new.train <- cbind(train[, 12:22], train[, 1:11]) 
train = rbind(train, new.train) 
y <- c(y, 1 - y) 

x <- rbind(train, test) 

(dat[,i]+lambda)/(dat[,j]+lambda) 

A.follow.ratio = calcRatio(x,1,2) 
A.mention.ratio = calcRatio(x,4,6) 
A.retweet.ratio = calcRatio(x,5,7) 
A.follow.post = calcRatio(x,1,8) 
A.mention.post = calcRatio(x,4,8) 
A.retweet.post = calcRatio(x,5,8) 
B.follow.ratio = calcRatio(x,12,13) 
B.mention.ratio = calcRatio(x,15,17) 
B.retweet.ratio = calcRatio(x,16,18) 
B.follow.post = calcRatio(x,12,19) 
B.mention.post = calcRatio(x,15,19) 
B.retweet.post = calcRatio(x,16,19) 

x = cbind(x[,1:11], 
      A.follow.ratio,A.mention.ratio,A.retweet.ratio, 
      A.follow.post,A.mention.post,A.retweet.post, 
      x[,12:22], 
      B.follow.ratio,B.mention.ratio,B.retweet.ratio, 
      B.follow.post,B.mention.post,B.retweet.post) 

AB.diff = x[,1:17]-x[,18:34] 
x = cbind(x,AB.diff) 
train = x[1:nrow(train),] 
test = x[-(1:nrow(train)),] 

set.seed(1024) 
cv.res <- xgb.cv(data = train, nfold = 3, label = y, nrounds = 100, verbose = FALSE, 
       objective = 'binary:logistic', eval_metric = 'auc') 

情节AUC图形这里

set.seed(1024) 
cv.res = xgb.cv(data = train, nfold = 3, label = y, nrounds = 3000, 
       objective='binary:logistic', eval_metric = 'auc', 
       eta = 0.005, gamma = 1,lambda = 3, nthread = 8, 
       max_depth = 4, min_child_weight = 1, verbose = F, 
       subsample = 0.8,colsample_bytree = 0.8) 

这里是我的代码遇到

突破

#bestRound: - subscript out of bounds 

bestRound <- which.max(as.matrix(cv.res)[,3]-as.matrix(cv.res)[,4]) 
bestRound 
cv.res 

cv.res[bestRound,] 

set.seed(1024) bst <- xgboost(data = train, label = y, nrounds = 3000, 
          objective='binary:logistic', eval_metric = 'auc', 
          eta = 0.005, gamma = 1,lambda = 3, nthread = 8, 
          max_depth = 4, min_child_weight = 1, 
          subsample = 0.8,colsample_bytree = 0.8) 
preds <- predict(bst,test,ntreelimit = bestRound) 

result <- data.frame(Id = 1:nrow(test), Choice = preds) 
write.csv(result,'submission.csv',quote=FALSE,row.names=FALSE) 

Answer 1

代码h的很多部分AVE没有什么意义我，但这里是构建模型所提供的数据的一个小例子：

数据：

train <- read.csv('train.csv', header = TRUE) 
y <- train[, 1] 
train <- as.matrix(train[, -1]) 

型号：

library(xgboost) 
cv.res <- xgb.cv(data = train, nfold = 3, label = y, nrounds = 100, verbose = FALSE, 
       objective = 'binary:logistic', eval_metric = 'auc', prediction = T) 

为了得到交叉验证的预测必须指定prediction = T当致电xgb.cv时。

为了获得最好的迭代：

it = which.max(cv.res$evaluation_log$test_auc_mean) 
best.iter = cv.res$evaluation_log$iter[it] 

绘制ROC曲线上的交叉验证的结果：

library(pROC) 
plot(pROC::roc(response = y, 
       predictor = cv.res$pred, 
       levels=c(0, 1)), 
    lwd=1.5) 

为了得到一个混淆矩阵（假设0.5的概率为阈值）：

library(caret) 
confusionMatrix(ifelse(cv.res$pred <= 0.5, 0, 1), y) 
#output 
      Reference 
Prediction 0 1 
     0 2020 638 
     1 678 2164 

       Accuracy : 0.7607   
       95% CI : (0.7492, 0.772) 
    No Information Rate : 0.5095   
    P-Value [Acc > NIR] : <2e-16   

        Kappa : 0.5212   
Mcnemar's Test P-Value : 0.2823   

      Sensitivity : 0.7487   
      Specificity : 0.7723   
     Pos Pred Value : 0.7600   
     Neg Pred Value : 0.7614   
      Prevalence : 0.4905   
     Detection Rate : 0.3673   
    Detection Prevalence : 0.4833   
     Balanced Accuracy : 0.7605   

     'Positive' Class : 0 

话虽这么说，一个目标应该是调整超参数与交叉验证，如ETA，γ，λ，子样本，colsample_bytree，colsample_bylevel等

最简单的方法是建立在您上使用expand.grid网格搜索作为自定义函数的一部分，使用xgb.cv的网格上的超参数的所有组合并使用lapply）。如果你需要更多的细节，请评论。