不同的聚类标签

Question

我试图聚集在培训期间没有看到的新数据，只包含测试数据。培训文件有5个类别，而测试数据有7个类别（5 +2），其中2个是新类别。现在，我想运行k-均值来为新添加的类找到适当的群集，或者为它们创建新的群集（如果它们不接近任何群集）。

这是我的代码的一部分：

print("Reading training data...") 
 
#mydata = pd.read_csv('.\KDDTrain.csv', header=0) 
 
mydata = pd.read_csv('.\PTraining.csv', header=0) 
 

 
# select all but the last column as data 
 
X_train = mydata.ix[1:, :-1] 
 
X_train = np.array(X_train) 
 
n_samples, n_features = np.shape(X_train) 
 
# print np.shape(X_train) 
 

 
# select last column as target/class 
 
y_train = mydata.ix[1:, n_features] 
 
y_train = np.array(y_train) 
 

 
# encode target labels with numeric values from 0 to no of classes 
 
# print "Encoding class labels..." 
 
from sklearn import preprocessing 
 

 
label_encoder = preprocessing.LabelEncoder() 
 
label_encoder.fit(y_train) 
 
# print list(label_encoder.classes_) 
 
# print 'total no of classes in dataset=' + str(len(label_encoder.classes_)) 
 
y_train = label_encoder.transform(y_train) 
 

 
# n_samples, n_features = data.shape 
 
n_digits = len(np.unique(y_train)) 
 

 
print("Training data statistics") 
 
print("n_attack_catagories: %d, \t n_samples %d, \t n_features %d" 
 
     % (n_digits, n_samples, n_features)) 
 

 
sample_size = 300 
 

 
# Read test data 
 
mytestdata = pd.read_csv('.\KDDTest+.csv', header=0) 
 

 
print("Reading test data...") 
 
# select all but the last column as data 
 
X_test = mytestdata.ix[1:, :-1] 
 
X_test = np.array(X_test) 
 
# print np.shape(X_test) 
 

 
# select last column as target/class 
 
y_test = mytestdata.ix[1:, n_features] 
 
# print "actual labels" 
 
# print y_test 
 
y_test = label_encoder.transform(y_test) 
 
# print "Encoded labels" 
 
# print y_test 
 
y_test = np.array(y_test) 
 

 
n_samples_test, n_features_test = np.shape(X_test) 
 
n_digits_test = len(np.unique(y_test)) 
 
print("Test data statistics") 
 
print("n_attack_catagories: %d, \t n_samples %d, \t n_features %d" 
 
     % (n_digits_test, n_samples_test, n_features_test)) 
 

 
print(79 * '_')

，给这个错误

File "C:/Users/aalsham4/PycharmProjects/clusteringtask/clustering.py", line 87, in <module> 
 
    y_test = label_encoder.transform(y_test) 
 
    File "C:\Users\aalsham4\AppData\Local\Continuum\Miniconda3\lib\site-packages\sklearn\preprocessing\label.py", line 153, in transform 
 
    raise ValueError("y contains new labels: %s" % str(diff)) 
 
ValueError: y contains new labels: ['calss6' 'class7' ]

现在，我不知道如果我这样做这正确地将聚类标记的类或不聚类。

任何建议

Answer 1

作为@ Anony-慕斯已经说了，这不是一个k均值问题。考虑到您想要的课程数量，k-means将找到“自然”分组。一旦你分配这些标签，进一步的更新不再是一个k-均值问题。

您可以使用各种统计分析，启发式来决定一个新的类是否“足够接近”现有类。这通常使用平均值和偏差（你已经拥有的K均值类）的量度，密度以及其他你认为与你的问题相关的东西。

我建议你研究谱聚类算法，并尝试他们在整个数据集;这些更适合查找空位，对密度作出反应等（取决于您为此应用选择的算法）。