CSV距离来三角距离矩阵在Python

Question

我有一个大的CSV关键字之间的相似性，我想将它转化成一个三角形的距离矩阵的（因为它是非常大而稀疏甚至会更好）进行分层聚类使用scipy。我现在的数据CSV样子：

a, b, 1 
b, a, 1 
c, a, 2 
a, c, 2 

我不知道如何做到这一点，我不能找到在python集群任何简单的教程。

感谢您的帮助！

Answer 1

有两个部分对这个问题：

• 你如何加载此格式的CSV距离为（也许稀疏）三角距离矩阵？

• 鉴于三角距离矩阵，你是怎么做到分层聚类SciPy的？

如何加载数据：我不认为scipy.cluster.hierarchy作品有稀疏的数据，让我们做密集。我也打算把它做成完整的方阵，然后拿出scipy想要的上三角，出于懒惰;如果你更聪明，你可以直接编入压缩版本。

from collections import defaultdict 
import csv 
import functools 
import itertools 
import numpy as np 

# name_to_id associates a name with an integer 0, 1, ... 
name_to_id = defaultdict(functools.partial(next, itertools.count())) 

with open('file.csv') as f: 
    reader = csv.reader(f) 

    # do one pass over the file to get all the IDs so we know how 
    # large to make the matrix, then another to fill in the data. 
    # this takes more time but uses less memory than loading everything 
    # in in one pass, because we don't know how large the matrix is; you 
    # can skip this if you do know the number of elements from elsewhere. 
    for name_a, name_b, dist in reader: 
     idx_a = name_to_id[name_a] 
     idx_b = name_to_id[name_b] 

    # make the (square) distances matrix 
    # this should really be triangular, but the formula for 
    # indexing into that is escaping me at the moment 
    n_elem = len(name_to_id) 
    dists = np.zeros((n_elem, n_elem)) 

    # go back to the start of the file and read in the actual data 
    f.seek(0) 
    for name_a, name_b, dist in reader: 
     idx_a = name_to_id[name_a] 
     idx_b = name_to_id[name_b] 
     dists[(idx_a, idx_b) if idx_a < idx_b else (idx_b, idx_a)] = dist 

condensed = dists[np.triu_indices(n_elem, 1)] 

然后，例如， scipy.cluster.hierarchy.linkage与condensed。要从索引映射到名称，可以使用类似于

id_to_name = dict((id, name) for name, id in name_to_id.iteritems())