将Pandas DataFrame转换为橙色表

Question

我注意到这是一个issue on GitHub already。有没有人有任何代码将熊猫数据框转换为橙色表格？

明确地说，我有下表。

 user hotel star_rating user home_continent gender 
0   1  39   4.0  1    2 female 
1   1  44   3.0  1    2 female 
2   2  63   4.5  2    3 female 
3   2  2   2.0  2    3 female 
4   3  26   4.0  3    1 male 
5   3  37   5.0  3    1 male 
6   3  63   4.5  3    1 male 

Answer 1

Orange软件包的文档没有涵盖所有细节。根据lib_kernel.cpp，Table._init__(Domain, numpy.ndarray)仅适用于int和float。

他们确实应该为pandas.DataFrames或至少支持numpy.dtype("str")提供一个C级接口。

更新：添加table2df,df2table通过对int和float使用numpy大大提高了性能。

将这段脚本保存在您的橙色Python脚本集合中，现在您在橙色环境中配备了熊猫。

使用：a_pandas_dataframe = table2df(a_orange_table)，a_orange_table = df2table(a_pandas_dataframe)

注意：此脚本只能在Python 2.x中，参考@DustinTang的answer为Python 3.x的兼容脚本。

import pandas as pd 
import numpy as np 
import Orange 

#### For those who are familiar with pandas 
#### Correspondence: 
#### value <-> Orange.data.Value 
####  NaN <-> ["?", "~", "."] # Don't know, Don't care, Other 
#### dtype <-> Orange.feature.Descriptor 
####  category, int <-> Orange.feature.Discrete # category: > pandas 0.15 
####  int, float <-> Orange.feature.Continuous # Continuous = core.FloatVariable 
####             # refer to feature/__init__.py 
####  str <-> Orange.feature.String 
####  object <-> Orange.feature.Python 
#### DataFrame.dtypes <-> Orange.data.Domain 
#### DataFrame.DataFrame <-> Orange.data.Table = Orange.orange.ExampleTable 
####        # You will need this if you are reading sources 

def series2descriptor(d, discrete=False): 
    if d.dtype is np.dtype("float"): 
     return Orange.feature.Continuous(str(d.name)) 
    elif d.dtype is np.dtype("int"): 
     return Orange.feature.Continuous(str(d.name), number_of_decimals=0) 
    else: 
     t = d.unique() 
     if discrete or len(t) < len(d)/2: 
      t.sort() 
      return Orange.feature.Discrete(str(d.name), values=list(t.astype("str"))) 
     else: 
      return Orange.feature.String(str(d.name)) 


def df2domain(df): 
    featurelist = [series2descriptor(df.icol(col)) for col in xrange(len(df.columns))] 
    return Orange.data.Domain(featurelist) 


def df2table(df): 
    # It seems they are using native python object/lists internally for Orange.data types (?) 
    # And I didn't find a constructor suitable for pandas.DataFrame since it may carry 
    # multiple dtypes 
    # --> the best approximate is Orange.data.Table.__init__(domain, numpy.ndarray), 
    # --> but the dtype of numpy array can only be "int" and "float" 
    # --> * refer to src/orange/lib_kernel.cpp 3059: 
    # --> * if (((*vi)->varType != TValue::INTVAR) && ((*vi)->varType != TValue::FLOATVAR)) 
    # --> Documents never mentioned >_< 
    # So we use numpy constructor for those int/float columns, python list constructor for other 

    tdomain = df2domain(df) 
    ttables = [series2table(df.icol(i), tdomain[i]) for i in xrange(len(df.columns))] 
    return Orange.data.Table(ttables) 

    # For performance concerns, here are my results 
    # dtndarray = np.random.rand(100000, 100) 
    # dtlist = list(dtndarray) 
    # tdomain = Orange.data.Domain([Orange.feature.Continuous("var" + str(i)) for i in xrange(100)]) 
    # tinsts = [Orange.data.Instance(tdomain, list(dtlist[i]))for i in xrange(len(dtlist))] 
    # t = Orange.data.Table(tdomain, tinsts) 
    # 
    # timeit list(dtndarray) # 45.6ms 
    # timeit [Orange.data.Instance(tdomain, list(dtlist[i])) for i in xrange(len(dtlist))] # 3.28s 
    # timeit Orange.data.Table(tdomain, tinsts) # 280ms 

    # timeit Orange.data.Table(tdomain, dtndarray) # 380ms 
    # 
    # As illustrated above, utilizing constructor with ndarray can greatly improve performance 
    # So one may conceive better converter based on these results 


def series2table(series, variable): 
    if series.dtype is np.dtype("int") or series.dtype is np.dtype("float"): 
     # Use numpy 
     # Table._init__(Domain, numpy.ndarray) 
     return Orange.data.Table(Orange.data.Domain(variable), series.values[:, np.newaxis]) 
    else: 
     # Build instance list 
     # Table.__init__(Domain, list_of_instances) 
     tdomain = Orange.data.Domain(variable) 
     tinsts = [Orange.data.Instance(tdomain, [i]) for i in series] 
     return Orange.data.Table(tdomain, tinsts) 
     # 5x performance 


def column2df(col): 
    if type(col.domain[0]) is Orange.feature.Continuous: 
     return (col.domain[0].name, pd.Series(col.to_numpy()[0].flatten())) 
    else: 
     tmp = pd.Series(np.array(list(col)).flatten()) # type(tmp) -> np.array(dtype=list (Orange.data.Value)) 
     tmp = tmp.apply(lambda x: str(x[0])) 
     return (col.domain[0].name, tmp) 

def table2df(tab): 
    # Orange.data.Table().to_numpy() cannot handle strings 
    # So we must build the array column by column, 
    # When it comes to strings, python list is used 
    series = [column2df(tab.select(i)) for i in xrange(len(tab.domain))] 
    series_name = [i[0] for i in series] # To keep the order of variables unchanged 
    series_data = dict(series) 
    print series_data 
    return pd.DataFrame(series_data, columns=series_name) 

Answer 2

像这样？

table = Orange.data.Table(df.as_matrix()) 

Orange中的列将获得通用名称（a1，a2 ...）。如果要从数据框中复制名称和类型，请从数据框中构建Orange.data.Domain对象（http://docs.orange.biolab.si/reference/rst/Orange.data.domain.html#Orange.data.Domain.init），并将其作为上面的第一个参数传递。

请参阅http://docs.orange.biolab.si/reference/rst/Orange.data.table.html中的构造函数。

Answer 3

为了将pandas DataFrame转换为橙色表，您需要构建一个指定列类型的域。

对于连续变量，您只需提供变量的名称，但对于离散变量，还需要提供所有可能值的列表。

下面的代码将构造一个域名为您的数据帧，并将其转换为橙色表：

import numpy as np 
from Orange.feature import Discrete, Continuous 
from Orange.data import Domain, Table 
domain = Domain([ 
    Discrete('user', values=[str(v) for v in np.unique(df.user)]), 
    Discrete('hotel', values=[str(v) for v in np.unique(df.hotel)]), 
    Continuous('star_rating'), 
    Discrete('user', values=[str(v) for v in np.unique(df.user)]), 
    Discrete('home_continent', values=[str(v) for v in np.unique(df.home_continent)]), 
    Discrete('gender', values=['male', 'female'])], False) 
table = Table(domain, [map(str, row) for row in df.as_matrix()]) 

地图（STR，行）所需步骤，橙色知道，数据中包含的离散特征值（而不是值列表中的值的索引）。

Answer 4

此代码从@TurtleIzzy修改为Python3。

import numpy as np 
from Orange.data import Table, Domain, ContinuousVariable, DiscreteVariable 


def series2descriptor(d): 
    if d.dtype is np.dtype("float") or d.dtype is np.dtype("int"): 
     return ContinuousVariable(str(d.name)) 
    else: 
     t = d.unique() 
     t.sort() 
     return DiscreteVariable(str(d.name), list(t.astype("str"))) 

def df2domain(df): 
    featurelist = [series2descriptor(df.iloc[:,col]) for col in range(len(df.columns))] 
    return Domain(featurelist) 

def df2table(df): 
    tdomain = df2domain(df) 
    ttables = [series2table(df.iloc[:,i], tdomain[i]) for i in range(len(df.columns))] 
    ttables = np.array(ttables).reshape((len(df.columns),-1)).transpose() 
    return Table(tdomain , ttables) 

def series2table(series, variable): 
    if series.dtype is np.dtype("int") or series.dtype is np.dtype("float"): 
     series = series.values[:, np.newaxis] 
     return Table(series) 
    else: 
     series = series.astype('category').cat.codes.reshape((-1,1)) 
     return Table(series)