解析XML并获取数据到熊猫数据框的问题

Question

我试图从一个XML文件中导入数据，该文件包含来自运动测试的呼吸数据。 的XML结构如下（简化以示出的一般结构）：

<?xml version="1.0"?> 
<Workbook xmlns="urn:schemas-microsoft-com:office:spreadsheet" 
    xmlns:o="urn:schemas-microsoft-com:office:office" 
    xmlns:x="urn:schemas-microsoft-com:office:excel" 
    xmlns:ss="urn:schemas-microsoft-com:office:spreadsheet" 
    xmlns:html="http://www.w3.org/TR/REC-html40"> 
    <Worksheet ss:Name="MetasoftStudio"> 
     <Table ss:ExpandedColumnCount="21" ss:ExpandedRowCount="458" x:FullColumns="1" x:FullRows="1" ss:StyleID="s62" ss:DefaultColumnWidth="53"> 
      <Column ss:StyleID="s62" ss:AutoFitWidth="0" ss:Width="137"/> 
      <Column ss:StyleID="s62" ss:AutoFitWidth="0" ss:Width="97"/> 
      <Column ss:StyleID="s62" ss:AutoFitWidth="0" ss:Width="137"/> 
      <Row ss:AutoFitHeight="0" ss:Height="26"> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">t</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">Phase</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">Marker</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">V'O2</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">V'O2/kg</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">V'O2/HR</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">HR</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">WR</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">V'E/V'O2</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">V'E/V'CO2</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">RER</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">V'E</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">BF</Data></Cell> 
      </Row> 
      <Row ss:Height="15"> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">h:mm:ss</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String"></Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String"></Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">L/min</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">ml/min/kg</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">ml</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">/min</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">W</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String"></Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String"></Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String"></Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">L/min</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">/min</Data></Cell> 
      </Row> 
      <Row ss:Height="15"> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">0:00:06</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String">Rest</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="String"></Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">0.27972413565454501</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">4.3706896196022598</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">4.5856415681072953</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">61</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">0</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">27.002532271037801</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">26.4113108545688</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">1.0223851598932201</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">10.155340000000001</Data></Cell> 
       <Cell ss:StyleID="Default"><Data ss:Type="Number">18.07</Data></Cell> 
      </Row> 
     </Table> 
    </Worksheet> 
</Workbook> 

我已经使用lxml解析和遍历XML文件，然后在每一个“细胞”中提取的“数据”追加到一个列表，然后追加该列表父列表使用的代码（给我的每一行的嵌套列表）：

from lxml import etree, objectify 
import pandas as pd 

with open('Python/cortex.xml') as infile: 
    xml_file = infile.read() 

    root = objectify.fromstring(xml_file) 

    header = [] 
    data = [] 

    for row in root.Worksheet.Table.getchildren(): 
     temp_row = [] 
     if not row.tag == '{urn:schemas-microsoft-com:office:spreadsheet}Column': 
      for cell in row.getchildren(): 
       temp_row.append(cell.Data) 
      data.append(temp_row) 
    header = data.pop(0) #remove the first 'row' and store in header list 
    del data[0] #remove 2nd line of superfluous data 

第一行给出了头，于是我pop是到自己的名单，以及行2包含每个变量的单位，所以我只是摆脱了这一点。到目前为止，所有的工作都很好（或者看起来似乎如此）...

现在我需要将它放入一个pd数据框以开始使用它。如果我去df = pd.DataFrame(data, columns=header)和我print(df)我得到： ValueError: Buffer has wrong number of dimensions (expected 1, got 32)

好了不知道那里发生了什么......如果我让DF不分配头和打印，我得到：

   0   1  2      3 \ 
0 [[[0:00:06]]] [[[Rest]]] [[[]]] [[[0.279724135654545]]] 
1 [[[0:00:09]]] [[[Rest]]] [[[]]] [[[0.465136232899829]]] 
2 [[[0:00:13]]] [[[Rest]]] [[[]]] [[[0.357975433456662]]] 
3 [[[0:00:19]]] [[[Rest]]] [[[]]] [[[0.543332419057909]]] 
4 [[[0:00:24]]] [[[Rest]]] [[[]]] [[[0.374604578743889]]] 

那并不是”你看起来不错！列表中列出的所有这些列表是从哪里来的！如果我迭代并打印嵌套列表data，它可以完美打印，但是一旦我尝试将其转换为df出现问题时。

任何人都可以启发我发生了什么，以及如何获得数据到PDDF？如果有更好的方法比我做得更好，那么我很乐意放弃它。

Answer 1

您可以通过构造函数创建列表清单，然后创建DataFrame。为了解析使用this solution：

from lxml import etree 

with (open('test.xml','r')) as f: 
    doc = etree.parse(f) 

namespaces={'o':'urn:schemas-microsoft-com:office:office', 
      'x':'urn:schemas-microsoft-com:office:excel', 
      'ss':'urn:schemas-microsoft-com:office:spreadsheet'} 

L = [] 
ws = doc.xpath('/ss:Workbook/ss:Worksheet', namespaces=namespaces) 
if len(ws) > 0: 
    tables = ws[0].xpath('./ss:Table', namespaces=namespaces) 
    if len(tables) > 0: 
     rows = tables[0].xpath('./ss:Row', namespaces=namespaces) 
     for row in rows: 
      tmp = [] 
      cells = row.xpath('./ss:Cell/ss:Data', namespaces=namespaces) 
      for cell in cells: 
#    print(cell.text); 
       tmp.append(cell.text) 
      L.append(tmp) 
print (L) 

---

[['t', 'Phase', 'Marker', "V'O2", "V'O2/kg", "V'O2/HR", 'HR', 'WR', 
    "V'E/V'O2", "V'E/V'CO2", 'RER', "V'E", 'BF'], 
['h:mm:ss', None, None, 'L/min', 'ml/min/kg', 'ml', 
'/min', 'W', None, None, None, 'L/min', '/min'], 
['0:00:06', 'Rest', None, '0.27972413565454501', '4.3706896196022598', 
    '4.5856415681072953', '61', '0', '27.002532271037801', '26.4113108545688', 
    '1.0223851598932201', '10.155340000000001', '18.07']] 

df = pd.DataFrame(L[2:], columns=L[0]) 
print (df) 
     t Phase Marker     V'O2    V'O2/kg \ 
0 0:00:06 Rest None 0.27972413565454501 4.3706896196022598 

       V'O2/HR HR WR   V'E/V'O2   V'E/V'CO2 \ 
0 4.5856415681072953 61 0 27.002532271037801 26.4113108545688 

        RER     V'E  BF 
0 1.0223851598932201 10.155340000000001 18.07