2017-02-10 126 views
0

从HTML财务数据我有这个income statement,我很好奇,怎么有些人会去从HTML数据提取到一个数组。我发布了我编制的东西。有关改进方法的任何意见。会对使用stdlib,panads,numpy等来查看方法感兴趣。由于填充阵列使用python

import requests 
import pandas as pd 
from bs4 import BeautifulSoup 
from collections import OrderedDict 

url = r'http://gambler-restaurant-15377.bitballoon.com/' 

r = requests.get(url, headers={"User-Agent" : b'Opera/9.80 (Windows NT 5.2; U; zh-cn) Presto/2.6.30 Version/10.6'}) 
soup = BeautifulSoup(r.text, 'lxml') 

financial_statement=OrderedDict() 
for row in soup.find_all('table', attrs={'id':'dollarTable'})[0].find_all('tr'): 
    row_items = [] 
    for item in row: 
     try: 
      row_items.append(str(item.get_text(strip=True))) 
     except (AttributeError, ValueError): 
      # headers dont have a tr tag, and thus raises AttributeError 
      # 'Fiscal Year Ending in 2011' raises ValueError 
      pass 
    financial_statement[row_items[0]]=row_items[1:] 



('For year ending Dec,', ['2007', '2008', '2009', '2010', '2011', '2012', '2013', '2014', '2015']) 
('Revenues', ['306,057.0', '--', '--', '733,438.0', '390,625.0', '407,708.0', '110,000.0', '18,833.0', '10,417.0']) 
+0

请问你的代码的工作?如果不是,哪些不起作用? “ – Evert

+0

”有兴趣看看使用stdlib,panads,numpy等等来实现它的方法。“通常情况下,他们会试图使用这样一个库,并且如果它没有按预期工作,就会要求提供反馈。如果您一般在寻找您的代码审查:有代替http://codereview.stackexchange.com/。 – Evert

+0

它,但是我在想一个更好的解决办法是将每个项目以某种方式映射到相关的年份(这是第一项在字典)。因此,例如:“2007”的“收入306,057.0” – mac

回答

0
import requests 
import pandas as pd 
from bs4 import BeautifulSoup 
from collections import OrderedDict 

url = r'http://gambler-restaurant-15377.bitballoon.com/' 

r = requests.get(url, headers={"User-Agent" : b'Opera/9.80 (Windows NT 5.2; U; zh-cn) Presto/2.6.30 Version/10.6'}) 
soup = BeautifulSoup(r.text, 'lxml') 

financial_statement=OrderedDict() 
for row in soup.find_all('table', attrs={'id':'dollarTable'})[0].find_all('tr'): 
    row_items = [] 
    for item in row: 
     try: 
      row_items.append(str(item.get_text(strip=True))) 
     except (AttributeError, ValueError): 
      # headers dont have a tr tag, and thus raises AttributeError 
      # 'Fiscal Year Ending in 2011' raises ValueError 
      pass 
    financial_statement[row_items[0]]=row_items[1:] 



('For year ending Dec,', ['2007', '2008', '2009', '2010', '2011', '2012', '2013', '2014', '2015']) 
('Revenues', ['306,057.0', '--', '--', '733,438.0', '390,625.0', '407,708.0', '110,000.0', '18,833.0', '10,417.0']) 
+0

虽然这可能理论上回答这个问题,这将是[优选包括答案的主要部分(http://meta.stackoverflow.com/q/8259)这里,并提供链路参考。 –