将网站中的数据提取到python中的字典中

以下是从Indeed.com提取特定作业数据的代码和相应输出。随着数据我有很多垃圾，我想分开标题，位置，工作描述和其他重要功能。我怎样才能将它转换成字典？将网站中的数据提取到python中的字典中

from bs4 import BeautifulSoup 
import urllib2 
final_site = 'http://www.indeed.com/cmp/Pullskill-techonoligies/jobs/Data-Scientist-229a6b09c5eb6b44?q=%22data+scientist%22' 
html = urllib2.urlopen(final_site).read() 
soup = BeautifulSoup(html) 
deep = soup.find("td","snip") 
deep.get("p","ul") 
deep.get_text(strip= True)

输出：

u'Title : Data ScientistLocation : Seattle WADuration : Fulltime/PermanentJob Responsibilities:Implement advanced and predictive analytics models usingJava,R, and Pythonetc.Develop deep expertise with Company\u2019s data warehouse, systems, product and other resources.Extract, collate and analyze data from a variety of sources to provide insights to customersCollaborate with the research team to incorporate qualitative insights into projects where appropriateKnowledge, Skills and Experience:Exceptional problem solving skillsExperience withJava,R, and PythonAdvanced data mining and predictive modeling (especially Machine learning techniques) skillsMust have statistics orientation (Theory and applied)3+ years of business experience in an advanced analytics roleStrong Python and R programming skills are required. SAS, MATLAB will be plusStrong SQL skills are looked for.Analytical and decisive strategic thinker, flexible problem solver, great team player;Able to effectively communicate to all levelsImpeccable attention to detail and very strong ability to convert complex data into insights and action planThanksNick ArthurLead Recruiternick(at)pullskill(dot)com201-497-1010 Ext: 106Salary: $120,000.00 /yearRequired experience:Java And Python And R And PHD Level Education: 4 years5 days ago-save jobwindow[\'result_229a6b09c5eb6b44\'] = {"showSource": false, "source": "Indeed", "loggedIn": false, "showMyJobsLinks": true,"undoAction": "unsave","relativeJobAge": "5 days ago","jobKey": "229a6b09c5eb6b44", "myIndeedAvailable": true, "tellAFriendEnabled": false, "showMoreActionsLink": false, "resultNumber": 0, "jobStateChangedToSaved": false, "searchState": "", "basicPermaLink": "http://www.indeed.com", "saveJobFailed": false, "removeJobFailed": false, "requestPending": false, "notesEnabled": true, "currentPage" : "viewjob", "sponsored" : false, "reportJobButtonEnabled": false};\xbbApply NowPlease review all application instructions before applying to Pullskill Technologies.(function(d, s, id){var js, iajs = d.getElementsByTagName(s)[0], iaqs = \'vjtk=1aa24enhqagvcdj7&hl=en_US&co=US\'; if (d.getElementById(id)){return;}js = d.createElement(s); js.id = id; js.async = true; js.src = \'https://apply.indeed.com/indeedapply/static/scripts/app/bootstrap.js\'; js.setAttribute(\'data-indeed-apply-qs\', iaqs); iajs.parentNode.insertBefore(js, iajs);}(document, \'script\', \'indeed-apply-js\'));Recommended JobsData Scientist, Energy AnalyticsRenew Financial-Oakland, CARenew Financial-5 days agoData ScientistePrize-Seattle, WAePrize-7 days agoData ScientistDocuSign-Seattle, WADocuSign-12 days agoEasily applyEngineer - US Citizen or Permanent ResidentVoxel Innovations-Raleigh, NCIndeed-8 days agoEasily applyData ScientistUnity Technologies-San Francisco, CAUnity Technologies-22 days agoEasily apply'

来源

2016-01-27 dsl1990

找到工作摘要元素，发现里面所有b元素，并用:分裂每个b元素的文本：

for elm in soup.find("span", id="job_summary").p.find_all("b"): 
    label, text = elm.get_text().split(" : ") 

    print(label.strip(), text.strip())

来源

2016-01-27 18:36:58 alecxe

感谢。这只是给标题，位置和位置。我希望在工作中获得工作职责和其他信息。无论出现在工作岗位上的任何信息都是有价值的。 – dsl1990

-1

如果您的输出始终具有相同的结构，你可以使用正则表达式来创建字典。

dict = {} 
title_match = re.match(r'Title : (.+)(?=Location)', output) 
dict['Title'] = title_match.group(1) 
location_match = re.match(r'Location : (.+)(?=Duration)', output) 
dict['Location'] = location_match.group(1)

当然，这是一个非常脆弱的解决方案，它可能会更好地为您使用BeautifulSoup的内置解析得到你想要的结果，我想他们可能是由标准的标签包围。

来源

2016-01-27 18:33:52 Barnabus

将网站中的数据提取到python中的字典中

回答

相关问题