我试图登录和刮一个工作网站,并发送任何关键词时,我发现了通知。我想我已经正确地跟踪了fepath的“xpath登录[iovation ]”但我不能提取的价值,这是我迄今所做的登陆如何使用python请求登录到网站
import requests
from lxml import html
header = {"User-Agent":"Mozilla/4.0 (compatible; MSIE 5.5;Windows NT)"}
login_url = 'https://www.upwork.com/ab/account-security/login'
session_requests = requests.session()
#get csrf
result = session_requests.get(login_url)
tree=html.fromstring(result.text)
auth_token = list(set(tree.xpath('//*[@name="login[_token]"]/@value')))
auth_iovat = list(set(tree.xpath('//*[@name="login[iovation]"]/@value')))
# create payload
payload = {
"login[username]": "[email protected]",
"login[password]": "pa$$w0rD",
"login[_token]": auth_token,
"login[iovation]": auth_iovation,
"login[redir]": "/home"
}
#perform login
scrapeurl='https://www.upwork.com/ab/find-work/'
result=session_requests.post(login_url, data = payload, headers = dict(referer = login_url))
#test the result
print result.text
运行浏览器,而'的JavaScript '并检查你是否看到这个值 - 也许这个值是由'JavaScript'添加的,并且请求不运行'JavaScript' – furas
感谢furas,当我禁用javascript时,loginp [iovation]的值字段消失了 –
你可以不用javascript登录?在这个页面上,我看到名字为[account-security-ui-combined.js]的文件(https://www.upwork.com/ab/account-security/static-assets/js/account-security-ui-combined .js文件?3482da8?3482da8)。也许它可以帮助找到这些代码。或者你将不得不使用'Selenium'来控制运行JavaScript的浏览器。 – furas