我有一段艰难的时间以编程方式从网站收集数据。我试图利用this example登录到服务器,但它不工作,因为我认为这是一种错误的登录类型。检索重定向到python中的登录页面的页面
当我尝试下载数据以解析html时,我尝试访问的网站重定向到登录页面。
这是网址:
https://mtred.com/rewards.html
和继承人代码:
# build opener with HTTPCookieProcessor
o = urllib2.build_opener(urllib2.HTTPCookieProcessor())
urllib2.install_opener(o)
# assuming the site expects 'user' and 'pass' as query params
p = urllib.urlencode({ 'UserLogin_username': 'mylogin', 'UserLogin_password': 'mypass' })
# perform login with params
f = o.open('http://www.mtred.com/user/login.html', p)
data = f.read()
f.close()
# second request should automatically pass back any
# cookies received during login... thanks to the HTTPCookieProcessor
f = o.open('https://www.mtred.com/rewards.html',p)
data = f.read()
print data
它踢我到登录页面,当我再次尝试打开奖励。我试图通过奖励自动做一些统计,因为这些信息不能通过公共API获得