谷歌应用程序引擎 - 基本Cookie处理用的URLFetch

Question

专家，

看来，即使检索使用谷歌应用程序引擎最基本的网站，可以说是相当具有挑战性！

在我的情况，我想在这个网址检索网站：

http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Index/9809

我愿意接受所有的饼干，然后发布到这个网址的响应（这是一个简单的表格后）：

http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Search

我想后的字符串是：

'POSTDATA': 'Q585=1&Q2926=1&Q586=200000&Q587=240000&Q588=&Q9166=07071&Q591=1&Q592=1&Q594=3&searchButton=Search' 

我得到的问题是，网页上显示我的网络浏览器上的“Cookie未启用”。

正如你从下面的代码可以看到我尝试手动添加cookie，但是这不是成功的。

请帮忙！ -Todd

import cgi 
import webapp2 
import gzip 
import StringIO 

from google.appengine.api import users 
from google.appengine.api import urlfetch 
from BeautifulSoup import BeautifulSoup 

class MainPage(webapp2.RequestHandler): 
    def get(self): 
#  self.response.headers['Content-Type'] = 'text/html' 
     url = "http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Index/9809" 
     result = urlfetch.fetch(url, 
           headers={'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3', 
               'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 
               'Accept-Language': 'en-us', 
               'Accept-Encoding': 'gzip', 
               'Connection': 'keep-alive'}) 

     cookie = result.headers.get('set-cookie') 
     input_text = {'User-Agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3', 
         'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 
         'Accept-Language': 'en-us', 
         'Accept-Encoding': 'gzip', 
         'Connection': 'keep-alive', 
         'Content-Type': 'application/x-www-form-urlencoded', 
         'Content-Length': '97', 
         'POSTDATA': 'Q585=1&Q2926=1&Q586=200000&Q587=240000&Q588=&Q9166=07071&Q591=1&Q592=1&Q594=3&searchButton=Search', 
         'SiteProfile':'ProfileId=9809', 
         's_sess':'c_m=undefinedfeedity.comfeedity.com; s_sq=; s_cc=true;;', 
         'bhCookieSaveSess':'1', 
         'bhPrevResults':'bhjs=1&bhrf=http://www.google.com/'} 

     input_text['set-cookie'] = cookie 
##  self.response.out.write(input_text) 

     url2 = "http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Search" 
     result2 = urlfetch.fetch(url2, method='POST',headers=input_text) 

#  self.response.out.write(result.headers) 
     f = StringIO.StringIO(result2.content) 
     c = gzip.GzipFile(fileobj=f) 
     content = c.read() 
     self.response.out.write(content) 
#  self.response.out.write(result.content) 


app = webapp2.WSGIApplication([('/', MainPage)], 
           debug=True) 

YAML文件：

application: fimrates 
version: 1 
runtime: python27 
api_version: 1 
threadsafe: true 

handlers: 
- url: /.* 
    script: fimrates.app 

Answer 1

你实际上并没有设置你的代码中的Cookie。这里是你的代码的修改版本。

#import this module to be able to create a cookie 
import Cookie 

class BankHandler(webapp2.RequestHandler): 

    #put this function to create the cookie header 
    def createCookieHeader(self, cookie): 
     cookieHeader = "" 
     for value in cookie.values(): 
      cookieHeader += "%s=%s; " % (value.key, value.value) 
     return cookieHeader 

    def get(self): 
     ... 
     self.cookie = Cookie.SimpleCookie() #create the cookie 

     result = urlfetch.fetch({..., 
         #inject the cookie header             
         'Cookie': self.createCookieHeader(self.cookie)}) 

     cookie = result.headers.get('set-cookie', '') 
     input_text = {...} 

     #the header is 'Cookie' 
     input_text['Cookie'] = cookie 

     url2 = "http://tdbank.mortgagewebcenter.com/PowerSite/CheckRates.aspx/Search" 
     result2 = urlfetch.fetch(url2, method='POST',headers=input_text) 
     ...