点击一个按钮后刮数据

Question

我想从这个website刮数据。下拉菜单填充进入基础数据，所以我提出多交的请求是这样的：

url = 'http://59.180.234.21:85/index.aspx' 

with requests.Session() as session: 
    response = session.get(url) 
    soup = BeautifulSoup(response.content, "html5lib") 
data = { 
    'ddlDistrict': '165', 
    '__VIEWSTATE': soup.find('input', {'name': '__VIEWSTATE'}).get('value', ''), 
    '__EVENTVALIDATION': soup.find('input', {'name': '__EVENTVALIDATION'}).get('value', ''), 
} 
response = session.post(url, data=data) 
soup = BeautifulSoup(response.content, "html5lib") 

data = { 
    'ddlDistrict': '165', 
    'ddlPS': '11', 
    '__VIEWSTATE': soup.find('input', {'name': '__VIEWSTATE'}).get('value', ''), 
    '__EVENTVALIDATION': soup.find('input', {'name': '__EVENTVALIDATION'}).get('value', ''), 
} 
response = session.post(url, data=data) 
soup = BeautifulSoup(response.content, "html5lib") 

data = { 
    'ddlDistrict': '165', 
    'ddlPS': '11', 
    'txtRegNo':'100', 
    'ddlYear': '2011', 
    '__VIEWSTATE': soup.find('input', {'name': '__VIEWSTATE'}).get('value', ''), 
    '__EVENTVALIDATION': soup.find('input', {'name': '__EVENTVALIDATION'}).get('value', ''), 
} 
response = session.post(url, data=data) 

这个做最后一页之后有一个按钮，我可以点击并查看报告中的HTML表格。我希望能够模拟点击按钮并获得我可以使用BS解析的响应。请让我知道如何做到这一点。样本输入，区：“新德里DIST”，警察局：“Con.Place”，FirNo：“100”，年：“2011”将给你一个冷杉观看。该按钮具有以下代码：

onclick="javascript:WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions("DgRegist$ctl03$imgDelete", "", true, "", "", false, false))" 

Answer 1

如果您可以生成按钮正在生成的http请求，那么您将获得所需的数据。如果按钮没有提出任何请求，那么数据已经存在，你只需要找到它并解析出来。

编辑：

在你的情况下，它提交表单数据重定向到同一页。为此，您需要在页面请求中包含表单数据，并在响应中包含结果数据。例如：

import requests 

headers = { 
    'Origin': 'http://59.180.234.21:85', 
    'Accept-Encoding': 'gzip, deflate', 
    'Accept-Language': 'en-US,en;q=0.8', 
    'Upgrade-Insecure-Requests': '1', 
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36', 
    'Content-Type': 'application/x-www-form-urlencoded', 
    'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8', 
    'Cache-Control': 'max-age=0', 
    'Referer': 'http://59.180.234.21:85/index.aspx', 
    'Connection': 'keep-alive', 
} 

data = [ 
    ('__EVENTTARGET', ''), 
    ('__EVENTARGUMENT', ''), 
    ('__LASTFOCUS', ''), 
    ('__VIEWSTATE', '/wEPDwUJMTQ2MDgwNjA1D2QWAgIDD2QWAgIFD2QWBAIBD2QWCGYPZBYEAgEPZBYCAgEPEGRkFgFmZAIDD2QWAgIBDxBkZBYAZAIBD2QWBAIBD2QWAgIBDxAPFgYeDURhdGFUZXh0RmllbGQFCENpdHlOYW1lHg5EYXRhVmFsdWVGaWVsZAUIQ2l0eUNvZGUeC18hRGF0YUJvdW5kZ2QQFREMLS0tU0VMRUNULS0tDUNFTlRSQUwgRElTVFQSQ1JJTUUgQU5EIFJBSUxXQVlTEEVBU1QgREVMSEkgRElTVFQJSUdJIERJU1RUD05FVyBERUxISSBESVNUVAtOT1JUSCBESVNUVBBOT1JUSCBFQVNUIERJU1RUEE5PUlRIIFdFU1QgRElTVFQLT1VURVIgRElTVFQLU09VVEggRElTVFQQU09VVEggRUFTVCBESVNUVBBTT1VUSCBXRVNUIERJU1RUElNQRUNJQUwgQ0VMTCBESVNUVA5TUFVXICYgQyBESVNUVAlWSUdJTEFOQ0UKV0VTVCBESVNUVBURDC0tLVNFTEVDVC0tLQMxNjIDMTY0AzE2OAMxNjkDMTY1AzE2NgMxNzMDMTcyAzE3NAMxNjcDOTU1AzE3MQM5NTQDOTUzAzE2MQMxNzAUKwMRZ2dnZ2dnZ2dnZ2dnZ2dnZ2cWAQIFZAIDD2QWAgIBDxAPFgYfAAUHUFNfTmFtZR8BBQdQU19Db2RlHwJnZBAVCQwtLS1TRUxFQ1QtLS0PQkFSQUtIQU1CQSBST0FEDUNIQU5BS1lBIFBVUkkKQ09OLiBQTEFDRQpFWEguIEdST1VOC01BTkRJUiBNQVJHClBULiBTVFJFRVQKVElMQUsgTUFSRwtUVUdMQUsgUk9BRBUJDC0tLVNFTEVDVC0tLQIwMgIwNwIxMQIxMgIxNQIyMgIzNQIzNhQrAwlnZ2dnZ2dnZ2cWAQIDZAICD2QWBAIBD2QWAgIBDw8WAh4JTWF4TGVuZ3RoAgRkZAIDD2QWAgIBDxBkDxYHAgECAgIDAgQCBQIGAgcWBxAFBDIwMTcFBDIwMTdnEAUEMjAxNgUEMjAxNmcQBQQyMDE1BQQyMDE1ZxAFBDIwMTQFBDIwMTRnEAUEMjAxMwUEMjAxM2cQBQQyMDEyBQQyMDEyZxAFBDIwMTEFBDIwMTFnZGQCAw9kFgRmD2QWAgIBDxBkZBYBAgNkAgIPZBYCAgEPDxYCHgdFbmFibGVkaGRkAgMPZBYCAgEPZBYCZg9kFgICAQ9kFgQCAQ88KwALAGQCAw88KwARAgEQFgAWABYADBQrAABkGAEFCmdyZG12dGhlZnQPZ2SPDrK3c7Ukzq5Wg/XtZSQMgDzEoWpRz8kXOVH1TO1LcA=='), 
    ('__EVENTVALIDATION', '/wEdAC8iT6D3HjIr+ivdq0yBTgClCsHRaAEHIr772zKgggdQ+5cM7ByNsRG4qWi12q7B1tveFDGmjlPiBn9IJO8m9jt8W1Wcqc3FqlgV9EENz1OdJenvj2TG96ujSrFeprbtr3RTWKEdLSZa5NFLztoz81urAMmLvBzV7Qyb4qeGafdxuGr4cVZnct4CZh3KKsvt+xdAs0fg094ls2+uRMaFDPjjvXQmtkg7agsuhug+xMVSXXqKkbM01pitokD3Lzhr/+Zrc1JkJBoj+hAGr8ppVSNG4Yj6XkYB+ZGeix5+udiv9J9IjbG0sujSnR9YEqeLFuIKGVNDezkrxdfUawGK33AxvjAuIFmExdxunofmSVMj2KhPcg/6G9KkHuC16bwbWAqSNP2Vcw4/0wky0Un3Ssd3cGZtjtv+8Amihean2n5uODEqvswSsIcl9+U0P3atZA9gLfz10VlY0S1jS6520f4SrEv7IkN+08PXTozm9OT6/xtTbG8qE+XuugkwabaWLRSnp8pclR+ltj186j/FXuFQADgLnY9pn1HgIJ6W1oaeYRGUECgQhKzewPcXKgm68keQY5UuqQXqAyLatchak9gZ0UXh+krR/3fyyNtTnsY2m8PCEGuPl86vYAMVmqqL9lXoXDEtci8mednFEKQQYva+qH6WXxs8JPfC5HROATEan29Lv0JBrmCBZS2sro8ULkaKOxbg8uzVwdeGr6v29r+3doU6WdnwFP0DXPL1dqxkGAcZoyyvxsCvu30nzr6m7V8lgJSWBob7Dm8GjVgW5r9J4pnX0P+2bLZvBfOH/t4fWMmWiUd3VkQPcKR+pddTuBtpJk290kZ4wQ4JdvCFsSKdBaNizvIH0xP0v3ruMbsMtxjvy3Vie7D95PeNV8/hUPt4D+GqPsOH44Eo2T+LfQkxwBWNveA+4s3aFDJlbkXzUPNrXlzDLLAaZVBaziFS2sS3u5FK3YA3jSyXSEoDlVEvjtTdVzRZn7DFyWrI8V/OY49Qu8R8qTviVpgIZnzlz1HnUusdQsXU9clbfRlGQn3F'), 
    ('ddlDistrict', '165'), 
    ('ddlPS', '11'), 
    ('txtRegNo', '100'), 
    ('ddlYear', '2011'), 
    ('txRegFromDt', ''), 
    ('txRegToDt', ''), 
    ('txtCompNM', ''), 
    ('btnSearch', 'Search'), 
] 

response = requests.post('http://59.180.234.21:85/index.aspx', headers=headers, data=data) 

print(response.content) 
>>> b'\r\n\r\n\r\n<!DOCTYPE html PUBLIC ...... FIR No.</a></td><td style="width:10%;"><a href="javascript:__doPostBack(&#39;DgRegist$ctl02$ctl03&#39;,&#39;&#39;)" style="color:Black;">Fir Year</a></td><td style="width:10%;">FIR Date</td><td style="width:15%;">\r\n           View FIR\r\n          </td>\r\n\t\t\t</tr><tr class="DataItemStyle ">\r\n\t\t\t\t<td>0100</td><td>2011</td><td>29-05-2011</td><td>\r\n           <input type="image" name="DgRegist$ctl03$imgDelete" id="DgRegist_ctl03_imgDelete" src="Images/print.gif" ... ... \r\n</form>\r\n</body>\r\n</html>\r\n'