Python的网页抓取 - UnicodeEncodeError

Question

我有一个代码从网站获取链接。问题很少。该代码在第二次运行时会生成正确的输出。第一次没有输出。第二我得到一个错误。

错误消息：

traceback (most recent call last): 
     File "C:\Users\dell\AppData\Local\Programs\Python\Python35\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py", line 326, in RunScript 
     exec(codeObject, __main__.__dict__) 
     File "C:\Users\dell\AppData\Local\Programs\Python\Python35\ld.py", line 25, in <module> 
     wr.writerows([tag.text]) 
     File "C:\Users\dell\AppData\Local\Programs\Python\Python35\lib\encodings\cp1252.py", line 19, in encode 
     return codecs.charmap_encode(input,self.errors,encoding_table)[0] 
    UnicodeEncodeError: 'charmap' codec can't encode character '\x91' in position 410: character maps to <undefined> 

---

代码：

import csv 
import urllib.parse 
import urllib.request 
from bs4 import BeautifulSoup 
url = "http://www.dhr.gov.in/" 
urls = [url] 
visited = [url] 
resultFile = open('t5.csv','w', newline ='') 
wr = csv.writer(resultFile, delimiter = ' ') 
while len(urls)>0: 
    try: 
     htmltext = urllib.request.urlopen(urls[0]).read() 
    except: 
     pass  
    soup = BeautifulSoup(htmltext) 
    urls.pop(0) 
    for tag in soup.findAll('a', href = True): 
     tag['href'] = urllib.parse.urljoin(url,tag['href']) 
     if url in tag['href']and tag['href'] not in visited: 
      urls.append(tag['href']) 
      visited.append(tag['href']) 
      print([tag['href']]) 
      print([tag.text]) 
      wr.writerows([tag['href']]) 
      wr.writerows([tag.text]) 
resultFile.close() 

Answer 1

有是如何摆脱一些黑客。但更好地使用python 3，错误会自行消失。