python3 URLError未知的url类型http

Question

我想使用urllib.request.urlretrieve与多处理模块一起下载一些文件并对它们进行一些处理。然而，每一个我尝试运行我的程序时，它给我的错误：

multiprocessing.pool.RemoteTraceback: 
""" 
Traceback (most recent call last): 
    File "/usr/lib/python3.4/multiprocessing/pool.py", line 119, in worker 
    result = (True, func(*args, **kwds)) 
    File "/usr/lib/python3.4/multiprocessing/pool.py", line 44, in mapstar 
    return list(map(*args)) 
    File "./thumb.py", line 13, in download_and_convert 
    filename, headers = urlretrieve(url) 
    File "/usr/lib/python3.4/urllib/request.py", line 186, in urlretrieve 
    with contextlib.closing(urlopen(url, data)) as fp: 
    File "/usr/lib/python3.4/urllib/request.py", line 161, in urlopen 
    return opener.open(url, data, timeout) 
    File "/usr/lib/python3.4/urllib/request.py", line 463, in open 
    response = self._open(req, data) 
    File "/usr/lib/python3.4/urllib/request.py", line 486, in _open 
    'unknown_open', req) 
    File "/usr/lib/python3.4/urllib/request.py", line 441, in _call_chain 
    result = func(*args) 
    File "/usr/lib/python3.4/urllib/request.py", line 1252, in unknown_open 
    raise URLError('unknown url type: %s' % type) 
urllib.error.URLError: <urlopen error unknown url type: http> 
""" 

The above exception was the direct cause of the following exception: 

Traceback (most recent call last): 
    File "./thumb.py", line 27, in <module> 
    pool.map(download_and_convert, enumerate(csvr)) 
    File "/usr/lib/python3.4/multiprocessing/pool.py", line 260, in map 
    return self._map_async(func, iterable, mapstar, chunksize).get() 
    File "/usr/lib/python3.4/multiprocessing/pool.py", line 599, in get 
    raise self._value 
urllib.error.URLError: <urlopen error unknown url type: http> 

，它似乎被噎死的URL是http://phytoimages.siu.edu/users/vitt/10_27_06_2/Equisetumarvense.JPG。这里是我的代码：

#!/usr/bin/env python3 

from subprocess import Popen 
from sys import argv, stdin 
import csv 
from multiprocessing import Pool 
from urllib.request import urlretrieve 

def download_and_convert(args): 
    num, url_list = args 
    url = url_list[0] 
    try: 
     filename, headers = urlretrieve(url) 
    except: 
     print(url) 
     raise 
    Popen(["convert", filename, "-resize", "250x250",\ 
      str(num)+'.'+url.split('.')[-1]]) 

if __name__ == "__main__": 
    csvr = csv.reader(open(argv[1])) 

    if(len(argv) > 2): nprocs = argv[2] 
    else: nprocs = None 

    pool = Pool(processes=nprocs) 
    pool.map(download_and_convert, enumerate(csvr)) 

我不知道为什么这个错误发生。难道是因为我正在使用多处理？如果任何人都可以帮助我，那将非常感激。

编辑：这是它尝试处理的第一个url，如果我更改它并不会更改错误。

Answer 1

经过一些评论的帮助，我找到了解决方案。看起来问题在于csv模块在字节顺序标记（BOM）上跳动。我可以通过encoding='utf-8-sig'作为建议here打开文件来修复它。

Answer 2

检查这段代码

>>> import urllib.parse 
>>> urllib.parse.quote(':') 
'%3A' 

正如你所看到的urllib解释：奇怪字符 ''。巧合的是你的程序挂在你身上的地方。

尝试urllib.parse.urlencode()应该把你放在正确的轨道上。