Python HttpConnection请求编码错误

Question

我想使用python脚本将zip文件上传到网站。该网站完全为此提供了一个API。但是，当我尝试使用它时，在加入所有要发送httplib连接请求的字符串时出现编码错误。我追查到的字符串是filedata（我的zip文件）。

Traceback (most recent call last): 
File "/Library/Application Junk/ProjectManager/Main.py", line 146, in OnUpload CurseUploader.upload_file('77353ba57bdeb5346d1b3830ed36171279763e35', 'wow', slug, version, VersionID, 'r', logText or '', 'creole', '', 'plain', zipPath) 
File "/Library/Application Junk/ProjectManager/CurseUploader.py", line 83, in upload_file 
content_type, body = encode_multipart_formdata(params, [('file', filepath)]) 
File "/Library/Application Junk/ProjectManager/CurseUploader.py", line 153, in encode_multipart_formdata 
body = '\r\n'.join(L) 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xcb in position 10: ordinal not in range(128) 

---

编辑：按照要求，全部代码。

EDIT2：按照建议，尝试将所有非ascii字符串编码为ascii。它会抛出相同的错误，但现在在L[i] = value.encode("ascii")。

from httplib import HTTPConnection 
from os.path import basename, exists 
from mimetools import choose_boundary 

try: 
    import simplejson as json 
except ImportError: 
    import json 

def get_game_versions(game): 
    """ 
    Return the JSON response as given from /game-versions.json from curseforge.com of the given game 

`game` 
    The shortened version of the game, e.g. "wow", "war", or "rom" 
""" 
conn = HTTPConnection('%(game)s.curseforge.com' % { 'game': game }) 
conn.request("GET", '/game-versions.json') 
response = conn.getresponse() 
assert response.status == 200, "%(status)d %(reason)s from /game-versions.json" % { 'status': response.status, 'reason': response.reason } 

assert response.content_type == 'application/json' 
data = json.loads(response.read()) 

return data 

def upload_file(api_key, game, project_slug, name, game_version_ids, file_type, change_log, change_markup_type, known_caveats, caveats_markup_type, filepath): 
""" 
Upload a file to CurseForge.com on your project 

`api_key` 
    The api-key from http://www.curseforge.com/home/api-key/ 

`game` 
    The shortened version of the game, e.g. "wow", "war", or "rom" 

`project_slug` 
    The slug of your project, e.g. "my-project" 

`name` 
    The name of the file you're uploading, this should be the version's name, do not include your project's name. 

`game_version_ids` 
    A set of game version ids. 

`file_type` 
    Specify 'a' for Alpha, 'b' for Beta, and 'r' for Release. 

`change_log` 
    The change log of the file. Up to 50k characters is acceptable. 

`change_markup_type` 
    Markup type for your change log. creole or plain is recommended. 

`known_caveats` 
    The known caveats of the file. Up to 50k characters is acceptable. 

`caveats_markup_type` 
    Markup type for your known caveats. creole or plain is recommended. 

`filepath` 
    The path to the file to upload. 
""" 

assert len(api_key) == 40 
assert 1 <= len(game_version_ids) <= 3 
assert file_type in ('r', 'b', 'a') 
assert exists(filepath) 

params = [] 

params.append(('name', name)) 

for game_version_id in game_version_ids: 
    params.append(('game_version', game_version_id)) 

params.append(('file_type', file_type)) 
params.append(('change_log', change_log)) 
params.append(('change_markup_type', change_markup_type)) 
params.append(('known_caveats', known_caveats)) 
params.append(('caveats_markup_type', caveats_markup_type)) 

content_type, body = encode_multipart_formdata(params, [('file', filepath)]) 
print('Got here?') 


headers = { 
    "User-Agent": "CurseForge Uploader Script/1.0", 
    "Content-type": content_type, 
    "X-API-Key": api_key} 

conn = HTTPConnection('%(game)s.curseforge.com' % { 'game': game }) 
conn.request("POST", '/projects/%(slug)s/upload-file.json' % {'slug': project_slug}, body, headers) 
response = conn.getresponse() 
if response.status == 201: 
    print "Successfully uploaded %(name)s" % { 'name': name } 
elif response.status == 422: 
    assert response.content_type == 'application/json' 
    errors = json.loads(response.read()) 
    print "Form error with uploading %(name)s:" % { 'name': name } 
    for k, items in errors.iteritems(): 
     for item in items: 
      print " %(k)s: %(item)s" % { 'k': k, 'name': name } 
else: 
    print "Error with uploading %(name)s: %(status)d %(reason)s" % { 'name': name, 'status': response.status, 'reason': response.reason } 

def is_ascii(s): 
return all(ord(c) < 128 for c in s) 

def encode_multipart_formdata(fields, files): 
""" 
Encode data in multipart/form-data format. 

`fields` 
    A sequence of (name, value) elements for regular form fields. 

`files` 
    A sequence of (name, filename) elements for data to be uploaded as files 
Return (content_type, body) ready for httplib.HTTP instance 
""" 
boundary = choose_boundary() 
L = [] 

for key, value in fields: 
    if value is None: 
     value = '' 
    elif value is False: 
     continue 

    L.append('--%(boundary)s' % {'boundary': boundary}) 
    L.append('Content-Disposition: form-data; name="%(name)s"' % {'name': key}) 
    L.append('') 
    L.append(value) 

for key, filename in files: 
    f = file(filename, 'rb') 
    filedata = f.read() 
    f.close() 
    L.append('--%(boundary)s' % {'boundary': boundary}) 
    L.append('Content-Disposition: form-data; name="%(name)s"; filename="%(filename)s"' % { 'name': key, 'filename': basename(filename) }) 
    L.append('Content-Type: application/zip') 
    L.append('') 
    L.append(filedata) 

L.append('--%(boundary)s--' % {'boundary': boundary}) 
L.append('') 

for i in range(len(L)): 
    value = L[i] 
    if not is_ascii(value): 
     L[i] = value.encode("ascii") 

body = '\r\n'.join(L) 
content_type = 'multipart/form-data; boundary=%(boundary)s' % { 'boundary': boundary } 
return content_type, body 

我该如何解决它呢？

---

EDIT3：按照要求，印刷瓦尔的全部结果

fields: [('name', u'2.0.3'), ('game_version', u'1'), ('game_version', u'4'), ('game_version', u'9'), ('file_type', 'r'), ('change_log', u'====== 2.0.3\n* Jaliborc: Fixed a bug causing wrong items to be shown for leather, mail and plate slots\n* Jaliborc: Items are now organized by level as well\n\n====== 2.0.2\n* Jaliborc: Completly rewritten the categories dropdown to fix a bug\n\n====== 2.0.1\n* Jaliborc: Updated for patch 4.2\n* Jaliborc: Included all Firelands items\n\n===== 2.0.0\n* Jaliborc: Now works with 4.1\n* Jaliborc: Completely redesigned and improved\n* Jaliborc: Includes **all** items in-game right from the start\n* Jaliborc: Searches trough thousands of items in a blaze\n* Jaliborc: Mostly //Load on Demand//\n* Jaliborc: Only works on English clients. Versions for other clients should be released in a close future.\n\n====== 1.8.7\n* Added linkerator support for multiple chat frames\n\n====== 1.8.6\n* Fixed a bug when linking an item from the chat frame. \n\n====== 1.8.5\n* Added compatibility with WoW 3.3.5\n\n====== 1.8.3\n* Bumped TOC for 3.3\n\n====== 1.8.2\n* Bumped TOC for 3.2\n\n====== 1.8.1\n* TOC Bump + Potential WIM bugfix\n\n===== 1.8.0\n* Added "Heirloom" option to quality selector\n* Fixed a bug causing the DB to be reloaded on item scroll\n* Cleaned up the code a bit. Still need to work on the GUI/localization\n* Altered slash commands. See addon description for details.\n\n====== 1.7.2\n* Bumped the max item ID to check from 40k to 60k. Glyphs, etc, should now appear.\n\n====== 1.7.1\n* Fixed a crash issue when linking tradeskills\n\n===== 1.7.0\n* Made Wrath compatible\n* Seems to be causing a lot more CPU usage now, will investigate later.'), ('change_markup_type', 'creole'), ('known_caveats', ''), ('caveats_markup_type', 'plain')] 

files: [('file', u'/Users/Jaliborc/Desktop/Ludwig 2.0.3.zip')] 

这似乎包含一些unicode字符串。我应该将它们全部编码吗？

Answer 1

ISO-8859-1很可能不是解决您的第一个问题的方法。你需要知道，any_random_gibberish.decode('ISO-8859-1')根本不能失败。其次，我不确定为什么上传文件时需要编码 - 当然，练习的目的是在服务器上准确地重现文件;将一个zip文件解码为unicode对象似乎很奇怪。

如果你要发布你得到的错误（“读取文件时出现编码错误”）和完整的回溯，那么这将是一个非常好的主意，那么有人可以帮助你。还需要：您提到的API的URL。

更新你说你有一个“ascii错误”的行body = '\r\n'.join(L)。一个合理的猜测，根据您的有限信息，是你有这样的问题：

>>> "".join([u"foo", "\xff"]) 
Traceback (most recent call last): 
    File "<stdin>", line 1, in <module> 
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128) 

u"foo" + "\xff"产生相同的结果。

发生什么事是您有unicode和str对象的混合。连接它们需要将str对象转换为unicode，并尝试使用默认编码，通常为ascii，当str对象不是ASCII时将会失败。

在这种情况下，问题不是str对象，而是unicode对象：您只是无法发送未编码的unicode对象。

我建议你将这段代码：

for key, filename in files: 
    f = file(filename, 'r') 
    filedata = f.read().decode("ISO-8859-1") 

与此：

for key, filename in files: 
    f = file(filename, 'rb') # Specify binary mode in case this gets run on Windows 
    filedata = f.read() # don't decode it 

，并立即进入该功能后，打印其ARGS，这样你可以清楚地看到哪些是unicode：

print "fields:", repr(fields) 
print "files:", repr(files) 

这很可能是通过执行（明确）unicode_object.encode("ascii")，可以将10个对象全部安全地强制为ascii。

更新2：这是值得研究的价值的一些如何是unicode，有些是str。看来，所有的unicode可以安全地编码为ascii：

new = [(k, v.encode('ascii') if isinstance(v, unicode) else v) for k, v in original]