保存gzipped JSON导致连接重置

我有一个gzipped JSON file，我尝试下载，但是当我尝试读取请求库的响应内容时，它会重置连接。保存gzipped JSON导致连接重置

data = requests.request("GET", i, stream=True) 
with gzip.open(i.rsplit("/")[-1], "wb") as fh: 
    for chunk in data.iter_content(chunk_size=1024): 
    fh.write(chunk)

这里是我所得到的，当我尝试阅读的内容：

Traceback (most recent call last): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 228, in _error_catcher 
    yield 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 501, in read_chunked 
    chunk = self._handle_chunk(amt) 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 461, in _handle_chunk 
    value = self._fp._safe_read(amt) 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\http\client.py", line 592, in _safe_read 
    chunk = self.fp.read(min(amt, MAXAMOUNT)) 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\socket.py", line 575, in readinto 
    return self._sock.recv_into(b) 
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\models.py", line 664, in generate 
    for chunk in self.raw.stream(chunk_size, decode_content=True): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 349, in stream 
    for line in self.read_chunked(amt, decode_content=decode_content): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 526, in read_chunked 
    self._original_response.close() 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\contextlib.py", line 77, in __exit__ 
    self.gen.throw(type, value, traceback) 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 246, in _error_catcher 
    raise ProtocolError('Connection broken: %r' % e, e) 
requests.packages.urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "H:/Programming/Python/warehouse/main.py", line 55, in <module> 
    compile_auctions(slugs) 
    File "H:/Programming/Python/warehouse/main.py", line 44, in compile_auctions 
    for chunk in data.iter_content(chunk_size=1024): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\models.py", line 667, in generate 
    raise ChunkedEncodingError(e) 
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

我可以在我的浏览器，在邮差阅读JSON，所以我不知道我在做什么错误。我在Windows 10上使用Python 3.5.1。

这是保存gzip JSON的正确方法吗？

来源

2016-06-12 mxplusb

请求已经解压缩数据。

你并不需要用gzip：

import requests 

req = requests.get("http://auction-api-us.worldofwarcraft.com/auction-data/4923213e5eb3ec3b7e03d22b632bda36/auctions.json", stream=True) 

with open("out.json", "wb") as f: 
    for chunk in req.iter_content(chunk_size=4096): 
     f.write(chunk)

来源

2016-06-12 15:53:47

在我的机器上，按预期工作。主机是否因为太多请求而将您列入黑名单？编辑：这是一个回复删除评论 –

事实证明，这个问题是别的东西，而不是我的代码。不过谢谢！ – mxplusb

@mynameismevin你能透露是什么问题？我正在努力解决同样的错误。 – Adam

所以，问题不在于实际上我使用的API，它要么与请求库for Windows或问题与Python 3.5的错误.1在Windows 10上。

当我使用Python 3.5.1在Ubuntu 16.04上运行这个确切的代码时，它工作得很好。

来源

2016-06-13 15:03:35 mxplusb

保存gzipped JSON导致连接重置

回答

相关问题