2012-06-15 132 views
2

我正在使用Google Cloud Storage Java API从Google App Engine应用程序读取存储在Google Cloud Storage存储区中的zip文件。 它适用于小文件,但是当我尝试读取我需要的实际文件时(25Mb压缩),我得到一个ApplicationError:10异常。测试完成在GAE上部署的应用程序。ApplicationError:10使用Google Cloud Storage Java API读取大型zip文件

我可能已经明白ApplicationError:10在文件打开时间太长(+30s以上)时抛出,请参阅:https://groups.google.com/forum/?fromgroups#!topic/google-appengine-java/5XBVaVDKvvs,但我仍然在努力让这件事情奏效。

感谢您的帮助。

还有就是我所采用的测试代码:

FileService fileService = FileServiceFactory.getFileService(); 
AppEngineFile readableFile = new AppEngineFile("/gs/xxxx-my-bucket/xxx-bigfile.zip"); 
FileReadChannel readChannel = fileService.openReadChannel(readableFile, false); 
ZipInputStream zis = new ZipInputStream(Channels.newInputStream(readChannel)); 
ZipEntry ze = null; 
while ((ze = zis.getNextEntry()) != null) { 
    resp.getOutputStream().println("Unzipping " + ze.getName()); 
    for (int c = zis.read(); c != -1; c = zis.read()) { 
     resp.getOutputStream().print((char) c); 
    } 
    zis.closeEntry(); 
} 
zis.close(); 

我得到的堆栈跟踪是:

java.io.IOException 
    at com.google.appengine.api.files.FileServiceImpl.translateException(FileServiceImpl.java:617) 
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:590) 
    at com.google.appengine.api.files.FileServiceImpl.read(FileServiceImpl.java:539) 
    at com.google.appengine.api.files.FileServiceImpl.read(FileServiceImpl.java:410) 
    at com.google.appengine.api.files.FileReadChannelImpl.read(FileReadChannelImpl.java:73) 
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:65) 
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:109) 
    at sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:103) 
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:235) 
    at java.io.BufferedInputStream.read1(BufferedInputStream.java:275) 
    at java.io.BufferedInputStream.read(BufferedInputStream.java:334) 
    at java.io.FilterInputStream.read(FilterInputStream.java:133) 
    at java.io.PushbackInputStream.read(PushbackInputStream.java:186) 
    at java.util.zip.InflaterInputStream.fill(InflaterInputStream.java:238) 
    at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:158) 
    at java.util.zip.ZipInputStream.read(ZipInputStream.java:163) 
    at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:122) 
    at fr.heliorama.skylinewebapp.servlets.TestGSZip.doGet(TestGSZip.java:30) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:617) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:717) 
    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1166) 
    at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:102) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) 
    at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) 
    at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) 
    at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) 
    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) 
    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) 
    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) 
    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) 
    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) 
    at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:249) 
    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) 
    at org.mortbay.jetty.Server.handle(Server.java:326) 
    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) 
    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923) 
    at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76) 
    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) 
    at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:135) 
    at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:477) 
    at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:449) 
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455) 
    at com.google.tracing.TraceContext.runInContext(TraceContext.java:695) 
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333) 
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325) 
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453) 
    at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251) 
    at java.lang.Thread.run(Thread.java:679) 
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 10: 
    at java.lang.Thread.getStackTrace(Thread.java:1495) 
    at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java:237) 
    at com.google.apphosting.runtime.ApiProxyImpl.access$000(ApiProxyImpl.java:68) 
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:182) 
    at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java:180) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:180) 
    at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java:68) 
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:101) 
    at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:50) 
    at com.google.appengine.api.files.FileServiceImpl.makeSyncCall(FileServiceImpl.java:586) 
    ... 48 more 
Caused by: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 10: 
    at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.failure(ApiProxyImpl.java:546) 
    at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher$1.runInContext(RpcStub.java:784) 
    at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455) 
    at com.google.tracing.TraceContext.runInContext(TraceContext.java:695) 
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333) 
    at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325) 
    at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453) 
    at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.rpcFinished(RpcStub.java:824) 
    at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher.failure(RpcStub.java:815) 
    at com.google.net.rpc3.impl.client.RpcClientInternalContext.runCallbacks(RpcClientInternalContext.java:895) 
    at com.google.net.rpc3.impl.client.RpcClientInternalContext.finishRpcAndNotifyApp(RpcClientInternalContext.java:798) 
    at com.google.net.rpc3.impl.client.RpcNetChannel.afterFinishingActiveRpc(RpcNetChannel.java:1054) 
    at com.google.net.rpc3.impl.client.RpcNetChannel.finishRpc(RpcNetChannel.java:902) 
    at com.google.net.rpc3.impl.client.RpcNetChannel.handleResponse(RpcNetChannel.java:2250) 
    at com.google.net.rpc3.impl.client.RpcNetChannel.messageReceived(RpcNetChannel.java:2057) 
    at com.google.net.rpc3.impl.client.RpcNetChannel.access$2000(RpcNetChannel.java:143) 
    at com.google.net.rpc3.impl.client.RpcNetChannel$TransportCallback.receivedMessage(RpcNetChannel.java:3112) 
    at com.google.net.rpc3.impl.client.RpcChannelTransportData$TransportCallback.receivedMessage(RpcChannelTransportData.java:599) 
    at com.google.net.rpc3.impl.wire.RpcBaseTransport.receivedMessage(RpcBaseTransport.java:417) 
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport.parseOneMessage(RpcClientTcpTransport.java:750) 
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport.parseMessages(RpcClientTcpTransport.java:634) 
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport.access$100(RpcClientTcpTransport.java:37) 
    at com.google.net.rpc3.impl.wire.RpcClientTcpTransport$1.dataReceived(RpcClientTcpTransport.java:295) 
    at com.google.net.async3.SocketConnection.handleNetworkReadEvent(SocketConnection.java:900) 
    at com.google.net.async3.SocketConnection.access$400(SocketConnection.java:43) 
    at com.google.net.async3.SocketConnection$NetworkReadHandlerImpl.run(SocketConnection.java:929) 
    at com.google.net.eventmanager.AbstractFutureTask$Sync.innerRun(AbstractFutureTask.java:260) 
    at com.google.net.eventmanager.AbstractFutureTask.run(AbstractFutureTask.java:121) 
    at com.google.net.eventmanager.EventManagerImpl.runTask(EventManagerImpl.java:575) 
    at com.google.net.eventmanager.EventManagerImpl.internalRunWorkerLoop(EventManagerImpl.java:999) 
    at com.google.net.eventmanager.EventManagerImpl.runWorkerLoop(EventManagerImpl.java:881) 
    at com.google.net.eventmanager.WorkerThreadInfo.runWorkerLoop(WorkerThreadInf...(length 8282) 
+0

我没有任何出色的想法,但也许你可以尝试下载文件到本地计算机,而不用解压缩它。然后您可以解压缩本地副本。也许这将摆脱时间问题,如果这确实是问题,或至少帮助你更好地确定问题实际上是什么。 –

+0

文件是否打开超过30秒? –

+0

@霍华德:好主意,我会尝试读解压缩的数据,就像二进制流,我们会看看它做了什么。顺便说一句,读者在GAE上,所以我不能把它下载到我的本地计算机上。 –

回答

2

OK,与谷歌应用程序引擎和谷歌云店工作时,你必然要数时间限制:

  • 30s在GS上读取文件。
  • 使用GAE URLFetch服务时的10秒
  • 对于GAE上的每个请求60秒。

在我的特殊情况下,我无法解决这些限制。

+0

嗨加布里埃尔,这些限制在任何地方指定?这很糟糕,因为我试图通过在任务队列限制(10分钟)内写入大文件块来节省内存,而且我不能这么做 – ZiglioUK

相关问题