AWS S3 transferutility上传从未上传但显示完整无错

Question

我收到了很多未上传到我的AWS存储桶的文件，但下面的代码始终显示完整。任何人都知道这是怎么回事？我应该如何查明上传是否失败，如果它总是返回完成？我确实看到了有关使用transferutility.EndUpload的一些信息，但我不确定如何使用它。另外我将如何实施重试？将对象状态传递给BeginUpload？任何帮助？

public class S3Upload 
{ 
    private string awsAccessKeyId = "XXXXXX"; 
    private string awsSecretAccessKey = "XXXXXX"; 
    private string bucketName = System.Configuration.ConfigurationManager.AppSettings.Get("BucketName"); 
    private Amazon.S3.Transfer.TransferUtility transferUtility; 
    private static log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType); 

    public S3Upload() 
    { 
     // Initialize log4net. 
     log4net.Config.XmlConfigurator.Configure(); 
     this.transferUtility = new Amazon.S3.Transfer.TransferUtility(awsAccessKeyId, awsSecretAccessKey); 
     log.Info("S3 instance initiated"); 

    } 

    public void UploadFile(string filePath, string toPath) 
    { 

     try 
     { 
      AsyncCallback callback = new AsyncCallback(uploadComplete); 

      log.Info("S3 upload started..."); 
      log.InfoFormat("S3 filePath: {0}", filePath); 
      log.InfoFormat("S3 toPath: {0}", toPath); 


      var uploadRequest = new Amazon.S3.Transfer.TransferUtilityUploadRequest(); 
      uploadRequest.FilePath = filePath; 
      uploadRequest.BucketName = bucketName; 
      uploadRequest.Key = toPath; 
      uploadRequest.StorageClass = Amazon.S3.Model.S3StorageClass.ReducedRedundancy; 
      uploadRequest.AddHeader("x-amz-acl", "public-read"); 
      transferUtility.BeginUpload(uploadRequest, callback, null); 
     } 
     catch (AmazonS3Exception amazonS3Exception) 
     { 
       log.ErrorFormat("An Error, number {0}, occurred when creating a bucket with the message '{1}", amazonS3Exception.ErrorCode, amazonS3Exception.Message);  
     } 
    } 

    private void uploadComplete(IAsyncResult result) 
    { 
     var x = result; 

     if (x.IsCompleted) 
     { 
      log.Info("S3 upload completed..."); 

     } 
    } 
} 

Answer 1

找到了解决办法！我添加了transferUtility.EndUpload（ar），如果它发生错误，它会将异常写入我的日志并重试放入请求。

发出请求PutObject时出错。 System.IO.IOException：无法将数据写入传输连接：现有连接被远程主机强制关闭。 --->

public class S3Upload 
{ 
    private string awsAccessKeyId = "XXXXX"; 
    private string awsSecretAccessKey = "XXXXX"; 
    private string bucketName = System.Configuration.ConfigurationManager.AppSettings.Get("BucketName"); 
    private Amazon.S3.Transfer.TransferUtility transferUtility; 
    private static log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType); 

    public S3Upload() 
    { 
     // Initialize log4net. 
     log4net.Config.XmlConfigurator.Configure(); 
     this.transferUtility = new Amazon.S3.Transfer.TransferUtility(awsAccessKeyId, awsSecretAccessKey); 
     log.Info("S3 instance initiated"); 

    } 

    public void UploadFile(string filePath, string toPath) 
    { 

     try 
     { 
      AsyncCallback callback = new AsyncCallback(uploadComplete); 

      log.Info("S3 upload started..."); 
      log.InfoFormat("S3 filePath: {0}", filePath); 
      log.InfoFormat("S3 toPath: {0}", toPath); 


      var uploadRequest = new Amazon.S3.Transfer.TransferUtilityUploadRequest(); 
      uploadRequest.FilePath = filePath; 
      uploadRequest.BucketName = bucketName; 
      uploadRequest.Key = toPath; 
      uploadRequest.StorageClass = Amazon.S3.Model.S3StorageClass.ReducedRedundancy; 
      uploadRequest.AddHeader("x-amz-acl", "public-read"); 
      IAsyncResult ar = transferUtility.BeginUpload(uploadRequest, callback, null); 
      transferUtility.EndUpload(ar); 
     } 
     catch (AmazonS3Exception amazonS3Exception) 
     { 
       log.ErrorFormat("An Error, number {0}, occurred when creating a bucket with the message '{1}", amazonS3Exception.ErrorCode, amazonS3Exception.Message); 
     } 
    } 

    private void uploadComplete(IAsyncResult result) 
    { 
     var x = result; 
    } 
} 

Answer 2

BeginUpload函数启动一个新的线程来启动并执行上载。调用EndUpload会导致当前线程阻塞，直到上传完成。此外，EndUpload函数捕获您将从上载中获得的任何异常。

如果您要调用BeginUpload，则需要从单独的线程调用EndUpload（），以便不会阻塞主线程。如果您打算阻止您的主线程，只需调用Upload（）函数而不是APM版本。

例如，产生一个新的线程，将坐等待上传完成。 （请注意，它没有做任何具体的，正好赶上了错误。）

public void UploadFile(string filePath, string toPath) 
{ 

    try 
    { 
     AsyncCallback callback = new AsyncCallback(uploadComplete); 

     log.Info("S3 upload started..."); 
     log.InfoFormat("S3 filePath: {0}", filePath); 
     log.InfoFormat("S3 toPath: {0}", toPath); 


     var uploadRequest = new Amazon.S3.Transfer.TransferUtilityUploadRequest(); 
     uploadRequest.FilePath = filePath; 
     uploadRequest.BucketName = bucketName; 
     uploadRequest.Key = toPath; 
     uploadRequest.StorageClass = Amazon.S3.Model.S3StorageClass.ReducedRedundancy; 
     uploadRequest.AddHeader("x-amz-acl", "public-read"); 
     IAsyncResult ar = transferUtility.BeginUpload(uploadRequest, callback, null); 
     // transferUtility.EndUpload(ar); 
     ThreadPool.QueueUserWorkItem(c => 
     { 
      try 
      { 
       transferUtility.EndUpload(ar); 
      } 
      catch (AmazonS3Exception ex) 
      { 
       // Retry, log the error, etc. 
      } 
      catch (WebException ex) 
      { 
       // Retry, log the error, etc. 
      } 
     }); 
    } 
    catch (AmazonS3Exception amazonS3Exception) 
    { 
      log.ErrorFormat("An Error, number {0}, occurred when creating a bucket with the message '{1}", amazonS3Exception.ErrorCode, amazonS3Exception.Message); 
    } 
} 

_{另外，该线程检查引发WebException。我不知道你是否可以从Beginupload中获得其中的一个，但是我正在处理的一些代码似乎注意到了这些。 （更安全，然后对不起。）}

此外，我发现你可以用CurrPorts Utility测试这个。该实用程序可让您将开放端​​口杀死为亚马逊（尽管它将重试直到其配置的MaxRetries）。