我有类似这样的帖子在多个环境问题(即不同的Redis Azure的实例):ServiceStack.Redis: Unable to Connect: sPort:与ServiceStack.Redis连接问题Azure的Redis的
但当然,我们不能移动或改变Redis的服务器,因为我们依赖于Azure Redis。如果是延迟问题,我们可能会被搞砸......
我们使用旧版本的SS(4.0.42.0),并且已经更新到最新版本(4.0.56.0),并且看到了相同的间歇性问题。
这里是一些背景: - 问题只出现在至少2K的请求(有时或多或少)之后。是的,我们正在使用最新的SS许可证。 - 这是非常间歇性的,大多数请求都是成功的,但失败的小失败通常会失败(1-5个左右),然后问题会消失一段时间 - 我尝试了RedisPoolManager,PooledRedisClientManager获得了相同的结果。 - 我已经为每个请求做了一个客户端统计报告,并且确保该池包含足够的客户端,没有出现错误等。我很少看到超过40个客户端在使用时间为40个。 -
不同例外: - IOException
与消息Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host
和堆栈跟踪,其中包括提及RedisClient。以下是完整的错误转储:
"exception": { "message": "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.", "source": "System", "targetSite": "Int32 Read(Byte[], Int32, Int32)", "stackTrace": " at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)\r\n at System.Net.FixedSizeReader.ReadPacket(Byte[] buffer, Int32 offset, Int32 count)\r\n at System.Net.Security._SslStream.StartFrameHeader(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)\r\n at System.Net.Security._SslStream.StartReading(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)\r\n at System.Net.Security._SslStream.ProcessRead(Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest)\r\n at System.Net.Security.SslStream.Read(Byte[] buffer, Int32 offset, Int32 count)\r\n at System.IO.BufferedStream.ReadByte()\r\n at ServiceStack.Redis.RedisNativeClient.ReadLine()\r\n at ServiceStack.Redis.RedisNativeClient.ReadData()\r\n at ServiceStack.Redis.RedisClient.<>c__DisplayClass1c
1.b__1b(RedisClient R)\ r \在ServiceStack.Redis.RedisClient.Exec [T](Func键ñ2 action)\r\n at ServiceStack.Redis.RedisClientManagerCacheClient.Get[T](String key)\r\n at API.ServiceInterface.RequestExtensions.GetUserSession(IRequest req, Boolean createIfNotExists) in F:\\src\\CCCAPI CD (DevLab)\\ServiceInterface\\Extensions\\RequestExtensions.cs:line 26\r\n at API.WebHost.AuthImpl.HandleBlacklistedUserSessions(IRequest req, IResponse httpResponse) in F:\\src\\CCCAPI CD (DevLab)\\WebHost\\Authentication\\AuthImpl.cs:line 30\r\n at ServiceStack.ServiceStackHost.ApplyPreRequestFilters(IRequest httpReq, IResponse httpRes)\r\n at ServiceStack.Host.RestHandler.ProcessRequestAsync(IRequest httpReq, IResponse httpRes, String operationName)", "type": "IOException", "innerException": { "message": "An existing connection was forcibly closed by the remote host", "source": "System", "targetSite": "Int32 Read(Byte[], Int32, Int32)", "stackTrace": " at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)", "type": "SocketException" } }
我们看到-Another例外是例外类型ServiceStack.Redis
与消息Unable to Connect: sPort: 50447
(这里有趣的是端口改变了,而且从来不是应该使用的真正的Azure Redis SSL端口,似乎池管理器可能无法将正确的端口传递给此客户端?)。下面是完整的转储: "exception": { "message": "Unable to Connect: sPort: 50447", "source": "ServiceStack.Redis", "targetSite": "ServiceStack.Redis.RedisException CreateConnectionError()", "stackTrace": " at ServiceStack.Redis.RedisNativeClient.CreateConnectionError()\r\n at ServiceStack.Redis.RedisNativeClient.SendExpectData(Byte[][] cmdWithBinaryArgs)\r\n at ServiceStack.Redis.RedisClient.<>c__DisplayClass1c
1.b__1b(RedisClient R)\ r \在ServiceStack.Redis.RedisClient.Exec [T](Func键2 action)\r\n at ServiceStack.Redis.RedisClientManagerCacheClient.Get[T](String key)\r\n at API.ServiceInterface.RequestExtensions.GetUserSession(IRequest req, Boolean createIfNotExists) in F:\\src\\CCCAPI CD (DevLab)\\ServiceInterface\\Extensions\\RequestExtensions.cs:line 26\r\n at API.WebHost.AuthImpl.HandleBlacklistedUserSessions(IRequest req, IResponse httpResponse) in F:\\src\\CCCAPI CD (DevLab)\\WebHost\\Authentication\\AuthImpl.cs:line 30\r\n at ServiceStack.ServiceStackHost.ApplyPreRequestFilters(IRequest httpReq, IResponse httpRes)\r\n at ServiceStack.Host.RestHandler.ProcessRequestAsync(IRequest httpReq, IResponse httpRes, String operationName)", "type": "RedisException", "innerException": { "message": "An existing connection was forcibly closed by the remote host", "source": "System", "targetSite": "Void Write(Byte[], Int32, Int32)", "stackTrace": " at System.Net.Sockets.NetworkStream.Write(Byte[] buffer, Int32 offset, Int32 size)", "type": "SocketException" }
林n,其中这一个挣扎... 任何帮助,将不胜感激。
不是我曾经希望的,因为我们对Azure太过hand - - 我将Auto Retry超时时间增加到了一个荒谬的值(20s),并且希望这足以缓解这些问题的影响。如果它确实是服务器端问题,我觉得很奇怪,没有完成报告来提醒我们存在一些限制/错误/加载问题。我们已经尝试增加Azure Redis实例的容量,远远超出我们所需的范围 - 没有运气。 谢谢你的明确解释。我们将考虑向Azure申请一张票.. – jglassco
A跟进此问题:我向Azure提交了支持凭单 - 他们承认Azure Redis和ServiceStack客户端存在问题。缓存支持团队手动修补了我们的redis实例,问题消失了。 我想他们会在不久的将来自动包含这个补丁,但现在我们必须要求它(对于标准实例,Premium显然已经有补丁)。 – jglassco
@jglassco对于后续行动来说很棒的thx。我假设受管理的Redis服务正在运行他们的Windows端口,因为Redis在Linux上运行得最好,所以这并不理想。 – mythz