在WildFly 8.2.1-Final高容量安装中,我们在大约一周内遇到降级性能问题,直到需要重新启动为止,并且因为发生JMS消息而出现越来越多的故障在状态ABORT_ONLY交易:WildFly - HornetQ连接超时和性能问题
Caused by: javax.ejb.EJBException: javax.jms.JMSException: Could not create a session: IJ000457: Unchecked throwable in managedConnectionReconnected()
Caused by: javax.resource.ResourceException: IJ000457: Unchecked throwable in managedConnectionReconnected() cl=org[email protected]3915e409[state=NORMAL managed [email protected] connection handles=0 lastUse=1494579251303 trackByTx=false po[email protected]691085ec [email protected][pool=HornetQConnectionDefinition] [email protected][xaResource=[email protected] pad=false overrideRmValue=null productName=HornetQ productVersion=2.0 jndiName=java:/JmsXA] txSync=null]
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:780)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:518)
at org.hornetq.ra.HornetQRASessionFactoryImpl.allocateConnection(HornetQRASessionFactoryImpl.java:948)
... 261 more
Caused by: javax.resource.ResourceException: IJ000461: Could not enlist in transaction on entering meta-aware object
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:551)
at org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:775)
... 263 more
Caused by: javax.transaction.SystemException: IJ000356: Failed to enlist: java.lang.Throwable: Unabled to enlist resource, see the previous warnings. tx=TransactionImple < ac, BasicAction: 0:ffff0a0001c3:682851c6:5900737b:dd09939 status: ActionStatus.ABORT_ONLY >
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.checkEnlisted(TxConnectionListener.java:848)
at org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:383)
at org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:544)
... 264 more
之所以事务处于状态ABORT_ONLY是以前记录的一个警告:
2017-05-30 11:33:15,977 WARN [com.arjuna.ats.jta] (Thread-123) ARJUNA016086: TransactionImple.enlistResource setTransactionTimeout on XAResource < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a0001c3:-7a698606:591fe1ff:82cea51, node_name=1, branch_uid=0:ffff0a0001c3:-7a698606:591fe1ff:830e516, subordinatenodename=null, eis_name=java:/JmsXA > threw: XAException.XAER_RMERR: javax.transaction.xa.XAException
Caused by: HornetQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=HQ119014: Timed out waiting for response when sending packet 51]
at org.hornetq.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:390)
at org.hornetq.core.client.impl.ClientSessionImpl.start(ClientSessionImpl.java:1901)
一些细节:
- 两种分组类型得到的超时是51(
SESS_XA_START
)和63(SESS_XA_SET_TIMEOUT
) - 在交易唯一的其他参与者是Oracle XA连接
- 现有0队列和1(耐久)话题邮件被从耗尽(消息计数处于或略高于0)
而相关的业务方法与@Lock(LockType.READ)
注释的消息从一个单EJB发送的,因此它可以同时被调用。 的EJB喷射java:/JmsXA
TopicConnectionFactory
和Topic
并创建关于该方法的每次调用一个新的TopicConnection
,Session
和MessageProducer
(被发送的每个消息),并负责关闭它们全部在一个finally {}
块。
由于超时发生发送数据包51(SESS_XA_START
)和63(SESS_XA_SET_TIMEOUT
)似乎开始会话很慢或挂起。
任何想法可能是什么原因?