EHCache3内存不足错误

我们希望在我们的springboot微服务中使用EHCache3来缓存二进制对象（PDF和一些元数据）。这些对象的大小在50K到12M之间。使用简单的测试，我可以加载缓存并查看它的工作情况。EHCache3内存不足错误

缓存大型PDF时会发生该问题。下面的堆栈跟踪显示

2017-02-03 21:08:47.936 INFO 11 --- [http-nio-8080-exec-4] .e.c.i.r.LoggingRobustResilienceStrategy : Ehcache key 5dd12ae803ea06c4d37afe949090b934 recovered from 

org.ehcache.core.spi.store.StoreAccessException: java.lang.reflect.UndeclaredThrowableException 
    at org.ehcache.core.exceptions.StorePassThroughException.handleRuntimeException(StorePassThroughException.java:72) 
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1098) 
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.put(AbstractOffHeapStore.java:306) 
    at org.ehcache.impl.internal.store.tiering.TieredStore.put(TieredStore.java:149) 
    at org.ehcache.core.Ehcache.put(Ehcache.java:196) 
    at com.pany.PdfGeneratorController.convertHtmlToPdf(PdfGeneratorController.java:79) 
    at com.pany.microservice.pdfgenerator.endpoints.PdfGeneratorEndpoint.convertHtmlToPdfUsingPOST(PdfGeneratorEndpoint.java:23) 
    at com.pany.microservice.pdfgenerator.api.PdfgeneratorcontrollerSpringEndpoint.convertHtmlToPdfUsingPOSTINNER(PdfgeneratorcontrollerSpringEndpoint.java:100) 
    at com.pany.microservice.pdfgenerator.api.PdfgeneratorcontrollerSpringEndpoint.convertHtmlToPdfUsingPOSTPOST(PdfgeneratorcontrollerSpringEndpoint.java:67) 
    at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at java.lang.reflect.Method.invoke(Method.java:498) 
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:220) 
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134) 
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) 
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) 
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) 
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) 
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) 
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) 
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) 
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) 
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:105) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:89) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:107) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:117) 
    at org.springframework.boot.web.support.ErrorPageFilter.access$000(ErrorPageFilter.java:61) 
    at org.springframework.boot.web.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:92) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:110) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) 
    at org.apache.catalina.core.StandardContextValve.__invoke(StandardContextValve.java:108) 
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java) 
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) 
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) 
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) 
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) 
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) 
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349) 
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783) 
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) 
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789) 
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) 
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.reflect.UndeclaredThrowableException: null 
    at com.sun.proxy.$Proxy122.encode(Unknown Source) 
    at org.terracotta.offheapstore.storage.PortabilityBasedStorageEngine.writeMapping(PortabilityBasedStorageEngine.java:64) 
    at org.terracotta.offheapstore.OffHeapHashMap.tryWriteEntry(OffHeapHashMap.java:703) 
    at org.terracotta.offheapstore.OffHeapHashMap.writeEntry(OffHeapHashMap.java:687) 
    at org.terracotta.offheapstore.OffHeapHashMap.computeWithMetadata(OffHeapHashMap.java:1947) 
    at org.terracotta.offheapstore.AbstractLockedOffHeapHashMap.computeWithMetadata(AbstractLockedOffHeapHashMap.java:582) 
    at org.terracotta.offheapstore.concurrent.AbstractConcurrentOffHeapMap.computeWithMetadata(AbstractConcurrentOffHeapMap.java:733) 
    at org.ehcache.impl.internal.store.disk.EhcachePersistentConcurrentOffHeapClockCache.compute(EhcachePersistentConcurrentOffHeapClockCache.java:158) 
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1083) 
    ... 83 common frames omitted 
Caused by: java.lang.reflect.InvocationTargetException: null 
    at sun.reflect.GeneratedMethodAccessor136.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at java.lang.reflect.Method.invoke(Method.java:498) 
    at org.ehcache.impl.internal.store.disk.OffHeapDiskStore$1.invoke(OffHeapDiskStore.java:454) 
    ... 92 common frames omitted 
Caused by: java.lang.OutOfMemoryError: Java heap space

我怀疑问题在于我使用了缓存的配置，但我无法弄清楚。我在堆和磁盘存储中使用约束值来测试缓存满时的正确驱逐。通常我会希望新内容从缓存中推出较旧的内容，并降低性能。但是现在我正在摆脱我无法解释的内存异常。有没有其他的东西要了解ehcache的分层？

<config 
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' 
    xmlns='http://www.ehcache.org/v3' 
    xmlns:jsr107='http://www.ehcache.org/v3/jsr107' 
    xsi:schemaLocation=" 
     http://www.ehcache.org/v3 http://www.ehcache.org/schema/ehcache-core-3.1.xsd 
     http://www.ehcache.org/v3/jsr107 http://www.ehcache.org/schema/ehcache-107-ext-3.1.xsd"> 

    <persistence directory="/root/cachexml"/> 
    <heap-store> 
     <max-object-size unit="MB">20</max-object-size> 
    </heap-store> 

    <cache alias="pdfCache" > 
     <key-type>java.lang.String</key-type> 
     <value-type>com.acquisio.microservice.pdfgenerator.msto.PDFTO</value-type> 
     <expiry> 
      <tti unit="hours">12</tti> 
     </expiry> 

     <resources> 
      <heap unit="entries">4</heap> 
      <disk persistent="true" unit="MB">64</disk> 
     </resources> 
    </cache> 

</config>

来源

2017-02-03 Yan Periard

Ehcache3的堆缓存[不正确支持]（https://groups.google.com/d/msg/ehcache-users/8Mu4LkDOmnc/sYvvgn37CAAJ）具有不同大小的条目，而不是在驱逐之下。这听起来不像是有计划解决它。 –

@BenManes这确实看起来像我看到的问题。你知道解决这个问题的方法吗？ –

Ehcache2是一款非常优秀的产品，我认为它仍然受到团队的支持。由于我在[备用缓存]（https://github.com/ben-manes/caffeine）上工作，我对Ehcache3的经验太有限，无法提供建议。 –

看着stacktrace和Ehcache 3代码 - 大概就像你没有给出确切的版本 - 看起来OOME发生在值序列化过程中，所以它可以存储在磁盘上。

当您在Ehcache 3中的多层缓存中放入一个值时，该值总是写入最低层 - 磁盘。但是为了做到这一点，必须将序列化的关键和价值。默认情况下，Ehcache 3依赖于Java序列化来完成这项工作，因此需要键和值来实现Serializable，根据您的设置情况，PDFTO就是这种情况。

详细地说，它的工作方式是将对象序列化，然后放入ByteBuffer，然后通过非堆内存将其写入磁盘。但是这意味着我们需要能够分配足够大的空间来继续。

所以问题不在于堆层，因为生活在那里的对象在被传递到缓存之前已经被分配了，但是传输到了磁盘层。

我建议查看你的整个堆的使用情况，并理解为什么你如此受限制，或者检查你的对象的序列化大小。如果他们有太多外向引用，可能会有潜在的惊喜。

来源

2017-02-07 09:04:28

我确实使用一个小堆重现了您的问题，与您的问题非常相似。你确实需要一些缓冲空间。做一个内存转储应该显示一堆'HeapByteBuffer's漂浮。 – Henri

接受4个条目onheap意味着可以有4个条目。无论大小。所以如果你的堆很小并且你的对象很大，你确实可以得到OOME。

您的解决方案是为您的堆存储提供最大内存大小（<heap unit="MB">256</heap>）而不是多个条目。

这样，在需要时以及在获得OOME之前（假设最大内存大小低于堆中剩余的内存），条目将被驱逐。

来源

2017-02-05 20:43:53 Henri

嗨，这不是我所看到的，例如使用64 MB，将填满堆中的空间，然后抛出内存异常。 - 你也知道为什么拥有4项最大大小不会驱逐最古老的项目？我的堆设置为512米，它应该没有任何问题。 –

这很有趣，意味着内存泄漏。你使用的是哪个确切版本的ehcache？驱逐是“随机的”（基于采样的明智随机）。因为在年龄上进行订购是非常昂贵的，所以在这里进行折衷。 – Henri

我评论了@ louis-jacomet的回答。由于我所解释的不是原因，我很确定他说的是。 – Henri

EHCache3内存不足错误

回答

相关问题