评估 - OutOfMemoryError：无法创建新的本地线程

Question

尽管这是由各种Stackoverflow用户报告的典型错误消息，但我的问题与如何评估建议的解决方案是否解决问题有关。

我读过各种讨论&与此错误相关的文章和大多数解决方案深入到Linux ulimits，我想这似乎也是我的情况。

我的ulimit值：

STACK 10240k, CORE 0k, NPROC 1024, NOFILE 4096; 

我猜想问题可能出在NOPROC/NOFILE太低（只有默认值）。

但是，我想知道是否有一种确切的方法来确定NOPROC已被超出等等的根本原因，以及是否有方法来确切地评估当前正在使用多少个进程/文件句柄;还是还有其他一些我应该关注的问题，可以进行统计评估？

仅供参考，当发生此问题时，heapdump未启用，并且在错误点处没有线程数据。

感谢您对评估和解决此问题的意见。

这里是简短的堆栈跟踪：

Caused by: java.lang.OutOfMemoryError: unable to create new native thread 
    at java.lang.Thread.start0(Native Method) 
    at java.lang.Thread.start(Thread.java:714) 

下面是系统值：

OS:Red Hat Enterprise Linux Server release 6.3 (Santiago) 
uname:Linux 2.6.32-279.el6.x86_64 #1 SMP Wed Jun 13 18:24:36 EDT 2012 x86_64 
libc:glibc 2.12 NPTL 2.12 
rlimit: STACK 10240k, CORE 0k, NPROC 1024, NOFILE 4096, AS infinity 
load average:0.11 0.10 0.03 
CPU:total 32 (8 cores per cpu, 2 threads per core) family 6 model 45 stepping 7, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, aes, ht, tsc, tscinvbit, tscinv 

/proc/meminfo: 
MemTotal:  74206252 kB 
MemFree:   2788244 kB 
Buffers:   1042212 kB 
Cached:   58454988 kB 
SwapCached:   2860 kB 
Active:   38242540 kB 
Inactive:  29129604 kB 

下面是从JVM崩溃报告中的信息 - hs_err_pidxxxxx.log：

# There is insufficient memory for the Java Runtime Environment to continue. 
# Cannot create GC thread. Out of system resources. 
... 
# Out of Memory Error (gcTaskThread.cpp:46), pid=20396, tid=140365307795200 

# JRE version: (7.0_80-b15) (build) 
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.80-b11 mixed mode linux-amd64 compressed oops) 
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again 

Current thread (0x00007fa95400a800): JavaThread "Unknown thread" [_thread_in_vm, id=20458, stack(0x00007fa9583f5000,0x00007fa9584f6000)] 
Stack: [0x00007fa9583f5000,0x00007fa9584f6000], sp=0x00007fa9584f4540, free space=1021k 

Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) 
V [libjvm.so+0x9a320a] VMError::report_and_die()+0x2ea 
V [libjvm.so+0x498d3b] report_vm_out_of_memory(char const*, int, unsigned long, char const*)+0x9b 
V [libjvm.so+0x55943a] GCTaskThread::GCTaskThread(GCTaskManager*, unsigned int, unsigned int)+0x11a 
V [libjvm.so+0x5589b8] GCTaskManager::initialize()+0x2b8 
V [libjvm.so+0x843438] ParallelScavengeHeap::initialize()+0x6f8 
V [libjvm.so+0x97509a] Universe::initialize_heap()+0xca 
V [libjvm.so+0x976269] universe_init()+0x79 
V [libjvm.so+0x5b2f25] init_globals()+0x65 
V [libjvm.so+0x95db4d] Threads::create_vm(JavaVMInitArgs*, bool*)+0x1ed 
V [libjvm.so+0x63b2e4] JNI_CreateJavaVM+0x74 
C [libjli.so+0x2f8e] JavaMain+0x9e 
Java Threads: (=> current thread) 
Other Threads: 
=>0x00007fa95400a800 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=20458, stack(0x00007fa9583f5000,0x00007fa9584f6000)] 
VM state:not at safepoint (not fully initialized) 
VM Mutex/Monitor currently owned by a thread: None 
GC Heap History (0 events): 
No events 
Deoptimization events (0 events): 
No events 
Internal exceptions (0 events): 
No events 
Events (0 events): 
No events 

Answer 1

I wanted to know if there is an exact way to identify the root cause say the NOPROC has been exceeded etc

JVM就像任何其他软件，最终都必须通过系统调用与内核通信。为了产生新线程，它必须使用clone系统调用，它可以返回各种错误代码（在手册页中有记录）。您可以使用strace来记录系统调用并查看它们的错误代码，这些代码可以提供比OOME更细致的信息。