Java program failing to allocate Native memory even after enough RAM is available. The program crashes repeatedly after failing with os_commit exceptions. Any recommendation to update to tune Java Heap etc. As per the logs you can see that there is enough memory available Resource Utilization JMap Output Java Version Redhat Release Answer You are possibly impacted by this As

Java program failing to map Native memory allocation

Java program failing to allocate Native memory even after enough RAM is available.

The program crashes repeatedly after failing with os_commit exceptions.

Any recommendation to update to tune Java Heap etc.

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (os_linux.cpp:2627), pid=35462, tid=140393234093824
#
# JRE version: Java(TM) SE Runtime Environment (8.0_65-b17) (build 1.8.0_65-b17)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.65-b01 mixed mode linux-amd64 compressed oops)
# Core dump written. Default location: /usr/hdp/share/hst/activity-analyzer/core or core.35462
#

As per the logs you can see that there is enough memory available

Memory: 4k page, physical 264266920k(135968600k free), swap 8393468k(8393468k free)

vm_info: Java HotSpot(TM) 64-Bit Server VM (25.65-b01) for linux-amd64 JRE (1.8.0_65-b17), built on Oct  6 2015 17:16:12 by "java_re" with gcc 4.3.0 20080428 (Red Hat 4.3.0-8)

time: Tue Feb  5 05:54:26 2019
elapsed time: 2928 seconds (0d 0h 48m 48s)

Resource Utilization

# free -mg
             total       used       free     shared    buffers     cached
Mem:           252        134        117          0          0         25
-/+ buffers/cache:        108        143
Swap:            8          0          8

JMap Output

# jmap -heap 2820
Attaching to process ID 2820, please wait...
Debugger attached successfully.
Server compiler detected.
JVM version is 25.65-b01

using thread-local object allocation.
Parallel GC with 33 thread(s)

Heap Configuration:
   MinHeapFreeRatio         = 0
   MaxHeapFreeRatio         = 100
   MaxHeapSize              = 33554432000 (32000.0MB)
   NewSize                  = 44564480 (42.5MB)
   MaxNewSize               = 11184635904 (10666.5MB)
   OldSize                  = 89653248 (85.5MB)
   NewRatio                 = 2
   SurvivorRatio            = 8
   MetaspaceSize            = 21807104 (20.796875MB)
   CompressedClassSpaceSize = 1073741824 (1024.0MB)
   MaxMetaspaceSize         = 17592186044415 MB
   G1HeapRegionSize         = 0 (0.0MB)

Heap Usage:
PS Young Generation
Eden Space:
   capacity = 9320267776 (8888.5MB)
   used     = 7558977896 (7208.803077697754MB)
   free     = 1761289880 (1679.696922302246MB)
   81.10258286209995% used
From Space:
   capacity = 262668288 (250.5MB)
   used     = 242645032 (231.40433502197266MB)
   free     = 20023256 (19.095664978027344MB)
   92.37698004869168% used
To Space:
   capacity = 346030080 (330.0MB)
   used     = 0 (0.0MB)
   free     = 346030080 (330.0MB)
   0.0% used
PS Old Generation
   capacity = 715653120 (682.5MB)
   used     = 395523640 (377.2007369995117MB)
   free     = 320129480 (305.2992630004883MB)
   55.267507252675706% used

22626 interned Strings occupying 2218928 bytes.

Java Version

# java -version
java version "1.8.0_65"
Java(TM) SE Runtime Environment (build 1.8.0_65-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.65-b01, mixed mode)

Redhat Release

# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.6 (Santiago)

Answer

You are possibly impacted by this

As per the post’s suggested workarounds :

There are 2 possible workaround here

Disable compressed Oops (-XX:-UseCompressedOops) with slight performance penalty. This will instruct JVM to run without the Compressed Oops feature and it would not try to fit the Java Heap in the first 4GB of heap

Keep the CompressedOops and set the base of the Java Heap with the JVM option -XX:HeapBaseMinAddress=n to specify the address where the Java Heap should start from set it to a higher address so that there is enough room available for the native mallocs

Advertisement

Answer