eclipse starts for some seconds and stop - eclipse

I suddendly start facing this bad problem
eclipse starts working and them after some seconds it stops.
i create a new eclipse from unzipping the orginal eclipse and i still have the same problem
when the eclipse stops, a new log file is automatilcally created in the eclipse directly.
this is what that log file states
#
# A fatal error has been detected by the Java Runtime Environment:
#
# EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000005ca3bb22, pid=6424, tid=3684
#
# JRE version: 7.0_25-b17
# Java VM: Java HotSpot(TM) 64-Bit Server VM (23.25-b01 mixed mode windows-amd64 compressed oops)
# Problematic frame:
# V [jvm.dll+0x47bb22]
#
# Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
# http://bugreport.sun.com/bugreport/crash.jsp
#
--------------- T H R E A D ---------------
Current thread (0x0000000008389000): JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=3684, stack(0x00000000098c0000,0x00000000099c0000)]
siginfo: ExceptionCode=0xc0000005, reading address 0xffffffffffffffff
Registers:
RAX=0x0000000003091222, RBX=0x00000000099bcee0, RCX=0x000100000b7b1210, RDX=0x00000000099bcee0
RSP=0x00000000099bc9d0, RBP=0x00000000099bcb00, RSI=0x000000000b7b10e8, RDI=0x00000000099bca60
R8 =0x00000000000007eb, R9 =0x0000000000000001, R10=0x000000005cb3eea8, R11=0x00000000000000eb
R12=0x0000000009c7abf4, R13=0x0000000000000100, R14=0x000000000b940bf0, R15=0x00000000ffffffff
RIP=0x000000005ca3bb22, EFLAGS=0x0000000000010202
Top of Stack: (sp=0x00000000099bc9d0)
0x00000000099bc9d0: 00000000099bcb88 0000000003091220
0x00000000099bc9e0: 00000000000003a0 0000000000000000
0x00000000099bc9f0: 000000000b7b10e8 000000005c975a13
0x00000000099bca00: 0000000003091500 00000000099bf170
0x00000000099bca10: 00000000099bf170 000000000b7b10e8
0x00000000099bca20: 0000000000000000 0000000000000000
0x00000000099bca30: 000000005caba8d0 00000000099bcab8
0x00000000099bca40: 0000000003091220 00000000030914e0
0x00000000099bca50: 0000000003091220 00000000099bcb88
0x00000000099bca60: 000000005caa8700 00000000030915a0
0x00000000099bca70: 0000000000000000 00000000030915a0
0x00000000099bca80: 00000000030915c0 00000000030915c0
0x00000000099bca90: 00000000030915c0 0000000003091614
0x00000000099bcaa0: 00000000030915a0 000000005c000000
0x00000000099bcab0: 00000000099bca60 0000000003091220
0x00000000099bcac0: 0000000003091220 0000000003091223
Instructions: (pc=0x000000005ca3bb22)
0x000000005ca3bb02: eb 08 48 8b 47 68 41 80 cb e8 c6 00 c1 48 ff 47
0x000000005ca3bb12: 68 48 8b 47 68 44 88 18 48 ff 47 68 48 8b 4e 50
0x000000005ca3bb22: 48 8b 01 ff 50 30 48 8b 4f 68 48 8b 5c 24 30 48
0x000000005ca3bb32: 8b 74 24 38 88 01 48 ff 47 68 48 83 c4 20 5f c3
Register to memory mapping:
RAX=0x00000000030911d0 [CodeBlob (0x00000000030911d0)]
Framesize: 0
BufferBlob (0x00000000030911d0) used for Compile::scratch_buffer
RBX=0x00000000099bcee0 is pointing into the stack for thread: 0x0000000008389000
RCX=0x000100000b7b1210 is an unknown value
RDX=0x00000000099bcee0 is pointing into the stack for thread: 0x0000000008389000
RSP=0x00000000099bc9d0 is pointing into the stack for thread: 0x0000000008389000
RBP=0x00000000099bcb00 is pointing into the stack for thread: 0x0000000008389000
RSI=0x000000000b7b10e8 is an unknown value
RDI=0x00000000099bca60 is pointing into the stack for thread: 0x0000000008389000
R8 =0x00000000000007eb is an unknown value
R9 =0x0000000000000001 is an unknown value
R10=0x000000005cb3eea8 is an unknown value
R11=0x00000000000000eb is an unknown value
R12=0x0000000009c7abf4 is an unknown value
R13=0x0000000000000100 is an unknown value
R14=0x000000000b940bf0 is an unknown value
R15=0x00000000ffffffff is an unallocated location in the heap
Stack: [0x00000000098c0000,0x00000000099c0000], sp=0x00000000099bc9d0, free space=1010k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
V [jvm.dll+0x47bb22]
Current CompileTask:
C2: 25098 1866 ! com.android.ide.common.resources.ValueResourceParser::startElement (589 bytes)
--------------- P R O C E S S ---------------
Java Threads: ( => current thread )
0x000000000b839000 JavaThread "Worker-10" [_thread_blocked, id=6320, stack(0x00000000102b0000,0x00000000103b0000)]
0x000000000b838800 JavaThread "Worker-9" [_thread_blocked, id=916, stack(0x00000000101b0000,0x00000000102b0000)]
0x000000000b80f800 JavaThread "Worker-8" [_thread_blocked, id=6488, stack(0x00000000100b0000,0x00000000101b0000)]
0x000000000b80e800 JavaThread "Worker-7" [_thread_blocked, id=2276, stack(0x00000000109c0000,0x0000000010ac0000)]
0x000000000b80e000 JavaThread "Worker-6" [_thread_blocked, id=5328, stack(0x00000000108c0000,0x00000000109c0000)]
0x000000000b80d000 JavaThread "Worker-5" [_thread_blocked, id=6548, stack(0x00000000107c0000,0x00000000108c0000)]
0x000000000b80c800 JavaThread "Worker-4" [_thread_blocked, id=5876, stack(0x00000000106c0000,0x00000000107c0000)]
0x000000000b80b000 JavaThread "Worker-3" [_thread_blocked, id=3336, stack(0x00000000104c0000,0x00000000105c0000)]
0x000000000b809800 JavaThread "JavaScript indexing" daemon [_thread_blocked, id=7100, stack(0x000000000fec0000,0x000000000ffc0000)]
0x000000000b80b800 JavaThread "Worker-2" [_thread_blocked, id=5848, stack(0x00000000105c0000,0x00000000106c0000)]
0x000000000b80a000 JavaThread "Java indexing" daemon [_thread_blocked, id=6648, stack(0x000000000fdc0000,0x000000000fec0000)]
0x000000000b808800 JavaThread "Worker-1" [_thread_blocked, id=4820, stack(0x000000000eaa0000,0x000000000eba0000)]
0x000000000be1c800 JavaThread "Bundle File Closer" daemon [_thread_blocked, id=6476, stack(0x000000000e9a0000,0x000000000eaa0000)]
0x000000000a904000 JavaThread "Worker-0" [_thread_in_native, id=872, stack(0x000000000d8d0000,0x000000000d9d0000)]
0x000000000a90d000 JavaThread "EMF Reference Cleaner" daemon [_thread_blocked, id=5372, stack(0x000000000c6c0000,0x000000000c7c0000)]
0x000000000ac3c000 JavaThread "Worker-JM" [_thread_blocked, id=6300, stack(0x000000000c3c0000,0x000000000c4c0000)]
0x000000000b761800 JavaThread "[Timer] - Main Queue Handler" daemon [_thread_blocked, id=7056, stack(0x000000000c2c0000,0x000000000c3c0000)]
0x000000000a963800 JavaThread "Framework Event Dispatcher" daemon [_thread_blocked, id=6132, stack(0x000000000c1c0000,0x000000000c2c0000)]
0x000000000a834800 JavaThread "Start Level Event Dispatcher" daemon [_thread_blocked, id=4844, stack(0x000000000be40000,0x000000000bf40000)]
0x000000000a4c1000 JavaThread "State Data Manager" daemon [_thread_blocked, id=6216, stack(0x000000000ad70000,0x000000000ae70000)]
0x000000000a4bc800 JavaThread "Framework Active Thread" [_thread_blocked, id=7096, stack(0x000000000ac70000,0x000000000ad70000)]
0x000000000839b000 JavaThread "Service Thread" daemon [_thread_blocked, id=3696, stack(0x00000000099c0000,0x0000000009ac0000)]
=>0x0000000008389000 JavaThread "C2 CompilerThread1" daemon [_thread_in_native, id=3684, stack(0x00000000098c0000,0x00000000099c0000)]
0x0000000008387800 JavaThread "C2 CompilerThread0" daemon [_thread_blocked, id=5440, stack(0x00000000097c0000,0x00000000098c0000)]
0x0000000008386800 JavaThread "Attach Listener" daemon [_thread_blocked, id=4364, stack(0x00000000096c0000,0x00000000097c0000)]
0x000000000837f000 JavaThread "Signal Dispatcher" daemon [_thread_blocked, id=5368, stack(0x00000000095c0000,0x00000000096c0000)]
0x00000000029c5800 JavaThread "Finalizer" daemon [_thread_blocked, id=6368, stack(0x00000000094c0000,0x00000000095c0000)]
0x00000000029be800 JavaThread "Reference Handler" daemon [_thread_blocked, id=5640, stack(0x00000000093c0000,0x00000000094c0000)]
0x00000000029dc800 JavaThread "main" [_thread_in_native, id=3976, stack(0x0000000000030000,0x0000000000130000)]
Other Threads:
0x00000000082f2000 VMThread [stack: 0x00000000092c0000,0x00000000093c0000] [id=6028]
0x00000000083ae800 WatcherThread [stack: 0x0000000009ac0000,0x0000000009bc0000] [id=1876]
VM state:not at safepoint (normal execution)
VM Mutex/Monitor currently owned by a thread: None
Heap
PSYoungGen total 128640K, used 78311K [0x00000000f5560000, 0x00000000ffbe0000, 0x0000000100000000)
eden space 102400K, 57% used [0x00000000f5560000,0x00000000f8eedd98,0x00000000fb960000)
from space 26240K, 73% used [0x00000000fd420000,0x00000000fe70c218,0x00000000fedc0000)
to space 27392K, 0% used [0x00000000fb960000,0x00000000fb960000,0x00000000fd420000)
ParOldGen total 86272K, used 65718K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 76% used [0x00000000e0000000,0x00000000e402da70,0x00000000e5440000)
PSPermGen total 148800K, used 97115K [0x00000000d0000000, 0x00000000d9150000, 0x00000000e0000000)
object space 148800K, 65% used [0x00000000d0000000,0x00000000d5ed6e48,0x00000000d9150000)
Card table byte_map: [0x0000000005b60000,0x0000000005cf0000] byte_map_base: 0x00000000054e0000
Polling page: 0x00000000003b0000
Code Cache [0x0000000002aa0000, 0x0000000003190000, 0x0000000005aa0000)
total_blobs=2542 nmethods=1732 adapters=758 free_code_cache=42413Kb largest_free_block=43096576
Compilation events (10 events):
Event: 25.012 Thread 0x0000000008387800 1870 com.android.ide.common.rendering.api.ResourceValue::<init> (17 bytes)
Event: 25.012 Thread 0x0000000008387800 nmethod 1870 0x0000000002fdb390 code [0x0000000002fdb4c0, 0x0000000002fdb538]
Event: 25.012 Thread 0x0000000008387800 1871 com.android.ide.common.rendering.api.ResourceValue::getResourceType (5 bytes)
Event: 25.012 Thread 0x0000000008387800 nmethod 1871 0x0000000002fdb190 code [0x0000000002fdb2c0, 0x0000000002fdb318]
Event: 25.018 Thread 0x0000000008387800 1872 com.android.ide.common.resources.MultiResourceFile::addResourceValue (86 bytes)
Event: 25.026 Thread 0x0000000008387800 nmethod 1872 0x00000000030222d0 code [0x00000000030224c0, 0x0000000003022c00]
Event: 25.044 Thread 0x0000000008387800 1873 com.android.ide.common.resources.MultiResourceFile::updateResourceItems (142 bytes)
Event: 25.055 Thread 0x0000000008387800 nmethod 1873 0x00000000030a0f10 code [0x00000000030a1140, 0x00000000030a1af0]
Event: 25.055 Thread 0x0000000008387800 1874 java.util.EnumMap::access$600 (5 bytes)
Event: 25.056 Thread 0x0000000008387800 nmethod 1874 0x00000000030220d0 code [0x0000000003022200, 0x0000000003022258]
GC Heap History (10 events):
Event: 15.097 GC heap before
{Heap before GC invocations=23 (full 3):
PSYoungGen total 116864K, used 30688K [0x00000000f5560000, 0x00000000fee20000, 0x0000000100000000)
eden space 90496K, 14% used [0x00000000f5560000,0x00000000f62828c0,0x00000000fadc0000)
from space 26368K, 65% used [0x00000000fadc0000,0x00000000fbe958c0,0x00000000fc780000)
to space 26432K, 0% used [0x00000000fd450000,0x00000000fd450000,0x00000000fee20000)
ParOldGen total 86272K, used 59553K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 69% used [0x00000000e0000000,0x00000000e3a28750,0x00000000e5440000)
PSPermGen total 147840K, used 88932K [0x00000000d0000000, 0x00000000d9060000, 0x00000000e0000000)
object space 147840K, 60% used [0x00000000d0000000,0x00000000d56d9398,0x00000000d9060000)
Event: 15.103 GC heap after
Heap after GC invocations=23 (full 3):
PSYoungGen total 129472K, used 3574K [0x00000000f5560000, 0x00000000fef20000, 0x0000000100000000)
eden space 103040K, 0% used [0x00000000f5560000,0x00000000f5560000,0x00000000fba00000)
from space 26432K, 13% used [0x00000000fd450000,0x00000000fd7cd8e8,0x00000000fee20000)
to space 26944K, 0% used [0x00000000fba00000,0x00000000fba00000,0x00000000fd450000)
ParOldGen total 86272K, used 59553K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 69% used [0x00000000e0000000,0x00000000e3a28750,0x00000000e5440000)
PSPermGen total 147840K, used 88932K [0x00000000d0000000, 0x00000000d9060000, 0x00000000e0000000)
object space 147840K, 60% used [0x00000000d0000000,0x00000000d56d9398,0x00000000d9060000)
}
Event: 15.103 GC heap before
{Heap before GC invocations=24 (full 4):
PSYoungGen total 129472K, used 3574K [0x00000000f5560000, 0x00000000fef20000, 0x0000000100000000)
eden space 103040K, 0% used [0x00000000f5560000,0x00000000f5560000,0x00000000fba00000)
from space 26432K, 13% used [0x00000000fd450000,0x00000000fd7cd8e8,0x00000000fee20000)
to space 26944K, 0% used [0x00000000fba00000,0x00000000fba00000,0x00000000fd450000)
ParOldGen total 86272K, used 59553K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 69% used [0x00000000e0000000,0x00000000e3a28750,0x00000000e5440000)
PSPermGen total 147840K, used 88932K [0x00000000d0000000, 0x00000000d9060000, 0x00000000e0000000)
object space 147840K, 60% used [0x00000000d0000000,0x00000000d56d9398,0x00000000d9060000)
Event: 15.282 GC heap after
Heap after GC invocations=24 (full 4):
PSYoungGen total 129472K, used 0K [0x00000000f5560000, 0x00000000fef20000, 0x0000000100000000)
eden space 103040K, 0% used [0x00000000f5560000,0x00000000f5560000,0x00000000fba00000)
from space 26432K, 0% used [0x00000000fd450000,0x00000000fd450000,0x00000000fee20000)
to space 26944K, 0% used [0x00000000fba00000,0x00000000fba00000,0x00000000fd450000)
ParOldGen total 86272K, used 58208K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 67% used [0x00000000e0000000,0x00000000e38d8098,0x00000000e5440000)
PSPermGen total 149888K, used 88932K [0x00000000d0000000, 0x00000000d9260000, 0x00000000e0000000)
object space 149888K, 59% used [0x00000000d0000000,0x00000000d56d9358,0x00000000d9260000)
}
Event: 15.761 GC heap before
{Heap before GC invocations=25 (full 4):
PSYoungGen total 129472K, used 76199K [0x00000000f5560000, 0x00000000fef20000, 0x0000000100000000)
eden space 103040K, 73% used [0x00000000f5560000,0x00000000f9fc9f40,0x00000000fba00000)
from space 26432K, 0% used [0x00000000fd450000,0x00000000fd450000,0x00000000fee20000)
to space 26944K, 0% used [0x00000000fba00000,0x00000000fba00000,0x00000000fd450000)
ParOldGen total 86272K, used 58208K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 67% used [0x00000000e0000000,0x00000000e38d8098,0x00000000e5440000)
PSPermGen total 149888K, used 89084K [0x00000000d0000000, 0x00000000d9260000, 0x00000000e0000000)
object space 149888K, 59% used [0x00000000d0000000,0x00000000d56ff1e8,0x00000000d9260000)
Event: 15.770 GC heap after
Heap after GC invocations=25 (full 4):
PSYoungGen total 112000K, used 8954K [0x00000000f5560000, 0x00000000fedc0000, 0x0000000100000000)
eden space 103040K, 0% used [0x00000000f5560000,0x00000000f5560000,0x00000000fba00000)
from space 8960K, 99% used [0x00000000fba00000,0x00000000fc2bebb0,0x00000000fc2c0000)
to space 26240K, 0% used [0x00000000fd420000,0x00000000fd420000,0x00000000fedc0000)
ParOldGen total 86272K, used 58208K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 67% used [0x00000000e0000000,0x00000000e38d8098,0x00000000e5440000)
PSPermGen total 149888K, used 89084K [0x00000000d0000000, 0x00000000d9260000, 0x00000000e0000000)
object space 149888K, 59% used [0x00000000d0000000,0x00000000d56ff1e8,0x00000000d9260000)
}
Event: 15.770 GC heap before
{Heap before GC invocations=26 (full 5):
PSYoungGen total 112000K, used 8954K [0x00000000f5560000, 0x00000000fedc0000, 0x0000000100000000)
eden space 103040K, 0% used [0x00000000f5560000,0x00000000f5560000,0x00000000fba00000)
from space 8960K, 99% used [0x00000000fba00000,0x00000000fc2bebb0,0x00000000fc2c0000)
to space 26240K, 0% used [0x00000000fd420000,0x00000000fd420000,0x00000000fedc0000)
ParOldGen total 86272K, used 58208K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 67% used [0x00000000e0000000,0x00000000e38d8098,0x00000000e5440000)
PSPermGen total 149888K, used 89084K [0x00000000d0000000, 0x00000000d9260000, 0x00000000e0000000)
object space 149888K, 59% used [0x00000000d0000000,0x00000000d56ff1e8,0x00000000d9260000)
Event: 15.923 GC heap after
Heap after GC invocations=26 (full 5):
PSYoungGen total 112000K, used 0K [0x00000000f5560000, 0x00000000fedc0000, 0x0000000100000000)
eden space 103040K, 0% used [0x00000000f5560000,0x00000000f5560000,0x00000000fba00000)
from space 8960K, 0% used [0x00000000fba00000,0x00000000fba00000,0x00000000fc2c0000)
to space 26240K, 0% used [0x00000000fd420000,0x00000000fd420000,0x00000000fedc0000)
ParOldGen total 86272K, used 65718K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 76% used [0x00000000e0000000,0x00000000e402da70,0x00000000e5440000)
PSPermGen total 148800K, used 89084K [0x00000000d0000000, 0x00000000d9150000, 0x00000000e0000000)
object space 148800K, 59% used [0x00000000d0000000,0x00000000d56ff028,0x00000000d9150000)
}
Event: 19.575 GC heap before
{Heap before GC invocations=27 (full 5):
PSYoungGen total 112000K, used 103040K [0x00000000f5560000, 0x00000000fedc0000, 0x0000000100000000)
eden space 103040K, 100% used [0x00000000f5560000,0x00000000fba00000,0x00000000fba00000)
from space 8960K, 0% used [0x00000000fba00000,0x00000000fba00000,0x00000000fc2c0000)
to space 26240K, 0% used [0x00000000fd420000,0x00000000fd420000,0x00000000fedc0000)
ParOldGen total 86272K, used 65718K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 76% used [0x00000000e0000000,0x00000000e402da70,0x00000000e5440000)
PSPermGen total 148800K, used 89916K [0x00000000d0000000, 0x00000000d9150000, 0x00000000e0000000)
object space 148800K, 60% used [0x00000000d0000000,0x00000000d57cf2c8,0x00000000d9150000)
Event: 19.587 GC heap after
Heap after GC invocations=27 (full 5):
PSYoungGen total 128640K, used 19376K [0x00000000f5560000, 0x00000000ffbe0000, 0x0000000100000000)
eden space 102400K, 0% used [0x00000000f5560000,0x00000000f5560000,0x00000000fb960000)
from space 26240K, 73% used [0x00000000fd420000,0x00000000fe70c218,0x00000000fedc0000)
to space 27392K, 0% used [0x00000000fb960000,0x00000000fb960000,0x00000000fd420000)
ParOldGen total 86272K, used 65718K [0x00000000e0000000, 0x00000000e5440000, 0x00000000f5560000)
object space 86272K, 76% used [0x00000000e0000000,0x00000000e402da70,0x00000000e5440000)
PSPermGen total 148800K, used 89916K [0x00000000d0000000, 0x00000000d9150000, 0x00000000e0000000)
object space 148800K, 60% used [0x00000000d0000000,0x00000000d57cf2c8,0x00000000d9150000)
}
Deoptimization events (10 events):
Event: 23.711 Thread 0x000000000a904000 Uncommon trap -12 fr.pc 0x000000000301b6d8
Event: 24.680 Thread 0x000000000a904000 Uncommon trap -34 fr.pc 0x0000000002ecd6bc
Event: 24.690 Thread 0x000000000a904000 Uncommon trap -34 fr.pc 0x0000000002ecd6bc
Event: 24.700 Thread 0x000000000a904000 Uncommon trap -34 fr.pc 0x0000000002ecd6bc
Event: 24.701 Thread 0x000000000a904000 Uncommon trap -34 fr.pc 0x0000000002ecd6bc
Event: 24.753 Thread 0x000000000a904000 Uncommon trap -34 fr.pc 0x0000000002cfd658
Event: 24.776 Thread 0x000000000a904000 Uncommon trap -12 fr.pc 0x0000000003017de4
Event: 24.826 Thread 0x000000000a904000 Uncommon trap -34 fr.pc 0x0000000003164cb0
Event: 24.866 Thread 0x000000000a904000 Uncommon trap -34 fr.pc 0x0000000003164cb0
Event: 24.909 Thread 0x000000000a904000 Uncommon trap -83 fr.pc 0x0000000002d25868
Internal exceptions (10 events):
Event: 19.981 Thread 0x000000000a904000 Threw 0x00000000f65d63e0 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jvm.cpp:1235
Event: 19.981 Thread 0x000000000a904000 Threw 0x00000000f65d6850 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jvm.cpp:1235
Event: 19.985 Thread 0x000000000a904000 Threw 0x00000000f65f0898 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jvm.cpp:1235
Event: 19.985 Thread 0x000000000a904000 Threw 0x00000000f65f0cd8 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jvm.cpp:1235
Event: 19.985 Thread 0x000000000a904000 Threw 0x00000000f65f1250 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jvm.cpp:1235
Event: 19.985 Thread 0x000000000a904000 Threw 0x00000000f65f1690 at C:\jdk7u2_64p\jdk7u25\hotspot\src\share\vm\prims\jvm.cpp:1235
Event: 23.699 Thread 0x000000000a904000 Implicit null exception at 0x0000000003160f07 to 0x000000000316234d
Event: 23.711 Thread 0x000000000a904000 Implicit null exception at 0x000000000301a309 to 0x000000000301b6bd
Event: 24.776 Thread 0x000000000a904000 Implicit null exception at 0x0000000003017bcb to 0x0000000003017dc9
Event: 24.909 Thread 0x000000000a904000 Implicit null exception at 0x0000000002d23539 to 0x0000000002d25859
Events (10 events):
Event: 24.893 Thread 0x0000000008389000 flushing nmethod 0x0000000003073010
Event: 24.893 Thread 0x0000000008389000 flushing nmethod 0x0000000003090e50
Event: 24.893 Thread 0x0000000008389000 flushing nmethod 0x00000000030a0dd0
Event: 24.894 Thread 0x0000000008389000 flushing nmethod 0x00000000030d5210
Event: 24.894 Thread 0x0000000008389000 flushing nmethod 0x00000000030d8d10
Event: 24.894 Thread 0x0000000008389000 flushing nmethod 0x00000000030df010
Event: 24.894 Thread 0x0000000008389000 flushing nmethod 0x00000000030e8a90
Event: 24.897 Thread 0x0000000008387800 flushing nmethod 0x000000000310f4d0
Event: 24.909 Thread 0x000000000a904000 DEOPT PACKING pc=0x0000000002d25868 sp=0x000000000d9cec00
Event: 24.909 Thread 0x000000000a904000 DEOPT UNPACKING pc=0x0000000002ad9164 sp=0x000000000d9ceb78 mode 2
Dynamic libraries:
0x0000000000400000 - 0x000000000044f000 E:\William\eclipse-jee-kepler-R-win32-x86_64\eclipse\eclipse.exe
0x0000000077c90000 - 0x0000000077e39000 C:\Windows\SYSTEM32\ntdll.dll
0x0000000077b70000 - 0x0000000077c8f000 C:\Windows\system32\kernel32.dll
0x000007fefdea0000 - 0x000007fefdf0b000 C:\Windows\system32\KERNELBASE.dll
0x0000000077a70000 - 0x0000000077b6a000 C:\Windows\system32\USER32.dll
0x000007feffdf0000 - 0x000007feffe57000 C:\Windows\system32\GDI32.dll
0x0000000000180000 - 0x000000000018e000 C:\Windows\system32\LPK.dll
0x000007fefe100000 - 0x000007fefe1c9000 C:\Windows\system32\USP10.dll
0x000007feffd50000 - 0x000007feffdef000 C:\Windows\system32\msvcrt.dll
0x000007fefc450000 - 0x000007fefc644000 C:\Windows\WinSxS\amd64_microsoft.windows.common-controls_6595b64144ccf1df_6.0.7601.17514_none_fa396087175ac9ac\COMCTL32.dll
0x000007fefe340000 - 0x000007fefe3b1000 C:\Windows\system32\SHLWAPI.dll
0x000007feffa60000 - 0x000007feffa8e000 C:\Windows\system32\IMM32.DLL
0x000007fefe8d0000 - 0x000007fefe9d9000 C:\Windows\system32\MSCTF.dll
0x0000000072000000 - 0x0000000072012000 E:\William\eclipse-jee-kepler-R-win32-x86_64\eclipse\plugins\org.eclipse.equinox.launcher.win32.win32.x86_64_1.1.200.v20130807-1835\eclipse_1503.dll
0x000007fefe9e0000 - 0x000007fefeabb000 C:\Windows\system32\ADVAPI32.dll
0x000007fefe0d0000 - 0x000007fefe0ef000 C:\Windows\SYSTEM32\sechost.dll
0x000007feffe70000 - 0x000007fefff9d000 C:\Windows\system32\RPCRT4.dll
0x000007fefcd30000 - 0x000007fefcd3c000 C:\Windows\system32\VERSION.dll
0x000007fefc7c0000 - 0x000007fefc816000 C:\Windows\system32\uxtheme.dll
0x000007fefeac0000 - 0x000007fefecc3000 C:\Windows\system32\ole32.dll
0x000007fefdb10000 - 0x000007fefdb1f000 C:\Windows\system32\CRYPTBASE.dll
0x000007fefe030000 - 0x000007fefe0c9000 C:\Windows\system32\CLBCatQ.DLL
0x000007feffc70000 - 0x000007feffd47000 C:\Windows\system32\OLEAUT32.dll
0x000000005c5c0000 - 0x000000005cccf000 C:\Program Files\Java\jdk1.7.0_25\jre\bin\server\jvm.dll
0x000007fefb130000 - 0x000007fefb139000 C:\Windows\system32\WSOCK32.dll
0x000007fefe2f0000 - 0x000007fefe33d000 C:\Windows\system32\WS2_32.dll
0x000007fefe0f0000 - 0x000007fefe0f8000 C:\Windows\system32\NSI.dll
0x000007fefbd00000 - 0x000007fefbd3b000 C:\Windows\system32\WINMM.dll
0x0000000077e60000 - 0x0000000077e67000 C:\Windows\system32\PSAPI.DLL
0x0000000071800000 - 0x00000000718d2000 C:\Windows\system32\MSVCR100.dll
0x0000000065900000 - 0x000000006590f000 C:\Program Files\Java\jdk1.7.0_25\jre\bin\verify.dll
0x000000005e5f0000 - 0x000000005e618000 C:\Program Files\Java\jdk1.7.0_25\jre\bin\java.dll
0x0000000065860000 - 0x0000000065875000 C:\Program Files\Java\jdk1.7.0_25\jre\bin\zip.dll
0x000007fefd470000 - 0x000007fefd487000 C:\Windows\system32\CRYPTSP.dll
0x000007fefd170000 - 0x000007fefd1b7000 C:\Windows\system32\rsaenh.dll
0x000007fefcf10000 - 0x000007fefcf2e000 C:\Windows\system32\USERENV.dll
0x000007fefdbe0000 - 0x000007fefdbef000 C:\Windows\system32\profapi.dll
0x0000000065840000 - 0x0000000065859000 C:\Program Files\Java\jdk1.7.0_25\jre\binx86_64\eclipse\configuration\org.eclipse.osgi\bundles\943\1\.cp\swt-gdip-win32-4333.dll
VM Arguments:
jvm_args: -Dosgi.requiredJavaVersion=1.6 -Xms40m -Xmx512m -XX:MaxPermSize=256m
java_command: <unknown>
Launcher Type: generic
Environment Variables:
JAVA_HOME=C:\Program Files\Java\jdk1.7.0_25
JRE_HOME=C:\Program Files (x86)\Java\jre7
PATH=C:/Program Files/Java/jdk1.7.0_25/bin/../jre/bin/server;C:/Program Files/Java/jdk1.7.0_25/bin/../jre/bin;C:/Program Files/Java/jdk1.7.0_25/bin/../jre/lib/amd64;C:\appOracle\product\11.2.0\client_2;C:\appOracle\product\11.2.0\client_2\bin;C:\Program Files\Java\jdk1.7.0_25\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\;C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\PrivateAssemblies\;C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files (x86)\Microsoft ASP.NET\ASP.NET Web Pages\v1.0\;C:\Program Files (x86)\Windows Kits\8.0\Windows Performance Toolkit\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Python27;C:\Python27\Scripts;C:\OpenSSL\bin;E:\William\eclipse-jee-kepler-R-win32-x86_64\eclipse;
USERNAME=WilliamK
OS=Windows_NT
PROCESSOR_IDENTIFIER=Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
--------------- S Y S T E M ---------------
OS: Windows 7 , 64 bit Build 7601 Service Pack 1
CPU:total 4 (4 cores per cpu, 1 threads per core) family 6 model 42 stepping 7, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, tsc, tscinvbit
Memory: 4k page, physical 8303600k(5555752k free), swap 16605356k(13414600k free)
vm_info: Java HotSpot(TM) 64-Bit Server VM (23.25-b01) for windows-amd64 JRE (1.7.0_25-b17), built on Jun 21 2013 12:58:32 by "java_re" with unknown MS VC++:1600
time: Wed Feb 19 08:57:30 2014
elapsed time: 25 seconds

Related

MongoDB slow - memory usage very high

I have a 5 note replicaset mongoDB - 1 primary, 3 secondaries and 1 arbiter.
I am using mong version 4.2.3
Sizes:
“dataSize” : 688.4161271536723,
“indexes” : 177,
“indexSize” : 108.41889953613281
My Primary is very slow - each command from the shell takes a long time to return.
Memory usage seems very high, and looks like mongodb is consuming more than 50% of the RAM:
# free -lh
total used free shared buff/cache available
Mem: 188Gi 187Gi 473Mi 56Mi 740Mi 868Mi
Low: 188Gi 188Gi 473Mi
High: 0B 0B 0B
Swap: 191Gi 117Gi 74Gi
------------------------------------------------------------------
Top Memory Consuming Process Using ps command
------------------------------------------------------------------
PID PPID %MEM %CPU CMD
311 49145 97.8 498 mongod --config /etc/mongod.conf
23818 23801 0.0 3.8 /bin/prometheus --config.file=/etc/prometheus/prometheus.yml
23162 23145 0.0 8.4 /usr/bin/cadvisor -logtostderr
25796 25793 0.0 0.4 postgres: checkpointer
23501 23484 0.0 1.0 /postgres_exporter
24490 24473 0.0 0.1 grafana-server --homepath=/usr/share/grafana --config=/etc/grafana/grafana.ini --packaging=docker cfg:default.log.mode=console
top:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
311 systemd+ 20 0 313.9g 184.6g 2432 S 151.7 97.9 26229:09 mongod
23818 nobody 20 0 11.3g 150084 17988 S 20.7 0.1 8523:47 prometheus
23162 root 20 0 12.7g 93948 5964 S 65.5 0.0 18702:22 cadvisor
serverStatus memeory shows this:
octopusrs0:PRIMARY> db.serverStatus().mem
{
"bits" : 64,
"resident" : 189097,
"virtual" : 321404,
"supported" : true
}
octopusrs0:PRIMARY> db.serverStatus().tcmalloc.tcmalloc.formattedString
------------------------------------------------
MALLOC: 218206510816 (208097.9 MiB) Bytes in use by application
MALLOC: + 96926863360 (92436.7 MiB) Bytes in page heap freelist
MALLOC: + 3944588576 ( 3761.9 MiB) Bytes in central cache freelist
MALLOC: + 134144 ( 0.1 MiB) Bytes in transfer cache freelist
MALLOC: + 713330688 ( 680.3 MiB) Bytes in thread cache freelists
MALLOC: + 1200750592 ( 1145.1 MiB) Bytes in malloc metadata
MALLOC: ------------
MALLOC: = 320992178176 (306122.0 MiB) Actual memory used (physical + swap)
MALLOC: + 13979086848 (13331.5 MiB) Bytes released to OS (aka unmapped)
MALLOC: ------------
MALLOC: = 334971265024 (319453.5 MiB) Virtual address space used
MALLOC:
MALLOC: 9420092 Spans in use
MALLOC: 234 Thread heaps in use
MALLOC: 4096 Tcmalloc page size
------------------------------------------------
Call ReleaseFreeMemory() to release freelist memory to the OS (via madvise()).
Bytes released to the OS take up virtual address space but no physical memory.
How can I detrmine what is causing this high memory consumption, and what can I do to return to nornal memory consumption?
Thanks,
Tamar

what are using large mem_reserve in a .net application?

We have an asp.net web service application that is having OutOfMemory issue with IIS w3wp.exe. We got mini memory dump of w3wp when OOM happened, and we see it has large mem_reserve but small managed heap size. Question is how can we find out what is using mem_reserve (virtual address space). I expect to see some mem_reserve but not 2.9G. Since mem_reserve doesn't add up to eeheap, would it be from unmanaged heap? then anyway to confirm that?
0:000> !address -summary
Mapping file section regions...
Mapping module regions...
Mapping PEB regions...
Mapping TEB and stack regions...
Mapping heap regions...
Mapping page heap regions...
Mapping other regions...
Mapping stack trace database regions...
Mapping activation context regions...
--- Usage Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
<unknown> 1664 d4c1d000 ( 3.324 GB) 92.34% 83.11%
Free 348 19951000 ( 409.316 MB) 9.99%
Image 1184 ad58000 ( 173.344 MB) 4.70% 4.23%
Heap 75 5ab4000 ( 90.703 MB) 2.46% 2.21%
Stack 193 1200000 ( 18.000 MB) 0.49% 0.44%
TEB 64 40000 ( 256.000 kB) 0.01% 0.01%
Other 10 35000 ( 212.000 kB) 0.01% 0.01%
PEB 1 1000 ( 4.000 kB) 0.00% 0.00%
--- Type Summary (for busy) ------ RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_PRIVATE 1323 d6f04000 ( 3.358 GB) 93.28% 83.96%
MEM_IMAGE 1828 dd75000 ( 221.457 MB) 6.01% 5.41%
MEM_MAPPED 40 1a26000 ( 26.148 MB) 0.71% 0.64%
--- State Summary ---------------- RgnCount ----------- Total Size -------- %ofBusy %ofTotal
MEM_RESERVE 746 b8b4e000 ( 2.886 GB) 80.16% 72.15%
MEM_COMMIT 2445 2db51000 ( 731.316 MB) 19.84% 17.85%
MEM_FREE 348 19951000 ( 409.316 MB) 9.99%
--- Protect Summary (for commit) - RgnCount ----------- Total Size -------- %ofBusy %ofTotal
PAGE_READWRITE 1024 1f311000 ( 499.066 MB) 13.54% 12.18%
PAGE_EXECUTE_READ 248 b2e6000 ( 178.898 MB) 4.85% 4.37%
PAGE_READONLY 613 1c1d000 ( 28.113 MB) 0.76% 0.69%
PAGE_WRITECOPY 219 103e000 ( 16.242 MB) 0.44% 0.40%
PAGE_EXECUTE_READWRITE 144 59c000 ( 5.609 MB) 0.15% 0.14%
PAGE_EXECUTE_WRITECOPY 69 21f000 ( 2.121 MB) 0.06% 0.05%
PAGE_READWRITE|PAGE_GUARD 128 144000 ( 1.266 MB) 0.03% 0.03%
--- Largest Region by Usage ----------- Base Address -------- Region Size ----------
<unknown> c2010000 4032000 ( 64.195 MB)
Free 86010000 4000000 ( 64.000 MB)
Image 72f48000 f5d000 ( 15.363 MB)
Heap 64801000 fcf000 ( 15.809 MB)
Stack 1ed0000 7a000 ( 488.000 kB)
TEB ffe8a000 1000 ( 4.000 kB)
Other fffb0000 23000 ( 140.000 kB)
PEB fffde000 1000 ( 4.000 kB)
0:000> !ao
---------Heap 1 ---------
Managed OOM occured after GC #2924 (Requested to allocate 0 bytes)
Reason: Didn't have enough memory to allocate an LOH segment
Detail: LOH: Failed to reserve memory (117440512 bytes)
---------Heap 7 ---------
Managed OOM occured after GC #2308 (Requested to allocate 0 bytes)
Reason: Could not do a full GC
0:000> !vmstat
TYPE MINIMUM MAXIMUM AVERAGE BLK COUNT TOTAL
~~~~ ~~~~~~~ ~~~~~~~ ~~~~~~~ ~~~~~~~~~ ~~~~~
Free:
Small 4K 64K 34K 219 7,475K
Medium 72K 832K 273K 80 21,903K
Large 1,080K 65,536K 7,954K 49 389,759K
Summary 4K 65,536K 1,204K 348 419,139K
Reserve:
Small 4K 64K 14K 436 6,519K
Medium 88K 1,020K 303K 169 51,303K
Large 1,088K 65,528K 21,052K 141 2,968,407K
Summary 4K 65,528K 4,056K 746 3,026,231K
Commit:
Small 4K 64K 16K 2,019 33,386K
Medium 68K 1,024K 287K 275 79,043K
Large 1,028K 65,736K 7,315K 87 636,435K
Summary 4K 65,736K 314K 2,381 748,866K
Private:
Small 4K 64K 28K 851 23,911K
Medium 88K 1,024K 313K 222 69,687K
Large 1,028K 65,736K 18,429K 186 3,427,951K
Summary 4K 65,736K 2,797K 1,259 3,521,550K
Mapped:
Small 4K 64K 18K 23 431K
Medium 68K 1,004K 385K 11 4,239K
Large 1,520K 6,640K 3,684K 6 22,104K
Summary 4K 6,640K 669K 40 26,775K
Image:
Small 4K 64K 9K 1,581 15,562K
Medium 68K 1,000K 267K 211 56,419K
Large 1,028K 15,732K 4,299K 36 154,787K
Summary 4K 15,732K 124K 1,828 226,771K
0:000> !eeheap -gc
Number of GC Heaps: 8
------------------------------
Heap 0 (01cd1720)
generation 0 starts at 0x44470de4
generation 1 starts at 0x43f51000
generation 2 starts at 0x02161000
ephemeral segment allocation context: none
segment begin allocated size
02160000 02161000 02dee4a8 0xc8d4a8(13161640)
43f50000 43f51000 444b9364 0x568364(5669732)
Large object heap starts at 0x12161000
segment begin allocated size
12160000 12161000 121bd590 0x5c590(378256)
c2010000 c2011000 c6041020 0x4030020(67305504)
Heap Size: Size: 0x5281dbc (86515132) bytes.
------------------------------
Heap 1 (01cd65a8)
generation 0 starts at 0x8c4b38e0
generation 1 starts at 0x8c011000
generation 2 starts at 0x04161000
ephemeral segment allocation context: none
segment begin allocated size
04160000 04161000 054114f0 0x12b04f0(19596528)
25c40000 25c41000 25fc7328 0x386328(3695400)
8c010000 8c011000 8c4c4778 0x4b3778(4929400)
Large object heap starts at 0x13161000
segment begin allocated size
13160000 13161000 13161010 0x10(16)
Heap Size: Size: 0x1ae9fa0 (28221344) bytes.
------------------------------
Heap 2 (01cdb5c0)
generation 0 starts at 0x4a71a420
generation 1 starts at 0x49f51000
generation 2 starts at 0x06161000
ephemeral segment allocation context: none
segment begin allocated size
06160000 06161000 06e89b18 0xd28b18(13798168)
27c40000 27c41000 27f8f6dc 0x34e6dc(3466972)
9a010000 9a011000 9a5900b8 0x57f0b8(5763256)
49f50000 49f51000 4a72cc2c 0x7dbc2c(8240172)
Large object heap starts at 0x14161000
segment begin allocated size
14160000 14161000 14161010 0x10(16)
Heap Size: Size: 0x1dd1ee8 (31268584) bytes.
------------------------------
Heap 3 (01ce05d8)
generation 0 starts at 0x34fbe540
generation 1 starts at 0x34d11000
generation 2 starts at 0x08161000
ephemeral segment allocation context: none
segment begin allocated size
08160000 08161000 08c26248 0xac5248(11293256)
2bc40000 2bc41000 2bfc9294 0x388294(3703444)
34d10000 34d11000 34fcdbb0 0x2bcbb0(2870192)
Large object heap starts at 0x15161000
segment begin allocated size
15160000 15161000 151c3b10 0x62b10(404240)
Heap Size: Size: 0x116cb9c (18271132) bytes.
------------------------------
Heap 4 (01ce55f0)
generation 0 starts at 0x934a7cec
generation 1 starts at 0x93011000
generation 2 starts at 0x0a161000
ephemeral segment allocation context: none
segment begin allocated size
0a160000 0a161000 0b4bf898 0x135e898(20310168)
45f50000 45f51000 45f70df4 0x1fdf4(130548)
93010000 93011000 934b83ec 0x4a73ec(4879340)
Large object heap starts at 0x16161000
segment begin allocated size
16160000 16161000 161ab050 0x4a050(303184)
Heap Size: Size: 0x186fac8 (25623240) bytes.
------------------------------
Heap 5 (01cec608)
generation 0 starts at 0x31f1d4d0
generation 1 starts at 0x31c41000
generation 2 starts at 0x0c161000
ephemeral segment allocation context: none
segment begin allocated size
0c160000 0c161000 0d2120b0 0x10b10b0(17502384)
7aea0000 7aea1000 7af5c074 0xbb074(766068)
31c40000 31c41000 31f2caac 0x2ebaac(3062444)
Large object heap starts at 0x17161000
segment begin allocated size
17160000 17161000 17179ad0 0x18ad0(101072)
Heap Size: Size: 0x14706a0 (21431968) bytes.
------------------------------
Heap 6 (01cf5a20)
generation 0 starts at 0x5fbb5598
generation 1 starts at 0x5f821000
generation 2 starts at 0x0e161000
ephemeral segment allocation context: none
segment begin allocated size
0e160000 0e161000 0eb22284 0x9c1284(10228356)
5f820000 5f821000 5fbc9838 0x3a8838(3835960)
Large object heap starts at 0x18161000
segment begin allocated size
18160000 18161000 18161010 0x10(16)
Heap Size: Size: 0xd69acc (14064332) bytes.
------------------------------
Heap 7 (01cfaa38)
generation 0 starts at 0xad530e00
generation 1 starts at 0xad011000
generation 2 starts at 0x10161000
ephemeral segment allocation context: none
segment begin allocated size
10160000 10161000 109bd218 0x85c218(8765976)
a4010000 a4011000 a42f418c 0x2e318c(3027340)
41f50000 41f51000 42a99090 0xb48090(11829392)
ad010000 ad011000 ad5399a0 0x5289a0(5409184)
Large object heap starts at 0x19161000
segment begin allocated size
19160000 19161000 1980ddb0 0x6acdb0(6999472)
Heap Size: Size: 0x225cb84 (36031364) bytes.
------------------------------
GC Heap Size: Size: 0xf950f98 (261427096) bytes.
0:000> !heap -s
SEGMENT HEAP ERROR: failed to initialize the extention
LFH Key : 0x3a251113
Termination on corruption : ENABLED
Heap Flags Reserv Commit Virt Free List UCR Virt Lock Fast
(k) (k) (k) (k) length blocks cont. heap
-----------------------------------------------------------------------------
Virtual block: 1a830000 - 1a830000 (size 00000000)
Virtual block: 1a9d0000 - 1a9d0000 (size 00000000)
007b0000 00000002 16384 15520 16384 198 418 5 2 3 LFH
00280000 00001002 3136 1608 3136 7 23 3 0 0 LFH
00a40000 00001002 1088 828 1088 79 25 2 0 0 LFH
00c30000 00001002 1088 308 1088 13 5 2 0 0 LFH
00100000 00041002 256 4 256 2 1 1 0 0
01560000 00001002 256 24 256 3 4 1 0 0
01510000 00001002 64 4 64 2 1 1 0 0
01670000 00001002 64 4 64 2 1 1 0 0
00290000 00011002 256 4 256 1 2 1 0 0
01870000 00001002 256 8 256 3 3 1 0 0
004e0000 00001002 256 4 256 1 2 1 0 0
019e0000 00001002 256 4 256 2 1 1 0 0
01af0000 00001002 256 4 256 2 1 1 0 0
02080000 00001002 1280 396 1280 1 34 2 0 0 LFH
01fc0000 00041002 256 4 256 2 1 1 0 0
1b700000 00041002 256 256 256 6 4 1 0 0 LFH
21840000 00001002 64192 1880 64192 1639 13 12 0 0 LFH
External fragmentation 87 % (13 free blocks)
-----------------------------------------------------------------------------
If we recycle w3wp, then all memory will go back to system. So there seems no memory leak. I understand that mem_reserve is mapped to virtual address, and are allocated space but not committed. Question is: is there a way to figure out what is using up all these virtual address.
I have memory dump, but I don't have access the the server to get performance counter or something.
Thanks folks!!

Emacs regular freezes: How to identify cause?

I'm having regular freezes in Emacs (every 15 Seconds and then for 2 to 3 Seconds).
I stated the Profiler with CPU:
- ... 4588 44%
Automatic GC 4588 44%
+ redisplay_internal (C function) 3405 33%
+ command-execute 1887 18%
+ timer-event-handler 384 3%
+ flyspell-post-command-hook 18 0%
+ undo-auto--add-boundary 8 0%
+ internal-echo-keystrokes-prefix 8 0%
+ delete-selection-pre-hook 7 0%
+ eldoc-schedule-timer 3 0%
tooltip-hide 3 0%
internal-timer-start-idle 2 0%
+ gui-set-selection 2 0%
+ global-visual-line-mode-check-buffers 1 0%
+ deactivate-mark 1 0%
How can i proceed from here to identify the root cause.
This kind of freeze is annoying when typing longer code or text.
Any help is highly appreciated.
Thanks

What is the difference bewteen SWAP and Virtual mmeory in Linux

Why I am not seeing any SWAP usage in below scenario, even though the virtual memory is showing 20gig ?
Tasks: 186 total, 1 running, 185 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2%us, 0.2%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 32880880k total, 17555744k used, 15325136k free, 300268k buffers
Swap: 16383996k total, 0k used, 16383996k free, 2287700k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10314 abcuser 20 0 20.3g 6.8g 25m S 0.7 21.8 23:47.38 java
25119 abcuser 20 0 20.3g 6.7g 25m S 0.3 21.4 23:38.25 java
total used free shared buffers cached
Mem: 31 16 14 0 0 2
-/+ buffers/cache: 14 17
Swap: 15 0 15
Total: 46 16 30
Regards
Sanjeev

Postgres Out of Memory

I am setting up a new install of postgres 10.4 and during load testing I keep receiving an out of memory error.
Configuration:
$ cat /etc/security/limits.conf
postgres hard memlock 508559360
postgres soft memlock 508559360
$ cat /etc/sysctl.conf
vm.nr_hugepages = 248320
vm.hugetlb_shm_group = 118
vm.overcommit_memory=2
vm.swappiness=1
vm.vfs_cache_pressure=50
kernel.sem = 250 32000 32 128
System Specs:
$ cat /proc/cpuinfo | grep "core id" | wc -l
32
Each CPU is an Intel(R) Xeon(R) CPU E5-2695 v4 # 2.10GHz. 16 physical cores, 32 logical as shown above.
$ free -m
free -m
total used free shared buff/cache available
Mem: 510969 498048 2434 3098 10485 8343
Swap: 1071 1071 0
Note: we have the memlock set at approximately 485 GB dedicated directly to postgres which shows in the high "used" column.
$ df -h /dev/shm
Filesystem Size Used Avail Use% Mounted on
tmpfs 250G 64K 250G 1% /dev/shm
$ cat /proc/meminfo
MemTotal: 523232496 kB
MemFree: 1216648 kB
MemAvailable: 8348808 kB
Buffers: 65220 kB
Cached: 10211780 kB
SwapCached: 7620 kB
Active: 5556092 kB
Inactive: 5432428 kB
Active(anon): 2208064 kB
Inactive(anon): 1593084 kB
Active(file): 3348028 kB
Inactive(file): 3839344 kB
Unevictable: 24128 kB
Mlocked: 24128 kB
SwapTotal: 1097724 kB
SwapFree: 0 kB
Dirty: 52 kB
Writeback: 0 kB
AnonPages: 728044 kB
Mapped: 62904 kB
Shmem: 3085592 kB
Slab: 1454276 kB
SReclaimable: 1345008 kB
SUnreclaim: 109268 kB
KernelStack: 11984 kB
PageTables: 22856 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 15770860 kB
Committed_AS: 7920136 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
HardwareCorrupted: 0 kB
AnonHugePages: 71680 kB
CmaTotal: 0 kB
CmaFree: 0 kB
HugePages_Total: 248320
HugePages_Free: 245883
HugePages_Rsvd: 1800
HugePages_Surp: 0
Hugepagesize: 2048 kB
DirectMap4k: 329600 kB
DirectMap2M: 534444032 kB
Postgres Info:
SELECT version();
PostgreSQL 10.4 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609, 64-bit
max_connections=600
shared_buffers= '8192 MB'
work_mem = '20 MB'
maintenance_work_mem = 2GB
max_parallel_workers = 8
wal_buffers = 16MB
max_wal_size = 20GB
min_wal_size = 1GB
checkpoint_completion_target = 0.9
effective_cache_size = '364 GB'
default_statistics_target = 1000
log_timezone = 'US/Eastern'
track_activities = on
track_counts = on
track_io_timing = on
stats_temp_directory = 'pg_stat_tmp'
datestyle = 'iso, mdy'
timezone = 'US/Eastern'
default_text_search_config = 'pg_catalog.english'
transform_null_equals = on
shared_preload_libraries = 'pg_stat_statements'
track_activity_query_size = 16384
track_functions = all
track_io_timing = true
pg_stat_statements.track = all
session_preload_libraries = 'auto_explain'
auto_explain.log_min_duration = '3s'
auto_explain.log_nested_statements='on'
auto_explain.log_analyze=true
Note: I've started w/ work memory set as high as 250 MB and have slowly brought it down to 20 MB and still receive the errors. I've also verified that connections don't go higher than 120 connections. We use PGBouncer in front of the instance in session mode.
The errors are too large for stack overflow but here they are linked:
https://codepad.co/snippet/derPU4E8
The standout errors are:
2018-06-18 15:02:22 EDT,28197,mydb,ERROR: could not resize shared memory segment "/PostgreSQL.1552129380" to 192088 bytes: No space left on device
I don't understand what device its talking about. I don't have any that appear to be even close to OOM let alone the tiny 192088 bytes its talking about.
2018-06-18 15:02:22 EDT,19708,mydb,ERROR: out of memory
2018-06-18 15:02:22 EDT,19708,mydb,DETAIL: Failed on request of size 7232.
2018-06-18 15:02:22 EDT,16688,,LOG: could not fork worker process: Cannot allocate memory
2018-06-18 15:02:22 EDT,4555,,ERROR: out of memory
2018-06-18 15:02:22 EDT,4555,,DETAIL: Failed on request of size 78336.
2018-06-18 15:02:22 EDT,4552,,LOG: could not open directory "/usr/lib/postgresql/10/share/timezone": Cannot allocate memory
2018-06-18 15:02:22 EDT,19935,mydb,ERROR: could not load library "/usr/lib/postgresql/10/lib/auto_explain.so": /usr/lib/postgresql/10/lib/auto_explain.so: failed to map segment from shared object
2018-06-18 15:02:22 EDT,28193,mydb,ERROR: out of memory
2018-06-18 15:02:22 EDT,28193,mydb,DETAIL: Failed on request of size 8192.
2018-06-18 15:02:22 EDT,26927,mydb,ERROR: could not resize shared memory segment "/PostgreSQL.1101931262" to 192088 bytes: No space left on device
Question: How do I debug this issue and more importantly how can I resolve it?
I followed Laurenz advice and no longer run OOM.
vm.overcommit_ratio = 100.
Additionally, I removed the memlock and huge pages definition in sysctl/limits.
Check to see if your queries are so complicated that they need to consume a large amount of temporary tablespace.