Recently my jersey application got high GC problems.
By inspecting the heap dump, I found that there're a lot of MethodWrapperImpl as well as LRUHybridCache$OriginThreadAwareFuture became unreachable objects.
(About 19700 MethodWrapperImpl alive and around 40k+ MethodWrapperImpl in unreacheable objects).
My question is:
Is it a normal behavior?
After all, I have only 1 resource and 1 resource method.
The heap dump shows that:
Total 32 instance of ClassReflectionHelperImpl and total 128 LRUHybridCache.
Btw, I'm using spring boot 2.0.5 and the jersey is on 2.26. (hk2 is 2.5.0-b42)
And it's hard to reproduce in local, only happens on production boxes with real traffic.
Leon
Related
We have a wildfly 8.2 app server which allocates 6GB of server RAM. Sometime due to havey transaction count wildfly has stop receiving incoming connections. But when I check server (not app server, it is our VM ) memory, it uses 4GB of RAM. Then I checked Wildfly app server's heap memory it did not use at least 25% of allocate heap size. Why is that? When I restart wildfly App server, All things work normally and when it comes that kind of load, above scenario happen again
Try to increase the connection-limit as suggested in this SO question
You can Dump HTTP requests as given here
Also, are you getting any errors in your console? Please post them as well.
I have a serious problem my memcached memory is overflow and server is getting down.
So how to handle memcached, If memcached memory is getting full then it will just throw error msg, not set the memcached anymore.
memcached is distributed cache system and can be works on different servers. What is your server? is it couchebase, is it elastic cache of AWS? you can use memcache on many different server and providers and when you are creating those servers you need to configure them and set the size of memory you want memcache to use. for example in the company I am working, the test environment uses couchbase but the live uses Amazon Elastic cache.
Memcached uses LRU (least recently used) algorithm to insert the new object into the memory if the table is full. You should not have problem because of full memory and handling memcached. The problem can raise from the full memory but not with this reason that memcached cannot handle it. Exceptions and other problem can be in other part of the system which is quite normal if your memory is full. if you configure the server correctly memcache usually does not throw exception.
Is the server that runs memcached same as the server that runs your application? Memcached can be put on another server and in this way you can prevent the memory to become full.
I am doing load testing on my application using jmeter and I have a situation where the cpu usage by the applications jvm goes to 99% and it stays there. Application still work, I am able to login and do some activity. But, it’s understandably slower.
Details of environment:
Server: AMD Optrom, 2.20 Ghz, 8 Core, 64bit, 24 GB RAM. Windows Server 2008 R2 Standard
Application server: jboss-4.0.4.GA
JAVA: jdk1.6.0_25, Java HotSpot(TM) 64-Bit Server VM
JVM settings:
-Xms1G -Xmx10G -XX:MaxNewSize=3G -XX:MaxPermSize=12G -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+UseCompressedOops -Dsun.rmi.dgc.client.gcInterval=1800000 -Dsun.rmi.dgc.server.gcInterval=1800000
Database: MySql 5.6 (in a different machine)
Jmeter: 2.13
My scenario is that, I make 20 users of my application to log into it and perform normal activity that should not be bringing huge load. Some, minutes into the process, JVM of Jboss goes up and it never comes back. CPU usage will remain like that till JVM is killed.
To help better understand, here are few screen shots.
I found few post which had cup # 100%, but nothing there was same as my situation and could not find a solution.
Any suggestion on what’s to be done will be great.
Regards,
Sreekanth.
To understand the root cause of the high CPU utilization, we need to check the CPU data and thread dumps at same time.
Capture 5-6 thread dumps at the time of the issue. Similarly capture CPU consumption thread-by-thread basis.
Generally the root cause of the CPU issue would be problems with threads like BLOCKED threads, long running threads, dead-lock, long running loops etc. That can be resolved by going through the stacks of the threads.
we have a strange performance problem with ObjectDB 2.5.3_01 and JBoss 7.1.1.
We have two ObjectDB databases on one ObjectDB server. Production database is running 5-10 times slower than test database. Their size and number of records are almost identical. When running production database in separate test environment its speed is very good.
We did a performance analysis on our 2 linux servers one running JBOSS and other running ObjectDB.
- CPU utilization of JBOSS server is max 5-10% (per core)
- CPU utilization of JBOSS server is 80-150% (this is per core)
Now is the interesting part, when running a query from ObjectDB explorer CPU utilization is minimal ie. 1%
Running a query from ObjectDB explorer on complete database which has 12000 records takes 30ms which we think is very good.
In our web application this behaviour gives aprox 10-13 s of refreshing a data table vs 2 second with test database.
Does anybody have any idea what could be wrong.
The problem was found to be a circular eager relationship, defined in that specific application, which required loading of many objects recursively with the query results.
The solution was to change the relationship setting from eager to lazy.
More details can be found in this forum thread.
We started using memcached on the test server for our social media project and having some problems on ram usage.
We have created a cluster with 1 server node running with just 1 cache bucket sized 128 mb but when we check memcached.exe ram usage from the task manager it' s ram usage rises continously 1mb per second.
Any workaround on this?
Thanks!
If you're using our 1.0.3 product (the current version of our Memcached server) there is a known issue where deleting the default bucket causes a memory leak. Can you let me know whether you deleted the default bucket?
Also, we just released beta 4 of our 1.6.0 product which has support for both Membase buckets as well as Memcached buckets. I would certainly appreciate you taking a look and trying it out. I know it has fixed the memory leak issue.
Thanks so much.
Perry