i have a Centos 5 server which hosted asterisk 13.
server works fine last week but now top command always show me a process with large amount of CPU usage. when i kill the process a few second later another command with large CPU usage started. many times processes command is ".syslog" but have other command like "qjennjifes", "vnvebynufu" and another unknown commands like that.
1) Check you have firewall and fail2ban recomended settings
2) Check you have no DoS/DDoS by "sip show channels"
3) Check your system not hacked/no broken soft on your host.
Related
I have a dedicated server running Ubuntu 20.04, with cPanel 106.11, MySQL 8, PHP 8.1, Elasticsearch 7.17.8 and i run magento 2.4.5-p1. Config Server Security & Firewall is enabled.
Every couple of days i get an monitoring alert to say my server doesnt respond to ping and the host has to do a hard reboot, they are getting frustrated with this and say they will turn off monitoring unless i sort this as they have checked all hardware which is fine.
This happens at different times and usually overnight.
I have looked through syslog, mysql log, elasticsearch log, magento 2 logs, apache log, kern.log and i cant find the cause of the issue.
I have enabled "sar" and the RAM usage around the time is 64%, cpu usage is between 5-10%.
What else can i look at to try and diagnose this issue?
Additional info requested by Wilson:
select count - https://justpaste.it/6zc95
show global status - https://justpaste.it/6vqvg
show global variables - https://justpaste.it/cb52m
full process list - https://justpaste.it/d41lt
status - https://justpaste.it/9ht1i
show engine innodb status - https://justpaste.it/a9uem
top -b -n 1 - https://justpaste.it/4zdbx
top -b -n 1 -H - https://justpaste.it/bqt57
ulimit -a - https://justpaste.it/5sjr4
iostat -xm 5 3 - https://justpaste.it/c37to
df -h, df -i, free -h and cat /proc/meminfo - https://justpaste.it/csmwh
htop - https://freeimage.host/i/HAKG0va
Server is using nvme drives, 32GB RAM, 6 cores, MySQL is running on same server as litespeed.
Server has not gone down again since posting this but the datacentre usually reboot within 15 - 20 mins and 99% of the time happens overnight. The server is not accessible over ssh when it crashes.
Rate Per Second = RPS
Suggestions to consider for your instance (should be available in your cpanel as they are all dynamic variables)
connect_timeout=30 # from 10 seconds to reduce aborted_connects RPHr of 75
innodb_io_capacity=900 # from 200 to use more of NVME IOPS capacity
thread_cache_size=36 # from 9 to reduce threads_created RPHr of 75
read_rnd_buffer_size=32768 # from 256K to reduce handler_read_rnd_next RPS of 5,805
read_buffer_size=524288 # from 128K to reduce handler_read_next RPS of 5,063
Many more opportunities exist to improve performance of your instance.
View profile for contact info, please. We are pushing the one question/one answer planned for this platform.
I use VS Code latest version 1.69.2 , and remote connect to my cloud VM. After one or two days， I found the cpu usage is very high .The detail of the process is :
my-user 18954 17082 0 12:10 ? 00:00:04 /home/my-user/work/.vscode-server/bin/3b889b090b5ad5793f524b5d1d39fda662b96a2a/node /home/my-user/work/.vscode-server/bin/3b889b090b5ad5793f524b5d1d39fda662b96a2a/out/bootstrap-fork --type=extensionHost --transformURIs --useHostProxy=false
There are 8 node processes in total，every cpu usage is large than 50%
The question is:
What process is this?
Why the cpu usage is so high?
When I close all the connected windows and then reconnect to my remote VM, these processes are still here. Why are these processes not closing automatically?
Is this the bug of VSCode 1.69.2?
VSCode uses Node, I suppose it's one of these process used for autocomplete that scans your files, but it's never ending. I have to kill them manually, 1 node process uses 100% of the CPU.
I solved the issue by removing the .vscode-server folder created in my home folder. You have to do this from a remote shell (not from vscode terminal).
I solved the issue by removing the Settings Sync plugin。
I am facing same issue regularly which happens 1-3 times in a month and mostly on weekends.
To explain, CPU utilization is exceeding past 100% from last 32 hours.
EC2 Instance is t3.medium
Postgres version is 10.6
OS : Amazon Linux 2
I have tried gather all the information I could get using command provided in reference https://severalnines.com/blog/why-postgresql-running-slow-tips-tricks-get-source
But didn't found any inconsistency or leak in my database, although while checking for process consuming all CPU resources I found following command is the culprit running for more than 32 hours.
/var/lib/postgresql/10/main/postgresql -u pg_linux_copy -B
This command is running from 3 separate processes at the moment and running from last 32 hours, 16 hours & 16 hours respectively.
Searching about about this didn't even returned a single result on google which is heartbreaking.
If I kill the process, everything turns back to normal.
What is the issue and what can I do to prevent this from happening again in future?
I was recently contacted by AWS EC2 Abuse team regarding my instance involved in some intrusion attack to some other server.
To my surprise, I found out that as I had used very week password root for default postgres account for my database and also had the postgres port public, the attacker silently gained access to instance and used my instance to try gaining access to another instance.
I am still not sure, how was he able to try ssh command by gaining access to master database account.
To summarise, One reason for unusual database spikes on server could be someone attacking your system.
I am using PostgreSQL 9.5 with a Java 8 Application on Windows OS (System: i5 2nd Generation). I noticed that when my application is in execution, there are several PostgreSQL processes / sub-processes that are created/removed dynamically.
These PostgreSQL processes use almost all of the CPU (>95%), due to which problem arises with other applications installed at my system.
I recently came to know about CPU affinity. For the time being, I am executing PowerShell script (outside of my Java application) which checks periodically and sets the desired value of cpu affinity for all PostgreSQL processes in execution.
I am looking for a way where I don't need to execute an external script(s) and/or there is one-time configuration required.
Is there a configuration supported by PostgreSQL 9.5 through which we can set max CPU cores to be used by PostgreSQL processes?
I looked for the solution, but could not find any.
There is no way to set this in the PostgreSQL configuration.
But you can start your PostgreSQL server from cmd.exe with:
start /affinity 3 C:\path\to\postgresql.exe C:\path\to\data\directory
That would allow PostgreSQL to run only on the twp “first” cores.
The cores are numbered 1, 2, 4, 8, 16 and so on, and you use the sum of the cores where you want PostgreSQL to run as argument to /affinity. For example, if you only want it to run on the third and fourth core, you would use /affinity 12.
This should work, since the Microsoft documentation says:
Process affinity is inherited by any child process or newly instantiated local process.
I am new administrator of PBS. I downloaded and installed torque-4.2.6 version. I used default configuration that is provided by torque.setup.
The OS is CentOS with kernel 2.6.18.
I stopped all the firewall. I confirmed that all the ssh/scp works bi-directionally between server and nodes.
after configuration, everything looks fine. small number of jobs have finished well.
When I submitted 10000 jobs, they finished about 70% of the jobs, but the remainders do not start to work. I found that the server_priv/jobs directory contains the jobs.
I checked the log fines... but I could not find any clue to the problem.
I checked disk space by using df, and there is 10% (more than 100GB) of free space and it looks enough to run PBS jobs.
Before I check other things, I ask help to the others in this site.