CPU usage near 100% all the time

Questions about SNMP, Power, System, Logs, disk, & RAID.
90Ninety
Know my way around
Posts: 152
Joined: Mon Jun 27, 2016 7:00 pm

Re: CPU usage near 100% all the time

Post by 90Ninety »

I am now getting a similar Issue , even after stopping all VM's , removing apps , full AV and Malware scans ( some removed ) , changing passwords . I am still getting giving a 20%-60% CPU thread to Kernel_Processes .

Literally the system is running idle with everything else and I have a constant spiking 20-60% thread going to Kernel_Processes

I opened a ticket with QNAP where I removed the known malware found in their FEB/MAR 2019 firmware . However the CPU is still spiking with this thread , even with everything disabled . My ports are still open , so will try closing or changing them next
alokeprasad
Easy as a breeze
Posts: 495
Joined: Tue Aug 25, 2015 7:06 pm

Re: CPU usage near 100% all the time

Post by alokeprasad »

90Ninety wrote: Mon Mar 11, 2019 8:59 pm .. the known malware found in their FEB/MAR 2019 firmware .
Which malware came with their firmware? I this the malware with unknown cause/method od infestation that is being discussed in
viewtopic.php?f=50&t=146352
and
https://www.qnap.com/en-us/security-adv ... -201902-13
NAS: TS-453Be
RAM:Crucial 8GB Kit (2 x 4GB) DDR3L-1600 SODIMM CT2KIT51264BF160B
QTS: 5.1.4
HDD's: RAID 6: Four 8TB WD Red (WD80EFAX)
USB HDD: One 12 TB WD Elements (WDBWLG0120HBK-NESN)
Switch: Netgear GS108
User avatar
OneCD
Guru
Posts: 12144
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: CPU usage near 100% all the time

Post by OneCD »

alokeprasad wrote: Tue Mar 12, 2019 8:08 am
90Ninety wrote: Mon Mar 11, 2019 8:59 pm .. the known malware found in their FEB/MAR 2019 firmware .
Which malware came with their firmware?
I don't think @90Ninety is saying QNAP distributed malware-infected QTS versions.

I think they meant they were running one of those firmware versions and, at some point, discovered malware running on it (thus, their firmware wasn't immune to infection). ;)

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
iam@nas
Easy as a breeze
Posts: 267
Joined: Wed Jun 15, 2016 2:49 am

Re: CPU usage near 100% all the time

Post by iam@nas »

You may really want to enable SSH access to your NAS and use the "top" command to identify the running process. 'Kernel_Processes' may be a random python process and knowing more about it wouldn't hurt. If you ever identify the PID of th process 'cat /proc/PID/cmdline" may help.
User avatar
herrcafe
Starting out
Posts: 38
Joined: Sun Apr 24, 2016 3:14 pm

Re: CPU usage near 100% all the time

Post by herrcafe »

90Ninety wrote: Mon Mar 11, 2019 8:59 pm I am now getting a similar Issue , even after stopping all VM's , removing apps , full AV and Malware scans ( some removed ) , changing passwords . I am still getting giving a 20%-60% CPU thread to Kernel_Processes .

Literally the system is running idle with everything else and I have a constant spiking 20-60% thread going to Kernel_Processes

I opened a ticket with QNAP where I removed the known malware found in their FEB/MAR 2019 firmware . However the CPU is still spiking with this thread , even with everything disabled . My ports are still open , so will try closing or changing them next
Any luck with this? I've been troubleshooting my QNAP 453mini for a while now (well, since we became aware of the Malware infection). System processes continue to average 60-70%, mostly due to manaRequest.cgi, disk_manage.cgi, and appRequest.cgi. If I run anything minimally demanding the GUI becomes unresponsive. QNAP took a look at it but they don't seem to know what the problem is. I wonder if you guys had better luck. The path to factory reset is getting closer by the day.
--
QNAP 453mini 16GB RAM RAID5 4 x 6TB HGST HUS726060ALA640
QNAP UX-500P Expansion Unit RAID5 5 x 4TB HGST HUS724040ALA640
User avatar
dolbyman
Guru
Posts: 35249
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: CPU usage near 100% all the time

Post by dolbyman »

if you think you have a malware infection, a complete reset should be done asap
neenjapt
First post
Posts: 1
Joined: Sat May 05, 2018 11:02 pm

Re: CPU usage near 100% all the time

Post by neenjapt »

herrcafe wrote: Fri Oct 11, 2019 5:33 am
90Ninety wrote: Mon Mar 11, 2019 8:59 pm I am now getting a similar Issue , even after stopping all VM's , removing apps , full AV and Malware scans ( some removed ) , changing passwords . I am still getting giving a 20%-60% CPU thread to Kernel_Processes .

Literally the system is running idle with everything else and I have a constant spiking 20-60% thread going to Kernel_Processes

I opened a ticket with QNAP where I removed the known malware found in their FEB/MAR 2019 firmware . However the CPU is still spiking with this thread , even with everything disabled . My ports are still open , so will try closing or changing them next
Any luck with this? I've been troubleshooting my QNAP 453mini for a while now (well, since we became aware of the Malware infection). System processes continue to average 60-70%, mostly due to manaRequest.cgi, disk_manage.cgi, and appRequest.cgi. If I run anything minimally demanding the GUI becomes unresponsive. QNAP took a look at it but they don't seem to know what the problem is. I wonder if you guys had better luck. The path to factory reset is getting closer by the day.
Were you able to solve? If so, how did you manage to do it?

I've been having the exact same issue with the exact same processes and still can't figure out the cause.

I believe this only started with the QTS version that introduced the QuMagic apps.

Thanks
carlos--
New here
Posts: 7
Joined: Fri Jul 01, 2016 2:31 am

Re: CPU usage near 100% all the time

Post by carlos-- »

I also noticed an increased load on my TS-453A in the last few weeks(/months?). The web itnerface is responding very slowly. Using a SSH connection with the top command it is clear that the load goes up all the time when the web interface is up. What are all those "system processes" in resource monitor?
laserfocus
New here
Posts: 8
Joined: Mon Oct 09, 2017 6:43 am

Re: CPU usage near 100% all the time

Post by laserfocus »

I am in the same boat. I was at 100% CPU all the time on a 451+. Worked with tech support to quit virtually every process. Now CPU sometimes drops to 40% but currently at 99%. Please fix, Qnap.
User avatar
dolbyman
Guru
Posts: 35249
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: CPU usage near 100% all the time

Post by dolbyman »

so what process is using your cpu then ?

asking qnap to do something here will do absolutely nothing
laserfocus
New here
Posts: 8
Joined: Mon Oct 09, 2017 6:43 am

Re: CPU usage near 100% all the time

Post by laserfocus »

Thanks. The reason qnap support could not really fix it is that the guilty processes constantly hop around. Sometimes Qmail is at 50%L-75%. Then it drops and manarequest.cgi, or msgexport.sh or diskmanage.cgi or chartreq.cgi or hal_daemon or daemon.mgr jump up. Support said that Qmail should not be the problem, but could not locate the issue otherwise.
spekkie2
First post
Posts: 1
Joined: Fri Jan 24, 2020 4:02 am

Re: CPU usage near 100% all the time

Post by spekkie2 »

This is going for quite some time. I Have 2 Qnap devices TS251 and TS472.
Different firmware etc does not really matter.

The thing to watch is mariadb. alias mysqld this is for always the culprit.


What always helps for me is to restart mariadb.

ssh into the NAS and " /etc/init.d/mariadb.sh restart "
This 95% of times fixes high CPU load. Still not sure why it happens sure I use a lot of stuff. vm's, containers, snapshots etc etc.

For sure I know Plex was one of the apps causing it for quite some times I am not using that anymore. Still I have this problem one a while.
My suspect is in backup stuff but not sure off that neither do I have evidence of this at the moment.

Never the less it is annoying this problems is there for a long time allready and should be addressed by Qnap.
fshort
First post
Posts: 1
Joined: Sun Feb 09, 2020 9:38 pm

Re: CPU usage near 100% all the time

Post by fshort »

Had the same issue. Noticed on the dashboard I had an background indexing task running. When I stopped that, the CPU reduced and the mysqld process went down to a normal level.
laserfocus
New here
Posts: 8
Joined: Mon Oct 09, 2017 6:43 am

Re: CPU usage near 100% all the time

Post by laserfocus »

I am in the same boat. Same issues. Same culprits in Resource Monitor. I have worked with support for months, shutting down tons of processes, reducing a lot of functionality, making Qmail check as infrequently as possible, pausing all indexing, etc. But my CPU is still near 100% most of the time.

Qnap can't fix it. I'm starting to think this 451+ is a lemon.
Flotsch123
Know my way around
Posts: 148
Joined: Sun Mar 12, 2017 9:03 pm

Re: CPU usage near 100% all the time

Post by Flotsch123 »

same here with a ts-253A. cpu above 50% almost all the time. goes up to 90% time by time with no load. i wa snot able to find malware with malware remover or antivirus. no suspicous cron entries.
to be safe, i just made a firmware recovery and am now initializing the nas. Hope the problem is gone now but it wastes time to set all things up new.
edit:
i am now configuring qts and cpu is around 5%. looks good.
edit:
realtime mcafee scan pushes a little bit. at the moment, the nas is synchronizing the raid (raid 1, 2x 8tb wd red) and running mcafee realtime scan and manual scan: cpu usage is ~30%. i guess this is normal load for a Intel® Celeron® N3150/N3160 1.6 GHz quad-core processor. this cpu is not the strongest.
without manual scan, so only realtime scan and synchronizing, it is back at 10%.
i keep an eye on the cpu load the next days.
this nas will only be a iscsi target and share target, and will run only nakivo backup and replication for esxi hosts, no other work to do.
Post Reply

Return to “System & Disk Volume Management”