disk_manage.cgi hogging CPU usage

[JarnoVanDerLinden]

I believe it's the exploit described by https://www.qnap.com/en/support/con_show.php?cid=109.

[Hulli]

Question for you. Do you find any file called "disk_manage.cgi" under /mnt/HDA_ROOT ?? Also, ssh to your server, perform a process list command "ps -ef", what do you find with disk_manage.cgi?

Hi I have exact this file there.
I have had a close look to the file and find some readable text in it that included some words like minerserver etc etc.
So yes I think this is the file which do the high processor usage.

I create a copy of this file to my PC and deletet the file on the NAS.
After NAS reboot processor usage was back to normal.

Do I have to check something addidtionaly??

brgds

Hulli


My config : TVS663 with QTS 4.3.3.0154

[Hulli]

Hi,
I think only deleting the file is not enough.
I saw that I have now 12 Zombie processes with disk_manage.cgi...
Any idea how to get rid of this??

brgds
Hulli

[dolbyman]

backup your data (if not done already) and start your nas from scratch

[Hulli]

Hi,

Thats exactly what I want to avoid...
I am searching an alternative solution.
Maybe someone​ can explain where the NAS has the autostart options to clean it up.

Brgds Hulli

[dolbyman]

viewtopic.php?f=25&t=131771#p609632

but you can never be sure if there is not other backdoors installed, a compromised system is better dealt with fire

[JarnoVanDerLinden]

When checking for disk_manage.cgi processes, make sure you look for the ones from /mnt/HDA_ROOT/, because there is a valid disk_manage.cgi used by the OS.

Also remove the crontab entry that periodically tries to launch the miner. You have to edit the /etc/config/crontab text file by deleting the line that looks like "*/3 * * * * /mnt/ext/opt/apache/bin/php /mnt/HDA_ROOT/rcu_shed"

From what I saw, this particular hack doesn't add anything to the autostart. It depends on crontab to launch it through php.

[Hulli]

I'm having the same issue. Looks like the disk_manage.cgi got started within the last 24 hours.
There is no autorun.sh present.
I'm fairly sure the admin password was not guessed.
TS-251A, 4.2.2 Build 20161214
I think there is an exploit somewhere.

I also just noticed that along with the disk_manage.cgi come qwatchdogd, rcu_shed and rcu_shed.json files in HDA_ROOT.

Further digging, crontab has gained an entry:
*/3 * * * * /mnt/ext/opt/apache/bin/php /mnt/HDA_ROOT/rcu_shed

same issue here.
the disk_manage.cgi seems to be the miner.
The rcu and qwatchdogd seems to be either a Tor accesspoint or a Tor gateway..

I deleted everything from the NAS and also killed the CRON Job.
Now it seems to be much better from the performance view.

I have still some processor issues with Perl5 20.1 task which takes ca. 20 to 25% CPU utilisation.
Do I need this perl programming envoirement?
Before the update to the new OS 4.3.3 there was no Perl on my NAS I assume....


brgds

Hulli

[Hulli]

seems that everything is back to normal after reboot...
processor runs between 3 and 8% which is normal within my envoirement.

So cross fingers that the full miner backdoor was eliminated.


brgds
Hulli

[amigoccs]

Hi,

This mining program sees spread in Taiwan from 2017/4/28 quickly and discuss in local community. I have collect related information and write a blog about this at Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program.

Here is a short version about my post. I didn't include the samples running linux commands on both NAS to save words. Because some reports the mining program is launched again after removal and firmware upgrade, I will update my post and follow up this issue.

1. What happen

A mining program is injected to using your NAS to work for mineXMR.com using your NAS to work for mineXMR.com.

According to the [Connection Details] section in [Get Started] tab on mineXMR.com, port 4444 is for Low end CPU. And also the [Mining Apps] section, this program should be CPUMiner (forked by LucasJones & Wolf) which is available on GitHub: OhGodAPet/cpuminer-multi. In the README.md file, it is x86-64 only.

2. How to Identify if CPUMiner is Running on my NAS

2.1 High CPU Utilization

You probably will see high CPU utilization (30% or higher) in [Control Panel] → [System Settings] → [System Status] → [Resource Monitor] → [CPU usage] tab when low network access.

But if your are using QTS 4.3.3, don't get fooled with the [Resource Monitor] gadget in [Dashboard] which may be launched by upper right corner. It's not always updated automatically.

2.2 Strange Running Process

Enable Allow SSH connection in [Control Panel] → [Network Services] → [Telnet/SSH] tab and login as admin and search for process disk_manage.cgi. If /mnt/HDA_ROOT/disk_manage.cgi is found, you probably get infected. Check schedule tasks in next section.

disk_manage.cgi is a standard QTS program but /mnt/HDA_ROOT/disk_manage.cgi isn’t. It’s a fake with the same name to fool you.

There are actually 3 suspicious processes running in the background:

a. /mnt/HDA_ROOT/disk_manage.cgi
b. /mnt/HDA_ROOT/qwatchdogd.cgi
c. /mnt/HDA_ROOT/rcu_shed.cgi


2.3 Strange Schedule Program

ssh login to QTS as admin and search for schedule task rcu_shed. If /mnt/HDA_ROOT/rcu_shed is found, you probably get infected.

3. Solution

3.1 Kill the Process

[~] # kill -KILL PID_OF_/mnt/HDA_ROOT/disk_manage.cgi
[~] # kill -KILL PID_OF_/mnt/HDA_ROOT/qwatchdogd.cgi
[~] # kill -KILL PID_OF_/mnt/HDA_ROOT/rcu_shed.cgi

3.2 Stop Auto-reload

To stop reload the mining program, remove "*/3 * * * * /mnt/ext/opt/apache/bin/php /mnt/HDA_ROOT/rcu_shed" in crontab configuration. Use vi to load /mnt/HDA_ROOT/.config/crontab, delete following line, and overwrite.

Some report crontab -e doesn't work which I cannot confirm and have no ideas why.

3.3 Get the Patch

Get all latest patch ASAP especially Security Vulnerabilities Addressed in QTS 4.2.3 Builds 20170121 and 20170124 and Security Vulnerabilities Addressed in QTS 4.2.4 Build 20170313.

3.4 Delete Mining Program and Related

Remember to delete disk_manage.cgi, rcu_shed, rcu_shed.json, and qwatchdogd in /mnt/HDA_ROOT/ at the end. There is no need to keep them.

3.5 Use QNAP Malware Remover

There is a [Malware Remover] in [App Center] in your QTS but not available on QNAP App Center page yet. The latest version 2.1.0 may remove this mining program and related files completely.

You may also download from here which is a direct link to QNAP. You need to unzip the download file and upload the QDK_2.2.14.qpkg in [App Center] in QTS.

[App Center] may be launch in [Main Menu] on upper left corner. [Malware Remover] can be found in [Utilities] category or just search for “malware”.

There is no interactive interface for this program. It just work in the background but you may read messages from it in [Control Panel] → [System Settings] → [System Logs]

Please read Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program to see samples of detect-and-remove and nothing-has-been-detectd.

Last words

It's not necessary to upgrade your firmware to 4.3.3 if yours is 4.2.x. Just apply the patch.

Please read Synology Security Issue and How-to Harden your NAS if you want to know how to secure your NAS.

Just my two cents.

[Trexx]

Thanks for the great write-up.

[amigoccs]

Hi all,

Because QNAP has release Malware Removal 2.1.0 r this issue, I have update my write-up. It's easier to use the app to remove manually.

I also write another post Detail Explain of QNAP Malware Remover 2.1.0 to explain what it contains. It is actually a shell script. There is no x86-64 specific programs. Therefore, you may install and run on any QNAP NAS without worry about x86-64/ARM model and firmware version.

Wish it helps!

Best regards,

Amigo

[amigoccs]

Update: 2.1.1 is released within 15 hours after 2.1.0. It adds schedule scan at 3:00AM everyday.

Detail comparison of the two version please see "Update: 2.1.1 Add To Scan at 3:00AM Everyday" section in the post.

FYI, there is new firmware available for 5.3.3 users. QTS 4.3.3 Build 20170503 provides security patch for:

- This build includes security fixes for multiple PHP vulnerabilities. The version of PHP running on the remote web server has been updated to 5.6.30.
- This build includes security fixes for a password change vulnerability (CVE-2017-7629).
- This build includes security fixes for a command injection vulnerability (CVE-2017-7876).

But I am not sure if it may stops the CPUMiner issue recently, recommend for 4.3.x users. For those who want to stay with 4.2.x, maybe you should check Security Bulletins and Advisories instead.

Just my two cents.

[netgear54]

My system was also infected with this minning tool. Looks like the coolprit was the "mighty" PhotoStation (Firmware 4.2.4 Build20170313).
The hacker got access by executing a series of commands using the folowing urls (in the order they were requested):

> [QNAP_Daniel] - content has been removed for security reasons

Now i did install/run the qnap malware remover and it looks like it deleted the files and the cron job added by the above commands, so hope that the changes done by the hacker are removed. In any case lesson learned: QNAP IS A PRIVATE LAN PRODUCT NOT MEANT FOR INTERNET EXPOSURE

[OneCD]

Some good info there.