CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
User avatar
amigoccs
Starting out
Posts: 38
Joined: Sun Aug 11, 2013 5:21 pm
Contact:

CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by amigoccs » Sun May 14, 2017 10:50 pm

Hi,

This a summarize of this mining malware in Taiwan from 2017/4/28. Here is a short version about my post. Please read Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program for detail.

1. What happen

A mining program is injected to using your NAS to work for mineXMR.com using your NAS to work for mineXMR.com.

According to the [Connection Details] section in [Get Started] tab on mineXMR.com, port 4444 is for Low end CPU. And also the [Mining Apps] section, this program should be CPUMiner (forked by LucasJones & Wolf) which is available on GitHub: OhGodAPet/cpuminer-multi. In the README.md file, it is x86-64 only.

2. How It Hacks

On 2017/5/13, netgear54 shared his hacking forensic investigation on disk_manage.cgi hogging CPU usage. It’s a command injection through Photo Station. Read the detail in Security Vulnerability Addressed in Photo Station 5.4.1 and 5.2.7.

3. How to Identify if CPUMiner is Running on my NAS

3.1 High CPU Utilization

You probably will see high CPU utilization (30% or higher) in [Control Panel] → [System Settings] → [System Status] → [Resource Monitor] → [CPU usage] tab when low network access.

But if your are using QTS 4.3.3, don't get fooled with the [Resource Monitor] gadget in [Dashboard] which may be launched by upper right corner. It's not always updated automatically.

3.2 Strange Running Process

Enable Allow SSH connection in [Control Panel] → [Network Services] → [Telnet/SSH] tab and login as admin and search for process disk_manage.cgi. If /mnt/HDA_ROOT/disk_manage.cgi is found, you probably get infected. Check schedule tasks in next section.

disk_manage.cgi is a standard QTS program but /mnt/HDA_ROOT/disk_manage.cgi isn’t. It’s a fake with the same name to fool you.

There are actually 3 suspicious processes running in the background:

a. /mnt/HDA_ROOT/disk_manage.cgi
b. /mnt/HDA_ROOT/qwatchdogd.cgi
c. /mnt/HDA_ROOT/rcu_shed.cgi

3.3 Strange Schedule Program

ssh login to QTS as admin and search for schedule task rcu_shed. If /mnt/HDA_ROOT/rcu_shed is found, you probably get infected.

4. Patch Photo Station

Because we know how it hacks, the first step is to upgrade to the latest Photo Station 5.4.1 (for QTS 4.3.x) and 5.2.7 (for QTS 4.2.x). Use App Center to update current version or download the latest from Photo Station page.

5. Remove Malware

5.1 Kill the Process

[~] # kill -KILL PID_OF_/mnt/HDA_ROOT/disk_manage.cgi
[~] # kill -KILL PID_OF_/mnt/HDA_ROOT/qwatchdogd.cgi
[~] # kill -KILL PID_OF_/mnt/HDA_ROOT/rcu_shed.cgi

5.2 Stop Auto-reload

To stop reload the mining program, remove "*/3 * * * * /mnt/ext/opt/apache/bin/php /mnt/HDA_ROOT/rcu_shed" in crontab configuration. Use vi to load /mnt/HDA_ROOT/.config/crontab, delete following line, and overwrite.

Some report crontab -e doesn't work which I cannot confirm and have no ideas why.

5.3 Delete Mining Program and Related

Remember to delete disk_manage.cgi, rcu_shed, rcu_shed.json, and qwatchdogd in /mnt/HDA_ROOT/ at the end. There is no need to keep them.

5.4 Use QNAP Malware Remover

There is a [Malware Remover] in [App Center] in your QTS but not available on QNAP App Center page yet. The latest version 2.1.2 may remove this mining program and related files completely.

You may also download from here which is a direct link to QNAP. You need to unzip the download file and upload the QDK_2.2.14.qpkg in [App Center] in QTS.

[App Center] may be launch in [Main Menu] on upper left corner. [Malware Remover] can be found in [Utilities] category or just search for “malware”.

There is no interactive interface for this program. It just work in the background but you may read messages from it in [Control Panel] → [System Settings] → [System Logs]

Please read Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program to see samples of detect-and-remove and nothing-has-been-detectd.

5.5 Install Security Patch

Although this attack use a command injection through Photo Station, it's a good idea to check the latest security patch ASAP especially Security Vulnerabilities Addressed in QTS 4.2.3 Builds 20170121 and 20170124 and Security Vulnerabilities Addressed in QTS 4.2.4 Build 20170313.

Last words

If you are interested in How to Prevent from Command Injection, read it at Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program.

If you are interested in Malware Remover, read Detail Explain of QNAP Malware Remover 2.1.0

If you want to secure your NAS, read Synology Security Issue and How-to Harden your NAS

Just my two cents.

User avatar
OneCD
Ask me anything
Posts: 6535
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by OneCD » Mon May 15, 2017 12:33 am

amigoccs wrote:Some report crontab -e doesn't work which I cannot confirm and have no ideas why.

For QNAP NAS, the lack of 'crontab -e' is well-known. ;)

https://wiki.qnap.com/wiki/Add_items_to_crontab

production NAS: TS-569 Pro with Debian 10.2 'Buster' (power on/off times are < 1 minute)
backup NAS: TS-559 Pro+ with QTS 4.2.6 #20191107

one.cd.only@gmail.com

Image Image Image Image

User avatar
amigoccs
Starting out
Posts: 38
Joined: Sun Aug 11, 2013 5:21 pm
Contact:

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by amigoccs » Tue May 16, 2017 11:46 pm

OneCD wrote:
amigoccs wrote:Some report crontab -e doesn't work which I cannot confirm and have no ideas why.

For QNAP NAS, the lack of 'crontab -e' is well-known. ;)

https://wiki.qnap.com/wiki/Add_items_to_crontab


Hi OneCD,

Thank you for your explain!

I have update your sharing in "Update on 2017/5/16 about "crontab -e" section in Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program.

Thank you very much!

User avatar
amigoccs
Starting out
Posts: 38
Joined: Sun Aug 11, 2013 5:21 pm
Contact:

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by amigoccs » Wed May 17, 2017 12:50 am

Hi all TS-269H users,

It seems there are some issue to TS-268H users once they install Malware Remover 2.1.2. Here is the story from my Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program:

TS-269H User with Malware Remover 2.1.2 Issue and Solutions

In [求救] QNAP 中咗malware on HKEPC , TS-269H users reports high CPU utilization after installing/upgrading to the latest Malware Remover 2.1.2. Because of this, everything becomes extremely slow.

The solutions is to roll back to 2.1.1. tcbyxx shared his experience in #63 of this post as below:

1. Get Malware Removal 2.1.1 on QNAP.
2. Open the URL to your TS-269H. You probably won't be able to see it due to busy CPU.
3. Use power button on TS-269H to turn off.
4. Press power to turn on.
5. Stay close with the URL or reload until you see the login page.
6. Login immediately.
7. Launch App Center to remove Malware Remover 2.1.2 ASAP.
8. Reboot your TS-269H again immediately.
9. Login and install Malware Removal 2.1.1 in App Center.

By the time you read this section, QNAP should restore 2.1.1 for TS-269H. You probably won't see 2.1.2 in your App Center.

Update at 2017/5/17 12:03

It has been reported by tcbyxx in #69 of this post that Malware Removal 2.1.3 is available to download which is not available in release note of Malware Remover yet. It has been tested by stevencheuk that everything goes back normal.

Therefore, you may try to install Malware Removal 2.1.3 in step 7 or 9.

Wish it helps!

User avatar
amigoccs
Starting out
Posts: 38
Joined: Sun Aug 11, 2013 5:21 pm
Contact:

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by amigoccs » Wed May 17, 2017 2:47 am

Hi,

I just add updates with the new Malware Remover 2.1.2, 2.1.3 in Detail Explain of QNAP Malware Remover 2.1.0.

1. Update: 2.1.2 Removes another Malware
2. Update: 2.1.3 is a Fix for TS-269H Only

Have a nice dream!

Danniello
New here
Posts: 5
Joined: Sat Apr 12, 2014 9:11 pm

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by Danniello » Thu May 18, 2017 5:03 am

Great. Just great...
Yesterday I found suspicious catalogue /homes/admin/.tor/ with Tor cache files constantly updated.

What? My QNAP infected?!?
I thought that I did configuration quite OK. Only port 22 was forwarded to QNAP. Two-factor authentication enabled. Disks fully encrypted. E-mail alerts enabled. One big mistake - IP ban "temporary" disabled, but probably it would not prevent infection...
Also I disabled multimedia, but I didn't removed Video/Photo and other Stations... Probably all multimedia functions should be disabled AND stations uninstalled! https://QNAPNAS01/photo/ is accessible even if multimedia is disabled (Photo Station message that it is in offline mode)! Generic error will be only after remove Photo Station...

Lesson learned. Remove everything that is not used. No external access. Eventually external access enable only from trusted IP or enable it only temporary (and after that check system integrity).

PS. Next time better do it yourself - small PC with normal Linux with data stored on separate RAID disks, so I could update/format/change system without touching data. I do not need fancy web interface with plenty of !@#$ Stations that are opening more and more vulnerabilities even if they are "disabled"...

User avatar
amigoccs
Starting out
Posts: 38
Joined: Sun Aug 11, 2013 5:21 pm
Contact:

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by amigoccs » Thu May 18, 2017 8:30 pm

Hi Danniello,

Danniello wrote:Yesterday I found suspicious catalogue /homes/admin/.tor/ with Tor cache files constantly updated.


Did you use Download Station?

What? My QNAP infected?!?
Danniello wrote:I thought that I did configuration quite OK. Only port 22 was forwarded to QNAP. Two-factor authentication enabled. Disks fully encrypted. E-mail alerts enabled. One big mistake - IP ban "temporary" disabled, but probably it would not prevent infection...


Port 22 is for ssh

I think you have done a log on port-level and authentication.

In one of my study "Synology Security Issue and How-to Harden your NAS", QTS doesn't have iptables. Instead, you are using a TCP Wrapper.

"...I test with iptable in command line. It seems they didn’t enable kernel compiling for iptable. You need to read to follow this discussion thread and install it yourself..."

From my personal experience working as a channel sales in McAfee distributor and project manager in IP-guard, it's far from enough to prevent attack from Internet. You need not only a firewall but also a web application firewall (aka WAF). And make sure you use a harden linux. Next you need to regularly audit all logs. If you have intrusion detection system (IDS) or intrusion prevention system (IPS), you need to analyze them to find any suspicious attack pattern.

I setup my TS-251a as a business machine which will disable or don't install many packages by default. I also use it within intranet because I don't have enough protection.

You might be interested in "Find out more Available Service by your NAS" to get a nmap to scan your server.

This CPUMiner uses command injection to hack your NAS. Even if you have a firewall, it's still useless because it's a different level attach. It's like people living in 3-dimension world looking at everything in 2-dimension. OSI model has a very detail explain on them.

I have explain how to prevent command injection in "How to Prevent from Command Injection section" in Check And Solve If Your QNAP NAS Has been Injected a CPUMiner Program. But it's just PHP, there are more running in QTS.

I suggest you to use VPN to access via Internet which is more secure than exploring your web service to Internet. And remember to patch you VPN service regularly.

Just my two cents.

Danniello
New here
Posts: 5
Joined: Sat Apr 12, 2014 9:11 pm

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by Danniello » Fri May 19, 2017 9:04 pm

Thanks for the advice.

I do not why but very often I type 22, but in fact I mean https:)
Also I forgot about OpenVPN, so in fact I had open ports: 443 and 1194.

Perhaps it is solution - open external access only for OpenVPN. In fact it is "the real access". These fancy QTS website I used only to "fast" access when there is no time for VPN, etc.
OpenVPN also could be compromised, but probably it is better secured than QTS web access.

User avatar
karlegas
Know my way around
Posts: 120
Joined: Thu Apr 26, 2012 2:38 pm
Contact:

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by karlegas » Sat May 20, 2017 12:54 am

Hi I have a TS-212 and the malware remover detects something that doesn't be the malware that you are talking, my firmware version is QTS 4.2.4 not the 4.2.5 so I think is a different version.

Please see the picture below

2017-05-19_09-40-11.png


Please advise, thanks

Karl
You do not have the required permissions to view the files attached to this post.
Favorite Apps: Deluge - Plex, Twonky and TVmobili streaming
Models: TVS-463 QTS 4.3.4 / TS-221 QTS 4.3.3
Website: http://www.karlegas.com Blog
Website: http://www.naseros.com Tutorials and News in Spanish for QNAP NAS

dolbyman
Guru
Posts: 15245
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by dolbyman » Sat May 20, 2017 1:07 am

I would ask qnap via ticket

User avatar
amigoccs
Starting out
Posts: 38
Joined: Sun Aug 11, 2013 5:21 pm
Contact:

Re: CPUMiner being Injected through Vulnerable Photo Station on QNAP x86 Models

Post by amigoccs » Sat May 20, 2017 12:12 pm

karlegas wrote:Hi I have a TS-212 and the malware remover detects something that doesn't be the malware that you are talking, my firmware version is QTS 4.2.4 not the 4.2.5 so I think is a different version.


Hi Karl,

Don't worry. I have read your log and it removes a malware found before CPUMiner.

Because your QTS has been detected with the malware, it's a good idea to upgrade your firmware to the latest 4.2.x or 4.3.x and install NAS-201705-04 and NAS-201705-12 in Security Bulletins and Advisories.

I have check QTS-212 and there is only one "4.2.4 build 20170313". Is this the firmware you are using now? If there is any concern that you refused to upgrade to the latest 4.2.x or 4.3.x, I think it's better to reinstall your firmware and install NAS-201705-04, NAS-201705-12 and NAS-201703-21 in Security Bulletins and Advisories. This should give you a clean QTS with latest security patch. Is this okay for you?

Just for your information, Malware Remover 2.1.2 has some problems with TS-269H and release 2.1.3 to fix. You TS-212 only need 2.1.2 and should work fine.

I also write Detail Explain of QNAP Malware Remover 2.1.x to help you understand how it works if you are uncomfortable with Malware Remover.

Wish it helps!

Post Reply

Return to “System & Disk Volume Management”