TVS-673 shuts down periodically for no apparent reason

[fowlkes]

I purchased a TVS-673-64GB and configured it with Seagate IronWolf 10TB drives.
It is running almost no services besides nsfv4 and virtual switch configuration. It
has been spontaneously shutting itself down every 15-30 hours for the last week
despite installing latest firmware image (4.3.3.0299) and trying to get rid of any
extra cruft.

I've been loging the hal reported temperatures and kernel messages out to an
external log and can see clearly it isn't an issue of overheating. There is absolutely
zero indication in the kmsg log of any events correlated with it going down.

Has anyone else had a similar experience?

[Trexx]

No. I have not seen that out of mine. My initial thoughts are either power supply, thermal, or maybe memory.
I would open a helpdesk ticket with QNAP to start with.

Do you have nightly backup/snapshot jobs setup? How are your FANS speeds configured?

[fowlkes]

Hi Paul --

Thanks for the response. I have had a ticket open for 24+ hours now and
no response from QNAP.

No nightly backups/snapshot jobs. There is nothing in the default qnap logs
that is correlated with it shutting down. I have now had it shut down on me
several times and I know immediately because my NFS share hangs.

I've run with fans in both performance mode and regular. It has shutdown
in either case.


I have logged into the machine and left running a command like this:

while [ 0 ]; do ps aux > PSLOG.1; ps aux > PSLOG.2; ps aux > PSLOG.3; done

which is continuously dumping the list of currently running processes to
a log files in a round-robin style. When the system dies, I thus get a "snapshot"
of what was running at the time (basically one of the 3 files will get truncated
as the thing goes down).... but no useful information. There are no obvious
processes starting up that seem to be correlated with it shutting down
(at least as far as I can tell).

Below is the results of the last non-truncated process listing in case someone
happens to recognize something unusual.




----


PID Uid VmSize Stat Command
1 admin 1804 S init
2 admin SW [kthreadd]
3 admin SW [ksoftirqd/0]
5 admin SW< [kworker/0:0H]
7 admin SW [rcu_sched]
8 admin SW [rcu_bh]
9 admin SW [migration/0]
10 admin SW [migration/1]
11 admin SW [ksoftirqd/1]
12 admin SW [kworker/1:0]
13 admin SW< [kworker/1:0H]
14 admin SW [migration/2]
15 admin SW [ksoftirqd/2]
17 admin SW< [kworker/2:0H]
18 admin SW [migration/3]
19 admin SW [ksoftirqd/3]
20 admin SW [kworker/3:0]
21 admin SW< [kworker/3:0H]
22 admin SW< [khelper]
23 admin SW [kdevtmpfs]
24 admin SW< [netns]
27 admin SW< [perf]
471 admin SW< [writeback]
473 admin SW [kworker/1:1]
474 admin SWN [ksmd]
475 admin SW< [crypto]
476 admin SW< [kintegrityd]
477 admin SW< [bioset]
478 admin SW< [kblockd]
687 admin SW< [tifm]
693 admin SW< [ata_sff]
711 admin SW< [md]
728 admin SW< [cfg80211]
829 admin SW< [rpciod]
830 admin SW [kworker/2:1]
852 admin SW [kswapd0]
859 admin SW [fsnotify_mark]
860 admin SW [ecryptfs-kthrea]
862 admin SW< [nfsiod]
863 admin SW< [cifsiod]
897 admin SW< [kthrotld]
1507 admin SW< [nvme]
1547 admin SW< [kmpath_rdacd]
1554 admin SW [scsi_eh_0]
1558 admin SW< [scsi_tmf_0]
1574 admin SW [scsi_eh_1]
1575 admin SW< [scsi_tmf_1]
1576 admin SW [scsi_eh_2]
1579 admin SW< [scsi_tmf_2]
1580 admin SW [scsi_eh_3]
1583 admin SW< [scsi_tmf_3]
1584 admin SW [scsi_eh_4]
1586 admin SW< [scsi_tmf_4]
1605 admin SW [scsi_eh_5]
1606 admin SW< [scsi_tmf_5]
1609 admin SW [scsi_eh_6]
1610 admin SW< [scsi_tmf_6]
1611 admin SW [scsi_eh_7]
1612 admin SW< [scsi_tmf_7]
1617 admin SW [scsi_eh_8]
1618 admin SW< [scsi_tmf_8]
1636 admin SW [scsi_eh_9]
1637 admin SW< [scsi_tmf_9]
1638 admin SW [scsi_eh_10]
1640 admin SW< [scsi_tmf_10]
1668 admin SW [scsi_eh_11]
1671 admin SW< [scsi_tmf_11]
1674 admin SW [scsi_eh_12]
1675 admin SW< [scsi_tmf_12]
1707 admin SW [scsi_eh_13]
1708 admin SW< [scsi_tmf_13]
1711 admin SW [scsi_eh_14]
1712 admin SW< [scsi_tmf_14]
1715 admin SW [scsi_eh_15]
1716 admin SW< [scsi_tmf_15]
1719 admin SW [scsi_eh_16]
1720 admin SW< [scsi_tmf_16]
1760 admin SW [rc0]
1766 admin SW< [raid5wq]
1768 admin SW< [dm-block-clone]
1770 admin SW< [dm_bufio_cache]
1771 admin SW< [kmpathd]
1772 admin SW< [kmpath_handlerd]
1798 admin SW< [ttm_swap]
1799 admin SW [gfx]
1800 admin SW [comp_1.0.0]
1801 admin SW [comp_1.0.1]
1802 admin SW [comp_1.0.2]
1803 admin SW [comp_1.0.3]
1804 admin SW [comp_1.0.4]
1805 admin SW [comp_1.0.5]
1806 admin SW [comp_1.0.6]
1807 admin SW [comp_1.0.7]
1808 admin SW [sdma0]
1809 admin SW [sdma1]
1810 admin SW [uvd]
1811 admin SW [vce0]
1812 admin SW [vce1]
1813 admin SW< [dm_timer_queue]
1863 admin SW [kworker/0:2]
1916 admin SW< [deferwq]
2014 admin SW [kworker/2:2]
2182 admin SW< [bioset]
2183 admin SW< [kcopyd]
2184 admin SW< [bioset]
2185 admin SW< [kcopyd_tracked]
2186 admin SW< [bioset]
2191 admin SW< [bioset]
2192 admin SW< [drbd-reissue]
2197 admin SW [scsi_eh_17]
2198 admin SW< [scsi_tmf_17]
2199 admin SW [usb-storage]
2226 admin SW< [kworker/2:1H]
2227 admin SW< [kworker/3:1H]
2228 admin SW< [kworker/1:1H]
2230 admin SW< [kworker/0:1H]
2296 admin 1644 S < udevd --daemon
2401 admin SW< [bioset]
2402 admin SW [md9_raid1]
2413 admin SW [kjournald]
2429 admin SW< [bioset]
2430 admin SW [md13_raid1]
6663 admin 6656 S /sbin/hal_daemon -f
6725 admin SW< [bioset]
6726 admin SW [md256_raid1]
6762 admin SW< [bioset]
6763 admin SW [md1_raid6]
6774 admin SW [kworker/3:2]
6789 admin SW< [drbd1_submit]
6797 admin SW [drbd_w_r1]
6853 admin SW< [kdmflush]
6855 admin SW< [bioset]
6860 admin SW< [kdmflush]
6862 admin SW< [bioset]
6865 admin SW< [kdmflush]
6867 admin SW< [bioset]
6871 admin SW< [kdmflush]
6873 admin SW< [bioset]
6892 admin SW< [kdmflush]
6893 admin SW< [bioset]
6894 admin SW< [kcopyd]
6895 admin SW< [bioset]
6896 admin SW< [dm-thin]
6897 admin SW< [dm-convert-thin]
6898 admin SW< [dm-tier]
6899 admin SW< [dm-tier]
6900 admin SW< [dm-tier]
6901 admin SW< [dm-tier-discard]
6902 admin SW< [kcopyd]
6903 admin SW< [bioset]
6904 admin SW< [dm-tier-cache-t]
6905 admin SW< [bioset]
6911 admin SW< [kdmflush]
6913 admin SW< [bioset]
6919 admin SW< [kdmflush]
6921 admin SW< [bioset]
7002 admin 2140 S /usr/bin/portmap -h 127.0.0.1 -h 192.168.2.1
7085 admin 116 S /usr/sbin/rpc.rquotad -p 30002
7094 admin 7404 S /usr/sbin/rpc.mountd -p 30000 -F
7109 admin SW< [nfsd4]
7110 admin SW< [nfsd4_callbacks]
7120 admin SW [nfsd]
7121 admin SW [nfsd]
7122 admin SW [nfsd]
7123 admin SW [nfsd]
7124 admin SW [nfsd]
7125 admin SW [nfsd]
7126 admin SW [nfsd]
7127 admin SW [nfsd]
7131 admin SW< [kdmflush]
7133 admin 3316 S /usr/sbin/rpc.idmapd
7134 admin SW< [bioset]
7138 admin 4920 S /usr/sbin/rpc.statd -p 30001
7180 admin SW [jbd2/dm-0-8]
7181 admin SW< [ext4-rsv-conver]
7182 admin 3804 S /usr/sbin/sshd -f /etc/config/ssh/sshd_config -p 22
7207 admin 3848 S < /usr/local/apache/bin/fcgi-pm -k start -f /etc/apache-sys-proxy.conf
7268 admin 616 S /sbin/lvmetad
7335 admin SW [kworker/u8:0]
7390 admin SW [kjournald]
7422 admin 10204 S < /usr/local/apache/bin/apache_proxy -k start -f /etc/apache-sys-proxy.conf
7751 admin 3436 S /sbin/daemon_mgr.nvr
7754 admin 4216 S /usr/local/apache/bin/fcgi-pm -k start -f /etc/apache-sys-proxy-ssl.conf
7758 admin 10704 S /usr/local/apache/bin/apache_proxys -k start -f /etc/apache-sys-proxy-ssl.conf
7808 admin SW [notify thread]
7817 admin 760 S < qWatchdogd: keeping alive every 5 seconds...
8120 admin 1380 S /sbin/netwatchdog -d
8312 admin 648 S /sbin/modagent
8555 admin 3560 S /sbin/daemon_mgr
9197 admin 23888 S /mnt/ext/opt/Python/bin/python ./manage.py runfcgi method=threaded socket=/tmp/netmgr.sock pidfile=/tmp/netmgr.pid
9262 admin 3372 S /mnt/ext/opt/netmgr/util/redis/redis-server *:0
9326 admin 16168 S /mnt/ext/opt/Python/bin/python /mnt/ext/opt/netmgr/api/core/ip_monitor.py
9337 admin 13912 S /mnt/ext/opt/Python/bin/python /mnt/ext/opt/netmgr/api/core/redisVnicListener.py
9504 admin 9568 S /mnt/ext/opt/Python/bin/python /mnt/ext/opt/netmgr/api/core/nsd.py
10075 admin SW< [bond0]
10111 admin SW< [bond1]
11280 admin 9440 S N /mnt/ext/opt/netmgr/api/core/dhcpdLink/dhcpd_eth4 -cf /etc/dhcpd_eth4.conf -lf /var/state/dhcp/dhcpd_eth4.leases -pf /mnt/ext/opt/netmgr/api/core/dhcpdLink/eth4.pid
13979 admin SW< [iscsi_eh]
14005 admin SW [qnap_et]
14019 admin 1440 S /sbin/iscsid --config=/etc/config/iscsi/sbin/iscsid.conf --initiatorname=/etc/iscsi/initiatorname.iscsi
14020 admin 2724 S < /sbin/iscsid --config=/etc/config/iscsi/sbin/iscsid.conf --initiatorname=/etc/iscsi/initiatorname.iscsi
14028 admin 5244 S /sbin/vdd_control -d
14177 admin 2384 S /bin/sh /usr/local/mariadb/bin/mysqld_safe --defaults-file=/usr/local/mariadb/my-mariadb.cnf --basedir=/usr/local/mariadb --datadir=/share/CACHEDEV1_DATA/.system/data --user=admin --default-storage-engine=MyISAM --wait_timeout=2628000 --pid-file=/var/lock/mariadb.pid --tmpdir=/share/CACHEDEV1_DATA/.system/tmp
14574 admin 61096 S /usr/local/mariadb/bin/mysqld --defaults-file=/usr/local/mariadb/my-mariadb.cnf --basedir=/usr/local/mariadb --datadir=/share/CACHEDEV1_DATA/.system/data --plugin-dir=/usr/local/mariadb/lib/plugin --user=admin --default-storage-engine=MyISAM --wait-timeout=2628000 --tmpdir=/share/CACHEDEV1_DATA/.system/tmp --log-error=/share/CACHEDEV1_DATA/.system/data/atlas.err --pid-file=/var/lock/mariadb.pid --socket=/tmp/mysql_mediadb.sock --port=3310
14994 admin 7756 S /sbin/qpkgd -d0
15043 guest 2292 S /usr/sbin/dbus-daemon --system
15527 admin 4704 S /usr/local/sbin/Qthttpd -p 80 -nor -nos -u admin -d /home/Qhttpd -c **.*
15553 admin 3312 S /usr/sbin/cupsd -C /etc/config/cups/cupsd.conf -s /etc/config/cups/cups-files.conf
15788 admin 4696 S /usr/local/sbin/_thttpd_ -p 58080 -nor -nos -u admin -d /home/httpd -c **.* -h 127.0.0.1 -i /var/lock/._thttpd_.pid
16058 admin 12172 S php-fpm: master process (/etc/php-fpm-sys-proxy.conf)
16059 admin 8412 S php-fpm: pool www
16060 admin 8412 S php-fpm: pool www
16108 admin 6960 S < /usr/local/apache/bin/apache_proxy -k start -f /etc/apache-sys-proxy.conf
16500 admin 6444 S /usr/local/sbin/remote_folder_daemon --reset
16797 admin 3284 S /usr/sbin/ntpdated
16905 admin 5752 S /usr/sbin/upsutil
17019 admin 9732 S /usr/local/apache/bin/apache_proxys -k start -f /etc/apache-sys-proxy-ssl.conf
17294 admin 1756 S /usr/sbin/crond -l 9 -c /tmp/cron/crontabs
17456 admin 3496 S /usr/bin/lunportman
17528 admin 7460 S /sbin/bcclient
17777 admin 1516 S N /sbin/acpid
17808 admin 3280 S /usr/sbin/rsyslogd -f /etc/rsyslog.conf -c4 -M /usr/local/lib/rsyslog/
17826 admin 656 S /sbin/syslog_maild -d
17872 admin 2552 S /sbin/gen_bandwidth -r -i 5
17970 admin 4728 S /sbin/lcdmond
18066 admin 1736 S qNoticeEngined: Write notice is enabled...
18068 admin 4892 S qLogEngined: Write log is enabled...
18077 admin 2440 S /bin/sh /etc/init.d/klogd.sh start
18082 admin 3836 S /sbin/qsyslogd
18087 admin 4032 S /sbin/qShield
18158 admin 928 S /bin/dd if /proc/kmsg of /mnt/HDA_ROOT/.logs/kmsg bs 1 count 1024000
18209 admin SW [kworker/u8:2]
19046 admin 4668 S /sbin/sdmd --daemon
19087 admin 6604 S /usr/bin/qsnapman
19135 admin 4080 S /usr/local/bin/qb_daemon -d 7
19353 admin Z [lock_timer.sh]
19354 admin Z [detect_defaultg]
19383 admin 5096 S /usr/bin/qsyncman
19987 admin 6552 S /usr/bin/qsnapman-alive
20214 admin 6892 S /usr/bin/qsnapman-recyc
20329 admin 992 S /bin/sleep 1
20431 admin 9736 S /usr/share/qnas_console_install/drawPic
20433 admin Z [sh]
20471 admin 2348 R ps aux
20722 admin 3740 S /bin/sh /sbin/nas_soldier
20783 admin SW [kworker/u8:4]
20924 admin 6816 S /sbin/upnpcd -i 300
21023 admin 1732 S /sbin/getty 115200 tty1
21024 admin 1988 S /sbin/getty 115200 tty2
21027 admin 1988 S /sbin/getty -L ttyS0 115200 vt100
21435 admin 19844 S /mnt/ext/opt/Python/bin/python /mnt/ext/opt/netmgr/api/core/asd.py
23999 admin SW [kworker/u8:1]
26324 admin SW [kworker/0:0]
26335 admin 8140 S sshd: admin@pts/0
26354 admin 11336 S -sh

[Trexx]

When you boot back up do you see log errors about unexpected shutdown or unclean file system? If you aren’t then it wouldn’t appear to be a “hard/crash” type shutdown.

I am wondering if either you have a sleep option configured somewhere or maybe a flakey UPS is causing it to shutdown.

What’s you Qnap Helpdesk ticket #?


Sent from my iPad using Tapatalk

[fowlkes]

My ticket# is GXH-325-82168

The file system isn't clean when I power it back up... fortunately
have never had the raid get inconsistent.

Disk standby mode is off
WoL disabled
Power recovery is "resume previous state"
Power schedule is not enabled
No UPS

Not sure where sleep would appear in config but here is the crontab

-----

# m h dom m dow cmd
0 3 * * * /usr/local/sbin/ImR_all -soft /Qmultimedia
0 2 * * * /sbin/qfstrim
0 3 * * 0 /etc/init.d/idmap.sh dump
0 4 * * * /sbin/hwclock -s
0 3 * * * /sbin/vs_refresh
0 3 * * * /sbin/clean_reset_pwd
0-59/15 * * * * /etc/init.d/nss2_dusg.sh
30 7 * * * /sbin/clean_upload_file
0-59/10 * * * * /etc/init.d/storage_usage.sh
30 3 * * * /sbin/notice_log_tool -v -R
*/10 * * * * /sbin/config_cache_util 0
0-59/20 3 * * * /sbin/adjust_time
10 15 * * * /usr/bin/power_clean -c 2>/dev/null
0 0 * * 1 /sbin/hal_event --pd_self_test dev_id=0x00000008,action=1
0 0 * * 1 /sbin/hal_event --pd_self_test dev_id=0x00000007,action=1
0 0 * * 1 /sbin/hal_event --pd_self_test dev_id=0x00000006,action=1
0 0 * * 1 /sbin/hal_event --pd_self_test dev_id=0x00000005,action=1
0 0 * * 1 /sbin/hal_event --pd_self_test dev_id=0x00000004,action=1
0 0 * * 1 /sbin/hal_event --pd_self_test dev_id=0x00000003,action=1
0 9,21 * * * /sbin/notify_update -m -s -p 1>/dev/null 2>&1
* * * * * /var/cache/netmgr/lock_timer.sh
* * * * * /var/cache/netmgr/detect_defaultgw_internet.sh
4 3 * * 3 /etc/init.d/backup_conf.sh
48 20 * * * /mnt/ext/opt/QcloudSSLCertificate/bin/ssl_agent_cli

[Trexx]

I would do a couple things Install Qnap’s malware remover app just to help rule that out as a possibility.

Also install Qnap’s diagnostic tool. Once that is installed, dump download logs, and then run a memory test.


Sent from my iPad using Tapatalk

[fowlkes]

I've dumped the logs with the diagnonstic tool several times, nothing apparent.

memtest86+ managed 1 pass, running it again in a different mode.