random reboot issues

Questions about SNMP, Power, System, Logs, disk, & RAID.
NicolasK
First post
Posts: 1
Joined: Mon Sep 23, 2019 4:55 pm

Re: random reboot issues

Post by NicolasK »

Hi Fellas

Just wanted to mention that the problem is still there in the official release. I upgraded to 4.4.1, what caused a raid resync and random reboots every 4 to 1 hours. I first believed I had a faulty drive, but how high is the probability of that happening exactly after a firmware upgrade? I did a firmware downgrade (without reset, just installing) and my NAS (TS-453A) is totally stable again. No data loss occured. I just had to un- and reinstall 3 applications (including VM), but their config was still intact afterwards.

For those who have the problem that crashing appears to often to downgrade: I observed that giving load on disks and cpu can provoke the crash. Deactivate all background apps and set raid rebuild priority to the "service first" setting in the storage manager. That got me at least 2 hours between reboots.

Best
Nicolas

P.S.: I reported to helpdesk after the downgrade, but they asked me to reinstall 4.4.1. because I sent a log after downgrade, but I will definately NOT do that :)
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: random reboot issues

Post by Moogle Stiltzkin »

NicolasK wrote: Mon Sep 23, 2019 5:15 pm Hi Fellas

Just wanted to mention that the problem is still there in the official release. I upgraded to 4.4.1, what caused a raid resync and random reboots every 4 to 1 hours. I first believed I had a faulty drive, but how high is the probability of that happening exactly after a firmware upgrade? I did a firmware downgrade (without reset, just installing) and my NAS (TS-453A) is totally stable again. No data loss occured. I just had to un- and reinstall 3 applications (including VM), but their config was still intact afterwards.

For those who have the problem that crashing appears to often to downgrade: I observed that giving load on disks and cpu can provoke the crash. Deactivate all background apps and set raid rebuild priority to the "service first" setting in the storage manager. That got me at least 2 hours between reboots.

Best
Nicolas

P.S.: I reported to helpdesk after the downgrade, but they asked me to reinstall 4.4.1. because I sent a log after downgrade, but I will definately NOT do that :)
i know right it's insane.

i also reported random reboot and sent them the dump logs BEFORE i downgraded. seems thats not enough.

so they told me to install 4.4.1 again....

but the weird part is, no mention to perform backup PRIOR to upgrading back to 4.4.1 that WILL cause random reboot = raid resync + raid rebuild etc (you wanna chance your data that something will not manage to recover right?)

upgrade back to 4.4.1 to troubleshoot fine, but they should have at least mentioned to backup first :S ....


But anyway i'll friggin do it just so i can get 4.4.1 bug free for ownself, but i told i'm only going to do it AFTER I BACKUP. so they got to wait.

Last time i took all the time and trouble to help with this 4.4.1 random reboot issue, so i'm still not happy about that experience i must say :(



Kudos by the way for making that link. Maybe now we can get to the bottom of what is causing this issue. My troubleshoot merely was able to place the blame on qts 4.4.1 and not on my hardware (since 4.3.6 was rock stable). so it's good you managed to find out more about this.
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: random reboot issues

Post by Moogle Stiltzkin »

NicolasK wrote: Mon Sep 23, 2019 5:15 pm
For those who have the problem that crashing appears to often to downgrade: I observed that giving load on disks and cpu can provoke the crash. Deactivate all background apps and set raid rebuild priority to the "service first" setting in the storage manager. That got me at least 2 hours between reboots.
i'll also add, perhaps it's better also add Bitmap

What is "Enable Bitmap" on my Qnap NAS? Will it allow me to recover data in case of failures?
"Bitmap" is used to allow faster RAID rebuild times, but you will be able to recover without it if you have a proper RAID level enabled.
When an array has a write-intent bitmap, a spindle (a device, often a hard drive) can be removed and re-added, then only blocks changes since the removal (as recorded in the bitmap) will be resynced.

Therefore a write-intent bitmap reduces rebuild/recovery (md sync) time if:

-the machine crashes (unclean shutdown)
-one spindle is disconnected, then reconnected
-If one spindle fails and has to be replaced, a bitmap makes no difference.

A write-intent bitmap:

-does not improve performance
-may cause a degradation in write performance, it varies upon:
(A)the size of the chunk data (on the RAID device) mapped to each bit in the bitmap, as expressed by cat /proc/mdstat. The ratio (bitmap size / RAID device size )
(B)workload profile (long sequences of writes are more impacted, as spindle heads go back and forth between the data zone and the bitmap zone)
-can be removed/added at any time
https://serverfault.com/questions/32118 ... y-qnap-nas


otherwise those rebuilds take very long. mine took 13 hours :S

but the better option is simply to downgrade to 4.3.6 when you begin to notice random reboots occuring on 4.4.1
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
paulsey
First post
Posts: 1
Joined: Tue Jul 09, 2019 9:40 pm

Re: random reboot issues

Post by paulsey »

I too have been experiencing random reboots on my TS-451+ 8GB. I upgraded to 4.4.1 pretty much 'straight out of the box', so it had very little in terms of stored data and installed apps and was not a pleasent first experience. The reboots often would coincide with high usage activity (CPU/RAM/Disk), but I also had reboots at 4:30am a couple of days in a row when no-one was on or using the device!! I pretty soon got sick of restarting containers and doing disk checks on 'unclean disks' so I have this morning performed the downgrade to 4.3.6 with much the same experience as described by fellow posters - needed to remove and reinstall virtual station, media streaming add-on, but what little data was on the storage remained in tact. Too early to tell if this will fix the random renoots, but fingers crossed (sad to have to say that!!). Dissappointing that a firmware gets released to production in such a poor state!!
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: random reboot issues

Post by Moogle Stiltzkin »

paulsey wrote: Fri Sep 27, 2019 2:18 pm ...
don't forget to sign up here :(
viewtopic.php?f=142&t=150707&start=15
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
Ramrunner
New here
Posts: 9
Joined: Tue Jul 07, 2015 1:01 pm

Re: random reboot issues

Post by Ramrunner »

Add me to your list guys - unfortunately only allowed my QNAP to update to 4.1.1 three days ago. God I wish I'd read your comments first. Nothing but reboots between 8-29 hours apart TS-453A for me if that helps any. I've disabled the ONLY third party app I have running - Serviio, but not much success getting this thing stable. Mine is COMPLETELY STOCK wrt RAM, the disks are on the compatibility list, the thing was ROCK STABLE before the upgrade. It's had plenty of upgrades in the years I've had it, but it's NEVER done anything like this!!!

Put the help desk request in, commented on it myself over the next 3 days to give more information every reboot - to no avail. QNAP have been silent for me so far.

Someone want to post a nice downgrade procedure - which firmware to grab, what to uninstall/reinstall before/after, etc?

I've never had to do this before, shouldn't have to do it, but it seems QNAP aren't helping me so I have to try something before my data craps itself.

For now it only has unclean volume every time which seem to fix every time OK.
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: random reboot issues

Post by Moogle Stiltzkin »

Ramrunner wrote: Wed Oct 09, 2019 12:15 pm the thing was ROCK STABLE before the upgrade. It's had plenty of upgrades in the years I've had it, but it's NEVER done anything like this!!!
welcome to the club Image

just to confirm you got this issue on QTS 4.4.1.1081 build 20191005 ?

i added u to the list here
viewtopic.php?f=142&t=150707



for me, my issue sort of went away. qnap did do a remote session, but when i asked what did they do to fix it, they said they did nothing. so i'm completely clueless how come my issue sort of resolved itself.

it is at least not random rebooting every few hours and days now.

I'm still testing if it can get through a month since boot up without any random reboots whatsoever.


i haven't heard from any other users with this issue that had similar success like me :' my guess is they all went back to 4.3.6 to ride this one out....
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
matlue
New here
Posts: 4
Joined: Wed Jul 20, 2016 11:26 pm

Re: random reboot issues

Post by matlue »

Same problem here with 4.4.1.1081 and TVS-663.
Just happened during volume unlocking, before that restart instead of sleep mode.
That's really annoying, hope they will fix it soon.
mlai
Starting out
Posts: 41
Joined: Wed Jul 15, 2009 9:16 pm

Re: random reboot issues

Post by mlai »

I am having the random reboot on my TS453B as well, but not on my old TS453pro and TVS682.
On the TS453B, random reboot chances are higher when the system is stressed with higher CPU loads/Disk activities. For example, when generating large number of thumbnails, QuMagic Face/Object recognition, or even HBS3 qdedup backups.

But on TS453pro (which is considerably older) and the TVS682 (modded with an i5 cpu and 24GB RAM), they are stable with QTS 4.4.1 1081. Wondering if the random reboot issue is model specific......
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: random reboot issues

Post by Moogle Stiltzkin »

NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
dsduarte
Getting the hang of things
Posts: 73
Joined: Fri Oct 11, 2019 2:28 am

Re: random reboot issues

Post by dsduarte »

I'm having this problem here with my new TS-235Be. I just bought it and seted it up for the first time rigth on oct/05, when 4.4.1.1081 was released. During set up process, system prompted me to upgrade and I agreed. Got several reboots in short amount of time.
I though that was a hard drive problem because one of my drives was alerting a SMART problem, sugesting to replace it.
Searching other users posts, I found an user who sugested downgrade... so I did it and got no more reboots....
Now I have sent logs to QNAP help desk and hoping that they find a solution soon to discart a defective unit.
QNAP TS-253Be
QNAP TS-251+
User avatar
Moogle Stiltzkin
Guru
Posts: 11448
Joined: Thu Dec 04, 2008 12:21 am
Location: Around the world....
Contact:

Re: random reboot issues

Post by Moogle Stiltzkin »

dsduarte wrote: Fri Oct 11, 2019 2:35 am ....
yeah you may want to replace problematic hdd. but the effect that downgrading solves the random reboot tells me though that this is the same as the qts 4.4.1 random reboot some of us are experiencing
viewtopic.php?f=142&t=150707&p=727628#p727628


since you're new, sometimes (very rarely) there are bad qts builds that may cause a random reboot. apparently this is one of those times.

so it's not necessarily that the hardware is bad (the fact that your issue goes away when you downgraded back to 4.3.6 implies as much).

as such it would be premature to RMA just yet since helpdesk MAY release a future qts that solves this issue. I in fact had prematurely rma some ram which i had initially thought at first was the culprit. i should have tested downgrading first :S
Last edited by Moogle Stiltzkin on Fri Oct 11, 2019 6:40 am, edited 1 time in total.
NAS
[Main Server] QNAP TS-877 (QTS) w. 4tb [ 3x HGST Deskstar NAS & 1x WD RED NAS ] EXT4 Raid5 & 2 x m.2 SATA Samsung 850 Evo raid1 +16gb ddr4 Crucial+ QWA-AC2600 wireless+QXP PCIE
[Backup] QNAP TS-653A (Truenas Core) w. 4x 2TB Samsung F3 (HD203WI) RaidZ1 ZFS + 8gb ddr3 Crucial
[^] QNAP TL-D400S 2x 4TB WD Red Nas (WD40EFRX) 2x 4TB Seagate Ironwolf, Raid5
[^] QNAP TS-509 Pro w. 4x 1TB WD RE3 (WD1002FBYS) EXT4 Raid5
[^] QNAP TS-253D (Truenas Scale)
[Mobile NAS] TBS-453DX w. 2x Crucial MX500 500gb EXT4 raid1

Network
Qotom Pfsense|100mbps FTTH | Win11, Ryzen 5600X Desktop (1x2tb Crucial P50 Plus M.2 SSD, 1x 8tb seagate Ironwolf,1x 4tb HGST Ultrastar 7K4000)


Resources
[Review] Moogle's QNAP experience
[Review] Moogle's TS-877 review
https://www.patreon.com/mooglestiltzkin
relyonit
Starting out
Posts: 37
Joined: Mon Jun 10, 2013 7:59 pm

Re: random reboot issues

Post by relyonit »

I’m done with QNAP. they release dodgy firmware and tbh support isn’t up to much. Anyone else returning their hardware?
mlai
Starting out
Posts: 41
Joined: Wed Jul 15, 2009 9:16 pm

Re: random reboot issues

Post by mlai »

mlai wrote: Thu Oct 10, 2019 11:19 am I am having the random reboot on my TS453B as well, but not on my old TS453pro and TVS682.
On the TS453B, random reboot chances are higher when the system is stressed with higher CPU loads/Disk activities. For example, when generating large number of thumbnails, QuMagic Face/Object recognition, or even HBS3 qdedup backups.

But on TS453pro (which is considerably older) and the TVS682 (modded with an i5 cpu and 24GB RAM), they are stable with QTS 4.4.1 1081. Wondering if the random reboot issue is model specific......
I removed a 8GB RAM module from my TS453B (so running on a single 8GB RAM module at the moment) and everything seems to be stable.
So it can be QTS 4.4.1 having a problem with over 8GB RAM on the TS453B (which officially only supports 8GB max) or there is some problem with one of the RAM slots......
Ramrunner
New here
Posts: 9
Joined: Tue Jul 07, 2015 1:01 pm

Re: random reboot issues

Post by Ramrunner »

Moogle Stiltzkin wrote: Wed Oct 09, 2019 12:41 pm for me, my issue sort of went away. qnap did do a remote session, but when i asked what did they do to fix it, they said they did nothing. so i'm completely clueless how come my issue sort of resolved itself.

it is at least not random rebooting every few hours and days now.

I'm still testing if it can get through a month since boot up without any random reboots whatsoever.

i haven't heard from any other users with this issue that had similar success like me :' my guess is they all went back to 4.3.6 to ride this one out....
I'm seeing some stability now myself.

The ONLY things I've disabled that I would normally be running are JRE8 and Serviio (JRE8 is of course required by Serviio)

No reboots for 3 days or so.

I've also just now updated to 20191010.

I'm a bit chicken to turn JRE8 and Serviio back on though. Media Console is not as good at presenting the media as Serviio is but it's sort of getting me by.
Post Reply

Return to “System & Disk Volume Management”