QNAP Hangs on Shutdown or Reboot

Questions about SNMP, Power, System, Logs, disk, & RAID.
MRVigliotti
New here
Posts: 7
Joined: Sat Apr 02, 2016 6:21 am

QNAP Hangs on Shutdown or Reboot

Post by MRVigliotti »

I'm having an issue where my TS-853A hangs during a shutdown or reboot. I can tell it unmounts all the drives and starts the shutdown process but then just hangs at "rebooting system". I have to do a hard power off (with the power button). It then boots cleanly when I turn it back on with no errors or filesystem checks.

I was able to solve this problem about a year ago with the help of a forum post and it turned out to be related to Plex. Basically, Plex was not shutting down in time and hanging the QNAP shutdown process. I rooted into the console and did some sort of prioritization of what was shut down. When I placed Plex higher in the order, it had more time to shutdown and everything worked.

Low and behold, a year passes, and the issue is back (probably a firmware update) and I can't find the link.

Can anyone help?
User avatar
Toxic17
Ask me anything
Posts: 6468
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: QNAP Hangs on Shutdown or Reboot

Post by Toxic17 »

have your run the QNAP Diagnostics tool? if not i would strongly suggest running this (available in app center) once you have run it (Dump log) check the errors and see if it points to anything in particular.

You have not said what firmware version and build number you are on, i will guess 4.3.3.0238 ????

I suggest downloading this app to make the correct app priority.

[QPKG] Sort my QPKGs
Regards Simon

Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
MRVigliotti
New here
Posts: 7
Joined: Sat Apr 02, 2016 6:21 am

Re: QNAP Hangs on Shutdown or Reboot

Post by MRVigliotti »

Thanks very much for the info! Yes, I'm on 4.3.3.0238. I was able to install and run the diagnostic tool. It's a very powerful tool and good to have but I'm getting log overload here and I'm not exactly sure what I should be looking for. The event log output for the last hang is below, not much to infer from it.

Code: Select all

1115,  0,2017-07-05,21:45:01,admin,192.168.10.1,---,[Network Recycle Bin] Network recycle bin emptied.,  0,1499305501,
1116,  0,2017-07-05,21:45:26,admin,192.168.10.1,---,[Firmware Update] Start downloading firmware 4.3.3.0238 Build 20170703.,  0,1499305526,
1117,  0,2017-07-05,21:46:50,admin,192.168.10.1,---,[Firmware Update] Start unzipping TS-X53A_20170703-4.3.3.0238.zip,  0,1499305610,
1118,  0,2017-07-05,21:47:02,admin,192.168.10.1,---,[Firmware Update] Start updating firmware 4.3.3.0238 Build 20170703.,  0,1499305622,
1119,  0,2017-07-05,21:47:18,System,127.0.0.1,localhost,[Firmware Update] Started updating the firmware.,  0,1499305638,
1120,  0,2017-07-05,21:49:01,System,127.0.0.1,localhost,[Firmware Update] System updated successfully from 4.3.3.0229(20170624) to 4.3.3.0238(20170703).,  0,1499305741,
1121,  0,2017-07-05,21:50:05,admin,192.168.10.1,---,[Power Management] System restarting.,  0,1499305805,
1122,  0,2017-07-05,21:50:14,System,127.0.0.1,localhost,Stopping Plex Media Server...,  0,1499305814,
1123,  0,2017-07-05,21:50:16,System,127.0.0.1,localhost,[Network & Virtual Switch]Stop Network & Virtual Switch.,  0,1499305816,
1124,  0,2017-07-05,21:51:12,System,127.0.0.1,localhost,System was shut down on Wed Jul  5 21:51:12 EDT 2017.,  0,1499305872,
1125,  0,2017-07-05,22:04:08,System,127.0.0.1,localhost,System started.,  0,1499306648,
1126,  0,2017-07-05,22:05:31,System,127.0.0.1,localhost,[App Center] Resource Monitor enabled.,  0,1499306731,
1127,  0,2017-07-05,22:06:02,System,127.0.0.1,localhost,[App Center] Helpdesk 1.1.04 has been installed in /mnt/HDA_ROOT/update_pkg/helpdesk successfully.,  0,1499306762,
1128,  0,2017-07-05,22:06:02,System,127.0.0.1,localhost,[App Center] Helpdesk enabled.,  0,1499306762,
1129,  0,2017-07-05,22:06:10,System,127.0.0.1,localhost,Starting Plex Media Server...,  0,1499306770,
1130,  0,2017-07-05,22:06:44,System,127.0.0.1,localhost,[App Center] Helpdesk 1.1.04 has been installed in /mnt/HDA_ROOT/update_pkg/helpdesk successfully.,  0,1499306804,
1131,  0,2017-07-05,22:06:45,System,127.0.0.1,localhost,[App Center] Helpdesk enabled.,  0,1499306805,
1132,  0,2017-07-05,22:07:39,System,127.0.0.1,localhost,[App Center] Qsync Central enabled.,  0,1499306859,
1133,  0,2017-07-05,22:07:40,System,127.0.0.1,localhost,[App Center] Qsync Central 3.0.1 installation succeeded.,  0,1499306860,
Also, the sort my QPKG that was recommended, does this tool also have the capability of dictating the shutdown order? If I'm having the same problem as last time, there is a hang on the shutdown, not on boot up.
User avatar
Toxic17
Ask me anything
Posts: 6468
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: QNAP Hangs on Shutdown or Reboot

Post by Toxic17 »

one the diagtool has been run download the file it creates (think it is a zip file) and click on the HTML file inside. this will list a few headings, and links. look for "errors"

You would need to ask the author of that program. though I doubt it controls the shutdown procedure process. it is only listing start up of QPKGs and not all services or processes.

have you checked all your hard disks via the SMART testing, and also setup a schedule to check your hard disk array (Data Scrubbing) one of which maybe causing your issue.
Regards Simon

Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
MRVigliotti
New here
Posts: 7
Joined: Sat Apr 02, 2016 6:21 am

Re: QNAP Hangs on Shutdown or Reboot

Post by MRVigliotti »

I have been diving deeper into this issue and I am positive that the Plex QPKG is causing this issue. I have no errors, the SMART testing in clear, and I data scrub every week as is now default. In fact, when the shutdown hangs, the disk array is shutdown before. Everything else is working fine.

I disabled the Plex QPKG before shutdown and the NAS rebooted cleanly. Enabled it back and restarted again it hangs. This is the exact problem that I fixed in the past. Plex just needs a few extra seconds to stop it's service. I fixed it by moving it up higher in the shutdown priority in the console. I have been searching to no avail to find the walkthrough that helped me.

Does anyone know which configuration file controls the QPKG shutdown process? There has to be one as I edited before, I just can't find it.
User avatar
Toxic17
Ask me anything
Posts: 6468
Joined: Tue Jan 25, 2011 11:41 pm
Location: Planet Earth
Contact:

Re: QNAP Hangs on Shutdown or Reboot

Post by Toxic17 »

are you using the latest package from Plex themselves?

1.7.5.4035-313f93718 June 28, 2017

might be worth asking on their forums if you can replicate this when Plex is enabled.

https://forums.plex.tv/categories/qnap
Regards Simon

Qnap Downloads
MyQNap.Org Repository
Submit a ticket • QNAP Helpdesk
QNAP Tutorials, User Manuals, FAQs, Downloads, Wiki
When you ask a question, please include the following


NAS: TS-673A QuTS hero h5.1.2.2534 • TS-121 4.3.3.2420 • APC Back-UPS ES 700G
Network: VM Hub3: 500/50 • UniFi UDM Pro: 3.2.9 • UniFi Network Controller: 8.0.28
USW-Aggregation: 6.6.61 • US-16-150W: 6.6.61 • 2x USW Mini Flex 2.0.0 • UniFi AC Pro 6.6.62 • UniFi U6-LR 6.6.62
UniFi Protect: 2.11.21/8TB Skyhawk AI • 3x G3 Instants: 4.69.55 • UniFi G3 Flex: 4.69.55 • UniFi G5 Flex: 4.69.55
User avatar
OneCD
Guru
Posts: 12010
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: QNAP Hangs on Shutdown or Reboot

Post by OneCD »

MRVigliotti wrote:Plex just needs a few extra seconds to stop it's service. I fixed it by moving it up higher in the shutdown priority in the console. I have been searching to no avail to find the walkthrough that helped me.

Does anyone know which configuration file controls the QPKG shutdown process? There has to be one as I edited before, I just can't find it.
The order QPKGs are shutdown is the reverse of the order they are loaded. e.g. last QPKG launched on bootup is the first one killed on shutdown.

There are a couple of things that might help you here:
  • change the kill order name for Plex in (/etc/rcK.d) so it has a lower number. So, if the automatically generated symlink is currently something like 'QK120Plex', rename this to 'QK100Plex'.

    Note: this will not be a persistent change and will need to be re-applied after each NAS startup, so maybe create an autorun script to do this.
  • or you could modify the Plex init script 'stop' function and force it to sleep for a few seconds. This would be a persistent change and would not need to be re-applied after NAS startup.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
MRVigliotti
New here
Posts: 7
Joined: Sat Apr 02, 2016 6:21 am

Re: QNAP Hangs on Shutdown or Reboot

Post by MRVigliotti »

Thanks for the revelant info OneCD, I'm almost there.

I did a test of your first suggestion and it worked great, the machine rebooted cleanly! But as you said, it was not done persistently and is back to normal after reboot. Was wondering if you could guide me a little further on setting up an autorun script?

I didn't really explore option 2 since I'm not that great with scripting and I would think that we want the shutdown of the process to start earlier vs. have it take longer? It's taking too long in the first place.
User avatar
OneCD
Guru
Posts: 12010
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: QNAP Hangs on Shutdown or Reboot

Post by OneCD »

MRVigliotti wrote:I didn't really explore option 2 since I'm not that great with scripting and I would think that we want the shutdown of the process to start earlier vs. have it take longer? It's taking too long in the first place.
I suggested that as you said:
MRVigliotti wrote:Plex just needs a few extra seconds to stop it's service.
Sleeping the 'stop' function would give it a few extra seconds.
MRVigliotti wrote:I did a test of your first suggestion and it worked great, the machine rebooted cleanly! But as you said, it was not done persistently and is back to normal after reboot. Was wondering if you could guide me a little further on setting up an autorun script?
No worries. Checkout this post - it will create the autorun system for you. Then all you need is a script to run.

Yours will be quite simple:

Code: Select all

#!/usr/bin/env bash
mv /etc/rcK.d/QK???Plex /etc/rcK.d/QK100Plex
I don't use Plex so I don't know the name of the symlink that appears on a NAS with Plex installed, so you may have to tweak that command string. ;)

Save the contents above into a new text file and place it in the autorun scripts directory. Mark it executable. Run it manually to check. Done.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
User avatar
OneCD
Guru
Posts: 12010
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: QNAP Hangs on Shutdown or Reboot

Post by OneCD »

I've just had a thought - if your autorun script runs before Plex is re-integrated during NAS boot, then nothing will happen.

It may be better to rename the kill symlink as part of the Plex init 'start' function. ;)

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
MRVigliotti
New here
Posts: 7
Joined: Sat Apr 02, 2016 6:21 am

Re: QNAP Hangs on Shutdown or Reboot

Post by MRVigliotti »

Can confirm, just rebooted, no change. Will test out integrating the kill symlink into the init start tonight when I get home. Many thanks for pointing me in the right direction.
User avatar
OneCD
Guru
Posts: 12010
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: QNAP Hangs on Shutdown or Reboot

Post by OneCD »

No worries. Please post the Plex init script if you need a hand.

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
MRVigliotti
New here
Posts: 7
Joined: Sat Apr 02, 2016 6:21 am

Re: QNAP Hangs on Shutdown or Reboot

Post by MRVigliotti »

Just got around to this and tested and working!

I ended up pasting this command:

Code: Select all

mv /etc/rcK.d/QK105PlexMediaServer /etc/rcK.d/QK100PlexMediaServer
into/etc/init.d/plex.sh right under the start section.

Sure enough, works like a charm. I may need to do this after every Plex update but right now I am just ecstatic that I can reboot the NAS remotely.

Huge thanks to OneCD, really appreciate the guidance, couldn't have fixed it without you!
User avatar
OneCD
Guru
Posts: 12010
Joined: Sun Aug 21, 2016 10:48 am
Location: "... there, behind that sofa!"

Re: QNAP Hangs on Shutdown or Reboot

Post by OneCD »

That's great. Nicely done. :D

But I've just noticed - you've hardcoded the original kill symlink name. Please note: this can change depending on what else is being loaded. That's why my original suggestion had those 3 question marks. Each one acts as a wildcard meaning "a single unknown character in this position".

The QPKG creator can specify a preferred start number (and the preferred kill number is 1000 minus the start number) from 100 to 999 but the NAS assigns the next one available. So, your 105 may end up being something else if you add/remove QPKGs.

Maybe try this instead:

Code: Select all

mv /etc/rcK.d/QK???PlexMediaServer /etc/rcK.d/QK100PlexMediaServer

ImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImageImage
MRVigliotti
New here
Posts: 7
Joined: Sat Apr 02, 2016 6:21 am

Re: QNAP Hangs on Shutdown or Reboot

Post by MRVigliotti »

Good catch, updated and tested. Thanks.
Post Reply

Return to “System & Disk Volume Management”