Very Strange Hang, Coincidence?

[FSC830]

Strange, MR automated update is running here for months (since 5.0 public beta), even with latest 5.0.0.1891 since Dec., 24th 2021.
So why it is hanging, not for the process itself but for the updated data???
And time when NAS freezes did not correspondend with time scheduled for the automated update at my end.
A user in German forum reports that all of his NAS have been affected, which have scheduled snapshots activated. NAS without scheduled snapshots did not hang!?
Looking in my screenshot shows that also a snapshot action was done just seconds before NAS stops logging.
So we have a choice: MR or snapshots? Or both?

Regards

Edit: Whereas another reports none of his NAS was affected, but all have snapshots scheduled.
Also HBS3 is using snapshots.

Seems that only one thing is for sure: no one knows what was causing this "event". As yet.

[phcranston]

We had the same issue over the weekend affect about a third of QNAPs (6 of 17) we manage for customers. The only common thread between all of them is that we use myqnapcloud for remote management. They are various models running various services. All have malware protection enabled (which is not reporting any issues) and they are not directly accessible from the internet. For us the resolution was the same on all, we powered off the qnap via the power button on the front panel or by power cycle a managed power strip connected to the device. The devices rebooted and came back up normally. Another interesting fact is that none of them reported needing a file system check after reboot which typically happens if a device is unexpectedly rebooted.

[JDNAS]

Last log action seen before hang on the first unit: (I'm waiting on shutting down the second because it is still accessing one disk in the mirror set and I want to see if the first unit can be successfully reset before risking the second).

Rapid SMART tests were performed successfully on on all disks, then:

Code: Select all

Information	2022-01-09	00:15:02	admin	127.0.0.1	Storage & Snapshots	Snapshot	[Storage & Snapshots] Started creating scheduled snapshot. Volume: Vol-QNAPSystemDefault.
Information	2022-01-09	00:15:10	admin	127.0.0.1	Storage & Snapshots	Snapshot	[Storage & Snapshots] Deleted snapshot "GMT-05_2022-01-01_0015". Volume: Vol-QNAPSystemDefault.
Information	2022-01-09	00:15:11	admin	127.0.0.1	Storage & Snapshots	Snapshot	[Storage & Snapshots] Reached volume/LUN snapshot limit. Volume: Vol-QNAPSystemDefault. The oldest snapshot will be deleted whenever a new snapshot is created.

The Snapshot Manager shows no error for that snapshot.

ETA:
The second unit has also been reset and shows essentially the same log history. (A few seconds difference it time. They are configured with the same basic "structure" - Disks population, RAID mirror, apps, etc. Only config difference is the specifics of the volumes)
I have not yet found any actual resulting problems except for missed jobs and time/anxiety.
All apps have now been updated but continuing with 4.5.4 firmware for now.

[FSC830]

It's getting more and more confusing .
I do not use myqnapcloud in any way. The NAS is not exposed to the internet. Access to LAN from internet is only possible via VPN and a firewall.
But at my NAS also the snapshot mechanism "reached/volume snapshot limit" was hit.
So what is really common by all affeced NAS?

Or is there maybe no common issue to find?

[infotecmb]

+3 QNAP NASes with the most recent 5.0.0.1891 firmware lost all connectivity except PING.
No cloud, no external access. One of them has even internal access disabled to everything except one Samba shared folder.
All have snapshots enabled.

[syncthing]

I think I also got hit by this one on the device I forgot do disable malware remover after an firmware update, other NAS without that "nice app" running didn't have that problem

https://www.qnap.com/en/support/con_show.php?cid=158

Solution for NAS Disconnection Caused by Malware Remover Update

A recent Malware Remover service update might cause system disconnection on certain QNAP NAS devices. This issue mainly affects devices where users have enabled auto update in Malware Remover > Settings. We have updated our service policies to fix this issue.

If your NAS stops responding or cannot be accessed after this service update, we recommend following these steps to manually restart the NAS.

Press and hold the power button until the NAS powers off.
Press the power button again to power on the NAS.

Most affected devices should return to normal after a manual restart. If this issue still persists, please contact QNAP Technical Support for further assistance.

[genmaitya]

Is that "solution" (https://www.qnap.com/en-us/support/con_show.php?cid=158) new? It's not very useful without any dates in the article to define what "recent" means.

Links to this page are displayed in red on the top page of qnap.com and will disappear automatically.
It is a very easy-to-understand announcement

[roaima]

I've just received a reply from QNAP pointing me toward the Malware Remover issue (https://www.qnap.com/en/support/con_show.php?cid=158).

Among other things I've suggested they might like to add creation/revision dates on their public knowledgebase documents so we can get an idea of how relevant they might be to a new situation

[boomorton]

I got my NAS back up.
after several attempts with a fresh disk and new new setup, I succeeded when I hit the reset button during the boot process. Then a new setup would install correctly and be reachable again after boot.
I installed all the previous disks and hit the reset button during boot and the system came back up completely. Everything works, only a volume file check needed to be performed, due to the hard reboots.

I tried the the reset button before of course, but probably too late when the system went into unreachable state.

[Overwerk]

I was getting ready to ship the unit back. I was on a call with tech support (QNAP) and nothing was said as I mentioned the less then adequate behavior from my NAS AFTER downloading the latest firmware.
The day after the unit started doing “self reboots” . After about 6 or so it quit and just sits. No contact via Qfinder.
I have all of the incorrect power down messages. I’m new to this forum. I’ll never update firmware without checking here first :{

[Overwerk]

@boomorton
I just had a few questions as you appear to be one who has regained utility of the affected NAS.
Is you NAS currently still working? What model number and firmware?
"Fresh disks and a new setup"? (I don't have any "fresh disks laying around, but what is a fresh setup?)
As I understand (correct me if I'm wrong) the firmware resides on a portion of the disks. I assume what you accomplished is a fresh firmware install on new drives.
I'm not experienced enough yet with these as to know what hitting the reset (one on the back of unit reachable only with very fine needle, pin paper clip) does during a boot.

Lastly, do you still have all of your data?

I am happy that your system is good! Thank you for your post

[2manycables]

I've just received a reply from QNAP pointing me toward the Malware Remover issue (https://www.qnap.com/en/support/con_show.php?cid=158).

Among other things I've suggested they might like to add creation/revision dates on their public knowledgebase documents so we can get an idea of how relevant they might be to a new situation

That QNAP support is pointing you to this with respect to this specific problem is interesting. I don't know what "We have updated our service policies to fix this issue." means, and as you note, there is no date on when that bulletin was issued. If you go to Malware Remover's changelog https://www.qnap.com/en/app_releasenote ... areRemover, there is no reference to any recent 'fixes' that seem applicable to this problem. I've disabled it after the hard power kill (press power button > 5 sec) reboot which resulted in one data-only file system needing to be checked (no data loss). So far no repeat issues, but its only been a few days. Have others left Malware Remover running (auto update checked or unchecked) and also encountered no further problems?

[FSC830]

...
Have others left Malware Remover running (auto update checked or unchecked) and also encountered no further problems?

Yes, I do!
Only one of my NAS was affected. After power cycling I did not change anything in configuration, even the MR and its automated update.
But to be honest, this is only a test system for QTS 5.0, and since the last weekends hang it is still running.
I am curious if this was a one-time-issue or if this will happen again .

Regards

[_Dejan_]

I have same problem on 3 of 5 qnap NAS...
This NAS have problem:
TS-873, fw 5.0.0.1891
TS-432PXU-RP, fw 5.0.0.1891
TS-1232PXU-RP, fw 5.0.0.1891

On TS-873 WEB&SMB&Virtualization stop working so need to be rebooted. On TS-432PXU-RP WEB&SMB stop working so it also need to be rebooted.
On TS-1232PXU-RP WEB stop working BUT ISCSI still normaly work so Im not reboot it because reboot need to be planned...

TS-431K fw 5.0.0.1858 work OK
TS-251+ fw 5.0.0.1891 work OK

[LRATOZ]

I just spent 4 hours trying to log into my QNAP TS-453D but to no avail!!! (And then I read all this).
The funny thing is that I still got access to the mapped drives on my PC but I can not log on into the NAS!
I am so ** off with the whole thing, especially after what we experienced last year with the ransomware.
QNAP is one of the worst companies to deal with when something goes wrong.
QNAP: If you read this then get of your lazy arses and get your act together!
This is going to be the straw that broke the camel's back!