Clarification on the Jan 27 update to protect from Deadbolt

[QNAPDanielFL]

I wanted to offer some clarification on what happened with the update on January 27 that many have talked about. This update patched the known Deadbolt attack. But we are still recommending you don't expose your NAS to the internet.

There are people who don't remember turning on "Recommended Update" in "Auto Update", but their NAS still updated. So here is an explanation of what happened.

With QTS 4.5.0 we added the "Auto Update" feature. But it was off by default.

With QTS 4.5.3 we by default enabled "Recommended Version".

Recommended version does not apply to every update. So people did not realize recommended update was enabled on their NAS. But after Deadbolt, we released a recommended update to protect from deadbolt. Because this upate was set as a "recommended version", NAS with "recommended version" enabled updated.

Having recommended version enabled by default did allow us to protect many NAS units. But if anyone does not want this feature, they can disable it.

Here is more information.

https://www.qnap.com/en/security-news/2 ... on-feature

We understand that services could be interrupted during the update. We are constantly looking for ways to improve our products. Future system software updates may include changes that help users better manage the update process.

[AlastairStevenson]

Thanks for the helpful update.
And the detailed (and dated ...) explanation in the linked security news article.

[Moogle Stiltzkin]

I wanted to offer some clarification on what happened with the update on January 27 that many have talked about. This update patched the known Deadbolt attack. But we are still recommending you don't expose your NAS to the internet.

There are people who don't remember turning on "Recommended Update" in "Auto Update", but their NAS still updated. So here is an explanation of what happened.

With QTS 4.5.0 we added the "Auto Update" feature. But it was off by default.

With QTS 4.5.3 we by default enabled "Recommended Version".

Recommended version does not apply to every update. So people did not realize recommended update was enabled on their NAS. But after Deadbolt, we released a recommended update to protect from deadbolt. Because this upate was set as a "recommended version", NAS with "recommended version" enabled updated.

Having recommended version enabled by default did allow us to protect many NAS units. But if anyone does not want this feature, they can disable it.

Here is more information.

https://www.qnap.com/en/security-news/2 ... on-feature

We understand that services could be interrupted during the update. We are constantly looking for ways to improve our products. Future system software updates may include changes that help users better manage the update process.

ty for the clarification daniel.

basically it was a issue with miscommunication....

i get that auto update may have helped solve some of the issues for people not updating qts as often as they should. but at the same time, some of the users may have legitimate reasons not to update depending on their specific situation.

for myself last time, my nas kept on random rebooting. so the only solution for me was to stick to an older firmware that did not have this issue, until a future firmware fixed this. so in that kind or some specific other situation they don't want to be auto updated.

questionable whether auto update as default enabled is a good thing or not.

that said, seeing as there is an option to disable it, that is good. but it has to be communicated to the users how to do so.

Maybe during initial qts setup, you clarify whether to enable or disable auto update... so it's clear from the initial setup on this.

the recent inclusion of being able to defer updates so you don't get a day 0 update is a welcome one. most of us usually wait a week or more before we update.... just in case....

personally i do everything manual cause i check often enuff. but other people who want less hassle has this automated option

[tribunal3117]

I have a TS-851. There should have been some kind of warning about this when it was introduced. By which I mean a popup after the firmware was installed. This isn't a minor config change. My NAS is not internet facing and I purposely lag my firmware. After this update, my NAS would crash every night at 3AM due to the malware removal scan (how's that for irony?). I still can't bring it down cleanly; it told me it had to force stop the NAS and triggered a resync process each time. It may be related to iSCSI as the first night the malware removal didn't crash the NAS, but it did foul the iSCSI connection causing me to need to bounce the ESXI host. The next 2 nights it crashed entirely so I deactivated the malware scan, and its managed to survive the past 3 nights. I haven't tried to bring the NAS down cleanly because frankly I'm just happy its staying up. I have a ticket open, but its not getting any movement as far as troubleshooting the shutdown issue. The NAS was wonderfully stable prior to your forced update to 5.0.0.

[FSC830]

This is an issue you can not really blame QNAP!
See this release note for QTS 4.5.3.x, dated from April 2021! https://www.qnap.com/en/release-notes/q ... 2/20210428
The new function is clearly mentioned: "QTS now automatically installs recommended firmware updates by default. Administrators can specify a schedule to check for and perform updates."
But no one reads the release notes! Or at least not carefully enough!
In most cases users just install the newer QTS, it is newer, so it must be better .

Regards

Edit:
Only thing you can blame QNAP: if you are not at the latest FW, using origin admin account or 2FA is not activated at every login a nasty message occurs about that.
They should have add the same if the Auto-update function is enabled!

[Moogle Stiltzkin]

This is an issue you can not really blame QNAP!
See this release note for QTS 4.5.3.x, dated from April 2021! https://www.qnap.com/en/release-notes/q ... 2/20210428
The new function is clearly mentioned: "QTS now automatically installs recommended firmware updates by default. Administrators can specify a schedule to check for and perform updates."
But no one reads the release notes! Or at least not carefully enough!
In most cases users just install the newer QTS, it is newer, so it must be better .

Regards

Edit:
Only thing you can blame QNAP: if you are not at the latest FW, using origin admin account or 2FA is not activated at every login a nasty message occurs about that.
They should have add the same if the Auto-update function is enabled!

exactly.

2 things they could have done here, when the user updates (manually) to the new firmware with this policy change, there needs to be a pop up notification to ask user permission to change it from manual to auto update.

also for new users first time initializing qts/quts will have during the setup process a setting indicator whether to allow auto updates or not.

those 2 things should do the trick, as i doubt newsletter will be sufficient enough (not everyone subscribes to them unfortunately. or like u said read changelogs carefully or all the time).

but never should something like this be imposed simply like this (regardless of any good intentions. like i said some people have their own situation to not necessarily update to the latest, or do automated updates. some people are not careful when managing it this way, but it's for them to learn hard lessons if they don't manage their nas's well).

hopefully going forward this type of thing won't happen again


if security is on the agenda, i suggest qnap look at truenas as an example and perhaps in a future qts/quts to follow something like that to improve on security.

that said i still like qnap, i still use them very frequently 24/7 , but there is obviously still room to improve

[FSC830]

Good intention is the little sister of badly done!

Regards

[Cbrad01]

When ever you update the firmware or applications you should always read the change logs, regardless of them platform.
Many times vendors change, remove or update features and functionality.
If the vendor changes, removes or updates something and documents in the change log as “bug fix” without any explanation shame on them. But if the document as “auto-updates are enabled as a default” and I miss it, shame on me. I don’t think the vendor needs to keep putting pop-ups and reminders in telling me what they listed in the change log. In fact I hate them. I can’t stand that Qnap keeps popping up warnings in container, virtualization station and the main screens telling me what they have told me hundreds of times.
No matter what the vendor does, change logs, pop ups, banners, emails, or whatever someone will miss it and moan later.
Vendors should list everything that changes, regardless of if it’s security, bug, enhancements, features, whatever in the change log.
When you install updates a clear and detailed change log should be presented before the update so we can make a choice.
Vendors are not our parents, nor are we their children.
We each have responsibilities.


Sent from my iPhone using Tapatalk

[nasinterested]

I have a TS-653D which i installed the latest firmware at the time (should have been 4.5.4.1723 20210708). It was this version on which i disabled *any* automatic update (BTW, data is backed up and NAS not exposed to the outside).

I later updated to a newer 4.5.4.1892. So, either this update has set the values already (which itself is a breach of trust) or there is more to your story.

Anyhow, security issues will always happen. i‘d prefer transparency rather intimidating my infrastructure. i know what i do (and if not, i am willing to pay the price for it).


Gesendet von iPhone mit Tapatalk Pro

[FSC830]

... or there is more to your story.
...

I dont know what you exactly mean with?
Are you hit by an update to 5.0.0.1891?

If so, you can check out easily in your log file if auto-update is the culprit:

Auto_upd_09a.png

Regards

[nasinterested]

i dont know why you are explaining this to me but most appreciated. I did not install the update so what else than auto update should have caused it and triggered to write the above?


BTW, everything seems to be working after fixing the issues

[P3R]

But after Deadbolt, we released a recommended update to protect from deadbolt.

As 4.5.4.1892 according to this security advisory should also have the patch for the Deadbolt vulnerability. I can't see anything else than that all users running 4.5.41892 was unnecessarily force upgraded to 5.0.0.1891.

For all the users that have problems with the still long list of "Known Issues" (including Thunderbolt that is mentioned in the release notes as a reason to not update to version 5.0), is a downgrade back to version 4.5.4.1892 supported?

[FSC830]

Full explanation:

2. An forced update is a rumor! There is no forced update triggered from QNAP! If there was an unexpected update you should check your Auto-Update options.

Most sites seem to disagree and state there was indeed a forced update, under certain conditions. However, neither of my 4.5.4.1892 units were touched during this time so that my have been a limiting factor too.

https://www.bleepingcomputer.com/news/s ... 0-devices/

https://www.reddit.com/r/qnap/comments/ ... jan_2627th

https://www.zdnet.com/article/decryptor ... s-devices/

https://www.techspot.com/news/93220-qna ... users.html

https://www.pcmag.com/news/dont-ignore- ... installing

Pretty sure I had a link to QNAP admitting to it as well, but can't easily find it again at the moment.

Sorry to say, but most sites (all sites ) are wrong with this!
Why?
All of these sites quote users and forum posts, who are claiming for this forced update, I dont see any serious investigation for this, it is just clickbaiting!
No, I am not a 100% loyal QNAP user (I confess) going to defend QNAP, far away from this: I do not rely in any QNAPs security feature, not for single cent!
QNAP proved so many times that they do not understand how to implement security.

So I did a test by myself, I removed all disks from my TS-473A, inserted other disks and installed the NAS from scratch with the firmware which was installed at time of delivery (4.5.2-1566).
The auto-update option was disabled.
Afterwards I updated to latest 4.5.4 available at QNAPs download site.
This update enabled the auto update option!
I disabled the auto-update option and was still waiting for an "forced" update -> nothing happens, there is no forced update!
After about 48 hours I enabled to auto-update option and the very next day (matching the time in autoupdate option) the NAS updated to 5.0.0.1891!

So I am absolutely sure, that all who claim an forced update, did not check the auto-update option after a previous v4.5.x update.
This is the mystery behind the so-called forced update - nothing else.

And as long, as no one can make a 100% evidence that auto-update was disabled at the point of time when "forced" update was done, as long I do not believe of that, regardless, how many sites share this "fake news"!

And all who doubt about this:
Make yourself the same test: downgrade you model to latest available 4.5.x version, disable both auto-update options (recommended version and latest version) and wait!

Here you will see the screenshots in chronological order from the test mentioned above:

Auto_upd_03.png

Auto_upd_04.png

Auto_upd_05.png

Auto_upd_06.png

Auto_upd_06a.png

Auto_upd_07.png

Auto_upd_08.png

Auto_upd_09a.png

Regards

[QNAPDanielFL]

But after Deadbolt, we released a recommended update to protect from deadbolt.

As 4.5.4.1892 according to this security advisory should also have the patch for the Deadbolt vulnerability. I can't see anything else than that all users running 4.5.41892 was unnecessarily force upgraded to 5.0.0.1891.

For all the users that have problems with the still long list of "Known Issues" (including Thunderbolt that is mentioned in the release notes as a reason to not update to version 5.0), is a downgrade back to version 4.5.4.1892 supported?

4.5.4.1892 and 5.0.0.1891 are both patched for the DEADBOLT vulnerability. But before deciding if you should downgrade, I would recommend checking with tech support. If you are not using Thunderbolt, you might find that QTS5 works much better now than before.

[Skwor]

But after Deadbolt, we released a recommended update to protect from deadbolt.

As 4.5.4.1892 according to this security advisory should also have the patch for the Deadbolt vulnerability. I can't see anything else than that all users running 4.5.41892 was unnecessarily force upgraded to 5.0.0.1891.

For all the users that have problems with the still long list of "Known Issues" (including Thunderbolt that is mentioned in the release notes as a reason to not update to version 5.0), is a downgrade back to version 4.5.4.1892 supported?

4.5.4.1892 and 5.0.0.1891 are both patched for the DEADBOLT vulnerability. But before deciding if you should downgrade, I would recommend checking with tech support. If you are not using Thunderbolt, you might find that QTS5 works much better now than before.

There are STILL ALOT of known issues with 5.0, it is not just a thunderbolt problem. I have held off until most the lingering issues are addressed, commercial OS's really should not have so many listed known problems for so long, see below for the list from your own company. I do want to go to 5.0 but it still seems unfinished to me, leaving me wondering what else may be a problem.

[Known Issues]
- Twonky Server cannot function normally on the TS-h973AX running the latest versions of QTS.
- Some applications cannot access the NAS when secure connection and TLS 1.3 are enabled. This is due to a known issue in the applications. We will fix this issue in upcoming app releases.
- macOS Finder takes a long time to display content in SMB shared folders when users connect the Mac to the NAS via Thunderbolt. This problem may be due to Mac device driver issues. It only occurs to Mac devices with Intel processors and macOS 11 (or later versions).
- Thunderbolt write speeds are lower than expected in QTS 5.0.0. Note: Due to Thunderbolt driver compatibility issues, if you are using macOS 11/12 devices with Intel processors, we do not recommend updating QTS to 5.0.0 for the time being.
- QTS and QuTS hero with newer kernel versions do not support ATTO Fibre Channel adapters. If you have already installed an ATTO Fibre Channel adapter on your device, we do not recommend updating the firmware to QTS 5.0.0 or QuTS hero h5.0.0 for the time being.
- Control Panel cannot display the information of the TPU installed in the M.2 slot on the QGD-1602P.
- After users rename a shared folder, QuLog Center still displays the original folder name in Accessed Resources.
- The WordPress folder would disappear from the NAS Web Folder after users updated QTS to 5.0.0 and WordPress to 5.7.2. (WordPress could not keep the previous settings during the update.)
- On certain ARM-based models, non-administrator users cannot access subfolders in the @Recently-Snapshot folder when advanced shared folder permission settings are enabled.
- A file system issue (EXT4 error) might occur when users disabled or removed SSD cache after using SSD cache.