NAS doesn't boot anymore, but still beeps at poweron

[Jens2]

Hi everybody,

my TS 873 was working normally until yesterday morning. I did not mess with it in any way at that time. Installed firmware had been running for some weeks without issues. There hasn't been any sort of power- or switch outage. In addition it's connected to a UPS that should guarantee a stable voltage.

Suddenly it became unreachable over network. I wasn't here, so I asked someone else to power it by button-long-press and power it on again, didn't help. When coming back I realized the network ports didn't even have a link anymore (with link aggregation even).

I made some more attempts to turn it off and on again. In 2 of those attempts I got the network address to become pingable, but nothing more. SSH oder the web interface all returned connection refused upon connection attempts.

I've tried to do these additional things:

• use only the aftermarket ram installed
• use only the ram originally shipped with the NAS
• Try it with and without disks installed

Eventually I installed a display adapter hoping to see some helpful error message. This is where the system stops and doesn't proceed:


I've also noticed there's a blue LED that lid up in few of the boot attempts, but not the others attempts. Not that the LED is important for operation, but to someone who knows what it signals it can probably tell how far the boot process went in those cases. It's roughly here where there's a marking on the image:

In addition here's a video showing the boot attempt and including the beep sounds: https://server47.de/download/temp/20210308_222515.mp4

As it always is - legal warranty has expired just two months ago...

Is there any diagnosis you folks can make of this information?

[dolbyman]

try to boot from a usb stick(forum search) to see if your dom is shot

[Jens2]

What do you mean by dom ?

You don't happen to know what I need to press to boot from stick?

[dolbyman]

DOM = Disk on a Module..the thing you circled in red
I used the forum search for you

viewtopic.php?t=133899

[Jens2]

Thank you, I did search, just not in this forum, but search engine in general. There it only brough up Document Object Model.

Just as an intermediary report:

With the DSL on a stick recommended by QNAP I wasn't able to use a keyboard at all. With a knoppix I was with this workaround:
- leave a keyboard connected while starting
- initiate the boot until including when linux says "Boot:", press enter and unplug the keyboard again asap before the boot process can get get stuck again
- plug it back in again when the (stick-) OS is fully up.
- but even then this OS may freeze eventually

With that I made dd copies of the partitions, just in case I may need them later. In addition I ran fsck -y on all of the partitions, but it complained about only the minor "unclean dismount" things.

It took several attempts to get this far. In general the system seems rather erratic. When I am just beginning to think it may be working again it'll go south again. My feeling is that the longer the box has been "cold" the more likely SOMETHING may work again. This makes me lean more towards some kind of hardware problem. The boot process seems to get blocked by some USB device. Even when booting from a stick the DOM (which seems to be connected via USB) seems to be a show stopper sometimes.

The last attempt I did was the most successful one. fsck as described, restart, reconnect the smaller disk array (that also has some space for the box itself) and suddenly the QNAP OS came up.

Unfortunately I did not think of creating a config backup in that moment, but was presented the option to upgrade the firmware and took it, hoping that refreshing the firmware again may be a good idea.

So installed the firmware, everything was reported to have gone smoothly, restart and then - silence again.

I'll give it some hours before doing another attempt. At least I'd like to get a valid config backup for any full resets that may follow.

In general - do you know how the firmware and settings are stored? There's this small dom, but based on QNAP-Wiki and forum entries there also seems to be something stored on the disks I inserted. And is this DOM used for logs as well? I mean probably because the QNAP is generally usable without disks.

[Mousetick]

The main firmware is stored on the DOM. It contains the bare minimum to boot and setup the machine without disks, plus some tarballs containing apps, system utilities and services that get installed to disk once at least one disk is initialized.

MBR partitions on a 512 MB DOM:
1 ext2 5 MB contains boot loader
2 ext2 235 MB contains kernel image, root filesystem image, tarballs for add-ons to be installed to disk
3 ext2 235 MB copy of #2. This is what QNAP refers to by "Dual boot OS protection" in its products specs. It would have been more useful IMHO to put a recovery firmware here that never gets updated than simply an identical copy of the latest firmware.
4 --- Extended partition (needed for more than 4 partitions with MBR)
5 ext2 8 MB purpose unknown. On my NAS, it contains a single, empty 'qlicense' directory. On a running system, this partition is mounted at /mnt/boot_config. It might be used to store passphrases or keys to automatically unlock encrypted drives if this feature is used.
6 ext2 8 MB grab-bag of a few configuration files that appear to be only stored here during a firmware update. Also contains autorun.sh if you use one.

Once there is at least one disk initialized, two md RAID1 devices are setup to store extra software that's part of the firmware, and to store configuration and logs, respectively:
- md9 (array of /dev/sdX1 partition(s)) ext3 ~500 MB mounted at /mnt/HDA_ROOT contains configuration in (hidden) .config directory, and logs in (hidden) .logs directory, and various other bits. Most logs are sqlite databases that need to be dumped before they can be read, except the kernel log kmsg which is plain text.
- md13 (array of /dev/sdX4 partition(s)) ext3 ~400 MB mounted at /mnt/ext contains the remainder of the firmware installed from the DOM tarballs.

Finally yet more pieces of the "firmware", such as the stupid Multimedia Console and Help Desk apps which can't be removed, are installed in the form of QPKG on the system volume in the (hidden) .qpkg directory. If the system volume is stored on the first storage pool, it would be in /share/CACHEDEV1_DATA/.qpkg.

In general - do you know how the firmware and settings are stored?

See above. I used my NAS as a reference, yours works probably the same way with an X86 CPU and the same DOM size, but things may be quite different on newer models equipped with much larger DOMs.

And is this DOM used for logs as well?

No stateful information ever gets stored to the DOM while the NAS is running, even when there are no disks, it's used purely as a readonly boot device. It only gets written to for firmware updates. When the system is booted off the DOM and there are no initialized disks, it runs entirely from RAM and all system state data is lost at power-off or reboot.

Information provided for informational purpose only and not intended to be an exhaustive inventory of QNAP firmware's scattered mess. I wouldn't recommend you attempt to backup and restore any of it. Heck, even QNAP themselves don't dare offering a 'System backup/restore' feature to their users!

[Jens2]

Thank you for that really insightful information. I learned a lot!

I've been able to get the NAS working one more time, so I was able to save my configuration.

I attempted the firmware recovery which worked in terms of there haven't been any errors in the process. But the NAS failed to boot afterwards as it had before the firmware recovery.

I guess I have to consider the DOM broken if it works that unreliably.

As far as I understand I have these options now:
* send the thing in to the vendor
* buy a new one
* order a replacement mainboard
* replace the DOM by a USB stick as described here: viewtopic.php?t=133899
* replace the DOM (is this possible in terms of can one buy that thing separately)?

Are there any further options I haven't yet thought about?

[Mousetick]

I guess I have to consider the DOM broken if it works that unreliably.

If the NAS boots off a USB stick reliably all the time, while it boots off the DOM only once in a while, then yes I believe it's reasonable to conclude the DOM is defective.

As far as I understand I have these options now:

Yes but QNAP doesn't sell spare parts. They only deal with RMA.

* replace the DOM (is this possible in terms of can one buy that thing separately)?

Maybe, if the DOM is plugged into a connector on the mainboard, and if you can find the same type from aftermarket vendors.

[dolbyman]

Those EUSB- DOM can be purchased off ebay/Aliexpress/etc.They are generic parts

[Toxic17]

Thank you for that really insightful information. I learned a lot!

I've been able to get the NAS working one more time, so I was able to save my configuration.

I attempted the firmware recovery which worked in terms of there haven't been any errors in the process. But the NAS failed to boot afterwards as it had before the firmware recovery.

I guess I have to consider the DOM broken if it works that unreliably.

As far as I understand I have these options now:
* send the thing in to the vendor
* buy a new one
* order a replacement mainboard
* replace the DOM by a USB stick as described here: viewtopic.php?t=133899
* replace the DOM (is this possible in terms of can one buy that thing separately)?

Are there any further options I haven't yet thought about?

the firmware recovery. what image did you use? I ask as all the ones listed on the main firmware recovery wiki pages are afaik cut down versions, but the page has none for the TS-x73 only the TVS-x73 image

[Jens2]

This one here:

TS-873U / TS-1273U / TS-1673U
http://eu1.qnap.com/Storage/tsd/fullima ... -1.3.0.img

I've been in contact with QNAP support in parallel. They explicitly told me TS-873U is identical in terms of firmware (just rack -mounted).

[Toxic17]

ahh ok.

[Jens2]

Ok, what happened today: I tried the variant with making a USB stick my permanent boot device. That worked just as well as using the original DOM - not.

With nothing connected to the internal usb port I discovered my problems were just as weird. I had similar problems getting the box to work (reliably) using some boot stick.

QNAP refused to make any concession towards me in terms of repair price (because the device is only 2 months over warranty end). They sent me a repair quote, giving me a 400$ estimate. Considering the original price of roughly 1000$ I find that pretty impertinent.
I guess for that price I'd rather buy a new one and at least get a new warranty period with that. But I'm not there, yet.

I've tried different combinations of the memory DIMMs again, but that hasn't made a difference. My current working theory is that maybe the CPU gets too hot. While having just the BIOS open the CPU temperature rises to a max of 62° (from what I could observe). That's certainly not low, but the specs of the AMD R-Series RX-421ND is up to 90°. So I'd rule that out.

Today I had a new situation after reconnecting the DOM. I powered one the device and it beeped as usual. But after a while it made 3 LONGER beep sounds in a row.
Just started looking in other posts what this may mean. Seems to point in the direction of dead capacitors. I may add this happened in cases when the system would not boot and in one when it did boot successfully.

[Jens2]

So far I didn't get any more ideas other than replacing the motherboard. But to find one with the same form factor and connectors wouldn't have been easy.
With a heavy heart I've sent it in. Let's see what they do with it.

I'm pretty disappointed about the quality here. From what I read here in the forums electronics defects aren't that rare anymore. From regular sized computers I'm not used to non-mechanical components breaking at all or at least not within the first 5 years.