HS3 "Failed to upload metadata"

[P3R]

Also, this is a terrible error handling message...

I agree. What did Qnap support tell you that the message really meant?

I still get the error Failed to Upload Metadata Check your network settings.

Are you doing any filtering on outgoing traffic from the Qnap?

Also what Qnap model running what QTS version and what HBS version?

What data is it? Type and amount of data, small/large files? Is there any difference with different folders (if you're doing multiple)?

[onecolumbyte]

I have a support ticket out to qnap, nothing yet.

The error is reported in HSB3 and shows in the log. I don't have additional detail... I suppose I will have to try again with debug loggin enabled to see if more detail is available.

Does anybody know how to adjust wait time as mentioned by Client 2020?

[nickexists]

I've had a support ticket for this open for months with nothing useful coming out of it. In my case I am trying to backup to Box. The issue doesn't seem to occur until the backup I'm trying to create gets sufficiently large. Once it gets to a certain size I'll get this error every time I try to start the backup within an hour.

If I download the debug log from HBS I see this errorin `backupengine-error.log`:

requests.exceptions.HTTPError: 409 Client Error: Conflict for url:

[2021-02-21 23:27:27,703][ Thread-2][E][qnap.hbs3.library.qts.ssd_detect][tool_func.py:logger_error:91] : ['Traceback (most recent call last):\n', ' File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/qnap/hbs3/library/qts/ssd_detect.py", line 142, in get_disk_type_by_poolID\n', 'Exception\n']
[2021-02-23 23:08:20,790][ Thread-2][E][qnap.hbs3.library.qts.ssd_detect][tool_func.py:logger_error:91] : ['Traceback (most recent call last):\n', ' File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/qnap/hbs3/library/qts/ssd_detect.py", line 142, in get_disk_type_by_poolID\n', 'Exception\n']
[2021-03-03 21:41:07,763][ BakWorker][E][qnap.cloudconnector3.common.executor][executor.py:__call__:284] : Subtask error: TransferTask<{'_state': 'ERROR', '_exc_info': (<class 'ucc.exception.ConflictError'>, ConflictError(<Response [409]>), <traceback object at 0x7fdb740d9d48>), '_cancel': <threading.Event object at 0x7fdb540e6198>, '_on_finish': None, '_on_error': None, 'src_cc': <ucc.local.LocalClient object at 0x7fdb8864b2e8>, 'src_key': '0000000_0000000000117202.qsv', 'dst_cc': <qswiss.observer.MethodCallObservable object at 0x7fdb88652320>, 'dst_key': 'dedup/vault/0000000_0000000000117/0000000_0000000000117202.qsv', 'src_record': <class 'qnap.cloudconnector3.backup.replication.worker.Record'>, 'dst_record': None, 'data': <_io.BytesIO object at 0x7fdb303bda40>, 'preprocessors': [], 'operation': 'UPLOAD', 'total_size': 8388664, 'transferred_size': 0, 'src_transferred_size': 0, 'dst_transferred_size': 0, 'name': '0000000_0000000000117202.qsv', 'start_time': 1614829191.5476763, 'check_time': 0, 'storage_class': None, 'tagging': None, 'on_progress': None, 'client_mtime': None}>
Traceback (most recent call last):
File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/ucc/common.py", line 94, in outter_func
File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/ucc/box.py", line 644, in create_file
File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/ucc/box.py", line 825, in _create_or_overwrite_file
File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/ucc/chunk_uploader.py", line 83, in upload
File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/ucc/box.py", line 717, in _normal_upload
File "/var/lib/jenkins/jobs/CC3_PY3_QPKG/workspace/ARCH/x86_64/master/master/qpkg/output/shared/python/lib/python3.7/site-packages/ucc/box.py", line 190, in _upload_file
File "/share/CACHEDEV1_DATA/.qpkg/HybridBackup/CloudConnector3/python/lib/python3.7/site-packages/requests/models.py", line 937, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 409 Client Error: Conflict for url: https://upload.box.com/api/2.0/files/content

[schferenc]

Hi,

I've just had the same issue with Google Cloud Storage backup jobs, but found another topic, which helped me:
viewtopic.php?t=140823

Briefly, you need to create ONE WAY Sync job instead of backup job. Now it works for me.

[ian.addison]

See my warning in that thread. A one-way sync is not the same as a backup.

[P3R]

See my warning in that thread. A one-way sync is not the same as a backup.

That's correct and to hopefully clarify, the most important difference between a one-way-sync and a backup in HBS3 is that with the backup it's possible to enable versions if you have enough storage capacity available on your backup destination. With a versioned backup you have the possibility to "go back in time" and select different backup versions (similar to snapshots) from different points in time. That's typically critical if you have been hit by a ransomware and you haven't disabled your automatically scheduled one-way-sync or single-instance backup before it run, as that will overwrite your still unharmed backup copy with the ransomware encrypted data files.

The by far best protection against the consequences of ransomware though is to not directly expose the NAS on the Internet.

A NAS that isn't directly available from a public network have a very low risk of intrusions and ransomware compared to exposed systems.

If remote access is necessary, then use a remote access VPN connetion to protect it.

A versioned backup is very valuable for many threats, not only ransomware, so it's very much recommended if you have the storage capacity available.

[manwecke]

Same problem since app. one week.

Fixed the issue with reduction of "Concurrent file processing limit = 1 rule
See attached screen shot.

With this parameter backups with app. 70GB run sucessfull.

[NAStrovia]

I have had the "Failed to upload metadata" OneDrive backup problem since QTS 5.0 rolled out under two different HBS3 versions (currently running 18.0.1125).

I submitted a ticket to QNAP support and heard back today that the problem will be solved in upcoming HBS3 version 18.0.1206.

In the meantime, I just tried the solution proposed by manwecke above (reduced "Concurrent file processing limit" from the default of 5 to 1) and it also worked for me. Thanks!

[grifos]

Hi,
Just to up give a small updated, issue still occuring with HBS 19.0.0303, trying to backup on Icedrive with webdav.

[grifos]

I opened a ticket with the support.
The official reply they gave me was: The HBS supports the WebDAV protocol, providing compatibility with solutions including WebDAV server (QNAP NAS), SME CloudDAV, DriveHQ, myDrive, OpenDrive, and pCloud

So hbs support'ish Webdav