[How-To] The Fastest Way to Move Your Files Internally

Discussion on setting up QNAP NAS products.
Locked
ja666
Know my way around
Posts: 186
Joined: Sat Nov 29, 2014 5:21 pm

[How-To] The Fastest Way to Move Your Files Internally

Post by ja666 »

Supplementing FAQ topic (locked):
From your computer use Total Commander (for Windows).
Please set left and right TC panels with the same network neighborhood path (!!), i.e. \\Qnap\folder_name\... (or mapped_drive:\... ).
Then use TC [F6] move command to move your files within seconds.
User avatar
Spider99
Experience counts
Posts: 1951
Joined: Fri Oct 21, 2011 11:14 pm
Location: UK

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by Spider99 »

Or use File Station on the Nas :)
Tim

TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
ja666
Know my way around
Posts: 186
Joined: Sat Nov 29, 2014 5:21 pm

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by ja666 »

Spider99 wrote:Or use File Station on the Nas :)
The topic is The Fastest Way, not 'The Most Cumbersome and Long QNAP Way' :lol: Spider99, tell us please how do you quickly move i.e. 50 files to 50 different destinations (different folders) with File Station.
My post was about Total Commander omitting Samba limitations.
User avatar
Spider99
Experience counts
Posts: 1951
Joined: Fri Oct 21, 2011 11:14 pm
Location: UK

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by Spider99 »

:lol:

stop being silly - your post is titled "the fastest way to move your files internally" and as you are doing it externally via File commander .... :roll:

50 files to 50 locations - where did that come from :lol:

Samba limitation nothing in your original post about that :lol:

If you want to create a useful thread - then at least explain why its useful and in what situations and for what clients etc etc -rather than ..it good because i think so...
Tim

TS-853A(16GB): - 4.3.4.0483 - Static volume - Raid5 - 8 x 4TB HGST Deskstar NAS
Windows Server + StableBit Drivepool and Scanner ~115 TB Backup Server
TS-412 & TS-459 Pro II: Retired
Clients: 3 x Windows 10 Pro(64bit)
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by schumaku »

ja666 wrote:Supplementing FAQ topic (locked):
From your computer use Total Commander (for Windows).
Please set left and right TC panels with the same network neighborhood path (!!), i.e. \\Qnap\folder_name\... (or mapped_drive:\... ).
Then use TC [F6] move command to move your files within seconds.
ja666 wrote:My post was about Total Commander omitting Samba limitations.
Lol ... and how do you guess \\QNAP\folder_name\ will talk to the NAS? it _is_ SAMBA permitting to move folders and files direct on the NAS, within some limitations. Windows Explorer (you where opening multiple instance windows is possible, too) or any other reasonable file manager can do the job.

When it comes to moving within the same shared folder - it's always fast. When it comes to moving files from one shared folder to another shared folder even if located on the same volume (I guess this is the limitation you talk of) a copy process will be done instead - and when you are on a WLAN or a remote site using a VPN connection, File Station (or for the sake of it using the NAS shell) will be faster then copy the data over the slow network ot the client "moving" ... TC does not help a s**t when using a \\NAS\share connection ... which is SMB handled by SAMBA.

Now you might understand thy the original topic (not referenced why ever) is locked.
m1r0slav
New here
Posts: 3
Joined: Sun Dec 02, 2018 11:13 pm

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by m1r0slav »

Hi guys, I believe original post IS definitely usefull. When I want to:
- a) manipulate the files (execute, copy, move, delete, edit) on my NAS as an USER, I want to use my client machine and it's native applications as much as possible in the most convenient way
- b) fiddle with the settings and installation of new features of my NAS as an ADMIN, then it's ok to log in to QNAP's web interface
ADMIN and USER are (or should be) 2 completely different roles and that's why moving / copying files quickly just using file station is NOT acceptable for me. Advice to use file station migh be OK for someone who is using NAS as a "toy" as a standalone user, but i believe nas should be a professional solution used by companies.

I owned Synology DS918+, but returned it and changed for QNAP TS-453BT3, because of much better connectivity (3 more USB 3.0 ports + quick copy button, 2 TB3 ports, 1 10Gbe LAN port and 2 HDMI 1.4 ports + remote control). Synology's DSM was much more user friendly and polished, than QNAP's QTS, but since QNAP offers more useful features (except few like hybrid raid, btrfs, pcie nvme ssd for cache, ...), I'd like to be able to live with and get used to QTS. Currently I'm strugling with manipulation with files, because moving files is not working as expected on my primary windows machine (I use Total Commander as well, but it's the same with windows explorer) using samba.

- moving files quickly works OK within the same shared folder on QNAP using windows client as ja666 and Spider99 already mentioned
- moving files quickly from one shared folder to another is way too slow (files are not being moved, but downloaded, copied and deleted) on windows client (Total Commander / SMB) on QNAP, although they are in the same volume. Moving using QNAP's internal file station manager is fast (=NOT like Spider99 mentioned) and I believe this shouldn't be restriction of SMB, but it's an issue of QNAP's implementation, because this was working perfect on my previous synology NAS.
- situation with speed of copying files within the same shared folder or between shared folders is exactly the same

My question is - why is copying / moving files working like this and is it possible to change some settings to force the same behaviour to QNAP as it was on the Synology ? Do you use any workarounds as a client, not admin ? I was considering using just one big shared folder for everyone, but I don't like this solution too much.

For me this might be a dealbreaker and I am considering returning back to Synology, but i would miss HDMI + remote control and would need to choose a different model with at least one 10Gbe connection, which could be used as TB3 with additional HW (QNAP's TB3 is 10Gb/s, not 40Gb/s anyways), because let's be honest, data storage and file manipulation is the main purpose of NAS and everything else is just an extra feature on top of it.

Thank you for possible solutions !
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by schumaku »

m1r0slav wrote: Mon Dec 03, 2018 1:27 am ADMIN and USER are (or should be) 2 completely different roles and that's why moving / copying files quickly just using file station is NOT acceptable for me.
Users can login and can be limited to user features only easily - just don't add them to the administrators role.
m1r0slav wrote: Mon Dec 03, 2018 1:27 am - moving files quickly works OK within the same shared folder on QNAP using windows client as ja666 and Spider99 already mentioned
- moving files quickly from one shared folder to another is way too slow (files are not being moved, but downloaded, copied and deleted) on windows client (Total Commander / SMB) on QNAP, although they are in the same volume.
There are plenty of reasons why the SAMBA maintainers restrict the Server-Side Copy Offload feature as well as the Btrfs Enhanced Server-Side Copy Offload to within the same shared folder.

There is however a difference: The SAMBA generic Server-Side Copy Offload does for now only support the FSCTL_SRV_COPYCHUNK_WRITE - this is what a common file copy does make use of - the access to FSCTL_DUPLICATE_EXTENTS_TO_FILE is currently supported in general only on BTRFS.

For example, moving file in File Station does not properly deploy and migrate the file ACL based on the target shared folder - as it does a fast but s***d mv.
ja666
Know my way around
Posts: 186
Joined: Sat Nov 29, 2014 5:21 pm

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by ja666 »

50 files to 50 locations came from real life (i.e. copying/moving files from flash memory) : I feel pis...-off with QNAP/Synology file managers targeting copy/move to to the root of the NAS with every copy/move to job.
While TC remembers last used directory (source and target).
Actually, TC for NAS would be welcome :-D


-----
Spider99 wrote: Sat Jul 30, 2016 3:51 am50 files to 50 locations - where did that come from
bighenry
First post
Posts: 1
Joined: Tue Dec 04, 2018 1:24 am

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by bighenry »

I fully agree with M1r0slav. It is totally nonsense that you use a client to move huge amount of data on a server! A file server must have reliable and fast tools to move data on it without creating any overhead. Otherwise it is not a server.
At this moment QTS can't be used as a file server and that leaves me astonished. By the way, previous versions (let's say until end 2017) could still let you work. Last releases are a serious obstacle to reliability. I have a TVS871RP-U i5, 32GB, equipped with 7 WD Red 2TB disks in RAID6. 4 Volumes, one of which with snapshot set. Virtual Station running
Disks are used at about 50% but lately it is impossible to copy 2 GB even if you wait half an hour, and besides it is a countinuous crash: it crashes while copying files, it crashes while moving files, it crashes while scrubbing. After a crash, the first file system check (usually on 3 volumes out of 4) ends with "Completed" in the log but when you open Storage & Snapshots you see that a File System check is still required (!). After you launch a new check, it ends with "Unmounted" error. After that you reboot and it seems to be ok (no more errors if you run filesystem check). It is 3 years that I look forward to hear of someone who has been helped by help desk: nevere heard of a solved issue. Issues get solved by QTS updates (if they do) and no trick or workaround is ever suggested by help desk.
And I do not mind you telling what happened in 2016 when they developed heavy changes on Snapshot engine: I had to throw away 1 month of backups because of an update. This is not a business tool: this is rather a toy. I need a second NAS and I am wondering if I should turn to a more professional product and also replace the one I own now. In the business you can't loose time in this way!
User avatar
schumaku
Guru
Posts: 43579
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by schumaku »

bighenry wrote: Tue Dec 04, 2018 2:48 am I fully agree with M1r0slav. It is totally nonsense that you use a client to move huge amount of data on a server!
Well, the functionality @M1r0slav mentioned and was talking of is a feature Windows clients can use, and do use when copy within the same shared folder (QNAP, standard SAMBA server) or within a volume (Synology is overriding the SAMBA limitation) are used by tools on the Windows client, like Windows Explorer, Robocopy, ... so not really the same.

Ref. your NAS having problems with data copy and crashing, not much related to this topic. Run a memory test (Diagnostics Tool from the App Center) plus some, alternate https://wiki.qnap.com/wiki/Memory_Test .. then it would be interesting to see the post-mortem information collected hopefully in the Kernel/console log.
bighenry wrote: Tue Dec 04, 2018 2:48 am And I do not mind you telling what happened in 2016 when they developed heavy changes on Snapshot engine: I had to throw away 1 month of backups because of an update.
Many users were hit by this back then - at least those who were relying on snapshots and backup.
StardustOne
Know my way around
Posts: 137
Joined: Sat Sep 15, 2007 5:18 am

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by StardustOne »

From my experience, FileStation 5 was able to move one folder with all it's content in one second. I tried with another one, same one second. But this is only happening if the data is sitting on the same volume.

I have another task running that moves about 250 GB of data from one volume to another volume, on the same NAS also with FileStation 5 Move, depending on the time it shows to complete this task, it depends, sometimes it tells me that it should complete this task in 3 hours, but I also see estimated time for doing this shown as in 16 Days.

I do see the point in having a NAS and tools to move/copy when the time it takes do do this job is taking so long. And yes my Qnap NAS has a 128GB SSD Cache, it does not help a bit.

And the only reason why I even use FileStation, I do not want the creation date for my folders to change. When I do a sync, then the subfolders all get the creation date from today. When I do a Filestation move, thie creation date of subfolders remains unchanged.
ja666
Know my way around
Posts: 186
Joined: Sat Nov 29, 2014 5:21 pm

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by ja666 »

Regarding folders' time & dates: pls check Folder Time Update software, I run it from PC (Win) to adjust folder attributes on my NAS-es according to my needs.
User avatar
Don
Guru
Posts: 12289
Joined: Thu Jan 03, 2008 4:56 am
Location: Long Island, New York

Re: [How-To] The Fastest Way to Move Your Files Internally

Post by Don »

Locked to prevent further necroposting
Use the forum search feature before posting.

Use RAID and external backups. RAID will protect you from disk failure, keep your system running, and data accessible while the disk is replaced, and the RAID rebuilt. Backups will allow you to recover data that is lost or corrupted, or from system failure. One does not replace the other.

NAS: TVS-882BR | F/W: 5.0.1.2346 | 40GB | 2 x 1TB M.2 SATA RAID 1 (System/VMs) | 3 x 1TB M.2 NMVe QM2-4P-384A RAID 5 (cache) | 5 x 14TB Exos HDD RAID 6 (Data) | 1 x Blu-ray
NAS: TVS-h674 | F/W: 5.0.1.2376 | 16GB | 3 x 18TB RAID 5
Apps: DNSMasq, PLEX, iDrive, QVPN, QLMS, MP3fs, HBS3, Entware, DLstation, VS, +
Locked

Return to “Turbo Station Installation & Setup”