Slow upload to Amazon S3

Post Reply
mikaux
New here
Posts: 9
Joined: Mon Jul 13, 2015 6:16 am

Slow upload to Amazon S3

Post by mikaux »

TVS-471, 4 x 3TB drives in Raid 6.
Gigabyte ethernet home network to internet via a 75MB down 15MB up fibre at cabinet connection. I get those speeds regularly with Speed Test.

I am replicating the TVS to Amazon S3 (Europe) and it is very slow. Admittedly I am uploading around 1.5 TB of data, video, photos etc however looking at the bandwidth usage in the resource monitor I am only uploading at a constant 1.8MB/s.

I have re patched the unit and physically relocated it to my IT cabinet where it is connected to a Netgear 16 port gigabyte switch which is in turn connected to the router (ASUS RT-AC66U).

Am I missing something, should I be seeing faster upload speeds.

Transferring between the mac and the NAS gives great speeds, so I know that the LAN is working fine. I am seeing around 75MB/s transferring video files from Mac to NAS.

The speed for the LAN transfer was showing on Ethernet 2 (I have two Ethernet cables running from the NAS to the Switch). The speed for the uploads to Amazon S3 are showing in Ethernet 1.

Any suggestions appreciated.
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow upload to Amazon S3

Post by P3R »

mikaux wrote:Gigabyte ethernet home network to internet via a 75MB down 15MB up fibre at cabinet connection.
I doubt that. Bandwidth is almost always specified as bits/s by networking professionals not Bytes/s as reported in most end user fronting operating systems.

B/s=Bytes/s=Bps
b/s=bits/s
I get those speeds regularly with Speed Test.
If that is http://www.speedtest.net/, you'll notice that they also use the unit (M)bits/s.
I am uploading around 1.5 TB of data, video, photos etc however looking at the bandwidth usage in the resource monitor I am only uploading at a constant 1.8MB/s.
The bandwith reported in Resource monitor give you Byte/s (abbreviated with capital letter B) and 1.8 MByte/s *8 (the number of bits in a byte)=14.4 Mbits/s, so close to your specified maximum internet upload speed.
The speed for the LAN transfer was showing on Ethernet 2 (I have two Ethernet cables running from the NAS to the Switch).
When connecting both network cables to a switch, the configuration of it all is critical. It is often incorrectly configured by many Qnap users, and if so may give strange problems in the future, even when now seemingly working perfect as for you.

If you want the community to assist you in verifying your network configuration, please explain the Qnap network configuration in detail (ip addresses, masks, default gateway, any port trunking used and so on) and please share the exact model of Netgear switch in use and if manageable/configurable, please explain how you have it configured.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
mikaux
New here
Posts: 9
Joined: Mon Jul 13, 2015 6:16 am

Re: Slow upload to Amazon S3

Post by mikaux »

Thanks for the quick reply and thanks also for the clear education on MBs and Mbps. Now I understand.

In regards to the Networking side of things. Its pretty simple.

Netgear GS116 Gigabyte unmanaged switch. (http://www.netgear.co.uk/business/produ ... -techspecs). The NAS is plugged directly to that switch from Ethernet ports 1 and 2. The switch then connects to the Router.

There is only one router on my network and two other Netgear switches.

Attached are three screen grabs that may be helpful.
You do not have the required permissions to view the files attached to this post.
P3R
Guru
Posts: 13190
Joined: Sat Dec 29, 2007 1:39 am
Location: Stockholm, Sweden (UTC+01:00)

Re: Slow upload to Amazon S3

Post by P3R »

mikaux wrote:Netgear GS116 Gigabyte unmanaged switch. (http://www.netgear.co.uk/business/produ ... -techspecs). The NAS is plugged directly to that switch from Ethernet ports 1 and 2.
The NAS shouldn't have multiple adresses in the same ip network.

If you want to use multiple interfaces connected into the same ip network, you should use port trunking.

The only port trunking options supported when using an unmanaged switch (called General switches in the Qnap documentation) are:
  • Active Backup - fault tolerance only.
  • Balance-tlb (Adaptive Transmit Load Balancing) - load balancing and fault tolerance.
  • Balance-alb (Adaptive Load Balancing) - load balancing and fault tolerance.
Personally I have used Balance-tlb on unmanaged switches and it worked for me.
RAID have never ever been a replacement for backups. Without backups on a different system (preferably placed at another site), you will eventually lose data!

A non-RAID configuration (including RAID 0, which isn't really RAID) with a backup on a separate media protects your data far better than any RAID-volume without backup.

All data storage consists of both the primary storage and the backups. It's your money and your data, spend the storage budget wisely or pay with your data!
mikaux
New here
Posts: 9
Joined: Mon Jul 13, 2015 6:16 am

Re: Slow upload to Amazon S3

Post by mikaux »

Thanks again for the prompt and useful information. I will make the Port Trunking changes today. It is pretty obvious I need to do some proper reading of the documentation and get familiar with this unit.
mikaux
New here
Posts: 9
Joined: Mon Jul 13, 2015 6:16 am

Re: Slow upload to Amazon S3

Post by mikaux »

Port trunking complete and it all seems to be working as suggested. Now to start understanding the power of this machine.
nickrego
New here
Posts: 2
Joined: Wed Apr 26, 2017 11:09 pm

Re: Slow upload to Amazon S3

Post by nickrego »

P3E: Does the Hybrid Backup app report transfer rates in Bytes, rather than Bits because its label it MBps ?
You do not have the required permissions to view the files attached to this post.
Post Reply

Return to “Amazon S3”