TS-509 unstable network performance

[Angelo]

Hi,

I'm using a TS-509 in my mac environment for multimedia storage. The TS-509 is wired with two Ethernet cables into a Linksys Gigabit switch (EG008W). The network tab on the QNAP shows that it's connected at a speed of 1000MBPS. While I'm transferring 700GB from one disk to another on the same NAS I've noticed varying transfer speeds. The highest speed I get is about 50MB/s (mainly the first 15 minutes of a transfer) versus 345Kb/s at the end. But the speed varies during the whole transfer of the data. It's almost a wave pattern...slow...speed is pickup...and slowing down again. The speed varies, after the initial high speed of 15 minutes, between 1.0 MB/s and 27.0 MB/s. See below the graphs:

cpu usage.jpg

Bandwith.jpg

Anybody an idea how to solve this? (BTW: Twonky, TM and all other applications are switched off)

My network is setup with port trunking with the option set on Active Backup(Fail Over). The speed doesn't change if I change the port trunking option to Balance-rr. So in the end I left it to Active Backup. I start to wonder if Port Trunking actually works on the linksys switch? Anybody knows?

Many thanks for any help.

[sodium]

you tell us that you transfer a file from 1 disk on the nas to another...
How do you copy the file? the only 2 correct ways are
- use filemaneger on the nas
- ssh into the box and move the file
any other way would pull the file to your desktop and then transfer it back to the nas... (while the above 2 options moves the file locally without transferring it over the network)

you have to tell us more, about your setup...

filesystem used (ext3 or ext4)
network filesystem used (afp, smb, nfs)
client setup (wifi or cabled)
mtu size (1500 or bigger)
client OS (OSX, Linux or Windows)
etc

There are a lot of parameters influencing the speed of the nas (and the graphs on the nas do not help they are only to indicate not to measure)

all the balance methods only help when there are multiple hosts writing to the nas... it does nothing for single host transfers to the nas...

[Angelo]

@sodium,

First thanks for the reply; it's clear you know your way around this.

You are right about copying files between disk on the NAS. I did it in a stupid way using Forklift a mac app...so indeed all the files are being transferred over the network. Was focussed on getting the NAS quickly in the network using the default settings not even touching the web file manager. The shop where I bought the QNAP installed the disks and formatted them into ext4 which as you said, in an earlier post, isn't most stable so I've reformatted all disks back to ext3.

So to answer your questions;
-filesystem: ext3
-network filesystem: afp
-client setup: wired macpro, two wired media streamers (popcorn c200), another NAS Buffalo Terastation, macbooks (wifi)
-MTU size 1500
-client os: OSX, Linux on Buffalo NAS, not sure about OS on popcorn but I suppose Linux as well

I was copying the files from one disk on the QNAP via macpro to another disk on QNAP.

Even though it wasn't the smartest way of moving files across the QNAP it's still strange that the network performance is so varying. And as the graphs showed also the CPU usage was going up and down in the same rhythm as the transfer speed. How to solve?

About balancing. Your point is understood but would it make sense to set it up as balanced-rr if I would use multiple hosts? Like using multiple macbooks, macpro and streamers etc?

[sodium]

[...]
your settings are all oke.

I was copying the files from one disk on the QNAP via macpro to another disk on QNAP.

As you may have noticed, this is not the fastest way

Even though it wasn't the smartest way of moving files across the QNAP it's still strange that the network performance is so varying. And as the graphs showed also the CPU usage was going up and down in the same rhythm as the transfer speed. How to solve?

are you sure the rebuild of your disks is complete (home >> Disk mangement >> RAID management) Look at the Status it should say: Ready if you see anything else its still rebuilding the RAID set and that is the reason the systems is slow...

if you have a status ready then we have a difficult one... (and an even harder on to prove) but most likely it's tech stuff

my advise would be :

install the Optware-ipkg (at Home >> Applications >> QPKG plugins)
install the dstat (at the command prompt

Code: Select all

ipkg install dstat

)
start

Code: Select all

dstat -a

then you will see something like this

Code: Select all

usr sys idl wai hiq siq| read  writ| recv  send|  in   out | int   csw 
  2   2  95   2   0   0| 220k  414k|   0     0 |   3B    8B| 229   285 
  5   7  86   2   0   0|  33k   96k| 538B 1428B|   0     0 | 300   794 
  1   1  98   0   0   0|   0     0 |1289B 3415B|   0     0 |  91   246 
  1   1  98   0   0   0|   0     0 | 156B  428B|   0     0 |  68    98 
  0   0 100   0   0   0|   0   220k| 140B 2781B|   0     0 | 119   147 
  0   0 100   0   0   0|   0     0 | 280B  680B|   0     0 |  83   106 
  1   0  99   0   0   0|   0    80k| 140B  476B|   0     0 |  96   126 
  0   2  98   0   0   0|   0     0 | 140B  476B|   0     0 |  91   251 
  1   0  99   0   0   0|   0     0 |  70B  358B|   0     0 |  75   105 
  0   0 100   0   0   0|   0     0 | 140B  476B|   0     0 |  74   107 
  1   1  98   0   0   0|   0     0 | 140B  476B|   0     0 |  82   110 
  0   0 100   0   0   0|  12k   64k| 140B  476B|   0     0 | 102   130 
  1   1  98   0   0   0|   0     0 | 140B  492B|   0     0 | 102   253 
  0   1  99   0   0   0|   0     0 |  70B  358B|   0     0 |  68   100 
  0   0 100   0   0   0|   0     0 | 140B  476B|   0     0 |  86   111 
  1   0  99   0   0   0|   0     0 | 140B  476B|   0     0 |  75   104 
  3   6  89   2   0   0|  18k   24k| 140B  476B|   0     0 | 201   426 
  2   3  94   1   0   0|  15k   72k|1288B 3415B|   0     0 | 197   624 

what you see here is my nas while nothing is going on

then the most imprtand columns are wai (waiting on I/O) and off course read (Data read) write (Data Write)

keep this running for a while and look at the numbers while doing things on the NAS... then you will probably prove that when the performance is degraded the system is writing data to the disks (and calculating the RAID) and this is greatly determined by the RAID level and speed of your disks (and the amount of cache on them)

About balancing. Your point is understood but would it make sense to set it up as balanced-rr if I would use multiple hosts? Like using multiple macbooks, macpro and streamers etc?

I've never been fond of balance-rr myself - yes, it does allow a single stream to go faster than one link, but the cost is reordered packets that will trigger additional TCP ACK segments, and if there is enough reordering, spurious 'fast' retransmissions. Always keep in mind that the speed culprit is not the network -- it's always the disk(s)... and my performance tests did show that all multiple nick modes are only helpful when the number of nodes > 20 (accessing the NAS)

my 2 cents

[Angelo]

Installed Optware and then dstat but when I try to run it it gives me;

Code: Select all

[/etc/init.d] # dstat -a
Terminal width too small, trimming output.

What can I do about this?

[sodium]

Installed Optware and then dstat but when I try to run it it gives me;

Code: Select all

[/etc/init.d] # dstat -a
Terminal width too small, trimming output.

What can I do about this?

make your terminal window bigger

eg by pressing the green button or by pressing the "command" key and - (same as View + Smaller do several times to make the font smaller)

HTH

[Angelo]

Duh!

Of course I tried that...it doesn't matter if I make terminal 200-500 characters width it's till too small???

[sodium]

ok ok
but does it produce any output? and if can you copy it here (please enclose output in code blocks -- next to the Quote)

[Angelo]

Pfff..really annoying...no it doesn't produce any output. I just see an empty terminal screen with a cursor crawling downwards

[Angelo]

Maybe there's something wrong with dstat look at this:

Code: Select all

[~] # dstat --version
Dstat 0.7.0
Written by Dag Wieers <dag@wieers.com>
Homepage at http://dag.wieers.com/home-made/dstat/

Platform posix/linux2
Kernel 2.6.30.6
Python 2.5.5 (r255:77872, Feb 17 2010, 04:07:04) 
[GCC 4.2.1]

Terminal type: linux (color support)
Terminal size: 0 lines, 0 columns

Processors: 1
Pagesize: 4096
Clock ticks per secs: 100

internal:
        
        aio, 
        cpu, 
        cpu24, 
        disk, 
        disk24, 
        disk24old, 
        epoch, 
        fs, 
        int, 
        int24, 
        io, 
        ipc, 
        load, 
        lock, 
        mem, 
        net, 
        page, 
        page24, 
        proc, 
        raw, 
        socket, 
        swap, 
        swapold, 
        sys, 
        tcp, 
        time, 
        udp, 
        unix, 
        vm

It looks like dstat has somehow defined his own terminal width to zero. Any ideas? Reinstall or use different version?

[sodium]

I have the same version...
What is your TERM value

Code: Select all

[~] # echo $TERM

it should say
rxvt

[Angelo]

it says...linux

Code: Select all

[/etc] # echo $TERM
linux

[sodium]

start the OSX terminal program (command + Spacebar >> type Terminal [enter])
In the Command menu click on Terminal >> and choose Preferences
press the plus sign on the left and create a new settings window (default this is called "new settings")
press the default button (making this the new default)
click on advanced tab
click on declare terminal as: rxvt
close the preferences pane
click on shell >> new window >> new settings (or whatever you called it)
connect to the NAS
try dstat again

[Angelo]

Don't understand it. I've done exactly as you said...changed the default terminal setting.

terminal.jpg

Used it but again it shows up as linux. And still it gives me

Code: Select all

[~] # echo $TERM
linux

Do I need to restart my mac for this?

[sodium]

then there must be a local (*1) file overriding the environment variable:

start the terminal as you normally do

type (before you connect to the NAS)

Code: Select all

$ export TERM=rxvt

then connect tot the NAS

(*1) other way to tackle this would be to create a new user on your system and ssh from that user to the NAS (but there is still a possibility the change happens globally on your system so option 1 is safer)