FTPS connection failed

[pevely]

Hi guys,

I have Qnap TS-459 pro with last 3.4.0 firmware. I have problem with FTPS connection. The FTP clients (File-zilla or Total Commander) login to the server succesfully and starting downloading file (12GB mkv file). In 15 - 20 minutes period connections dropped. Logfile from File-Zilla say:

GTnuTLS error -37: Rehandshake was requested by the peer.

Whitout SSL FTP works perfect. FTPS connection dropped from outside (internet) and from local network too.
Has anybody same problem?

[schumaku]

Hi,

Welcome to the wonderful world of security!

Unless I'm wrong, your tranfer is stopped after exactly 1 GB (1'073'741'824) _or_ always after the same time on slower connections. This is the moment the ftp server is asking for a re-handshake. What is correct from the security view, is likely a show stopper for many ftp clients: Many seem to be built on the same code - which simply does not support re-negotiation (and a new key exchange - that's why it's required). I don't knw about Total Commander - but certianly FileZilla does not support the certificate re-handshake yet.

In other words: Everybody using these (no doubt - very popular) ftp clients is likely experiencing the same actually.

It's a pitty we can't simply edit proftpd.conf due to the tight integration on the NAS. Very likley, supressing the TLSRenegotiation would do the job - this statement is missing in the configuration file:

Code: Select all

TLSRenegotiate required off

I'll push this to QNAP again.

Edit: Did some searching on the net again - here it's explained, too: http://www.proftpd.org/docs/howto/TLS.html

Regards,
-Kurt.

[popoman]

Hi Pevely
Sorry I don't have your problem , what is your FileZilla version and FTPS via LAN or WAN ?

[schumaku]

Sorry I don't have your problem , what is your FileZilla version and FTPS via LAN or WAN ?

The TLS Rehandshake is initiated by the ftp sever on a per-file base every 1 GB. Are you able to transfer files larger than 1 GB by ftps ?

[AlexKe]

Hi Folks,

Sorry, we cannot reproduce this issue. Would you like to provide the remote link and we will check onsite?

[schumaku]

Alex,

The facts are clear: According to http://www.proftpd.org/docs/directives/linked/config_ref_TLSRenegotiate.html :

By default, mod_tls will perform renegotiations if supported, on the control channel after 4 hours, and on the data channel after one gigabyte of transferred data. The default timeout for a renegotiation is 30 seconds.

Because of ther are almost no ftp clients able to handle this in a correct way, I kindly ask QNNAP to add the single config statement to the ftp configuration:

Code: Select all

TLSRenegotiate required off

This does not fulls suppress the option. If a client explicitly requests a reregotiation, it still can be done.

Plenty of reports in the worldwide QNAP forums around according to this problem - either at one GB data, or after four hours. Following up because there is jsut a new post here viewtopic.php?f=189&t=43522 asking for a general timeout after 2.5 hours - sometjing I'm not aware of. WIll challenge the user if this is ftps and after 1 GB, too.

-Kurt.

[AlexKe]

Hi Folks,

We cannot reproduce in our lab. I also tried to copy one file 16GB to TS-119P+ via FTP with SSL/TLS (Explicit) in the 10/100MB LAN network, and the file can be transferred correctly without disconnection issue.

[schumaku]

Probably time to configure your test client to really used ftpes ... in FileZilla, create a new host entry, and select Encryption: Require explicit FTP over TLS...

Client:

FileZilla Client
----------------
Version: 3.4.0

Build information:
Compiled for: i586-pc-mingw32msvc
Compiled on: x86_64-unknown-linux-gnu
Build date: 2011-03-27
Compiled with: i586-mingw32msvc-gcc (GCC) 4.2.1-sjlj (mingw32-2)
Compiler flags: -g -O2 -Wall -g -fexceptions

Linked against:
wxWidgets: 2.8.12
GnuTLS: 2.10.4

Operating system:
Name: Windows Server 2003 (build 3790, Service Pack 2)
Version: 5.2
Platform: 32 bit system

Server: QNAP TS-509 Pro Version 3.4.2 build 0331T

Case 1 - normal:

Closed session and FIleZille, login, then get file:

Status: Resolving address of ts-509pro
Status: Connecting to 10.10.1.109:21...
Status: Connection established, waiting for welcome message...
Response: 220 NASFTPD Turbo station 2.x 1.3.2e Server (ProFTPD) [::ffff:10.10.1.109]
Command: AUTH TLS
Response: 234 AUTH TLS successful
Status: Initializing TLS...
Status: Verifying certificate...
Command: USER user
Status: TLS/SSL connection established.
Response: 331 Password required for user
Command: PASS ********
Response: 230 User user logged in
Command: OPTS UTF8 ON
Response: 200 UTF8 set to on
Command: PBSZ 0
Response: 200 PBSZ 0 successful
Command: PROT P
Response: 200 Protection set to Private
Status: Connected
Status: Starting download of /Public/4gb
Command: CWD /Public
Response: 250 CWD command successful
Command: PWD
Response: 257 "/Public" is the current directory
Command: TYPE A
Response: 200 Type set to A
Command: PASV
Response: 227 Entering Passive Mode (10,10,1,109,217,66).
Command: RETR 4gb
Response: 150 Opening ASCII mode data connection for 4gb (4294967296 bytes)
Error: GnuTLS error -37: Rehandshake was requested by the peer.
Error: Could not read from transfer socket: ECONNABORTED - Connection aborted
Response: 450 Transfer aborted. Link to file server lost
Error: File transfer failed after transferring 1'073'741'824 bytes in 72 seconds
...

Case 2 - Rehandshake requested every single GB tranfered - why ever, FileZilla is (under some conditions) able to continue retrieving the file using the continuation capabilities. However, QNAP can't build in this likely specific FileZilla case:

Status: Resolving address of ts-509pro
Status: Connecting to 10.10.1.109:21...
Status: Connection established, waiting for welcome message...
Response: 220 NASFTPD Turbo station 2.x 1.3.2e Server (ProFTPD) [::ffff:10.10.1.109]
Command: AUTH TLS
Response: 234 AUTH TLS successful
Status: Initializing TLS...
Status: Verifying certificate...
Command: USER user
Status: TLS/SSL connection established.
Response: 331 Password required for user
Command: PASS ********
Response: 230 User user logged in
Command: SYST
Response: 215 UNIX Type: L8
Command: FEAT
Response: 211-Features:
Response: MDTM
Response: MFMT
Response: UTF8
Response: AUTH TLS
Response: MFF modify;UNIX.group;UNIX.mode;
Response: MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.mode*;UNIX.owner*;
Response: LANG en-US*
Response: PBSZ
Response: PROT
Response: REST STREAM
Response: SIZE
Response: 211 End
Command: OPTS UTF8 ON
Response: 200 UTF8 set to on
Command: PBSZ 0
Response: 200 PBSZ 0 successful
Command: PROT P
Response: 200 Protection set to Private
Status: Connected
Status: Retrieving directory listing...
Command: PWD
Response: 257 "/" is the current directory
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (10,10,1,109,220,125).
Command: MLSD
Response: 150 Opening ASCII mode data connection for MLSD
Response: 226 Transfer complete
Status: Directory listing successful
Status: Retrieving directory listing...
Command: CWD Public
Response: 250 CWD command successful
Command: PWD
Response: 257 "/Public" is the current directory
Command: PASV
Response: 227 Entering Passive Mode (10,10,1,109,218,144).
Command: MLSD
Response: 150 Opening ASCII mode data connection for MLSD
Response: 226 Transfer complete
Status: Directory listing successful
Status: Resolving address of ts-509pro
Status: Connecting to 10.10.1.109:21...
Status: Connection established, waiting for welcome message...
Response: 220 NASFTPD Turbo station 2.x 1.3.2e Server (ProFTPD) [::ffff:10.10.1.109]
Command: AUTH TLS
Response: 234 AUTH TLS successful
Status: Initializing TLS...
Status: Verifying certificate...
Command: USER user
Status: TLS/SSL connection established.
Response: 331 Password required for user
Command: PASS ********
Response: 230 User user logged in
Command: OPTS UTF8 ON
Response: 200 UTF8 set to on
Command: PBSZ 0
Response: 200 PBSZ 0 successful
Command: PROT P
Response: 200 Protection set to Private
Status: Connected
Status: Starting download of /Public/soundeffects.rar
Command: CWD /Public
Response: 250 CWD command successful
Command: PWD
Response: 257 "/Public" is the current directory
Command: TYPE I
Response: 200 Type set to I
Command: PASV
Response: 227 Entering Passive Mode (10,10,1,109,219,93).
Command: RETR soundeffects.rar
Response: 150 Opening BINARY mode data connection for soundeffects.rar (2565961553 bytes)
Error: GnuTLS error -37: Rehandshake was requested by the peer.
Error: Could not read from transfer socket: ECONNABORTED - Connection aborted
Response: 450 Transfer aborted. Link to file server lost
Error: File transfer failed after transferring 1'073'741'824 bytes in 73 seconds
Status: Starting download of /Public/soundeffects.rar
Command: PASV
Response: 227 Entering Passive Mode (10,10,1,109,220,115).
Command: REST 1073741824
Response: 350 Restarting at 1073741824. Send STORE or RETRIEVE to initiate transfer
Command: RETR soundeffects.rar
Response: 150 Opening BINARY mode data connection for soundeffects.rar (1492219729 bytes)
Error: GnuTLS error -37: Rehandshake was requested by the peer.
Error: Could not read from transfer socket: ECONNABORTED - Connection aborted
Response: 450 Transfer aborted. Link to file server lost
Error: File transfer failed after transferring 1'073'741'824 bytes in 75 seconds
Status: Starting download of /Public/soundeffects.rar
Status: Testing resume capabilities of server
Command: PASV
Response: 227 Entering Passive Mode (10,10,1,109,220,88).
Command: REST 2565961552
Response: 350 Restarting at 2565961552. Send STORE or RETRIEVE to initiate transfer
Command: RETR soundeffects.rar
Response: 150 Opening BINARY mode data connection for soundeffects.rar (1 bytes)
Response: 226 Transfer complete
Command: PASV
Response: 227 Entering Passive Mode (10,10,1,109,220,68).
Command: REST 2147483648
Response: 350 Restarting at 2147483648. Send STORE or RETRIEVE to initiate transfer
Command: RETR soundeffects.rar
Response: 150 Opening BINARY mode data connection for soundeffects.rar (418477905 bytes)
Response: 226 Transfer complete
Status: File transfer successful, transferred 418'477'905 bytes in 29 seconds

Case 3:

On the control session disconnection, the TLS session is STILL not correctly shutdown - just let the (ftpes) connection idle for ten minutes and you will see this:

...
Response: 421 Idle timeout (600 seconds): closing control connection
Error: GnuTLS error -9: A TLS packet with unexpected length was received.
Status: Server did not properly shut down TLS connection
Error: Could not read from socket: ECONNABORTED - Connection aborted
Error: Disconnected from server

[WonkoTheSane]

Hi,

I have the same issue on my TS-410 running firmware 3.4.4 Build 0718T.

It would be great if the configuration value mentioned by Kurt could be
added to the next firmware update.


Regards

Matthias

[schumaku]

Shame...

Code: Select all

TLSRenegotiate required off

...still no part of the hardcoded proftpd.conf in v3.5.0

[WonkoTheSane]

Shame...

Code: Select all

TLSRenegotiate required off

...still no part of the hardcoded proftpd.conf in v3.5.0

Yeah, this still isn't fixed. Personally, I've worked around this problem
using autorun.sh, but that's definitely not for everyone.

You can easily reproduce the connection problems by transferring a file >1GB
using WinSCP or TotalCommander. Happens every single time.


Regards

Matthias

[AlexKe]

Thanks for reminding. We will check it.

- connection drop while transferring large file over 12GB via FTPS

[WonkoTheSane]

Thanks for reminding. We will check it.

Good to hear. Another thing to look into would be that when the external IP
address changes, the FTP server keeps responding with the outdated one to
the PASV command, which always causes a timeout during the initial LIST
command that follows.

It would be great to ditch my autorun.sh workaround for this issue as well.

Regards

Matthias

[AlexKe]

Hi WonkoTheSane

Do you mean the NAS function as a FTP server behind a router? After the WAN ip address changed, the FTP server still response the previous WAN IP address?

[schumaku]

Do you mean the NAS function as a FTP server behind a router? After the WAN ip address changed, the FTP server still response the previous WAN IP address?

Yes, this is what we see, too. the WAN IP is not checked frequently enough, or proftpd fake WAN IP is not refreshed in a timely manner.