Problem maintaining CIFS mount from Ubuntu Virtual Machine to NAS after 4.3.3 update

[majeedg]

I have a TS-870 Pro with Ubuntu 16.04 LTS (kernel 4.4.0-75-generic) running in a VM using CIFS to mount the various shares set up on the host. For the last year it's been running fine just fine, and and prior to that it ran with 14.10 with no issues.

Since updating to QTS 4.3.3.0154 Build 20170413 I've begun to notice that after a brief, but unknown, amount of time that the shares are inaccessible. My first indication was attempting to do a simple 'ls' resulted in the error:

Code: Select all

ls: cannot open directory '.': Host is down

Using 'df' shows that while not all the shares are mounted, they are being remounted... slowly. So slowly that you can catch it in progress as seen below.

Code: Select all

//192.168.1.251/Surveillance 1.5T  1.5T   39G  98% /media/NAS/Surveillance
//192.168.1.251/Backups       22T   14T  8.1T  63% /media/NAS/Backups
//192.168.1.251/Scans        1.5T  1.5T   39G  98% /media/NAS/Scans
//192.168.1.251/Public       1.5T  1.5T   39G  98% /media/NAS/Public

Only the Backups share is fully mounted. Not all shares will remount despite repeatedly using 'df' to view them and reinitiate whatever process remounts them, so a quick 'sudo mount -a' usually does the trick. While 'df' seems to start things, other commands like 'ls' do not, or at least they don't appear to, since I'm not sure how to confirm it without resorting to the 'df' command.

I'm not sure where to begin to diagnose this. The Ubuntu VM has been updated regularly so it could be something there, but the config settings for the mounts have not changed. They look like this:

Code: Select all

//192.168.1.251/Video /media/NAS/Video cifs credentials=/home/majeed/.smbcredentials,rw,iocharset=utf8,uid=1000,gid=1000 0 0

On the NAS the advanced settings for Microsoft Networking only have Enable Asynchronous I/O checked and Highest SMB Version is 3.0.

Not sure what else to add. Hopefully someone out there has a solution.

Thanks, Majeed.

[MrPaul]

I think I'm impacted by the same issue. I'm on a VM and shares seem to time out.

QNAP TS-451 version 4.3.3
4x4tb WD Red
RAID is 2 RAID 1 mirrors (two separate shares)
Host: Windows 10
Hypervisor: VirtualBox 5.1.20
Guest: Ubuntu 16.04.2 LTS
Guest Network: Bridged

I ususally need to reboot to remedy the issue. I get the following when trying to remount.

[16:12]:sudo mount -a
mount error(121): Remote I/O error
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

[majeedg]

So, I seemed to have figured it out.

It turns out that both QTS 4.3.x AND Ubuntu are both to blame. QTS for setting the SMB version to 3.0 without warning of possible issues (that I noticed anyway... maybe it's in the fine print somewhere). Ubuntu for not realizing that the SMB versions were now different and magically changing its settings accordingly. /sarcasm

The solution is to add vers=3.0 to the list of CIFS options on the mount command.

I changed all the entries in my fstab from,

Code: Select all

//192.168.1.251/Video /media/NAS/Video cifs credentials=/home/majeed/.smbcredentials,rw,iocharset=utf8,uid=1000,gid=1000 0 0

to

Code: Select all

//192.168.1.251/Video /media/NAS/Video cifs credentials=/home/majeed/.smbcredentials,rw,iocharset=utf8,uid=1000,gid=1000,vers=3.0 0 0

and it's been stable since I made the change 12 hours ago.

Hope that works for you too.

Another solution would be to simply change the SMB version in the Microsoft Networking Advanced Settings section of the control panel to something more compatible, but where's the fun in that?

[MrPaul]

Thanks alot. This seems to have remedied two of my issues. After over 12hr my mount is stable and on top of that the CPU utilization on my NAS went from > 80% (preventing snapthots from occuring) to < 5%. Unless I follow up here again I'm going to consider this resolved.

[schumaku]

QTS for setting the SMB version to 3.0 without warning of possible issues

QTS does support SMB 1.0, 2.0, 2.1, and 3.0 ... the 3.0 is the max setting.

Ubuntu for not realizing that the SMB versions were now different and magically changing its settings accordingly.

Every other Windows release does introduce newer SMB protocol versions, too. mount.cifs has some defaults (check what is used...), along with the CIFS Kernel module, defining the maximum possible SMB protocol version. Different from a Windows system, mount.cifs does (to my knowledge) not negotiate the best possible protocol.

The solution is to add vers=3.0 to the list of CIFS options on the mount command.

Would be interesting to know which default protocol is implemented. This protocol might run into an issue in QTS 4.3.3 SAMBA implementation.

[be!bob]

The solution is to add vers=3.0 to the list of CIFS options on the mount command.

Would be interesting to know which default protocol is implemented. This protocol might run into an issue in QTS 4.3.3 SAMBA implementation.

According to the mount.cifs manpage (mount.cifs version 6.4):

vers=
SMB protocol version. Allowed values are:
· 1.0 - The classic CIFS/SMBv1 protocol. This is the default.

I have been having a similar problem since updating to the 4.3.3 firmware on my TP-453. This does not seem to fix it for me.

[majeedg]

QTS for setting the SMB version to 3.0 without warning of possible issues

QTS does support SMB 1.0, 2.0, 2.1, and 3.0 ... the 3.0 is the max setting.

I had seen that option in the settings, but to be honest, I'd never before paid attention to what SMB version it had been set to, because it just always worked. I have a number of Desktops and laptops and VMs all running either Ubuntu or MacOS, and it's never been an issue, until this latest firmware update. But still, a warning somewhere, even in the firmware changelog stating, "Hey, FYI, we're pushing up the SMB version. If you notice something odd with your various client's behaviour on mounted shares, maybe check thereabouts." would have gotten me down the road to a solution WAY sooner. By days. Just sayin'.

Ubuntu for not realizing that the SMB versions were now different and magically changing its settings accordingly.

Every other Windows release does introduce newer SMB protocol versions, too. mount.cifs has some defaults (check what is used...), along with the CIFS Kernel module, defining the maximum possible SMB protocol version. Different from a Windows system, mount.cifs does (to my knowledge) not negotiate the best possible protocol.

This was pure sarcasm, btw, but thank you. I know how to use mount.cifs to get the job done, but had never considered it would need to negotiate anything.

The solution is to add vers=3.0 to the list of CIFS options on the mount command.

Would be interesting to know which default protocol is implemented. This protocol might run into an issue in QTS 4.3.3 SAMBA implementation.

I tried to figure that out, but did not come across anything. I see someone else found it. For now the solution for me seems to be to just keep an eye out for changes to the SMB version in the settings after any update and change my mounts accordingly.

[schumaku]

Earlier we have read the complaint why QTS does not push up the max SMB version to the max

Regardless, all SMB versions are supported. Windows and mac OS SMB clients negotiate the max protocol, unless it's fixed somehow - just the mount.cifs is a little bit hmmm limited.

There are some oddities when using Mac OS automount I've not came behind yet. Effect is similar to what you have seen originally, the auto mount point does fall back to root,wheel and the files are not accessible due to the U**x permissions in place. When sudo forcing the access, we see the (auto)mount does still take place, and the files are accessible.

So never say there are no problems, it's still software.

[majeedg]

Well, like they say, you can't please all of the people all of the time.

I've not come across that MacOS bug either. Perhaps it's specific to an older version? All my (3) Macs are updated to Sierra, and Yosemite before that. They don't automount nicely at boot all the time but they've been pretty solid.

Thanks for the feedback.

[thempstead]

Nice to know I'm not alone. I have an Ubuntu VM (16.04) which is showing the same issues with a NAS (TS-431X) running 4.3.3 as the original poster. This is even if I tell the NAS to talk SMB 1.0 rather than 3.0.

I would add the following:
- If the share is mounted read-only onto the Ubuntu VM from the NAS it seems more stable for me. Only seems to drop if it is mounted read-write.
- If the share is mounted read-write then you copy a load of data to it then that seems to work ok. it only seems to drop once it has been idle for a while (even if you have a shell session sitting in that directory on the Ubuntu box). I'm wondering if it's dropping when the NAS spins down the disks as being idle ...?
- The Ubuntu VM will quite happily still have several mounted (read-only) shares from the same NAS even though it's complaining about not being able to remount the read-write one with a host down error.

[thempstead]

Updates on my post above ...

- does still happen if share is mounted read only just less seemingly less often.
- if the share has "failed" as above and I cannot remount it from Ubuntu as above then if I login to the NAS web GUI and do something which exercises the disks, e.g. tell it to look at utilisation in Storage Manager, then if I can go back to the Ubuntu VM and the remount now works.

To me it looks like the NAS is closing the connection on idle.

[oyvindo]

My 3 QNAP servers have always been listed in Network Neighborhood on my Win10 client PC, along with a number of other Windows machines. But after upgrading to 4.3.3 and updating Windows to build 1703 (KB4016871), suddenly only Windows machines are listed. All QNAP NAS servers are gone. They can be reached easily by either mapping a network drive or by typing the UNC manually, but I'm still missing them from the Network listing. All my QNAP servers have SMB 3.0 enabled.
Does anyone have some idea of what's going on?

[dave.michael]

I was having the same issue. The solution of specifying 'vers=3.0' seems to have worked for my TS-451 on 4.3.3. Thanks, OP!

My TS-419P II (fw v4.3.3.0154) , however, seems to still exhibit the same problem regardless of version specified. I tried 3.0 initially, then 2.1, 2.0 then 1.0. Only 1.0 allows me to mount but eventually still, my ACD backups fail and Plex server has issues accessing the media on that device. The TS-419P II also doesn't have the option to set highest SMB version like the newer devices.

Any other tips?

[schumaku]

The TS-419P II also doesn't have the option to set highest SMB version like the newer devices.

It has. First scratch v4.3.3.0154 and update to .0174 from qnap.com manual download.

When it comes to the max SMB version - the control has not made it to the UI of the CAT1 systems, you need to call the readily available smb[N|NN]enable, and check the current settings using smb2status form the NAS shell:

Up to QTS 4.2.x, SMB 2.1 is the max available:

[~] # smb21enable
...

QTS 4.3.x will bring SMB up to the 3.0 subset available in SAMBA 4.4 to all QNAP Marvell Kirkwood NAS.

[~] # smb3enable
...

Check status:

[~] # smb2status

smbd (samba daemon) Version 4.4.9
smbd (samba daemon) is running.
max protocol SMB 3.0 enabled.

[~] # smb<TAB>
smb21enable smb2disable smb2enable smb2status smb3enable smbtools

Enjoy!

-Kurt.

[dave.michael]

The TS-419P II also doesn't have the option to set highest SMB version like the newer devices.

It has. First scratch v4.3.3.0154 and update to .0174 from qnap.com manual download.

When it comes to the max SMB version - the control has not made it to the UI of the CAT1 systems, you need to call the readily available smb[N|NN]enable, and check the current settings using smb2status form the NAS shell:

Up to QTS 4.2.x, SMB 2.1 is the max available:

[~] # smb21enable
...

QTS 4.3.x will bring SMB up to the 3.0 subset available in SAMBA 4.4 to all QNAP Marvell Kirkwood NAS.

[~] # smb3enable
...

Check status:

[~] # smb2status

smbd (samba daemon) Version 4.4.9
smbd (samba daemon) is running.
max protocol SMB 3.0 enabled.

[~] # smb<TAB>
smb21enable smb2disable smb2enable smb2status smb3enable smbtools

Enjoy!

-Kurt.

Worked like a charm. Persists through reboot too! Thanks so very much!