SMB file names

Questions about using NAS on Mac OS.
m_emelchenkov
Starting out
Posts: 32
Joined: Fri Aug 21, 2020 2:33 pm

SMB file names

Post by m_emelchenkov »

Hello. How to allow "/" characters in file names in SMB? I have many such files because used AFP before. Now Apple made AFP outdated but did not offer equal replacement. SMB breaks file naming.
torch1
Know my way around
Posts: 137
Joined: Thu Jul 04, 2019 4:53 am

Re: SMB file names

Post by torch1 »

Was the "/" used in the file name or the path to the file? Unusual in the filename for a *nix....
You may have to SSH into your QNAP, find the files with the illegal "/" and replace it with a character that is allowed. I don't know which linux utility you will have to use (bash?).
m_emelchenkov
Starting out
Posts: 32
Joined: Fri Aug 21, 2020 2:33 pm

Re: SMB file names

Post by m_emelchenkov »

@torch1, "/" in the file name is _legal_ character for macOS. I do not want to change file names, I want to make QNAP's SMB implementation compatible with it, if possible.
torch1
Know my way around
Posts: 137
Joined: Thu Jul 04, 2019 4:53 am

Re: SMB file names

Post by torch1 »

Perhaps "/" is legal in the macOS, but I suspect that the "/" has been converted to a ":" when stored on the disk. In Terminal, list the files in the directory

Code: Select all

>ls -al
m_emelchenkov
Starting out
Posts: 32
Joined: Fri Aug 21, 2020 2:33 pm

Re: SMB file names

Post by m_emelchenkov »

Of course it is. It is how macOS FS works internally. And what? It can't help solve my situation in any way. If you know the solution (i.e. options for smb.conf), please share. But I see you don't know.
torch1
Know my way around
Posts: 137
Joined: Thu Jul 04, 2019 4:53 am

Re: SMB file names

Post by torch1 »

You are assuming there is an easy solution that doesn't involve running some command on the QNAS directly. Well, good luck.
https://www.samba.org/samba/docs/curren ... onf.5.html
Mousetick
Experience counts
Posts: 1081
Joined: Thu Aug 24, 2017 10:28 pm

Re: SMB file names

Post by Mousetick »

You should be able to create files with / in their names from the macOS client to the SMB server on the NAS.

You didn't provide enough information about your situation. You said you have many existing files with / characters. Where are these files currently stored, is the / currently transcoded in the native storage, and from where are you trying to access them?
m_emelchenkov
Starting out
Posts: 32
Joined: Fri Aug 21, 2020 2:33 pm

Re: SMB file names

Post by m_emelchenkov »

@Mousetick, thanks, you gave me an idea. I tried to create "Hello / World" folder via SMB. OK, created. I can open this folder, create more folders inside. Good. But: if you'll open the same volume with AFP, you'll get this: https://i.postimg.cc/J7fNBycL/badchar.png (and you can open it). If I create "Hello / World" folder in AFP, it will snows correctly in SMB, but you can't open this folder. Currently, I have many folders copied via AFP so it's not possible to open them with SMB. I see it's encoding bug somewhere, and I wonder if I can fix it with config option, or need to re-convert encoding for all such folders?
Mousetick
Experience counts
Posts: 1081
Joined: Thu Aug 24, 2017 10:28 pm

Re: SMB file names

Post by Mousetick »

Unfortunately / and other special characters in file names which are not supported on Windows but are perfectly valid on macOS and Linux are not interoperable between the AFP (Netatalk) and SMB (Samba) services on the NAS. This is a shortcoming of QNAP's configuration of those services.

/ on macOS becomes / on Linux NAS via AFP
/ on Linux NAS becomes / on macOS via AFP
/ on macOS becomes ⍰ on Linux NAS via SMB
/ on Linux NAS becomes ⍰ on macOS via SMB

When macOS uses SMB it assumes that the server is Windows-based and cannot handle / or other invalid characters on Windows, so it transcodes them into Unicode private range. What's happening to you is the same process but in reverse: the / currently natively stored on Linux NAS because it came from AFP transfer, is transcoded back to Unicode private range, because the NAS assumes the client is Windows and cannot handle /.

AFAIK You have 3 possible options to resolve the problem:
1) Copy the files from AFP to SMB through the mac client. This will transcode the filenames in the correct direction. After this step you must stop using AFP completely and only use SMB. And it's quite inefficient and time-consuming depending on how much stuff you have to copy. I recommend copying rather than moving as it's safer if something goes wrong you still have the originals.
2) Rename the files on the NAS to apply transcoding manually and directly, replacing / with the appropriate Unicode character, using shell scripts. You can do this directly on the NAS filesystems while logged in via SSH, or you can do this on the mac Terminal while the network share is mounted via AFP. You'll need to figure out the correct Unicode private range character to use, and how to write the script. After this step you must stop using AFP completely and only use SMB.
3) Change the Samba SMB server configuration on the NAS so that it handles / and other special characters natively, nullifying the transcoding done by macOS over SMB. You'll need to edit the Samba configuration file on the NAS. Those changes are not supported by QNAP and may be lost after a restart, after modifying the service configuration through the QTS admin UI, after adding/modifying shared folders, or after a firmware update.

Following are the settings I use on my non-QNAP Samba server (these should normally be put in the [global] section):

Code: Select all

mangled names = no
case sensitive = auto
preserve case = yes
fruit:encoding = native
If there are existing settings as above in the smb.conf, they need to be commented out or removed. For example, on QNAP NASes:

Code: Select all

mangled names = yes
needs to be changed to

Code: Select all

mangled names = no
This configuration stores / ? * and similar invalid Windows file name characters as-is on the NAS, and they appear as-is on macOS. The handling is a bit convoluted but the results are worthwhile: / on NAS gets transcoded into ⍰ by Samba SMB server before sending filename to macOS, then macOS transcodes ⍰ back to /, and vice-versa in the other direction. This configuration should be interoperable between AFP and SMB so you should be able continue using either or both with this setup. And you don't need to rename the existing files.

Good luck.
m_emelchenkov
Starting out
Posts: 32
Joined: Fri Aug 21, 2020 2:33 pm

Re: SMB file names

Post by m_emelchenkov »

@Mousetick "fruit:encoding = native" did the trick! Thanks a lot! I am impressed by your such detailed description of my situation, it's almost an article, not a forum post. Also, you replied in the same detailed and useful manner on my another problem (about POSIX permissions). So I want to express my gratitude, if it is acceptable to you, I'd like to make a donation to you, please let me know can I do it, i.e. PayPal/Paysend/etc.—I don't see how to contact you via PM, so please left your contacts here or mail me at [removed]. Thanks again!!!
Last edited by m_emelchenkov on Wed May 19, 2021 2:54 pm, edited 1 time in total.
Mousetick
Experience counts
Posts: 1081
Joined: Thu Aug 24, 2017 10:28 pm

Re: SMB file names

Post by Mousetick »

m_emelchenkov wrote: Mon May 17, 2021 4:57 pm I don't see how to contact you via PM, so please left your contacts here or mail me at xxxxxx.
BTW you may want to remove your email address (by editing your previous post) rather than leave it for the world to see. You might get spam because of it, or worst case, someone else impersonating me may contact you asking for a donation :)
m_emelchenkov
Starting out
Posts: 32
Joined: Fri Aug 21, 2020 2:33 pm

Re: SMB file names

Post by m_emelchenkov »

Mousetick wrote: Tue May 18, 2021 4:20 pm
m_emelchenkov wrote: Mon May 17, 2021 4:57 pm I don't see how to contact you via PM, so please left your contacts here or mail me at xxxxxx.
BTW you may want to remove your email address (by editing your previous post) rather than leave it for the world to see. You might get spam because of it, or worst case, someone else impersonating me may contact you asking for a donation :)
Spam not a problem, but potential cheater is a security flaw, you are right :). Removed.
mikebw
First post
Posts: 1
Joined: Thu Jun 10, 2021 6:07 am

Re: SMB file names

Post by mikebw »

m_emelchenkov wrote: Mon May 17, 2021 4:57 pm @Mousetick "fruit:encoding = native" did the trick!
@m_emelchenkov, Could you please provide an update on whether or not the change(s) you made to the config file persist after a restart or system update as suggested could be possible by @Mousetick ?

"may be lost after a restart, after modifying the service configuration through the QTS admin UI, after adding/modifying shared folders, or after a firmware update."
m_emelchenkov
Starting out
Posts: 32
Joined: Fri Aug 21, 2020 2:33 pm

Re: SMB file names

Post by m_emelchenkov »

@mikebw, yes, config will be overwritten after service restart. To prevent this I wrote a custom autorun script, it patches necessary files (and do some other things not related to SMB but related to my other needs). If you are interested in my solution please mail m aaat emelchenkov dooot pro.
Mousetick
Experience counts
Posts: 1081
Joined: Thu Aug 24, 2017 10:28 pm

Re: SMB file names

Post by Mousetick »

mikebw wrote: Thu Jun 10, 2021 6:23 am
Similar request and solution in this thread:
need help making a change in smb.conf "stick" after reboot
The specific changes to smb.conf are different, but the general approach is the same (using autorun script).
Post Reply

Return to “Mac OS”