SMB file names
-
- Starting out
- Posts: 32
- Joined: Fri Aug 21, 2020 2:33 pm
SMB file names
Hello. How to allow "/" characters in file names in SMB? I have many such files because used AFP before. Now Apple made AFP outdated but did not offer equal replacement. SMB breaks file naming.
-
- Know my way around
- Posts: 137
- Joined: Thu Jul 04, 2019 4:53 am
Re: SMB file names
Was the "/" used in the file name or the path to the file? Unusual in the filename for a *nix....
You may have to SSH into your QNAP, find the files with the illegal "/" and replace it with a character that is allowed. I don't know which linux utility you will have to use (bash?).
You may have to SSH into your QNAP, find the files with the illegal "/" and replace it with a character that is allowed. I don't know which linux utility you will have to use (bash?).
-
- Starting out
- Posts: 32
- Joined: Fri Aug 21, 2020 2:33 pm
Re: SMB file names
@torch1, "/" in the file name is _legal_ character for macOS. I do not want to change file names, I want to make QNAP's SMB implementation compatible with it, if possible.
-
- Know my way around
- Posts: 137
- Joined: Thu Jul 04, 2019 4:53 am
Re: SMB file names
Perhaps "/" is legal in the macOS, but I suspect that the "/" has been converted to a ":" when stored on the disk. In Terminal, list the files in the directory
Code: Select all
>ls -al
-
- Starting out
- Posts: 32
- Joined: Fri Aug 21, 2020 2:33 pm
Re: SMB file names
Of course it is. It is how macOS FS works internally. And what? It can't help solve my situation in any way. If you know the solution (i.e. options for smb.conf), please share. But I see you don't know.
-
- Know my way around
- Posts: 137
- Joined: Thu Jul 04, 2019 4:53 am
Re: SMB file names
You are assuming there is an easy solution that doesn't involve running some command on the QNAS directly. Well, good luck.
https://www.samba.org/samba/docs/curren ... onf.5.html
https://www.samba.org/samba/docs/curren ... onf.5.html
-
- Experience counts
- Posts: 1081
- Joined: Thu Aug 24, 2017 10:28 pm
Re: SMB file names
You should be able to create files with / in their names from the macOS client to the SMB server on the NAS.
You didn't provide enough information about your situation. You said you have many existing files with / characters. Where are these files currently stored, is the / currently transcoded in the native storage, and from where are you trying to access them?
You didn't provide enough information about your situation. You said you have many existing files with / characters. Where are these files currently stored, is the / currently transcoded in the native storage, and from where are you trying to access them?
-
- Starting out
- Posts: 32
- Joined: Fri Aug 21, 2020 2:33 pm
Re: SMB file names
@Mousetick, thanks, you gave me an idea. I tried to create "Hello / World" folder via SMB. OK, created. I can open this folder, create more folders inside. Good. But: if you'll open the same volume with AFP, you'll get this: https://i.postimg.cc/J7fNBycL/badchar.png (and you can open it). If I create "Hello / World" folder in AFP, it will snows correctly in SMB, but you can't open this folder. Currently, I have many folders copied via AFP so it's not possible to open them with SMB. I see it's encoding bug somewhere, and I wonder if I can fix it with config option, or need to re-convert encoding for all such folders?
-
- Experience counts
- Posts: 1081
- Joined: Thu Aug 24, 2017 10:28 pm
Re: SMB file names
Unfortunately / and other special characters in file names which are not supported on Windows but are perfectly valid on macOS and Linux are not interoperable between the AFP (Netatalk) and SMB (Samba) services on the NAS. This is a shortcoming of QNAP's configuration of those services.
/ on macOS becomes / on Linux NAS via AFP
/ on Linux NAS becomes / on macOS via AFP
/ on macOS becomes ⍰ on Linux NAS via SMB
/ on Linux NAS becomes ⍰ on macOS via SMB
When macOS uses SMB it assumes that the server is Windows-based and cannot handle / or other invalid characters on Windows, so it transcodes them into Unicode private range. What's happening to you is the same process but in reverse: the / currently natively stored on Linux NAS because it came from AFP transfer, is transcoded back to Unicode private range, because the NAS assumes the client is Windows and cannot handle /.
AFAIK You have 3 possible options to resolve the problem:
1) Copy the files from AFP to SMB through the mac client. This will transcode the filenames in the correct direction. After this step you must stop using AFP completely and only use SMB. And it's quite inefficient and time-consuming depending on how much stuff you have to copy. I recommend copying rather than moving as it's safer if something goes wrong you still have the originals.
2) Rename the files on the NAS to apply transcoding manually and directly, replacing / with the appropriate Unicode character, using shell scripts. You can do this directly on the NAS filesystems while logged in via SSH, or you can do this on the mac Terminal while the network share is mounted via AFP. You'll need to figure out the correct Unicode private range character to use, and how to write the script. After this step you must stop using AFP completely and only use SMB.
3) Change the Samba SMB server configuration on the NAS so that it handles / and other special characters natively, nullifying the transcoding done by macOS over SMB. You'll need to edit the Samba configuration file on the NAS. Those changes are not supported by QNAP and may be lost after a restart, after modifying the service configuration through the QTS admin UI, after adding/modifying shared folders, or after a firmware update.
Following are the settings I use on my non-QNAP Samba server (these should normally be put in the [global] section):
If there are existing settings as above in the smb.conf, they need to be commented out or removed. For example, on QNAP NASes:
needs to be changed to
This configuration stores / ? * and similar invalid Windows file name characters as-is on the NAS, and they appear as-is on macOS. The handling is a bit convoluted but the results are worthwhile: / on NAS gets transcoded into ⍰ by Samba SMB server before sending filename to macOS, then macOS transcodes ⍰ back to /, and vice-versa in the other direction. This configuration should be interoperable between AFP and SMB so you should be able continue using either or both with this setup. And you don't need to rename the existing files.
Good luck.
/ on macOS becomes / on Linux NAS via AFP
/ on Linux NAS becomes / on macOS via AFP
/ on macOS becomes ⍰ on Linux NAS via SMB
/ on Linux NAS becomes ⍰ on macOS via SMB
When macOS uses SMB it assumes that the server is Windows-based and cannot handle / or other invalid characters on Windows, so it transcodes them into Unicode private range. What's happening to you is the same process but in reverse: the / currently natively stored on Linux NAS because it came from AFP transfer, is transcoded back to Unicode private range, because the NAS assumes the client is Windows and cannot handle /.
AFAIK You have 3 possible options to resolve the problem:
1) Copy the files from AFP to SMB through the mac client. This will transcode the filenames in the correct direction. After this step you must stop using AFP completely and only use SMB. And it's quite inefficient and time-consuming depending on how much stuff you have to copy. I recommend copying rather than moving as it's safer if something goes wrong you still have the originals.
2) Rename the files on the NAS to apply transcoding manually and directly, replacing / with the appropriate Unicode character, using shell scripts. You can do this directly on the NAS filesystems while logged in via SSH, or you can do this on the mac Terminal while the network share is mounted via AFP. You'll need to figure out the correct Unicode private range character to use, and how to write the script. After this step you must stop using AFP completely and only use SMB.
3) Change the Samba SMB server configuration on the NAS so that it handles / and other special characters natively, nullifying the transcoding done by macOS over SMB. You'll need to edit the Samba configuration file on the NAS. Those changes are not supported by QNAP and may be lost after a restart, after modifying the service configuration through the QTS admin UI, after adding/modifying shared folders, or after a firmware update.
Following are the settings I use on my non-QNAP Samba server (these should normally be put in the [global] section):
Code: Select all
mangled names = no
case sensitive = auto
preserve case = yes
fruit:encoding = native
Code: Select all
mangled names = yes
Code: Select all
mangled names = no
Good luck.
-
- Starting out
- Posts: 32
- Joined: Fri Aug 21, 2020 2:33 pm
Re: SMB file names
@Mousetick "fruit:encoding = native" did the trick! Thanks a lot! I am impressed by your such detailed description of my situation, it's almost an article, not a forum post. Also, you replied in the same detailed and useful manner on my another problem (about POSIX permissions). So I want to express my gratitude, if it is acceptable to you, I'd like to make a donation to you, please let me know can I do it, i.e. PayPal/Paysend/etc.—I don't see how to contact you via PM, so please left your contacts here or mail me at [removed]. Thanks again!!!
Last edited by m_emelchenkov on Wed May 19, 2021 2:54 pm, edited 1 time in total.
-
- Experience counts
- Posts: 1081
- Joined: Thu Aug 24, 2017 10:28 pm
Re: SMB file names
BTW you may want to remove your email address (by editing your previous post) rather than leave it for the world to see. You might get spam because of it, or worst case, someone else impersonating me may contact you asking for a donationm_emelchenkov wrote: ↑Mon May 17, 2021 4:57 pm I don't see how to contact you via PM, so please left your contacts here or mail me at xxxxxx.
-
- Starting out
- Posts: 32
- Joined: Fri Aug 21, 2020 2:33 pm
Re: SMB file names
Spam not a problem, but potential cheater is a security flaw, you are right . Removed.Mousetick wrote: ↑Tue May 18, 2021 4:20 pmBTW you may want to remove your email address (by editing your previous post) rather than leave it for the world to see. You might get spam because of it, or worst case, someone else impersonating me may contact you asking for a donationm_emelchenkov wrote: ↑Mon May 17, 2021 4:57 pm I don't see how to contact you via PM, so please left your contacts here or mail me at xxxxxx.
-
- First post
- Posts: 1
- Joined: Thu Jun 10, 2021 6:07 am
Re: SMB file names
@m_emelchenkov, Could you please provide an update on whether or not the change(s) you made to the config file persist after a restart or system update as suggested could be possible by @Mousetick ?
"may be lost after a restart, after modifying the service configuration through the QTS admin UI, after adding/modifying shared folders, or after a firmware update."
-
- Starting out
- Posts: 32
- Joined: Fri Aug 21, 2020 2:33 pm
Re: SMB file names
@mikebw, yes, config will be overwritten after service restart. To prevent this I wrote a custom autorun script, it patches necessary files (and do some other things not related to SMB but related to my other needs). If you are interested in my solution please mail m aaat emelchenkov dooot pro.
-
- Experience counts
- Posts: 1081
- Joined: Thu Aug 24, 2017 10:28 pm
Re: SMB file names
Similar request and solution in this thread:
need help making a change in smb.conf "stick" after reboot
The specific changes to smb.conf are different, but the general approach is the same (using autorun script).