AFP (netatalk) - CNID data still stored in MYSHARE/.AppleDB

Questions about using NAS on Mac OS.
Post Reply
xander-27
New here
Posts: 2
Joined: Mon Sep 15, 2014 4:56 am

AFP (netatalk) - CNID data still stored in MYSHARE/.AppleDB

Post by xander-27 » Mon Sep 15, 2014 6:22 am

Hello,

When a shared folder is accessed via AFP, a .AppleDB folder is created at the root of the share to store the CNID database. However, from versions 3.x of netatalk, and with actual configuration, I think it should not.

- Netatalk is in version 3.0.5 (QNAP firmware up-to-date 4.1.0) (current netatalk version : 3.1.6)
- CNID backend is DBD
- dbpath for all shared folders are : /var/netatalk/CNID//.@cnid/SHAREDFOLDERNAME (default location)

What do you think ? Is that a bug of version 3.0.5 ?

Informations that could be useful :
- From version 3.0.6 of netatalk, there is an option 'vol dbnest' to force the behaviour (default to false) I describe :
NEW (Changes in 3.0.6) : Option "vol dbnest", when set to true, the CNID database for a volume is stored in the volume root of a share in a directory .AppleDB like in Netatalk 2. Defaults to false. From FR#84.
- Here is an example of the same issue resolved by using the vol dbnest option (too bad we can't try it on 3.0.5...) : https://bugs.freenas.org/issues/4666

Thanks for your help !

Xander. France. Using TS-669L.

User avatar
schumaku
Guru
Posts: 43668
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: AFP (netatalk) - CNID data still stored in MYSHARE/.Appl

Post by schumaku » Mon Sep 15, 2014 10:57 pm

xander-27 wrote:- dbpath for all shared folders are : /var/netatalk/CNID//.@cnid/SHAREDFOLDERNAME (default location)


QTS 4.1.0 2014061x as well as QTS 4.1.1 20140828 does make use of the "classic" .AppleDB subfolder again:

[~] # ls -ls /share/MD0_DATA/Multimedia/.AppleDB/cnid2.db
388 -rwxrwx--- 1 admin administ 389120 Sep 12 14:42 /share/MD0_DATA/Multimedia/.AppleDB/cnid2.db*

However, during QTS 4.1.0 Beta, there were some builds which used the storage volume based .@cnid (and AFAIK this was also used with CDB instead of DBD):

[~] # ls -ls /share/MD0_DATA/.@cnid/Multimedia/.AppleDB/cnid2.db
88 -rw-r--r-- 1 admin administ 86016 Feb 6 2014 /share/MD0_DATA/.@cnid/Multimedia/.AppleDB/cnid2.db

No idea what "bug" you are behind here: DBD is default, as well as the storage location for the CNID DB (certainly on the QNAP afpd builds) - this config is not used in the QNAP dynamically created and maintained /etc/afpd.conf anyway. Correct my if I'm wrong, have not spent a lot of time to check.

I'm much more concerned about QNAP reversal to the distributed CNID DB aka DBD (with the CNID DB corruption issues resulting form unclean closed AFP client connections) from the NAS-based centralized one - leading to some more overhead on the initial connection - than on the storage location...

Disputable, what such system data does within the user storage space... Disputable, why we have this **** DBD back again, the first complaints on corruption are back in the community already.

User avatar
doktornotor
Ask me anything
Posts: 7521
Joined: Tue Apr 24, 2012 5:44 am

Re: AFP (netatalk) - CNID data still stored in MYSHARE/.Appl

Post by doktornotor » Mon Sep 15, 2014 11:04 pm

schumaku wrote:Disputable, why we have this **** DBD back again, the first complaints on corruption are back in the community already.


Indeed. Plus, if you recall the Photoshop and MS Office complaints, all those are known issues fixed upstream.

Outdated known buggy junk everywhere. Starting from base system and ending with this. Just about everything is more outdated than the practically unmaintained Optware stuff. Sigh.
Last edited by doktornotor on Mon Sep 15, 2014 11:11 pm, edited 1 time in total.
I'm gone from this forum till QNAP stop wasting volunteers' time. Get help from QNAP helpdesk instead.
Warning: offensive signature and materials damaging QNAP reputation follow:
QNAP's FW security issues
QNAP's hardware compatibility list madness
QNAP's new logo competition
Dear QNAP, kindly fire your clueless incompetent forum "admin" And while at it, don't forget the webmaster!

User avatar
schumaku
Guru
Posts: 43668
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: AFP (netatalk) - CNID data still stored in MYSHARE/.Appl

Post by schumaku » Mon Sep 15, 2014 11:10 pm

doktornotor wrote:Indeed. Plus, ...
Oh, there is more... When I connect a Mac to the network, I immediately see the Bonjour discovered Time Capsule, the N****** NAS, the S******* NAS, .... anything but the QNAP NAS. Minutes later, the QNAP NAS start to become listed. Nasty if you think about this might be a wake-up for Time Machine, or to quickly fetch a presentation before leaving the office. EDIT: Oh, this discovery issue was confirmed by QNAP QA in October 2013 already.

xander-27
New here
Posts: 2
Joined: Mon Sep 15, 2014 4:56 am

Re: AFP (netatalk) - CNID data still stored in MYSHARE/.Appl

Post by xander-27 » Tue Sep 16, 2014 5:16 am

schumaku wrote:QTS 4.1.0 2014061x as well as QTS 4.1.1 20140828 does make use of the "classic" .AppleDB subfolder again:

You're right, it does store the CNID DB in /share/SHARENAME/.AppleDB. What I don't understand is why.

schumaku wrote:No idea what "bug" you are behind here: DBD is default, as well as the storage location for the CNID DB (certainly on the QNAP afpd builds)

Well at first I wondered what was the CNID backend used. Since there was no cnid scheme option in /etc/afp.conf (except in [Qsync] section) and that it's the default backend since Netatalk 2.1 (netatalk documentation), I guessed DBD was used.

But I was wrong.
According to http://netatalk.sourceforge.net/3.0/htm ... D-backends, I should have a cnid_dbd daemon, which I have not.

Code: Select all

10740 admin      1564 S   /usr/local/sbin/cnid_metad -F /etc/afp.conf
10768 admin      2396 S   /usr/local/sbin/afpd -F /etc/afp.conf -p /var/afpd3.pid


The page http://netatalk.sourceforge.net/3.0/htm ... onf.5.html confused me, it says cnid_metad daemon instead of cnid_dbd daemon (see CNID backends section at the end of the page).

About the storage location, since there is no vol dbpath option in /etc/afp.conf, according to http://netatalk.sourceforge.net/3.0/htm ... onf.5.html (vol dbpath), and given, morever, the results of dbd -F /etc/afp.conf below, the CNID DB should be located under /var/netatalk/CNID/, but it is not.

Code: Select all

[/share/DATA] # dbd -fF /etc/afp.conf /share/CACHEDEV1_DATA/DATA/
Sep 15 20:11:26.557780 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Web' is '/var/netatalk/CNID//.@cnid/Web'
Sep 15 20:11:26.558195 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Public' is '/var/netatalk/CNID//.@cnid/Public'
Sep 15 20:11:26.558400 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'homes' is '/var/netatalk/CNID//.@cnid/homes'
Sep 15 20:11:26.558591 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Multimedia' is '/var/netatalk/CNID//.@cnid/Multimedia'
Sep 15 20:11:26.558817 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Download' is '/var/netatalk/CNID//.@cnid/Download'
Sep 15 20:11:26.559048 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Recordings' is '/var/netatalk/CNID//.@cnid/Recordings'
Sep 15 20:11:26.559277 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Qsync' is '/var/netatalk/CNID//.@cnid/Qsync'
Sep 15 20:11:26.559462 dbd[19019] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'DATA' is '/var/netatalk/CNID//.@cnid/DATA'
"/share/CACHEDEV1_DATA/DATA" isn't a "dbd" CNID volume


The dbd command results also confirm that the share isn't a DBD CNID volume. So what is the default CNID backend (@DEFAULT_CNID_SCHEME@) that was set at compilation and that is used on the QNAP ?

For information, here is the results of afpd -V :

Code: Select all

afpd 3.0.5 - Apple Filing Protocol (AFP) daemon of Netatalk

This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
Foundation; either version 2 of the License, or (at your option) any later
version. Please see the file COPYING for further information and details.

afpd has been compiled with support for these features:

          AFP versions: 2.2 3.0 3.1 3.2 3.3
         CNID backends: cdb dbd last tdb
      Zeroconf support: Avahi
  TCP wrappers support: No
         Quota support: Yes
   Admin group support: Yes
    Valid shell checks: Yes
      cracklib support: No
            EA support: ad | sys
           ACL support: Yes
          LDAP support: No
         D-Bus support: No
         DTrace probes: No

              afp.conf: /usr/local/etc/afp.conf
           extmap.conf: /usr/local/etc/extmap.conf
       state directory: /var/netatalk/
    afp_signature.conf: /var/netatalk/afp_signature.conf
      afp_voluuid.conf: /var/netatalk/afp_voluuid.conf
       UAM search path: /usr/local/etc/netatalk/uams//
  Server messages path: /var/netatalk/msg/


Now, what if I try (for test purpose) to force the use of DBD for a shared folder (DATA for example) ?

Add the option cnid scheme in the DATA volume section of /etc/afp.conf :

Code: Select all

[DATA]
cnid scheme = dbd


Comment the following line in the start() section of /etc/init.d/atalk.sh (mkafpconf was re-creating the file /etc/afp.conf every time netatalk was (re)started) :

Code: Select all

#/sbin/mkafpconf > /dev/null 2>&1


Create CNID directory in /var/netatalk/ (had to do it manually).

Restart netatalk :

Code: Select all

# /etc/init.d/atalk.sh restart
Shutting down AppleTalk service: OK
Starting AppleTalk service: OK


Well, the dbd command is not crying anymore :

Code: Select all

 dbd -fF /etc/afp.conf /share/CACHEDEV1_DATA/DATA/
Sep 15 20:24:50.561214 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Web' is '/var/netatalk/CNID//.@cnid/Web'
Sep 15 20:24:50.561650 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Public' is '/var/netatalk/CNID//.@cnid/Public'
Sep 15 20:24:50.561885 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'homes' is '/var/netatalk/CNID//.@cnid/homes'
Sep 15 20:24:50.562086 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Multimedia' is '/var/netatalk/CNID//.@cnid/Multimedia'
Sep 15 20:24:50.562298 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Download' is '/var/netatalk/CNID//.@cnid/Download'
Sep 15 20:24:50.562508 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Recordings' is '/var/netatalk/CNID//.@cnid/Recordings'
Sep 15 20:24:50.562706 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'Qsync' is '/var/netatalk/CNID//.@cnid/Qsync'
Sep 15 20:24:50.562909 dbd[23166] {netatalk_conf.c:718} (W:AFPDaemon): dbpath for 'DATA' is '/var/netatalk/CNID//.@cnid/DATA'


I now have a cnid_dbd daemon :

Code: Select all

21348 admin      2632 S   /usr/local/sbin/afpd -F /etc/afp.conf -p /var/afpd3.pid
22680 admin      1564 S   /usr/local/sbin/cnid_metad -F /etc/afp.conf
23167 admin     11020 S   /usr/local/sbin/cnid_dbd -F /etc/afp.conf -p /share/CACHEDEV1_DATA/DATA -t 6 -l 4


And the CNID DB is created under /var/netatalk/CNID/ instead of /share/SHARENAME/.AppleDB/ :

Code: Select all

# ll /share/DATA/
drwxrwxrwx    3 admin    administ      4.0k Sep 15 20:25 ./
drwxrwxrwx   26 admin    administ      4.0k Sep 15 19:31 ../
#
# ll /var/netatalk/CNID/.\@cnid/DATA/
drwxrwxrwx    3 admin    administ      1.0k Sep 15 20:24 ./
drwxrwxrwx   10 admin    administ      1.0k Sep 15 20:23 ../
drwxr-xr-x    2 admin    administ      1.0k Sep 15 20:30 .AppleDB/


I could use DBD, but I'm not comfortable about the CNID backend differences, and I have read you guys, so I prefer not.

So the questions are : What is the CNID scheme that is used ? And why the dbpath is ignored with it ?

schumaku wrote:I'm much more concerned about QNAP reversal to the distributed CNID DB aka DBD (with the CNID DB corruption issues resulting form unclean closed AFP client connections) from the NAS-based centralized one - leading to some more overhead on the initial connection - ..........

What is that "NAS-based centralized one" ? Is that the CNID scheme actually used (4.1.0) ? Does QNAP want to use DBD again in future firmware version (4.1.1 ?) ?

schumaku wrote:......... than on the storage location...

I understand, so I appreciate we can discuss about my storage location issue :D

Xander.

User avatar
schumaku
Guru
Posts: 43668
Joined: Mon Jan 21, 2008 4:41 pm
Location: Kloten (Zurich), Switzerland -- Skype: schumaku
Contact:

Re: AFP (netatalk) - CNID data still stored in MYSHARE/.Appl

Post by schumaku » Tue Sep 16, 2014 5:36 am

Useless proof using standard Netatalk defaults, waste of time and bandwidth - QNAP does use different defaults as mentioned :idea: :idea: :idea: :

No idea what "bug" you are behind here: DBD is default, as well as the storage location for the CNID DB (certainly on the QNAP afpd builds)


xander-27 wrote:What is that "NAS-based centralized one"
[edit] I must have a bad minute when dfining CDB andDBD last time here...to avoid the confusion (sorry please) - here are the correct definitions borrowed from http://netatalk.sourceforge.net/3.1/htm ... D-backends:

cdb

The "concurrent database" backend is based on Berkeley DB. With this backend, several afpd daemons access the CNID database directly. Berkeley DB locking is used to synchronize access, if more than one afpd process is active for a volume. The drawback is, that the crash of a single afpd process might corrupt the database. cdb should only be used when sharing home directories for a larger number of users and it has been determined that a large number of cnid_dbd processes is problematic.

dbd

Access to the CNID database is restricted to the cnid_dbd daemon process. afpd processes communicate with the daemon for database reads and updates. The probability for database corruption is practically zero.

This is the default backend since Netatalk 2.1.


No, 4.1.1 (which is out as 20140822 for some NAS models) does again the same.

If you seek a fully configureable and customizable system, afraid you are wrong with the majority of NAS vendors using embedded Linux systems. Here (as with many other services) - there is virtually nothing freely configureable (not SAMBA; no ProFTPD, not Netatalk, ...). And gain, the QNAP has built afpd do override some common defaults again, too. So don't waste your time...

Post Reply

Return to “Mac OS”