Differences

This shows you the differences between the selected revision and the current version of the page.

--- technology:unix:lms:linux_log 02.25.2012 21:06
+++ technology:unix:lms:linux_log 03.18.2013 04:30 current
@@ Line 46: / Line 46: @@
 ====== AFP and Avahi for Apple filesharing ======
-AFP = Apple Filing Protocol
+Instructions modified from:
+  * http://www.kremalicious.com/2008/06/ubuntu-as-mac-file-server-and-time-machine-volume/
+  * http://plugcomputer.org/plugwiki/index.php/AFP
 ===== Install Netatalk =====
-  * updated System > Admin > Soft Sources to get source code from 'Main Server'
+  * Netatalk install <code>sudo apt-get install netatalk</code>
-  * Build Dependancies?? <code>sudo apt-get build-dep netatalk</code>
+    * I used to have to install this manually because it did not formerly come with the correct crypto options. Now, this is included in the latest netatalk install. Info from [[http://www.kremalicious.com/2008/06/ubuntu-as-mac-file-server-and-time-machine-volume/#comment-86412|here]].
-  * Install needed packages <code>sudo apt-get install cracklib2-dev fakeroot libssl-dev</code>
+    * Setting up netatalk (2.2~beta4-1) ...
-  * Netatalk install <code>sudo apt-get source netatalk</code>
-    * many errors / warnings like dpkg-shlibdeps:warning:../netatalk/...
-    * .../... shouldnt be linked with libcrack.so.Z (it uses none of it symbols)
-    * all else looks OK
-    * built ~/netatalk_2.0.3x-9_amd64.deb
-  * Show Info for package (see if installed) <code>sudo dpkg -i ~/netatalk_2.0.3-9_amd64.deb</code>
-  * Do not update netatalk?? <code>echo "netatalk hold" | sudo dpkg --set-selections</code>
-    * This tells software update to not touch it
 ===== Config Netatalk (AFP) =====
-  * edited /etc/default/netatalk to
+  * Which daemons to run?
+    * edited /etc/default/netatalk
     * <code>
-        ATALKD_RUN = NO
+        ATALKD_RUN=no
-        PAPD_RUN = NO
+        PAPD_RUN=no
-        CNID_METAD_RUN = YES
+        CNID_METAD_RUN=yes
-        AFPD_RUN = YES
+        AFPD_RUN=yes
-      </code>
+        TIMELORD_RUN=no
-  * edited /etc/netatalk/afpd.conf
+        A2BOOT_RUN=no</code>
+  * Set various settings for afp daemon
+    * edited /etc/netatalk/afpd.conf
     * <code>
-        - -transall -vamlist vams_randnum.so, vams_dhx.so -nosavepassword -advertise_ssh
+        - -transall -uamlist uams_randnum.so,uams_dhx.so,uams_dhx2.so -savepassword -advertise_ssh</code>
-      </code>
+    * - says that this line applies to the default afpd server; you could, optionally, have more than one, with names in place of this dash
+    * -transall uses all protocols, tcp, udp, ddp (could also optionally choose any one or mix of these)
+    * -uamlist sets user auth modules to use; [[http://www.trollop.org/2011/03/06/netatalk-afp-mac-os-x-10-7-lion/|uams_dhx2.so]] is needed for OSX Lion
+    * -savepassword allows clients to save the pass locally; can also be -nosavepassword
+    * -advertise_ssh allows afp encrypted and sent over a ssh session; assumes sshd is setup and running correctly; not advised by netatalk [[http://netatalk.sourceforge.net/2.0/htmldocs/afpd.conf.5.html|docs]]
 ===== Config Shared Volumes =====
-  * edited /etc/netatalk/AppleVolumes.default
+  * Set what to share and to whom
+    * edited /etc/netatalk/AppleVolumes.default
     * <code>
-        /media/documents LinuxMediaServer allow:nyeates1 cnidscheme:cdb options:usedots
+        /media/documents LinuxMediaServer allow:nyeates,shyeates,shyeates513 cnidscheme:cdb options:usedots,upriv
       </code>
+      * allow: usernames that can login to the afp server; needs to be defined users I think
+      * cnidscheme: use dbd; cdb doesnt work anymore on netatalk 2.1+
+      * options: usedots: allow proper .blah files; upriv: afp3 unix privs - do not use if you have OS X tiger
   * Restarted Apple Files Server
-    * <code> sudo /etc/init.d/netatalk restart</code>
+    * <code>sudo /etc/init.d/netatalk restart</code>
+  * Test connectivity
+    * Go over to mac computer on same network
+    * In finder, hit Ctrl+K
+    * Type in "afp://lms" and hit connect
+      * lms is the dns name of my server; could also use IP address
+    * Type in auth info - user and pass
+    * It should load and show mount options (if more than one mount was defined in AppleVolumes.default)
+    * Files and directories should load and show
 ===== Install Avahi (Bonjour) =====
-  * Avahi install <code>sudo apt-get install avahi-bonjour</code>
+  * Avahi install <code>sudo apt-get install avahi-daemon</code>
-  * Other needed packages?? <code>sudo apt-get install libnss-mdns</code>
+  * Other needed package?? <code>sudo apt-get install libnss-mdns</code>
     * both already installed !!
   * edited /etc/nsswitch.conf
     * added "mdns" to "hosts:" line
+    * <code>hosts:          files mdns4_minimal [NOTFOUND=return] dns mdns4 mdns</code>
 ===== Config Avahi and advertise services =====
   * edited /etc/avahi/services/afpd.service
-    * inserted XML: "port 548" (&amp; 5353)
+    * inserted XML: "port 548"
+    * &lt;code>
+        <?xml version="1.0" standalone='no'?><!--*-nxml-*-->
+        <!DOCTYPE service-group SYSTEM "avahi-service.dtd">
+        <service-group>
+        <name replace-wildcards="yes">%h</name>
+        <service>
+        <type>_afpovertcp._tcp</type>
+        <port>548</port>
+        </service>
+        <service>
+        <type>_device-info._tcp</type>
+        <port>0</port>
+        <txt-record>model=Xserve</txt-record>
+        </service>
+        </service-group></code>;
   * Restarted Avahi Daemon
     * <code>sudo /etc/init.d/avahi-daemon restart</code>
+    * <code>sudo restart avahi-daemon</code>
 ===== Test on Macs =====
@@ Line 1418: / Line 1450: @@
 Went into mounted dir and its all there! Yay!
+====== Got AFP and Bonjour equivalents running ======
+See [[#afp_and_avahi_for_apple_filesharing|above section]] that I updated on this
+====== Dubugging dir list slowness ======
+In using the AFP shares on mac mini for movies with plex media server, I noticed that the directory listing was molasses slow.
+Also, the entire 'movies' dir wouldnt load at all after some time.
+I started digging into log files and people with similar problems.
+Error log for Netatalk, the AFP daemon:
+  tail -f /media/documents/.AppleDB/db_errlog
+This file was giving same error over and over:
+  cnid2.db/cnid2.db: DB_SECONDARY_BAD: Secondary index inconsistent with primary
+  Finding last valid log LSN: file: 2 offset 7783453
+  Recovery starting from [2][7782965]
+  Recovery complete at Tue Mar  6 17:17:04 2012
+  Maximum transaction ID 80000004 Recovery checkpoint [2][7783453]
+Users [[http://forums.freebsd.org/showthread.php?t=20324|here]] had similar issues and they basically were deleting their entire CNID databases (the database that keeps track of IDs -> file names. They were then restarting. Another set of information [[http://netatalk.sourceforge.net/2.1/htmldocs/configuration.html#CNID-backends|under the official Netatalk docs]] explained about this CNID db and how it operates, what its role is, and about a new command:
+  dbd -s .
+The -s scans. -r rebuilds what the scan finds.
+I decided I would move the /media/documents/.AppleDB dir (same as deleting it), and rebuild.
+I shutdown netatalk
+moved .AppleDB
+ran a cnid rebuild
+  dbd -r /media/documents/
+The above command started checking and writing new CNIDs for every single file. Tons of lines flew by, one for each file it seemed.
+mins of CNID rewrites, and it stops. I now start up netatalk. Logs and db seem to clear out from new.
+Initial directory loads take a bit of time still, but once they are loaded, they are cached and fast.
+====== Move data to Mac Mini Server ======
+I want to get all of the data on this linux server over to a mac server, held on our current mac mini that will come from the living room. Now that we have 2 apple TV devices, we dont need the mac mini as a TV device in living room, freeing it for server use. My hope is that 1) I dont have to mess with such low level stuff and piecing together 10 open source components when setting up hard drives and file shares and bonjour, etc 2) So that Sue can more easily modify or restart or understand the data on this server.
+===== Plan =====
+Purchased a 3 TB HDD, and a USB enclosure that can take 4 drives.
+  - Plug this enclosure in, with the 3TB drive in it, into the Linux Media Server.
+  - Format and mount the 3 TB drive on LMS, now we have the 2 TB raid and 3 TB hdd
+  - Copy the files over to 3 TB hdd
+    - check that copy is correct
+  - Install mac mini and get server edition running on it (cheap app store purchase)
+  - Bring 3TB hdd over to mac mini (via usb enclosure), mount it,
+    - check that files show
+    - check that streaming is fast enough
+  - Start Backblaze backup immediately
+  - Unplug 3 x 1TB drives from LMS, slide them into USB enclosure on mac mini
+  - Format the 3 x 1TB drives into a mac raid-jbod (just a bunch of disks)
+    - there is no fault tolerance here, but i will setup a backup solution via backblaze or the likes
+  - Copy the files from 3 TB hdd to 3 x 1 TB raid-jbod
+  - Format the 3 TB hdd on mac mini
+  - Add the 3 TB hdd into the raid-jbod array, to create a 6TB JBOD array
+  - Install menu-bar utility that allowed SMART monitoring status and emails
+===== Format and Mount 3 TB as UDF =====
+Read an article that I should try UDF file system, so that when I take it to the mac, it can still be read natively. Ubuntu/linux also can format it. Little did I know, there was a bit of researched needed to format and mount it on linux.
+Turns out UDF does not use normal MBR (master boot record) at the begining of the disk. It does its own thing, and you just gobble up the entire disk. So I messed with partitions at first, without knowing I didnt need to. Instead, it is good to first clear the MBR at the start of the disk, and then format with udftools on ubuntu. See commands below.
+sudo dd if=/dev/zero of=/dev/sde bs=512 count=1
+sudo mkudffs --media-type=hd --blocksize=512 /dev/sde
+Got the info from here:
+  - http://superuser.com/questions/39942/using-udf-on-a-usb-flash-drive/
+  - http://superuser.com/a/567559/110601
+  - http://superuser.com/questions/526174/what-unix-like-filesystem-can-easily-and-safely-be-mounted-on-both-the-mac-and-l/
+When I look at the drive capacity though, with `df -h` it says the drives size is 747G. Not 3 TB :-(
+I could still test the 3TB UDF drive on the mac mini to see if it mounts and has files on it.