Well… not much of a guide really.  Just a very quick how-to on what I did to get Time Machine on my Macbook Pro with OSX Lion 10.7 to save the backup data to my Linux server network share using AFP, rather than SMB (Windows Share), a Time Capsule, or and Apple Server.  I will attach my working config files for your reference as well.  Note that, 1) I am using the source code for Netatalk, compiled and installed, not the Ubuntu or Debian package, and 2) I am assuming you are comfortable with installing software from source.  Last note, I owe a HUGE THANK YOU to post I found on the basic’s of this configuration. (located here: http://www.trollop.org/2011/07/23/os-x-10-7-lion-time-machine-netatalk-2-2/)  I had a hard time getting anything working otherwise, and that post mostly got things working for me.

I really love this setup, because:

A) I am running a tiny ITX case with a 1.5TB ZFS mirror in it for data integrity. (back when drives were cheap, this makes a powerful and inexpensive NAS)  (oh and I am running this using the zfs-native package on Ubuntu Lucid! Love it!)

B) I love having the network-based Time Machine running, because it just runs whenever I am on my home network. I don’t have to think about, “Well, should I plug in my USB drive and backup tonight?”  It just runs all the time.

C) I also have Crashplan running on that ZFS pool, to save all my local machines here as well.

None of that has anything to do with getting an AFP share running, so, moving on.  The instructions below don’t cover the creation of your sparce file for the AFP share, which Time Machine uses.  Once you get to the Sixth step below, you’ll need to create that sparce bundle and copy it to your share. I used the script located on this post to generate my sparce bundle and copy it over to my server share. Anyway, to get your Mac talking to your Linux box, setup Netatalk like below. :)

---

FIRST:

Go get the latest Netatalk source code from Sourceforge.  http://netatalk.sourceforge.com/

Download and un-tar that code into a directory of your choice. But before we configure and compile this…  we need some supporting software.

---

SECOND:

Get the supporting software packages for Avahi and other items.  We need Avahi for zeroconf and other libs for authentication purposes.

I ran this apt-get command:

apt-get install avahi-daemon mdns-scan \
 avahi-utils libavahi-common-dev \
 libavahi-compat-libdnssd1 libavahi-compat-libdnssd-dev \
 libssl-dev libacl1-dev libwrap0-dev libgcrypt11-dev libdb4.8 libdb4.8-dev

Install the above items, then we can move on to the compile and install of afpd/Netatalk.

---

THIRD:

Now, “cd” to the directory where you un-tar’d the netatalk download code.  For example, mine was in this folder:

cd /mpool/Shares/Shared/netatalk-2.2.1

Assuming all the above packages installed successfully, let’s run ./compile. Here’s my command line:

./configure --enable-debian --with-acls --enable-ddp  --enable-zeroconf

Then, if that ran successfully, you should end up with some summary information as follows:  (may not be exact)

Using libraries:
    LIBS = -lpthread  -L$(top_srcdir)/libatalk
    CFLAGS = -I$(top_srcdir)/include -D_U_="__attribute__((unused))" -g -O2 -I$(top_srcdir)/sys
    SSL:
        LIBS   =  -L/usr/lib64 -lcrypto
        CFLAGS =  -I/usr/include/openssl
    LIBGCRYPT:
        LIBS   = -lgcrypt
        CFLAGS =
    BDB:
        LIBS   =  -L/usr/lib64 -ldb-4.8
        CFLAGS =
Configure summary:
    Install style:
         debian
    AFP:
         Large file support (>2GB) for AFP3: yes
         Extended Attributes: ad | sys
    CNID:
         backends:  dbd last tdb
    UAMS:
         DHX     ( SHADOW)
         DHX2    ( SHADOW)
         RANDNUM ( SHADOW)
         passwd  ( SHADOW)
         guest
    Options:
         DDP (AppleTalk) support: yes
         CUPS support:            no
         Apple 2 boot support:    no
         SLP support:             no
         Zeroconf support:        yes
         tcp wrapper support:     yes
         quota support:           yes
         admin group support:     yes
         valid shell check:       yes
         cracklib support:        no
         dropbox kludge:          no
         force volume uid/gid:    no
         ACL support:             yes
         LDAP support:            no

Notice the section on “UAMS” containing “DHX” and “DHX2″?  You must have those listed, or your Mac OSx Lion system wont work on it.  It needs the encrypted authentication.  If you don’t see those listed, you are missing some libraries on your system.

 

---

FOURTH:

Now if all is well…  run make.

make

make install

Run the above command, while still in that source directory.

Netatalk will install binaries for you under: /usr/local/sbin  and config under: /usr/local/etc/netatalk.
It also installs a basic startup script under your /etc/init.d directory called, easy enough, “netatalk.”

 

---

FIFTH:

Setup your config.  I’ll attach my configs for your reference, but here’s a couple basics.

1. afpd.conf  - comment out anything else, and add this line:

- -udp -noddp -uamlist uams_randnum.so,uams_dhx.so,uams_dhx2.so -nosavepassword

2. AppleVolumes.default – also comment out others, and add these.
These are just an example, based on my config. You’ll need to adjust the names and paths.
(NOTICE the “tm” at the end of the time machine share below, you must have that for TM to work.)  

/dapool/bu/tm TimeMachine allow:greg cnidscheme:dbd options:usedots,upriv,tm
/mpool/Shares/AppleShare AppleShare allow:greg cnidscheme:dbd options:usedots,upriv

3. netatalk.conf – I didn’t have this file on one system, so I just created it and added the config below:

#### machine's AFPserver/AppleTalk name.
ATALK_NAME=`echo ${HOSTNAME}|cut -d. -f1`

#### server (unix) and legacy client (<= Mac OS 9) charsets
ATALK_UNIX_CHARSET='LOCALE'
ATALK_MAC_CHARSET='MAC_ROMAN'

#### Don't Edit. export the charsets, read form ENV by apps
export ATALK_UNIX_CHARSET
export ATALK_MAC_CHARSET

#########################################################################
# AFP specific configuration
#########################################################################

#### Set which daemons to run.
#### If you use AFP file server, run both cnid_metad and afpd.
CNID_METAD_RUN=yes
AFPD_RUN=yes

#### maximum number of clients that can connect:
AFPD_MAX_CLIENTS=20

#### UAMs (User Authentication Modules)
#### available options: uams_dhx.so, uams_dhx2.so, uams_guest.so,
####                    uams_clrtxt.so(legacy), uams_randnum.so(legacy)
AFPD_UAMLIST="-U uams_dhx.so,uams_dhx2.so"

#### Set the id of the guest user when using uams_guest.so
AFPD_GUEST=nobody

#### config for cnid_metad. Default log config:
CNID_CONFIG="-l log_note"

#########################################################################
# AppleTalk specific configuration (legacy)
#########################################################################

#### Set which legacy daemons to run.
#### If you need AppleTalk, run atalkd.
#### papd, timelord and a2boot are dependent upon atalkd.
ATALKD_RUN=no
PAPD_RUN=no
TIMELORD_RUN=no
A2BOOT_RUN=no

#### Control whether the daemons are started in the background.
#### If it is dissatisfied that legacy atalkd starts slowly, set "yes".
ATALK_BGROUND=no

#### Set the AppleTalk Zone name.
#### NOTE: if your zone has spaces in it, you're better off specifying
####       it in afpd.conf
ATALK_ZONE=@AFP

---

SIXTH:

Start ‘em up!

/etc/init.d/netatalk start

That should start up the services. I did a quick check using “netstat -tapn” and if you see these below, you should be running:

tcp        0      0 0.0.0.0:548             0.0.0.0:*               LISTEN      22146/afpd
tcp6       0      0 ::1:4700                :::*                    LISTEN      22144/cnid_metad

Again, may look a bit different on your system, but you should at minimum see the afpd process on port 548 and soemthing listed for “cnid_metad” on its default port of 4700.  I don’t know why mine says tcp6, but it works, so I am not complaining.

I should note, I set this up on two systems.  One worked right away, the other wouldn’t start cnid_metad.  Your startup should respond with:

..Starting Netatalk services (this will take a while):  cnid_metad afpd.

See that “cnid_metad” listed in the services starting?  My problem system wouldn’t show this, it only showed afpd.  As a result, Time Machine and Finder on my Mac would give me an error about CNID DB not working and “using temporary” or something like that.  Time Machine failed everytime.

What I did to fix it, was a cheesy hack.  I just found the lines below in my /etc/init.d/netatalk script:

# prepare startup of file services
if [ "x$CNID_METAD_RUN" = "xyes" -a -x /usr/local/sbin/cnid_metad ] ; then
   echo -n " cnid_metad"
   /usr/local/sbin/cnid_metad $CNID_CONFIG
fi

I changed the above to look like this:

# prepare startup of file services
if [ -x /usr/local/sbin/cnid_metad ] ; then
   echo -n " cnid_metad"
   /usr/local/sbin/cnid_metad $CNID_CONFIG
fi

I don’t care if CNID_METAD is “yes” or not, I just want it to run!  Everytime!  So I just check if that binary is executable and launch it.  This worked!!  And Time Machine connected to my AFP share (which was this: afp://draygon/TimeMachine ) and started backing up!

I won’t go into the Time Machine config on your Mac here. All you really need to do is in the “Select Disk” button, it will list your available disks, just select the one on your server.  I did make sure I could connect to it first.  In Finder, go to “Go” on the menu and select “Connect to Server”.  Type in the server path or browse to it.  ( afp://YOUR_SERVER_NAME/YOUR_SHARE_NAME  )  If that opens, and you can create files in there in Finder, you should be good to go!

It should be obvious, as well, that on your Linux system you need a user account created with a matching password (which Finder on your Mac may ask for).  You should also have file system permissions configured on your Linux system accordingly.  If not, you will have trouble connection and/or creating files.  Therefore, Time Machine won’t work either.  So make sure the “sharing” stuff works.

My Config Files:

Located in /usr/local/etc/netatalk, the links below will open up at pastebin.com.

afpd.conf  - http://pastebin.com/3kvHLLph  

AppleVolumes.default - http://pastebin.com/Z0L7pzGi

netatalk.conf - http://pastebin.com/agVs4RW0

 

Extra Note:

I wanted to mention this link:  http://lifehacker.com/5691649/an-easier-way-to-set-up-time-machine-to-back-up-to-a-networked-windows-computer 

In order to get Time Machine to use that AFP share, you need to create a Sparse bundle file which saves your data.  The link above has some great instructions on that.  And they link you to this script for your Mac to create the file and copy it, makes it easy!

 EDIT 12/02/11:

On OSX Lion, you apparently DO NOT need to manually create that sparse file.  I had to remove my backup data and recreate it, and when I did, I did not create the sparse file.  I simply opened Time Machine, clicked Select Disk, and there was my network share. I selected that network share and it created a new sparse file and started to backup automatically. Nice!!

Maybe that’s a Lion thing, I don’t know. BUT YOU DO NEED to tweak your Mac to allow backup to an unsupported network share.  The articles listed above mention using the “secrets pref pane” in OSX and a checkbox to allow the unsupported backup location. PrefPane can be downloaded from this site:
http://secrets.blacktree.com/ 

(you can really BREAK your Mac with that program with some of those settings, so use at your own risk!  I only used it to set my Time Machine to use Linux.)

Its a quick-n-easy install.  Open that program once installed, go to the Time Machine section and check the box to allow unsupported backup locations.  There is a manual, command line way to do this as  well.

 

 

 

 

That’s right, I don’t like SpiderOak enough to switch from Dropbox.  Why? No LAN sync!   SpiderOak is also a backup service. I use CrashPlan along with Dropbox on my computers (Mac, Linux, Windows, even a FreeBSD server) and SpiderOak is not quite good enough to replace CrashPlan either.

Let me just say first that I do like SpiderOak and it works well.  It’s nice looking, easy to use, and does what its supposed to do.  It runs on Linux! It does it securely too! And I really like that! I can’t really complain about its functionality.  So my opinion here is based primarily on how its feature set can fit into my computer use and needs.

Here’s what I don’t like about SpiderOak.

1. It doesn’t have LAN sync. Therefore, I can’t replace DropBox.

2. It doesn’t have a background service, so you have to be logged in for it to backup. (and therefore, I can’t replace CrashPlan)

3. It doesn’t have multiple destinations, and specifically, destinations to my other computers or friends computers as backup.  This is a major reason why I use CrashPlan too. Not having multiple destinations and on local systems is big negative for me. (more on that in a minute, because it DOES have a local backup option)

4. Sync folders must be inside a backup folder. What?  That’s not going to work for me.  You CAN backup multiple and different folders, unlike Dropbox, but those must be inside of a “Backup” folder. You can’t just select one to sync.  That’s partially useful feature, but I don’t want that restriction. And considering that it DOES NOT have LAN sync (#1 above), it really makes it less useful to me. Although its true, I typically want to backup what I an syncing.

If security were a top priority, I’d definitely NOT use Dropbox and use SpiderOak, but that lack of LAN sync makes it tough to want to use it.  Files that start to sync have to be backed up, then redownloaded on another computer.  I watched progress on this and it was moving and the speed of my internet connection, which is 1.5Mbps.  When you want your files to sync up quick, like Dropbox does on my 1Gbps network, you really begin to loose interest in SpiderOak. For me, personally, I will not be using SpiderOak until they at minimum have LAN sync.

Now, if they were to also add multiple destinations, get me the ability to backup to/from Linux, Mac, Windows, and FreeBSD, setup a system service/daemon, and have backup sets on top of LAN sync, I’d seriously consider ditching both Dropbox and Crashplan.  (Ditching Crashplan will be tough though, there’s a lot of nice features and its been extremely reliable for me.)

There’s another option on the Dropbox side of things I am playing with, Windows Live Mesh.  As a replacement to Dropbox, it is working pretty well.  Not perfect, not as fast, and NOT on Linux, but it is on Mac.  It does allow me to sync any folder, over the LAN or Internet, to multiple systems, and its fairly quick once it syncs up. But its kind of a resource hog, and sometimes it seems to hang on some files.  Not a good replacement for Dropbox, except if you are concerned about security, because you can set syncing between your computers only if you want.

I did want to mention that SpiderOak does have a “local copy” type of feature. So your backup folders can be set to keep a local copy on another drive or on the network for fast recovery. That’s a nice feature. But it had one huge flaw, in my opinion, it only backed up to the local storage from that point in time.  What that means is, anything you had backed up before that, wasn’t in your local storage.  This is a huge drawback for me, and compared to how Crashplan works with local backup storage (destinations), I just don’t like it.

SpiderOak is an “O K” backup program that can also sync some folders. Its secure, if you don’t trust Dropbox with your data. For that reason alone, you could use SpiderOak as a Dropbox alternative.  I just don’t keep anything in my Dropbox that would require that security, or I use a Truecrypt drive on it.

I service small businesses in my area with anything from virus cleanup to network setup to backup systems. I cannot recommend SpiderOak to them.  But, SpiderOak people, if you are listening, if you can make your program LAN sync and give me a good multi-destination-running as a service-file-delta-level-backup like Crashplan does, I’ll seriously consider using your product and recommend it to my clients.  On the other hand, if you just add LAN sync, I’ll drop Dropbox and use your product. :)

 

 

A how-to for installing Crashplan backup software on FreeBSD, my favorite server software with my favorite backup software.

We owe Kim Scarborough and Aaron Baff for this information, BIG TIME! Thank you, Thank you, Thank you, Kim and Arron for your help in making this possible!

Kims original HOW-TO is here:
http://kim.scarborough.chicago.il.us/do/nerd/tips/crashplan

I have no contact or reference info for Aaron, exept, here’s the original CrashPlan forum thread with our conversations:
https://crashplan.zendesk.com/entries/112498

All credit for this really belongs to Arron and Kim! I only wanted to record my own notes and share my experience here.

Greg’s How-to: How to install Crashplan on FreeBSD!

Note, this install is on a FreeBSD 8.1-RELEASE system. Kim installed on FreeBSD 7. Also, this doesn’t seem to matter for installation purposes, but my system also runs with ZFS on a GPT root mirror, with other ZFS pools on the system as well using RAIDz2. (where I will store my Crashplan data)

Below I mention “install some/port”, which is not intended to be a literal command. I’ll leave the details of installing ports to you, which you can find online on other sites.

1. install emulators/linux_base-f10
Note: I had to find glibc-common online and manually load into distfiles when I first tried to install on 8.1.
Update 3/30/11: As of 3/30, and running on FreeBSD 8.2, I did not need to find glibc manually, it just loaded first try. Maybe the first time I tried the mirrors were unavailable?

1.b Check /etc/rc.conf
After you install the Linux base packages, check /etc/rc.conf that linux is enabled and add it if not.
linux_enable="YES"

2. Add to fstab and mount:
linproc /compat/linux/proc linprocfs rw 0 0

Then…
mount linproc

Update 3/30/11 FreeBSD 8.2: I wasn’t able to mount linproc at this point. I rebooted the system, then it mounted. So, reboot first, then add this to your fstab and make sure it mounts before you reboot.

3. install sysutils/linux-procps

Update 3/30/11 FreeBSD 8.2: In the next steps 3, 4, 5, 6, 7, I needed to add the following to my /etc/make.conf file. Note that the “f10″ is NOT in quotes. Once this was added, these extra ports installed fine.

OVERRIDE_LINUX_BASE_PORT=f10
OVERRIDE_LINUX_NONBASE_PORTS=f10
LINUX_DIST_SUFFIX=f10


4. install /usr/ports/textproc/linux-f10-expat

5. install /usr/ports/x11-fonts/linux-f10-fontconfig

6. install /usr/ports/x11/linux-f10-xorg-libs

7. install /usr/ports/java/linux-sun-jre16
note: you’ll need to manually find and download the jre-6u3 file from sun/oracle.

You should have these installed. (or similar, maybe other versions)

linux-f10-expat-2.0.1 = up-to-date with port
linux-f10-fontconfig-2.6.0 = up-to-date with port
linux-f10-xorg-libs-7.4_1 = up-to-date with port
linux-procps-3.2.5 = up-to-date with port
linux-sun-jre-1.6.0.03 = up-to-date with port
linux_base-f10-10_3 = up-to-date with port

8. change to linux bash, as root:
/compat/linux/bin/bash

Note that this puts you into the linux compat environment. If you go looking for /usr/local/bin/crashplan, it won’t be there unless you are in the linux compat environment first. The actual location is here:
/compat/linux/usr/local/crashplan

9. Install Crashplan.
go to the Crashplan-install folder (where you extracted it) and start the ./install.sh script. Use all defaults and finish the script.

10. Edit run.conf. In the /usr/local/crashplan/bin folder,
edit the run.conf file. There are two long lines defining
Java statup parameters. Add the following to the SVR line.

-Djava.nio.channels.spi.SelectorProvider=sun.nio.ch.PollSelectorProvider

That should go within the quotes!

11. Try starting the service:
/usr/local/crashplan/bin/CrashPlanEngine start

You’ll need to start this manually (from within linux bash), and will not start at boot.
(until one of us writes an rc script)
Update 3/30/11: Here’s an rc.d startup script how to for Crashplan:

http://www.1stbyte.com/2011/03/30/an-rc-d-startup-script-for-crashplan-running-on-freebsd/

12. Use the “headless” install connection per the Crashplan FAQ’s online to connect from another Linux/Windows/Mac system to manage the service.

Good luck!!

I am not expert on these things (encryption), but I have done some reading and found some issues with Encrypted File System I don’t like. I may not describe the issues correctly, so this is just my opinion more than anything.

1. In Windows 2000, don’t even bother. It can be bypassed with their recovery agent or administrator. So if you lose your laptop, the data can be accessed.

2. In XP, it is better and more secure. I think there is no data recovery agent, but I think a local administrator account on a non-domain install of XP will still have the private keys.

3. The private keys are on that hard drive!

4. You still see all the files. The file names are all viewable, and that may be a security risk for some companies. It’s better than nothing, but I don’t like that too much.

5. You can’t encrypt the whole system. Or a whole partition for that matter. You must encrypt a folder, and at that, only the files in that folder are encrypted.

6. Here’s the one I like least… with EFS, when you open a file, it is decrypted to a tmp file. This file is deleted once you finish with it, but as you know, files are not “wiped” from the drive when they delete, they just remove the pointer to it. So unless data is overwritten in that place of the drive, that data is accessible to anyone. If you had a spreadsheet with SSN’s or credit card numbers, and you just happen to lose your system to someone who knows what to do with it, you got a big problem!!

7. There’s more, I just can’t think of them.

Anyway, after doing some reading… I found that Bitlocker in Vista will be a very nice solution. But you have to buy Enterprise or Ultimate versions of Vista to get it. Bitlocker can encrypt the entire OS partition. Now that is nice! That is exaclty what we wanted! And if you set it up correctly, using a key or PIN at boot, it will make an extremely secure setup. One drawback, you can only encrypt the partition the OS is on, not other partitions. You’ll need to use normal EFS for them.

That’s nice, but I have Vista Business. And I don’t want to spend more money right now. Plus, on my main system and pretty much all my clients, they have 2000 and XP. Guess what I found to get me by? TrueCrypt. www.truecrypt.org. Nice product!! And it’s open-source and free!!!!

With TrueCrypt, you can password protect an entire partition with AES 256-bit encryption. You can use multiple ciphers and even key based access using a USB drive. (Bitlocker can do the USB drive thing too!) It’s a tiny program running in the systray. And in my case, I am just running a password authentication and 256bit AES on a separate partition, so my performance is pretty good too, though not as fast without encryption. Now, with XP I will be making redirections to My Documents to that private drive, and saving all my “work-in-progress” there. That, to me, operates reasonably, and pretty darn secure. I could do more to secure it, like use a key file on my usb key drive. Then you cannot get into any of my private data without the key drive inserted! But I need to test that first.

TrueCrypt can also create a virtual drive from a file.� That might be handy, but performance is just a little slower that way.� It cannot encrypt your OS partition though, which is a drag, but at least I can encrypt a separate partition and you cannot see the file system structure.� It has a lot of neat features. Definitely worth trying if you want lock down things.

Uber quick howto:  (based on Debian)
1. Make sure ssh, rsync and sudo are installed and working.
2. Add a user account,  on remote system.
2.a Add a certificate with openssl or ssh-keygen (look that up elsewhere)
2.b Make sure the cert is unencrypted with no password. Yes,  that is a slight security concern, HOWEVER, if you are very careful to secure that private key, you are ok.  In other words, don’t share it or let it out!
2.c Add your public key to your new users /home/username/.ssh/authorized_keys file.  (how to’s for this stuff are on the web)
2.d Test this user’s login and make sure it logs you in from your local machine.
3.  Now, this new user is unprivileged, so you need to use sudo for running the remote rsync command.  Add this to your remote machine /etc/sudoers file:

 nameofnewuser	remotemachinename=NOPASSWD:/usr/bin/rsync

Above, you replace with the appropriate names.

4. Copy your private key from the remote machine and save it on the local machine where you will be backing up to.  For example, save it in the local user’s .ssh directory.  /home/localuseraccount/.ssh/private.key

5. You need to create a script.  In the example below, I have an exclude.txt file also, so I can exclude directories and files.  Look that up in the rsync how-to’s.

#!/bin/bash

rsync -avz --rsync-path="sudo /usr/bin/rsync" 	

	--exclude-from=exclude.txt -e

 	"ssh -p 22 -i /home/localuseraccount/.ssh/private.key"

  	remoteuseraccount@remote.server.com:/ /backup/to/path

In case you didn’t catch that, the section above with the rsync command is all one line!