Tag: ZFS
Install FreeBSD on a ZFS pool that’s mirrored using GPT partitions
by Greg on Aug.30, 2010, under FreeBSD
Here’s my notes for ZFS root installation with FreeBSD 8.1. (amd64 on my system)
Boot up the FreeBSD install disk, you’ll need one with the Fixit system. In the examples listed, I had 2 disks, ad4 and ad6. Yours may be different, replace as necessary.
1. Add the modules on boot, hit “6″ at the BTX loader menu, then enter these lines at the “OK” prompt.
load ahci.ko load opensolaris.ko load zfs.ko boot
2. At the sysinstall menu, go to Fixit.
3. Create the GPT disks and partitions.
- Delete any partions already on the disk, and destroy. (only if needed) On my disks, I had 2 partitions created from another OS installation. “-i 2” indicates the partition number. You may need to add a third or more.
gpart delete -i 2 ad4 gpart delete -i 1 ad4 gpart delete -i 2 ad6 gpart delete -i 1 ad6 gpart destroy ad4 gpart destroy ad6
- Create the gpt disk.
gpart create -s gpt ad4 gpart create -s gpt ad6
- (NOTE: “gpart show” and “gpart show –l” will display the partitions.
- You may want to layout different partitions, but we’ll just create a boot, swap, and zfs partition here, with GPT labels. (swap is better left off ZFS for crash dumps)
gpart add -s 128 -t freebsd-boot -l boot0 ad4 gpart add -s 128 -t freebsd-boot -l boot1 ad6 gpart add -s 8388608 -t freebsd-swap -l swap0 ad4 gpart add -s 8388608 -t freebsd-swap -l swap1 ad6 gpart add -t freebsd-zfs -l disk0 ad4 gpart add -t freebsd-zfs -l disk1 ad6
-
Next, add the bootcode to the disk.
gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 ad4 gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 ad6
4. OK, now we’ve got some GTP disks setup for a mirror and boot.
IMPORTANT!!! Make sure you don’t skip this step, create the /boot/zfs directory for the zpool.cache.
mkdir /boot/zfs
5. Now we can start the ZFS fun! On the FreeBSD wiki and on other sites I used as reference, there may be listed other options and settings for your ZFS layout. Here, I am going to keep it very simple, but I’ll list out my options later.
Also, you may prefer different names or even separate location/directory/ZFS filesystem for your root pool. My preference is to call the root pool “rpool” and place the FreeBSD system in the ROOT filesystem. (looks like this: rpool/ROOT) In addition, my preference is to place “/home” outside of the ROOT fs. Which allows is to separate system from data and management of snapshots is more flexible. But this is totally a preference thing. One of the beautiful things about ZFS is the flexibility and ease of use.
- Let’s create the pool. Notice the gpt/disk# items, which correspond to your gpt labels. Also, we’ll disable a mountpoint on rpool.
zpool create rpool mirror gpt/disk0 gpt/disk1 zfs set mountpoint=none rpool
- Add the ROOT filesystem. Again, here you may prefer to add all ZFS filesystems for places like /usr, /var, /tmp, and other system locations. I don’t recommend it, but in this example we’ll keep it all in one filesystem, except /home.
zfs create -o mountpoint=/mnt rpool/ROOT zfs create -o mountpoint=/mnt/home rpool/home
(OPTIONAL file systems)
zfs create rpool/ROOT/usr zfs create -o compression=lzjb -o setuid=off rpool/ROOT/usr/ports zfs create -o compression=off -o exec=off -o setuid=off rpool/ROOT/usr/ports/distfiles zfs create -o compression=off -o exec=off -o setuid=off rpool/ROOT/usr/ports/packages zfs create -o compression=lzjb -o setuid=off rpool/ROOT/usr/src zfs create rpool/ROOT/var zfs create -o exec=on -o setuid=off rpool/ROOT/tmp
6. Install the base system. We’ll be intsalling the system into the /mnt directory for now, but we’ll change those mounts later.
-
cd /dist/8.1* export DESTDIR=/mnt for dir in base catpages dict doc games info lib32 manpages ports; do (cd $dir ; ./install.sh) ; done cd src ; ./install.sh all cd ../kernels ; ./install.sh generic cd /mnt/boot ; cp -Rlp GENERIC/* /mnt/boot/kernel/
7. Edit /mnt/boot/loader.conf and add these lines.
ahci_load="YES" zfs_load="YES" vfs.root.mountfrom="zfs:rpool/ROOT"
8. Edit /mnt/etc/rc.conf and add these lines. I have re0 as my network interface, your’s may be different.
zfs_enable="YES" ifconfig_re0="DHCP" hostname="systemname"
9. Edit /mnt/etc/fstab and add these lines. Setup your swap.
/dev/gpt/swap0 none swap sw 0 0 /dev/gpt/swap1 none swap sw 0 0
10. Copy the zpool.cache. IMPORTANT!!!!! Don’t forget this step!!
cp /boot/zfs/zpool.cache /mnt/boot/zfs/zpool.cache
11. Export this…
export LD_LIBRARY_PATH=/dist/lib
12. Unmount ZFS filesystems and set the correct mountpoints for the new root to boot. If you setup other/optional zfs filesystems, then you’ll want to set their correct mount points, too.
zfs umount -a zfs set mountpoint=legacy rpool/ROOT zfs set mountpoint=/home rpool/home
13. Set the bootfs property of the pool to rpool/ROOT
zpool set bootfs=rpool/ROOT rpool
That’s about it! Well, actually, you’ll still need to do a lot of post install stuff. (see the wiki (set passwd, time zone, etc)) This is only a very base load of FreeBSD, root doesn’t even have a password.
Keep in mind, at the time I wrote this, there is a bug in the zfs boot loader, where you can only boot from the first disk in the mirror. Kind of reduces the usefulness of a root mirror, that’s for sure. However, there is a patch that you can compile a new zfs loader with. There’s a procedure to do so, and I’ll write up a how-to on that sooner than later.
Here are the site’s I used as reference, as well as the forum post on the root mirror boot issue.
http://www.b0rken.org/freebsd/zfs.html
Unable to login to CIFS ZFS share and windows prompting for password
by Greg on Dec.19, 2009, under OpenSolaris
I setup my share right, followed all the instructions, set the /etc/pam.conf entry. Couldn’t get into my share, and my password wouldn’t work.
I found that you can test your PAM mapping like so:
smbutil login –c username
But that gave me:
Keychain entry not found.
Ok, had to do a little searching… found out that you can SET THE STORED PASSWORD FOR SMB! Duh! Why didn’t I think of that in the first place? LOL!
All you need to do is this:
smbutil login username@server
It will ask you for a password and suddenly your login from windows will work! Nice!
Keep in mind, that is assuming your server is setup in workgroup mode, joined to your workgroup not the domain. I think you would just specify “username@netbiosdomain” in that case.
ZFS CIFS Network Password Is Not Correct
by Greg on Oct.25, 2009, under OpenSolaris
Check the /etc/pam.conf has the smb line:
other password required pam_smb_passwd.so.1 nowarn
If not, add that to the end of the file. If you, check /var/smb/smbpasswd you should also see a line for your account with a hashed password listed. In my case, I had my username and an ID listed, but not hashed password. I didn’t have the line in pam.conf. After I added it, I had to run the passwd username command on my account, which generated the smb password and inserted it into the smbpasswd file. Then the error about password not being correct went away and I could get in from Windows. (Windows 7 too! With the “NTLMv2 if negotiated” option in policy.)
OpenSolaris ZFS rpool mirror
by Greg on Aug.08, 2009, under OpenSolaris
Notes to self:
Install your system to a PARTITION, NOT the whole drive. I seem to remember reading somewhere that this was required.
My system had 2 disks. rpool was setup on disk 0. (those are ZERO’s not O’s.)
c8t0d0s0 (disk 0 with rpool) c8t1d0s0 (disk 1 we want to mirror)
After installed and booted into new system… (as root)
format (choose your second disk, should be 1 and in this sys it was c8t1d0) fdisk (here, choose Y to select the 100% Solaris partion) exit to save changes
Now do…
prtvtoc /dev/rdsk/c8t0d0s0 | fmthard -s - /dev/rdsk/c8t1d0s0
(notice the “rdsk”, it’s not “dsk”, and it includes the slices. If you installed on a full drive, I don’t think you see the “s0″ slices)
zpool attach -f rpool c8t0d0s0 c8t1d0s0
And last setup grub on the second disk…
installgrub -m /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c8t1d0s0
All done! Make note that you use “rdsk” on the prtvtoc command. I kept getting “must be raw device” errors because I missed the “r” and had only /dev/dsk in the drive path. Also, on first attempts, I somehow messed up my second drive partitions and labels and it was EFI labeled. Nothing worked when it was EFI, and I couldn’t remove it either. Since I did this initially in a VMware guest, I just deleted the disk and added a new one. But that’s important to note, do not do anything to that drive except run “format” and then use the “fdisk” command on the new drive. It will come up and say something like you need a 100% Solaris partition, just say Y and exit that to save. THEN… you can do the prtvtoc thing to copy the partition table to the new drive.
ZFS CIFS and ACL Inheritance
by Greg on Jul.24, 2009, under Networking, OpenSolaris, Security
This is just another one of those things that didn’t make any sense and only partially does now. At least NOW I know there is more at play here than the simple solutions in Samba using create mask and create directory mask. In Linux, that’s how I would get around the issues of Windows directory permissions running on a Linux SMB share.
Now, I am learning to do things the OpenSolaris way. I am loving OpenSolaris and ZFS! However, coming from a Linux and Windows “way of life”, there are some differences that just aren’t clear. What kills me is, I try the RTFM thing, and somehow completely miss that one little thing that makes it all work. Off topic, but an example, coming from Linux, I would just type “su” and get root access. In OpenSolaris, that won’t work. Neither will “pfexec su”, nor “sudo su”. Then one day, after dealing with it for a week or so, I stumble upon a post where someone in an unrelated sample script typed “pfexec su – root”. There ya go! Argh!
Anyway, back on the ZFS/CIFS/ACL thing. It was driving me nuts that I couldn’t figure it out. I wanted a folder with this setup:
/pool/sharefs – owner:greg – group:domusers
greg and domusers should have full control and all folders under “sharefs” should inherit that.
So under linux/samba, that’s where I would do like “create mask = 770″ or simlar, and “force create group = domusers”. Something like that, can’t remember exactly. made it simple actually. It always wrote files with the right perms and ownership and other people in that group could read/write just fine.
Problem is, you can’t get very specific about who get’s what, where, and you can’t use more than one group. Well, sure enough, there’s a thing called “ACL” that handles that stuff now. It’s been around for a while now, but I never even heard of it until I started using OpenSolaris. I like how it seems to be more compatible with the way Windows handles ACL’s. What I don’t like is, it’s confusing. I get the NTFS/Share perms in Windows, been doing that a long time now. The CIFS/ZFS ACL thing kind of makes sense, and it will “click” at some point the more I use it.
After spending hours on this, I reached a point where I had to figure it out. Here’s what I did.
On the ZFS file system, create it normally for SMB access. Then I changed some properties for aclinherit and aclmode. Change those to “passthrough”:
zfs set -o aclinherit=passthrough -o aclmode=passthrough pool/sharefs
Then chmod/chown. OH! That’s another thing. You need to use /bin/chmod and /bin/ls! Not just type: chmod … That wont work. In OpenSolaris the default path points to /usr/gnu/bin/chmod, which doesn’t have the “A” or “V” options to set/view ACL’s. That was another thing that DROVE ME CRAZY!!! I read the man pages and manuals and docs online and I didn’t catch anything that said, “Hey, there are different versions of chmod and ls here!” I can’t believe the time wasting here! Back to the point, do this to put your own default perms on:
/bin/chmod 2774 /pool/sharefs (I actually am not positive that is needed, but I think it set group as inheritable) /bin/chmod -R A- /pool/sharefs (that will wipe out the current perms) /bin/chmod -R A=owner@:full_set:fd:allow /pool/sharefs (resets perms with only that acl) /bin/chmod -R A+group@:full_set:fd:allow /pool/sharefs (that appends the group perms, full control) /bin/chmod -R A+everyone@:read_set:fd:allow /pool/sharefs (above appends everyone read access)
In all the above that will preset INHERITABLE permissions for the subdirectories. Notice above there is one with “A=” on it? That will reset the perms and set only that perm. So I guess you may not even need the previous line for “A-” to reset. (I am just learning here ya know!)
It looks as if that makes a little sense now. You can view the current ACL’s like so: ”/bin/ls -V /pool/sharefs”
In my case, I might want to add another user or group:
/bin/chmod -R A+user:stacy:full_set:fd:allow /pool/sharefs /bin/chmod -R A+group:othergroup:full_set:fd:allow /pool/sharefs /bin/chmod -R A+group:yetanothergroup:read_set:fd:allow /pool/sharefs
So with this setup I can now open the share on the server and create a file or folder with inherited permissions. It does, however, save my username as a new owner, so keep that in mind. But if the group stays in there with “domusers” as full read/write access, I am happy.
Well, now I get it just a little and it makes more sense compared to Windows ACL’s. I didn’t go over any share specifics and authentication issues, this was just ACL’s! I still have to RTFM my way around that for a while. Next project, join OpenSolaris to a Windows domain. (Which, BTW, does not work in NT Domain style connections, you have to use Active Directory.)