FreeBSD 4 - Howto
This document serves as a step by step guide on how to make a PXE boot environment
for use in largescale FreeBSD deployments. The same thing is also possible for
at least Linux, NetBSD and OpenBSD. This document aims specifically at FreeBSD 4.x.
For detailed information on the different subsections of this howto, please refer to
the sections to your left.
0. Prerequisites
You should have in-depth knowlegde of your operating system. You need at least two
servers. The first server, called
bootserver, runs ISC DHCPd, has a TFTP
server and runs RSYNC in servermode. We make several assumptions:
- The PXE server is 192.2.0.16/24, its hostname 'pxe-test.ipng.nl', its MAC
address 00:02:b3:4c:ed:c8.
- The bootserver is 192.2.0.2/24
- The DHCP configuration file lives in /usr/local/pxe/dhcpd.conf
- The TFTP directory is /usr/local/pxe/tftpboot/
- The RSYNC configuration file lives in /usr/local/pxe/rsyncd.conf, the modules
live in /usr/local/pxe/rsync/
1. Enabling tftpd
1. FreeBSD's standard tftp server works just fine. Add:
tftp dgram udp wait root /usr/libexec/tftpd tftpd -l -s /usr/local/pxe/tftpboot/
to /etc/inetd.conf. Make sure your server starts inetd by adding inetd_enable=YES;
to /etc/rc.conf.
2. mkdir -p /usr/local/pxe/tftpboot/
3. Start /usr/sbin/inetd.
3. Building FreeBSD's PXE bootloader
1. In /etc/make.conf, add LOADER_TFTP_SUPPORT=YES; This enables pxeboot(8) to
retrieve the kernel via TFTP, it's default is via NFS.
2. cd /usr/src/sys/boot; make clean; make; This makes pxeboot(8) with TFTP support
3. cp /usr/obj/usr/src/sys/boot/i386/pxeldr/pxeboot /usr/local/pxe/tftpboot/
4. Configuring DHCP
In /usr/local/pxe/dhcpd.conf, add:
option rsync-path code 194 = text;
host pxe-test {
hardware ethernet 00:02:b3:4c:ed:c8;
fixed-address 192.2.0.16;
next-server 192.2.0.2;
option domain-name "ipng.nl";
option host-name "pxe-test";
filename "pxeboot";
option rsync-path "192.2.0.2::test/";
}
You now have a DHCP server which will offer the PXE server the pxeboot(8) loader you
just created in step 3.
5. Create an RSYNC module called 'test'
1. Install RSYNC; pkg_add -vr rsync
2. In /usr/local/pxe/rsyncd.conf, add:
use chroot = yes
max connections = 16
pid file = /var/run/rsyncd.pid
gid = wheel
uid = root
[test]
path = /usr/local/pxe/rsync/test/
comment = PXE root repository for pxe-test.ipng.nl
read only = yes
3. Create the module directory; mkdir -p /usr/local/pxe/rsync/test/
4. Populate the RSYNC module with files you want on your PXE server. In particular,
make sure you have the following files: /etc/rc.d/rc.inet1, /etc/rc.d/rc.inet2 and
/etc/rc.d/rc.local. Note that these files follow the Slackware filename scheme.
5. Start RSYNC /usr/local/bin/rsync -4 --daemon --config=/usr/local/etc/rsyncd.conf
I often run services in DJB daemontools, to ensure that the software gets restarted
in the event of a server or software crash. Your mileage may vary.
6. Build a PXE image:
1. cvs checkout pxe
2. cd pxe
3. ./build.sh
4. gzip -c initrd > /usr/local/pxe/tftpboot/192.2.0.16/initrd.gz
I keep all sorts of sizes laying around. The image is currently some 22MB in size, but sometimes I need quite a lot of extra stuff onboard, so I make varying filesystem sizes. As you can see, making larger ramdisks does not take much more space once you gzip them:
$ ls -l /usr/local/pxe/tftpboot/initrd*.gz
-rw-r--r-- 1 root
wheel 7871137 Feb 9 13:28 initrd_4.10-p5-128M.gz
-rw-r--r-- 1
root wheel 7704303 Jan 7 16:25 initrd_4.10-p5-32M.gz
-rw-r--r--
1 root wheel 7714334 Jan 5 10:30 initrd_4.10-p5-40M.gz
-rw-r--r-- 1 root wheel 7726349 Jan 5 10:30
initrd_4.10-p5-48M.gz
-rw-r--r-- 1 root wheel 7765873 Feb 11
13:15 initrd_4.10-p5-64M.gz
7. Build a kernel for the PXE server
1. Edit your kernel configuration in /usr/src/sys/i386/conf/PXE. Strip IDE, SCSI and
other stuff you don't have from it. GENERIC will do fine, by the way.
2. cd /usr/src; make buildkernel KERNCONF=PXE
3. Copy it into place (gzipping it): gzip -c /usr/src/sys/i386/compile/PXE/kernel >
/usr/local/pxe/tftpboot/192.2.0.16/kernel.gz
4. mkdir -p /usr/local/pxe/tftpboot/boot
5. cp /boot/loader /boot/boot[12] /usr/local/pxe/tftpboot/boot
6. Create /usr/local/pxe/tftpboot/boot/loader.rc with these contents:
echo Configuring for IP ${boot.netif.ip}
echo Loading kernel...
load ${boot.netif.ip}/kernel
echo Loading root filesystem...
load -t mfs_root ${boot.netif.ip}/initrd
echo Booting...
set vfs.root.mountfrom="ufs:/dev/md0c"
boot
When the PXE BIOS receives the DHCP reply, that reply will contain a filename to
load. It will load pxeboot(8) via TFTP, which in turn will load /boot/loader and
/boot/loader.rc from the TFTP server. The file loader.rc will then have the
pxeboot(8) loader fetch /192.2.0.16/kernel.gz and then /192.2.0.16/initrd.gz. It
then instructs the pxeboot loader to try to find its root filesystem from the
initrd. Note that gzipping the files saves a lot of TFTP'ing; the pxeboot(8) loader
supports unzipping and will always attempt to find .gz files first.
8. Test!
Basically, the following will happen when you turn on your PXE server:
- It will start the PXE BIOS.
- PXE will issue a DHCPDISCOVER and a DHCPREQUEST;
- Your dhcpd will tell it that it is 192.2.0.16/24 and that it should fetch
tftp://192.2.0.2/pxeboot;
- pxeboot will be fetched and executed. It will fetch tftp://boot/loader and
tftp://boot/loader.rc;
- loader will fetch tftp://192.2.0.2/kernel.gz and tftp://192.2.0.2/initrd.gz
- The kernel will be bootstrapped with the initrd as root filesystem.
- The kernel will start /etc/rc, which in turn will:
- Run /etc/rc.d/rc.S; It will issue another DHCP request; store the lease
variables in /etc/DHCP_ENVIRONMENT
- Read the environment and if the RSYNC_PATH variable is set (it is, because of
the dhcpd.conf you created in step 4), it will rsync that module over the root
filesystem.
- Execute djb daemontools (from /var/service)
- Execute /etc/rc.d/rc.M [present in the image], which does some basic sanity
checking and starts sshd; rc.M then starts:
- /etc/rc.d/rc.inet1, if present. You can configure network interfaces here.
- /etc/rc.d/rc.inet2, if present. You can start network services here.
- /etc/rc.d/rc.local, if present. You can start other things here, or symlink
/etc/*-service to /var/service to make daemontools run it.
DHCP Server config:
We define a specific option (number 194) which we declare to be a string. In
DHCP terms, this results in the following configuration line somewhere at the
top of your dhcpd.conf:
option rsync-path code 194 = text;
...
host pxe-test {
hardware ethernet 00:01:80:57:5A:40;
fixed-address 192.2.0.16;
next-server 192.2.0.2;
option domain-name "ipng.nl";
option host-name "pxe-test";
filename "pxeboot";
option rsync-path "192.2.0.2::test/";
}
The clause above looks mostly like a normal static hostname declaration. We
include
host-name,
domain-name and a specific BOOTP option called
next-server. The PXE bios will use this IP address to fetch
the argument to
filename, and will start executing that filename.
dhclient configuration in the image
The client side is contained within the PXE bootimage (initrd.gz).
In the image, we have /etc/dhclient.conf, which also defines the
rsync-path dhcp option and tells the dhclient program to request this
information from the server:
option rsync-path code 194 = text;
request host-name, domain-name, ntp-servers, rsync-path;
With these two lines in /etc/dhclient.conf, your PXE server will know
how to find its post-boot files, using the line 'option
rsync-path "192.2.0.2::test/";' in the bootservers dhcpd.conf.
The INITRD:
The initrd aims to create an environment which is the same regardless of
operating system choice. To that end, I shuffeled binary locations
around a bit, mostly resembling the FreeBSD filesystem layout.
We have the following binaries:
/bin: df ln rm echo ls rmdir cat ed mkdir chmod expr mv sleep
cp grep ping stty csh hostname ping6 sync date kill ps test dd
ksh pwd bash
/sbin: dhclient init mount_nfs route dhclient-script mount_procfs
rtsol dmesg mountd swapon fdisk ldconfig newfs sysctl fsck md5 tunefs
halt mknod nologin umount ifconfig mount swapctl mount_mfs reboot
/usr/bin: awk bitkeys chflags crontab cut du env envdir envuidgid fghack
find fstat ftp grep gzip id less logger login more multilog netstat
nfsstat rpcinfo rsync scp sed setlock setuidgid sftp showmount
softlimit ssh ssh-add ssh-agent ssh-keygen ssh-keyscan supervise svc
svok svscan svstat systat tai64n tai64nlocal tail tar telnet top touch
uname uptime vi view vmstat w wc ex wget who passwd su which sort
uniq printf cmp head xargs egrep fgrep zegrep zgrep compress gunzip
gzcat uncompress zcat ex view nawk slogin
/usr/sbin: arp chown cron dev_mkdb inetd iostat ntpd ntpdate
pwd_mkdb rtsold sshd syslogd tcpdump traceroute traceroute6 vipw;
There's a CVS repository I maintain which contains a shell script which
tries to create an initrd with the proper files in their proper
locations. Of course different operating systems (and different
distributions based upon the Linux kernel) tend to have binaries in
different places. If a binary is not present (ie swapctl for Linux),
it is not copied to the image.
/etc is sparsely populated. We only have critical files such as:
dhclient.conf ntp.conf protocols services ssh
crontab hosts profile rc.d shells
termcap fstab group master.passwd
rc.firewall ttys auth.conf gettytab
login.conf rc rc.firewall6
Note that the root-user has a preset password, only known to me. It
seems like a good idea to override the initrd image with local
configuration files in your RSYNC repository.
Disclaimer: This page was created for your convenience. I do not offer support on
PXE servers at all. If these pages aren't enough to get you going, please use Google
or other means of information. Do not bother me with questions. I know for certain
this stuff works, BIT runs at least 15 production servers with this PXE image.