====== Root Filesystem over NFS ======
This page explains how to use NFS to mount the root filesystem of a (WRAP) client from another server (a PC for example). One typical use is for development purposes; a NFS root filesystem mount makes it easy to access the same filesystem from both your development platform and your target platform so that you can test new features (programs, or kernel modules, or configurations) on a running platform, without rebooting or copying files over and over.
===== Pre-amble =====
Most systems that mount their root filesystems over NFS *also* boot the kernel over the network. See the [[pxe_voyage|PXE page]] about how to boot a diskless client over the network using PXE. I suggest you get that working first, then proceed with this.
===== Boot Process =====
==== PXE Boot Process ====
A quick summary of PXE booting:
**1.** the BIOS runs a built-in PXE agent (WRAP has one in the newer BIOS releases).
**2.** This agent uses DHCP and TFTP to load a PXE bootloader, PXELINUX in our case.
**3.** PXELINUX gets its configuration over TFTP, chooses a label to boot.
**4.** PXELINUX gets the Linux kernel and initial ramdisk root filesystem.
**5.** After this, the device proceeds standalone and no longer needs the server for its root filesystem.
====PXE + NFS Boot Process====
When using a root filesystem over NFS, steps 4 and 5 of the boot process behave as follows:
**4.** PXELINUX gets the Linux kernel and boots it.
**5.** The Linux kernel autoconfigures an IP interface and mounts an NFS filesystem as root filesystem.
======Setup======
Your server needs to run a NFS server (typically, the Linux kernel mode NFS server) and needs to export (make available) a root filesystem over NFS.
For installing NFS on your server, see this concise tutorial: http://www.metaconsultancy.com/whitepapers/nfs.htm
I have configured my NFS server exports as follows:
/etc/exports:
/home/leon/sandbox/wrap/voyage-0.2pre4 10.1.*.*(sync,rw,no_root_squash,no_all_squash)
See man exportfs and man exports.
The client must have the Linux network device driver being built-in the kernel (not as a module):
Device Drivers --->
Network device support --->
[*] Network device support
[*] EISA, VLB, PCI and on board controllers
<*> National Semiconductor DP8381x series PCI Ethernet support
Additionally NFS (rootfs) support must be compiled built-in:
<*> NFS file system support
[*] Provide NFSv3 client support
[ ] Provide client support for the NFSv3 ACL protocol extension
[ ] Provide NFSv4 client support (EXPERIMENTAL)
[ ] Allow direct I/O on NFS files (EXPERIMENTAL)
< > NFS server support
[*] Root file system on NFS
This kernel must be booted with special kernel command line options (configured on your server's pxelinux.cfg/default file):
ip=dhcp root=/dev/nfs
This tells the kernel to first auto-configure itself over DHCP, then mount the root filesystem over NFS.
During boot, keep an eye on the kernel log.
[ 266.463689] eth0: link up.
[ 266.499268] eth0: Setting full-duplex based on negotiated link capability.
[ 266.596386] Sending DHCP requests ., OK
[ 266.647679] IP-Config: Got DHCP answer from 10.1.0.1, my address is 10.1.0.105
[ 266.748590] IP-Config: Complete:
[ 266.788752] device=eth0, addr=10.1.0.105, mask=255.255.0.0, gw=10.1.0.1,
[ 266.882926] host=10.1.0.105, domain=foo.bar, nis-domain=(none),
[ 266.966968] bootserver=10.1.0.1, rootserver=10.1.0.1, rootpath=/home/leon/s
andbox/wrap/voyage-0.2pre4
[ 267.099381] Looking up port of RPC 100003/2 on 10.1.0.1
[ 267.169399] Looking up port of RPC 100005/1 on 10.1.0.1
[ 267.244698] VFS: Mounted root (nfs filesystem) readonly.
Note that I have my DHCP server fully set up to pass all relevant NFS options to the (WRAP) client:
subnet 10.1.0.0 netmask 255.255.0.0 {
range 10.1.0.100 10.1.0.200;
option broadcast-address 10.1.255.255;
option routers 10.1.0.1;
option domain-name-servers 192.168.1.254;
host wrap {
hardware ethernet 00:0D:B9:03:0F:D4;
filename "wrap/pxelinux.0"; # for PXE / TFTP
next-server 10.1.0.1; # for NFS
option root-path "/home/leon/sandbox/wrap/voyage-0.2pre4"; # for NFS
}
}
Alternatively, you can also supply or override the NFS server and root path on the Linux kernel command line as follows:
ip=dhcp root=/dev/nfs nfsroot=10.1.0.1:/home/leon/sandbox/wrap/voyage-0.2pre4
(See the kernel sources Documentation/kernel-parameters.txt file for details on the arguments.)
man dhcpd.conf
===== Remaining Issues in Voyage-0.2pre4 =====
This are bugs in the 0.2pre4 networking scripts. Both bugs result in the network temporarely going down. **This also happens if you run Voyage 0.2pre4 from CF disk**, but because the network goes up again soon after, you won't notice. For NFS mounts, the bugs are fatal though.
====mountnfs.sh deletes DHCP client lease files====
I was experiencing NFS disconnections after 5 minutes or so.
voyage:~# eth0: remaining active for wake-on-lan
nfs: RPC call returned error 101
nfs: RPC call returned error 101
nfs: RPC call returned error 101
<...repeats...>
My root filesystem was mounted as follows:
voyage:~# mount
rootfs on / type rootfs (rw)
/dev/root on / type nfs (rw,noatime,v2,rsize=4096,wsize=4096,hard,nolock,proto=udp,
addr=10.1.0.1)
<...>
Debuggin showed that /etc/rcS.d/S45mountnfs.sh called bootclean() from /etc/init.d/bootclean.sh, which cleared the PID and leases file that the DHCP client wrote just earlier. As a consequence, the DHCP client borked on renewing the lease, brings the eth0 interface down. This is fatal for a NFS root filesystem mount.
The solution is to remove, or comment out the last line in /etc/init.d/mountnfs.sh:
#
# Clean /tmp, /var/lock, /var/run
#
#bootclean mountnfs
The dependency on a reliable network is also the Achilles heel of a NFS mount. For operational use, I suggest using a ramdisk over NFS approach, or the conventional on-disk ramdisk approach.
====DHCP Client confused by Time Jumps====
After I debugged the fault above, the next one came along; the NFS mount still failed over time. AGAIN at the DHCP renew interval. Sigh.
I tested with the default Voyage 0.2pre4.tar.gz and it also has this problem: you see the eth0 interface brought down, then quickly up again.
Here I try to describe this subtle bug:
voyage:~# ls -al /var/run/
-rw-r--r-- 1 root root 457 Jan 1 2000 dhclient.eth0.leases
-rw-r--r-- 1 root root 5 Jan 1 2000 dhclient.eth0.pid
Note the time stamps of the DHCP files. The DHCP must have been started before the system got a NTP time. (This can be verified by the start-up order in /etc/rcS.d/ and /etc/rc2.d/.) The DHCP client writes the absolute lease ending time to its configuration file:
cvoyage:~# cat /var/run/dhclient.eth0.leases
lease {
interface "eth0";
fixed-address 10.1.0.108;
filename "wrap/pxelinux.0";
<...>
renew 6 2000/1/1 00:03:16; # <---------------- Note these timestamps
rebind 6 2000/1/1 00:05:08;
expire 6 2000/1/1 00:05:46;
}
Internally, the DHCP client has a timer counting down until renew time. In the meantime, the init.d/ntp script runs and suddenly the system time jumps forward by some 6 years and more. When the DHCP clients reaches its renewal time it decides that its lease is expired. The dhclient-script follows by bringing down the interface (on expire). The dhclient restarts DHCP discovery and lease request. The server gives out a new address (because it does not know what happened to the old one, as dhclient does not release its lease by default). Upon receiving a new IP address dhclient-script brings the interface up again (on IP address change).
This can be seen in the DHCP server log:
Mar 5 23:48:58 localhost dhcpd: DHCPOFFER on 10.1.0.107 to 00:0d:b9:03:0f:d4 via eth0
Mar 5 23:49:00 localhost dhcpd: DHCPREQUEST for 10.1.0.107 from 00:0d:b9:03:0f:d4 via eth0
Mar 5 23:49:00 localhost dhcpd: DHCPACK on 10.1.0.107 to 00:0d:b9:03:0f:d4 via eth0
<...>
Mar 5 23:54:02 localhost dhcpd: DHCPDISCOVER from 00:0d:b9:03:0f:d4 via eth0 <--------- Huh? Expecting REQUEST!
Mar 5 23:54:02 localhost dhcpd: Abandoning IP address 10.1.0.107: pinged before offer
Mar 5 23:54:08 localhost dhcpd: DHCPDISCOVER from 00:0d:b9:03:0f:d4 via eth0
Mar 5 23:54:09 localhost dhcpd: DHCPOFFER on 10.1.0.108 to 00:0d:b9:03:0f:d4 via eth0
Mar 5 23:54:11 localhost dhcpd: DHCPREQUEST for 10.1.0.108 from 00:0d:b9:03:0f:d4 via eth0
Mar 5 23:54:11 localhost dhcpd: DHCPACK on 10.1.0.108 to 00:0d:b9:03:0f:d4 via eth0
We have a chicken-egg problem here: NTP can only start after networking is up. But the DHCP client should not be exposed to time jumps.
I am looking for a solution still, a workaround is to disable the ifconfig down actions in the dhclient-script.
===== Further Notes =====
Normally, when you mount NFS filesystems on a client, you need to run a portmapper. This seems unnecessary for a root NFS mount, but if you want to mount any other filesystems later, you need a portmapper anyway:
So I installed these two packages on my (WRAP) client:
http://ftp.nl.debian.org/debian/pool/main/p/portmap/portmap_5-9_i386.deb
http://ftp.nl.debian.org/debian/pool/main/n/nfs-utils/nfs-common_1.0.6-3.1_i386.deb
====== Final Words ======
* This was written by Leon Woestenberg, inspired by the [[pxe_voyage|PXE Wiki page by Assaf Gordon]]. I copied parts of his Wiki page formatting.
* If you find any mistakes, correct them (this is a Wiki, after all).
* This is free documentation. It is distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantability or fitness for a particular purpose.
* Some of the commands in the guide should be executed as ''root'' - **this might damage your system!** \\ Use at your own risk.