These days, we take certain things for granted: there’s always DHCP, PXE-enabled clients and modern UEFI firmware with rich UI and mouse support.

Imagine we are in the 1980s: the Xerox Alto already exists, but it’s still too early even for BOOTP. Though the world looked very different back then, the problem of network booting was already there. The concept is simple: computer contains bootstrap code stored in a non-volatile memory (e.g. ROM), which allows it to connect to a server and retrieve the kernel over a network.

I’m not going to take you through the full history – this is just to give you a rough feeling of where we are. So how about bootstapping a machine by using TFTP and RARP?

I personally find it very interesting to learn about the evolution of the process, as it helps build a better understanding of the current state of things and why certain design decitions were made.

Thanks to QEMU and its wide range of emulated devices, we don’t need real hardware to set up a lab environment for booting an old machine over the network and getting the kernel up and running. After checking list of supported devices I found that there are ne2k_pci and ne2k_isa which emulate a very popular NE2000 Ethernet network card that supports RARP. We could also use a floppy disk to start the network booting.

NE2000 NE2000 image taken from https://de.wikipedia.org/wiki/NE2000

Wherever I May ROM

Emulated hardware is great, but it doesn’t help much without a proper boot ROM. Fortunately we can build our own ROM using Etherboot. Of course it won’t behave like the original one, but it is enough for our small experiment.

“Etherboot is a free software package for making boot ROMS for booting GNU/Linux and other operating systems on x86 PCs over a network using Internet Protocols” – Free Software Directory

I assume you can follow QEMU documentation and configure it on your host machine. As for now, we will need QEMU VM to compile our custom ROM with RARP support (because DHCP/BOOTP is not invented yet). The reason to use QEMU at this stage is because Etherboot requires i386 and I don’t want to deal with cross-compilation. I decided to use Ubuntu Trusty, but you can use any other Linux distro with 32-bit support.

  1. Create a new virtual disk image
❯ qemu-img create -f qcow2 trusty.qcow2 20G
  1. Create a new VM (emulating i386)
❯ qemu-system-i386 \
    -m 2048M \
    -cdrom ubuntu-14.04.6-server-i386.iso \
    -hda trusty.qcow2 \
    -vga std \
    -netdev user,id=net0,hostfwd=tcp::2222-:22 \
    -device e1000,netdev=net0
  1. Proceed with a normal Ubuntu installation and once it is up and running it’s time to prepare our environment
❯ apt update
❯ apt install -y git make gcc
❯ git clone https://github.com/etherboot/etherboot
❯ cd etherboot/src
  1. Etherboot supports various configuration options and thats how we disable DHCP support and enable RARP.
❯ EXTRA_CFLAGS="-DNO_DHCP_SUPPORT -DRARP_NOT_BOOTP" make bin/ne.rom

Most of the boot ROMs in the past either had a hardcoded (preconfigured) TFTP address or had some logic where they would scan the network themselves.

From ROM With Love

By default QEMU will run SeaBIOS and iPXE, but we don’t want this behaviour, so we will use Option ROM by passing -option-rom argument.

  1. Fetch ROM from QEMU VM (used in previous steps) onto your host
❯ scp -P 2222 'ubuntu@localhost:/home/ubuntu/etherboot/src/bin/ne.rom' .
  1. Prepare a bridge interface because RARP works only within a broadcast domain. I am not going to dive deep into details how to create one as I already had my bridge configured. Depending on your host OS things might be a bit different, but you can search for qemu br0 tap0 interface on the Internet.
# Create bridge
❯ ip link add br0 type bridge
❯ ip link set br0 up

# Create TAP device
❯ ip tuntap add tap0 mode tap
❯ ip link set tap0 up
❯ ip link set tap0 master br0

# Create veth pair
❯ ip link add veth0 type veth peer name veth1
❯ ip link set veth0 up
❯ ip link set veth1 up
❯ ip link set veth0 master br0

# Assign IP to the br0
❯ ip addr add 10.166.116.1/24 dev br0
  1. Start a new QEMU VM with a custom ROM and bridged interface
❯ qemu-system-i386 \
    -m 128M \
    -boot n \
    -option-rom ne.rom \
    -net nic,model=ne2k_isa \
    -net tap,ifname=tap0 \
    -nographic
  1. Once VM is started you should see that it detected our [NE*000] and was able to read MAC address and now is looking for a RARP server.
SeaBIOS (version 1.15.0-1)
ROM segment 0xca00 length 0x0000 reloc 0x00000000
Etherboot 5.4.4+ (GPL) http://etherboot.org
Drivers: NE*000   Images: NBI ELF PXE   Exports: PXE
Protocols: RARP TFTP
Relocating _text from: [00093640,0009f9f0) to [07ef3c50,07f00000)
Boot from (N)etwork or (Q)uit? N

Probing pci nic...
Probing isa nic...
[NE*000]
NE2000 base 0x0300, addr 52:54:00:12:34:56
Searching for server (RARP)...
  1. Time to install and configure rarpd. This is the server-side implementation of RARP that returns IP address on each RARP request. I am doing it on my host directly as it can be easily removed later.
❯ sudo apt install rarpd
# Let's validate that it is up and running
❯ sudo service rarpd status
● rarpd.service - LSB: RARP lookup daemon
     Loaded: loaded (/etc/init.d/rarpd; generated)
     Active: active (running)
  1. Configure rarpd by adding MAC-IP pair to /etc/ethers. I will let you discover yourself what IP and MAC are needed in your case. RARP uses a centralized table that maps MAC addresses to IP addresses. Each MAC must be manually registered by an administrator. The protocol provides no data beyond the IP address. It could be that running sudo service rarpd restart is required afterwards.
❯ cat /etc/ethers
52:54:00:12:34:56 10.166.116.76
❯ sudo service rarpd restart

Now if we run our QEMU VM again we’ll see a bit different output. Machine got an IP address and it is now trying to fetch the kernel over TFTP.

NE2000 base 0x0300, addr 52:54:00:12:34:56
Searching for server (RARP)...
Me: 10.166.116.76, TFTP: 10.166.116.1
Loading 10.166.116.1:/tftpboot/kernel.10.166.116.76 .

When I was checking network traffic with tcpdump I excepted to see MAC, but to my surprise it was showing an IP.

15:25:17.753553 ARP, Reverse Request who-is 10.166.116.76 tell 10.166.116.76, length 46
15:25:17.753694 ARP, Reverse Reply 10.166.116.76 at 10.166.116.76, length 28

However when inspecting using Wireshark it was showing our MAC wireshark-pcap

I hope you had fun and reached the point where the machine is asking for a kernel. Hopefully, preparing and booting the kernel will be a post for another day.