Installing pfSense on APU boards over a serial cable

This tutorial should work on Linux, OSX and Windows.

Warning: if you have a TLSense router, please use a monitor+HDMI cable and download a VGA image instead.  

4 steps overview:

  1. Download memstick-serial release of the operating system you want to install.
  2. Burn the image on the USB drive
  3. Connect the USB drive and serial cable to APU board
  4. Start putty and connect

Now, more details:

Download pfSense image

Go to https://www.pfsense.org/download/ and select the "USB Memstick Installer" from the platform dropdown.

 

Flash the pfSense image on the USB drive

If you are using Linux/OSX, you should use dd to prepare the USB stick:

sudo dd if=pfSense-CE-memstick-2.4.0-RC-amd64.img of=/dev/sdX ; sync

Make sure that if= specifies the exact location of the downloaded file and of= specifies your USB device. If you make a mistake here, you might overwrite your hard drive!

If you are using Windows, use Rufus: https://rufus.akeo.ie/

Connect USB drive and serial cable to the APU board

If you have a USB-to-serial cable, it will look like on the picture. Regular null-modem cable also works, but looks slightly different. 

If you are using OSX, make sure that your tty device shows up in /dev/.See MacOS instructions here. 

If you are using Windows, see the windows instructions here.

If you are using Linux, you should see the /dev/ttyUSB0, if you have a regular cable you should see /dev/ttyS0

Verify with ls: 

root@homedesktop:# ls /dev/ttyUSB0 
/dev/ttyUSB0

All good. Ready to connect.

Connect

Make sure you have putty installed, and start it as root:

sudo putty

If you use Windows, launch putty with administrative privileges.

Enter the values as you see below: 

 

Click open and your session should start. It should look similar to this:

 

Wait for the USB to boot, and follow the instructions on the screen:

Press F10 to make sure your USB stick has been detected. It should look similar to this:

1. this should be your USB stick
2. here should be your SSD drive

 

If you are not seeing USB stick in the boot menu, it's possible that your USB brand is not supported by APU.

Some USB sticks simply don't work. Try a different brand.

Press 1 and follow the instructions on the screen.

Note, BIOS will always say "Booting from Hard Disk..." even if it's booting from USB.

Recording of the installation process is shown below. Note, we have installed pfSense with ZFS instead of UFS to minimize the filesystem corruption problem.

Troubleshooting

pfSense 2.5.0 is based on FreeBSD 12.2. This OS has some USB driver bug that often manifests itself by printing 

Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM
(probe0:umass-sim0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00                                                                                              
(probe0:umass-sim0:0:0:0): CAM status: CCB request completed with an error                                                                              
(probe0:umass-sim0:0:0:0): Retrying command, 3 more tries remain

over and over again. This warning appears only when a USB device is connected to APU.

Since you need a USB stick to install pfSense, this can be problematic. About half of my USB sticks don't work due to this bug, but some of them do. If you keep getting this issue, try a different USB brand.

If I find a workaround for this issue, I'll post it here. 

Here's a recording from a successful installation: https://asciinema.org/a/392645

Good luck!