Booting via GRUB 2 prompt

Did your update-initramfs not complete before you rebooted your zfs-on-root Ubuntu install? Here's how to boot it from the GRUB prompt using a previous kernel, assuming that your boot filesystem is on (hd0,gpt3), the zfs dataset name of your boot filesystem is bpool/BOOT/ubuntu/HEAD, the zfs dataset name of your root filesystem is rpool/ROOT/ubuntu/HEAD, and you're wanting to boot version 5.4.0-42-generic of the kernel:

grub> insmod zfs
grub> set root=(hd0,gpt3)
grub> linux /BOOT/ubuntu/HEAD/@/vmlinuz-5.4.0-42-generic boot=zfs zfs_force=1 root=ZFS=rpool/ROOT/ubuntu/HEAD net.ifnames=0 biosdevname=0 iommu=pt iommu=1
grub> initrd /BOOT/ubuntu/HEAD/@/initrd.img-5.4.0-42-generic
grub> boot

That should load the kernel and initial ramdisk. If you are then dropped to an (initramfs) shell, you'll have to manually import the zpool containing the root filesystem:

(initramfs) /sbin/modprobe zfs
(initramfs) zpool import rpool -R /root
(initramfs) exit

Note: The following:

net.ifnames=0 biosdevname=0

was added to the kernel parameters to ensure that all Ethernet device names use the naming scheme ethX for consistency. That part is not necessary. And the following:

iommu=pt iommu=1

was added to set some IOMMU options I need. That part is not necessary.

You'll then want to update your initial ramdisk, update grub, and re-install GRUB to be on the safe side.

• Verify /boot is mounted from your boot pool:

root@daytapod1 ~ $ findmnt /boot/
TARGET SOURCE                 FSTYPE OPTIONS
/boot  bpool/BOOT/ubuntu/HEAD zfs    rw,nodev,noatime,xattr,posixacl

• Update initial ramdisks for all kernels:

root@daytapod1 ~ $ update-initramfs -k all -u -v

• Update grub config:

root@daytapod1 ~ $ update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
done

• Install grub on all disks in the boot pool:

root@daytapod1 ~ $ zpool status bpool
  pool: bpool
 state: ONLINE
  scan: scrub repaired 0B in 00:00:03 with 0 errors on Sun Jan 10 00:24:04 2021
config:

        NAME                                                STATE     READ WRITE CKSUM
        bpool                                               ONLINE       0     0     0
          ata-Hitachi_HUA722020ALA330_JK1151YAH6E00Z-part3  ONLINE       0     0     0

errors: No known data errors

root@daytapod1 ~ $ grub-install /dev/disk/by-id/ata-Hitachi_HUA722020ALA330_JK1151YAH6E00Z
Installing for i386-pc platform.
Installation finished. No error reported.