Failed to start File System Check on Root Device

[sp01109]

fwup: v1.8.3 / v1.8.4
Platform: RPI 3B+
Linux kernel: 4.19.66-v7+

Problem

[FAILED] Failed to start File System Check on Root Device. See 'systemd-fsck-root.service' for details.

Every time, after using fwup to upgrade the system, error messages are shown and the system will shut down by itself. At the 2nd time to restart the system, it works as normal again.

In /boot/cmdline.txt, fsck.mode=auto fsck.repair=yes is used. If set it to fsck.mode=skip fsck.repair=no, since fsck won't be executed, the message won't be shown. However, after using fsck, the message shows again once.

I wonder if it may be related to an issue found and fixed in v1.8.4 as one of the bug fixes notes states that "... fsck.fat checks in dosfstools ..."

P.S. The way to create root file image is:

1. dcfldd to freeze the root file system partition
2. e2fsck & resize2fs to reduce the root file system image size

[fhunleth]

Are you trying to update a filesystem in place? I.e., are you booted and using the root filesystem and then running fwup to upgrade it?

[sp01109]

I think the answer is yes. After booting up the system, there is an application will call fwup via system command to execute the upgrade. After upgrading, the application will restart the system to make the new root file system become the new one.

However, an A/B system update mechanism is configured on the disk. FYI.

[fhunleth]

Hmm. I'm not sure if I'm understanding. You'll need to make sure that the filesystem that you're upgrading is unmounted. That's why I asked. If the filesystem isn't unmounted, it is really easy to corrupt the upgrade since almost any write that the OS does will corrupt it. For example, when you reboot and the OS unmounts everything, it will write anything that's still in case.

What you're describing really sounds like you're either writing to a mounted partition or your A/B partitions overlap. If you really think that it's fwup's fault, could you try manually upgrading your system using dd? fwup is basically a scripted dd with a few extra features, so I feel like you should be able to reproduce the issue with dd and perhaps that will give you a hint as to what's wrong.

[sp01109]

@fhunleth, Thank you very much for helping me. I am sorry about the confusion here. I actually just reused the example of fwup.conf in your another project, bbb-buildroot-fwup. I am referring to the pingpong example of RPi.

I thought I could run fwup in the active filesystem (such as a root filesystem in A partition) and have fwup put the new filesystem to the B partition. So the next time, as the system starts, the system in B partition will be used. I thought in this case, B partition is unmounted since it is not being used. (i.e. It is not shown as an active one as df command is used.)

Please correct me if my understanding of the ping-pong method is wrong.

[fhunleth]

Oh, I have not updated that project in far too long. Are you running fwup -a -d /dev/mmcblk0 -i myfirmware.fw -t upgrade?

Another thing to mention is that fwup runs the first task in the configuration who's prefix matches the task specified on the command line. So -t upgrade will tell fwup to look at both the upgrade.a and upgrade.b tasks. The require function in the upgrade.* tasks will be checked. One of them will work and fwup will run the commands in it. That's how fwup figures out whether to upgrade A or B. You can also specify -t upgrade.a or -t upgrade.b on the command line to try to force as task as a way of debugging.

My last thought is for you to look at fwup.conf files for Raspberry Pis that I update more regularly: https://github.com/nerves-project/nerves_system_rpi4/blob/main/fwup.conf and https://github.com/nerves-project/nerves_system_rpi0/blob/main/fwup.conf? They're almost identical, but if you're using a Pi4, I'd start with the first and if you're using another Pi, I'd start with the second one. They both use the info function in their configuration so you can see which partition that you're upgrading.

[sp01109]

I am using fwup -anU -d /dev/mmcblk0 -i myfirmware.fw -t upgrade, which include additional U (no-unmount).

I was thinking since the B partition has been unmounted so it is safe and unnecessary to have fwup unmount it. I have had been using it since fwup version 1.0.0 (or maybe an earlier version). Do you think it will cause any issue with new fwup version 1.8.4?

I will check the new reference you posted here and compare my configuration. I will try it and share my observation with you later.