cannot boot from emmc [SOLVED]

aleix · July 01, 2024, 10:10:34 PM

Hello,

Board: T2-OLinuXino-LIME2-e8Gs16M-IND Rev.L

Installed OS: official image from https://images.olimex.com/release/a20/

Once it is installed through a microSD card, this command is used:

Code Select

olinuxino-sd-to-emmc

Everything works fine and the OS is booted from emmc.

Then, a few packages are installed (after apt update & upgrade), a firewall is set and root account is disabled (passwd -l root and nologin from /etc/passwd).

Then, when rebooting, cannot boot anymore, and the same behaviour is tested on three identical boards.

This is the the output:

Code Select

U-Boot SPL 2021.04+olimex-1-20231103.141125 (Nov 03 2023 - 14:12:28 +0000)
DRAM: 1024 MiB
CPU: 912000000Hz, AXI/AHB/APB: 3/2/2
Loading configuration from EEPROM: OK
Verifying data: OK
Trying to boot from MMC2


U-Boot 2021.04+olimex-1-20231103.141125 (Nov 03 2023 - 14:12:28 +0000) Allwinner Technology

CPU:   Allwinner A20 (SUN7I)
ID:    T2-OLinuXino-LIME2-e8Gs16M-IND Rev.L
SN:    00025829
MAC:   30:1F:9A:D1:36:BE
I2C:   ready
DRAM:  1 GiB
SF: Detected w25q128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB
MMC:   mmc@1c0f000: 0, mmc@1c11000: 1
Loading Environment from EXT4... *** Warning - bad CRC, using default environment

Loading Environment from FAT... ** No device specified **
In:    serial
Out:   serial
Err:   serial
Allwinner mUSB OTG (Peripheral)
Net:   eth0: ethernet@1c50000, eth1: usb_ether
starting USB...
Bus usb@1c14000: USB EHCI 1.00
Bus usb@1c14400: USB OHCI 1.0
Bus usb@1c1c000: USB EHCI 1.00
Bus usb@1c1c400: USB OHCI 1.0
scanning bus usb@1c14000 for devices... 1 USB Device(s) found   
scanning bus usb@1c14400 for devices... 1 USB Device(s) found   
scanning bus usb@1c1c000 for devices... 1 USB Device(s) found   
scanning bus usb@1c1c400 for devices... 1 USB Device(s) found   
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc1(part 0) is current device
Scanning mmc 1:1...
Found U-Boot script /boot/boot.scr
2274 bytes read in 2 ms (1.1 MiB/s)
## Executing script at 43100000
Boot script loaded from mmc
Checking for /uEnv.txt...
Checking for /boot/uEnv.txt...
1156 bytes read in 1 ms (1.1 MiB/s)
Loaded environment from /boot/uEnv.txt
Loading FIT image...
29502207 bytes read in 1611 ms (17.5 MiB/s)
## Loading kernel from FIT Image at 58000000 ...
   Using 'config-9243' configuration
   Trying 'kernel-1' kernel subimage
     Description:  Linux kernel 5.10.0-30-rt-armmp
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x580000d4
     Data Size:    5050880 Bytes = 4.8 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x40080000
     Entry Point:  0x40080000
     Hash algo:    crc32
     Hash value:   65d19ea1
     Hash algo:    sha1
     Hash value:   a27f53b86204312d87ebfc878360ffdd422ff938
   Verifying Hash Integrity ... crc32+ sha1+ OK
## Loading ramdisk from FIT Image at 58000000 ...
   Using 'config-9243' configuration
   Trying 'ramdisk-1' ramdisk subimage
     Description:  Ramdisk for kernel 5.10.0-30-rt-armmp
     Type:         RAMDisk Image
     Compression:  Unknown Compression
     Data Start:   0x584d140c
     Data Size:    24164998 Bytes = 23 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x4fe00000
     Entry Point:  0x4fe00000
     Hash algo:    crc32
     Hash value:   219c745e
     Hash algo:    sha1
     Hash value:   cc65dc7beda697ff7c2bc9111557f32bff2819fe
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading ramdisk from 0x584d140c to 0x4fe00000
WARNING: 'compression' nodes for ramdisks are deprecated, please fix your .its file!
## Loading fdt from FIT Image at 58000000 ...
   Using 'config-9243' configuration
   Trying 'fdt-4' fdt subimage
     Description:  unavailable
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x59befbc8
     Data Size:    26587 Bytes = 26 KiB
     Architecture: ARM
     Load Address: 0x4fa00000
     Hash algo:    crc32
     Hash value:   f5ec9034
     Hash algo:    sha1
     Hash value:   4db0e46ea10ec1b5b306e7881dc999f4003045c9
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x59befbc8 to 0x4fa00000
## Loading fdt from FIT Image at 58000000 ...
   Trying 'overlay-1' fdt subimage
     Description:  unavailable
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x59c1ee24
     Data Size:    1001 Bytes = 1001 Bytes
     Architecture: ARM
     Load Address: 0x4fa10000
     Hash algo:    crc32
     Hash value:   d1d51f29
     Hash algo:    sha1
     Hash value:   fa0c14efb55dbe4bdf8aa7241a3e600b486e3ef6
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x59c1ee24 to 0x4fa10000
failed on fdt_overlay_apply(): FDT_ERR_NOTFOUND
base fdt does did not have a /__symbols__ node
make sure you've compiled with -@
Could not find a valid device tree
SCRIPT FAILED: continuing...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Card did not respond to voltage select! : -110
Scanning disk mmc@1c0f000.blk...
Disk mmc@1c0f000.blk not ready
Scanning disk mmc@1c11000.blk...
Found 2 disks
No EFI system partition
Applying overlay: '/usr/lib/olinuxino-overlays/sun7i-a20/spi1-spidev.dtbo'...
408 bytes read in 5 ms (79.1 KiB/s)
No FDT memory address configured. Please configure
the FDT address via "fdt addr <address>" command.
Aborting!
Failed to apply overlay.
Restoring the original FDT blob...
BootOrder not defined
EFI boot manager: Cannot load any image
Card did not respond to voltage select! : -110
scanning bus for devices...
SATA link 0 timeout.
AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
flags: ncq stag pm led clo only pmp pio slum part ccc apst

Device 0: unknown device

Device 0: unknown device
ethernet@1c50000 Waiting for PHY auto negotiation to complete......... TIMEOUT !
Could not initialize PHY ethernet@1c50000
using musb-hdrc, OUT ep1out IN ep1in STATUS ep2in
MAC de:ad:be:ef:00:01
HOST MAC de:ad:be:ef:00:00
RNDIS ready
The remote end did not respond in time.missing environment variable: pxeuuid
missing environment variable: bootfile
Retrieving file: pxelinux.cfg/01-30-1f-9a-d1-36-be
ethernet@1c50000 Waiting for PHY auto negotiation to complete...

Code Select

=> mmcinfo         
Device: mmc@1c11000
Manufacturer ID: 13
OEM: 14e
Name: Q2J55 
Bus Speed: 52000000
Mode: MMC High Speed (52MHz)
Rd Block Len: 512
MMC version 5.0
High Capacity: Yes
Capacity: 7.1 GiB
Bus Width: 4-bit
Erase Group Size: 512 KiB
HC WP Group Size: 8 MiB
User Capacity: 7.1 GiB WRREL
Boot Capacity: 16 MiB ENH
RPMB Capacity: 4 MiB ENH
Boot area 0 is not write protected
Boot area 1 is not write protected

Would appreciate any help.

Thanks

aleix · July 01, 2024, 11:15:53 PM

Narrowed down the step at which it stops booting. After apt update & upgrade it boots fine. Then this is installed:

Code Select

# apt install curl ufw wireguard-tools sox alsa-utils opus-tools rsync

And after installing this, and rebooting, the problem appears.

The first line of discrepancy before and after the problem is

Code Select

failed on fdt_overlay_apply(): FDT_ERR_NOTFOUND

Also tried installing the same packages without apt upgrade, and same problem. These packages lead to this:

Code Select

Selecting previously unselected package linux-image-5.10.0-30-rt-armmp.
Preparing to unpack .../08-linux-image-5.10.0-30-rt-armmp_5.10.218-1_armhf.deb ...
Unpacking linux-image-5.10.0-30-rt-armmp (5.10.218-1) ...
Selecting previously unselected package linux-image-rt-armmp.
Preparing to unpack .../09-linux-image-rt-armmp_5.10.218-1_armhf.deb ...
Unpacking linux-image-rt-armmp (5.10.218-1) ...

Perhaps this linux image is the culprit?

LubOlimex · July 02, 2024, 08:14:13 AM

You can try something different - boot again from the SD card, run apt update and apt upgrade and install the packages, see if it boots fine from the SD card and THEN run the eMMC install script and check without a card.

aleix · July 02, 2024, 09:06:29 AM

Quote from: LubOlimex on July 02, 2024, 08:14:13 AMYou can try something different - boot again from the SD card, run apt update and apt upgrade and install the packages, see if it boots fine from the SD card and THEN run the eMMC install script and check without a card.

Just tried it. After apt update + upgrade, and installing packages, no problem booting from SD card. Then, after running the eMMC install script, cannot boot, with the same log messages as before. Interestingly, if inserting the previous SD card again, the OS does not boot either, either on that same board or in a new one. The boot log in this case:

Code Select

U-Boot SPL 2021.04+olimex-1-20231103.141125 (Nov 03 2023 - 14:12:28 +0000)
DRAM: 1024 MiB
CPU: 912000000Hz, AXI/AHB/APB: 3/2/2
Loading configuration from EEPROM: OK
Verifying data: OK
Trying to boot from MMC1


U-Boot 2021.04+olimex-1-20231103.141125 (Nov 03 2023 - 14:12:28 +0000) Allwinner Technology

CPU:   Allwinner A20 (SUN7I)
ID:    T2-OLinuXino-LIME2-e8Gs16M-IND Rev.L
SN:    0002582A
MAC:   30:1F:9A:D1:36:BF
I2C:   ready
DRAM:  1 GiB
SF: Detected w25q128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB
MMC:   mmc@1c0f000: 0, mmc@1c11000: 1
Loading Environment from EXT4... *** Warning - bad CRC, using default environment

Loading Environment from FAT... ** No device specified **
In:    serial
Out:   serial
Err:   serial
Allwinner mUSB OTG (Peripheral)
Net:   eth0: ethernet@1c50000, eth1: usb_ether
starting USB...
Bus usb@1c14000: USB EHCI 1.00
Bus usb@1c14400: USB OHCI 1.0
Bus usb@1c1c000: USB EHCI 1.00
Bus usb@1c1c400: USB OHCI 1.0
scanning bus usb@1c14000 for devices... 1 USB Device(s) found
scanning bus usb@1c14400 for devices... 1 USB Device(s) found
scanning bus usb@1c1c000 for devices... 1 USB Device(s) found
scanning bus usb@1c1c400 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0 is current device 
Scanning mmc 0:1...
Found U-Boot script /boot/boot.scr
2274 bytes read in 9 ms (246.1 KiB/s)
## Executing script at 43100000
Boot script loaded from mmc
Checking for /uEnv.txt...
Checking for /boot/uEnv.txt...
1156 bytes read in 4 ms (282.2 KiB/s)
Loaded environment from /boot/uEnv.txt
Loading FIT image...   
29502407 bytes read in 1615 ms (17.4 MiB/s)
## Loading kernel from FIT Image at 58000000 ...
   Using 'config-9243' configuration
   Trying 'kernel-1' kernel subimage
     Description:  Linux kernel 5.10.0-30-rt-armmp
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x580000d4
     Data Size:    5050880 Bytes = 4.8 MiB
     Architecture: ARM 
 OS:           Linux
     Load Address: 0x40080000
     Entry Point:  0x40080000
     Hash algo:    crc32
     Hash value:   65d19ea1
     Hash algo:    sha1
     Hash value:   a27f53b86204312d87ebfc878360ffdd422ff938
   Verifying Hash Integrity ... crc32+ sha1+ OK
## Loading ramdisk from FIT Image at 58000000 ...
   Using 'config-9243' configuration
   Trying 'ramdisk-1' ramdisk subimage
     Description:  Ramdisk for kernel 5.10.0-30-rt-armmp
     Type:         RAMDisk Image
     Compression:  Unknown Compression
     Data Start:   0x584d140c
     Data Size:    24165199 Bytes = 23 MiB
     Architecture: ARM 
     OS:           Linux
     Load Address: 0x4fe00000
     Entry Point:  0x4fe00000
     Hash algo:    crc32
     Hash value:   c32aeb6a
     Hash algo:    sha1
     Hash value:   c16b0d0c10db1902362d33213696b12f22978fe8
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading ramdisk from 0x584d140c to 0x4fe00000
WARNING: 'compression' nodes for ramdisks are deprecated, please fix your .its file!
## Loading fdt from FIT Image at 58000000 ...
   Using 'config-9243' configuration
   Trying 'fdt-4' fdt subimage
     Description:  unavailable
 Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x59befc90
     Data Size:    26587 Bytes = 26 KiB
     Architecture: ARM 
     Load Address: 0x4fa00000
     Hash algo:    crc32
     Hash value:   f5ec9034
     Hash algo:    sha1
     Hash value:   4db0e46ea10ec1b5b306e7881dc999f4003045c9
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x59befc90 to 0x4fa00000
## Loading fdt from FIT Image at 58000000 ...
   Trying 'overlay-1' fdt subimage
     Description:  unavailable
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x59c1eeec
     Data Size:    1001 Bytes = 1001 Bytes
     Architecture: ARM 
     Load Address: 0x4fa10000
     Hash algo:    crc32
     Hash value:   d1d51f29
     Hash algo:    sha1
     Hash value:   fa0c14efb55dbe4bdf8aa7241a3e600b486e3ef6
   Verifying Hash Integrity ... crc32+ sha1+ OK
   Loading fdt from 0x59c1eeec to 0x4fa10000
failed on fdt_overlay_apply(): FDT_ERR_NOTFOUND
base fdt does did not have a /__symbols__ node
make sure you've compiled with -@
Could not find a valid device tree
SCRIPT FAILED: continuing...
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Scanning disk mmc@1c0f000.blk...
Scanning disk mmc@1c11000.blk...
** Unrecognized filesystem type **
Found 3 disks
No EFI system partition
Applying overlay: '/usr/lib/olinuxino-overlays/sun7i-a20/spi1-spidev.dtbo'...
408 bytes read in 9 ms (43.9 KiB/s)
No FDT memory address configured. Please configure
the FDT address via "fdt addr <address>" command.
Aborting!
Failed to apply overlay.
Restoring the original FDT blob...
BootOrder not defined  
EFI boot manager: Cannot load any image
switch to partitions #0, OK
mmc1(part 0) is current device
** No partition table - mmc 1 **
scanning bus for devices...
SATA link 0 timeout.   
AHCI 0001.0100 32 slots 1 ports 3 Gbps 0x1 impl SATA mode
flags: ncq stag pm led clo only pmp pio slum part ccc apst

Device 0: unknown device

Device 0: unknown device
Speed: 100, full duplex
BOOTP broadcast 1
BOOTP broadcast 2
BOOTP broadcast 3
BOOTP broadcast 4
*** Unhandled DHCP Option in OFFER/ACK: 125
*** Unhandled DHCP Option in OFFER/ACK: 125
DHCP client bound to address 192.168.1.149 (3929 ms)
*** Warning: no boot file name; using 'C0A80195.img'
Using ethernet@1c50000 device
TFTP from server 192.168.1.1; our IP address is 192.168.1.149
Filename 'C0A80195.img'.
Load address: 0x42000000
Loading: T T T T T T T

LubOlimex · July 02, 2024, 10:23:57 AM

How big is the SD card? Is it bigger than the eMMC?

Also can you try with minimal image instead of base?

https://www.olimex.com/forum/index.php?topic=9253.0

LubOlimex · July 02, 2024, 02:18:45 PM

I am doing some testing meanwhile trying to replicate the issue. So far:

1) Prepared a card with Etcher with latest base image and then boot from it and transferred it without changes with "olinuixno-sd-to-emmc". Then tested if it works off eMMC. It did.

2) Updated the same card with apt update and apt upgrade and transferred it to eMMC again with "olinuixno-sd-to-emmc". Tested without card - still worked fine.

3) Put in the same card and add "apt install curl ufw wireguard-tools sox alsa-utils opus-tools rsync" wait until these install and type reboot. The board no longer boots. So the issue is in that command line. It is not related to eMMC or SD card boot. Just that line kills the boot process. I don't know which part of that installation is to blame exactly. Maybe try installing each one separately and see after which one the boot process dies.

aleix · July 02, 2024, 04:26:41 PM

Quote from: LubOlimex on July 02, 2024, 10:23:57 AMHow big is the SD card? Is it bigger than the eMMC?

32 GB

QuoteAlso can you try with minimal image instead of base?

will do

QuoteI am doing some testing meanwhile trying to replicate the issue.

big thanks

QuoteMaybe try installing each one separately and see after which one the boot process dies.

will do and report back soon

aleix · July 02, 2024, 05:12:20 PM

Minimal image burned and installed to SD. Then sd to emmc. Then apt update and upgrade. Reboot OK.

rsync was part of the image, so didn't need to install it. Same for alsa-utils.

apt install curl, needs libcurl4, reboot OK.

apt install opus-tools, needs libflac8 libogg0 libopus0, reboot OK.

apt install sox, needs libgsm1 libltdl7 libmagic-mgc libmagic1 libopencore-amrnb0 libopencore-amrwb0 libpng16-16 libsndfile1 libsox-fmt-alsa libsox-fmt-base libsox3 libvorbis0a libvorbisenc2 libvorbisfile3 libwavpack1, reboot OK.

apt install ufw, needs libmpdec3 libpython3-stdlib libpython3.9-minimal libpython3.9-stdlib media-types python3 python3-minimal python3.9 python3.9-minimal, reboot OK

That means wireguard-tools should be the culprit. It says it needs apparmor firmware-linux-free linux-image-5.10.0-30-rt-armmp linux-image-rt-armmp. So, instead of doing apt install wireguard-tools, I am going to try:

Code Select

apt install --no-install-recommends wireguard-tools

And voilà, boot is OK.

So it seems wireguard-tools recommendations break boot.

Thanks for the help.

aleix · July 02, 2024, 05:14:50 PM

Could the subject be changed so that it can be more useful? Something like:

apt install wireguard-tools messes up boot (avoid with the --no-install-recommends flag)

LubOlimex · July 03, 2024, 08:17:36 AM

I can change it. But I think it is better if I just put [SOLVED]. Thank you for debugging it and updating the steps you took. It is public forum anyway and if somebody is searching for the solution would find it anyway. It is probably the image and some of the recommends of the package are incompatible at the moment.