Debugging IMX8MP SOM boot

[thj]

I have the imx8mp SOM and EVB, the other day I got buildroot-202405212131.img booting on the board the other day and I was successfully able to connect on the network and pull a dmesg off the board.

Returning to board today I'm not able to get the board to boot, I managed twice to get an error from spl, but without any consistency.

I got this error after pulling out the SD card:

Code Select Expand

U-Boot SPL 2020.04 (May 21 2024 - 18:08:43 +0000)
DDRINFO: start DRAM init
DDRINFO: DRAM rate 4000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Normal Boot
Trying to boot from BOOTROM
image offset 0x8000, pagesize 0x200, ivt offset 0x0
ROMAPI: download failure offset 0x60000 size 0x200
SPL: failed to boot from all boot devices
### ERROR ### Please RESET the board ###

The boot switches on delivery were set to NAND 8-BIT 512 Pages (1010).
I have tried setting them to SD card (1100) and USB Serial Download (1000) with a XHCI debug cable attached and 4 different SD cards, but none of these options let me boot.

I have a tigard ftdi device attached to A53_DBG1 at 115200 which I could use for terminal access before.

I'm not sure what to try get. I have verified the voltages are correct on the test points on the SOM. When in NAND boot the sys_status led illuminates which it did when I could boot successfully.

I can add a jtag header, but I'd prefer other steps before soldering to the evb.

[LubOlimex]

Yes, the default mode is ON, OFF, ON, OFF this is booting from SD card. I will investigate the table at the back I suspect it is not good.

[thj]

The table on the back seems matches with my reading of the datasheet.

The datasheet says that if all boot modes fail I should end up with the board in usb serial mode, I don't think my XHCI debug cable should work (https://www.datapro.net/products/usb-3-0-super-speed-a-a-debugging-cable.html) and I have to make one to test this mode.

I'll play with the switches and the image. It would be helpful if the ROM output anything on serial when it can't boot, but I don't read anything like that in the soc datasheet

[LubOlimex]

Yeah I just checked "Table 6-1. Boot MODE pin settings" (from chapter "Chapter 6 SNVS, Reset, Fuse, and Boot") of the "IMX8MPRM.pdf" and it seems the same as the back of the board.

Why the board boots off SD card when set to 1-0-1-0, probably some fallback if selected mode fails.

[LubOlimex]

It seems that from the 9 boot options available setting the board in only 3 would make the SD card boot fail. In the other 6 the board boot would fallback to SD card.

The three modes that would lead to board not booting are #1, #2, #8 - 0000, 1000, 1110.

So if the bootloader is not happy with the way the NAND is configured it would boot from card.

[thj]

I think it was falling back to manufacturer mode:

Code Select Expand

6.1.9 SD/MMC manufacture mode
When the internal boot and recover boot (if enabled) failed, the boot goes to the
SD/MMC manufacture mode before the serial download mode. In the manufacture mode,
one bit bus width is used despite of the fuse setting.
By default, the SD/MMC manufacture mode is enabled. Blow the fuse of the
RECOVERY_SDMMC_BOOT_DIS to disable it.
I think that explains what was happening with the default switches.

I have added a jtag header and probed the tcard slot with a flex cable adapter and I can see the sd card get clock after a reset in the valid boot modes, but not in an invalid boot mode.

Even with that I'm still not getting any boot. I will try to get jtag connecting next. Because I can see a clock for the sd card I'm more confident the module isn't dead.

[thj]

Probing the SD card with an extension with a scope I see a short clock pulse on reset, but it stops quickly.

I have connected a jtag header, but I don't have 0402 10k resistors to add pull ups. Openocd with a tigard board detects a non-nxp device:


<code>
Info : JTAG tap: auto0.tap tap/device found: 0x1cf80553 (mfg: 0x2a9 (Telegent Systems USA Inc), part: 0xcf80, ver: 0x1)
</code>

I put together a usb cable with 5V removed, but D+ D- and GND connected through, but with the EVB set to usb serial boot I don't see any device enumerating. This was a guess for what the cable should be so it might be entirely wrong.

I'm a bit limited by tools I have a available and ideas for what to try that will make some progress figuring out why the som and evb aren't working together.

[LubOlimex]

What is the error that you get when booting from the SD card with the buildroot? What do you get over the serial?

[thj]

I don't get anything on serial when using the the buildroot image.

I have successfully booted the image, but can no longer.

I have tried:
- 5 different sd cards
- two different usb serial devices

[LubOlimex]

How do you download to the SD card? Did you try to re-download the archive with the image?

Can it be related to powering or serial debug cable?

[LubOlimex]

I have no idea how you got any output without SD card. I can't get any output on the serial if I don't have SD card. Also the boot with card starts very different from what you got:

Code Select Expand

U-Boot SPL 2020.04 (Jul 05 2024 - 10:18:39 +0300)
DDRINFO: start DRAM init
DDRINFO: DRAM rate 4000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Normal Boot
Trying to boot from BOOTROM
image offset 0x8000, pagesize 0x200, ivt offset 0x0
NOTICE:  BL31: v2.4(release):2022.08-462-gcc43a0c3c5-dirty
NOTICE:  BL31: Built : 10:18:26, Jul  5 2024


U-Boot 2020.04 (Jul 05 2024 - 10:18:39 +0300)


[thj]

I have been creating the sd card like so:

<code>
$ curl -O http://images.olimex.com/release/imx8mp/buildroot-202405212131.img.gz
$ sha256sum buildroot-202405212131.img.gz
9422671397373070cecfc946677fa56b60b5af991a07cb7de7d16ecb29da1bc5  buildroot-202405212131.img.gz
$ gunzip buildroot-202405212131.img.gz
$ sha256sum
600d77b9c36b9769ae22e9f880644cba195862afb029dd3307c348c9d46386ab  buildroot-202405212131.img
$ file buildroot-202405212131.img
buildroot-202405212131.img: DOS/MBR boot sector; partition 1 : ID=0xc, active, start-CHS (0x1,5,5), end-CHS (0x9,45,36), startsector 16384, 131072 sectors; partition 2 : ID=0x83, start-CHS (0x9,45,37), end-CHS (0x18,121,33), startsector 147456, 245760 sectors
$ sudo dd if=buildroot-202405212131.img of=/dev/da1 bs=1m
$ gpart show da1
=>      63  15523777  da1  MBR  (7.4G)
        63     16321       - free -  (8.0M)
     16384    131072    1  fat32lba  [active]  (64M)
    147456    245760    2  linux-data  (120M)
    393216  15130624       - free -  (7.2G)
</code>

I just followed these steps and tested again.

I am using a tigard for serial configured at 1.8v. I verified the tigard serial is functioning by connecting it to a ch340 based usb serial and successfully sending bytes between the two devices. I think my usb serial is working.

When I could boot successfully the start of my log looked like this:

<code>
U-Boot SPL 2020.04 (May 21 2024 - 18:08:43 +0000)
DDRINFO: start DRAM init
DDRINFO: DRAM rate 4000MTS
DDRINFO:ddrphy calibration done
DDRINFO: ddrmix config done
Normal Boot
Trying to boot from BOOTROM
image offset 0x8000, pagesize 0x200, ivt offset 0x0
NOTICE:  BL31: v2.4(release):2022.08-462-g759e1257e7
NOTICE:  BL31: Built : 13:35:27, May 21 2024


U-Boot 2020.04 (May 21 2024 - 18:08:43 +0000)

CPU:   i.MX8MP[8] rev1.1 1600 MHz (running at 1200 MHz)
CPU:   Industrial temperature grade (-40C to 105C) at 44C
Reset cause: POR
Model: NXP i.MX8MPlus LPDDR4 EVK board
DRAM:  4 GiB
tcpc_init: Can't find device id=0x50
setup_typec: tcpc port2 init failed, err=-19
tcpc_init: Can't find device id=0x50
setup_typec: tcpc port1 init failed, err=-19
MMC:   FSL_SDHC: 1, FSL_SDHC: 2
Loading Environment from MMC... *** Warning - bad CRC, using default environment
</code>

It looks like the uboot I have is different to the one you have. If I look in the image I downloaded just now it is also from May 21 2024.

<code>
$ strings buildroot-202405212131.img| grep "U-Boot SPL"
U-Boot SPL 2020.04 (May 21 2024 - 18:08:43 +0000)
</code>

When I could boot I would get errors on some boots leading to panics. I am wondering if maybe the connections in the sd card slot are bad in some way.

[LubOlimex]

Yes, we've rebuild the image recently so it says July, the buildroot from the web-site should say May. This is not a problem.

Umm, I am not sure if the dd command you use is alright. I feel like you write to the second partition of the card by writing to da1, leaving some problematic partition on the card. Specifying the block size is not needed either. I have two suggestions:

1. Can you try preparing the card with:

sudo dd if=buildroot-202405212131.img of=/dev/da

If that still shows a problem, try:

2. Downloading the image to the card with free version of BalenaEtcher instead? This would exclude problems related to the preparation of the card.

Let me know how it goes.

[thj]

Umm, I am not sure if the dd command you use is alright. I feel like you write to the second partition of the card by writing to da1, leaving some problematic partition on the card. Specifying the block size is not needed either. I have two suggestions:

Sorry for the confusion, this is how FreeBSD names devices. 'da' is a disk and the number is the index on the system. So that is me writing to the second card reader on my laptop (an external one rather than the framework module).

1. Can you try preparing the card with:

sudo dd if=buildroot-202405212131.img of=/dev/da

If that still shows a problem, try:

2. Downloading the image to the card with free version of BalenaEtcher instead? This would exclude problems related to the preparation of the card.

Let me know how it goes.

I tried flashing both ways with 2 different SD cards.

I also put a scope on the tx line from the board a long with the usb serial to be sure the usb serial wasn't at fault, but other than the line pulling when pressing RST1 I see nothing on the tx line.

Here is a picture showing the tx line on A53_DBG1 after pressing RST1



Here is a capture showing the clock line on the SD card after pressing RST1 with a trigger on RST



I could only probe the clock line on the sd card, but there is some activity from the soc, even when there is no boot

[LubOlimex]

The setup looks fine. Maybe there is some bigger problem here. It might be better for us to have a look at those two boards here. Drop me an e-mail at support@olimex.com and link me this thread (so we don't waste time identifying the problem).