Can any one read kernel panics? (Solved R17)

[dingbatca]

Kernel panic!  What does it mean?

Code Select Expand


[  495.930000] Internal error: Oops - undefined instruction: 0 [#1] ARM
[  495.930000] Modules linked in: smsc75xx usbnet mii
[  495.930000] CPU: 0    Not tainted  (3.7.0-rc2-next-20121022 #4)
[  495.930000] PC is at tcp_ack+0x338/0xf2c
[  495.930000] LR is at tcp_rcv_established+0x364/0x5d8
[  495.930000] pc : [<c02ff210>]    lr : [<c0300514>]    psr: 20000013
[  495.930000] sp : c283fb50  ip : 00004c89  fp : 00000013
[  495.930000] r10: c04bb620  r9 : 00000000  r8 : 00000754
[  495.930000] r7 : c3b26c80  r6 : c3b26c98  r5 : 00000402  r4 : c3b84de0
[  495.930000] r3 : 00000000  r2 : 00126966  r1 : 00000013  r0 : 00000000
[  495.930000] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment user
[  495.930000] Control: 0005317f  Table: 43bcc000  DAC: 00000015
[  495.930000] Process iperf (pid: 1042, stack limit = 0xc283e1b8)
[  495.930000] Stack: (0xc283fb50 to 0xc2840000)
[  495.930000] fb40:                                     00000001 c00085a4 0000007e 00000014
[  495.930000] fb60: 93a16811 00000001 93a15cc1 c283fc28 00000000 00000402 00000013 00000000
[  495.930000] fb80: c02f9b98 60000013 ffffffff c283fbcc 00000000 00000013 00000000 00000000
[  495.930000] fba0: 00000000 894c0000 c3b84de0 c28c004e c3b26c80 c3b26c98 00000020 c3b84de0
[  495.930000] fbc0: c28c004e c3b26c80 c3b26c98 00000020 c04b6738 00000000 00000001 c0300514
[  495.930000] fbe0: c030a408 00000020 ffffffff c283fc2c c28c003a c3b26c80 c3b84de0 c28e4b80
[  495.930000] fc00: c3b84de0 c28c003a 00000000 c03079b8 c28c004e c3b26c98 c28c003a c04b6738
[  495.930000] fc20: 00000000 00000001 00000020 c3b26c80 c04d90b4 c28c004e c3b84de0 c030a908
[  495.930000] fc40: c28c003a c3b26c98 c3b26c80 c04b6738 c3b26c80 c3b26c80 c04b6c20 00000006
[  495.930000] fc60: c3b26c98 c3b26c80 c04b6738 00000000 00000001 c02e9718 c02e9698 c3b26c80
[  495.930000] fc80: c04d90b4 c28c003a c3b26c98 c02e9adc 0000007e 00000001 00000001 c04b6720
[  495.930000] fca0: c04b6744 00000008 c39e2800 c3b26c80 00000000 c02c8d10 93a0b311 c386c180
[  495.930000] fcc0: c283e000 00400000 c283fd4c 00000006 c04dd7c8 c3b26c80 c3b26500 c04b6744
[  495.930000] fce0: c3b26460 c04dd800 00000002 c04dd7c0 00000000 0000000a c04dd7f4 c04dd7d4
[  495.930000] fd00: 00200200 c02c9110 c02c9084 c04dd800 00000001 00000040 0000012c 00000003
[  495.930000] fd20: c04bb620 c04dd7c0 c04dd7c8 c02c94a8 00000005 00004c8b 00000000 00000001
[  495.930000] fd40: c0500f6c c283e000 00020000 00000003 00000101 0000000c c0500f60 c002466c
[  495.930000] fd60: 00000000 c02c24c4 00000008 00400000 33323130 60000013 c3b84de0 000002f0
[  495.930000] fd80: 00020000 00000000 014df0f0 00000000 00000000 c0024854 c283e000 c00249ac
[  495.930000] fda0: 00000000 c3850840 c3b84de0 c02f6630 c04b6c20 00000006 c3b1bbf8 00020000
[  495.930000] fdc0: c04b6738 00000000 00000001 c02e9718 00000000 000005a8 c3b84e64 00000000
[  495.930000] fde0: c283e000 00000000 00000000 00000000 c283ff40 00000001 00000000 00000000
[  495.930000] fe00: c39e2800 c3b1bbe0 7fffffff 0000fe88 0000007e c283feb8 c283feb8 00020000
[  495.930000] fe20: c286d580 c283ff88 c283e000 c3bd7440 014e4990 c03182f0 c3b1b640 c04dd800
[  495.930000] fe40: 00000005 c04dd7c0 c283ff38 c02b8428 c04dd7f4 00000001 00200200 c02c9110
[  495.930000] fe60: c02c9084 00020000 c358e580 00000040 00000000 c283fe78 00000000 00000000
[  495.930000] fe80: c283ff38 00000001 00000000 00000000 00000000 c283feb8 c0500f6c c283feb8
[  495.930000] fea0: c286d580 c283ff38 fffffdee c00c73e4 00000000 00000000 00000000 00000000
[  495.930000] fec0: 00000000 00000001 ffffffff c286d580 00000000 00000000 00000000 00000000
[  495.930000] fee0: c3bd7440 00000000 00000000 00000000 00000000 00000000 c283fe58 00000000
[  495.930000] ff00: 00020000 00000000 00020000 00000000 00000000 00000000 00000000 00000000
[  495.930000] ff20: 00000000 00000000 00000000 00000000 00000000 00000000 014bf0f0 00020000
[  495.930000] ff40: 00020000 c286d580 00020000 014bf0f0 c283ff88 c283e000 00000000 c00c79d0
[  495.930000] ff60: c283ff90 014e4990 014e4980 00000000 00000000 c286d580 00000000 00020000
[  495.930000] ff80: 00000000 c00c7c0c 00000000 00000000 014bf0d0 014bf0f0 014e4980 00000004
[  495.930000] ffa0: c000eca8 c000eb40 014bf0d0 014bf0f0 00000003 014bf0f0 00020000 014bf008
[  495.930000] ffc0: 014bf0d0 014bf0f0 014e4980 00000004 0001d210 00000001 00020000 014e4990
[  495.930000] ffe0: 00000000 bea67424 0000a714 b6e38d2c 40000010 00000003 00000000 00000000
[  495.930000] [<c02ff210>] (tcp_ack+0x338/0xf2c) from [<c0300514>] (tcp_rcv_established+0x364/0x5d8)
[  495.930000] [<c0300514>] (tcp_rcv_established+0x364/0x5d8) from [<c03079b8>] (tcp_v4_do_rcv+0xe0/0x238)
[  495.930000] [<c03079b8>] (tcp_v4_do_rcv+0xe0/0x238) from [<c030a908>] (tcp_v4_rcv+0x61c/0x734)
[  495.930000] [<c030a908>] (tcp_v4_rcv+0x61c/0x734) from [<c02e9718>] (ip_local_deliver+0x80/0x150)
[  495.930000] [<c02e9718>] (ip_local_deliver+0x80/0x150) from [<c02e9adc>] (ip_rcv+0x2f4/0x4f8)
[  495.930000] [<c02e9adc>] (ip_rcv+0x2f4/0x4f8) from [<c02c8d10>] (__netif_receive_skb+0x2a4/0x618)
[  495.930000] [<c02c8d10>] (__netif_receive_skb+0x2a4/0x618) from [<c02c9110>] (process_backlog+0x8c/0x144)
[  495.930000] [<c02c9110>] (process_backlog+0x8c/0x144) from [<c02c94a8>] (net_rx_action+0xb4/0x1ec)
[  495.930000] [<c02c94a8>] (net_rx_action+0xb4/0x1ec) from [<c002466c>] (__do_softirq+0xb0/0x1e8)
[  495.930000] [<c002466c>] (__do_softirq+0xb0/0x1e8) from [<c0024854>] (do_softirq+0x4c/0x58)
[  495.930000] [<c0024854>] (do_softirq+0x4c/0x58) from [<c00249ac>] (local_bh_enable+0x9c/0xb0)
[  495.930000] [<c00249ac>] (local_bh_enable+0x9c/0xb0) from [<c02f6630>] (tcp_sendmsg+0x60/0xd2c)
[  495.930000] [<c02f6630>] (tcp_sendmsg+0x60/0xd2c) from [<c03182f0>] (inet_sendmsg+0x3c/0x7c)
[  495.930000] [<c03182f0>] (inet_sendmsg+0x3c/0x7c) from [<c02b8428>] (sock_aio_write+0xf0/0xfc)
[  495.930000] [<c02b8428>] (sock_aio_write+0xf0/0xfc) from [<c00c73e4>] (do_sync_write+0x9c/0xd0)
[  495.930000] [<c00c73e4>] (do_sync_write+0x9c/0xd0) from [<c00c79d0>] (vfs_write+0x17c/0x184)
[  495.930000] [<c00c79d0>] (vfs_write+0x17c/0x184) from [<c00c7c0c>] (sys_write+0x44/0x70)
[  495.930000] [<c00c7c0c>] (sys_write+0x44/0x70) from [<c000eb40>] (ret_fast_syscall+0x0/0x2c)
[  495.930000] Code: e1a05009 e58d301c e1a0a009 ea000042 (e3180002)
[  495.930000] ---[ end trace a1e2c3593d08947c ]---
[  495.930000] Kernel panic - not syncing: Fatal exception in interrupt


[dpwhittaker]

Is this an olinuxino micro?  Have you seen:

https://www.olimex.com/forum/index.php?topic=59.0

If that looks like your issue, try removing R17.

[dingbatca]

That looks like it!   I will pull r17 tonight and see if I can crash it.  Thanks dpwhittaker!!!!

[BJFreeman]

it is saying that the driver smsc75xx usbnet mii
it says that the driver is missing a link to code.

does the board use this driver?

Note my board arrived today so just getting into the guts

[dpwhittaker]

I thought that meant there are 3 modules that DID get linked in, smsc75xx, usbnet, and mii.  The point being that any of these modules could be responsible for the issue (plus anything statically linked or the kernel itself).  The actual error is:

undefined instruction: 0

Which is caused either by compiling the kernel with the wrong cross-compiler, or a bad read from RAM, which has been shown to be caused by R17.

[BJFreeman]

did not think was cross compilers since the PC is already executing code.
PC is at tcp_ack+0x338/0xf2c

so yes either the memory or a link to the kernel did not get satisfied at linking process.

[dingbatca]

OK, more info on my setup.
I have the Micro board, and have been unable to get any distro to run stable.

I have 10+ different wireless/wired USB network cards.  In this case I was using my 1Gbit/s USB NIC, running the smsc75xx module. I can cause a kernel panic on any distro, kernel rev, bootlets, or network card. Which leads me to think the R17 is the only common factor.

Currently I am running Gentoo with a 3.7-rc2 kernel.  I have clean source from kernel.org.  No extra patches have been applied.

I will remove R17 tonight and run a few tests.  I will report back soon.

[BJFreeman]

it would seem if you got booted and operational, then you have this error.
that R17 should have effected more memory accesses than just this one.

be interesting to see you results.

BTW my board is a maxi so no way to validate.

[dingbatca]

In both Arch, and Gentoo (3.7-rc2) my Micro board crashed.
My two tests are:
iperf -t 3600
kernel compile: 3.7-rc2 from kernel.org. Local swap on SD.

Removed R17 from the bottom of my micro and both tests passed.  Looks like Olimex need to update this board, and put a sticky post at the top of the forum.  Also a warning on the product page would help.