lwIP on PIC32-EMZ64

[jrmymllr]

Has anyone by chance got lwIP working on a PIC32-EMZ64? I'm having a lot of trouble getting the PIC32 MAC to behave even without worrying about the PHY, i.e. MAC registers aren't reflecting what the transmission of a packet. At least I can use the demo Olimex provided to reverse engineer how it's done, but that Harmony code sure is convoluted.

[kyrk.5]

It this that you mean?
https://en.wikipedia.org/wiki/LwIP

I even did not know that such a thing is existing, but thanks for the tip

Maybe Harmony provides not the best code, but it is possible to cut the TCPIP stack from it out and use it as a Module. I done this with USB, since the newer PICs have a different USB and I needed an USB stack, but the old one from (Michrocihp Library something) was not supporting the newer PIC32s. I guess the same is possible with the TCPIP stack, so that you do not have to have all the Harmony stuff only TCPIP. Not sure why you exactly need LwIP, is there any reason why the Microchip Code is not good for you?

I think if you do not find somebody that already did LwIP on this board, you can try to cut out the TCPIP stack from harmony out. (Tehnically it is not so much change, but I think cutting out helps a lot to de convolute the Microchip Code . )So that you have only that C and H files that is necessary to have run the stack. Then you can start analyse what is the difference between your LwIP and Microchip TCPIP stack, and hopefully find out how to solve your problem.

Maybe over christmas I will give a look to this LWiP, looks intresting. An maybe much more compacter than the Harmony code. Can you please provide a link, where you found your LwIP C code? I guess it would make sense if I start with the exact one that you have, so that I can give you feedback if I have success with it.

[jrmymllr]

It this that you mean?
https://en.wikipedia.org/wiki/LwIP

I even did not know that such a thing is existing, but thanks for the tip

Maybe Harmony provides not the best code, but it is possible to cut the TCPIP stack from it out and use it as a Module. I done this with USB, since the newer PICs have a different USB and I needed an USB stack, but the old one from (Michrocihp Library something) was not supporting the newer PIC32s. I guess the same is possible with the TCPIP stack, so that you do not have to have all the Harmony stuff only TCPIP. Not sure why you exactly need LwIP, is there any reason why the Microchip Code is not good for you?

I think if you do not find somebody that already did LwIP on this board, you can try to cut out the TCPIP stack from harmony out. (Tehnically it is not so much change, but I think cutting out helps a lot to de convolute the Microchip Code . )So that you have only that C and H files that is necessary to have run the stack. Then you can start analyse what is the difference between your LwIP and Microchip TCPIP stack, and hopefully find out how to solve your problem.

Maybe over christmas I will give a look to this LWiP, looks intresting. An maybe much more compacter than the Harmony code. Can you please provide a link, where you found your LwIP C code? I guess it would make sense if I start with the exact one that you have, so that I can give you feedback if I have success with it.

Oh wow, someone actually replied! I wish I had checked earlier.

I have a lot of time invested in lwIP and want to move a project I did on an ARM over to PIC32MZ, which uses lwIP. lwIP is supposed to be very good and it seems reliable. I haven't looked much into the MCHP stack but Harmony is such a turn-off. It's so convoluted and such a big learning curve.

The best example I've found is here, near the bottom: http://lwip.wikia.com/wiki/Available_device_drivers

The only problem is it's for PIC32MX. Now, the MX and MZ MAC appear to be identical, but even getting the status registers to indicate something has transmitted is a problem. I have a clock going into EREFCLK and the config bits set correctly. Must be something obvious, but haven't found it yet.

I'll check back more often now that I'm working on this again.

[kyrk.5]

Hi,

I think I can do a little bit debugging. Recently I tried out the Ethernet demo for the EMZ64 board, so it will be no problem to analyse for me how Microchip is doing the low level stuff. And you have luck because I added also some ethernet cable to my desk for test purpose. Until now I hade to run over to the modem and connect the board.

What I do not have is the Ethernet starter kit for MX. And I think I do not have any MX device with ethernet. So a running lwIP I can not analyse, but I think this will not be a problem. I think it will be enough for me if I see the code and I will try to port it to the EMZ hardware. I guess you used this as starting:
https://github.com/banneliv/lwip_port_pic32

[jrmymllr]

Hi,

I think I can do a little bit debugging. Recently I tried out the Ethernet demo for the EMZ64 board, so it will be no problem to analyse for me how Microchip is doing the low level stuff. And you have luck because I added also some ethernet cable to my desk for test purpose. Until now I hade to run over to the modem and connect the board.

What I do not have is the Ethernet starter kit for MX. And I think I do not have any MX device with ethernet. So a running lwIP I can not analyse, but I think this will not be a problem. I think it will be enough for me if I see the code and I will try to port it to the EMZ hardware. I guess you used this as starting:
https://github.com/banneliv/lwip_port_pic32

Yes, I used the link you provided. However I can go one step further. I didn't have any luck with the code from the link despite changing all registers to the correct ones for MZ. So my goal was then to be able to send ethernet frames in a predicable manner, and even this is causing problems.

See the code below. This is for PIC32MZ2048EFG064, which is, for these purposes, essentially identical (I believe). I do have PIC32-EMZ64 but for now I am using the other chip by itself on essentially a breadboard, so I can probe MAC connections easily. The code below is supposed to transmit three frames from the MAC configured as RMII. If lines 172-174 are uncommented, it transmits three frames according to TX_EN, but TXD0 and TXD1 contains data only in the first frame. If those lines are instead commented, nothing is transmitted. This doesn't makes sense to me as I thought the MAC is blindly sending whatever is thrown at it.

If the length of the data buffer is changed by modifying the values in lines 162-164, the length of the TX_EN pulse changes as one would expect. For example, if the value is changed from 0xc4000180 to 0xc2000180 for one of the three descriptors, the TX_EN pulse is half the width of the others. This is good. But again the TXD lines are mostly silent!

Try this code out as it's a good way to rapidly get up to the point I'm at. It does *something*, just not what I'd expect. Thanks for your interest.


(compiled on xc32 1.44)

Code Select Expand

#include <xc.h>
#include <sys/kmem.h>

#pragma config FMIIEN = OFF // Ethernet RMII/MII Enable (MII Enabled)
#pragma config FETHIO = ON // Ethernet I/O Pin Select (Default Ethernet I/O)
#pragma config PGL1WAY = OFF // Permission Group Lock One Way Configuration (Allow multiple reconfigurations)
#pragma config PMDL1WAY = OFF // Peripheral Module Disable Configuration (Allow multiple reconfigurations)
#pragma config IOL1WAY = OFF // Peripheral Pin Select Configuration (Allow multiple reconfigurations)
#pragma config FUSBIDIO = ON // USB USBID Selection (Controlled by the USB Module)

// DEVCFG2
#pragma config FPLLIDIV = DIV_1 // System PLL Input Divider (1x Divider)
#pragma config FPLLRNG = RANGE_5_10_MHZ // System PLL Input Range (5-10 MHz Input)
#pragma config FPLLICLK = PLL_FRC // System PLL Input Clock Selection (FRC is input to the System PLL)
#pragma config FPLLMULT = MUL_50 // System PLL Multiplier (PLL Multiply by 44)
#pragma config FPLLODIV = DIV_8 // System PLL Output Clock Divider (16x Divider)
#pragma config UPLLFSEL = FREQ_24MHZ // USB PLL Input Frequency Selection (USB PLL input is 24 MHz)

// DEVCFG1
#pragma config FNOSC = SPLL // Oscillator Selection Bits (System PLL)
#pragma config DMTINTV = WIN_127_128 // DMT Count Window Interval (Window/Interval value is 127/128 counter value)
#pragma config FSOSCEN = OFF // Secondary Oscillator Enable (Disable SOSC)
#pragma config IESO = OFF // Internal/External Switch Over (Disabled)
#pragma config POSCMOD = OFF // Primary Oscillator Configuration (Primary osc disabled)
#pragma config OSCIOFNC = OFF // CLKO Output Signal Active on the OSCO Pin (Disabled)
#pragma config FCKSM = CSECME // Clock Switching and Monitor Selection (Clock Switch Enabled, FSCM Enabled)
#pragma config WDTPS = PS1048576 // Watchdog Timer Postscaler (1:1048576)
#pragma config WDTSPGM = STOP // Watchdog Timer Stop During Flash Programming (WDT stops during Flash programming)
#pragma config WINDIS = NORMAL // Watchdog Timer Window Mode (Watchdog Timer is in non-Window mode)
#pragma config FWDTEN = OFF // Watchdog Timer Enable (WDT Disabled)
#pragma config FWDTWINSZ = WINSZ_25 // Watchdog Timer Window Size (Window size is 25%)
#pragma config DMTCNT = DMT31 // Deadman Timer Count Selection (2^31 (2147483648))
#pragma config FDMTEN = ON // Deadman Timer Enable (Deadman Timer is enabled)

// DEVCFG0
#pragma config DEBUG = OFF // Background Debugger Enable (Debugger is disabled)
#pragma config JTAGEN = OFF // JTAG Enable (JTAG Disabled)
#pragma config ICESEL = ICS_PGx1 // ICE/ICD Comm Channel Select (Communicate on PGEC1/PGED1)
#pragma config TRCEN = ON // Trace Enable (Trace features in the CPU are enabled)
#pragma config BOOTISA = MIPS32 // Boot ISA Selection (Boot code and Exception code is MIPS32)
#pragma config FECCCON = OFF_UNLOCKED // Dynamic Flash ECC Configuration (ECC and Dynamic ECC are disabled (ECCCON bits are writable))
#pragma config FSLEEP = OFF // Flash Sleep Mode (Flash is powered down when the device is in Sleep mode)
#pragma config DBGPER = PG_ALL // Debug Mode CPU Access Permission (Allow CPU access to all permission regions)
#pragma config SMCLR = MCLR_NORM // Soft Master Clear Enable bit (MCLR pin generates a normal system Reset)
#pragma config SOSCGAIN = GAIN_2X // Secondary Oscillator Gain Control bits (2x gain setting)
#pragma config SOSCBOOST = ON // Secondary Oscillator Boost Kick Start Enable bit (Boost the kick start of the oscillator)
#pragma config POSCGAIN = GAIN_2X // Primary Oscillator Gain Control bits (2x gain setting)
#pragma config POSCBOOST = ON // Primary Oscillator Boost Kick Start Enable bit (Boost the kick start of the oscillator)
#pragma config EJTAGBEN = NORMAL // EJTAG Boot (Normal EJTAG functionality)

// DEVCP0
#pragma config CP = OFF // Code Protect (Protection Disabled)


#define PIC32MZ_ETH_TX_BUFFER_COUNT 3
#define PIC32MZ_ETH_RX_BUFFER_SIZE 1536
#define PIC32MZ_ETH_RX_BUFFER_COUNT 6
#define PIC32MZ_ETH_TX_BUFFER_SIZE 1536

typedef unsigned int uint32_t;
typedef char char_t;
typedef signed int int_t;
typedef unsigned int uint_t;
typedef unsigned char uint8_t;

typedef struct
{
   uint32_t control;
   uint32_t address;
   uint32_t status1;
   uint32_t status2;
   uint32_t next;
} Pic32mzTxBufferDesc;

typedef struct
{
   uint32_t control;
   uint32_t address;
   uint32_t status1;
   uint32_t status2;
   uint32_t next;
} Pic32mzRxBufferDesc;

//Transmit buffer
static uint8_t txBuffer[PIC32MZ_ETH_TX_BUFFER_COUNT][PIC32MZ_ETH_TX_BUFFER_SIZE]
   __attribute__((aligned(4)));
//Receive buffer
static uint8_t rxBuffer[PIC32MZ_ETH_RX_BUFFER_COUNT][PIC32MZ_ETH_RX_BUFFER_SIZE]
   __attribute__((aligned(4)));
//Transmit buffer descriptors
static Pic32mzTxBufferDesc txBufferDesc[PIC32MZ_ETH_TX_BUFFER_COUNT]
   __attribute__((aligned(4)));
//Receive buffer descriptors
static Pic32mzRxBufferDesc rxBufferDesc[PIC32MZ_ETH_RX_BUFFER_COUNT]
   __attribute__((aligned(4)));


int main(void)
{
   __builtin_disable_interrupts();

   _CP0_BIS_CAUSE(_CP0_CAUSE_IV_MASK);

   INTCONSET = _INTCON_MVEC_MASK;
   
ANSELB = 0;
ANSELE = 0;
ANSELG = 0;
   
    EMAC1CFG2bits.FULLDPLX = 1;
   //Disable Ethernet interrupts
   IEC4CLR = _IEC4_ETHIE_MASK;
   //Turn the Ethernet controller off
   ETHCON1CLR = _ETHCON1_ON_MASK | _ETHCON1_TXRTS_POSITION | _ETHCON1_RXEN_MASK;

   //Wait activity abort by polling the ETHBUSY bit
   while(ETHSTAT & _ETHSTAT_ETHBUSY_MASK);

   //Enable the Ethernet controller by setting the ON bit
   ETHCON1SET = _ETHCON1_ON_MASK;

   //Clear Ethernet interrupt flag
   IFS4CLR = _IFS4_ETHIF_MASK;
   //Disable any Ethernet controller interrupt generation
   ETHIEN = 0;
   ETHIRQ = 0;
   //Clear the TX and RX start addresses
   ETHTXST = 0;
   ETHRXST = 0;

   //Reset the MAC using SOFTRESET
   EMAC1CFG1SET = _EMAC1CFG1_SOFTRESET_MASK;
   EMAC1CFG1CLR = _EMAC1CFG1_SOFTRESET_MASK;

   //Reset the RMII module
   EMAC1SUPPSET = _EMAC1SUPP_RESETRMII_MASK;
   EMAC1SUPPCLR = _EMAC1SUPP_RESETRMII_MASK;

   //Issue an MIIM block reset by setting the RESETMGMT bit
   EMAC1MCFGSET = _EMAC1MCFG_RESETMGMT_MASK;
   EMAC1MCFGCLR = _EMAC1MCFG_RESETMGMT_MASK;
   
      //Initialize hash table
   ETHHT0 = 0;
   ETHHT1 = 0;

   //Configure the receive filter

   EMAC1MAXF = 1518;
   ETHRXST = (uint32_t) KVA_TO_PA(&rxBufferDesc[0]);
   ETHTXST = (uint32_t) KVA_TO_PA(&txBufferDesc[0]);
   ETHCON2 = PIC32MZ_ETH_RX_BUFFER_SIZE;
   
   //Enable the reception by setting the RXEN bit
   ETHCON1SET = _ETHCON1_RXEN_MASK;
   EMAC1SUPPbits.SPEEDRMII = 1;
 
   txBufferDesc[0].address = (uint32_t) KVA_TO_PA(&(txBuffer[0][0]));
   txBufferDesc[1].address = (uint32_t) KVA_TO_PA(&(txBuffer[1][0]));
   txBufferDesc[2].address = (uint32_t) KVA_TO_PA(&(txBuffer[2][0]));
   
   txBufferDesc[0].control = 0xc4000180;
   txBufferDesc[1].control = 0xc4000180;
   txBufferDesc[2].control = 0xc4000180;
   
   txBufferDesc[0].next = (uint32_t) KVA_TO_PA(&txBufferDesc[1]);
   txBufferDesc[1].next = (uint32_t) KVA_TO_PA(&txBufferDesc[2]);
   txBufferDesc[2].next = (uint32_t) KVA_TO_PA(&txBufferDesc[0]);

//if these memset statements are commented, the MAC transmits zero frames
//if they are not commented, it transmits 3 frames with data only in the first
// memset(&(txBuffer[0][0]), 0x14, PIC32MZ_ETH_TX_BUFFER_SIZE);
// memset(&(txBuffer[1][0]), 0x14,PIC32MZ_ETH_TX_BUFFER_SIZE);
// memset(&(txBuffer[2][0]), 0x14, PIC32MZ_ETH_TX_BUFFER_SIZE);
 
   //Set TXRTS bit to start the transmission
   ETHCON1SET = _ETHCON1_TXRTS_MASK;
   
   
   while(1)
   {
       if(ETHCON1bits.TXRTS == 0)
       {
             ETHCON1SET = _ETHCON1_TXRTS_MASK;
       }
   }
   
}

[kyrk.5]

Hi,

just read your answer, can only test the code later. But one thing came to my mind:
Cache
Ofthen this type of problem comes from a not coherent cache. For example written data is still in the cache but not in the RAM. So when Ethernet is accessing it, the RAM will have other values than it should. Try to add cache invalidation or allocate the buffer in the non cached space.
If caching is a problem, it could checked out if other projects maybe let them generraly be disabled. Not quite a good idea but often developers chose the easiest way to come around problems... I  think cache settings are done is some kind of a function with the text: performance or like this. At least in PLIB. Harmony is also doing such things I think. Good point to check cache setting.

This might not work on PIC32: Another quick test is to just step over instructions rather then running and let a breakpoint hit. Ofthen stepping force the system to throw away the cache. At least on PowerPC it is good testing method. Some code does not run but stepping over works. Ofthen this is a cacheing problem.

[jrmymllr]

Hi,

just read your answer, can only test the code later. But one thing came to my mind:
Cache
Ofthen this type of problem comes from a not coherent cache. For example written data is still in the cache but not in the RAM. So when Ethernet is accessing it, the RAM will have other values than it should. Try to add cache invalidation or allocate the buffer in the non cached space.
If caching is a problem, it could checked out if other projects maybe let them generraly be disabled. Not quite a good idea but often developers chose the easiest way to come around problems... I  think cache settings are done is some kind of a function with the text: performance or like this. At least in PLIB. Harmony is also doing such things I think. Good point to check cache setting.

This might not work on PIC32: Another quick test is to just step over instructions rather then running and let a breakpoint hit. Ofthen stepping force the system to throw away the cache. At least on PowerPC it is good testing method. Some code does not run but stepping over works. Ofthen this is a cacheing problem.

I believe all caching is disabled by default according to the PRECON register, and I'm not changing any of those bits. I did experiment by turning cache on and off while measuring the performance of the PIC32 running a MP3 decoder, but for stuff like this I set optimization off and cache off.

[kyrk.5]

Not the cache
Sorry still did not had enough time to debug (possible on weekend), but here are two links:
http://ww1.microchip.com/downloads/en/DeviceDoc/60001155D.pdf
Example 35-4: Example for sending message. Slightly different than yours, they have multiple descriptors but first is start and last is stop. In your example all 3 are start and stop. Kind of a difference. Might be a point to try this way.

Here is also an example:
https://www.oryx-embedded.com/doc/pic32mx__eth_8c_source.html
It looks like they prepare the linked list in a circullar manner. And then load one element of it and then set ETH_TX_CTRL_EOWN to 1 and then let it transmit. In your example you are setting each buffer to HW owned. Maybe you could try to send first buffer by setting EOWN to 1 and trigger send, then set second buffer by setting EOWN to 1 and trigger and so on. Maybe this will work better.

[JohnS]

Check for other operating systems which drive the PIC32MZ ethernet hardware and look at the code.

E.g. LiteBSD

By way of example it has code in such as https://github.com/sergev/LiteBSD/blob/master/sys/mips/dev/if_en.c

John

[jrmymllr]

Check for other operating systems which drive the PIC32MZ ethernet hardware and look at the code.

E.g. LiteBSD

By way of example it has code in such as https://github.com/sergev/LiteBSD/blob/master/sys/mips/dev/if_en.c

John

This is interesting and fairly easy to read. Thanks.

[jrmymllr]

Not the cache
Sorry still did not had enough time to debug (possible on weekend), but here are two links:
http://ww1.microchip.com/downloads/en/DeviceDoc/60001155D.pdf
Example 35-4: Example for sending message. Slightly different than yours, they have multiple descriptors but first is start and last is stop. In your example all 3 are start and stop. Kind of a difference. Might be a point to try this way.

Here is also an example:
https://www.oryx-embedded.com/doc/pic32mx__eth_8c_source.html
It looks like they prepare the linked list in a circullar manner. And then load one element of it and then set ETH_TX_CTRL_EOWN to 1 and then let it transmit. In your example you are setting each buffer to HW owned. Maybe you could try to send first buffer by setting EOWN to 1 and trigger send, then set second buffer by setting EOWN to 1 and trigger and so on. Maybe this will work better.

I have looked extensively through 60001155D and used that to write what I did. But I will try changing the start and stop bits as you suggest. As I understand it's ok to have start and stop on the same message if it represents the entire frame, but I'm open to ideas.

I did also come across Cyclone TCP and followed their code.  It's really good and easy to follow. I think I'm going to take a second look at this though.

[jrmymllr]

Not the cache
Sorry still did not had enough time to debug (possible on weekend), but here are two links:
http://ww1.microchip.com/downloads/en/DeviceDoc/60001155D.pdf
Example 35-4: Example for sending message. Slightly different than yours, they have multiple descriptors but first is start and last is stop. In your example all 3 are start and stop. Kind of a difference. Might be a point to try this way.

Here is also an example:
https://www.oryx-embedded.com/doc/pic32mx__eth_8c_source.html
It looks like they prepare the linked list in a circullar manner. And then load one element of it and then set ETH_TX_CTRL_EOWN to 1 and then let it transmit. In your example you are setting each buffer to HW owned. Maybe you could try to send first buffer by setting EOWN to 1 and trigger send, then set second buffer by setting EOWN to 1 and trigger and so on. Maybe this will work better.

Something I recently found. Maybe it is the cache. It seems the cache is likely enabled by default despite my previous assumptions. And, RAM used for DMA, which ethernet uses, should be put into KSEG1, or the 'coherent' linker attribute can be used. This might explain some of the frustrating inconsistencies I've encountered. Stand by, I will report back when I can try it.

[jrmymllr]

Not the cache
Sorry still did not had enough time to debug (possible on weekend), but here are two links:
http://ww1.microchip.com/downloads/en/DeviceDoc/60001155D.pdf
Example 35-4: Example for sending message. Slightly different than yours, they have multiple descriptors but first is start and last is stop. In your example all 3 are start and stop. Kind of a difference. Might be a point to try this way.

Here is also an example:
https://www.oryx-embedded.com/doc/pic32mx__eth_8c_source.html
It looks like they prepare the linked list in a circullar manner. And then load one element of it and then set ETH_TX_CTRL_EOWN to 1 and then let it transmit. In your example you are setting each buffer to HW owned. Maybe you could try to send first buffer by setting EOWN to 1 and trigger send, then set second buffer by setting EOWN to 1 and trigger and so on. Maybe this will work better.

It was the cache! I thought it was disabled by default. Too bad the Ethernet datasheet doesn't mention this. All DMA memory must be in KSEG1! Now I can make some progress...

[kyrk.5]

Cool

Actually I think it is possible to invalidate the cache by code and have the memory in KSEG0. This would result in a more portable code, since on other architectures the memory is not mirrored like this. Often it is configurable with some region descriptors (MPU or SMPU) that a given part of the memory should not be cached or it can be invalidated by c code.