Best way for events between PRU and processor?

[Ken Shirriff]

I'm trying to send information back and forth between the processor and the PRU, and I'm looking for suggestions on the best way to do this.

Currently I'm using PRU_EVTOUT0 to send events from the PRU. The processor code does a select() on the PRU_EVTOUT_0 fd to find out when an event has happened. Then I do a prussdrv_pru_wait_event() and prussdrv_pru_clear_event() to get rid of the event. (The select is because I also want to wait for network data.)

However, this is kind of a mess of race conditions, since an event can come in between the select and the clear. Or two events can happen before the select. So I have various status flags that the PRU sets in memory. But that leads to other race conditions.

So, I'm wondering if there's a better way to handle events back and forth. Other people must have dealt with this and come up with good solutions.

I've seen stuff about Remoteproc - is that the cool new technology? Its mailboxes seem like a good model. However, I'd rather stick with the UIO model instead of moving to a new kernel and rewriting everything if possible. 

My application, in case it's relevant: I'm building a network gateway with the PRU bit-banging a 3 megabit/second Ethernet. So the processor sends packets to the PRU to transmit, and the PRU tells the processor about incoming packets. The PRU needs to tell the processor when a send is completed, or when a packet has arrived. 

Thanks for any help,

Ken

[Przemek Klosowski]

On Mon, Nov 6, 2017 at 9:09 PM, Ken Shirriff <ken.sh...@gmail.com> wrote:

My application, in case it's relevant: I'm building a network gateway with the PRU bit-banging a 3 megabit/second Ethernet.

 

Oh, I've seen that on your blog---the bridge from  bitbanged Alto 3Mbps to a 10/100 ethernet---great work!

I've seen stuff about Remoteproc - is that the cool new technology? Its mailboxes seem like a good model. However, I'd rather stick with the UIO model instead of moving to a new kernel and rewriting everything if possible. 

Yeah, many people on this forum like UIO but it seems that Remoteproc is the wave of the future: it's the default in newer kernels, etc.  UIO is of course supported too, though. 

I don't  know the answer to your question about synchronization, sorry; TJF might know.

[din...@gmail.com]

Hi,

FYI, recent remoteproc RPMSG versions have moved from mailboxes to interrupts for communication: https://git.ti.com/pru-software-support-package/pru-software-support-package/commit/69805828df0f262fb60363c2db189d1b8d0b693c

A race-free algorithm would require the interrupts simply to wake the peer, and rely on shared memory FIFO for handling events. AFAIK, that's the idea used by virtio/RPMSG. In pseudo-code:

1. Wait for interrupt.

2. Clear interrupt.

3. Drain the events-FIFO located in shared memory.

Regards,

Dimitar

[TJF]

Am Dienstag, 7. November 2017 18:42:58 UTC+1 schrieb din...@gmail.com:

Hi,

FYI, recent remoteproc RPMSG versions have moved from mailboxes to interrupts for communication: https://git.ti.com/pru-software-support-package/pru-software-support-package/commit/69805828df0f262fb60363c2db189d1b8d0b693c0

This is a small step in the right direction. When they're finally finishing their way, they'll end up where the prussdrv concept started.

[TJF]

Hi Ken Shirriff!

Here's what I do in the libpruio project:

I don't use interrupts and I don't stop the PRU main loop (no prussdrv_pru_wait_event() and prussdrv_pru_clear_event()).

Instead the PRU main loop is running continually and I use a certain area in the DRam to exchange status information.

Before the ARM sends a new command, it has to wait until the PRU is ready (max. 10 to 15 PRU cycles).

And when the PRU has data for the ARM part, it stores the data in a ring buffer and tells the ARM the current ring buffer position.

I'm not keen on ethernet issues, but I think a similar concept should work for you as well. Use a FIFO buffer for outgoing data and a ring buffer for incomming. Before the ARM can send the next package, it has to wait until the current one is done. (When this limitation is insufficient, you'll have to use a ring buffer for outgoing data as well.)

BR

[Ken Shirriff]

Thanks everyone for the suggestions. I used Dimitar's approach and it works reliably and made my code more comprehensible.

I now have a single event loop that does the wait/clear/process, rather than trying to handle things semi-synchronously and expecting to get an interrupt event in response to a particular PRU request. I also made "ownership" of each buffer explicit between the PRU and the ARM. When the ARM has a buffer ready for the PRU, it marks the owner as "PRU". When the PRU is done with a buffer, it marks the owner as  "ARM" and sends an interrupt.  So when the ARM gets an interrupt, it doesn't assume anything is done, but checks the owner tags to see what it should do.

The shorter explanation is that before I was using the interrupt event to indicate a particular task was done, which was a race condition mess. Now I use the interrupt event to indicate that something has (probably) changed and then check to see what changed.

Ken

[dr.rolan...@gmail.com]

Hi Ken Shirriff,

could you please give a code example or a link about how to manage the interrupt handling on the Linux user code side?

Thanks in advance.

RoSchmi

[Ken Shirriff]

Hi RoSchmi,

My code is at https://github.com/shirriff/alto-ethernet-interface/tree/master/src

I also wrote a blog post discussing interrupts: http://www.righto.com/2016/09/how-to-run-c-programs-on-beaglebones.html

Note that it is for the 3.8.13 kernel and everything works differently in newer kernels. Also, I'm not an expert here so I don't guarantee that my code is the best way to do things.

Ken

[Roland Schmidt]

Hi Ken

thank you for the links.

I will have a look on it tomorrow.

RoSchmi

[RoSchmi]

Am Montag, 4. Dezember 2017 22:02:14 UTC+1 schrieb Ken Shirriff:

Hi RoSchmi,

My code is at https://github.com/shirriff/alto-ethernet-interface/tree/master/src

I also wrote a blog post discussing interrupts: http://www.righto.com/2016/09/how-to-run-c-programs-on-beaglebones.html

Note that it is for the 3.8.13 kernel and everything works differently in newer kernels. Also, I'm not an expert here so I don't guarantee that my code is the best way to do things.

Ken

Hi Ken,
I had a look on both of your links. As you said, the programs use the old PRU <-> ARM communication concept. However I'd more like to use the new PRU remoteproc, PRMsg and intc concept. Did you already manage to set up the user space code to receive interrupts from the PRUs using the new concept? If yes, are you willing to share the code?
Kind regards
RoSchmi

[pierric...@gadz.org]

Hi RoSchmi,

I am using the PRU with RPMsg for a school project, I have just made a documentation about how to set it up:
https://github.com/PierrickRauby/PRU-RPMsg-Setup-BeagleBoneBlack

I maybe out off the track with this answer but I hope it help.

Thanks

Pierrick