Bug in bcm2835_spi_transfern?

[Tobias Spitzbart]

Hi,

I try a SPI connection between RPi and FPGA.

I use the function: bcm2835_spi_transfern

When I try to send more then 64 Bytes there is a big delay.

For e.g. I try to send 256 bytes.

On the pic you can see the first 64 bytes then a delay from about 24us then the secound 64 bytes an so on.

Is the function or the DMA limit to 64 bytes?

Thanks for you help

Greetings Tobias

[Arjan van Vught]

Hi Tobias,

Which Raspberry Pi board are you using?

Are there any other applications running in de background?

Can you also provide the CS and CLK signal?

There is no DMA used with this library function. 

Meanwhile, I will try to reproduce your situation on a Raspberry Pi Model 3B board.

- Arjan 

https://github.com/vanvught/rpidmx512

<tek00006.png>

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/b1136916-afb7-4bc5-b6cd-a9f56e03ff22%40googlegroups.com.
<tek00006.png>

[Arjan van Vught]

Hi Tobias, see my analyzer outputs and source code below. The Raspberry Pi board is generating a good SPI.

 lsb_release -a

No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.8 (stretch)
Release:        9.8
Codename:       stretch

- Arjan

https://github.com/vanvught/rpidmx512

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <stdint.h>

#include "bcm2835.h"

int main(int argc, char **argv) {
    if (bcm2835_init() == 0) {
        fprintf(stderr, "Function bcm2835_init() failed\n");
        return -1;
    }

    if (bcm2835_spi_begin() == 0) {
        fprintf(stderr, "Function bcm2835_spi_begin() failed\n");
        return -1;
    }

    bcm2835_spi_set_speed_hz(640000);
    bcm2835_spi_chipSelect(BCM2835_SPI_CS0);
    bcm2835_spi_set_speed_hz(4000000);
    bcm2835_spi_setDataMode(BCM2835_SPI_MODE0);

    char bytes[1024];

    uint32_t i = 0;

    for (i = 0; i < 1024; i++) {
        bytes[i] = i;
    }

    for (;;) {
        bcm2835_spi_transfern((char *) bytes, 1024);
        for (i = 0; i < 1024; i++) {
            bytes[i] = i;
        }

        bcm2835_delayMicroseconds(50000);
    }
}

Op 19-7-2019 om 20:19 schreef Arjan van Vught:

[Tobias Spitzbart]

Hi Arjan,

thanks for your replay.

I testet it with the RPI4 and Debain Buster.

My SPI clock was 50MHz. Can you maybe test your system mit a clock speed from 50Mhz?

Greetings Tobias

[Arjan van Vught]

Hi Tobias,

The 50Mhz is pretty high on the limitation. Moreover, this library is not formally released for the Rasperry Pi Model 4.

I do not have a Model 4 (won’t never buy it), but I can give this 50MHz a try on a Model 3B+ board with Stretch.

Meanwhile, please also try 10MHz, 20MHz and 30MHz clock spseed for the SPI. 

- Arjan

https://github.com/vanvught/rpidmx512

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/53a0c22c-489d-4dc5-b0dc-2024e23b846a%40googlegroups.com.

[Tobias Spitzbart]

Hello Arjan,

sorry for the late anser.

I try the clock speed with 10MHz, 20MHz and 30MHz.

Above 10MHz there is a gab between first 64 bytes and the next 64 bytes.

The gab increase with the clock speed.

Can you please test it with your setup.

Greetings Tobias

 

Am Freitag, 19. Juli 2019 23:07:06 UTC+2 schrieb Arjan van Vught:

Hi Tobias,

The 50Mhz is pretty high on the limitation. Moreover, this library is not formally released for the Rasperry Pi Model 4.

I do not have a Model 4 (won’t never buy it), but I can give this 50MHz a try on a Model 3B+ board with Stretch.

Meanwhile, please also try 10MHz, 20MHz and 30MHz clock spseed for the SPI. 

- Arjan

https://github.com/vanvught/rpidmx512

Op 19 jul. 2019, om 22:37 heeft Tobias Spitzbart <tobias....@gmail.com> het volgende geschreven:

Hi Arjan,

thanks for your replay.

I testet it with the RPI4 and Debain Buster.

My SPI clock was 50MHz. Can you maybe test your system mit a clock speed from 50Mhz?

Greetings Tobias

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bcm...@googlegroups.com.

[Tobias Spitzbart]

Hello,

I tested it with the same setup with a raspberry pi 3+.
The problem is similar.
Above 15 MHz there is a gab between two 64Byte Blocks.

I tried to use the Real-time Kernel Patch. But that makes the things worse.

Have anyone have a Idea. Or is the SPI limited around 3MB/s?

Greetings Tobias

[Arjan van Vught]

Hi Tobias,

Have you tried to add the code as mentioned in section: "Real Time performance constraints”-> https://www.airspayce.com/mikem/bcm2835/

- Arjan

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/02830b7d-78f3-4b75-8ad3-708e903a8f5b%40googlegroups.com.

[Arjan van Vught]

Hi Tobias,

My apologize; it is still on my list to give it a test. Also with bare metal - no Linux. However, I could imagine that it does not make any difference. This library does not use any Linux drivers. Hence it would be something with the SoC itself.

- Arjan

https://github.com/vanvught/rpidmx512

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/02830b7d-78f3-4b75-8ad3-708e903a8f5b%40googlegroups.com.

[Tobias Spitzbart]

Hi Arjan,

thanks for your answer.

It is a shame that there is no Datasheet from the SoC.

Maybe the SPI-Controller in the SOC has a 64Byte buffer and when it is empty the next 64Bytes a load into the Buffer...

But without a Datasheet it is only speculation

Greetings Tobias

Am Freitag, 30. August 2019 09:49:38 UTC+2 schrieb Arjan van Vught:

Hi Tobias,

My apologize; it is still on my list to give it a test. Also with bare metal - no Linux. However, I could imagine that it does not make any difference. This library does not use any Linux drivers. Hence it would be something with the SoC itself.

- Arjan

https://github.com/vanvught/rpidmx512

Op 27 aug. 2019, om 12:26 heeft Tobias Spitzbart <tobias....@gmail.com> het volgende geschreven:

Hello,

I tested it with the same setup with a raspberry pi 3+.
The problem is similar.
Above 15 MHz there is a gab between two 64Byte Blocks.

I tried to use the Real-time Kernel Patch. But that makes the things worse.

Have anyone have a Idea. Or is the SPI limited around 3MB/s?

Greetings Tobias

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bcm...@googlegroups.com.

[Tobias Spitzbart]

In the Version history I found this point:

1.28 Fixed a problem where bcm2835_spi_writenb() would have problems with transfers of more than 64 bytes dues to read buffer filling. Patched by Peter Würtz.

Maybe that is similar to my problem...

Greetings Tobias

[Arjan van Vught]

Hi Tobias,

Op 30 aug. 2019, om 14:40 heeft Tobias Spitzbart <tobias.s...@gmail.com> het volgende geschreven:

Hi Arjan,

thanks for your answer.

It is a shame that there is no Datasheet from the SoC.

Maybe the SPI-Controller in the SOC has a 64Byte buffer and when it is empty the next 64Bytes a load into the Buffer...

But without a Datasheet it is only speculation

Herewith the datasheet -> https://www.raspberrypi.org/app/uploads/2012/02/BCM2835-ARM-Peripherals.pdf

- Arjan

Greetings Tobias

Am Freitag, 30. August 2019 09:49:38 UTC+2 schrieb Arjan van Vught:

Hi Tobias,

My apologize; it is still on my list to give it a test. Also with bare metal - no Linux. However, I could imagine that it does not make any difference. This library does not use any Linux drivers. Hence it would be something with the SoC itself.

- Arjan

https://github.com/vanvught/rpidmx512

Op 27 aug. 2019, om 12:26 heeft Tobias Spitzbart <tobias....@gmail.com> het volgende geschreven:

Hello,

I tested it with the same setup with a raspberry pi 3+.
The problem is similar.
Above 15 MHz there is a gab between two 64Byte Blocks.

I tried to use the Real-time Kernel Patch. But that makes the things worse.

Have anyone have a Idea. Or is the SPI limited around 3MB/s?

Greetings Tobias

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bcm...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/02830b7d-78f3-4b75-8ad3-708e903a8f5b%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/3e0f529b-c54d-4864-a37e-f6699a0c9057%40googlegroups.com.

[Arjan van Vught]

Op 30 aug. 2019, om 14:58 heeft Tobias Spitzbart <tobias.s...@gmail.com> het volgende geschreven:

In the Version history I found this point:

1.28 Fixed a problem where bcm2835_spi_writenb() would have problems with transfers of more than 64 bytes dues to read buffer filling. Patched by Peter Würtz.

As this function is a write only, the FIFO read was ignored. The latter caused the issue. 

Patch: 

/* Read from FIFO to prevent stalling */

while (bcm2835_peri_read(paddr) & BCM2835_SPI0_CS_RXD)

   (void) bcm2835_peri_read_nb(fifo);

Maybe that is similar to my problem…

There could be an issue that the FIFO is not filled quickly enough. That’s why I am eager to give it a test on baremetal. 

Can you give it a try with the Linux SPI DMA driver? 

Please note that this Linux driver test must be done without any code from this library. 

Or, add a GPIO pin toggle in void bcm2835_spi_transfernb where you check if we must wait on a Full FIFO. This is somehow scary as you are working with high clock speed and a GPIO toggle is relatively costly. 

- Arjan

Greetings Tobias

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/b5112dc1-df79-4e7f-98d0-e55f6db6268a%40googlegroups.com.

[Arjan van Vught]

Hi Tobias,

I’ve done a test with a 128 byte’s transfer at 20 MHz. There is no (visible) gap in the middle. I have added a .7z file with the kernel?.img on Google Drive -> https://drive.google.com/open?id=1B4EElQ7-Ke_i5iB-m8eHFHG3w18tcLY7

This is baremetal. When you want to give it a try, then please use an empty FAT formatted SDCard and copy the contents of the .7z file onto the SDCard.

There is no output on the screen. The green led will bink at 1Hz. With the source code below and the Makefile, you could also build your own test version -> clone my GitHub and create your own project. Run make and both kernel?.img files will be created. 

My baremetal implementation is similar to the Linux version -> https://github.com/vanvught/rpidmx512/blob/master/lib-bcm2835/src/bcm2835_spi.c#L128

- Arjan

https://github.com/vanvught/rpidmx512

CH1 = SPI CLK, CH4 = SPI_CS ans CH3 = SPI_MOSI

#include <stdio.h>

#include "bcm2835_spi.h"

#include "c/led.h"

void notmain(void) {

int i;

bcm2835_spi_begin();

bcm2835_spi_chipSelect(BCM2835_SPI_CS0);

bcm2835_spi_setDataMode(BCM2835_SPI_MODE0);

bcm2835_spi_set_speed_hz(20000000);

char buffer[1024];

for (i = 0; i < sizeof(buffer) ; i++) {

buffer[i] = 0xFF;

}

for (;;) {

bcm2835_spi_transfern(buffer, 128);

for (i = 0; i < sizeof(buffer) ; i ++) {

buffer[i] = 0xFF;

}

udelay(500000);

led_blink();

}

}

Makefile

#

DEFINES = ENABLE_MMU

#

LIBS = 

#

SRCDIR = firmware

include ../firmware-template/Rules.mk

prerequisites:

<tek00006.png>

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/b1136916-afb7-4bc5-b6cd-a9f56e03ff22%40googlegroups.com.
<tek00006.png>

[Tobias Spitzbart]

Hello Arjan

thanks for your replay.

I try use the direct way over the /dev/spidev0.0 with c# and pyhton3 with 50Mhz Clock

When I transfer 2x64 Bytes there is a gab between every Byte and between the 64Byte blocks..
When I transfer 3x64 Bytes there is a gab between every Byte and between the 64Byte blocks.
...
When I transfer 287 Bytes there is a gab between every Byte and between the 64Byte blocks.
When I transfer 288 Bytes the gabs are diaper...


For Pyhton I use this code:
import spidev
import time

spi= spidev.SpiDev()
spi.open(0,0)
spi.max_speed_hz=100000000
Data=[0]*287 #287 -> gab; 288 -> no gab


while 1:
    spi.xfer2(Data);
    time.sleep(2)

This is really a strange behavior...

Can somebody please confirm that behavior...

Greetings Tobias

Op 19 jul. 2019, om 20:09 heeft Tobias Spitzbart <tobias....@gmail.com> het volgende geschreven:

Hi,

I try a SPI connection between RPi and FPGA.

I use the function: bcm2835_spi_transfern

When I try to send more then 64 Bytes there is a big delay.

For e.g. I try to send 256 bytes.

On the pic you can see the first 64 bytes then a delay from about 24us then the secound 64 bytes an so on.

Is the function or the DMA limit to 64 bytes?

Thanks for you help

Greetings Tobias

<tek00006.png>

--
You received this message because you are subscribed to the Google Groups "bcm2835" group.

To unsubscribe from this group and stop receiving emails from it, send an email to bcm...@googlegroups.com.

[Tobias Spitzbart]

Here is a picture

Pyhton Code:

import spidev
import time

spi= spidev.SpiDev()
spi.open(0,0)
spi.max_speed_hz=100000000

Data1=[0]*287
Data2=[0]*288


while 1:
    spi.xfer2(Data1);

    spi.xfer2(Data2);
    time.sleep(2)

[Arjan van Vught]

Hi Tobias,

Python is an interpreter language, therefore it is not the best choice when performance is expected.

What I see when using bcm2835_spi_transfern running without Linux, there are no gaps. This tells me that Linux, with its (many) interrupts, is not doing well for high SPI clock speeds. And that there is no bug in bcm2835_spi_transfern.

A good test would be writing a user-space DMA SPI solution with this library -> porting this https://github.com/rsta2/circle/blob/master/lib/spimasterdma.cpp to user-space.

- Arjan

To unsubscribe from this group and stop receiving emails from it, send an email to bcm2835+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/1a2da386-e3e1-4129-91d4-c9e490173389%40googlegroups.com.

[Tobias Spitzbart]

Hi Arjan,

Thanks for your replay.

Okay. Then it is a Linux-Interrupt-SPI-Polling-Mode problem. ;)

I am not smart enough to writing a user-space DMA SPI solution...

So I must live with this limitation.

Thanks for your support.

Greetings Tobias

Am Donnerstag, 5. September 2019 12:27:34 UTC+2 schrieb Arjan van Vught:

Hi Tobias,

Python is an interpreter language, therefore it is not the best choice when performance is expected.

What I see when using bcm2835_spi_transfern running without Linux, there are no gaps. This tells me that Linux, with its (many) interrupts, is not doing well for high SPI clock speeds. And that there is no bug in bcm2835_spi_transfern.

A good test would be writing a user-space DMA SPI solution with this library -> porting this https://github.com/rsta2/circle/blob/master/lib/spimasterdma.cpp to user-space.

- Arjan

To view this discussion on the web visit https://groups.google.com/d/msgid/bcm2835/1a2da386-e3e1-4129-91d4-c9e490173389%40googlegroups.com.