'Speeding up' ArtNet

[Adam Davies]

Hi there,

I'm trying to make an 8x8x8 LED cube to represent channels in a universe. With so many LEDs, each column has the +ve wired together and each layer has the -ve wired together (like most tutorials)

Currently, when receiving ArtNet at 30fps some of the LEDs are flickering as their connected pins are being turned off, then back on again, as other channels in their layer or column change. Due to the 'slow' speed of ArtNet, this flicker is noticeable.

I'm thinking that if I can have the ArtNet frame create/replace the array at 30fps, I can poll this array 100 times per second to 'speed up' the changes and hopefully remove the flicker. Howveer I'm struggling where in my script to place the loop to achieve this

Has anyone tried this? Have you any thoughts? My current code is below. I know some of the labels are odd, I'm following tutorials and haven't cleaned it up yet

Thanks

Adam

_______________________________________________________________________________________

from led import *

from database import *

from ola.ClientWrapper import ClientWrapper

from time import sleep

import sys

#Sets all pins to off

clear()

threshold = 4

def NewData(data):

  

  #Create list of all channels in a universe

  channels = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148,149,150,151,152,153,154,155,156,157,158,159,160,161,162,163,164,165,166,167,168,169,170,171,172,173,174,175,176,177,178,179,180,181,182,183,184,185,186,187,188,189,190,191,192,193,194,195,196,197,198,199,200,201,202,203,204,205,206,207,208,209,210,211,212,213,214,215,216,217,218,219,220,221,222,223,224,225,226,227,228,229,230,231,232,233,234,235,236,237,238,239,240,241,242,243,244,245,246,247,248,249,250,251,252,253,254,255,256,257,258,259,260,261,262,263,264,265,266,267,268,269,270,271,272,273,274,275,276,277,278,279,280,281,282,283,284,285,286,287,288,289,290,291,292,293,294,295,296,297,298,299,300,301,302,303,304,305,306,307,308,309,310,311,312,313,314,315,316,317,318,319,320,321,322,323,324,325,326,327,328,329,330,331,332,333,334,335,336,337,338,339,340,341,342,343,344,345,346,347,348,349,350,351,352,353,354,355,356,357,358,359,360,361,362,363,364,365,366,367,368,369,370,371,372,373,374,375,376,377,378,379,380,381,382,383,384,385,386,387,388,389,390,391,392,393,394,395,396,397,398,399,400,401,402,403,404,405,406,407,408,409,410,411,412,413,414,415,416,417,418,419,420,421,422,423,424,425,426,427,428,429,430,431,432,433,434,435,436,437,438,439,440,441,442,443,444,445,446,447,448,449,450,451,452,453,454,455,456,457,458,459,460,461,462,463,464,465,466,467,468,469,470,471,472,473,474,475,476,477,478,479,480,481,482,483,484,485,486,487,488,489,490,491,492,493,494,495,496,497,498,499,500,501,502,503,504,505,506,507,508,509,510,511,512]

  #Create array combining channel numbers and channel values

  fruit_dictionary = dict(zip(channels, data))

  for key, value in fruit_dictionary.items():

#If channel has dropped but was previously on, turn off

    if value <threshold and wiringpi.digitalRead((key+100)) == 1:

      wiringpi.digitalWrite((key+100),0)

#If channel is now on but was previously off

    elif value >threshold and wiringpi.digitalRead((key+100)) ==0:

      ledinstructions[key]()

      

universe = 1

try:

  wrapper = ClientWrapper()

  client = wrapper.Client()

  client.RegisterUniverse(universe, client.REGISTER, NewData)

  wrapper.Run()

  

except KeyboardInterrupt:

  clear()

  print("The End")

[E.S. Rosenberg]

Hey Adam,

This would depend on your "last mile", if from ArtNet you are going straight to some native LED protocol some of them I believe support higher refresh rates if I'm not mistaken.

If on the other hand your "last mile" is DMX and you are using all 512 channels then your maximum refresh rate is 41Hz.

HTH,

Eli

Op do 25 feb. 2021 om 20:13 schreef Adam Davies <adam....@tsllighting.com>:

--
The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/9b6b457c-c15d-4544-827c-5c0e5a9bde10n%40googlegroups.com.

[Adam Davies]

Hi Eli,

Apologies, perhaps I didn't explain it very well. The Raspberry Pi will listen to ArtNet (being broadcast from elsewhere) and control 512 simple 3V LEDs connected to the GPIO pins of the Raspberry Pi. No PWM dimming, a simple on or off.

Obviously I don't have 512 pins so by soldering like I've done, this reduces the pin count to 72. I'm using WiringPi as a library to control MCP23017 chips via I2C - these chips basically act as GPIO expander chips.

Currently I get 30fps of ArtNet coming 'in', but I need to find a way, in the Python script, to 'cache' that frame and then open or close GPIO pins at 100fps (or potentially more) so that the flicker isn't noticeable, until the next frame of ArtNet arrives and the GPIO pins then open or close according to the channel values in that frame.

Make sense?

Adam

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/CAHxFV_tU_2%2Bm2sWfP30Vjsd-nD5MAeZ5Y6593Hcbr_Ueh-YC8A%40mail.gmail.com.

--

Adam Davies Control & System Support t: +44(0)20 8629 2025 |w: www.tsllighting.com

This message contains confidential information and is intended only for the intended recipients. If you are not an intended recipient you should not disseminate, distribute or copy this e-mail. Please notify us immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. Therefore we do not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version.

 

[Peter Stuge]

Adam Davies wrote:
> I'm using WiringPi as a library to control MCP23017 chips via I2C -
> these chips basically act as GPIO expander chips.

Perhaps WiringPi but especially I2C are significant bottlenecks.

I2C is only guaranteed at 100kHz and the software stack you're using
will likely be sending two or three bytes per IO state output.

If we calculate the absolute maximum 100000/8/2 that's 6250 outputs
per second, 6250/72 = 86 fps. That may seem good but in practice you
will get nowhere near that because the main CPU isn't constantly
sending I2C data out, but doing a ton of other stuff;

* running Linux
* running Python
* receiving ArtNet

Whenever you run anything in userspace of a general purpose operating
system like Linux you have to be aware that the kernel scheduler does
not make any guarantees whatsoever about when your program will run,
and context switches between kernel (which happen for every I2C operation)
are fairly costly.

> Currently I get 30fps of ArtNet coming 'in', but I need to find a way, in
> the Python script, to 'cache' that frame and then open or close GPIO pins
> at 100fps (or potentially more) so that the flicker isn't noticeable

That's just not doable with I2C and your current software stack, if
with I2C at all.

Maybe *maybe* you can reach your target if you throw WiringPi out and
call kernel ioctl():s directly, writing 8 IOs with every syscall.

Also, don't have any conditions in your hot path. Make the hot path
deterministic, make it write all pixels every time. Then you'll have
constant frame rate always, and the main CPU can cache better.


> *Adam Davies*
..

> This message contains confidential information and is intended only for the

Don't pretend to send confidential information to public mailing lists.
Please talk with your systems administrator to remove that silly paragraph.


//Peter

[Adam Davies]

Hi Peter,

Ok that's all really useful, thanks for the info.

I was aware I'd never guarantee a solid higher rate as Linux needs to continue to run, I was just hoping that I'd be able to store and re-process that frame as a kind of buffer or something to artificially speed up the refresh rate going to the LEDs

I fear it may be time to return to the drawing board to see how else I can achieve this

Adam

[Steve French of Volt Vision]

Would a SPI interface work for your IO expanders? (some can work with both I2C or SPI).  SPI can be clocked much faster than I2C, but I am not sure about potential bottlenecks on the Linux side of it.  Also TI makes quite a few Constant Current LED drivers with digital shift registers that can be daisy chained for higher counts.  These are made for driving LED matrices I think.

--

Respectfully,
VoltVisionFrenchy aka Steve French

www.voltvision.com

--

The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/f0814d52-530c-4c42-8799-32866981c40dn%40googlegroups.com.

[Stefan Krüger]

as Steve pointed to -

all the big led screens are using some sort of (spi) shift-registers that can handle the high data-rates..

the *anti-flickering* frequency updates are then in bast case handled by the driver chips -
most of them need some external clock signal to control these update-rates.

some of the TexasInstruments datasheets /  reference designs / videos for there LED-Driver portfolio are nice resources to get an idea of what is done to get flicker-free leds and how multiplexing is handled in these devices....
most of the chips have an *internal memory* to hold the current led-values - and do a higher refresrate without the host cpu putting in the same data again-
some of the multiplexing drivers have internal ram to buffer up even the next data for the next set of multiplexed leds - just to be fast enough with  switching to avoid flicker...
https://www.ti.com/power-management/led-drivers/led-display-drivers/overview.html
some reference designes i found quickly:
https://www.ti.com/tool/TIDA-01615
https://www.ti.com/tool/TIDA-00161

i myself have not done any multiplexing designs jet - (just out of simplicity and no needs for it ;-) )

i  think it is by fare easier to get your raw *3D multiplexing* flicker-free by using a dedicated uC that is connected per spi or i2c or usb to your rPI -
then you only need to only make sure that on this connection you can transfer a full data-set of all 512 pixels fast enough and hopefully without longer interruptions.
and if you use one of the cheaper and fast ones out there - i personally would use an an ItsyBitsyM4  board (ATSAMD51 Cortex M4 - 120 MHz)  as iam a little bit familiar with this chip.
there are a bunch of others out there..
this way you don't have to optimize your code that heavily and get the task done easier ;-)

sunny greetings

stefan

[michael]

Hi Adam.

Do you have experience with fpga programming? If so you could use an ice40 dogs together with symbiflow as toolchain. You can write an fpga logic, that receives the led values via spi and stores it in local memory. The fpga logic could also be written to support brightness values.

There are low cost ice40 devkits lots of love ports. Mine has about 100.

Regards

Michael

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Adam Davies <adam....@tsllighting.com>

Datum: 25.02.21 23:43 (GMT+01:00)

An: open-l...@googlegroups.com

Betreff: Re: [open-lighting] 'Speeding up' ArtNet

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/CAOtnWMCJyxXG8Rwekw%2BCsfKd1-M5b%3DjZ3_5UqahVjyQFk%2BaUzQ%40mail.gmail.com.

[Adam Davies]

Hi Michael,

I have 0 experience with any of that! I've tweaked the python slightly (to iterate through all channels, switch off anything it needs to and keep on what is already on as opposed to switching off all pins then switching back on what it needs to) and I'm hoping that once I finish the cube the flicker will not be as noticeable. If it's still bad, I'll research what you suggested

Thanks

Adam

[Peter Stuge]

Adam,

Adam Davies wrote:
> I've tweaked the python slightly (to iterate through all channels,
> switch off anything it needs to and keep on what is already on as
> opposed to switching off all pins then switching back on what it needs to)
> and I'm hoping that once I finish the cube the flicker will not be as
> noticeable.

Switching once is for sure better than switching twice, but unless
you always switch everything you're not achieving deterministic frame
duration; ie. the update time depends on the contents, which looks
unpleasant if noticeable.

> If it's still bad, I'll research what you suggested

No need to jump to an FPGA for a cube, but the advice to move from
I2C to SPI is excellent.

You can easily clock SPI at 10 MHz, so up to 100 times faster than
I2C, letting you comfortably output the entire frame.

Here's a python library for SPI hardware: (you don't need transact, only write)

https://github.com/tomstokes/python-spi

And you're lucky that there's an SPI version of your IO expander: MCP23S17

Give that a go, you should notice a real difference.


//Peter

[Peter Stuge]

Peter Stuge wrote:
> And you're lucky that there's an SPI version of your IO expander: MCP23S17
>
> Give that a go, you should notice a real difference.

If ~8mA output per pin is sufficient (this depends on your LEDs) you
could remove a lot of overhead by using e.g. 74LVC594A for SPI-to-IO-expansion.

MCP23S17 has two byte overhead for every data byte (200% overhead, ouch!!)
while 74LVC594A has zero.

Connect SPI CS to all STCP pins, SCK to all SHCP pins, MOSI to the first
DS pin and tie all SHR and STR pins high. Daisy-chain by connecting Q7S
on the first 74LVC594A to DS on the second, and so on.

Then call spi.write() from Python with one byte per daisy-chained 74LVC594A;
the SPI hardware shifts out all the bits and the following rising CS edge
makes all 74LVC594A outputs drive the new data.


//Peter

[michael]

You are right. SPI with 9 x 74LVC594A would be the best sollution. 8 for the layer and one for the column. I would create 8 spi transfers in advance and schedule them after each other in a thread using SCHED_RR policy. The second thread fills the SPI buffers with bit7 from each channel. Note that one byte in each SPI is the colmn selector and can be set once on program start.  You might need to switch to c/c++ just for the realtime loop, because of the timing requirements. That can be a small module that gets the channels fed in.

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Peter Stuge <pe...@stuge.se>

Datum: 28.02.21 21:11 (GMT+01:00)

An: open-lighting <open-l...@googlegroups.com>

Betreff: Re: [open-lighting] 'Speeding up' ArtNet

--
The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/20210228201111.14294.qmail%40stuge.se.

[michael]

The FPGA is an overkill for the current use case, but it becomes the right solution once the LEDs should become dimmable. It is easy.to put 64 pm or ppm into the fpga and let it do the scheduling of the columns. It can use a 512byte blockram and receive the 512 intensities via spi or even directly via a dmx512 receives in the fpga.

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Peter Stuge <pe...@stuge.se>

Datum: 28.02.21 19:17 (GMT+01:00)

An: open-lighting <open-l...@googlegroups.com>

Betreff: Re: [open-lighting] 'Speeding up' ArtNet

--
The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/20210228181723.13214.qmail%40stuge.se.

[michael]

Hi Adam.

Can you somehow show the result, for example with a video?

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Adam Davies <adam....@tsllighting.com>

Datum: 28.02.21 17:05 (GMT+01:00)

An: open-lighting <open-l...@googlegroups.com>

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/a3e2a727-fdb3-4a49-9e29-33659b56081fn%40googlegroups.com.

[Adam Davies]

Thanks all for the continued support

I have ordered some of the MCP23S17 chips which will hopefully arrive soon but this brings me back to my original question:

Are there any Python Gurus out there that can help me edit the script? I guess I need to push the values from the cached dictionary to the cube over SPI fast enough that the human eye won't notice the ghosting and just update that dictionary at 30fps, or however often a new ArtNet frame appears

Adam

[Peter Stuge]

Adam Davies wrote:
> I have ordered some of the MCP23S17 chips which will hopefully arrive soon
> but this brings me back to my original question:

Cool! I'm excited to learn about the results.

> Are there any Python Gurus out there that can help me edit the script?

I'm happy to help a bit with the script. The original script you sent
seems to use a couple of files not included and which I don't know
where to find - the database and led libraries in particular, I suppose
that ledinstructions is part of the latter.

And how exactly will you connect the 23S17 chips to the LED matrix?
Is that already fixed or is it still possible to make adjustments?


//Peter

[Adam Davies]

Thanks so much Peter,

Since first posting that code snippet, what I'm now working with is largely different so best to ignore that. Once the chips arrive and I have 'something' that works, I'll re-post the code or link to my GitHub.

In terms of connecting the chips, nothing confirmed yet. What were you thinking??

Adam

[michael]

I strongly recommend using the suggested shift registers (74594 or 16bit 74675) as you only need output and do not need the flexibility and interrupt support of the 23s17.

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Adam Davies <adam....@tsllighting.com>

Datum: 02.03.21 21:11 (GMT+01:00)

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/5e489c71-1183-404c-a226-95e6de26ca05n%40googlegroups.com.

[Peter Stuge]

Adam Davies wrote:
> best to ignore that

Okay.

> In terms of connecting the chips, nothing confirmed yet. What were you
> thinking??

One, daisy-chain the 23S17 all using a single CS.

Two, recommend to connect all LEDs "within" GPIOA and GPIOB, respectively.
It's fine to connect LEDs between GPIOA pins of different chips, and
between GPIOB pins of different chips, but avoid LEDs between any GPIOA
and GPIOB pins. This takes some connection planning.

That gives software more options. It becomes possible to drive some LEDs
by only writing to one of the two GPIO registers. For the cube it may not
actually matter because writing both registers at once provides more
throughput, which is important for looks.

Using 595 would be simpler, but the 23S17 will work well. The software
needs to do more setup for 23S17 but nothing bad.


//Peter

[michael]

You can not chain the mcp23s17 but you can chain the shift registers. I attached the schematic as pdf.

The extreme solution would be to connect the CS to a PWM output with the expected frequency and a duty cycle so that the loading of the parallel registers is done with the precise pwm and with the pwm pin connected to a gpip that is used as an interrupt. The interrupt can then be used to start the next spi transfer. This would create the most efficient and precise solution.

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Peter Stuge <pe...@stuge.se>

Datum: 03.03.21 12:23 (GMT+01:00)

An: open-lighting <open-l...@googlegroups.com>

Betreff: Re: [open-lighting] 'Speeding up' ArtNet

--
The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/20210303112317.11826.qmail%40stuge.se.

[Peter Stuge]

michael wrote:
> You can not chain the mcp23s17

Why do you say that?

Indeed I haven't done it with this particular chip, but like all SPI
devices they have MOSI and MISO pins connected to their internal shift
register, so I expect that it would work well to connect several of
them together, and at runtime take CS low, then clock out data for all
of them and finally take CS high to make all chips drive the new
signals on that rising CS edge.

Have you tried this with MCP23S17 and found that it doesn't work?


//Peter

[michael]

Because it does have a fixed size shift register. You can write multiple consecutive registers at a time because it has an auto increment mechanism for the register address. Also the SO pin is high impedance during a write. You can connect up to 8 mcp23s17 to one CS, but each of that has a 3 bit address and the first byte decides which chip to select and if it is a read or a write. The spi implementation resembles much of the i2c implementation. And your suggestion for the shift registers is so simple and therefor so brilliant.

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Peter Stuge <pe...@stuge.se>

Datum: 03.03.21 14:37 (GMT+01:00)

An: open-l...@googlegroups.com

Betreff: Re: [open-lighting] 'Speeding up' ArtNet

--
The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/20210303133733.14821.qmail%40stuge.se.

[michael]

This is how it could look like.

8x8 pins are for the vertical copper pipes and 8 column pins with mos-fets are for the 8 layers. With 500hz repeat rate you have more than 50hz for each layer. With a 500khz pwm, as I suggested earlier it should be possible to make it flicker free.

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: michael <mic...@cubic.org>

Datum: 03.03.21 20:36 (GMT+01:00)

[Adam Davies]

Peter,

That PCB looks amazing! Do you have the ability to print them too, or just design them?

Is that board design for the MCP23S17 or the chips you suggested?

For the uninitiated, I’ve used SPI before but only for controlling pixel LED tape. How would I go about controlling the other chips? Are there libraries out there or tutorials? 

Apologies for all the questions. Am still very new to this - normally a lighting designer by trade but trying to keep my brain busy during Covid times!

Thanks

Adam

 

On 3 Mar 2021, at 20:43, michael <mic...@cubic.org> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/E1lHYLI-0006NP-7M.spam-suxx-2021-03-03-12-43-36%40roasted.cubic.org.

<LedCube8x8x8-3D.png>

[Arjan van Vught]

Op woensdag 3 maart 2021 om 11:14:58 UTC+1 schreef mic...@cubic.org:

I strongly recommend using the suggested shift registers (74594 or 16bit 74675) as you only need output and do not need the flexibility and interrupt support of the 23s17.

Yes, better to use shift-registers. The 594 (or AHC595) are also much faster than a 23S17. 

- Arjan

http://www.orangepi-dmx.org/orange-pi-dmx512-rdm/ws28xx-configuration/prototype-boards/multi/8-output

[Peter Stuge]

Hi Adam,

Adam Davies wrote:
> Peter,
>
> That PCB looks amazing!

I can't take credit for that - michael posted that PCB image. :)

> Do you have the ability to print them too, or just design them?

I order the PCBs I design from local fabs. I've made my own in the
past, but since I have excellent suppliers nearby, I support them and
I receive fantastic quality.

> Is that board design for the MCP23S17 or the chips you suggested?

Looking at the image, it seems to be a design for the 595. The chips
have fewer pins than the MCP23S17.

> For the uninitiated, I’ve used SPI before but only for controlling pixel
> LED tape. How would I go about controlling the other chips?
> Are there libraries out there or tutorials?

Right, there is a library that I linked to before, for SPI from Python.

But we'll (I'll) have to investigate whether the MCP23S17 can be
daisy-chained or not. See more in the next mail.

If daisy-chaining is not possible then that means more SPI overhead,
but because SPI is 25x-100x faster than I2C that should still be okay.


//Peter

[Peter Stuge]

michael wrote:
> > > You can not chain the mcp23s17
> >
> > Why do you say that?
>

> Because it does have a fixed size shift register. You can write
> multiple consecutive registers at a time because it has an auto
> increment mechanism for the register address.

Aren't most shift registers fixed size? And yes, the auto increment
can be useful within the chip, but I think the 16-bit "Byte mode with
IOCON.BANK = 0" is even more useful in this particular application.

> You can connect up to 8 mcp23s17 to one CS, but each of that has a
> 3 bit address and the first byte decides which chip to select and
> if it is a read or a write.

I noticed those addresses, but if daisy-chaining works then they're
not really neccessary.

> Also the SO pin is high impedance during a write.

> The spi implementation resembles much of the i2c implementation.

It's okay if SO is high impedance while command and data is clocked
into the MCP23S17, the question is whether it will clock out further
bits onto SO.

The documentation is not particularly detailed about this, which is
disappointing, so I've also ordered a few chips to investigate how it
behaves.

In any case, Adam, you'll certainly be able to use the MCP23S17 and
they should perform plenty fast, but worst case there will be 200%
overhead per chip, causing some slow-down, but I think not at all
relevant.

> And your suggestion for the shift registers is so simple and
> therefor so brilliant.

I guess you may have had the same idea michael. ;)


Kind regards

//Peter

[michael]

Hi,

I have completed the PCB design but as I have no 74'594 at hand I came
across the idea to simulate them all with an ICE40HX8K.
If I have or build a model of the mcp23s17 I can even test that circuit.
And it can be used to test the version with 64 PWM/PPMs.

Am 2021-03-04 21:25, schrieb Peter Stuge:
> Hi Adam,
>
> Adam Davies wrote:
>> Peter,
>>
>> That PCB looks amazing!
>
> I can't take credit for that - michael posted that PCB image. :)

I dont care, Adam allready mentioned his mistake, but Peter came up with
the idea to use the shift registers.

>> Do you have the ability to print them too, or just design them?
>
> I order the PCBs I design from local fabs. I've made my own in the
> past, but since I have excellent suppliers nearby, I support them and
> I receive fantastic quality.

I also do not build them myself anymore. To much dirt.

>> Is that board design for the MCP23S17 or the chips you suggested?
>
> Looking at the image, it seems to be a design for the 595. The chips
> have fewer pins than the MCP23S17.

The board is for the '594 but with a small change the '595 can also be
used.

>> For the uninitiated, I’ve used SPI before but only for controlling
>> pixel
>> LED tape. How would I go about controlling the other chips?
>> Are there libraries out there or tutorials?
>
> Right, there is a library that I linked to before, for SPI from Python.
>
> But we'll (I'll) have to investigate whether the MCP23S17 can be
> daisy-chained or not. See more in the next mail.
>
> If daisy-chaining is not possible then that means more SPI overhead,
> but because SPI is 25x-100x faster than I2C that should still be okay.

It should be possible to do it in a combined spi transfer.
I dont know if that is supported in the python library.

The linux spidev uapi supports sending multiple spi-transfers with one
ioctl and you can even controll the behavior of the cs-line.

struct spi_ioc_transfer transfers[5];
transfers[i].cs_change = 1; // deassert cs between each transfer
ioctl(m_fd, SPI_IOC_MESSAGE(5), transfers);

If transfers[0..4] are filled with the access to the 5 mcp23s17 they are
accessed with a very cmall gap inbetween (a few ns to maybe a us).
It would feel as if they are accessed at the same time.

[Peter Stuge]

michael wrote:
> Hi,I have completed the PCB design but as I have no 74'594 at hand I

> came across the idea to simulate them all with an ICE40HX8K.

The iCE40 is a great idea for scaling up - but remember that its IO
pins can't source/sink nearly as much current as e.g. 74LVC;
a few mA vs. ~20 mA. Probably enough for high-brightness LEDs though.

And watch out for Imax for the whole chip. It could get hot. :)


//Peter

[Peter Stuge]

Hi Adam,

Peter Stuge wrote:
> > Also the SO pin is high impedance during a write.
>

> It's okay if SO is high impedance while command and data is clocked
> into the MCP23S17, the question is whether it will clock out further
> bits onto SO.

So the MCP23S17 arrived and I've investigated how it behaves.

michael's understanding of the chip was unfortunately exactly right -
my guess that the MCP23S17 could be daisy-chained proved wrong.

Unlike other SPI chips I've seen this one doesn't shift bits through
to SO if it receives "too many" bits. Annoying. But all is not lost.

The chip has three address pins, so you can still connect up to 8 of
them together for up to 128 outputs, just with more overhead.

If I remember correctly you need 72 outputs; that's 5 of them?

Attached is a schematic showing how they would be connected;
SPI in parallel but the addresses differ.

Attached is also some Python code to blink LEDs connected from GPA1
to GPA0, and so on, on the first chip, using the linked spi.py.

I hope this makes some sense?


//Peter

[michael]

It supports a 3wire SPI where you can tie miso  and mosi  together. I have made some schematics and a test program for both variants, the 5 mcp23s17 as well as the shift registers. I made some traces from running it on a rpi2.

Http://stickel.org/LightingConsoleHardware/LedCube/ledcube-1.0.tar.gz

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Peter Stuge <pe...@stuge.se>

Datum: 06.03.21 23:27 (GMT+01:00)

An: open-l...@googlegroups.com

Betreff: Re: [open-lighting] 'Speeding up' ArtNet

--
The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/20210306222731.22853.qmail%40stuge.se.

[Adam Davies]

Hi Peter,

How did you manage to get an MCP23S17 chip so quickly?! I'm guessing from your email address that you're in Sweden? Without paying a silly amount, I was only able to find ones in China for sale here in England so I'm still waiting for it to arrive...

You're correct yes, I need 72 outputs (64 columns + 8 layers (with each layer having a transistor on the end to turn it 'on')) so it would be 5 chips. The code makes sense (even though I can't run it yet). Now I just need to work out how to combine that with the OLA's dmxreceive Python example.

I've uploaded my progress with the MCP23017 I2C chip so far to https://github.com/napsterxp/ArtNet-LED-Cube so you can see the route I was taking. I'm yet to begin adjusting it for the 'S' chip. I know it's not the most elegant piece of code ever, but everyone has to start somewhere!

Adam

[Peter Stuge]

michael wrote:
> It supports a 3wire SPI where you can tie miso  and mosi  together.

Right, unfortunately it supports _only_ that. Oh well.

> I have made some schematics and a test program for both variants,
> the 5 mcp23s17 as well as the shift registers.

They look great, thanks for sharing! Much more complete than my
basic illustration.

> I made some traces from running it on a rpi2.

I don't have the Saleae software at hand but setting the scheduler
priority is a good idea; it should reduce jitter some.

I'm curious whether there's any visible difference between a C/C++
implementation and a Python one, with or without scheduler priority.


//Peter

[michael]

You can download the Saleae software for free and load the files with it, even without an interface.

Von meinem Samsung Galaxy Smartphone gesendet.

-------- Ursprüngliche Nachricht --------

Von: Peter Stuge <pe...@stuge.se>

Datum: 07.03.21 13:37 (GMT+01:00)

An: open-l...@googlegroups.com

Betreff: Re: [open-lighting] 'Speeding up' ArtNet

--
The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/20210307123742.31512.qmail%40stuge.se.

[Peter Stuge]

Hi Adam,

Adam Davies wrote:

> How did you manage to get an MCP23S17 chip so quickly?!

I checked what was in stock at a component distributor where I needed
to order some other parts anyway, and included them with that order.

> I'm guessing from your email address that you're in Sweden? Without
> paying a silly amount, I was only able to find ones in China for sale
> here in England so I'm still waiting for it to arrive...

Here's a component search engine showing distributor stock:

https://www.findchips.com/search/MCP23S17

I believe at least RS Components and Farnell have UK warehouses. Some
distributors may not sell to private individuals, but I believe RS does.
Digi-Key and Mouser should do so too, but they have US warehouses so
that's another day and customs and such.

> You're correct yes, I need 72 outputs (64 columns + 8 layers (with each
> layer having a transistor on the end to turn it 'on')) so it would be 5
> chips. The code makes sense (even though I can't run it yet).

Great. Code to translate from channel values to register bytes is
obviously missing, but since that depends on how you connect the LEDs
I made a simple example with just a single LED.

> Now I just need to work out how to combine that with the OLA's dmxreceive
> Python example.

Let me know if you get stuck.

One challenge is that there are two sources of events; on one hand data
arrives from OLA, on the other the output function needs to cycle through
the LED matrix repeatedly.

Data might arrive at say 30 Hz but the output function should run
more frequently, and independently of whether OLA receives channel
values, so the two shouldn't be linked in time.

Since the receive function is the source of new channel values and because
it runs less frequently that should do all the work to prepare the bytes
to be sent to the IO chips by the output function.

With the OLA code you can use wrapper.AddEvent() instead of repeat() in
my example, but you'll have to also add a call to wrapper.AddEvent()
first thing in the output function to make it repeat.

Note that wrapper.AddEvent() expects milliseconds instead of seconds.

Sub-10 ms delay is probably too short in user-space, but well, try it out. :)

> I've uploaded my progress with the MCP23017 I2C chip so far to
> https://github.com/napsterxp/ArtNet-LED-Cube so you can see the
> route I was taking.

Thanks! Okay, now I understand it.

> I'm yet to begin adjusting it for the 'S' chip.

I'd recommend starting fresh, because wiringpi will be replaced with
spi output and because the second time around results are usually
significantly better, for me at least.

> I know it's not the most elegant piece of code ever, but everyone
> has to start somewhere!

There's a very good saying "First make it work, then make it fast".
Your code works, but can't become fast enough because I2C holds it back.

For elegance, one powerful concept is to use variables/data to express
all possible differences in repetitive code, allowing for much shorter
code, but don't use more complex variables/data than required, so that
the code also stays simple.

Don't worry, you're on the right track!


//Peter

[michael]

Hacked together a web page containing the essence of the discussion about Led cube and to have a place to share the boards I derived off of the discussion.

Http://stickel.org/LedCube/

[Adam Davies]

Evening all,

Loving the webpage! You can get the 23S17 chip from RS or Mouser here in England but they charge huge amounts for postage (Mouser charge around £1 for the chip and £14.95 for postage and RS charge around £10 for postage) so I'm trying to be patient and save some money waiting for some from eBay/China until my work is next doing an RS order and I might 'accidentally' add some to the end. I've now tidied the GitHub slightly, adding a separate folder for any progress I make on the SPI version.

Peter - I have included your Python as an example on Github. I hope you don't mind? Please let me know if you'd like it removed. Have you got any more information on wrapper.AddEvent or an example? I'm really struggling to see how it would fit in which I think is due to lack of knowledge and understanding of how the other wrapper.xyz() functions work.

Currently struggling to work out if this can all be contained within one Python script or if I need one script that's receiving the ArtNet frame and storing it *somewhere* , (no idea where) and a separate script that pushes this info to the LEDs. Would it be possible to take Michael's work and 'make' it into a Python library with a function that's able to cycle through the array created from the ArtNet frame if you first define what pins correspond to what channels in the cube?

Am going to revisit in the morning with a fresh set of eyes, some more sleep and some more coffee.

Thank you so much to you all for your efforts, I never thoughtmy first post would get anymore interest than a link to a StackOverflow article somewhere!

Adam

--

The Open Lighting Project: open-l...@googlegroups.com, #openlighting (irc.freenode.org)
To unsubscribe from this group, send email to open-lightin...@googlegroups.com
For more options, visit https://groups.google.com/groups/opt_out?hl=en
---
You received this message because you are subscribed to the Google Groups "open-lighting" group.
To unsubscribe from this group and stop receiving emails from it, send an email to open-lightin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/open-lighting/E1lIwsR-0006mf-UQ.spam-suxx-2021-03-07-09-07-36%40roasted.cubic.org.

[Peter Stuge]

Adam Davies wrote:
> Peter - I have included your Python as an example on Github. I hope you
> don't mind?

Don't mind at all.

> Have you got any more information on wrapper.AddEvent or an example?

Sure.

> I'm really struggling to see how it would fit in which I think is due
> to lack of knowledge and understanding of how the other wrapper.xyz()
> functions work.

So wrapper itself, a ClientWrapper object, wraps around the client
that communicates with OLA. The wrapper adds among other things event
handling, which everyone using the client will need at a minimum
to recognize when OLA has made new channel values available, but
which also allows to add other, time-based, events.

> Currently struggling to work out if this can all be contained within
> one Python script

Because time-based events can be added to the wrapper using AddEvent()
the answer to this is yes! It's possible to rework my SPI example and
combine it with the OLA receive example with a few changes:

* remove the repeat() function since we'll use wrapper.AddEvent() instead
* for clarity, rename the SPI data variable to spidata
* call wrapper.AddEvent() first thing in the output() function, because
AddEvent() events are one-shot, not repeating, first thing to achieve
the desired frequency as closely as possible
* remove the call to the repeat() function
* add https://www.openlighting.org/ola/developer-documentation/python-api/ code
* call wrapper.AddEvent() before wrapper.Run() to run output() the first time

Attached is what the result could look like. Compare it to the
previous version, and see if the changes make sense.

Note that NewData() does not process channel values at all, it just
prints them. So while this hopefully makes both channel reception and
SPI output run in one script, the two are not (yet) connected.

> or if I need one script that's receiving the ArtNet frame and
> storing it *somewhere* , (no idea where) and a separate script that
> pushes this info to the LEDs.

This is an excellent thought! It is very much possible, and this
would have been my suggested solution in case the wrapper had not
supported adding other, non-OLA events.

In that case I'd have suggested for the receiver to still do the
neccessary processing for the 23S17 chips and your particular LED
connections before it stores the calculated GPIO values in a region
of shared memory, which could be read by the second script and then
output to SPI. But it looks like we can get away with a single script.

> Would it be possible to take Michael's work and 'make' it into a
> Python library with a function that's able to cycle through the
> array created from the ArtNet frame if you first define what
> pins correspond to what channels in the cube?

Not easily a Python library, but they could be two separate programs,
both would need to be started individually, and then it would be
possible for them to cooperate and share data. But since Michael's
program already has a fair amount of functionality, I would probably
suggest for it to connect directly to OLA instead.

It might turn out that Python itself is causing too much overhead for a
pleasant result, in which case that's certainly worth pursuing, but
for me it's interesting to see what we can get out of pure Python code.


Kind regards

//Peter

[Peter Stuge]

Adam Davies wrote:
> You're correct yes, I need 72 outputs (64 columns + 8 layers (with each
> layer having a transistor on the end to turn it 'on')) so it would be 5
> chips.

All right. Please make sure to use 4 chips for all columns and connect
all 8 transistors to one single GPIO port of the fifth chip, leaving
the other GPIO port of the fifth chip unused in the cube.

You can use that last port for other things though, maybe status LEDs,
like ArtNet signal reception.


Connecting the cube in that way allows the shortest SPI output; 6
transactions instead of 10, a significant improvement. SPI output is
the bottleneck ultimately determining whether the cube has visible
flicker or not, so this helps a lot. The SPI transactions are:

* switch off all transistors (only one should ever be on, but why not)
* set 4x 16 columns for next layer
* switch on transistor for next layer


//Peter