DMX512 Generated directly by Raspberry Pi board

[jonshouse1]

Some people may be interested in this

http://www.jonshouse.co.uk/rpidmx512.cgi

Thanks,
Jon

[Jannis Achstetter]

Hi Jon,

Am 25.05.2014 21:00, schrieb jonshouse1:
> Some people may be interested in this
>
> http://www.jonshouse.co.uk/rpidmx512.cgi

this is very interesting news.
Since you're not using the UARTs does it mean we could drive several
universes? One via GPIO? (no reading and as such no direction selection
needed)

Best regards,
Jannis

[jonshouse1]

Yes, I don't see why not. The overhead of clocking N bits in parallel should be only slightly greater than clocking one stream.  I am rapidly finding the limitations of this technique though.  It makes X unusable and mp4 playback is also a no go.  But if what you are after is ethernet to N x DMX as a bridge I see no reason why this would not work, just hold enable true on all the modules and clock them in parallel.

Thanks,
Jon

[Richard Ash]

On Sun, 25 May 2014 15:44:39 -0700 (PDT)
jonshouse1 <j...@jonshouse.co.uk> wrote:

> On Sunday, 25 May 2014 22:38:36 UTC+1, Kripton wrote:
> >
> > Hi Jon,
> >
> > Am 25.05.2014 21:00, schrieb jonshouse1:
> > > Some people may be interested in this
> > >
> > > http://www.jonshouse.co.uk/rpidmx512.cgi

It's an interesting approach, but the non-compatibility with audio
playback wouldn't have worked for what I did with mine [1], but I got
the UART approach working (very close to merging into OLA trunk, when I
get some re-arrangement into library functions done).
https://github.com/richardash1981/ola

[1]:
http://eastertrail.blogspot.co.uk/2014/04/command-and-control-ii.html

Richard

[Richard Ash]

On Wed, 28 May 2014 22:52:35 +0100
jon <j...@jonshouse.co.uk> wrote:
Sorry for the delay getting back to you on this - I have been

> I do not know OLA in any detail, so have need to read my way through
> the source tree to know where to find the code. I am also not a great
> fan of c++ hence my lack to desire to poke around in the OLA
> codebase :-)
All in the plugins/uartdmx/ directory within the source. It's mostly a
copy of the FTDI output module, but tweaked to use the standard UART
calls rather than the FTDI-specific library. I started off with a
stand-alone C demo program which I should be able to find and send to
you if it's useful.

> I did work out what I did wrong with my UART code but have not fixed
> it yet. I forget to toggle the GPIO pin between the UART and GPIO
> modes, as a result the BREAK and MAB I generated performed no action
> - though it did output the serial stream at the correct 250k bit.
Bit confused by this - you should be able to generate BREAK and MAB
with ioctls() to the UART without any extra gpio pins needed.

> I added this to the little Pi gpio interface I was using:
>
> // Set a (possibly high) baud rate without linux kernel knowing about
> it. Useful for DMX maybe ? // To get 250k buad you must add
> "init_uart_clock=16000000" to /boot/config.txt and pass 4,0 to this
> function set_baudrate_divisors(unsigned int divisorwhole, unsigned
> int divisorpartial) { *(gpio+1024) = divisorwhole;
> *(gpio+1028) = divisorpartial;
> }
You can select 250kbaud mode in a slightly more standard way (it works
for any (supported) UART on Linux, but only on Linux) with the termios2
interface (which accepts baud rate as an integer in bits per second,
rather than symbolic constants). This is in UartLinuxHelper.cpp in my
sources, which is really just a C function wrapped up into a static
method.

> Its all pretty crude, I suspect I can make it work if I put a couple
> of hours into it but I got distracted with other projects.
>
> The one problem I cant see a solution for is that when outputting the
> DMX values the Linux scheduler could come back late and fail to fill
> the UART in time, do you see any flashes from your code (miss timed
> packets) or does it look rock solid?
It's rock solid for me, with the proviso that I know the LED fitting I
am using will hold state indefinitely if the DMX input is removed (it
doesn't drop back to black on no signal). So I am confident that the
513 bytes of DMX lighting data go out as a block, with no glitches in
them. I agree that the parts of the waveform timed by the Linux
userspace scheduler may be late, but this should only affect the
durations of the break, mark-after-break and inter-frame gaps (the
things timed with usleep()). The DMX spec is (I think) relatively
forgiving about the upper limit of these timing elements - certainly
the timing does vary, you can see this if you have a 'scope on the DMX
line.

What has to be consistent is the run of 513 bytes in a frame,
which have to be (close to) back to back in order to stop the devices
dropping out of lock and reading the rest of the frame as a new frame
(which certainly will flicker!). I think this shouldn't be a problem
using the Pi's built-in UART, because the UART has both a hardware
buffer and a direct interrupt line from the UART to the processor (they
being on the same chip!).

So when my code calls write(), 513 bytes go
into a buffer in the Linux kernel. The kernel driver for the UART then
pushes 16 bytes out to the hardware UART, which starts putting them on
the wire. As that 16 byte buffer empties, an interrupt is raised by the
UART to get it re-filled. This causes the Linux kernel to stop what it
is doing, and put some more of the 513 byte kernel buffer contents into
the 16 byte UART buffer, and repeat until all is done. This is all done
in kernel mode, and I think all in an interrupt (I haven't looked at
the actual driver, but would expect this to be the case). As a result,
the user space scheduler doesn't come into it, and the only question is
what the IRQ latency of the Linux kernel on the Pi is - which I would
expect to be very small compared to the time taken to push out 16 bytes
at 250kbaud (576 microseconds, if my maths is right).

This argument doesn't work for USB-connected serial adaptors, because
they don't have hardware interrupt lines - they rely upon the USB host
controller polling the peripherals, which inevitably has higher
latency. So you need bigger buffers in the USB peripheral, or you get
glitches in the output when the USB doesn't fill the buffer in the
serial converter in time. I haven't played with this, but I can see
it's much harder to get to work.

The disadvantage is you can only have one UART on the Pi because that
is all the hardware there is - your approach can be scaled up because
there are more GPIO pins.

I was wondering about using the GPIO blaster technique to produce
analogue outputs from the Pi, so it could act as an
ethernet-to-analogue-dimmer bridge unit (like a DMX demux, but
straight from Ethernet) ...

Richard

[Peter Stuge]

Richard Ash wrote:
> an interrupt is raised by the UART to get it re-filled. This causes
> the Linux kernel to stop what it is doing

That's not neccessarily true at all. Linux is a general purpose system
which does not promise you that it will stop what it is doing when there
is a UART IRQ. As an example, a higher priority IRQ may be keeping the
processor busy for long enough that your wire timing breaks, and you'll
never know.

If you are implementing a protocol which relies on timing then please
save yourself and others some trouble and use an environment which will
indeed guarantee the required timing. Linux is not it.


//Peter

[jon]

On Sat, 2014-05-31 at 12:34 +0100, Richard Ash wrote:
> On Wed, 28 May 2014 22:52:35 +0100
> jon <j...@jonshouse.co.uk> wrote:
> Sorry for the delay getting back to you on this - I have been
>
> > I do not know OLA in any detail, so have need to read my way through
> > the source tree to know where to find the code. I am also not a great
> > fan of c++ hence my lack to desire to poke around in the OLA
> > codebase :-)
> All in the plugins/uartdmx/ directory within the source. It's mostly a
> copy of the FTDI output module, but tweaked to use the standard UART
> calls rather than the FTDI-specific library. I started off with a
> stand-alone C demo program which I should be able to find and send to
> you if it's useful.
>
> > I did work out what I did wrong with my UART code but have not fixed
> > it yet. I forget to toggle the GPIO pin between the UART and GPIO
> > modes, as a result the BREAK and MAB I generated performed no action
> > - though it did output the serial stream at the correct 250k bit.
> Bit confused by this - you should be able to generate BREAK and MAB
> with ioctls() to the UART without any extra gpio pins needed.

Thanks, I noticed this. I discarded my bit twiddling and changed it for
the ioctl instead :-)

> > I added this to the little Pi gpio interface I was using:
> >
> > // Set a (possibly high) baud rate without linux kernel knowing about
> > it. Useful for DMX maybe ? // To get 250k buad you must add
> > "init_uart_clock=16000000" to /boot/config.txt and pass 4,0 to this
> > function set_baudrate_divisors(unsigned int divisorwhole, unsigned
> > int divisorpartial) { *(gpio+1024) = divisorwhole;
> > *(gpio+1028) = divisorpartial;
> > }
> You can select 250kbaud mode in a slightly more standard way (it works
> for any (supported) UART on Linux, but only on Linux) with the termios2
> interface (which accepts baud rate as an integer in bits per second,
> rather than symbolic constants). This is in UartLinuxHelper.cpp in my
> sources, which is really just a C function wrapped up into a static
> method.

Ok, I made your baud rate function work for me :-) Thanks. You are
correct about #include problems. I spent a couple of hours trying to mix
termios and termios2 in the same source, getting compile errors or
segfaults. In the end I put the buad rate function in a file of its own
and compiled it stand alone then linked it against my code.
It all works a treat now.

>
> > Its all pretty crude, I suspect I can make it work if I put a couple
> > of hours into it but I got distracted with other projects.
> >
> > The one problem I cant see a solution for is that when outputting the
> > DMX values the Linux scheduler could come back late and fail to fill
> > the UART in time, do you see any flashes from your code (miss timed
> > packets) or does it look rock solid?
> It's rock solid for me, with the proviso that I know the LED fitting I
> am using will hold state indefinitely if the DMX input is removed (it
> doesn't drop back to black on no signal). So I am confident that the
> 513 bytes of DMX lighting data go out as a block, with no glitches in
> them. I agree that the parts of the waveform timed by the Linux
> userspace scheduler may be late, but this should only affect the
> durations of the break, mark-after-break and inter-frame gaps (the
> things timed with usleep()). The DMX spec is (I think) relatively
> forgiving about the upper limit of these timing elements - certainly
> the timing does vary, you can see this if you have a 'scope on the DMX
> line.
>

Yes, looked with a scope and BREAK and MAB are a little variable in
length, but so far the lights seem ok with that. I assume olc already
does this but I am marking my process real time with the scheduler,
seems to improve the timings a bit
#include <sched.h>
void set_realtime(void)
{
struct sched_param sparam;
sparam.sched_priority = sched_get_priority_max(SCHED_FIFO);
sched_setscheduler(0, SCHED_FIFO, &sparam);

}


> What has to be consistent is the run of 513 bytes in a frame,
> which have to be (close to) back to back in order to stop the devices
> dropping out of lock and reading the rest of the frame as a new frame
> (which certainly will flicker!). I think this shouldn't be a problem
> using the Pi's built-in UART, because the UART has both a hardware
> buffer and a direct interrupt line from the UART to the processor (they
> being on the same chip!).
>
> So when my code calls write(), 513 bytes go
> into a buffer in the Linux kernel. The kernel driver for the UART then
> pushes 16 bytes out to the hardware UART, which starts putting them on
> the wire. As that 16 byte buffer empties, an interrupt is raised by the
> UART to get it re-filled. This causes the Linux kernel to stop what it
> is doing, and put some more of the 513 byte kernel buffer contents into
> the 16 byte UART buffer, and repeat until all is done. This is all done
> in kernel mode, and I think all in an interrupt (I haven't looked at
> the actual driver, but would expect this to be the case). As a result,
> the user space scheduler doesn't come into it, and the only question is
> what the IRQ latency of the Linux kernel on the Pi is - which I would
> expect to be very small compared to the time taken to push out 16 bytes
> at 250kbaud (576 microseconds, if my maths is right).

Yes, all sounds correct ish. Whatever the IRQ is doing the buffer fill
is always at the mercy of linux scheduler. With luck the driver top half
has buffered the 513 bytes and the bottom half of the driver refills the
UART when it gets a fifo empty interrupt, but its a big assumption to
assume so without looking at the code (no I also could not be bothered
to go and look !)

The Linux kernel seems much more real time than it used to be but beware
pauses and strange effects. The Pi has some pauses caused by something
the Broadcom firmware is doing, it can be turned off by using
disable_pvt=1 is /boot/config.txt. I noticed regular scheduler pauses
until someone pointed this one out to me.

> This argument doesn't work for USB-connected serial adaptors, because
> they don't have hardware interrupt lines - they rely upon the USB host
> controller polling the peripherals, which inevitably has higher
> latency. So you need bigger buffers in the USB peripheral, or you get
> glitches in the output when the USB doesn't fill the buffer in the
> serial converter in time. I haven't played with this, but I can see
> it's much harder to get to work.

I wrote a driver for uDMX (does 30 frames per second of 512 channels)
and the Enttec units (43 frames per second of 512 channels). No real
problems. USB host controller is using DMA (I think, Linux does on most
platforms..) and you are pushing data from the host (Pi board) to the
USB device so I doubt that much polling needs to be done. USB is meant
to be real (ish) time so I guess its been designed not to use deep
buffering. The latency looked ok, I an doing light effects synced to
sound and didn't notice any problems.

> The disadvantage is you can only have one UART on the Pi because that
> is all the hardware there is - your approach can be scaled up because
> there are more GPIO pins.
>
> I was wondering about using the GPIO blaster technique to produce
> analogue outputs from the Pi, so it could act as an
> ethernet-to-analogue-dimmer bridge unit (like a DMX demux, but
> straight from Ethernet) ...

It is limited to 1us (1Mhz). That may be fast enough to produce some
kind of meaningful PWM. The snag is an analogue dimmer for mains
lighting really should be synchronized with the mains itself for most
designs (zero cross). I don't see how the Pi could do that.
If you just wanted a set of PWM outputs from the Pi a PCA9685 based
board might be a better way. I was intending to use this for light
control until I found some cheap 18 and 27 channel low side DMX driver
boards on ebay.

http://www.adafruit.com/product/815

To do proper mains dimming I would use an microcontroller and sync the
PWM to the mains zero cross and switch with Triacs. You could feed SPI
or serial or something at the Pi to control that.

Thanks for your help,
Jon

[Richard Ash]

On Sat, 31 May 2014 14:16:24 +0100
jon <j...@jonshouse.co.uk> wrote:
> Ok, I made your baud rate function work for me :-) Thanks. You are
> correct about #include problems. I spent a couple of hours trying to
> mix termios and termios2 in the same source, getting compile errors or
> segfaults. In the end I put the buad rate function in a file of its
> own and compiled it stand alone then linked it against my code.
> It all works a treat now.

I think I had a comment about the need to not-include-any-other-headers
- I've certainly been through that experience, it's why it is a helper
all unto itself!

> Yes, looked with a scope and BREAK and MAB are a little variable in
> length, but so far the lights seem ok with that. I assume olc already
> does this but I am marking my process real time with the scheduler,
> seems to improve the timings a bit
> #include <sched.h>
> void set_realtime(void)
> {
> struct sched_param sparam;
> sparam.sched_priority = sched_get_priority_max(SCHED_FIFO);
> sched_setscheduler(0, SCHED_FIFO, &sparam);
> }

The dmx4linux source code I started from had code for this, but I've
never got that far - I think there are some ways to crash the machine
if you don't have comprehensive error handling on the realtime thread.
Probably should be added with a config file option to control it though.

> Yes, all sounds correct ish. Whatever the IRQ is doing the buffer
> fill is always at the mercy of linux scheduler. With luck the driver
> top half has buffered the 513 bytes and the bottom half of the driver
> refills the UART when it gets a fifo empty interrupt, but its a big
> assumption to assume so without looking at the code (no I also could
> not be bothered to go and look !)
>
> The Linux kernel seems much more real time than it used to be but
> beware pauses and strange effects.

Been here in the past. I found that the more often your code yeilded to
the scheduler, the more likely it was to get run when you wanted it.
Put the other way, hogging the CPU is penalised by the scheduler quite
strongly. I had to fix this in a graphics render to make text crawl
across the screen nicely - without the sleeps, it was smooth for a bit,
then jumped horribly.

> The Pi has some pauses caused by
> something the Broadcom firmware is doing, it can be turned off by
> using disable_pvt=1 is /boot/config.txt. I noticed regular scheduler
> pauses until someone pointed this one out to me.

Good to know if it's causing anyone problems.

> > I was wondering about using the GPIO blaster technique to produce
> > analogue outputs from the Pi, so it could act as an
> > ethernet-to-analogue-dimmer bridge unit (like a DMX demux, but
> > straight from Ethernet) ...
> It is limited to 1us (1Mhz). That may be fast enough to produce some
> kind of meaningful PWM. The snag is an analogue dimmer for mains
> lighting really should be synchronized with the mains itself for most
> designs (zero cross).

I meant just filtered to a 0 to 10V DC output to feed into an existing
(legacy) lighting dimmer, replacing a bunch of physical faders with
something more automatable. I agree dimming the mains directly isn't
possible!

> I don't see how the Pi could do that.
> If you just wanted a set of PWM outputs from the Pi a PCA9685 based
> board might be a better way. I was intending to use this for light
> control until I found some cheap 18 and 27 channel low side DMX driver
> boards on ebay.
>
> http://www.adafruit.com/product/815

I've got one of those as well, for the same idea, but not built it yet.
I just wondered if a fewer parts approach was possible! Would be
relevant to small systems (think lots of Pis, each close to the lights
controlled) in particular.

> To do proper mains dimming I would use an microcontroller and sync the
> PWM to the mains zero cross and switch with Triacs. You could feed SPI
> or serial or something at the Pi to control that.

I think this is how most modern DMX dimmers work. Oddly, 15 years ago
it wasn't - most had a DMX-to-analogue board, feeding into an
all-analogue dimmer which used variable phase delays to drive the
TRIACs. Always seemed like a lot of hard work.

Richard

[jon]

> I've got one of those as well, for the same idea, but not built it yet.
> I just wondered if a fewer parts approach was possible! Would be
> relevant to small systems (think lots of Pis, each close to the lights
> controlled) in particular.

Yes, I see no reason why not. If a PWM sequence was setup via the
pigpio library and set to repeat (important) then it should work well.
Just build new PWM waveforms and replace the repeating one to update the
lights :-)

It would also be possible to make a multi-channel DMX output by using
GPIO in parallels. The DMA engine is designed to toggle a 32 bit word
each time it clocks so updating multiple GPIO lines would only cause a
little extra CPU workload and zero extra DMA work. It would be possible
to take multiple DMX universes from ethernet and bridge them to multiple
RS485 converters, that might be an interesting experiment.

As a technique for day to to day use though the UART wins hands down.
the DMA code uses 30% CPU and does not play nice with audio and video.
The UART code uses 3% and works ok when playing video, though the update
rate becomes a little erratic.

Jon

[Woody Poulard]

Is it possible to use this GPIO hardware configuration with the OLA GPIO plugin once updated?

[Arjan van Vught]

For the bare-metal programmers, I have two versions for DMX512 directly by Raspberry Pi board. Examples :

*. Send function : main.c
*. IRQ continues output : main.c

Op zondag 25 mei 2014 21:00:17 UTC+2 schreef jonshouse1:

[Peter Newman]

Hi Woody,

I don't know exactly which GPIO hardware config you mean, but the GPIO plugin just allows individual pins to be turned on or off based on incoming DMX from an OLA universe currently.

[Woody Poulard]

Peter,

Thank you for your reply! I am attempting to send incoming art-net signal to the GPIO pins so that I may connect to a dimmer using a standard 3-pin XLR cable. I do see the GPIO plugin in OLA however the documentation is a little rough and im not sure how to implement it. Any insight on this would be great. I was under the impression that one would need to use a certain configuration of the hardware to do this.

[Peter Newman]

Ah okay, you want to output DMX (I assume that's what you mean by 3-pin XLR; technically the standard is 5 pin) from the Pi.

Currently we only support DMX generation directly from the Pi using it's onboard UART via the UART DMX plugin; as far as I know Jon hasn't submitted a Pull Request for multiple universes of bit banged DMX via the GPIO pins.

[jon]

On Mon, 2015-03-02 at 10:49 -0800, Peter Newman wrote:
> Ah okay, you want to output DMX (I assume that's what you mean by
> 3-pin XLR; technically the standard is 5 pin) from the Pi.
>
>
> Currently we only support DMX generation directly from the Pi using
> it's onboard UART via the UART DMX plugin; as far as I know Jon hasn't
> submitted a Pull Request for multiple universes of bit banged DMX via
> the GPIO pins.

I use my own lighting system and have never had change to even try OLA
yet alone develop for it. I offered the code to the group for reference
in case anyone wanted to pick up the technique.

As I said in my description really the only advantage of the technique
would be to generate multiple simultaneous DMX streams from a single
board.

Interestingly the DMA technique is not bit-banged in the conventional
sense as the wave table is clocked by the DMA controller in the RPI SOC
rather than by the processor itself.

If only one stream is required then the UART technique is best.

The physical interface described on my web page should also work with
the OLA code, has anyone tried it ?

http://www.jonshouse.co.uk/rpidmx512.cgi

Thanks,
Jon

[Bastien Vide]

Hi!

I just wanted to know if there is any news on GPIO plugin for OLA. I just bought a TTL to RS485 without really knowing what I was doing...

So I just wanted to know how to generate DMX512 via GPIO, or if I can use UART and plug it to the board I just bought (I assume it obviously shouldn't). 

Also, after some research, I was thinking about buying this: https://www.sparkfun.com/products/10124

I think it would work with UART, but could someone confirm my thoughts?

Thank you for all the help!

Once I will have built my awesome ArtNet2DMX interface, I'll make some tutorials on OLA, as it isn't really documented (especially for plugins like the UART or the GPIO one) and hard to understand for electronics noobs, like me :(

[Arjan van Vught]

Hi,

The only option we currently have is to use the UART. I have created an explanation here https://sites.google.com/site/rpidmx512/raspberry-pi-ola-dmx512-sender

- Arjan

Op vrijdag 5 juni 2015 11:40:02 UTC+2 schreef Bastien Vide:

[Simon Newton]

Bastien: take a look at the uartdmx OLA plugin that Richard added. He
has a good write up at
http://eastertrail.blogspot.co.uk/2014/04/command-and-control-ii.html

Simon

> --
> The Open Lighting Project: open-l...@googlegroups.com, #openlighting
> (irc.freenode.org)
> To unsubscribe from this group, send email to
> open-lightin...@googlegroups.com
> For more options, visit https://groups.google.com/groups/opt_out?hl=en