Delay between RGB color being commanded vs sent to FAST hardware

[Brian Cox]

Hi guys,

This is after updating to MPF 0.31.3 (latest), and working out some bugs in the virtual game, I'm finally re-attaching to hardware.

What I'm finding is that there is a delay (about 12 seconds) between an RGB color being commanded versus when it actually gets send out to the FAST controller (log attached).

For the sake of simplicity, you can track this one event (I can point out others if needed) where a target has a yellow light blinking in front of it, the switch is hit, and then the light should go green.

• The switch is "dropTargetTM5" (0x24) with a tagged event of "sw_tatMyst_5". This first gets hit around the time 21:29:51.568.
  
• The LED "lightTatMyst_5" (0x05) gets the command to turn green (0,255,0) at time 21:29:51:572 (less than a second later, good).
• The next "RS:" output to the FAST controller occurs at time 21:29:51.579. At this time you can see this LED is getting "05f0f000", which is almost full-on yellow (what it's supposed to be up until this time).
• At time 21:29:51.724 you see the LED goes to it's low-yellow state (which is the blinking show it had before being hit) of "05202000".
• It's not until time 21:30:03.819 that the FAST "RS:" command actually gets the first correct value for the LED, which is "0500ff00".

So, over twelve seconds elapse between the time that the green color was commanded for this LED at 21:29:51.568, and the time that the "RS:" command to the FAST controller is showing the correct value, 21:30:03.819

In this log, there should be at least four or five LEDs ("lightTatMyst_#") that change from blinking yellow to solid green on a "sw_tatMyst_#" event, but I've only pointed this one out for simplicity.

I see this effect across the board, for all light shows, including attract modes, etc...

Let me know if you need more info,

Thanks,

BC

[Brian Madden]

I think you're experiencing the results of the new "flow control" features we added to the FAST platform in 0.31.

The FAST hardware does not do flow control, so we found that at times, MPF was sending data to the FAST controller quicker than it could process it, and we were filling up the buffer which led to dropped packets. (This was the root cause of what was causing some people to experience switch debounce settings not being applied to all switches.)

So what we do in 0.31 is keep track of how many commands we send and how many acknowledgement responses we get back to make sure we don't get too far ahead.

I searched your log for "flow" and found lots of entries like this:

2016-09-25 21:31:12,343 : DEBUG : FAST : Enabling Flow Control for RGB connection. Messages in flight: 4, Max setting: 3

So that's what's causing this here.

It looks like you have 158 LEDs? In my testing, I found that the FAST controller could handle 128 LEDs at 50fps, and 256 at 30fps. So the first thing I would do is lower the update rate of your LEDs too. The default is 50 times per sec, but fast does hardware fades, so really 50 is probably overkill. Try lowering that to 40 or 30 or something. That setting is in the mpf: section of your config, called default_led_hw_update_hz:. Then search the debug log until you don't get those messages.

Also you can adjust the setting for the flow control to the RGB processor by adding an rgb_buffer: setting to your fast: section. The default is 3, so you can try increasing that which will let MPF send more messages faster. You can try 4, 5, 6, etc.. until you don't see those messages. If you want to disable this feature all together, just enter some huge number like 100. (But in that case you'll probably drop some packets, which is probably what has been happening all along and you just never noticed.)

Now that they've identified this as an issue, FAST will increase their buffer and optimize the way they read from it in a future firmware release. In the meantime, playing with these two settings should get results that are good for you.