Pixy tweaked: New Color Model + Auto Bright and White

[humlet.s...@gmail.com]

Hi Rich,

It's almost a year ago that I have posted something in the Pixy forum … and I haven't answered your last mail. Sorry for that. But finally I found some time to play with the Pixy cam and I have tried to implement some of the ideas that I wrote down in my little pamphlets I've posted some time ago in the Pixy forum:

http://cmucam.org/boards/9/topics/5770?r=5791#message-5791

http://cmucam.org/attachments/1190/PixyPimp.pdf

http://cmucam.org/attachments/1192/PixyPrefersPink.pdf

You can find the results - some demo videos, a prebuild Pixy firmware and the corresponding windows build of PixyMon - here:

https://onedrive.live.com/redir?resid=49AC1078706EAF34!551&authkey=!AAyk28qaxuvrrJw&ithint=folder%2cmp4

(In the auto white demo video sometime you will notice faint flashes in the camera view, when the mouse is inactive. This is the moment the light source has been changed from a cool to a warm one or vice versa)

The corresponding sources are available here:

https://github.com/humlet/pixy/commit/8dd774524bd92be5d0f60037c463aecb2a6cbe47

 

New Features:

• Experimental Signatures
• Runs on Pixy and in PixyMon.
• You can switch between the original and the new color model. The function of the original signature model is effectively untouched.
• Based HSV color space. For each signature the following HSV parameters can be configured individually:
• HSV hue acceptance angle
• HSV saturation min and max
• HSV value min and max
• … OK more sliders, but also more control and it's still intuitive.
• Black, White and grey signatures are supported!
• Fair share model used in the look up table (LUT) based pixel preselection on the M0 core:
• Supported only for the new color signature model.
• The color model used in the LUT is still the same: Fast and simple but suboptimal separation of low and highly saturated colors.
• Now, instead of the signature ID (in case of collisions the signature with lowest ID wins) the LUT returns a 7bit wide bitmap of compatible signatures. Later on the M4 the pixel is assigned to the signature with the smallest distance in the HSV (u,v)-chrominance plane.
• A luminance y-LUT has been added. For each individual signature combination observed in the chrominance uv-LUT setup, minimum and maximum luminance cuts are prepared. Especially for low saturated signatures (e.g. grey) this reduces the number of false positively preselected pixels.
• The number of pixel synchronization NOPs in the M0 assembler code were just sufficient to implement this.  
• The color model used is more or less the same as presented in the PixyPimp.pdf document above. The only difference is that the polar coordinates in the uv-chrominance plane have been normalized. This transforms the hexagon shaped spectrum into a nice circle. The polar angle correspond to the HSV hue whereas the r coordinate is equivalent to the HSV saturation.
• Auto Brightness:
• Pixy automatically optimizes the brightness setting (OV9715 camera's AEC target set point) to maximize the HSV value of accepted pixels. E.g. Pixy increases brightness in backlight situations.
• It's still experimental and works only in combination with the new signature model. But first results are promising. See demo video above.
• Auto White Balance:
• Pixy automatically tries to optimize the white balance by analyzing for accepted pixels the distribution of the uv-chrominance deviation from the corresponding signature uv-median.
• It's still experimental and works only in combination with the new signature model. But first results are promising. See demo video above.
• Skip Frames:
• Pixy performs the run length analysis, including the auto white and auto bright closed control loops, but omits the generation of new color blobs for a selectable number of 20ms camera frames. This grants slower client controllers more time to read out a higher number of blobs per frame at a slower frame rate.
• Brightness dead band escape:
• Pixy temporarily reduces the OV9715 camera's AEC stabilization window after brightness setting changes (Implemented in a rather dirty way.) 

 

Remarks:

• If PixyMon is started within the QT Creator frame work, the LUT is dumped as ascii-art to the "application output" view, if the "Logging" checkbox is activated in the "expSig" sub tab of "PixyMon Parameters".
• The alternative uv-LUT approach I described in the PixyPimp.pdf document above is still in the code. It can be activated for testing in cooked mode only. It still runs, even though the LUT granularity suffered a bit during the speed optimizations for the LUT approach that has been finally used.
  Maybe I'm wrong, but I don't think that it can ever be used for the M0 pre-selection. Even though it's implemented without divisions and multiplications, I think, it's too complex/slow.  
• I've tried to document my changes. At least some further information can be found in the source code. … Just ask, if something isn't clear. I'm sure a lot explanation is still missing.
• The way I freed the memory for the additional y-LUT is rather spooky. Just increasing the LUT space a bit (0x200 bytes) crashed the Pixy. So I took it from the scratch memory. However, it seems to run stably.
• I'm afraid that I have messed with the parameter interface. I didn't know better. The parameters interface features ("shadow", "dirty", "radio", "chirping" …) is more or less documentation free, or I just haven't found it.
• I find the source code differ provided in the QT Creator framework (Menu: "Tools"->"Git"->"Local Repository"->"Branches" … select branch to compare to, then press diff) very useful.
• I have changed the M4 optimization setting from –Os to –O2. This increased the size of the hex file by only 20K while speeding up e.g. the LUT generation by 9%.
• I've got a weird error message, when I tried to build "build_this_first" within LPCXpresso
  "cp ../common/src/*  ../libpixy_m0/src
  [main] make 10460 C:\cygwin64\bin\make.exe: *** fatal error in forked process - fork: can't reserve memory for parent stack …"
  Quite an error for some simple copy commands, but the make works smoothly, when dispatched in the Cygwin shell
• Comments are very welcome

Happy New Year,

Helmut

[Rich LeGrand]

Hi Helmut,

Thanks for the post!

We look forward to giving this a try.  It looks very cool!    

best,

--rich

--
You received this message because you are subscribed to the Google Groups "pixy_firmware_sdk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pixy_firmware_...@googlegroups.com.
To post to this group, send email to pixy_fir...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pixy_firmware_sdk/ce186225-bfc1-45fe-a4f9-a5b8e90c36cd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Charmed Labs
www.charmedlabs.com