Raspberry pi pnp head cam?

[Karl Lew]

This week I got my Raspberry Pi to drive TinyG. It was good wireless fun sitting on the couch making TinyG zip back and forth from my iPad.

Today I read about the RPi camera now available in UK. It made me think wacko thoughts....ahhhh, why not....

What do folks think about putting an RPi with its camera on the PnP head?

[DAniel Dumitru]

Hello KArl,
Will you please elaborate a bit over your setup ?
I mean : a tablet, I guess wi-fi, raspberry with a wi-fi dongle, serial
port to tinny g ?

Please give some sw details too !
BR
Daniel

[chris]

RPi is not stable with OpenCV, use the Olimex board, this works and the cost is similar.

[Karl Lew]

Daniel, you have it right regarding my setup. Tablet->wifi->edimax ew-7811un->RPi-> plugable 7-usb serial hub->TinyG.
Also have Tiny Java Web Server and Java 8 preview with FX hard FP on the Debian wheezy-2013 RPi. The logical next step would be to create an OpenPnP web app. That would let me use OpenPnp with tablet and sit on couch while my friend plays League of Legends on my puter. He likes my gaming setup I got for highres Skyrim. I think he is nuts to play LOL. 8)

Having wireless RPi on the head could actually reduce cable snarl. It was such an odd idea I thought I should get a sanity check with you guys.

Sad about OpenCV instability on RPi. Wonder if that might be due to sw float emulation in previous RPi javas? The OpenCV instability makes me sad for now but hopeful for a distant (?) future where some energetic kind souls might fix it given the new incentive of RPi cam. Did not know about Olimex board. Thanks for tip, Chris.

[Jason von Nieda]

I personally think the RasPi is underpowered for real pick and place work when you add vision into the mix. We'll see, though. Board Forge (Tempo Automation, now) is going with a Node.js web app frontend running on RasPi.

I may break OpenPnP up into a job runner and a front end at some point, so that it can run headless, but it's low on the list of priorities.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/0fd4e737-3325-4174-a2c4-ba21834c91e2%40googlegroups.com?hl=en-US.

For more options, visit https://groups.google.com/groups/opt_out.

[Karl Lew]

Jason, thanks for the Board Forge tip. Here is the related link http://www.adafruit.com/blog/2013/03/15/boardforge-raspberry-pi-powered-oshw-pick-and-place-machine-piday-raspberrypi-raspberry_pi/

Looks like summer will see at least two pnp kickstarters!

[Karl Lew]

Here is a picture of a 36mm board taken at a distance of 45mm using the Raspberry Pi camera, illuminated by a single white led. The Pi camera lens is normally in focus at 60cm-infinity. I unscrewed the lens entirely then screwed it back in about one full turn. I think this brings up some interesting possibilities and now I'm seriously looking into a RaspPnP head. (...so named because "PnPi" didn't sound too appetizing...) 

(oh and others have looked into OpenCV with the RPi cam, so it's looking like perhaps a good fit for OpenPnp...)

[Karl Lew]

https://github.com/firepick1/FirePick/wiki/EPIC

[matthew granfield]

been planning on using a PI for vision and a arduino 256 for the rest, you got any ideas what you're gonna use for vision software on the pi?

[chris]

Opencv, in the meantime the stability is increased and driver for it is available.
4mp camera on the head makes only sense
If you want cover two heads using one
downlooking camera in addition to
Vga downloking camera for fiducial and
Part location.

[DAniel Dumitru]

Hello All,

I have seen 2 interesting videos with pick and place machines and one of
them has also detailed instructions on machine build :
http://www.youtube.com/watch?v=-qG2Mjf1Lyc
and has details here :
http://www.vbesmens.de

and this one :
http://www.youtube.com/watch?v=OTaeEG4P-E4

BR,
Daniel

[Karl Lew]

Images acquired using FirePick on-head Raspberry Pi camera using look-up mirror located on PCB tray.

Images processed using OpenCV.

http://youtu.be/a6q6oHu2UsU

[chris]

Hello Karl, i´m interested how fast RPI is for low level image processing compared to host cpu.
I could send you preprocessed C code and some images if you could implement a timing measurement for RPI.
I have actually this macros for timing:
If you have similar code, or could implement one using something like this: http://mindplusplus.wordpress.com/2013/05/21/accessing-the-raspberry-pis-1mhz-timer/
#define TimerDecl(x)    long long x=0L
#define TimerInit()        QueryPerformanceFrequency(&frequency_)
#define Start(x)        QueryPerformanceCounter(&timer_),x=timer_-x
#define Stop(x)            QueryPerformanceCounter(&timer_),x=timer_-x
#define Timer(x)        elapsedTime_ = x * 1000.0 / frequency_, printf("timing-> %s: %fms\n",#x,elapsedTime_)

It would be nice to know some numbers in order to estimate the real performance.

[Karl Lew]

Chris,

Sorry for the delay in answering. Dayton gave me a shout and told me you had posted to the group...

My interest in RPI OpenCV stems from this recent link: http://www.raspberrypi.org/archives/4207

In Pierre's post, we see facial recognition running on RPI using OpenCV and VIDEO. Wow! Pierre's research is inspiring!

I asked myself some questions and here is what I found:

Q: Does the direct camera connection give the RPI an advantage over USB cameras?

A: Yes it does, because the image goes directly into RPI memory, so there's no need to serialize the image and stream it bit-by-bit over USB.

Q: Can the RPI camera provide close range pictures?

A: Yes, it can. See 1206 part on 15gauge spindle in picture, above.

Q: Can OpenCV process the RPI image usefully?

A: Yes it can. The output of the BackgroundSubtractorMOG2 and subsequent minAreaRect transformations provide exactly what OpenPnP needs to determine part location and orientation. (the Pink rectangle is an OpenCV RotatedRect data structure)

Q: Can the RPI process images quickly?

A: Probably. I'm researching that. Currently, raspistill takes 1-2 seconds to take and save a 200x800 pixel JPG. However, we don't need the JPG, We just need the part location and orientation. My hypothesis is the 1-2 seconds might be just the time it takes to generate the JPG.

Q: Can the RPI run OpenCV and OpenPnP at the same time?

A: Maybe. I'm researching that. RPI can run my Java web server. It can also run OpenCV. With luck, it can run both simultaneously. If not, I'll just add an RPI driver to OpenPnP, so I'm not too worried.

Q: Why not use BeagleBoneBlack?

A: BBB is a faster, bigger computer than RPI. However, RPI has that magic little $20 camera that fits neatly on the PnP head between two 15 gauge spindles located 48mm apart. That small form factor lets FirePick do three things with a single $20 camera: 1) inspect component tape, 2) identify part pickup error, and 2) identify PCB position and orientation. At the time of this writing, BBB has no such cheap, universal camera with a closeup lens.

Q: Why not use Pixy?

A: Pixy has a lot of promise, given that OpenCV is on the board in a dedicated chip. However, the large size of the Pixy board would make the PnP head a bit large and unwieldy. In contrast, look at this picture to see how small the FirePick PnP head is with an onboard RPI, RPI camera, wireless, 1-2 vacuum solenoids, a MakerSlide carriage and two (2) independent 15 gauge spindles (one shown). All of that, and only 350g or barely 12oz. Dayton is excited about the Pixy, as am I. We just haven't figured out how to squeeze the PnP head down to RPi scale.

https://github.com/firepick1/FirePick/wiki/AZP2

[chris]

My interest in RPI OpenCV stems from this recent link: http://www.raspberrypi.org/archives/4207

surely you have seen this: http://ryanmessina.wordpress.com/2013/06/10/performance-of-raspberry-pi-with-opencv/  , the data refers to usb cam.
The RPI camera has the advantage that you can preview the camera image on video with 30 fps using the hw video accelerator and doing the background processing
on slower frame rate. This is very nice.

In Pierre's post, we see facial recognition running on RPI using OpenCV and VIDEO. Wow! Pierre's research is inspiring!

I asked myself some questions and here is what I found:

Q: Can the RPI process images quickly?

A: Probably. I'm researching that. Currently, raspistill takes 1-2 seconds to take and save a 200x800 pixel JPG. However, we don't need the JPG, We just need the part location and orientation. My hypothesis is the 1-2 seconds might be just the time it takes to generate the JPG.

From that response, i think you don´t have installed OpenCV with RPI camera support or the video4linux RPI driver and default OpenCV.

Q: Can the RPI run OpenCV and OpenPnP at the same time?

A: Maybe. I'm researching that. RPI can run my Java web server. It can also run OpenCV. With luck, it can run both simultaneously. If not, I'll just add an RPI driver to OpenPnP, so I'm not too worried.

That is easy, but eventually you could programm you´r code in java using JavaCV, otherwise JINI... solution are possible. OpenCV is a library, not a programm and OpenPNP can
call it as it does.

I have found a good thresholding application for pnp usage, r00..r22. are pointers to image row for speedup
conv3x3(find_cross,
//  0 1 0
//  1 1 1
//  0 1 0
        v=*r01+++*r10+++*r11+++*r12+++*r21++;if(v>600) v=255; else v=0;
        )
Translated to OpenCV is this:
Note, input and output cannot be the same matrix/image.
This implementation assume that the image is gray  image or equivalent,
the first channel is accessed, it works for YUV image too, or for BGR images using the B channel
without color conversion, solution possible for pnp imaging. You can use color image as source
and gray image as destination.

    // Pixel argument x have sideeffects, this is needed
    // Pixel(img,c+x,y) for accessing channels
    #define Pixel_(img,x,y)  ((uchar*)(img->imageData + img->widthStep*(y)))[x]
    #define Pixel(img,x,y)        Pixel_(img,x*img->nChannels,y)

#define find_cross(img,dst) \
  do {
  for(int y=1;y<img->height-1;y++) \
  for(int x=0;y<img->width;x++)     \
     Pixel(dst,x,y)=Pixel(img,(x-1),y)+Pixel(img,x,y)+Pixel((x+1),y)+Pixel(img,x,y-1)+Pixel(img,x,y+1)>600?255:0; \
  memmove(&Pixel(dst,0,0),&Pixel(dst,0,1),dst->widthStep);memmove(&Pixel(dst,0,dst->height-1),&Pixel(dst,0,dst->height-2),dst->widthStep); \
 } while(0)

[Karl Lew]

Chris,

That's fascinating! No, I just knew about Pierre's post, not Ryan's, or your own research.

If you develop an RPICam solution, we could happily just use yours for FirePick. I was just going to smash together something workable and not worry about performance, since CPH isn't a big concern for FirePick. FirePick is focused on ease of assembly and use. Later versions would be optimized.

Would you be interested in developing an open source RaspiCam PnP library for us to use in FirePick and OpenPnP?

You could roll your own or we could work on it together. Either way...

:) Karl

[chris]

Hi, can you send me a email to ope...@forward.cat .