Open Hardware Vision Camera

[Angus Bishop]

Hi All, I've just ordered a load of custom boards for a OpenPnP camera I have designed.

Current specs below;

40x47mm size. 4X M3 mounting holes on 30x30 spacing.

720p 30fps Global Shutter Image sensor (30fps limit due to a clock incompatibility which I hope to sort out by rev002)

Uncompressed RGB Data out over USB 3.0.

5 Integrated and triggered LED's for illumination.

Syncronous triggered 5v power and ground header for LED expansion.

In-chip lens correction.

Target BOM cost is $30-40

Let me know if there's something I should be adding or specific features that I should concentrate on.

[Jason von Nieda]

Sounds nice Angus! What lens mount do you have planned?

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/b00be590-7a59-4ef5-8245-bceb684bb8dcn%40googlegroups.com.

[Angus Bishop]

M12, Currently got one from Sunex in, but it's all pretty standard

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CA%2BQw0jxxzVN-D6ASz6m17Wihja6khmFiswxAym4WZQ9-E0eYBA%40mail.gmail.com.

[ma...@makr.zone]

Wow. Could you add a trigger for stills?

It was repeatedly discussed here for fly-by vision i.e. moving the parts over the camera without stopping, so real-time referencing to the controller motion is needed.

_Mark

[Niclas Hedhman]

I have not decided on camera solutions yet, so naturally I am interested...

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CAJo-XVVX%2BRW%3Dg6hWmq0vTxiBDrcK2v-sxfTYffavs-pjgDQbOQ%40mail.gmail.com.

[Angus Bishop]

I know the sensor can do stills, But I'm not sure how I can implement it with the UVC wrapper I have at the moment.

[ma...@makr.zone]

I guess it would be fine to just send the last triggered still image as a video frame over and over again. 

The important thing is that the triggering is exact in time, i.e. waiting for the next 30fps frame to "just happen" is no good.

Ideally, the "triggered stills" mode would be optional. ;-)

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/1a6fa39e-3434-4fdb-ae42-ab67a276a7c3n%40googlegroups.com.

[Angus Bishop]

Would it be best to implement the "triggered stills" in USB or would it be easier to just trigger it with a voltage? I ask as the bridge chip that I'm using has some functionality to trigger a snapshot, but it might introduce a fair bit of latency.

[ma...@makr.zone]

No it must be in hardware. The signal must either come from the motion controller (sending a pulse when a certain position is crossed) or even from a contact/optical sensor physically triggered by the nozzle moving above the camera. It must reflect the machine position in real-time.

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/78b7ab6e-105c-4d26-b6b6-d52d825ec564n%40googlegroups.com.

[Angus Bishop]

Latency between hardware trigger input and exposure is around 0.5ms is this too long for real time?

[ma...@makr.zone]

I guess it does not matter as long as it is very consistent. The fly-by vision will have to be calibrated for common offsets, anyway.

The fly-by motion too must be at very consistent/constant feed-rate (enough run-up to be past any acceleration phase).

This will in no way be "easy-peasy" but if your camera can do the triggering, we would be a huge step closer ;-)

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/34d7af27-3eb4-4d90-8bfc-8fef1f37f2edn%40googlegroups.com.

[Niclas Hedhman]

Peanut Gallery chiming in; Does the nozzle/head contain "fiducials" which are used by the imaging software for position determination of the component? In my mind, that seems to be a requirement to manage to do this at all...

Cheers

Niclas

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/6e705507-d6ba-74db-5638-bb132ba9d7f1%40makr.zone.

[ma...@makr.zone]

The way I envision it, fiducials are only needed if the triggered frame timing vs. the motion is not consistent enough or if you have no triggering at all.

I guess fiducials would be quite hard to interpret unless they are in the same focal plane as the part. Putting them in the same focal plane would probably mean the fiducials are fixed in Z (on the head), arranged around the nozzle and the nozzle retracts into the same focal plane with the part on the tip(?).

See the illustration:

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CADmm%2BKfZzpNF4zXN0H%2B7qY7xLDHcguroZRGLGepd0NRqVf9JaA%40mail.gmail.com.

[Mike Menci]

Mark

interesting mirror vision approach with fixed camera and mirror to X axis is here:

https://twitter.com/GregDavill/status/1263053390155706368

and here - see 5:50 min of the video:

https://www.youtube.com/watch?v=ZaqvEftw7aI&feature=youtu.be&t=368

Mike

[Niclas Hedhman]

Mark,

Right, but getting that consistency of speed and acceleration over the camera is not that trivial, is it? Assuming camera as near to feeders as possible, there would be a lot of difference for component "next door" vs one far away, and possibly coming in over the camera from different directions. To me (complete newbie) it sounds somewhat easier with the fiducials (on the focal plane), as it should be less sensitive to vibrations in the whole gantry.

From my PoV; fly-by-camera is such "super feature" that having that in OpenPNP just blows my mind...

Cheers

Niclas

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/8b5613fc-19bd-62b4-1fc2-671b2ae52a2a%40makr.zone.

[ma...@makr.zone]

Niclas,

you can never fully speed up over the camera. This will always be a controlled, rather slow fly-by. Ideal for multi-nozzle heads that have the nozzles in a row and want to quickly align them all. Even a slow  fly-by would still be much faster than the stop and go plus camera settling for each nozzle. But this will only work for parts that can be aligned in one shot. Probably for small passives, not for huge fine-pitched ICs...

Note, this is just a theoretical discussion started by Angus asking ...

> Let me know if there's something I should be adding or specific features that I should concentrate on.

... it does not mean that OpenPnP will have this feature anytime soon ... but having such a global shutter+trigger camera (affordable) would certainly be a door opener!

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CADmm%2BKckDkgWLMn%2Bd5iQY6DOiG2qqOC7G%2BBkQoEwyOpOGXBCzA%40mail.gmail.com.

[Sandra Carroll]

_Mark

 

Just Curious,

Does OpenPNP support the use of Multiple Up Cameras?

Ie if I have 2 nozzles can I have 2 cameras and position them so in 1 pass both nozzles are inspected.

 

Or is this what your referring to as a super feature?

 

Sandra

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/5fb5a8b1-ec63-d960-323d-5c52dc5543ea%40makr.zone.

[ma...@makr.zone]

> Does OpenPNP support the use of Multiple Up Cameras?

No, the current implementation does not support the use of multiple cameras, but yes, OpenPnP  does allow you to define multiple Up Cameras and has the architecture to support this with relatively few things missing and (more importantly) nothing standing in the way. Always amazes me how great Jason's underlying architecture is!

Quasi-parallel vision i.e. just dedicating one camera for each nozzle but still performing the alignments one after the other could probably be done in a few lines of code (the GUI to associate a nozzle with a camera being the hardest part ;-). The gain is reduced motion time to position each nozzle plus some avoided settle times. This is probably only worth it, if you do the vision at retracted nozzle height, so no Safe Z up/down motion is needed. Doing it at a different focal plane than the PCB plane introduces some parallax problems that we luckily already solved (Marek has this on his machine and his testing helped me develop a solution) :-)

https://makr.zone/improved-runout-compensation-and-bottom-camera-calibration/346/

However, true ganged-up bottom vision i.e. doing it at the very same time would be a much taller order. Much more to reprogram in OpenPnP. Iterative/multi-pass bottom vision would either only work for rotation (only if you have non-shared C axes) or spoil most of the time gained. You would have to calibrate camera (pixel) centers and/or provide a way to mechanically adjust the cameras to the nozzles in X/Y and/or plane. Again this only makes sense if done at retracted nozzle height.

One important thing to keep in mind is added CPU load through running so many quality USB cameras.

https://makr.zone/camera-fps-cpu-load-and-lighting-exposure/519/

So Angus' camera with triggered stills could still be instrumental, even for this variant.

For one-pass vision for small passives, the fly-by solution would still be much faster, I believe.

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/0a4201d6cb72%248978e480%249c6aad80%24%40gmail.com.

[ma...@makr.zone]

Hi Mike

that's not the same as fly-by and much more demanding in terms of precision mechanics and optics, but also potentially much, much faster of course. With today's ultra small cameras I guess you could even flip the camera itself.

https://thepihut.com/products/zerocam-camera-for-raspberry-pi-zero

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/88edf10b-080f-4924-8874-030ce78c68b5n%40googlegroups.com.

[Angus Bishop]

Hi Guys,

Thanks for the support so far.

I noted a few issues with my V001, so unfortunatley V001 will probably not work.

I've just finished V002, aside from a very messy schematic.

It is up on GitHub for those who want to bring this up alongside. I won't be ordering my boards probably until after Christmas because I'm pretty skint at the moment.

https://github.com/angusgbishop/OpenPnPVis

Triggering is brought out onto a 2.54mm pitch header.

Triggering should be a pretty consistent delay between trigger pin high and the start of exposure (154us). Total time between trigger high and end of exposure is a little more variable, and I can't say that for sure on everyone's setup. The previously stated value of 0.5ms is between trigger high and start of frame readout.

Let me know either here or on GitHub if there's anything I buggered up.

Thanks again!

[Ian Arkver]

Hi Angus,

I took a look at the schematic. Looks like you're using the 8-bit parallel interface at maybe up to 33MHz, though I don't see the sensor's PCLK connected to the bridge. This won't be fast enough for 720p uncompressed RGB. 1280x720x30fps x 3bytes-per-pixel is about 83MHz without any blanking overhead. It's not even fast enough for 4:2:2 YUV. Pity there wasn't a USB3.0 UVC bridge that took MIPI input.

i guess the values in the schematic are just placeholders? For example 100nF is obviously wrong for the crystal parallel load caps, and 100R seems very low for the power LEDs.

All the best,
Ian

[Angus Bishop]

Hi Ian,

The USB bridge chip drives a data clock signal at either 66 or 100 MHz, the sensor itself has an ext clock of between 17-40MHz.

The image sensor has its own PLL and clocks out it's data at a software definable frequency.

The aim is to get the image sensor to clock it's data out at 66Mhz and have the data clock out of the USB chip halved into the input of the image sensor. There's some latency in the clock propagation, but I think it's within the limits of the input.

If this doesn't work I can use the USB chips' crystal as the input, this would remove the propagation delay of the flip flop, but I'm not sure how it correlates to the data in clock.

It would require different software.

All of this is in an effort to remove the need for an intermediate FPGA.

The component values are all mostly placeholders until I have some chance of evaluating things.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/d07841aa-32fa-4499-8725-d58fdebb09adn%40googlegroups.com.

[Ian Arkver]

Sure, I just don't think you'll get anything above SD on an 8-bit
parallel bus at these sort of rates. But then, I don't have the OnSemi
datasheet. Maybe I'm wrong.

I notice that the two other module manufacturers with AR0144 modules
(Framos and Leopard) both have MIPI interfaces.

Anyway, good luck!

All the best,
Ian

>> the triggered frame timing vs. the motion is *not
>> *consistent enough or if you have no triggering at all.

>>
>> I guess fiducials would be quite hard to interpret
>> unless they are in the same focal plane as the part.
>> Putting them in the same focal plane would probably
>> mean the fiducials are fixed in Z (on the head),
>> arranged around the nozzle and the nozzle retracts
>> into the same focal plane with the part on the tip(?).
>>
>> See the illustration:
>>
>>

>> _Mark
>>
>>
>> Am 05.12.2020 um 11:35 schrieb Niclas Hedhman:
>>>
>>> Peanut Gallery chiming in; Does the nozzle/head
>>> contain "fiducials" which are used by the imaging
>>> software for position determination of the component?
>>> In my mind, that seems to be a requirement to manage
>>> to do this at all...
>>>
>>> Cheers
>>> Niclas
>>>
>>> On Fri, Dec 4, 2020 at 4:33 PM ma...@makr.zone
>>> <ma...@makr.zone> wrote:
>>>
>>> I guess it does not matter as long as it is very

>>> *consistent*. The fly-by vision will have to be

>>> calibrated for common offsets, anyway.
>>>

>>> The fly-by /motion /too must be at very

>>>>>> <https://groups.google.com/d/msgid/openpnp/b00be590-7a59-4ef5-8245-bceb684bb8dcn%40googlegroups.com?utm_medium=email&utm_source=footer>.

>>>>>>
>>>>>> --
>>>>>> You received this message
>>>>>> because you are subscribed
>>>>>> to the Google Groups
>>>>>> "OpenPnP" group.
>>>>>> To unsubscribe from this
>>>>>> group and stop receiving
>>>>>> emails from it, send an
>>>>>> email to
>>>>>> openpnp+u...@googlegroups.com.
>>>>>> To view this discussion on
>>>>>> the web visit

>>>>>> https://groups.google.com/d/msgid/openpnp/CA%2BQw0jxxzVN-D6ASz6m17Wihja6khmFiswxAym4WZQ9-E0eYBA%40mail.gmail.com
>>>>>> <https://groups.google.com/d/msgid/openpnp/CA%2BQw0jxxzVN-D6ASz6m17Wihja6khmFiswxAym4WZQ9-E0eYBA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>>>>>>
>>>>>> --
>>>>>> You received this message
>>>>>> because you are subscribed to
>>>>>> the Google Groups "OpenPnP" group.
>>>>>> To unsubscribe from this group
>>>>>> and stop receiving emails from
>>>>>> it, send an email to
>>>>>> openpnp+u...@googlegroups.com.
>>>>>>
>>>>>> To view this discussion on the
>>>>>> web visit

>>>>>> https://groups.google.com/d/msgid/openpnp/CAJo-XVVX%2BRW%3Dg6hWmq0vTxiBDrcK2v-sxfTYffavs-pjgDQbOQ%40mail.gmail.com
>>>>>> <https://groups.google.com/d/msgid/openpnp/CAJo-XVVX%2BRW%3Dg6hWmq0vTxiBDrcK2v-sxfTYffavs-pjgDQbOQ%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>>>>>>
>>>>>> --
>>>>>> You received this message because you
>>>>>> are subscribed to the Google Groups
>>>>>> "OpenPnP" group.
>>>>>> To unsubscribe from this group and
>>>>>> stop receiving emails from it, send an
>>>>>> email to openpnp+u...@googlegroups.com.
>>>>>> To view this discussion on the web
>>>>>> visit

>>>>>> https://groups.google.com/d/msgid/openpnp/1a6fa39e-3434-4fdb-ae42-ab67a276a7c3n%40googlegroups.com
>>>>>> <https://groups.google.com/d/msgid/openpnp/1a6fa39e-3434-4fdb-ae42-ab67a276a7c3n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>>>>>
>>>>> --
>>>>> You received this message because you are
>>>>> subscribed to the Google Groups "OpenPnP"
>>>>> group.
>>>>> To unsubscribe from this group and stop
>>>>> receiving emails from it, send an email to
>>>>> openpnp+u...@googlegroups.com.
>>>>> To view this discussion on the web visit

>>>>> https://groups.google.com/d/msgid/openpnp/78b7ab6e-105c-4d26-b6b6-d52d825ec564n%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/openpnp/78b7ab6e-105c-4d26-b6b6-d52d825ec564n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>>>>
>>>> --
>>>> You received this message because you are
>>>> subscribed to the Google Groups "OpenPnP" group.
>>>> To unsubscribe from this group and stop
>>>> receiving emails from it, send an email to
>>>> openpnp+u...@googlegroups.com.
>>>> To view this discussion on the web visit

>>>> https://groups.google.com/d/msgid/openpnp/34d7af27-3eb4-4d90-8bfc-8fef1f37f2edn%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/openpnp/34d7af27-3eb4-4d90-8bfc-8fef1f37f2edn%40googlegroups.com?utm_medium=email&utm_source=footer>.

>>> --
>>> You received this message because you are
>>> subscribed to the Google Groups "OpenPnP" group.
>>> To unsubscribe from this group and stop receiving
>>> emails from it, send an email to
>>> openpnp+u...@googlegroups.com.
>>> To view this discussion on the web visit

>>> https://groups.google.com/d/msgid/openpnp/6e705507-d6ba-74db-5638-bb132ba9d7f1%40makr.zone
>>> <https://groups.google.com/d/msgid/openpnp/6e705507-d6ba-74db-5638-bb132ba9d7f1%40makr.zone?utm_medium=email&utm_source=footer>.

>>>
>>> --
>>> You received this message because you are subscribed
>>> to the Google Groups "OpenPnP" group.
>>> To unsubscribe from this group and stop receiving
>>> emails from it, send an email to
>>> openpnp+u...@googlegroups.com.
>>> To view this discussion on the web visit

>>> https://groups.google.com/d/msgid/openpnp/CADmm%2BKfZzpNF4zXN0H%2B7qY7xLDHcguroZRGLGepd0NRqVf9JaA%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/openpnp/CADmm%2BKfZzpNF4zXN0H%2B7qY7xLDHcguroZRGLGepd0NRqVf9JaA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

>>
>> --
>> You received this message because you are subscribed to
>> the Google Groups "OpenPnP" group.
>> To unsubscribe from this group and stop receiving emails
>> from it, send an email to openpnp+u...@googlegroups.com.
>> To view this discussion on the web visit

>> https://groups.google.com/d/msgid/openpnp/88edf10b-080f-4924-8874-030ce78c68b5n%40googlegroups.com
>> <https://groups.google.com/d/msgid/openpnp/88edf10b-080f-4924-8874-030ce78c68b5n%40googlegroups.com?utm_medium=email&utm_source=footer>.

>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenPnP" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to openpnp+u...@googlegroups.com

> <mailto:openpnp+u...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/d07841aa-32fa-4499-8725-d58fdebb09adn%40googlegroups.com
> <https://groups.google.com/d/msgid/openpnp/d07841aa-32fa-4499-8725-d58fdebb09adn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "OpenPnP" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/openpnp/O-J2KPbHCE4/unsubscribe
> <https://groups.google.com/d/topic/openpnp/O-J2KPbHCE4/unsubscribe>.
> To unsubscribe from this group and all its topics, send an email to
> openpnp+u...@googlegroups.com
> <mailto:openpnp+u...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/openpnp/CAJo-XVUBwkx8ZsCER9a2jwEqsaFG0DQc3ArA_0GPej9o9U2SaA%40mail.gmail.com
> <https://groups.google.com/d/msgid/openpnp/CAJo-XVUBwkx8ZsCER9a2jwEqsaFG0DQc3ArA_0GPej9o9U2SaA%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Ian Arkver]

Apologies for all the "You received this message" spam. Thunderbird sucks at Google Groups apparently (or I suck at deleting all the included cruft).

CU,
Ian

[Ian Arkver]

R12 I think it is, looks like it's under the edge of the camera lens mount. Top right corner of lens mount anyway.

CU,
Ian

[Angus Bishop]

Cheers for that, I think the Courtyard has fallen off the footprint for the lens mount since the last time I used it.

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/1d18b334-37b7-49ae-a9f0-390056be5e6bn%40googlegroups.com.

[Ian Arkver]

No worries, happy to help. I'd also be a little nervous about those vias
under the lens mount edge too. The C_DAT ones and an LED_power thingy
too. Maybe the vias are tented over with solder resist, maybe not. :-)

Or maybe the lens mount is plastic?

Anyway, good luck with it.
CU,
Ian

[Ian Arkver]

Last comment :-)

CYUSB3064 would be a better bridge here than the End-of-life FT602Q. It's a little bigger (10x10 vs 9x9) and another BGA which would be a pain, and It's about £5 more in 100+ qty, but it is MIPI so would handle the resolution and framerate.

I've no experience with either though. No idea how the software side would pan out.

CU,
Ian

[Angus Bishop]

Rev A is EoL, the chip is on Rev B at the mo, Its reasonably recent.

I've noted the CYUSB3064 before, MIPI would be a slightly easier routing job, but the extra cost and having to deal with 2 BGA's have precluded me from putting it in. However it's a damn sight easier to implement than the FT602 in tandem with a FPGA.

If the FT602 doesn't work out then you've already found my next step!

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/ae25e90b-25fe-42bf-943e-2446cc19afffn%40googlegroups.com.

[ma...@makr.zone]

I'm not sure I read the PCB correctly, but make sure the mounting holes are not grounded.  IMHO, it would usually be wrong to ground the PC's USB to the machine.

None of the USB cameras I have (ELP, others) has grounded mounting holes.

https://makr.zone/grounding-the-machine/283/

_Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CAJo-XVUaFDKpBKFx-bUpug2c5YnRKBmQJ2wfnNQDYYHFkrerLg%40mail.gmail.com.

[Angus Bishop]

The mounting holes are AC coupled to ground for EMI, they aren't DC connected.

AC coupling can be removed by no-fitting the 4 ac coupling caps.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/094ec254-bf07-b038-dc1f-d90e564f33b2%40makr.zone.

[Shai]

Subscribing as this is the first time I'm seeing an open source camera tuned for PNP, great work! Curious - are you going to be writing a firmware for this as well?

[Shai]

And one more thing - any chance of an Eagle CAD version? :) I'm not much of a KiCAD fan and don't believe they have an export option to Eagle? If not, that's fine as being open source is already appreciated!

[Angus Bishop]

I'm not planning an Eagle conversion, but once I've spun a revision that looks OK, I'll release gerbers for others to make their own.

I will be writing firmware. My intention is to avoid writing any PC-side, I've flip-flopped on this point but ultimately it'll be more useful if it just runs off native UVC drivers.

[tf38385656]

Hi  Angus Bishop
Your  Uncompressed forucc =RGB  camera work Okey ?

I think i have the same problem for that .

Openpnp just for MJPEG formats and YUV formats

How did you do ?

Angus Bishop 在 2020年12月7日 星期一下午4:30:45 [UTC+8] 的信中寫道：