TVM920 Vision

[BendRocks]

Has anyone gotten vision working on the TVM920? I'm not able to open the cameras with any of the drivers. The cameras are analog with an analog mux prior to the capture/encode card. So, there's just one stream of video coming and you need to toggle the mux to switch between down looking and up looking cameras. 

[Jason von Nieda]

As far as I know no one has messed with OpenPnP and the TVM920 so far except Glen English, who is writing the driver. I'm not sure if he has tried using the cameras yet or not. Hopefully he can comment.

We recently added Before and After capture scripting events, so getting the MUX working shouldn't be too much of an issue assuming it can be controlled via the TCP protocol.

Jason

On Mon, Jan 2, 2017 at 6:15 PM BendRocks <mwtse...@gmail.com> wrote:

Has anyone gotten vision working on the TVM920? I'm not able to open the cameras with any of the drivers. The cameras are analog with an analog mux prior to the capture/encode card. So, there's just one stream of video coming and you need to toggle the mux to switch between down looking and up looking cameras. 

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/5bd6c0af-7a4c-4084-b50f-3fc9be5d2e51%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[BendRocks]

The capture card is on the Atom PC controllers in the TVM, so the mux selection won't need a UDP message. I'll see if I can get a standalone OpenCV test app working first in case nobody else has looked at this yet and from that standalone hopefully you and others can give advice.

[BendRocks]

I can access the video feed via DirectShow in c# and I can also change the input mux so I can see up/down video as needed.

But no luck via OpenCV yet.

Looking around, it seems there's a fair number of video capture cards that use DirectShow that cannot be seen from OpenCV.

Thinking out loud, there's dshow wrappers for java, java could call into a native DLL, or ???? Jason, any thoughts on this?

Does the OpenCVCamera just need to grab a frame from the camera? It looks like it runs in a thread and captures a frame every x ms. 

[Cri S]

Do you have used directshow driver?
camera number 700 and 1400 ?
If yes, videolib can handle analog camera.
Default compiled is ntsc. Javacv have videolib interface, but if other format, as example pal is needed, wrapper code must be written.

[Jason von Nieda]

That's odd, then. The OpenCV capture implementation for Windows uses DirectShow under the hood. Can you try opening the camera in OpenCV using device ID 700? This will tell OpenCV to specifically use the DirectShow capture implementation.

There is more information about why that works at https://github.com/opencv/opencv/blob/master/modules/videoio/include/opencv2/videoio.hpp#L88 and https://github.com/opencv/opencv/blob/master/modules/videoio/src/cap.cpp#L151.

If we can't get it to work with OpenCV, then yes, we could potentially do a DirectShow wrapper. The Camera system is designed to work with multiple implementations. In the past we've had native Video For Windows and lti-civil implementations but OpenCV turned out to be good enough to handle everything so those were deprecated and dumped.

The camera implementations, like OpenCvCamera are responsible for capturing images on demand and also for streaming images for the video interface. Most of the implementations, as you noticed, just capture in a loop in a thread and send the results out. 

Jason

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/7269a41e-2ec8-4360-aaa5-ecc2f254212b%40googlegroups.com.

[Glen English]

My comments

I've been running my openPNP on a separate machine- my 'big PC' ,  NOT the internal PC.

I run my TVM driver on the internal atom PC. 

I have video cables running (at the moment) across the floor to capture inputs on my big PC.

I was using USB capture interface s"Easy capture $10) but their drivers are crap so I have ordered a couple of HDMI $50 capture boards

In my machine I split 3 ways using a powered 4 way video splitter, one to the coax, 

splits:

---one to a Composite>HDMI converter which goes to a HDMI monitor on top of the machine specifically for viewing.

--- one to the composite input of the existing video board

-- one to a coax running across the floor to my big PC, via a $10 Composite>HDMI converter into an HDMI capture board.

*******

Now;

Rather than use two capture boards, it might be easier to just switch the composite video on input to the 4 way composite splitter.

A USB IO board would fix that

I would have a USB IO board in the internal PC with the TVMdriver

OpenPNP would send the TVM driver camera-select commands.

-g

[Jason von Nieda]

Glen,

Thanks for posting that info. So, just to be clear, you have not tried (or succeeded, possibly) to use OpenPnP to capture video directly from the TVM's built in capture hardware?

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/a8351e2e-daf5-4d98-8a8d-70b23351403b%40googlegroups.com.

[Glen English]

I have not tried, , but Robert in Vancouver has had a go. frustrating result apparently.

On Wednesday, January 4, 2017 at 8:47:09 AM UTC+11, Jason von Nieda wrote:

Glen,

Thanks for posting that info. So, just to be clear, you have not tried (or succeeded, possibly) to use OpenPnP to capture video directly from the TVM's built in capture hardware?

Jason

gle.com/d/optout.

[Robert Walter]

So far no joy on the openCv driver. The webcam driver sees the capture card as a viable device, but no video stream. I can easily load a third party webcam software, and I can obtain video from the primary (first) channel, as this capture card is multiplexed. We would have to have Glen amend his driver to handle camera switching duties to make this capture card a somewhat viable option if the VFW driver was to make a return, or if OpenCV driver could be made to accomodate, but considering the multiplexed nature, I don't see it as a long term solution. The 920 appears to have standard ELP board camera mounts for both up and down looking cameras, so swaping out would be a relatively simple affair cabling aside. Redoing the up camera cabling would be a simple job as it is all loose cabling in the machine under the table. Routing a new flexible USB cable through all the cable track will be a bit more work.

R.

[Jason von Nieda]

Robert,

Which third party software did you use that was able to access it? Did you need to install any drivers? And was this on the built in Atom or on your PC? Any info you can provide about the stack would be helpful. Maybe the third party software has an info window that gives details about the board, etc? In addition, if any of you know the make/model of the capture board that would be helpful too.

Jason

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/89e4a8f2-4d8c-447f-aebb-96404c73fb76%40googlegroups.com.

[Glen English]

When I get the HDMI capture boards I'll see if they play happily with the OpenPNP ecosystem 

(IE use existing cameras and just convert to HDMI with $10 box).

All these video capture boards USB and alike and PCI etc and their drivers unless you spend to big bucks appear to be really half baked.

[Robert Walter]

Jason,

Was just an freeware webcam viewer called "WebCamViewer".  www.bustatech.com

It is a single file executable, that I ran on the Atom. Also have Glen's driver, as well as openPNP loaded on the Atom as well. Everything behaving well, except for video....

Lastest development build of openPNP downloaded via the updater. Then copied Glens modified openPNP.jar file into the openPNP directory. Just JRE installed (no development). No other drivers installed.

The capture board is a offshore model developed by www.10moons.com. The driver / sdk is somewhat available on the net. It is referred to as SDK-3000

But aside from that, no mods to the system. Just loaded JRE, openPNP and Glens driver.

Rob.

[BendRocks]

As noted above, the TVM920 camera opens in a few lines of AForge as the default video source on the machine, and the input can be selected via the DShow AvailableCrossbarVideoInputs property. The inputs for the top/bottom camera are 0 and 1. But there's also a 2 and 3 input. Rob, perhaps when you opened in the webcam app the right input wasn't selected. it takes explicitly setting the input to make it start working. 

Attached is a dump of the devcaps. Perhaps there's a guid or something in there that can be made sense of for opencv

[Robert Walter]

Yeah, I am sure it can be done with a little coding. The SDK3000 API looks pretty simple, but as said, since it is multiplexed, you can only have once channel active at any one time. This makes it impossible to run side by side video windows of both up and down cameras simulateously. I guess one could write a custom driver for it and keep changing channels back and forth to get dual (albeit reduced frame rate) video sources, but I think I will just swap out the TVM cameras for ELP 2MP USB cameras once we are positive that Glen's driver is production ready (I can say it is really close). I already got under the head on the 920 and the stock camera bolt pattern is identical to the USB ELP camera's I have on hand, so it will be a super simple changeover, aside from running a 4m USB cable through the cable chain, and down to the PC.

R.

[Glen English]

I had a bit of a go getting the internal SDK3000 board on the internal Atom PC to work

no joy in getting OPNP to see it... Probably workable if you are budget limited , for me, too much f**king around.

Hmmm the less I have to do with that little pc the better i think, IE dump Chinese windows (although I have changed to locale to USA)

Conversely, I plugged in a couple of USB cameras and it all just worked immediately, easy. Like too easy...

the mounts for the internal camera are the same as much ELP brand USB cameras.. which is handy.

Remember- get a  black & white (monochrome) camera if you can, otherwise >half the pixels are wasted with the Bayer Filter (thanks Pete!) .

[Mark Harris]

Colour cameras can let you use colour filters to remove things like a green background  to get better contrast on the parts.. if its just a black background in those machines though, who cares :)

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+unsubscribe@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/51a661e9-5fdc-4c60-b07d-77fb60aa563b%40googlegroups.com.

[Glen English]

HI Mark.

I agree. That is true. I guess if you can  start off with enough resolution, go color.... 

On Friday, January 6, 2017 at 3:24:18 PM UTC+11, Mark Harris wrote:

Colour cameras can let you use colour filters to remove things like a green background  to get better contrast on the parts.. if its just a black background in those machines though, who cares :)

On 5 January 2017 at 20:44, Glen English <gleneng...@gmail.com> wrote:

I had a bit of a go getting the internal SDK3000 board on the internal Atom PC to work

no joy in getting OPNP to see it... Probably workable if you are budget limited , for me, too much f**king around.

Hmmm the less I have to do with that little pc the better i think, IE dump Chinese windows (although I have changed to locale to USA)

Conversely, I plugged in a couple of USB cameras and it all just worked immediately, easy. Like too easy...

the mounts for the internal camera are the same as much ELP brand USB cameras.. which is handy.

Remember- get a  black & white (monochrome) camera if you can, otherwise >half the pixels are wasted with the Bayer Filter (thanks Pete!) .

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

[Mark Harris]

The SIPLACE 80 machines I have use black and white cameras (due to age), and they have a completely black plate behind where the part would be, and the nozzles are all black too - this way the computer only sees the pins. If the Neoden machines are similar - then black and white would be perfect :)

To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+unsubscribe@googlegroups.com.

To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/c60fc5b8-5cca-4f63-ba13-7795dc70e181%40googlegroups.com.

[BendRocks]

I can get OpenPNP on the TVM to show an image from the top and bottom camera, but it requires an external program.

Here's how:

1) Use external program to set the video input (top or bottom) on the TVM920 capture card and display image. Close it.

2) Open OpenPNP, the same camera capture shows up in the OpenCV camera

3) Close OpenPNP, open external app. Change the input, view the image from the other camera on the external app. Close the app.

3) Open OpenPNP, the other camera image shows up in the OpenCV camera.

So, there's just a bit of init code required to make at least one camera work.

Looking around some more, there's a post on the link below that indicates that as of 2015, OpenCV + VideoInput library creates the crossbar and routes in the pins, but it doesn't insert the crossbar into the filter graph by default. That causes issues for all mult-input catpure cards, including USB. So, just switching to a USB multi-input capture device doesn't mean this gets fixed given everything else. 

I think there's enormous value in getting the built-in cameras working and cards working. I get the idea of ripping everything out and putting new USB cameras in. But that will mean a whole bunch of folks that could be running OpenPNP will not. It also means no going back ever. 

So, can someone explain what it would take to insert the crossbar into the filter graph in OpenCV? In the link, he's doing it in the original source it looks like. Can it be done from Java? 

Also, Jason, there's a DLL used by OpenPNP called org_vonnieda_vfw_capturedevice.dll....what is this for? I can't find anything that seems to use this. As I noted earlier, and external DLL that will fetch either cam might easiest here.

Here's the link with a change to to the OpenCV cpp code to insert the crossbar into the filter graph.

http://answers.opencv.org/question/55477/read-analog-video-signal-supported-grabber/

[BendRocks]

One more point: I thought maybe I could grab 10 fps or so and switch the input mux between each frame. That doesn't work. The mux select takes a bit of time. Maybe you could change it once per second top speed. But you can see this if you select the "change input" button on the TVM920 UI. The incoming image changes slowly, with some noise in between. 

Jason, could you elaborate a bit on what you said earlier in this thread about how the "before and after capture scripting events" work?

[Cri S]

Can you show the code that you use on opencv with videoinputlib?
Yes, it can be do with java, that require javacv as videoinputlib is deprecated because it uses old c interface that is not supported on opencv,
where javacv support it. Even on C(++) the old interface is officially depredicated but still supported up to 2.x .

[BendRocks]

I don't use any OpenCV or VideoInputLib code. I'm using an external program I wrote in c# to change the input mux via directshow. And my point is that an external program can set the hardware up and then it works in OpenPNP. 

So, based on some other reports I linked to above, I think what is needed is the crossbar needs to be inserted into the filter graph. But it appears to me that has to be done in the cpp code of  VideoInput

https://github.com/ofTheo/videoInput/issues/30

https://github.com/ofTheo/videoInput/blob/master/videoInputSrcAndDemos/libs/videoInput/videoInput.cpp

.

[Cri S]

It works using videoinputlib too without code change. Inside video input.h comments, there is a small program that show how to use videoinput with opencv. As javacv is poorly documented, I suggest having first working C program and then convert it to javacv converting then the final result to mat from IpImage. Its however possible to open the 1400 camera on opencv and use external program to switch AMC.crossswitch and format from NTSC to pal_b , or doing this before opening the camera.
Its unknown if other settings works under opencv control, like brightness, ... or if external program need take care of it.
As you probably have a script for changing AMC.crossswitch for user control, automatic switching is inside onCameraBeforeCapture script depending on up looking and down looking setting on camera and remembered state.
Further you need aliasCamera that proxy another camera in order opencv recognize the same camera as top and bot camera.

[Jason von Nieda]

Hi BendRocks,

org_vonnieda_vfw_capturedevice.dll is an old Java wrapper for Video For Windows. It was how OpenPnP used to capture images in Windows before we switched to OpenCV. It's no longer used.

Writing Java wrappers for native code has gotten much easier in the past few years, so if it turns out you need a native interface to get the cameras working it wouldn't be that hard. See https://github.com/java-native-access/jna and https://github.com/nativelibs4java/BridJ.

You could also just call out to an external command line program if it doesn't need to be particularly fast.

The scripting event stuff is documented here: https://github.com/openpnp/openpnp/wiki/Scripting#scripting-events

If that doesn't cover your question, let me know and I can go into specifics if you need me to.

Something to keep in mind: OpenPnP doesn't *need* video streaming to work. All vision operations go through the capture() API which just returns a single frame and it's only called when one is needed. So you could just have your machine stream from the top camera, since this is the most useful for setup and just have it switch inputs when capture is called on the bottom camera and then switch right back.

Jason

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/CAJGcfUjaXEngyUYMBGVEfDaSFRAT0wf5hbMgyde9pu0w05pDyg%40mail.gmail.com.

[SMdude]

Also worth a try, back up and delete your machine xml, then let openpnp create a new one.
Then go in to openpnp and set up your top and bottom cameras again.

On my machine, it got to a point where I could only see video streaming from one camera or the other(elp's). As soon as I would try and initialise the camera that was not working, the other stream would stop.
Backed up and deleted machn.xml, I then set both cameras up, which then worked.
Copied the camera sections from the new back to my old machine.xml and restored it and both cams are back.

I suspect I could probably just deleted them from openpnp in the machine setup, or even just deleted the <cameras> section, and resetup the cameras.

So if you haven't already, start fresh and see..

Cheers

[Glen English]

Given that in YUY2 (uncompressed mode) for the ELP USB2 cameras, at 1MP, you are only getting  ~ 7-9 fps, does OpenCV just block until it gets a new frame ? or would it use whatever was the latest one  IE  that just came in, but was captured 150mS ago ?

that is to say,  can you flush the vision pipeline to ensure the grab is the latest  (which of course incurs a delay but that's fine.

The ELP cameras sure have transfer limitations (and me must use YUY2 in my opinion- not MJPEG) , but at $50 they are alot cheaper than the USB3 Pt Grey (which I also have) at $300-$600 ....

I guess that's one advantage of the composite cameras with PCI interface- essentially full D1 resolution and low latency at full frame rate.

Composite capture cards are getting hard to find but there are plenty of PCI HDMI captures that you can use a cheap Composite to HDI adaptor...

 

[BendRocks]

Thanks Jason and Cri

Today, I let my TVM920 update itself to Win10. And it went to 100% Chinese mode. Not a western character anywhere. And worse, there's not an option to switch to English even after you set to english on the langauge screen...However, I did launch the TVM app, and it seemed to work under Win10 though the width was a bit wonky.

This started because I made a change to the VideoInput files that I wanted to run on the TVM. No luck. It required a bunch of MS runtime dll  files and the 2015 redisti was failing some for reason. So, I figured I'd install VS. But the HD was almost full. The drive is 32Gbyte, but split into 16+16, with a D drive holding some Altium stuff. So, I dumped the altium and D drive, grew the C drive to 32G. And then, VS2015 wouldn't install because it no longer support Win7. So I said wth and updated to Win10 and it went to single language Chinese. This is a known issue, and the fix is to install English Win10 over it--the product keys should persist. That will let you keep files, but it will forget the app installs as they can't be assumed to be supporting english.

And it's slow as molasses installing. 

But, tomorrow I should learn if the standalone VideoInput app with crossbar support can do what needs to be done. And hopefully I can get RemoteDesktop working tomorrow night as the machine is far away from my main dev machine. 

FWIW, my TVM920 came with Win7 Ultimate 32-bit installed. 

[ca...@airdtf.com]

I'm able to use up/down cameras now, just a little script in OpenPNP needs to be written to enable this. Using the external command line tool FFmpeg you're able to capture from a specified device and crossbar and pass this to an image file, which OpenPNP is able to process with the ImageCamera source.

Due to a limitation in OpenCV (which persists to 3.2 and is an architectural problem with their dshow crossbar implementation) the down-camera will never be able to be used "live" until they fix their bug. Additionally, crossbar support has to be implemented in OpenPNP but this is rather simple.

So this is the script that has to be written when the down-camera is taking a picture:

1) Turn off the OpenCV Up-camera.

2) Turn off the up-lights and turn on the down-lights.

3) Invoke FFmpeg and take a picture using the down-camera.

4) Turn the down-lights back off and up-lights on.

5) Turn on the OpenCV Up-camera.

In this setup the Up-camera is always "live" which should make for fast imaging during placement. The Down-camera is only activated occasionally during fiducial grabbing so this setup should still be very fast during production. Annoyingly the down-camera can't be used "live" so initial fiducial-setting will be annoying unless an additional script is written for this, or a live view is used using an external program, etc.

The FFmpeg invocation is simple:

ffmpeg -y -f dshow -crossbar_video_input_pin_number 0 -i video="SDK3000 video capture" -vframes 1 <image name>.jpg

then the ImageCamera object in openPNP is pointed to that jpg. the Down camera is on crossbar 0 and the Up camera is on crossbar 1.

Carl

On Monday, January 2, 2017 at 7:15:56 PM UTC-5, BendRocks wrote:

Has anyone gotten vision working on the TVM920? I'm not able to open the cameras with any of the drivers. The cameras are analog with an analog mux prior to the capture/encode card. So, there's just one stream of video coming and you need to toggle the mux to switch between down looking and up looking cameras. 

[ca...@airdtf.com]

Not so easy... OpenPNP crashes a lot on camera.close() and camera.run()

[Jason von Nieda]

Can you provide any details on that? I can't fix bugs I don't know about :)

Jason

--

You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/1600c774-13c8-4a31-a27e-baf0c080dee5%40googlegroups.com.

[Cri S]

I have noticed running opencv without camera generate core dumps, dont remember if it works with one CAM, and opencv don't like closing and opening the camera again, some driver still allows that.

You must use ffmpeg to switch crossbar, not taking image. Then with utility like splitcam or any other tool that from the single CAM making 3 cameras, the orginal and two TEE filter of it.
On opencv dont close camera, one camera just display the wrong thing, because both cameras always display the same image where the crossswitch dictate up or down camera.
Hope this helps. Possible alternative is creating aliasCamera that proxy annother camera,
technically both solutions works.
Videoinputlib was tested?

[BendRocks]

This is good info Carl.

Under Win10 the capture card has become erratic at best. One channel cannot be selected any longer. I've just ordered an Osprey 100 PCI video capture card that (3 composite in and 1 S video in) and I'll see how that does. As I dug around looking for drivers, there's a ton of SA7134 capture cards, and all their SW seemed to come from Philips. But it's all very old. Osprey seems to keep updating their current drivers, claims 100,000 of these cards in the field and support for DirectShow. 

If this works, then I think I also understand how to switch the video crossbar in the standalone VideoInputBasic test code for VideoInput. That compiles fine, but doesn't do anything right now on the TVM, likely because of the same issue above. The VLC Media Player also isn't able to do it either. 

If this card will get things going, I'll be happy to pay the price to get Win10 and VS2015. If not, then I'll roll back to Win7. 

FWIW, if anyone needs to re-install the original SDK3000 drivers, there a site that is called MyDrivers.com. Search on SDK3000 and you'll find a date of 2011-8-23 on one of the zips. The contents are all whql signed sys and cats, so likely safe. But as I said, under Win10 I'm still having trouble with a channel.

[ca...@airdtf.com]

Jason, not much more to it than that - I put machine.getCameras()[0].close(); in a script file, ran it, and OpenPNP crashed. OpenCV type camera.

Sometimes OpenPNP comes back after several seconds, and the camera has been switched off - success. Subsequently running machine.getCameras()[0].run(); causes another crash that never recovers.

This also happens on ImageCamera type, and happens whether or not a camera is currently being displayed in the live view.

There also doesn't seem to be a way to invoke a single capture command, ie update the system via script with a single capture command, capture() has the image as return type but I'm not sure where to put it.

Unfortunately upgrading OS or swapping capture cards is not an option, the machine is currently working fine with QiHE's software, maybe once OpenPNP proves to be a viable alternative, but there's some work to be done yet.

[ca...@airdtf.com]

Any idea what "unable to convert string in GetNozIndex" means? can't find this function anywhere, currently unable to move the head to preset locations.

[Cri S]

Try it, use splitcamera. This gives you two identical cameras, one bottom, THE other top.
You don't have to close cameras, just switch cross switch without taking any image, because the camera is already used by split camera, changing parameters without taking pictures work.
Or at least black image.

[Jason von Nieda]

Carl,

The close and run methods are internal functions, not intended to be called by scripts or really by anyone. They are used to run the camera system. 

run() is the thread function, and is called when starting the thread that runs the camera. 

close() shuts the camera down and is only called as the program is terminated, just before exiting.

capture() is used throughout the program to get an image from the camera. This *is* intended to be called by external callers and it should return an image. I'm not sure what you mean by "update the system". 

In general, you should only call public methods from the service interfaces. For instance, in Camera the service interface is http://openpnp.github.io/openpnp/develop/org/openpnp/spi/Camera.html and all of those methods are safe to call.

If you tell me what you are trying to accomplish by calling these I can guide you on the right way to do it. 

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/3bca411c-bc17-4804-b9ae-34cff4c2ed23%40googlegroups.com.

[Jason von Nieda]

That doesn't appear anywhere in the OpenPnP code. Are you using a fork?

Jason

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/a4338c31-a70c-491a-80b1-01456f00f6f0%40googlegroups.com.

[ca...@airdtf.com]

In order for FFmpeg to access the device and change the crossbar / take a picture, the driver has to be released by OpenPNP. It doesn't look like this function is available in the Camera service interface.

by "update the system" I mean capture an image and update the live view with a current capture. I can't seem to find any button or menu to do this manually with ImageCamera.

Installed splitcamera - this supports the crossbar, and the input can be selected manually via a menu. However, FFmpeg still can't change the crossbar value without taking over the driver, so splitcamera has to be closed. And when it starts again, it defaults to crossbar = 1. Still, it's an improvement :)

[ca...@airdtf.com]

Yes - part of the TVMdriver. This string appears in his driver, so I'll contact him, thanks.

[Jason von Nieda]

On Tue, Jan 10, 2017 at 9:51 AM <ca...@airdtf.com> wrote:

In order for FFmpeg to access the device and change the crossbar / take a picture, the driver has to be released by OpenPNP. It doesn't look like this function is available in the Camera service interface.

Yes, modifications will need to be made to support that functionality. The correct way to go about that is to create a new implementation of Camera (like OpenCvCamera) and put your custom functionality there. 

As a temporary hack, you can call OpenCvCamera.setDeviceIndex(some_large_number) to shut down the camera and then call setDeviceIndex(the_original_value) to reopen it. This is how the configuration pane handles updating the device.

by "update the system" I mean capture an image and update the live view with a current capture. I can't seem to find any button or menu to do this manually with ImageCamera.

New frames are sent to the camera subsystem automatically by the camera implementations. There is no way to send a new frame manually. It sounds like what you are trying to do is change the image that ImageCamera shows - to do that you should use ImageCamera.setSourceUri(). Look at the source to see how to use it.

Again, the right way to do this is to create a new Camera type that handles your functionality. If you are able to reliably get frames with FFmpeg then create a new FfmpegCamera that talks to it the way you want and you should be able to get reliable images.

Jason

 

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/df56960f-0173-4cbb-a94c-b52da641dd1c%40googlegroups.com.

[ca...@airdtf.com]

Alright, thanks - I'll try again when these features are implemented. It would be great to try OpenPNP in a real production environment, so I have no problem waiting until this machine is supported.

[ca...@airdtf.com]

As a temporary hack, you can call OpenCvCamera.setDeviceIndex(some_large_number) to shut down the camera and then call setDeviceIndex(the_original_value) to reopen it. This is how the configuration pane handles updating the device.

This does work - it takes several seconds on each switch using scripting, but it works!

 

New frames are sent to the camera subsystem automatically by the camera implementations. There is no way to send a new frame manually. It sounds like what you are trying to do is change the image that ImageCamera shows - to do that you should use ImageCamera.setSourceUri(). Look at the source to see how to use it.

The source URI is set to a .jpg file - but changing the file (via ffmpeg in this case) doesn't update the live view.

[Jason von Nieda]

ImageCamera loads the file when setSourceUri is called and then caches it. Loading it from disk for every frame would be far too slow. You can refresh it by calling setSourceUri with the same filename. This will cause it to reload the file.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/001821e1-4d81-4031-86b3-4eead9b6d5ac%40googlegroups.com.

[ca...@airdtf.com]

Thanks for that info - I'll use that for now.

I've already submitted an issue to OpenCV for their crossbar support, hopefully it's fixed in the next release, then your openCV driver can be used to great success.

[Cri S]

ffmpeg -f dshow -show_video_crossbar_dialog true -crossbar_video_input_pin_number 1 -i video="AVerMedia BDA Analog Capture"
This can work, but then you need kill it.

Try this: If you switch Id, as example having two opencv cameras with id 2000 and 2001, respective up and down camera.
Now you take a image with ffmpeg and switching crossbar to up camera using dshow.
After that assign camera Id 1400 if camera 0 to upcamera. If you have set format to pal-b too it should work. For switching to down camera, restore Id 2000 to upcamera , invoke ffmpeg to switch crossbar and take picture, configure id 1400 to downcam, wait the 8 seconds, return.
If downcam or upcam already have Id 1400, where the script should switch to down/up-cam , return immediately as its already working.

[BendRocks]

Carl, I'm not sure that adding crossbar to openCV will fix this as there seems to be an issue that for some reason the VideoInput.cpp crossbar doesn't work with this card. I was in the middle of trying this but then was thwarted after I moved to Win10 (where it doesn't work for sure). 

But it's easy to try if you wish. Go to github and get the VideoInput sources. There is a VisualStudio project in there called VideoInputBasicDemo. It will compile fine with a single cast you will need to add (the compiler will tell you where). There is a file in there called triangleApp.cpp with a function called keydown()

If you add the two lines show below, then you can switch video crossbar via Z and X keys. If that works on TVM machine with Win7, then I think we're in good shape. But if not, then there's something deeper going on that needs to be understood in the VideoInputBasicDemo project. If we can't make it work there, then it adding crossbar support to OpenCV won't work because OpenCV relies on VideoInput (Cri is the expert here).

if (c == 'Z') VI.setupDevice(0, 0);

if (c == 'X') VI.setupDevice(0, 1);

https://github.com/ofTheo/videoInput

[Cri S]

Setup must be made before grabbing first image,
Line with CIF and format, otherwise default format is ntfs that's not work on pal and crossbar is not used.
Its possible that stopping camera, making cross switch and restarting camera is needed.
If that's works java wrappers are already written, otherwise crossswitch is accessible but this then require more info about device (graph builder) and updating java wrapper.
Videoinputlib can work on win10 too, and the sample application triangle... Works only with webcams.

[BendRocks]

I've put up the source of a first version of a proof-of-concept vision DLL for the TVM920. This was my last resort path, but the VideoInput library could not be made to work by me in various c++ test programs using 2 different video cards. And even if I could make that work, then that has to be fed back into OpenCV, and (I think) a new path to set input mux created. It's a lot of pieces to get working.

What I could make work (in 5 minutes, which is always a good sign) was test code in c# using aforge, and that worked with both cards--the original SAA7134 card and a new multi-input Osprey 100 (not the 100e! that won't fit the PCI). Unfortunately, the default QiHe program doesn't work with the new Osprey card. It almost does. The down camera is fine. But the up camera is using what appears to be the S-Video input.

I'll add the Osprey card is very good. The software works with everything I've tried and the config software is great. I suspect now that everything is settled down, if I went back to the original card on Win10 it would probably work. I'll try that later. 

The performance of the dll is reasonable. It takes 4 mS to grab a 720x576 32bpp image. Process memory is about 18Mbytes. 

If you look at the code, you'll see a bunch of DLLs. The code that does the work is C#.  C# DLLs aren't readily useful by any runtime except dotnet. And so, to get to a normal C DLL , you ease your way into it. You first create a bridge DLL in managed C++, and then you call that via a normal C DLL. The alternative was COM. No thanks. 

The DLL has just a few calls:

Stop(). This will stop the camera thread. Call before closing. Start happens automatically.

GetData(). Returns a linear array of bytes that is a direct copy of the uncompressed bitmap. The first 4 bytes of the array contain two words representing the width and height of the bitmap. Calling code needs to explicitly dispose of the image when done.

SetInput()

The setinput() will select the front-end mux of the video card. It takes a bit for the image to change, as much as a few hundred ms. Before then the image is very noisy as one input is turning off and another turning on. But in practice this isn't a big deal as the camera is looking down 100% of the time when you wish to go fast. 

The SetInput() will also send the UDP packets to change the lighting from uplight to downlight. There is a heartbeat UDP packet required on the TVM, and I was wondering if the light control would need to satisfy the heartbeat, and fortunately it does not. 

Next, I'll use JNI to call from a Java camera driver to get the image. 

Source link is below, though the current state probably wont' be useful to anyone. I doubt this will compile if you download it, but if you are comfortable with VS it should be pretty quick to get going. 

PS. I'll likely be sending byte array of PNG across the interface instead of image bytes. More later.

https://github.com/BendRocks/TVM920Vision

[Jason von Nieda]

Nice work BenRocks, glad to see this is coming along!

You may want to take a look at http://jni4net.com/ and https://github.com/nativelibs4java/BridJ. I don't have any experience with jni4net, but if it works it may allow you to skip several steps. If not, I do have experience with BridJ and it should allow you to at least skip the C step. It also makes it easy to bundle DLLs into a distributable jar file.

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/87cab7d7-bdc9-469b-b457-953c0e9172b2%40googlegroups.com.

[Glen English]

NOTE

"The SetInput() will also send the UDP packets to change the lighting from uplight to downlight. There is a heartbeat UDP packet required on the TVM, and I was wondering if the light control would need to satisfy the heartbeat, and fortunately it does not. "

Note all that is handled in my TVMdriver,exe

100% of it. every single thing..... I am expecting all camera operations to be handled directly from OpenPNP....

I could integrate your vision switching code into TVMdriver project, but figured that was a camera op....

glen

On Monday, January 16, 2017 at 6:48:13 PM UTC+11, BendRocks wrote:

[BendRocks]

It's just a few lines of code to turn the leds on and off, and since it's UDP, the client doesn't care (or know) where it came from. The heartbeat is similarly simple if it were needed. I've started working on java code for direct control of the TVM motion. That path makes the most sense to me.

Long term, I'd prefer to have OpenCV implement the switching and dump the camera DLL in order to keep everything in Java. But man, OpenCV is too big for me to get my arms around at the moment. 

[Glen English]

contact me directly for source and protocol if you need it,

it's on a public FTP. a few people have it

gpios are simply set and reset commands, so you are right - state free from client POV

Agreed, OpenCV takes a bit to get your around....

[BendRocks]

Well I have thrown in the towel on this. After getting the new video card and getting it working in c# quickly, I wasted 12 hours ultimately getting it called from OpenPNP via JNA. The JNA part was easy, the DLLs to go from normal C DLL to managed code was painful and long, with issues ranging from marshaling buffers, x32 and x64 DLL interop, to a memory leak. I finally got all that working from OpenPNP, but but it wouldn't stay working more than 30 mins or so. Memory use was fine, CPU was fine. The code just disappeared into JNA and never came back. And I figured I could either get really good at JNA **OR** try something else. 

And so, I got a $39 USB camera from Amazon, pulled out the up looking camera in the TVM and replaced it with the USB camera (all the screw holes just worked). And now I use the stock DL camera and the analog capture card for that, and the USB camera for the UL camera. And everything is happy. The swap was easy. The sensor in the stock UL camera was huge compared to the amazon camera, which meant the stock lens probably wouldn't work. But in the end it did--just barely--with 3 or 4 rotations of thread left. I have 22 mm of view, which is larger than the stock camera (which had about 18) and I get more resolution to boot. 

I cannot tell you how much I detest directshow. Beyond words. 

https://www.amazon.com/gp/product/B00UMX3HEG

[Jason von Nieda]

Sorry to hear it didn't work out. I know how frustrating this can be. It sounds like you now have a working solution for little trouble, so I guess that is a win :)

Jason

--
You received this message because you are subscribed to the Google Groups "OpenPnP" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openpnp+u...@googlegroups.com.
To post to this group, send email to ope...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/openpnp/df97f27f-a76d-49d4-a2c5-44b883df77b5%40googlegroups.com.

[Cri S]

OK, now its seems you are operational in short time. You surely have made real cph measurements
on original software and real PCB.
I'm interested to the different cph, original/openpnp on same PCB and feeder setup.