Suggested tactics for LIVE video with NO recognition?

[notarysojac]

A problem I have is that the ZXing engine is SO quick, that the video in the app window is not getting updated before the recognition process is completed.  

The result is I'm not seeing video in the 'frame' in my window because, when the recognition proces stops, the live video feed also stops. 

What would be cool is a way to tell the ZXing engine "Pass thru the video but don't perform recognition on the video contents until told to do so".

Is this even possible with the current architecture of the ZXing lib?  

BTW - I'm working with objective-C in MacOS - this seems to be less of an issue with the iOS environment.

Thanks for whatever tips and/or hints may be forthcoming.  I can probably figure out a way to modify the ZXing lib(s) but thought maybe somebody has already dealt with this issue. 

[notarysojac]

Well I pretty much answered my own question - to my satisfaction, anyway - but am certainly interested in hearing from anyone with a better idea.

My solution in Objective-C: 

1) create a BOOL (with property) named 'recognize' in the ZXCapture object

2) observe the 'recognize property in - (void) captureOutput.... (see below)

by bracketing the execution of the decode process in ZXCapture:: captureOutput 

...

if(YES == [self recognize])  // this conditional was added to the code

  {

  if (nil != captureDelegate)

    {

    ZXDecodeHints*  hints   = [[[ZXDecodeHints alloc] init] autorelease];

    ZXBinaryBitmap* bitmap  = [[[ZXBinaryBitmap alloc] initWithBinarizer:binarizer] autorelease];

    @try 

      {

      ZXQRCodeReader* reader = [[[ZXQRCodeReader alloc] init] autorelease];

      // NSLog(@"started decode");

      ZXResult* result = [reader decode:bitmap hints:hints];

      // NSLog(@"finished decode");

      [captureDelegate captureResult:self result:result];

      }

      

    @catch (ZXReaderException* rex) 

      {

      if (![rex.reason isEqualToString:@"Could not find three finder patterns"]) 

        {

        // NSLog(@"failed to decode, caught ReaderException '%@'", rex.reason);

        }

      }

      

    @catch (ZXIllegalArgumentException* iex) 

      {

      // NSLog(@"failed to decode, caught IllegalArgumentException '%@'", iex.reason);

      }

      

    @catch (id ue) 

      {

      NSLog(@"Caught unknown exception: %@", ue);

      }

    }

  }

...

4) Start the ZXingEngine - then enable 'recognize' by a call into the ZXCapture object when it makes sense

[Steven Parkes]

On Sep 6, 2012, at 6:58 AM, notarysojac <hydrat...@gmail.com> wrote:

> A problem I have is that the ZXing engine is SO quick, that the video in the app window is not getting updated before the recognition process is completed.
> The result is I'm not seeing video in the 'frame' in my window because, when the recognition proces stops, the live video feed also stops.
>
> What would be cool is a way to tell the ZXing engine "Pass thru the video but don't perform recognition on the video contents until told to do so".

Not sure the API makes it possible to break that out at this point. It might. You want to add the preview layer without enabling the encoding. There are "start" and "stop' kinds of things but they may start and stop the preview as well.

I think you can achieve something similar by just letting the decode run but not terminate it on the first successful decode. I think the little test app does that; it never terminates. This would spend more CPU on the decode when you were going to ignore the result but it doesn't sound like that would matter if it was brief, just until the preview layer got a chance to appear.

[notarysojac]

Thanks Steven, 

Of course, everyone's desired behaviors can be different.  For me, this toggle on/off of the recognize boolean works perfectly as per my wishes.  

The ZXing engine continues to run - the video is live - the recognition of the QRcode happens when I enable the bool - when the results show up the recognition bool is set to NO - the video remains active but I have a copy of the decoded QRcode - and the decode process can be rejoined at any time.  Life is good!  

If you would like a copy of my edited ZXCapture object, I'd be happy to send it to you for your review.  

Meanwhile, this is a perfect demonstration of the beauty of open-source code - because I get the behaviors I need and those changes may percolate into the source code - or not... either way, as Denzel said to Ethan Hawke in 'Training Day', "It's all good, bro..."   

BTW - the ZXing project is a joy to work with and I reeeeely appreciate the Objective-C work that been included.    Thanks MUCH!

[Steven Parkes]

I saw your comment after my post.

A patch against trunk would be better than a copy of your edited file.

If we were going to go this way, it'd be even better to not capture the frames. There's a fair amount of work involved there that is being wasted.

But like I said, just ignoring the decodes should work to. It does seem like this is a corner case ...

Do I remember correctly that if you don't set the delegate the preview doesn't run? If not, that's an easy choice, too.

> --
>
>
>

[Pratik]

Hello notarysojac,

I am working on some Application in which I want to have Live Video from my webcam, and There are QRcodes in that Live video. So, I want to decode that QR codes and Stream that Video. What I am doing is capturing the video and making frames out of it using opencv and javacv and then from those frames tring to decode the QRtag using Zxing Library. But sometimes it is able to decode and sometime not. Do you have any idea how can make it Perfect. Should I change frames into Grayscale or better resolution of Camera will work much better. Any suggestion, Ideas.....

Thanks

Pratik

[notarysojac]

Hi Pratik,

Not sure why it is that your QRcodes are not decoding - but it would seem that, based on the various implementations of the ZXing engine in mobile devices where the input to the decoder is video frames, you should be easily able to make things work with a webcam.  

If you are saving off individual frames and then sending those to a decoder, is it possible that you are compressing the frames and introducing artifacts into the image as a result?  I am amazed at how well the ZXing engine can - given the relatively small number of pixels in a video frame - do any decoding at all - but it appears to be able to work very well under incredibly demanding conditions.  

Another possibility - if you have QRcodes which are 'dense' - contain a large number of bytes of data - you may be able to improve your decoding by not stuffing so much info into the QRcoded image.  If that's not your problem, perhaps you are trying to decode in a situation where there's not much light?  Video cameras are notoriously noisy in low-light.   Also - if the QRcode is a tiny portion of the overall video image, you may have trouble decoding because of the fewer number of pixels which comprise the QRcode. Maybe the camera is moving too quickly and you're getting blurred images?   Does your webcam create 320x240 pixel or 640x480 pixel images?  Can you focus the webcam?  I have used a MacAlly ICECAM2 with very good results - If I have a 3" square QRcode printed on a piece of paper and hold it in front of the webcam, the results come back almost instantaneously.  

Well - as with most engineering problems - you will need to determine what really is the cause of the problem - then go about implementing a remedy.  

Good luck!

[notarysojac]

Hi Steven, 

Exactly where an intercept might occur so that the decoder would not be invoked is an important consideration.  I appreciate your observation that I have not chosen the most optimum place to place that 'switch'.   BTW - Yes - setting the delegate to nil causes the video stream to become inactive. 

As far as this capability being a corner case, it could be argued that, in the presence of multiple QRcodes (or other barcode formats) it would be useful to be able to inform ZXing, "Decode THIS one - not the one you just panned past".  Suffice to say, it's now working for me exactly as I would wish and I am so glad and grateful to be able to make use of the ZXing project materials. 

With regards creating and submitting a patch against trunk, it would seem a better path if somebody from your camp added <whatever changes> in the places and manner which make the most sense to you.  It's almost a sure bet my tweaks would get backed out and/or revised.  If nobody else cares about the capability to enable and disable the decoding without losing the video stream, then there's no loss.  

Thanx!!

[Steven Parkes]

On Sep 8, 2012, at 3:34 PM, notarysojac <hydrat...@gmail.com> wrote:

> As far as this capability being a corner case, it could be argued that, in the presence of multiple QRcodes (or other barcode formats) it would be useful to be able to inform ZXing, "Decode THIS one - not the one you just panned past".

Not saying it's not useful, just that it's a corner case. Even as described above, it seems to me a corner/uncommon case.

Being a corner case doesn't mean don't do it, just that it should not negatively impact the common case in either performance or code organziation.

> With regards creating and submitting a patch against trunk, it would seem a better path if somebody from your camp added <whatever changes> in the places and manner which make the most sense to you. It's almost a sure bet my tweaks would get backed out and/or revised.

I understand the feeling, but that doesn't (in my experience) work out that well in practice in open source. Everybody's busy, so often the most effective path is for people to scratch their own itches. That does often mean there is iteration where a patch gets feedback from others and then the author has to tweak things. This can seem aggravating but often it's the only way to make progress. I can give feedback more easily than I can take on the responsibility to write/rewrite something. The hope is that people learn though iteration and require less iteration in the future.