Camera Preview for ZXing
Anil
I am trying to build a custom QR Scanner using the ZXing for Android
and I hit a few roadblocks that I cannot get past:
1. I like how Barcode Scanner client captures a QR without clicking a
button and I want to replicate it but I cannot understand from the
code how this was accomplished. Since I don't want the button for
capturing QR, the takePicture() method is ruled out. But, I can't
understand how the Camera.Preview reads the QR without any user
interaction.
2. Also, how do I display the results back after the QR was read? I
understand onActivityResult is expected to do that but I am not sure
how to code it. I guess I am struggling because I don't see the
methods linked to one another.
3. Finally, should I look at creating each UI screen as an individual
Intent? Is that a better a design?
Thank you.
Best,
Anil.
Lachezar Dobrev
2011/3/11 Anil <gan...@gmail.com>:
> Hi:
>
> I am trying to build a custom QR Scanner using the ZXing for Android
> and I hit a few roadblocks that I cannot get past:
Unless you're trying to do something very unorthodox, consider using
Scanning-Via-Intent method.
http://code.google.com/p/zxing/wiki/ScanningViaIntent
> 1. I like how Barcode Scanner client captures a QR without clicking a
> button and I want to replicate it but I cannot understand from the
> code how this was accomplished. Since I don't want the button for
> capturing QR, the takePicture() method is ruled out. But, I can't
> understand how the Camera.Preview reads the QR without any user
> interaction.
The application uses Camera.startPreview() and then
Camera.setOneShotPreviewCallback().
The camera driver generates preview frames and sends them to the
preview call-back.
Take a look at:
http://developer.android.com/resources/samples/ApiDemos/src/com/example/android/apis/graphics/CameraPreview.html
> 2. Also, how do I display the results back after the QR was read? I
> understand onActivityResult is expected to do that but I am not sure
> how to code it. I guess I am struggling because I don't see the
> methods linked to one another.
The onActivityResult() is called when using Scanning-Via-Intent method.
If you decide to replicate the scanning method you will have to feed
the preview frame to an instance of a Reader hierarchy.
> 3. Finally, should I look at creating each UI screen as an individual
> Intent? Is that a better a design?
This is pretty much general Android question, and Activity is the
equivalent of a Window/Frame. Exceptions to this general rule do
exist.
> Thank you.
>
> Best,
> Anil.
Anil
Unfortunately, scanning via intent is not an option because that would
mean dependency on the Barcode Scanner. I'd love to just use Barcode
Scanner and not reinvent the wheel.
If I understand you correctly, setOneShotPreviewCallback() sends
single frames without any user interaction. Let me play around with
that.
But, when is a frame considered complete and how do I know that so
that I can process that frame like, send it to a decoder?
On Mar 11, 1:46 am, Lachezar Dobrev <l.dob...@gmail.com> wrote:
> Inline...
>
> 2011/3/11 Anil <gan...@gmail.com>:
>
> > Hi:
>
> > I am trying to build a custom QR Scanner using the ZXing for Android
> > and I hit a few roadblocks that I cannot get past:
>
> Unless you're trying to do something very unorthodox, consider using
> Scanning-Via-Intent method.
> http://code.google.com/p/zxing/wiki/ScanningViaIntent
>
> > 1. I like how Barcode Scanner client captures a QR without clicking a
> > button and I want to replicate it but I cannot understand from the
> > code how this was accomplished. Since I don't want the button for
> > capturing QR, the takePicture() method is ruled out. But, I can't
> > understand how the Camera.Preview reads the QR without any user
> > interaction.
>
> The application uses Camera.startPreview() and then
> Camera.setOneShotPreviewCallback().
> The camera driver generates preview frames and sends them to the
> preview call-back.
>
> Take a look at:
Lachezar Dobrev
2011/3/12 Anil <gan...@gmail.com>:
> Thank you for your response.
>
> Unfortunately, scanning via intent is not an option because that would
> mean dependency on the Barcode Scanner. I'd love to just use Barcode
> Scanner and not reinvent the wheel.
I don't exactly understand your point of view, although I am also
using just the library.
In my case I needed a bulk scanner with interactivity on every scan,
that was not achievable with the Scanning-Via-Intent method.
> If I understand you correctly, setOneShotPreviewCallback() sends
> single frames without any user interaction. Let me play around with
> that.
Well... You need to put a SurfaceView in your Activity, commonly at
the background plane. Then connect the camera to the surface to enable
preview.
The one-shot preview shoots ONLY ONE frame, and de-registers the
call-back. You will need to continuously poll for frames.
An alternative method is to set a long-term preview call-back with
the setPreviewCallback(...) method, but this shoots tens of frames per
second, and you may end up with a severe resource depletion if you're
unable to process frames fast enough, not to mention battery drain.
> But, when is a frame considered complete and how do I know that so
> that I can process that frame like, send it to a decoder?
The frame that you get IS complete, meaning it represents a full
frame of the preview. The call-back should send it to the decoder
immediately, and request a new One-Shot preview immediately after
processing the frame, or if you choose to do so, with a small delay.
Do not forget to call Camera.autoFocus(...) from time to time. While
running auto-focus you may or may not get preview frames. After the
auto-focus completes the call-back you provided should start
requesting frames again.
Anil
Barcode Scanner has to be pre-installed and unfortunately, that is not
an option that my company is willing to live with.
Per your suggestions, after using setOneShotPreviewCallback() I am
able to get a single frame but I am running into a NotFoundException
when the frame is sent for decoding when I execute the code using the
G1 which as 480X320 resolution. I used DecodeHintType as well but no
luck.
Of course, when I use the Barcode Scanner app, I am able to decode the
QR but not with my app. Would you be able specify what is that I am
doing wrong in my code?
public void onPreviewFrame(byte[] cameraData, Camera arg1) {
QRCodeReader qrReader = new QRCodeReader();
Result result = null;
Hashtable<DecodeHintType, Boolean> hints = new
Hashtable<DecodeHintType, Boolean>();
if (cameraData != null) {
Log.i(TAG, "$$$ Anil clicked image size is: " + cameraData.length +
" .$$$");
//Emulator: 176x144
//Phone (G1): 480x320
Log.i(TAG, "Size width: " + size.width + "Size height: " +
size.height);
PlanarYUVLuminanceSource source = new
PlanarYUVLuminanceSource(cameraData,
size.width, size.height, 100, 60, 380, 260);
//GlobalHistogramBinarizer binarizer = new
GlobalHistogramBinarizer(source);
HybridBinarizer binarizer = new HybridBinarizer(source);
BinaryBitmap bitmap = new BinaryBitmap(binarizer);
hints.put(DecodeHintType.TRY_HARDER, Boolean.TRUE);
Log.i(TAG, "Got bitmap from LuminanceSource");
try {
result = qrReader.decode(bitmap, hints);
Log.i(TAG, "Finished decoding the QR in onPreviewFrame: " +
result.getText());
} catch (NotFoundException nfe) {
Log.e(TAG, "Could not detect Barcode in the image: ");
nfe.printStackTrace();
} catch (ChecksumException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (FormatException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} finally {
qrReader.reset();
}
//now that we have the result
if (result != null) {
mCamera.stopPreview();
mCamera.release();
Log.i(TAG, "Result not null in onPreviewFrame");
onValidDecodeResult(result,
source.renderCroppedGreyscaleBitmap());
} else {
mCamera.setOneShotPreviewCallback(this);
} //end of if-else
}
} //end of onPreviewFrame
Parameters
Lachezar Dobrev
2011/3/14 Anil <gan...@gmail.com>:
> Well, if I have to have my app leverage Scanning Via Intent then,
> Barcode Scanner has to be pre-installed and unfortunately, that is not
> an option that my company is willing to live with.
Well... That's one way of putting it.
There is an Android Integration project, that handles whether the
Scanner is installed, and directs the user to install it if needed.
http://zxing.googlecode.com/svn/trunk/android-integration/
Take into consideration, that if you decide to do it yourself you
will have to write much code, that the ZXing project has already done.
> Per your suggestions, after using setOneShotPreviewCallback() I am
> able to get a single frame but I am running into a NotFoundException
> when the frame is sent for decoding when I execute the code using the
> G1 which as 480X320 resolution. I used DecodeHintType as well but no
> luck.
Getting a NotFoundException is very much expected.
The preview-based scanning is built around a brute-force approach.
This means, that the scanner relies on trying many times with
relatively bad quality frames and relatively low success rate, rather
than trying once with a higher success rate and better quality.
That said, you should use a GlobalHistogramBinarizer, because it is
faster although provides less quality. Also ditch the TRY_HARDER hint,
it makes things slower.
I think your problem is lack of focus. Call Camera.autoFocus(...).
As you may have noticed the scanner application does some continuous
auto-focusing.
Also... I don't understand why you are cropping the frame
(PlanarYUVLuminanceSource arguments). Use the full size.
> Of course, when I use the Barcode Scanner app, I am able to decode the
> QR but not with my app. Would you be able specify what is that I am
> doing wrong in my code?
That is one of the very serious reasons to use the Scanner
Application to do the job for you.
Anil Bhargav
I am not sure where to call the camera.autofocus() in my code because
I have an Activity1, once my App is launched, I click "Start Scan" in
Activity1 which calls Activity2 (where onPreviewFrame exists) that
sets up the camera. I looked at some examples but all of them call
autofocus in onClick but I can't do the same in my app.
Also, I cooked up those numbers in PlanarYUVLuminanceSource
considering that my G1 has 480x320 resolution and also to pass the if
condition, "if (left + width > dataWidth || top + right >
dataHeight)".
simplecy
regards
Simon
Sean Owen
Anil Bhargav
to do that as a last resort and there is no fun in building your app
that way.
Currently, I am just using the core.jar and hopefully once I resolve
the autofocus and PlanarYUVLuminanceSource issues, my app should be up
and running.
Anil Bhargav
terms aside, it is no fun calling it my app if all I have done is copy
your code.
simplecy
AFICT he's not planning on making a barcode reading app - he just wants to read barcodes within his app and is struggling to add ZXing code to his app because its very tricky to reverse engineer out the bits he needs.
So I'm suggesting going at it the other way, compile Zxing and add in his code (Whatever it is ) and remove the bits of ZXing he doesn't need.
I'd expect that his app would only resemble Zxing in that there'd be a window with a redline in it :)
A= ZXing code ,B =Anil's Code
A+B == B+A :)
regards
Simon
Lachezar Dobrev
2011/3/14 Anil Bhargav <gan...@gmail.com>:
> Yes, I understand and hence I mentioned, I am reinventing the wheel.
>
> I am not sure where to call the camera.autofocus() in my code because
> I have an Activity1, once my App is launched, I click "Start Scan" in
> Activity1 which calls Activity2 (where onPreviewFrame exists) that
> sets up the camera. I looked at some examples but all of them call
> autofocus in onClick but I can't do the same in my app.
Auto-focus is an asynchronous process.
You could call auto-focus every 2 seconds for instance. The else
case in the preview listener should do the trick: if it's more than
two seconds since last auto-focus, then do auto-focus, otherwise
request a new frame.
For instance, you may enhance your Preview Listener to implement
Autofocus Call Back:
//now that we have the result
if (result != null) {
mCamera.stopPreview();
mCamera.release();
Log.i(TAG, "Result not null in onPreviewFrame");
onValidDecodeResult(result, source.renderCroppedGreyscaleBitmap());
} else if ((this.lastAutoFocus + 2000) < System.currenTimeMillis())
// Request an Auto-Focus
this.lastAutoFocus = System.currentTimeMillis();
mCamera.autoFocus(this);
} else {
mCamera.setOneShotPreviewCallback(this);
} //end of if-else
...
//
public void onAutoFocus(boolean success, Camera camera) {
this.lastAutoFocus = System.currentTimeMillis();
// After auto-focusing request a new frame.
mCamera.setOneShotPreviewCallback(this);
}
Easy as 3.14.
> Also, I cooked up those numbers in PlanarYUVLuminanceSource
> considering that my G1 has 480x320 resolution and also to pass the if
> condition, "if (left + width > dataWidth || top + right >
> dataHeight)".
These are not conditions, but sanity checks.
Anil Bhargav
So what values should I be passing to PlanarYUVLuminanceSource
constructor? Again, I can't hard code values there because the values
will change for every device. Alternatively, I have to use ZXing's
CameraManager to get the LuminanceSource.
I am still getting the NotFoundException possibly because I am
chopping the image w/ inaccurate values in PlanarYUVLuminanceSource?
Lachezar Dobrev
> new PlanarYUVLuminanceSource(cameraData, size.width, size.height, 0, 0, size.width, size.height);
2011/3/14 Anil Bhargav <gan...@gmail.com>:
Anil Bhargav
scanner, my app doesn't wait for the user to focus on the barcode in
order to capture the QR. My app reads a frame, sends that to decode
and returns to home screen.
The camera is capturing the image, as I see the image size as 230400
bytes. However, the app is failing to detect the QR in this raw image.
The image quality cannot be bad because Barcode Scanner is able to
read it. I am assuming that my app will fail to process the QR even if
I saved the image on the device and sent it for decoding.
Apurv Gupta
Apurv Gupta
David Williams
the camera is constantly scanning until it finds the lines that
match a barcode edge
qr stands for quick response note the edges but once zebra identifies
the edge of the bit map then it reads the lines where there is
on or off pixel which translates into a bitmap value
no just define you barcode reader encoder and add it to
the types make your own encoder decoder
Lachezar Dobrev
2011/3/14 Anil Bhargav <gan...@gmail.com>:
> No luck, continue to get the NotFoundException. Unlike, barcode
> scanner, my app doesn't wait for the user to focus on the barcode in
> order to capture the QR. My app reads a frame, sends that to decode
> and returns to home screen.
No, that's not how it should work!
You need to constantly pull frames! You are NOT going to be able to
scan the first frame, unless you're extremely lucky!
You wait for either a successful read indefinitely.
Also handle the onPause() in your activity to disconnect from the camera.
> The camera is capturing the image, as I see the image size as 230400
> bytes. However, the app is failing to detect the QR in this raw image.
> The image quality cannot be bad because Barcode Scanner is able to
> read it. I am assuming that my app will fail to process the QR even if
> I saved the image on the device and sent it for decoding.
By bad quality I do not mean only bad picture quality, but also
over-exposures, under-exposures, focus blur, motion blur, etc. These
all contribute to the image being poorly analysable.
Initially when you start the camera, it commonly is either
over-exposed, or under-exposed, and I'm pretty sure it's out-of-focus.
Do continuous scanning.
Anil Bhargav
I did not make much progress.
Anil Bhargav
not if having that would help decode the image. I think I'll try and
incorporate the preview rectangle and see if that makes any
difference.
Anil Bhargav
stopped and the app returns to the main activity.
I can't make much of the stack trace either; here is what I see in
DDMS and I am not sure if the debug statements are giving out
something that I cannot decipher:
03-18 16:23:43.499: INFO/DEBUG(48): *** *** *** *** *** *** *** ***
*** *** *** *** *** *** *** ***
03-18 16:23:43.499: INFO/DEBUG(48): Build fingerprint: 'android-
devphone1/dream_devphone/dream/trout:1.6/DRC83/14721:user/gfh,test-
keys'
03-18 16:23:43.499: INFO/DEBUG(48): pid: 3453, tid: 3453 >>>
com.softclouds.eventscan.qrcrm <<<
03-18 16:23:43.509: INFO/DEBUG(48): signal 11 (SIGSEGV), fault addr
00000008
03-18 16:23:43.509: INFO/DEBUG(48): r0 00000000 r1 bec19780 r2
410546c0 r3 0000bc48
03-18 16:23:43.509: INFO/DEBUG(48): r4 bec19780 r5 4104ed90 r6
bec19778 r7 ad00e5c0
03-18 16:23:43.509: INFO/DEBUG(48): r8 00001071 r9 0000bc48 10
4104ed6c fp 00000000
03-18 16:23:43.509: INFO/DEBUG(48): ip 00000000 sp bec1972c lr
ad0594cb pc ad03a92a cpsr 80000030
03-18 16:23:44.639: DEBUG/WND-DBG(2171): Exception :Getting Active
Network state::null
03-18 16:23:44.839: DEBUG/WND-DBG(2171): Idle time : 1300490624713,
waiting for widgets to settle
03-18 16:23:45.819: INFO/DEBUG(48): #00 pc 0003a92a /system/
lib/libdvm.so
03-18 16:23:45.819: INFO/DEBUG(48): #01 lr ad0594cb /system/
lib/libdvm.so
03-18 16:23:45.819: INFO/DEBUG(48): stack:
03-18 16:23:45.819: INFO/DEBUG(48): bec196ec 00000001
03-18 16:23:45.819: INFO/DEBUG(48): bec196f0 00000007
03-18 16:23:45.819: INFO/DEBUG(48): bec196f4 ad047361 /system/
lib/libdvm.so
03-18 16:23:45.829: INFO/DEBUG(48): bec196f8 4108ea70 /dev/
ashmem/dalvik-LinearAlloc (deleted)
03-18 16:23:45.839: INFO/DEBUG(48): bec196fc 0000bc48 [heap]
03-18 16:23:45.839: INFO/DEBUG(48): bec19700 ad3442f9 /system/
lib/libandroid_runtime.so
03-18 16:23:45.839: INFO/DEBUG(48): bec19704 ad040b19 /system/
lib/libdvm.so
03-18 16:23:45.839: INFO/DEBUG(48): bec19708 4104ed44
03-18 16:23:45.839: INFO/DEBUG(48): bec1970c 41c6ed1d /system/
framework/framework.odex
03-18 16:23:45.839: INFO/DEBUG(48): bec19710 ad3442f9 /system/
lib/libandroid_runtime.so
03-18 16:23:45.839: INFO/DEBUG(48): bec19714 bec19780 [stack]
03-18 16:23:45.839: INFO/DEBUG(48): bec19718 bec19780 [stack]
03-18 16:23:45.839: INFO/DEBUG(48): bec1971c 00000001
03-18 16:23:45.849: INFO/DEBUG(48): bec19720 df002777
03-18 16:23:45.849: INFO/DEBUG(48): bec19724 e3a070ad
03-18 16:23:45.849: INFO/DEBUG(48): bec19728 4104ed60
03-18 16:23:45.849: INFO/DEBUG(48): #00 bec1972c ad0594cb /system/
lib/libdvm.so
03-18 16:23:45.849: INFO/DEBUG(48): bec19730 414e7480 /system/
framework/core.odex
03-18 16:23:45.849: INFO/DEBUG(48): bec19734 ad01319c /system/
lib/libdvm.so
03-18 16:23:45.849: INFO/DEBUG(48): bec19738 ad083e1c /system/
lib/libdvm.so
03-18 16:23:45.849: INFO/DEBUG(48): bec1973c ad084ee8 /system/
lib/libdvm.so
03-18 16:23:45.849: INFO/DEBUG(48): bec19740 bec19778 [stack]
03-18 16:23:45.849: INFO/DEBUG(48): bec19744 ad017b90 /system/
lib/libdvm.so
03-18 16:23:45.849: INFO/DEBUG(48): bec19748 0000032c
03-18 16:23:45.859: INFO/DEBUG(48): bec1974c bec19800 [stack]
03-18 16:23:45.859: INFO/DEBUG(48): bec19750 0000bc48 [heap]
03-18 16:23:45.859: INFO/DEBUG(48): bec19754 4104ef04
03-18 16:23:45.859: INFO/DEBUG(48): bec19758 00000000
03-18 16:23:45.859: INFO/DEBUG(48): bec1975c ad017be8 /system/
lib/libdvm.so
03-18 16:23:45.859: INFO/DEBUG(48): bec19760 00000028
03-18 16:23:45.859: INFO/DEBUG(48): bec19764 00000001
03-18 16:23:45.859: INFO/DEBUG(48): bec19768 0000bc48 [heap]
03-18 16:23:45.859: INFO/DEBUG(48): bec1976c ad017630 /system/
lib/libdvm.so
03-18 16:23:45.859: INFO/DEBUG(48): bec19770 00000048
Anil Bhargav
process it. So, I took the approach of using takePicture() and I could
get past the NotFoundException but now when I am processing the jpeg
image, I am getting the ChecksumException or FormatException.
Could this be attributed to the size of the image or the quality or
both?