Preliminary Patch for Issue 1351: Automatic flash setting when scene appears to be dark

[Nikolaus Huber]

I added the option to the preferences Activity and got the code in place for it to work.

The algorithm to determine if the torch should be turned on is still not very satisfying. For some reason the Image Format seems to be different than what it is supposed to be, I guess. It should be NV21, making the first width*height bytes the luminance levels between 16 and 235 (at least according to wikipedia) but this seems not to be the case =/

[Sean Owen]

This patch contains a whole lot of stuff, including your binary debug build. Can you whittle it down to just the changes that should go in?

The luminance plane should be first and is 8 bits per pixel AFAIK. You shouldn't have to care about it... really you want to operate on a LuminanceSource which abstracts this away from you. Hm, I think that is maybe why I thought it would be better to do this in DecodeHandler, since you have parsed the luminance at that stage already. It would be better not to parse twice.

(There's another complication in that the luminance source doesn't convert the whole image upfront as that would be wasteful. It converts rows on demand. But you you are scanning across a diagonal you'll touch many rows, which is expensive. You may want to instead sample a few rows.)

[Nikolaus Huber]

Here only the files I changed (I hope)

I looked at the DecodeHandler and I would honestly say that having the Auto Front Light Code in the decode() function would be quite weird (as it does not have anything to do with decoding). Furthermore, the byte Array held by LuminanceSource should be exactly the same byte Array I have access to in PreviewCallback, but with the restriction that I am only able to scan by rows. I couldn't find any code that actually manipulates the luminance values between PreviewCallback and LuminanceSource, so these should be the same. Having the code in PreviewCallback should make more sense (in my opinion) and also has easier access to CameraManager.

Thanks for helping me =)

[Sean Owen]

OK I will have a look tomorrow. It's not ideal to couple to the image format separately in another place but yes as a matter of practice the format is always going to be YUV planar. 

[Nikolaus Huber]

I just tweaked it a bit and the result is actually vaguely satisfying now (at least it detects my white door and black cupboard now). It should probably also have a time contraint somehow, so that it cannot switch between on and off too quickly.

[Nikolaus Huber]

What I could actually do, is transform the data into a LuminanceSource already in the PreviewCallback object and hand this to the Decoder instead of the data. This way the Auto Front Light function can be in PreviewCallback (and thus separate from the decoder and nearer to the CameraManager) and still only be accessed via LuminanceSource.

[Sean Owen]

OK, it's looking reasonably good. I've got the patch locally and can take it from here. I will try out the thresholds and see about making it a faster and using PlanarYUVLuminanceSource.

Daniel what do you think about adding another pref for this? There's some issue there.

[Sean Owen]

PS in NV21, the first 2/3 of the image data is the Y plane. The U and V planes are 4x smaller than Y because they are 2x2 subsampled, though there are 2 planes.

Starting 1/3 of the way through the data happens to be right as it's halfway into the Y plane.

From there it kind of doesn't matter what you sample. Looks like you are sampling 1/3 of a row, though the comment mentions sampling a diagonal. It might make more sense to sample a whole row. But, it doesn't matter a lot. You could sample 1000 pixels in a row and just use that and would be pretty fine too. It's less principled, but simpler, and more cache-friendly.

I think there are a few more loose ends here like making sure successive frames don't turn it on and off quickly, or handling a race condition with the user turning it on manually.

[Sean Owen]

My current patch.

The big issue I see right now is that when the light comes on, it becomes bright enough for the light to go off again. You can increase the threshold for 'off' but it may be a little problematic, since the device is also continually adjusting exposure to make dark images brighter.

Maybe it's not necessary to turn it off automatically?

Somehow I think the right answer here is to use the ambient light sensor (http://developer.android.com/reference/android/hardware/SensorEvent.html#values). It's easy to register a callback to detect current luminance is a more reliable way regularly.

I know not all devices have one, but most will.

---

http://developer.android.com/reference/android/hardware/SensorEvent.html#values

[Nikolaus Huber]

I built on top of your patch:

I included the ambient light sensor. It should also be stopped correctly when the activity is on hold (I hope. I used closeDriver from CameraManager). It now uses a combination of ambient light and the preview frame to determine the torch setting. Values might need a bit of tweaking.

I also changed it to the LuminanceSource class. The data now already gets transformed in PreviewCallback instead of in the decoder.

I was very happy when I saw the ambient light sensor working =D

On Wednesday, 7 November 2012 12:26:18 UTC, Sean Owen wrote:

My current patch.

The big issue I see right now is that when the light comes on, it becomes bright enough for the light to go off again. You can increase the threshold for 'off' but it may be a little problematic, since the device is also continually adjusting exposure to make dark images brighter.

Maybe it's not necessary to turn it off automatically?

Somehow I think the right answer here is to use the ambient light sensor (http://developer.android.com/reference/android/hardware/SensorEvent.html#values). It's easy to register a callback to detect current luminance is a more reliable way regularly.

I know not all devices have one, but most will.

On Wednesday, November 7, 2012 12:13:48 PM UTC, Sean Owen wrote:

[Sean Owen]

Hmm but if you use the light sensor then most of this change is not necessary. I actually made a similar change locally today and it works quite well and is quite simple. Let me prepare that change, which is pretty much the same as what you have, and graft in the rest like the new preference.

Like Daniel I am reluctant to add yet another setting. But I don't know that this behavior can be on by default as it will probably be surprising. I think the pref will have to become a three-way off/auto/on setting instead of a checkbox.

[Sean Owen]

Have a look at the change I just committed. It contains the ambient light sensor change, but not the preview frame change or the preference setting (yet). It's disabled but you can turn it on by uncommenting two lines in CaptureActivity. See how well it works for you, especially the thresholds. It works nicely for me.

Next I would try to make the current torch preference a three-way selection: on, off, auto. That way we don't need yet another setting.

[Nikolaus Huber]

I updated and tested it. Works really fine =)

I attached a patch to include the three-way option of on/off/auto in the preferencesActivity and changed all the necessary files to cope with it. =)

One thing I didn't manage to solve was how to put the options into the string.xml instead of the arrays.xml

[Sean Owen]

OK, I committed a variation on your changes -- same idea just fit in a little differently. It's pretty clean.

You can reference strings in the string-array declarations with @string/...  This commit included translations too.

Works pretty well for me. The default is still off. I think the automatic lighting is pretty neat.

[Nikolaus Huber]

Works really well =)

Thanks for helping me, I enjoyed it very much and learned a lot =)

Is there a resource for common words for software for all of these languages? Or did you just use dictionaries?

[Sean Owen]

There's an automated tool in the project called StringResourceTranslator (or something like that) that translates via Google Translate. The results are sometimes good, sometimes not, but usually better than leaving in English.

The API is no longer free, so you'd have to get an API key and plug it in to use the tool; I just pay the $1 / month that auto translation ends up costing.