Intent to implement and ship: ImageCapture

[Alfredo Yang]

Summary:
Allow web authors to take photo via gUM video track.

Bug:
Main tracking bug, https://bugzilla.mozilla.org/show_bug.cgi?id=888177

Spec:
https://dvcs.w3.org/hg/dap/raw-file/default/media-stream-capture/ImageCapture.html

Platform coverage:
All platforms.

Target release:
late 2014.

Pref:
dom.imagecapture.enabled

Background:
The spec is pretty much a draft. I focus on implementing subset needs to take advantage of high resolution camera hardware in platform like B2G [1].

Best regards,

Alfredo

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=1054905

[Ehsan Akhgari]

It seems like this API addresses at least some of the use cases of our
Camera API for Firefox OS (things such as being able to display a
preview video stream, being able to adjust some picture quality
parameters, etc.).

I am wondering if you know how much of the Camera API use cases this is
going to address, and whether it can be something that we can build
advanced camera apps on top of?

(CCing Mike, our Camera API guru.)

Thanks!
Ehsan

> _______________________________________________
> dev-platform mailing list
> dev-pl...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>

[Mike Habicher]

I've been steering the underlying ICameraControl implementation (dom/camera) towards better support for ImageCapture's usage model for a while.

If we can fill in support for the missing features we currently use (see CameraCapabilities.webidl) then I don't see why ImageCapture couldn't become the API used by the Camera app -- depending on what's involved in getting platform-supported video recording to work (or if it works already).

--m.

On 14-09-04 03:10 PM, Ehsan Akhgari wrote:

It seems like this API addresses at least some of the use cases of our Camera API for Firefox OS (things such as being able to display a preview video stream, being able to adjust some picture quality parameters, etc.).

I am wondering if you know how much of the Camera API use cases this is going to address, and whether it can be something that we can build advanced camera apps on top of?

(CCing Mike, our Camera API guru.)

Thanks!
Ehsan

On 2014-09-03, 5:15 AM, Alfredo Yang wrote:

[Jonas Sicking]

On Wed, Sep 3, 2014 at 2:15 AM, Alfredo Yang <ay...@mozilla.com> wrote:
> Summary:
> Allow web authors to take photo via gUM video track.
>
> Bug:
> Main tracking bug, https://bugzilla.mozilla.org/show_bug.cgi?id=888177
>
> Spec:
> https://dvcs.w3.org/hg/dap/raw-file/default/media-stream-capture/ImageCapture.html
>
> Platform coverage:
> All platforms.
>
> Target release:
> late 2014.
>
> Pref:
> dom.imagecapture.enabled
>
> Background:
> The spec is pretty much a draft. I focus on implementing subset needs to take advantage of high resolution camera hardware in platform like B2G [1].

What's the use case here? Note that in order to build even a
half-decent camera app just for taking selfies you need some amount of
control over of focus. For more advanced camera apps you also want
control over backlight compensation, flash etc.

This is an area where gUM and WebRTC so far has been very lacking
which means that it's not yet mature enough for those use cases. This
seems fine since we can't do everything at once. However we should be
intentional in what types of use cases we're trying to address when we
add features.

/ Jonas

[Robert O'Callahan]

On Fri, Sep 5, 2014 at 11:01 AM, Jonas Sicking <jo...@sicking.cc> wrote:

> What's the use case here? Note that in order to build even a
> half-decent camera app just for taking selfies you need some amount of
> control over of focus. For more advanced camera apps you also want
> control over backlight compensation, flash etc.
>

Yes. That's all in-scope for the ImageCapture spec. See
http://w3c.github.io/mediacapture-image/#photooptions. We don't implement
those options yet, but ultimately ImageCapture should be what our Camera
app uses.

Rob
--
oIo otoeololo oyooouo otohoaoto oaonoyooonoeo owohooo oioso oaonogoroyo
owoiotoho oao oboroootohoeoro oooro osoiosotoeoro owoiololo oboeo
osouobojoeocoto otooo ojouodogomoeonoto.o oAogoaoiono,o oaonoyooonoeo
owohooo
osoaoyoso otooo oao oboroootohoeoro oooro osoiosotoeoro,o o‘oRoaocoao,o’o
oioso
oaonosowoeoroaoboloeo otooo otohoeo ocooouoroto.o oAonodo oaonoyooonoeo
owohooo
osoaoyoso,o o‘oYooouo ofooooolo!o’o owoiololo oboeo oiono odoaonogoeoro
ooofo
otohoeo ofoioroeo ooofo ohoeololo.

[Jonas Sicking]

On Thu, Sep 4, 2014 at 4:22 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> On Fri, Sep 5, 2014 at 11:01 AM, Jonas Sicking <jo...@sicking.cc> wrote:
>>
>> What's the use case here? Note that in order to build even a
>> half-decent camera app just for taking selfies you need some amount of
>> control over of focus. For more advanced camera apps you also want
>> control over backlight compensation, flash etc.
>
>
> Yes. That's all in-scope for the ImageCapture spec. See
> http://w3c.github.io/mediacapture-image/#photooptions. We don't implement
> those options yet, but ultimately ImageCapture should be what our Camera app
> uses.

Ooh, that is great! Though surprisingly the API doesn't seem to have
any support for focus or flash control? Was that intentionally left
out?

Looking at the API though, why is it based on Events rather than
Promises? This seems to make the API much harder and more error prone
to use since you might have to worry about matching up the right
'photo' event with the right call to takePhoto(). The following seems
like an easier-to-use API.

[Constructor(VideoStreamTrack track)]
interface ImageCapture {
readonly attribute PhotoOptions photoOptions;
readonly attribute VideoStreamTrack videoStreamTrack;
readonly attribute MediaStream previewStream;

Promise<void> setOptions (PhotoSettings? photoSettings);
Promise<Blob> takePhoto ();
Promise<ImageData> grabFrame ();
};

/ Jonas

[Robert O'Callahan]

On Fri, Sep 5, 2014 at 11:39 AM, Jonas Sicking <jo...@sicking.cc> wrote:

> Ooh, that is great! Though surprisingly the API doesn't seem to have
> any support for focus or flash control? Was that intentionally left
> out?
>

I think it's just a case of not wanting the spec to get too far ahead of
implementations.

Looking at the API though, why is it based on Events rather than
> Promises?

http://w3c.github.io/mediacapture-image/#promise-extensions-to-imagecapture

[Jonas Sicking]

On Thu, Sep 4, 2014 at 4:45 PM, Robert O'Callahan <rob...@ocallahan.org> wrote:
> On Fri, Sep 5, 2014 at 11:39 AM, Jonas Sicking <jo...@sicking.cc> wrote:
>> Looking at the API though, why is it based on Events rather than
>> Promises?
>
> http://w3c.github.io/mediacapture-image/#promise-extensions-to-imagecapture

That seems super strange to me. We can't have web specs that contain
two alternative APIs. That does not make for a platform which works
consistently across implementations.

/ Jonas

[Robert O'Callahan]

Yeah, I'm not sure what they're thinking there. I'll raise it on the list.

[Henri Sivonen]

On Wed, Sep 3, 2014 at 12:15 PM, Alfredo Yang <ay...@mozilla.com> wrote:
> Summary:
> Allow web authors to take photo via gUM video track.

Does this have the same privacy protections as current gUM?

Is current gUM restricted to authenticated origins? If it isn't, is it
realistic to restrict it to authenticated origins?

I gather that gUM requires prompting even for packaged apps, which
seems good to me. Is that the case? However, the Camera app currently
has access to the camera without any prompting ever. Will the Camera
app start prompting or will it perhaps be pre-authorized somehow when
it migrates to gUM+this new API? If yes, could the pre-authorization
be reflected in App permissions? (Currently the Camera app clearly has
special powers that App permissions doesn't show, which makes me feel
I can't trust App permissions. See
https://bugzilla.mozilla.org/show_bug.cgi?id=1062246 )

--
Henri Sivonen
hsiv...@hsivonen.fi
https://hsivonen.fi/

[Robert O'Callahan]

On Fri, Sep 5, 2014 at 10:19 PM, Henri Sivonen <hsiv...@hsivonen.fi> wrote:

> Does this have the same privacy protections as current gUM?
>

Yes. You can only use this on a stream you've already acquired (e.g. via
current gUM, but other APIs also produce streams). You can already shunt a
MediaStream to a <video> element and then drawImage that to a canvas to get
stream pixels.

Is current gUM restricted to authenticated origins? If it isn't, is it
> realistic to restrict it to authenticated origins?
>

That's a good idea but it's a separate issue.