Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Proposal for a MediaControl API
108 views
Skip to first unread message
David Flanagan
Jun 14, 2013, 7:16:05 PM6/14/13
to dev-w...@lists.mozilla.org, Alive Kuo, Fabrice Desré, Gregor Wagner, Hema Koka, Vivien Nicolas, Tim Guan-tin Chien, Jonas Sicking
This is a proposal for an API for communication between apps that play
media and apps that want to control media playback. This is based on product
requirements for FirefoxOS v1.2 and on preliminary discussion on the
dev-gaia mailing list. (The basic idea is Vivien's though I have no idea
whether
this is what he had in mind.) I've cc'ed a bunch of people who have
participated in
those preliminary discussions or I suspect might be interested.
FirefoxOS use-cases:
1) The Gaia Music app can play music while in the background.
We need a way to allow the user to control playback from the
lockscreen and/or other parts of the system. This is a product
requirement for version 1.2, so we need to move quickly here.
2) It would be cool (but not currently required) if we could control
the FM radio app this way, too.
3) We'd like to be able to control music playback from bluetooth
devices like headsets and car stereo systems. (For example: if the
phone is sending music output to a car stereo via bluetooth, then
the user should be able to control playback from the buttons on the
car's steering wheel.)
4) An API like this would give us a nice way to pause media playback
when an incoming call is received. (We do that now, somehow, but I
assume that the implementation is something of a hack.)
5) Perhaps an API like this would give us a better way to control
media playback volume independently from phone ringer volume.
Desktop use-cases:
1) Would this API allow gecko to make intelligent use of the media
control keys on a user's keyboard?
2) Would this be a way to give the user the ability to stop media
playback in whatever tab has started playing something (that's a
common complaint, isn't it?)
Roles:
Media players and media controllers will both want to use this API but
the two distinct roles will use distinct parts of the API.
Permissions:
Media controller apps will require a "media-control" permission to
access the API. I don't think that media player apps should require
any special permissions to allow themselves to be controlled.
Basic API requirements:
At a minimum, this API should give media controller apps the ability
to pause and resume media playback and to set the playback time (seek)
even when the media player app is not cooperating and does not use the
player API. (This allows us to paus playback when a phone call comes
in on the phone or to easily silence advertisements on the desktop.)
This means that the controller API should have pause() and play()
methods that directly control whatever <audio> or <video> element is
currently playing.
(I'm making the assumption here that gecko already has some notion of
"the currently playing media element". Maybe the
navigator.mozAudioChannelManager API is relevant here, but I don't
actually know anything about that API.)
In addition to basic play and pause functionality, however, player
apps and controller apps should be able to cooperate to provide a
great media control UX:
- player apps must be able to specify a set of controls that they
offer (next track, previous track, fast forward, rewind, shuffle,
etc.)
- controller apps must be able to query the set of controls, so they
can display corresponding UX. (We don't want to display a next track
button, for example, for apps that don't have a next track.)
- player apps must be able to specify metadata (song title, artist,
album, e.g.) about the media currently being played.
- controller apps must be able to query the metadata so that they can
display it to the user.
- player apps need to receive notification when controller apps send a
control command. (Though as noted above, play and pause commands
work directly on the media element even if the player app isn't
listening for them.)
- controller apps need a way to determine the duration of the current
media and the current playback time.
- controller apps need to receive notifications when:
a) the player changes the current metadata
b) the player changes the set of controls it can respond to
c) the player stops or resumes playing or otherwise changes state
(seeking, buffering, etc.)
d) the currently controlled media element is destroyed (because the
app that contains it exits, for example)
e) a new media element becomes the currently controlled media element.
- maybe a player app needs to receive a notification when it is no
longer the current player (because another player app has taken
control). Maybe a player app needs an API to grab or relinquish the
API.
- maybe the controller app needs a way to make the player app become
visible? On the phone, this would switch to the media app. On
desktop it might reveal the tab that was playing music.
API sketch:
Media controller apps could use a MediaController object available as
navigator.mediaControl:
interface MediaController {
void play();
void pause();
readonly attribute double duration; // mirrors controlled media
element
attribute double currentTime; // mirrors controlled media
element
attribute String[] supportedCommands; // ['next', 'previous',
'shuffle'...]
showPlayer(); // reveal the playing app or tab?
attribute any metadata; // data about currently
playing track
sendCommand(cmd); // one of the commands above
attribute Function onstatechanged; // play, pause, seeking, etc.
attribute Function onplayerchanged; // what we're controlling has
changed
attribute Function onmetadatachanged; // Fired when new track info
availble
};
An alternative might be to give a media controller app access to an
object that completely mirrores the currently playing
HTMLMediaElement. This would allow controllers to listen for any media
event, query the buffered region of the media, and so forth and would
enable very full-featured controls.
The media player API could perhaps be defined as an extension to
HTMLMediaElement:
interface HTMLMediaElement {
...
attribute MediaControllable controls;
};
interface MediaControllable {
attribute String[] commands; // How we're willing to be controlled.
attribute any metadata; // Track name, etc., for display by
controller
attribute Function oncommand; // Fired when the controller sends a
command
attribute Function onstatechange; // Fired when we become the currently
// controlled element or lose that
status?
};
media and apps that want to control media playback. This is based on product
requirements for FirefoxOS v1.2 and on preliminary discussion on the
dev-gaia mailing list. (The basic idea is Vivien's though I have no idea
whether
this is what he had in mind.) I've cc'ed a bunch of people who have
participated in
those preliminary discussions or I suspect might be interested.
FirefoxOS use-cases:
1) The Gaia Music app can play music while in the background.
We need a way to allow the user to control playback from the
lockscreen and/or other parts of the system. This is a product
requirement for version 1.2, so we need to move quickly here.
2) It would be cool (but not currently required) if we could control
the FM radio app this way, too.
3) We'd like to be able to control music playback from bluetooth
devices like headsets and car stereo systems. (For example: if the
phone is sending music output to a car stereo via bluetooth, then
the user should be able to control playback from the buttons on the
car's steering wheel.)
4) An API like this would give us a nice way to pause media playback
when an incoming call is received. (We do that now, somehow, but I
assume that the implementation is something of a hack.)
5) Perhaps an API like this would give us a better way to control
media playback volume independently from phone ringer volume.
Desktop use-cases:
1) Would this API allow gecko to make intelligent use of the media
control keys on a user's keyboard?
2) Would this be a way to give the user the ability to stop media
playback in whatever tab has started playing something (that's a
common complaint, isn't it?)
Roles:
Media players and media controllers will both want to use this API but
the two distinct roles will use distinct parts of the API.
Permissions:
Media controller apps will require a "media-control" permission to
access the API. I don't think that media player apps should require
any special permissions to allow themselves to be controlled.
Basic API requirements:
At a minimum, this API should give media controller apps the ability
to pause and resume media playback and to set the playback time (seek)
even when the media player app is not cooperating and does not use the
player API. (This allows us to paus playback when a phone call comes
in on the phone or to easily silence advertisements on the desktop.)
This means that the controller API should have pause() and play()
methods that directly control whatever <audio> or <video> element is
currently playing.
(I'm making the assumption here that gecko already has some notion of
"the currently playing media element". Maybe the
navigator.mozAudioChannelManager API is relevant here, but I don't
actually know anything about that API.)
In addition to basic play and pause functionality, however, player
apps and controller apps should be able to cooperate to provide a
great media control UX:
- player apps must be able to specify a set of controls that they
offer (next track, previous track, fast forward, rewind, shuffle,
etc.)
- controller apps must be able to query the set of controls, so they
can display corresponding UX. (We don't want to display a next track
button, for example, for apps that don't have a next track.)
- player apps must be able to specify metadata (song title, artist,
album, e.g.) about the media currently being played.
- controller apps must be able to query the metadata so that they can
display it to the user.
- player apps need to receive notification when controller apps send a
control command. (Though as noted above, play and pause commands
work directly on the media element even if the player app isn't
listening for them.)
- controller apps need a way to determine the duration of the current
media and the current playback time.
- controller apps need to receive notifications when:
a) the player changes the current metadata
b) the player changes the set of controls it can respond to
c) the player stops or resumes playing or otherwise changes state
(seeking, buffering, etc.)
d) the currently controlled media element is destroyed (because the
app that contains it exits, for example)
e) a new media element becomes the currently controlled media element.
- maybe a player app needs to receive a notification when it is no
longer the current player (because another player app has taken
control). Maybe a player app needs an API to grab or relinquish the
API.
- maybe the controller app needs a way to make the player app become
visible? On the phone, this would switch to the media app. On
desktop it might reveal the tab that was playing music.
API sketch:
Media controller apps could use a MediaController object available as
navigator.mediaControl:
interface MediaController {
void play();
void pause();
readonly attribute double duration; // mirrors controlled media
element
attribute double currentTime; // mirrors controlled media
element
attribute String[] supportedCommands; // ['next', 'previous',
'shuffle'...]
showPlayer(); // reveal the playing app or tab?
attribute any metadata; // data about currently
playing track
sendCommand(cmd); // one of the commands above
attribute Function onstatechanged; // play, pause, seeking, etc.
attribute Function onplayerchanged; // what we're controlling has
changed
attribute Function onmetadatachanged; // Fired when new track info
availble
};
An alternative might be to give a media controller app access to an
object that completely mirrores the currently playing
HTMLMediaElement. This would allow controllers to listen for any media
event, query the buffered region of the media, and so forth and would
enable very full-featured controls.
The media player API could perhaps be defined as an extension to
HTMLMediaElement:
interface HTMLMediaElement {
...
attribute MediaControllable controls;
};
interface MediaControllable {
attribute String[] commands; // How we're willing to be controlled.
attribute any metadata; // Track name, etc., for display by
controller
attribute Function oncommand; // Fired when the controller sends a
command
attribute Function onstatechange; // Fired when we become the currently
// controlled element or lose that
status?
};
Alive Kuo
Jun 15, 2013, 4:47:18 AM6/15/13
to David Flanagan, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Hema Koka, Vivien Nicolas, Tim Guan-tin Chien, Jonas Sicking
David Flanagan <dfla...@mozilla.com> 於 2013/6/15 上午7:16 寫道：
> This is a proposal for an API for communication between apps that play
> media and apps that want to control media playback. This is based on product
> requirements for FirefoxOS v1.2 and on preliminary discussion on the
> dev-gaia mailing list. (The basic idea is Vivien's though I have no idea whether
> this is what he had in mind.) I've cc'ed a bunch of people who have participated in
> those preliminary discussions or I suspect might be interested.
>
> FirefoxOS use-cases:
>
> 1) The Gaia Music app can play music while in the background.
> We need a way to allow the user to control playback from the
> lockscreen and/or other parts of the system. This is a product
> requirement for version 1.2, so we need to move quickly here.
>
Through this proposal I still concern what's the behavior and life cycle of media controller.
> This is a proposal for an API for communication between apps that play
> media and apps that want to control media playback. This is based on product
> requirements for FirefoxOS v1.2 and on preliminary discussion on the
> dev-gaia mailing list. (The basic idea is Vivien's though I have no idea whether
> this is what he had in mind.) I've cc'ed a bunch of people who have participated in
> those preliminary discussions or I suspect might be interested.
>
> FirefoxOS use-cases:
>
> 1) The Gaia Music app can play music while in the background.
> We need a way to allow the user to control playback from the
> lockscreen and/or other parts of the system. This is a product
> requirement for version 1.2, so we need to move quickly here.
>
In our use case the media controller is lock-screen, but indeed it's part of system app.
So media controller is system app, media player is music app.
Is this correct?
Since system app never dies(only when b2g crashes) and music player has chances to die,
we may want to
1) Show the media control when music app starts playing.(Not just after app starts)
2) Hide the media control when music app is closed. No matter it's crashed or closed.
> Basic API requirements:
>
> At a minimum, this API should give media controller apps the ability
> to pause and resume media playback and to set the playback time (seek)
> even when the media player app is not cooperating and does not use the
> player API. (This allows us to paus playback when a phone call comes
> in on the phone or to easily silence advertisements on the desktop.)
> This means that the controller API should have pause() and play()
> methods that directly control whatever <audio> or <video> element is
> currently playing.
>
> (I'm making the assumption here that gecko already has some notion of
> "the currently playing media element". Maybe the
> navigator.mozAudioChannelManager API is relevant here, but I don't
> actually know anything about that API.)
>
> - maybe a player app needs to receive a notification when it is no
> longer the current player (because another player app has taken
> control). Maybe a player app needs an API to grab or relinquish the
> API.
This one is interesting. I don't see where to determine the "active" media player in this proposal. Am I missing something?
> - maybe a player app needs to receive a notification when it is no
> longer the current player (because another player app has taken
> control). Maybe a player app needs an API to grab or relinquish the
> API.
More accurately, how to determine who is "media player"?
Current audio competing policy is content channel is higher than normal channel,
but any visible(page visibility = true) app is allowed to play.
In the case when media player is paused/stopped by audio competing policy,
I think it's feasible to hide the media control.
And I propose that:
If the one who interrupts the previous channel is also using 'content channel',
than media control would switch the control to the new one.
Otherwise, hide or make the control inactive because we don't know if the new app is a media player.
So the rule to define one is media player or not could be content-channel permission requirement.
--
Alive C. Kuo, Front-end Eng., Mozilla Corp. (Taiwan, Taipei)
Chris Double
Jun 15, 2013, 10:42:20 AM6/15/13
to dev-w...@lists.mozilla.org
On Sat, Jun 15, 2013 at 11:16 AM, David Flanagan <dfla...@mozilla.com> wrote:
> API sketch:
>
> Media controller apps could use a MediaController object available as
> navigator.mediaControl:
You might want to pick a different name since there is already a
> API sketch:
>
> Media controller apps could use a MediaController object available as
> navigator.mediaControl:
MediaController in the HTML API:
<http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#mediacontroller>
--
http://www.bluishcoder.co.nz
David Flanagan
Jun 17, 2013, 1:51:43 PM6/17/13
to dev-w...@lists.mozilla.org
MediaController object and make it into a MediaRemoteController object
or something.
On the other hand, my proposal is attempting to add another layer of
control (like next song, previous song) on top of the base
HTMLMediaElement.... Maybe I should just focus on that and treat it as
an app-to-app (or tab-to-tab) communication protocol rather than also
trying to tie it to individual audio and video tags.
David
David Flanagan
Jun 17, 2013, 2:22:37 PM6/17/13
to Alive Kuo, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Hema Koka, Vivien Nicolas, Tim Guan-tin Chien, Jonas Sicking
On 6/15/13 1:47 AM, Alive Kuo wrote:
> David Flanagan <dfla...@mozilla.com <mailto:dfla...@mozilla.com>>
> Since system app never dies(only when b2g crashes) and music player
> has chances to die,
> we may want to
> 1) Show the media control when music app starts playing.(Not just
> after app starts)
> 2) Hide the media control when music app is closed. No matter it's
> crashed or closed.
>
That is the intent of the onplayerchanged event handler on my proposed
MediaController object. I may not have it quite right, but yes, the
system app would need notifications like those.
>> Basic API requirements:
>>
>> At a minimum, this API should give media controller apps the ability
>> to pause and resume media playback and to set the playback time (seek)
>> even when the media player app is not cooperating and does not use the
>> player API. (This allows us to paus playback when a phone call comes
>> in on the phone or to easily silence advertisements on the desktop.)
>> This means that the controller API should have pause() and play()
>> methods that directly control whatever <audio> or <video> element is
>> currently playing.
>
> pause/play is for sure necessary, but you don't need to do anything
> when the phone call comes.
>
I know we have a system in place that works for incoming phone calls. I
just wonder if we can make it better with an API like this. For
example, does the music app know to pause the current song when a phone
call arrives and then resume it, or does it just keep playing but get
muted? What API is currently used to communicate an incoming phone call
to the Music app? (I should have researched this before making this
proposal, I know!)
>>
>> (I'm making the assumption here that gecko already has some notion of
>> "the currently playing media element". Maybe the
>> navigator.mozAudioChannelManager API is relevant here, but I don't
>> actually know anything about that API.)
>
> gecko would automatically stops the playing content when higher
> priority channel occurs.
>>
>> - maybe a player app needs to receive a notification when it is no
>> longer the current player (because another player app has taken
>> control). Maybe a player app needs an API to grab or relinquish the
>> API.
>
> This one is interesting. I don't see where to determine the "active"
> media player in this proposal. Am I missing something?
> More accurately, how to determine who is "media player"?
>
I was actually you could help me with that, Alive :-). This is, I
think, the weakest part of my proposal. I honestly don't know whether
gecko would allow us to know what the currently active HTMLMediaElement
is, or whether we need an explict API for apps to say "I want to be the
currently active controllable music player".
This ties in to my response to Chris Double... A key part of my
proposal is that it allows application-level controls like "shuffle" and
"next track" in addition to player level controls like pause and play.
Perhaps I'm trying to do too much to allow application level controls
and tie the API to an individual audio or video tag. Maybe it just
needs to be an app-to-app communication protocol instead of integrated
at the HTMLMediaElement level.
> Current audio competing policy is content channel is higher than
> normal channel,
> but any visible(page visibility = true) app is allowed to play.
> In the case when media player is paused/stopped by audio competing policy,
> I think it's feasible to hide the media control.
>
> And I propose that:
> If the one who interrupts the previous channel is also using 'content
> channel',
> than media control would switch the control to the new one.
> Otherwise, hide or make the control inactive because we don't know if
> the new app is a media player.
>
> So the rule to define one is media player or not could be
> content-channel permission requirement.
>
Right now, it looks like Music, Video and Gallery (which plays videos)
have audio-channel-content permission. (Camera does not have it even
though it plays videos, too.) But only Music will keep playing even
when it goes into the background. What mechanism controls that? It seems
like that might be the distinction we want. If audio stops playing when
we switch apps or go to the homescreen or lock the phone, then there is
no need for controls on the lockscreen for those apps.
David
> David Flanagan <dfla...@mozilla.com <mailto:dfla...@mozilla.com>>
> 於 2013/6/15 上午7:16 寫道：
>
>> This is a proposal for an API for communication between apps that play
>> media and apps that want to control media playback. This is based on
>> product
>> requirements for FirefoxOS v1.2 and on preliminary discussion on the
>> dev-gaia mailing list. (The basic idea is Vivien's though I have no
>> idea whether
>> this is what he had in mind.) I've cc'ed a bunch of people who have
>> participated in
>> those preliminary discussions or I suspect might be interested.
>>
>> FirefoxOS use-cases:
>>
>> 1) The Gaia Music app can play music while in the background.
>> We need a way to allow the user to control playback from the
>> lockscreen and/or other parts of the system. This is a product
>> requirement for version 1.2, so we need to move quickly here.
>>
>
> Through this proposal I still concern what's the behavior and life
> cycle of media controller.
> In our use case the media controller is lock-screen, but indeed it's
> part of system app.
> So media controller is system app, media player is music app.
>
> Is this correct?
Yes.
>
>> This is a proposal for an API for communication between apps that play
>> media and apps that want to control media playback. This is based on
>> product
>> requirements for FirefoxOS v1.2 and on preliminary discussion on the
>> dev-gaia mailing list. (The basic idea is Vivien's though I have no
>> idea whether
>> this is what he had in mind.) I've cc'ed a bunch of people who have
>> participated in
>> those preliminary discussions or I suspect might be interested.
>>
>> FirefoxOS use-cases:
>>
>> 1) The Gaia Music app can play music while in the background.
>> We need a way to allow the user to control playback from the
>> lockscreen and/or other parts of the system. This is a product
>> requirement for version 1.2, so we need to move quickly here.
>>
>
> Through this proposal I still concern what's the behavior and life
> cycle of media controller.
> In our use case the media controller is lock-screen, but indeed it's
> part of system app.
> So media controller is system app, media player is music app.
>
> Is this correct?
> Since system app never dies(only when b2g crashes) and music player
> has chances to die,
> we may want to
> 1) Show the media control when music app starts playing.(Not just
> after app starts)
> 2) Hide the media control when music app is closed. No matter it's
> crashed or closed.
>
MediaController object. I may not have it quite right, but yes, the
system app would need notifications like those.
>> Basic API requirements:
>>
>> At a minimum, this API should give media controller apps the ability
>> to pause and resume media playback and to set the playback time (seek)
>> even when the media player app is not cooperating and does not use the
>> player API. (This allows us to paus playback when a phone call comes
>> in on the phone or to easily silence advertisements on the desktop.)
>> This means that the controller API should have pause() and play()
>> methods that directly control whatever <audio> or <video> element is
>> currently playing.
>
> pause/play is for sure necessary, but you don't need to do anything
> when the phone call comes.
>
just wonder if we can make it better with an API like this. For
example, does the music app know to pause the current song when a phone
call arrives and then resume it, or does it just keep playing but get
muted? What API is currently used to communicate an incoming phone call
to the Music app? (I should have researched this before making this
proposal, I know!)
>>
>> (I'm making the assumption here that gecko already has some notion of
>> "the currently playing media element". Maybe the
>> navigator.mozAudioChannelManager API is relevant here, but I don't
>> actually know anything about that API.)
>
> gecko would automatically stops the playing content when higher
> priority channel occurs.
>>
>> - maybe a player app needs to receive a notification when it is no
>> longer the current player (because another player app has taken
>> control). Maybe a player app needs an API to grab or relinquish the
>> API.
>
> This one is interesting. I don't see where to determine the "active"
> media player in this proposal. Am I missing something?
> More accurately, how to determine who is "media player"?
>
think, the weakest part of my proposal. I honestly don't know whether
gecko would allow us to know what the currently active HTMLMediaElement
is, or whether we need an explict API for apps to say "I want to be the
currently active controllable music player".
This ties in to my response to Chris Double... A key part of my
proposal is that it allows application-level controls like "shuffle" and
"next track" in addition to player level controls like pause and play.
Perhaps I'm trying to do too much to allow application level controls
and tie the API to an individual audio or video tag. Maybe it just
needs to be an app-to-app communication protocol instead of integrated
at the HTMLMediaElement level.
> Current audio competing policy is content channel is higher than
> normal channel,
> but any visible(page visibility = true) app is allowed to play.
> In the case when media player is paused/stopped by audio competing policy,
> I think it's feasible to hide the media control.
>
> And I propose that:
> If the one who interrupts the previous channel is also using 'content
> channel',
> than media control would switch the control to the new one.
> Otherwise, hide or make the control inactive because we don't know if
> the new app is a media player.
>
> So the rule to define one is media player or not could be
> content-channel permission requirement.
>
have audio-channel-content permission. (Camera does not have it even
though it plays videos, too.) But only Music will keep playing even
when it goes into the background. What mechanism controls that? It seems
like that might be the distinction we want. If audio stops playing when
we switch apps or go to the homescreen or lock the phone, then there is
no need for controls on the lockscreen for those apps.
David
Marco Chen
Jun 18, 2013, 1:59:47 AM6/18/13
to David Flanagan, Alive Kuo, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Jonas Sicking, Vivien Nicolas, Tim Guan-tin Chien, Hema Koka
dev-webapi mailing list
dev-w...@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-webapi
Alive Kuo
Jun 18, 2013, 2:34:03 AM6/18/13
to David Flanagan, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Hema Koka, Vivien Nicolas, Tim Guan-tin Chien, Jonas Sicking
>> David Flanagan <dfla...@mozilla.com> 於 2013/6/15 上午7:16 寫道：
>>
> That is the intent of the onplayerchanged event handler on my proposed MediaController object. I may not have it quite right, but yes, the system app would need notifications like those.
>>
>>
> That is the intent of the onplayerchanged event handler on my proposed MediaController object. I may not have it quite right, but yes, the system app would need notifications like those.
>>
> I know we have a system in place that works for incoming phone calls. I just wonder if we can make it better with an API like this. For example, does the music app know to pause the current song when a phone call arrives and then resume it, or does it just keep playing but get muted? What API is currently used to communicate an incoming phone call to the Music app? (I should have researched this before making this proposal, I know!)
If I am correct we have onmozinterruptbegin and onmozinterruptend event callback on audio element
when it is interrupted by higher priority audio channel.
See https://bugzilla.mozilla.org/show_bug.cgi?id=805333 (It's the beginning of everything including audio competing.)
and wiki https://wiki.mozilla.org/WebAPI/AudioChannels
>>
>> This one is interesting. I don't see where to determine the "active" media player in this proposal. Am I missing something?
>> More accurately, how to determine who is "media player"?
>>
> I was actually you could help me with that, Alive :-). This is, I think, the weakest part of my proposal. I honestly don't know whether gecko would allow us to know what the currently active HTMLMediaElement is, or whether we need an explict API for apps to say "I want to be the currently active controllable music player".
>
> This ties in to my response to Chris Double... A key part of my proposal is that it allows application-level controls like "shuffle" and "next track" in addition to player level controls like pause and play. Perhaps I'm trying to do too much to allow application level controls and tie the API to an individual audio or video tag. Maybe it just needs to be an app-to-app communication protocol instead of integrated at the HTMLMediaElement level.
Uh…not an easy one.
>> This one is interesting. I don't see where to determine the "active" media player in this proposal. Am I missing something?
>> More accurately, how to determine who is "media player"?
>>
> I was actually you could help me with that, Alive :-). This is, I think, the weakest part of my proposal. I honestly don't know whether gecko would allow us to know what the currently active HTMLMediaElement is, or whether we need an explict API for apps to say "I want to be the currently active controllable music player".
>
> This ties in to my response to Chris Double... A key part of my proposal is that it allows application-level controls like "shuffle" and "next track" in addition to player level controls like pause and play. Perhaps I'm trying to do too much to allow application level controls and tie the API to an individual audio or video tag. Maybe it just needs to be an app-to-app communication protocol instead of integrated at the HTMLMediaElement level.
It's not difficult to know who is playing audio now(occupying the active audio channel) -- but still needs gecko expose it --
but to define who is a music player needs new API if we don't want hardcode something in gaia and gecko.
The only thing I could figure out now is this:
1) Music registers as a remote control receiver. Maybe in manifest.
2) System checks the registration of media remote control, put the one who registers it on the lockscreen remote controller element like:
<controller remote="app://music.gaia-mobile.org"></controller>
3) Change the remote url/origin when active content playing document is changing to another app/page.
And as you said I wonder this is too complex to implement…if you really want application level control.
Maybe we need UX confirmation? Do we really want to control "shuffle" on lockscreen?
>
>> Current audio competing policy is content channel is higher than normal channel,
>> but any visible(page visibility = true) app is allowed to play.
>> In the case when media player is paused/stopped by audio competing policy,
>> I think it's feasible to hide the media control.
>>
>> And I propose that:
>> If the one who interrupts the previous channel is also using 'content channel',
>> than media control would switch the control to the new one.
>> Otherwise, hide or make the control inactive because we don't know if the new app is a media player.
>>
>> So the rule to define one is media player or not could be content-channel permission requirement.
>>
> Right now, it looks like Music, Video and Gallery (which plays videos) have audio-channel-content permission. (Camera does not have it even though it plays videos, too.) But only Music will keep playing even when it goes into the background. What mechanism controls that? It seems like that might be the distinction we want. If audio stops playing when we switch apps or go to the homescreen or lock the phone, then there is no need for controls on the lockscreen for those apps.
The app who would be playing at background is just the one who has content channel permission.
However, gallery and video now would and should stop playing when they observes 'mozvisibilitychange' event.
Both of them are using content channel to play video because another usage of it is to interrupt other playing content.
So, if we have the API to know who is playing audio and we(system app) for sure already now who is foreground,
it's still reasonable to use content channel as parameter here. Just check the one:
1) Going to background + 2) Still playing content audio channel.
P.S. The API is implemented by Andrea(:baku), see https://bugzilla.mozilla.org/show_bug.cgi?id=853101
Marco told me it's already in the 1.2 schedule though it's not a trivial change.
Henri Sivonen
Jun 18, 2013, 2:51:56 AM6/18/13
to David Flanagan, Gregor Wagner, dev-w...@lists.mozilla.org, Fabrice Desré, Alive Kuo, Jonas Sicking, Vivien Nicolas, Tim Guan-tin Chien, Hema Koka
On Sat, Jun 15, 2013 at 2:16 AM, David Flanagan <dfla...@mozilla.com> wrote:
> user to control playback from the
> lockscreen and/or other parts of the system.
Cool. It would be good to ensure this can mapped to the corresponding
> user to control playback from the
> lockscreen and/or other parts of the system.
system facility on Android.
> Desktop use-cases:
Ubuntu Unity has system-wide controls for previous track, play/pause
and next track and display of track name, album name, artist and cover
art in the system audio menu along with the traditional volume
control. Canonical even has a Firefox extension that exposes a
vendor-specific API to the Web so that Web apps that have been pinned
to the Unity launcher can integrate with the menu.
http://developer.ubuntu.com/api/ubuntu-12.04/javascript/index.html#mediamenu
In the interest of being able to standardize something that'd obsolete
Canonical's vendor-specific API, I think it would be good to ensure
that the Unity functionality is covered.
--
Henri Sivonen
hsiv...@hsivonen.fi
http://hsivonen.iki.fi/
Andrea Marchesini
Jun 18, 2013, 3:59:53 AM6/18/13
to David Flanagan, Gregor Wagner, dev-w...@lists.mozilla.org, Fabrice Desré, Alive Kuo, Jonas Sicking, Vivien Nicolas, Tim Guan-tin Chien, Hema Koka
> 5) Perhaps an API like this would give us a better way to control
> media playback volume independently from phone ringer volume.
I see many things in common between this API and the new AudioChannel API.
> media playback volume independently from phone ringer volume.
https://bugzilla.mozilla.org/show_bug.cgi?id=853101
https://wiki.mozilla.org/WebAPI/AudioChannels
This AudioChannelAPI allows/will allow:
. to control a volume per window/app.
. it's possible to mute a window/app.
. it's possible to retrieve the list of active channels per window/app.
The idea is to expose these features to the browser API.
Would be really great to merge this API somehow with what you are proposing.
Regards,
Andrea
Mounir Lamouri
Jun 18, 2013, 9:14:24 AM6/18/13
to dev-w...@lists.mozilla.org, Gregor Wagner, Hema Koka, Alive Kuo, Jonas Sicking, Vivien Nicolas, David Flanagan, Tim Guan-tin Chien, Fabrice Desré
Hi,
I understand the intent behind this API but I am not sure we should use
this instead of an API allowing communication between applications. An
inter app communication API would be a powerful tool that would allow
developers to do better than what we might end up doing with a
MediaController API. Could you give a rationale why this would be better
than a generic communication API?
A few comments miscellaneous comments.
On 15/06/13 01:16, David Flanagan wrote:
> 3) We'd like to be able to control music playback from bluetooth
> devices like headsets and car stereo systems. (For example: if the
> phone is sending music output to a car stereo via bluetooth, then
> the user should be able to control playback from the buttons on the
> car's steering wheel.)
I believe that D3E should already take care of that.
> 4) An API like this would give us a nice way to pause media playback
> when an incoming call is received. (We do that now, somehow, but I
> assume that the implementation is something of a hack.)
I believe we should already support that with the AudioManager.
> 5) Perhaps an API like this would give us a better way to control
> media playback volume independently from phone ringer volume.
I believe that the AudioManager solves that too.
> Desktop use-cases:
>
> 1) Would this API allow gecko to make intelligent use of the media
> control keys on a user's keyboard?
Isn't this taken care of by D3E?
> 2) Would this be a way to give the user the ability to stop media
> playback in whatever tab has started playing something (that's a
> common complaint, isn't it?)
It is not clear to me why this use case requires a Web facing API? Can't
a UI in Firefox solves this?
--
Mounir
I understand the intent behind this API but I am not sure we should use
this instead of an API allowing communication between applications. An
inter app communication API would be a powerful tool that would allow
developers to do better than what we might end up doing with a
MediaController API. Could you give a rationale why this would be better
than a generic communication API?
A few comments miscellaneous comments.
On 15/06/13 01:16, David Flanagan wrote:
> 3) We'd like to be able to control music playback from bluetooth
> devices like headsets and car stereo systems. (For example: if the
> phone is sending music output to a car stereo via bluetooth, then
> the user should be able to control playback from the buttons on the
> car's steering wheel.)
> 4) An API like this would give us a nice way to pause media playback
> when an incoming call is received. (We do that now, somehow, but I
> assume that the implementation is something of a hack.)
> 5) Perhaps an API like this would give us a better way to control
> media playback volume independently from phone ringer volume.
> Desktop use-cases:
>
> 1) Would this API allow gecko to make intelligent use of the media
> control keys on a user's keyboard?
> 2) Would this be a way to give the user the ability to stop media
> playback in whatever tab has started playing something (that's a
> common complaint, isn't it?)
a UI in Firefox solves this?
--
Mounir
Sotaro Ikeda
Jun 21, 2013, 4:34:26 PM6/21/13
to Mounir Lamouri, Alive Kuo, Fabrice Desré, dev-w...@lists.mozilla.org, Hema Koka, Gregor Wagner, Vivien Nicolas, David Flanagan, Tim Guan-tin Chien, Jonas Sicking
Hi,
I like "inter app communication API" for it. This way of implementation is similar to android's "MediaPlaybackService" in music app.
sotaro
----- Original Message -----
From: "Mounir Lamouri" <mou...@lamouri.fr>
To: dev-w...@lists.mozilla.org
Cc: "Gregor Wagner" <gwa...@mozilla.com>, "Hema Koka" <hk...@mozilla.com>, "Alive Kuo" <al...@mozilla.com>, "Jonas Sicking" <jo...@sicking.cc>, "Vivien Nicolas" <vnic...@mozilla.com>, "David Flanagan" <dfla...@mozilla.com>, "Tim Guan-tin Chien" <tch...@mozilla.com>, "Fabrice Desré" <fab...@mozilla.com>
Sent: Tuesday, June 18, 2013 9:14:24 AM
Subject: Re: Proposal for a MediaControl API
I like "inter app communication API" for it. This way of implementation is similar to android's "MediaPlaybackService" in music app.
sotaro
----- Original Message -----
From: "Mounir Lamouri" <mou...@lamouri.fr>
To: dev-w...@lists.mozilla.org
Cc: "Gregor Wagner" <gwa...@mozilla.com>, "Hema Koka" <hk...@mozilla.com>, "Alive Kuo" <al...@mozilla.com>, "Jonas Sicking" <jo...@sicking.cc>, "Vivien Nicolas" <vnic...@mozilla.com>, "David Flanagan" <dfla...@mozilla.com>, "Tim Guan-tin Chien" <tch...@mozilla.com>, "Fabrice Desré" <fab...@mozilla.com>
Sent: Tuesday, June 18, 2013 9:14:24 AM
Subject: Re: Proposal for a MediaControl API
Alive Kuo
Jun 23, 2013, 3:39:18 PM6/23/13
to David Flanagan, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Jonas Sicking, Vivien Nicolas, Andrea Marchesini, Tim Guan-tin Chien, Hema Koka
David,
With Andrea's effort on bug 853101, system app would gain the power to control play/pause of certain app.
However, as previous discussion, we still lack two important things:
1) How to identify the media player?
2) How to communicate?
(A) From music to system (Update the playing song tittle….)
(B) From system to music (Part of this would be achieve with new audio channel API)
I don't know the detail of so called 'inter-app communication API'…is it ongoing?
in case we in the long don't have this magical stuff,
I still have some ideas about these two questions, in a mid-term way.
The proposal is still my first proposal, using window.open(...) in music app,
"but" system app won't bring the opened frame to foreground. (Because people hate to have too many frame in foreground:)
So it is to say, let music app open a "background media player frame" to let system app knows it would like to be controlled.
System app would then check the (1) content channel permission and (2) is it playing?(Using audio channel API).
The opened frame could also be used for communication to system,
and system would use mozBrowser API to get event when it changes something(like title-change, uh, weird I know.)
This helps system app to identify who is going to be controlled, and, if multiple apps are going to be controlled,
the key to find whom is relied on the app switch mechanism in system app(window manager) to find who is active app and playing content and having an opened 'media-player-background-frame' now.
Also if we are lucky to have the inter-app communication API, the opened frame could be ignored here.
Just use the brand new communication API!
But it's still necessary to make music app to let system knows he is going to play content,
and I think this also solves the cases some app are using content channel but won't be controlled via lock-screen controller.
> https://bugzilla.mozilla.org/show_bug.cgi?id=853101
With Andrea's effort on bug 853101, system app would gain the power to control play/pause of certain app.
However, as previous discussion, we still lack two important things:
1) How to identify the media player?
2) How to communicate?
(A) From music to system (Update the playing song tittle….)
(B) From system to music (Part of this would be achieve with new audio channel API)
I don't know the detail of so called 'inter-app communication API'…is it ongoing?
in case we in the long don't have this magical stuff,
I still have some ideas about these two questions, in a mid-term way.
The proposal is still my first proposal, using window.open(...) in music app,
"but" system app won't bring the opened frame to foreground. (Because people hate to have too many frame in foreground:)
So it is to say, let music app open a "background media player frame" to let system app knows it would like to be controlled.
System app would then check the (1) content channel permission and (2) is it playing?(Using audio channel API).
The opened frame could also be used for communication to system,
and system would use mozBrowser API to get event when it changes something(like title-change, uh, weird I know.)
This helps system app to identify who is going to be controlled, and, if multiple apps are going to be controlled,
the key to find whom is relied on the app switch mechanism in system app(window manager) to find who is active app and playing content and having an opened 'media-player-background-frame' now.
Also if we are lucky to have the inter-app communication API, the opened frame could be ignored here.
Just use the brand new communication API!
But it's still necessary to make music app to let system knows he is going to play content,
and I think this also solves the cases some app are using content channel but won't be controlled via lock-screen controller.
--
Alive C. Kuo, Front-end Eng., Mozilla Corp. (Taiwan, Taipei)
Andrea Marchesini <amarc...@mozilla.com> 於 2013/6/18 下午3:59 寫道：
Alive C. Kuo, Front-end Eng., Mozilla Corp. (Taiwan, Taipei)
> https://bugzilla.mozilla.org/show_bug.cgi?id=853101
David Flanagan
Jun 24, 2013, 2:57:57 PM6/24/13
to Mounir Lamouri, Gregor Wagner, dev-w...@lists.mozilla.org, Fabrice Desré, Alive Kuo, Jonas Sicking, Vivien Nicolas, Tim Guan-tin Chien, Hema Koka
On 6/18/13 6:14 AM, Mounir Lamouri wrote:
> Hi,
>
> I understand the intent behind this API but I am not sure we should use
> this instead of an API allowing communication between applications. An
> inter app communication API would be a powerful tool that would allow
> developers to do better than what we might end up doing with a
> MediaController API. Could you give a rationale why this would be better
> than a generic communication API?
A generic interapp communication API would be great (and it would allow
> Hi,
>
> I understand the intent behind this API but I am not sure we should use
> this instead of an API allowing communication between applications. An
> inter app communication API would be a powerful tool that would allow
> developers to do better than what we might end up doing with a
> MediaController API. Could you give a rationale why this would be better
> than a generic communication API?
us to stop abusing the settings API for that purpose). Is there a
proposal? We need a solution relatively quickly here.
> A few comments miscellaneous comments.
>
> On 15/06/13 01:16, David Flanagan wrote:
>> 3) We'd like to be able to control music playback from bluetooth
>> devices like headsets and car stereo systems. (For example: if the
>> phone is sending music output to a car stereo via bluetooth, then
>> the user should be able to control playback from the buttons on the
>> car's steering wheel.)
> I believe that D3E should already take care of that.
think that is what you're talking about.
David Flanagan
Jun 24, 2013, 3:06:19 PM6/24/13
to Alive Kuo, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Jonas Sicking, Vivien Nicolas, Andrea Marchesini, Tim Guan-tin Chien, Hema Koka
On 6/23/13 12:39 PM, Alive Kuo wrote:
> David,
>
Thank you for moving this forward, Alive!
> With Andrea's effort on bug 853101, system app would gain the power to
> control play/pause of certain app.
>
I don't see that yet in https://wiki.mozilla.org/WebAPI/AudioChannels,
but I assume that Andrea is adding a play/pause feature in the bug
itself and the wiki is just out of date.
What we need in addition to that low-level API is a meta-level of
control so that we can send application-level commands like
"next-track". I wonder if there is any way we can justify adding that
feature to the AudioChannels API itself. I'm not at all sure that that
is a good idea, but consider this use-case: when an alarm sounds, it
would be nice to allow the user to snooze or silence it from the
lockscreen. (I suspect we already do that, actually). But a very clean
way to do that would be to send a meta-command "snooze" from the
lockscreen to the clock app.
> However, as previous discussion, we still lack two important things:
>
> 1) How to identify the media player?
In my first draft of the API I proposed, there was an explicit
navigator.something API for the music player to use. Then I tried to
integrate that API directly on to the HTMLMediaElement. In retrospect, I
think that was probably a mistake. So if we define a MediaControl API,
identifying the player will be easy: its the app that uses the player
part of the API.
> 2) How to communicate?
> (A) From music to system (Update the playing song tittle….)
> (B) From system to music (Part of this would be achieve with new
> audio channel API)
>
Note that the draft UX designs we have for music app controls on the
lockscreen call for album art, so the player app to controller app
communication channel has to be able to pass blobs.
> I don't know the detail of so called 'inter-app communication API'…is
> it ongoing?
> in case we in the long don't have this magical stuff,
> I still have some ideas about these two questions, in a mid-term way.
>
If we are going to get some kind of inter-app communication API, I think
that would be the best way to proceed here.
I'd even suggest hacking something together (using the settings API for
communication and saving album art to device storage instead of
transferring that through settings) while we wait for an inter-app
communications API.
> The proposal is still my first proposal, using window.open(...) in
> music app,
> "but" system app won't bring the opened frame to foreground. (Because
> people hate to have too many frame in foreground:)
>
If this mechanism only allows communication between apps and the system
app, and it does not actually open a visible window (i.e. if it is not
some kind of generic widget feature) then it seems like there ought to
be a simpler way, like allowing postMessage and onmessage for
communication with the system. Perhaps I'm mis-understanding the
nuances of your proposal, Alive.
David
> So it is to say, let music app open a "background media player frame"
> to let system app knows it would like to be controlled.
>
> System app would then check the (1) content channel permission and (2)
> is it playing?(Using audio channel API).
>
> The opened frame could also be used for communication to system,
> and system would use mozBrowser API to get event when it changes
> something(like title-change, uh, weird I know.)
>
> This helps system app to identify who is going to be controlled, and,
> if multiple apps are going to be controlled,
> the key to find whom is relied on the app switch mechanism in system
> app(window manager) to find who is active app and playing content and
> having an opened 'media-player-background-frame' now.
>
> Also if we are lucky to have the inter-app communication API, the
> opened frame could be ignored here.
> Just use the brand new communication API!
> But it's still necessary to make music app to let system knows he is
> going to play content,
> and I think this also solves the cases some app are using content
> channel but won't be controlled via lock-screen controller.
> --
> Alive C. Kuo, Front-end Eng., Mozilla Corp. (Taiwan, Taipei)
>
> Andrea Marchesini <amarc...@mozilla.com
> David,
>
Thank you for moving this forward, Alive!
> With Andrea's effort on bug 853101, system app would gain the power to
> control play/pause of certain app.
>
but I assume that Andrea is adding a play/pause feature in the bug
itself and the wiki is just out of date.
What we need in addition to that low-level API is a meta-level of
control so that we can send application-level commands like
"next-track". I wonder if there is any way we can justify adding that
feature to the AudioChannels API itself. I'm not at all sure that that
is a good idea, but consider this use-case: when an alarm sounds, it
would be nice to allow the user to snooze or silence it from the
lockscreen. (I suspect we already do that, actually). But a very clean
way to do that would be to send a meta-command "snooze" from the
lockscreen to the clock app.
> However, as previous discussion, we still lack two important things:
>
> 1) How to identify the media player?
navigator.something API for the music player to use. Then I tried to
integrate that API directly on to the HTMLMediaElement. In retrospect, I
think that was probably a mistake. So if we define a MediaControl API,
identifying the player will be easy: its the app that uses the player
part of the API.
> 2) How to communicate?
> (A) From music to system (Update the playing song tittle….)
> (B) From system to music (Part of this would be achieve with new
> audio channel API)
>
lockscreen call for album art, so the player app to controller app
communication channel has to be able to pass blobs.
> I don't know the detail of so called 'inter-app communication API'…is
> it ongoing?
> in case we in the long don't have this magical stuff,
> I still have some ideas about these two questions, in a mid-term way.
>
that would be the best way to proceed here.
I'd even suggest hacking something together (using the settings API for
communication and saving album art to device storage instead of
transferring that through settings) while we wait for an inter-app
communications API.
> The proposal is still my first proposal, using window.open(...) in
> music app,
> "but" system app won't bring the opened frame to foreground. (Because
> people hate to have too many frame in foreground:)
>
app, and it does not actually open a visible window (i.e. if it is not
some kind of generic widget feature) then it seems like there ought to
be a simpler way, like allowing postMessage and onmessage for
communication with the system. Perhaps I'm mis-understanding the
nuances of your proposal, Alive.
David
> So it is to say, let music app open a "background media player frame"
> to let system app knows it would like to be controlled.
>
> System app would then check the (1) content channel permission and (2)
> is it playing?(Using audio channel API).
>
> The opened frame could also be used for communication to system,
> and system would use mozBrowser API to get event when it changes
> something(like title-change, uh, weird I know.)
>
> This helps system app to identify who is going to be controlled, and,
> if multiple apps are going to be controlled,
> the key to find whom is relied on the app switch mechanism in system
> app(window manager) to find who is active app and playing content and
> having an opened 'media-player-background-frame' now.
>
> Also if we are lucky to have the inter-app communication API, the
> opened frame could be ignored here.
> Just use the brand new communication API!
> But it's still necessary to make music app to let system knows he is
> going to play content,
> and I think this also solves the cases some app are using content
> channel but won't be controlled via lock-screen controller.
> --
> Alive C. Kuo, Front-end Eng., Mozilla Corp. (Taiwan, Taipei)
>
> Andrea Marchesini <amarc...@mozilla.com
Mike Habicher
Jun 24, 2013, 4:39:13 PM6/24/13
to David Flanagan, Alive Kuo, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Hema Koka, Vivien Nicolas, Tim Guan-tin Chien, Mounir Lamouri, Jonas Sicking
On 13-06-24 02:57 PM, David Flanagan wrote:
> On 6/18/13 6:14 AM, Mounir Lamouri wrote:
>> On 15/06/13 01:16, David Flanagan wrote:
>>> 3) We'd like to be able to control music playback from bluetooth
>>> devices like headsets and car stereo systems. (For example: if the
>>> phone is sending music output to a car stereo via bluetooth, then
>>> the user should be able to control playback from the buttons on the
>>> car's steering wheel.)
>> I believe that D3E should already take care of that.
> What's D3E? Google suggests http://d3e.sourceforge.net/, but I don't
> think that is what you're talking about.
Deep Googling turned up this:
> On 6/18/13 6:14 AM, Mounir Lamouri wrote:
>> On 15/06/13 01:16, David Flanagan wrote:
>>> 3) We'd like to be able to control music playback from bluetooth
>>> devices like headsets and car stereo systems. (For example: if the
>>> phone is sending music output to a car stereo via bluetooth, then
>>> the user should be able to control playback from the buttons on the
>>> car's steering wheel.)
>> I believe that D3E should already take care of that.
> What's D3E? Google suggests http://d3e.sourceforge.net/, but I don't
> think that is what you're talking about.
https://bugzilla.mozilla.org/show_bug.cgi?id=166240
So I'm guessing "D3E" means "DOM 3 Events":
http://www.w3.org/TR/DOM-Level-3-Events/
--m.
Mounir Lamouri
Jun 25, 2013, 9:35:42 AM6/25/13
to dev-w...@lists.mozilla.org, Alive Kuo, Fabrice Desré, Hema Koka, Gregor Wagner, Jonas Sicking, Vivien Nicolas, David Flanagan, Tim Guan-tin Chien, Mike Habicher
to google, sorry :(
-- Mounir
Alive Kuo
Jun 27, 2013, 5:24:44 AM6/27/13
to David Flanagan, dev-w...@lists.mozilla.org, Fabrice Desré, Gregor Wagner, Jonas Sicking, Vivien Nicolas, Andrea Marchesini, Tim Guan-tin Chien, Hema Koka
CJ and me has a discussion about this again today.
Comments inline.
David Flanagan <dfla...@mozilla.com> 於 2013/6/25 上午3:06 寫道：
>> With Andrea's effort on bug 853101, system app would gain the power to control play/pause of certain app.
>>
> I don't see that yet in https://wiki.mozilla.org/WebAPI/AudioChannels, but I assume that Andrea is adding a play/pause feature in the bug itself and the wiki is just out of date.
>
> What we need in addition to that low-level API is a meta-level of control so that we can send application-level commands like "next-track". I wonder if there is any way we can justify adding that feature to the AudioChannels API itself. I'm not at all sure that that is a good idea, but consider this use-case: when an alarm sounds, it would be nice to allow the user to snooze or silence it from the lockscreen. (I suspect we already do that, actually). But a very clean way to do that would be to send a meta-command "snooze" from the lockscreen to the clock app.
IMO empower new Audio channel API is necessary. since we still don't have other communication way between apps...
David, do you have the precise spec now?
I think if we want to show the song title or album art in the lockscreen or utility tray, we need to pass the metadata of the audio via the new mozBrowser audio channel API. CJ would co-ordinate this change with Andrea or make someone under his team to do this.
Andrea, what do you think?
>
>> However, as previous discussion, we still lack two important things:
>>
>> 1) How to identify the media player?
> In my first draft of the API I proposed, there was an explicit navigator.something API for the music player to use. Then I tried to integrate that API directly on to the HTMLMediaElement. In retrospect, I think that was probably a mistake. So if we define a MediaControl API, identifying the player will be easy: its the app that uses the player part of the API.
new API is OK for me. (TODO: need to tell system about this in audio channel API I think?)
>
>> 2) How to communicate?
>> (A) From music to system (Update the playing song tittle….)
>> (B) From system to music (Part of this would be achieve with new audio channel API)
>>
That's what I said above ;) But is it easier to pass metadata to system app via audio mozBrowserAPI or let music parse it?
>
>> I don't know the detail of so called 'inter-app communication API'…is it ongoing?
>> in case we in the long don't have this magical stuff,
>> I still have some ideas about these two questions, in a mid-term way.
>>
> If we are going to get some kind of inter-app communication API, I think that would be the best way to proceed here.
>
> I'd even suggest hacking something together (using the settings API for communication and saving album art to device storage instead of transferring that through settings) while we wait for an inter-app communications API.
we don't need this hack. It's just a substitution of (2) if we don't have that in the long run.
Comments inline.
David Flanagan <dfla...@mozilla.com> 於 2013/6/25 上午3:06 寫道：
>> With Andrea's effort on bug 853101, system app would gain the power to control play/pause of certain app.
>>
> I don't see that yet in https://wiki.mozilla.org/WebAPI/AudioChannels, but I assume that Andrea is adding a play/pause feature in the bug itself and the wiki is just out of date.
>
> What we need in addition to that low-level API is a meta-level of control so that we can send application-level commands like "next-track". I wonder if there is any way we can justify adding that feature to the AudioChannels API itself. I'm not at all sure that that is a good idea, but consider this use-case: when an alarm sounds, it would be nice to allow the user to snooze or silence it from the lockscreen. (I suspect we already do that, actually). But a very clean way to do that would be to send a meta-command "snooze" from the lockscreen to the clock app.
David, do you have the precise spec now?
I think if we want to show the song title or album art in the lockscreen or utility tray, we need to pass the metadata of the audio via the new mozBrowser audio channel API. CJ would co-ordinate this change with Andrea or make someone under his team to do this.
Andrea, what do you think?
>
>> However, as previous discussion, we still lack two important things:
>>
>> 1) How to identify the media player?
> In my first draft of the API I proposed, there was an explicit navigator.something API for the music player to use. Then I tried to integrate that API directly on to the HTMLMediaElement. In retrospect, I think that was probably a mistake. So if we define a MediaControl API, identifying the player will be easy: its the app that uses the player part of the API.
>
>> 2) How to communicate?
>> (A) From music to system (Update the playing song tittle….)
>> (B) From system to music (Part of this would be achieve with new audio channel API)
>>
>
>> I don't know the detail of so called 'inter-app communication API'…is it ongoing?
>> in case we in the long don't have this magical stuff,
>> I still have some ideas about these two questions, in a mid-term way.
>>
> If we are going to get some kind of inter-app communication API, I think that would be the best way to proceed here.
>
> I'd even suggest hacking something together (using the settings API for communication and saving album art to device storage instead of transferring that through settings) while we wait for an inter-app communications API.
> If this mechanism only allows communication between apps and the system app, and it does not actually open a visible window (i.e. if it is not some kind of generic widget feature) then it seems like there ought to be a simpler way, like allowing postMessage and onmessage for communication with the system. Perhaps I'm mis-understanding the nuances of your proposal, Alive.
Yes as you said, but if we have both (1) audio API to control the player from system app (2) API tell system who is going to be controlled as you said: navigator.XXXXX for music to register
we don't need this hack. It's just a substitution of (2) if we don't have that in the long run.
Thinker K.F. Li
Jul 15, 2013, 9:41:47 AM7/15/13
to dev-w...@lists.mozilla.org
I don't think inter-app control like this should be promoted as an Web
API. It should be an convention, rules, or protocols among apps.; music
app, lockscreen and system app are apps. This is a kind of
customization features. There are infinite number of APIs if we want to
support this kind of features.
I suggest to left this only a part of Gaia. Define a set of rules or a
protocol between default music app and lockscreen over IAC that we are
working on. We can define a set of conventions for music players, so
every music player apps are encouraged to implement them to get better
integration with the device.
David Flanagan <dfla...@mozilla.com> writes:
> This is a proposal for an API for communication between apps that play
> media and apps that want to control media playback. This is based on product
> requirements for FirefoxOS v1.2 and on preliminary discussion on the
> dev-gaia mailing list. (The basic idea is Vivien's though I have no
> idea whether
> this is what he had in mind.) I've cc'ed a bunch of people who have
> participated in
> those preliminary discussions or I suspect might be interested.
>
> FirefoxOS use-cases:
>
> 1) The Gaia Music app can play music while in the background.
> We need a way to allow the user to control playback from the
> lockscreen and/or other parts of the system. This is a product
> requirement for version 1.2, so we need to move quickly here.
>
> 2) It would be cool (but not currently required) if we could control
> the FM radio app this way, too.
>
--
Sinker
--
天教懶漫帶疏狂
API. It should be an convention, rules, or protocols among apps.; music
app, lockscreen and system app are apps. This is a kind of
customization features. There are infinite number of APIs if we want to
support this kind of features.
I suggest to left this only a part of Gaia. Define a set of rules or a
protocol between default music app and lockscreen over IAC that we are
working on. We can define a set of conventions for music players, so
every music player apps are encouraged to implement them to get better
integration with the device.
David Flanagan <dfla...@mozilla.com> writes:
> This is a proposal for an API for communication between apps that play
> media and apps that want to control media playback. This is based on product
> requirements for FirefoxOS v1.2 and on preliminary discussion on the
> dev-gaia mailing list. (The basic idea is Vivien's though I have no
> idea whether
> this is what he had in mind.) I've cc'ed a bunch of people who have
> participated in
> those preliminary discussions or I suspect might be interested.
>
> FirefoxOS use-cases:
>
> 1) The Gaia Music app can play music while in the background.
> We need a way to allow the user to control playback from the
> lockscreen and/or other parts of the system. This is a product
> requirement for version 1.2, so we need to move quickly here.
>
> 2) It would be cool (but not currently required) if we could control
> the FM radio app this way, too.
>
Sinker
--
天教懶漫帶疏狂
0 new messages