Save/restore feature

[Fabrice Desre]

Hi all,

One feature currently missing in b2g is a way to manage/backup/restore
data from the phone on a host computer. There are various scripts
floating around, but they are no substitute for a real end-user oriented
solution. Here's a rough proposal on how to build that functionality.

First of all, no UI should be chrome UI: both the host and the device
applications will be web applications with the appropriate privilege
level, even if the host application will very likely run in a Firefox tab.

Basic UX flow possible (Josh, I remember you showed me wireframes, can
you post them somewhere for our enlightenment?):
1. Plug in your phone with the setting "enable backup/restore" checked.
2. Open the host application.
3. This will launch the device application.
4. On the host, choose whatever you want to backup/restore.

On the technical side:
1. The "enable backup/restore" settings enables a debugging protocol
actor that will listen to connections on port 4343. On the host, we
forward the 4343 and 4344 ports trough adb.
2. The host application connects to the device by opening a socket to
localhost:4343.
3. The backup actor then sends a system message to wake up the backup
application. Once this application is running and listening on port
4344, it sends back an ack to the host application.
4. The host application checks that the device app is running by trying
to open a connection on locahost:4344.
5. Once this is done, host and device use a custom protocol to exchange
data. The host application saves everything using indexedDB.

On the device, the backup application could access data like contacts,
sms, etc. directly itself, but will also use the datastore and/or the
inter application communication apis to gather data from other
applications. This will let users backup and restore data from 3rd party
applications.

On b2g the system application could provide the backup functionality,
but I like the idea of a dedicated application better, because that will
let us support Android as well.

Thoughts?

Fabrice
--
Fabrice Desré
b2g team
Mozilla Corporation

[James Burke]

On Sat, Jul 6, 2013 at 3:58 PM, Fabrice Desre <fab...@mozilla.com> wrote:
> On the technical side:

This may be going too far, but what about the possibility of
implementing WebUSB[1], and using that in some fashion to allow a
different firefox phone to back up another phone with a direct USB
connection? I could see that as being useful for people that do not
have a dedicated PC.

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=674718

James

[Fabrice Desre]

On 07/06/2013 07:28 PM, James Burke wrote:
> On Sat, Jul 6, 2013 at 3:58 PM, Fabrice Desre <fab...@mozilla.com> wrote:
>> On the technical side:
>
> This may be going too far, but what about the possibility of
> implementing WebUSB[1], and using that in some fashion to allow a
> different firefox phone to back up another phone with a direct USB
> connection? I could see that as being useful for people that do not
> have a dedicated PC.

While I agree that supporting different transports like USB will be
needed, I don't think that phone to phone backup through USB is a very
important use case. Bluetooth would be a better candidate here, but we
need to implement the possibility to write new BT profiles in JS.

[Alex Jordan]

On Jul 6, 2013 4:00 PM, "Fabrice Desre" <fab...@mozilla.com> wrote:
> 1. The "enable backup/restore" settings enables a debugging protocol
> actor that will listen to connections on port 4343. On the host, we
> forward the 4343 and 4344 ports trough adb.

Small question: how does adb end up on the host computer?

[Fabrice Desre]

On 07/07/2013 12:42 AM, Alex Jordan wrote:
> On Jul 6, 2013 4:00 PM, "Fabrice Desre" <fab...@mozilla.com

> <mailto:fab...@mozilla.com>> wrote:
>> 1. The "enable backup/restore" settings enables a debugging protocol
>> actor that will listen to connections on port 4343. On the host, we
>> forward the 4343 and 4344 ports trough adb.
> Small question: how does adb end up on the host computer?

The simulator add-on includes a prebuilt adb executable. We will likely
do that at first also, and later have something more built-in.

[James Lal]

Any chance we can back up directly to an sd card and then periodically
update that? Maybe this is less of an end-user concern and more of a
developer/dogfooder concern but its really nice how my music/images/etc...
"just" work when reflashing the phone it would be nice to have some sort of
FTU detection of "Hey you have a backup of a phone on here, want to use
that?".

Updating from the host machine is obviously also amazing assuming we make
sure its hard to forget to pull off the backup when needed but I think the
SD card approach would be easier from my dogfooder's perspective.

> _______________________________________________
> dev-b2g mailing list
> dev...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-b2g
>

[Mike Habicher]

On 13-07-07 01:44 PM, James Lal wrote:
> Any chance we can back up directly to an sd card and then periodically
> update that? Maybe this is less of an end-user concern and more of a
> developer/dogfooder concern but its really nice how my music/images/etc...
> "just" work when reflashing the phone it would be nice to have some sort of
> FTU detection of "Hey you have a backup of a phone on here, want to use
> that?".

If we decide to do this, we should make sure to encrypt the backups to
prevent mass-storage slurpage.

--m.

[hong tang]

Hi Fabrice,

This is Hong from Beijing office.
Regarding the feature you mentioned below (Save/restore feature), we
already have a couple engineers started to work on it a few months ago.

Currently what have been implemented are:
1. On the phone side, we implemented a privileged app.
The app is a background service that will listen to connections on
a special port 10010, it responds to the commands coming from the host
application.
2. On the PC side(Windows/Linux/Mac), we implemented a Firefox addon as
the host application.
Basically user needs to install the addon and bring up the
management interface on Firefox. The addon forwards the remote port
10010 to local
port 10010 through shipped |adb|, it manages the device by sending
commands to a TCP sockect that connects to localhost:10010.
3. Specially on Windows, we implemented a daemon service to detect the
FirefoxOS device.
When the user plugged-in a FirefoxOS device via USB cable, the
daemon program can detect whether the adb driver exists already, if not,
it will
automatically install the driver for the user.
4. So far we have finished managing/backuping/restoring data for contact
and SMS. In about a month time we will have picture/music/video
backup/restore completed.

I will send you a more detailed document some time this week.

Best regards,
Hong

>
>
> -------- Original Message --------
> Subject: [b2g] Save/restore feature
> Date: Sat, 06 Jul 2013 15:58:22 -0700
> From: Fabrice Desre <fab...@mozilla.com>
> To: dev-b2g <dev...@lists.mozilla.org>
> CC: Josh Carpenter <jcarp...@mozilla.com>
>
>
>
> Hi all,
>
> One feature currently missing in b2g is a way to manage/backup/restore
> data from the phone on a host computer. There are various scripts
> floating around, but they are no substitute for a real end-user oriented
> solution. Here's a rough proposal on how to build that functionality.
>
> First of all, no UI should be chrome UI: both the host and the device
> applications will be web applications with the appropriate privilege
> level, even if the host application will very likely run in a Firefox tab.
>
> Basic UX flow possible (Josh, I remember you showed me wireframes, can
> you post them somewhere for our enlightenment?):
> 1. Plug in your phone with the setting "enable backup/restore" checked.
> 2. Open the host application.
> 3. This will launch the device application.
> 4. On the host, choose whatever you want to backup/restore.
>
> On the technical side:

> 1. The "enable backup/restore" settings enables a debugging protocol
> actor that will listen to connections on port 4343. On the host, we
> forward the 4343 and 4344 ports trough adb.

> 2. The host application connects to the device by opening a socket to
> localhost:4343.
> 3. The backup actor then sends a system message to wake up the backup
> application. Once this application is running and listening on port
> 4344, it sends back an ack to the host application.
> 4. The host application checks that the device app is running by trying
> to open a connection on locahost:4344.
> 5. Once this is done, host and device use a custom protocol to exchange
> data. The host application saves everything using indexedDB.
>
> On the device, the backup application could access data like contacts,
> sms, etc. directly itself, but will also use the datastore and/or the
> inter application communication apis to gather data from other
> applications. This will let users backup and restore data from 3rd party
> applications.
>
> On b2g the system application could provide the backup functionality,
> but I like the idea of a dedicated application better, because that will
> let us support Android as well.
>
> Thoughts?
>

[Fabrice Desre]

Hi Hong,

On 07/08/2013 07:50 AM, hong tang wrote:
> Hi Fabrice,
>
> This is Hong from Beijing office.
> Regarding the feature you mentioned below (Save/restore feature), we
> already have a couple engineers started to work on it a few months ago.

Good to know that people are working on that.

But... where are the public bugs (and why no comment in
https://bugzilla.mozilla.org/show_bug.cgi?id=821985)? Anything announced
on mailing lists? Do you realize that if other teams started the same
work behind closed doors we would have duplicated effort for no real
benefit? I'm probably sounding like a broken record at this point, but I
urge you to disclose all such projects publicly.

> Currently what have been implemented are:
> 1. On the phone side, we implemented a privileged app.
> The app is a background service that will listen to connections on a
> special port 10010, it responds to the commands coming from the host
> application.
> 2. On the PC side(Windows/Linux/Mac), we implemented a Firefox addon as
> the host application.
> Basically user needs to install the addon and bring up the
> management interface on Firefox. The addon forwards the remote port
> 10010 to local
> port 10010 through shipped |adb|, it manages the device by sending
> commands to a TCP sockect that connects to localhost:10010.

I'm strongly against using an add-on for the UI. Let's ship everything
else in Firefox itself.

[Thinker K.F. Li]

I think BT is a very poor and slow option.
WIFI is much better than BT if want to backup data wirelessly.
With WIFI, we have much more protocols in the list; ex. WebDAV.

Sinker
--
天教懶漫帶疏狂

[Fabrice Desre]

On 07/08/2013 09:14 PM, Thinker K.F. Li wrote:
> I think BT is a very poor and slow option.
> WIFI is much better than BT if want to backup data wirelessly.
> With WIFI, we have much more protocols in the list; ex. WebDAV.

Sure Wifi is faster and overall more flexible, but do we have a nice way
to do pairing to initiate the connection over wifi?

[Yuan Xulei]

On Tue 09 Jul 2013 12:31:13 PM CST, Fabrice Desre wrote:
> On 07/08/2013 09:14 PM, Thinker K.F. Li wrote:
>> I think BT is a very poor and slow option.
>> WIFI is much better than BT if want to backup data wirelessly.
>> With WIFI, we have much more protocols in the list; ex. WebDAV.
>
> Sure Wifi is faster and overall more flexible, but do we have a nice way
> to do pairing to initiate the connection over wifi?

We can use a randomly generated passcode (like the passcode to pair
Firefox sync device), or event a 2D bar code.

Yuan

[Kan-Ru Chen (陳侃如)]

Or use WebRTC! Isn't it provides a way to communicate between clients?
Connection over wifi has to handle NAT traversal and setup the peers
which has been resolved by WebRTC.

Kanru

[Fabrice Desre]

On 07/08/2013 10:11 PM, Kan-Ru Chen (陳侃如) wrote:

> Or use WebRTC! Isn't it provides a way to communicate between clients?
> Connection over wifi has to handle NAT traversal and setup the peers
> which has been resolved by WebRTC.

Yep, very good idea!

[Yuan Xulei]

On 07/09/2013 01:11 PM, Kan-Ru Chen (陳侃如) wrote:
> Or use WebRTC! Isn't it provides a way to communicate between clients?
> Connection over wifi has to handle NAT traversal and setup the peers

> which has been resolved by WebRTC. Kanru
WebRTC seems a little complicated. If I'm not wrong, it needs an extra
server to help the host application and the phone to establish
connection. In the backup and restore user case, the extra server seems
unnecessary, as the host application and the phone are usually on the
same IP subnet and could be connected directly.

[Thinker K.F. Li]

For pairing, it includes two features.

- Service Discovery -- find each other.
- Authentication & Authorization.

UPnP is an obvious solution for service discovery. For authentication
and authorization, it can be a plain password or a 2D bar code.

--
Sinker
--
天教懶漫帶疏狂

[Shih-Chiang Chien]

For WebRTC, we can use any kinds of medium to deliver the SDP between host and device. It's not necessary to have a extra server for it. So for example, we could read the SDP from a QR code via camera.

Best Regards,
Shih-Chiang Chien
Mozilla Taiwan

On Jul 9, 2013, at 3:00 PM, Yuan Xulei <xy...@mozilla.com> wrote:

> On 07/09/2013 01:11 PM, Kan-Ru Chen (陳侃如) wrote:

>> Or use WebRTC! Isn't it provides a way to communicate between clients? Connection over wifi has to handle NAT traversal and setup the peers which has been resolved by WebRTC. Kanru

> WebRTC seems a little complicated. If I'm not wrong, it needs an extra server to help the host application and the phone to establish connection. In the backup and restore user case, the extra server seems unnecessary, as the host application and the phone are usually on the same IP subnet and could be connected directly.

[Panos Astithas]

I know that Brian Smith and Tanvi Vyas from our security team had done some
research on authentication and authorization solutions for the remote
debugging protocol (which we haven't implemented yet) that included
protocols like ZRTP and J-PAKE. It would be a good idea to talk to them
about this.

On Tue, Jul 9, 2013 at 10:54 AM, Thinker K.F. Li <thi...@codemud.net>wrote:

> For pairing, it includes two features.
>
> - Service Discovery -- find each other.
> - Authentication & Authorization.
>
> UPnP is an obvious solution for service discovery. For authentication
> and authorization, it can be a plain password or a 2D bar code.
>
> "Kan-Ru Chen (陳侃如)" <kc...@mozilla.com> writes:
>

> --
> Sinker
> --
> 天教懶漫帶疏狂

[Julien Wajsberg]

Le 08/07/2013 18:43, Fabrice Desre a écrit :
> 2. On the PC side(Windows/Linux/Mac), we implemented a Firefox addon
> as the host application. Basically user needs to install the addon and
> bring up the management interface on Firefox. The addon forwards the
> remote port 10010 to local port 10010 through shipped |adb|, it
> manages the device by sending commands to a TCP sockect that connects
> to localhost:10010.
> I'm strongly against using an add-on for the UI. Let's ship everything
> else in Firefox itself.

why that ? I mean, more used features were moved to addons, why this
specific feature should be in the default Firefox even for people that
don't use a FxOS device ?

--
Julien

[hong tang]

Hi Fabrice,

Basically our requirement is from China local partners. Everyone is
asking for Wandoujia liked product, which is Andoid mobile user
assistent and commonly used here in China market.

The major effort for this tool will be on PC side, not only
backup/restore information, user can also manage his content information
through PC management interface, like add/delete contacts, send sms, etc.

Thanks,
Hong

于 2013/7/9 0:43, Fabrice Desre 写道:
> Hi Hong,
>
> On 07/08/2013 07:50 AM, hong tang wrote:
>> Hi Fabrice,
>>
>> This is Hong from Beijing office.
>> Regarding the feature you mentioned below (Save/restore feature), we
>> already have a couple engineers started to work on it a few months ago.
> Good to know that people are working on that.
>
> But... where are the public bugs (and why no comment in
> https://bugzilla.mozilla.org/show_bug.cgi?id=821985)? Anything announced
> on mailing lists? Do you realize that if other teams started the same
> work behind closed doors we would have duplicated effort for no real
> benefit? I'm probably sounding like a broken record at this point, but I
> urge you to disclose all such projects publicly.
>
>> Currently what have been implemented are:
>> 1. On the phone side, we implemented a privileged app.
>> The app is a background service that will listen to connections on a
>> special port 10010, it responds to the commands coming from the host
>> application.

>> 2. On the PC side(Windows/Linux/Mac), we implemented a Firefox addon as
>> the host application.
>> Basically user needs to install the addon and bring up the
>> management interface on Firefox. The addon forwards the remote port
>> 10010 to local
>> port 10010 through shipped |adb|, it manages the device by sending
>> commands to a TCP sockect that connects to localhost:10010.
> I'm strongly against using an add-on for the UI. Let's ship everything
> else in Firefox itself.
>

> Fabrice

["Yuan Xulei(袁徐磊)"]

Hi all,

We create a demo tool for this. It allows user to manage, backup and restore data (such as contacts, sms, pictures, music...) of a Firefox OS 1.3 phone from a host computer. For details and screenshots, see:http://1drv.ms/1fUzBbk

The tool is composed of a host application running in a Firefox desktop tab and a standalone device application.

The host application can establish a connection to the device application by either USB cable or WIFI. Once connected, use can view the data and manage them directly. The host application and the device application keeps data synchronized after user's operations.

I have a idea that we may integrate these functions into the app manager of desktop and extend the app manager to something like a device manager. User can manage or backup their phones' data through Firefox on a computer without installing other tools.

The source code is available on github:
Host application - https://github.com/MozillaOnline/pc-sync-tool
Device application - https://github.com/MozillaOnline/pc-sync-gaia-app

Any thoughts?

Yuan