WEP 108: Preference Sync
Anant Narayanan
I wrote up a WEP that proposes two changes to the current preference
sync engine in order to obtain better results. https://wiki.mozilla.org/Labs/Weave/WEP/108
I look forward to your feedback!
Regards,
Anant
Mike Connor
My first question is "why are we syncing default values at all?"
Especially since some platforms have different default values. I
would think WBO-per-non-default-value would be reasonable.
How about something like this:
- On download, if local is default, apply the remote value.
- If local is a different non-default value, apply the remote value.*
- If the local value changes to the default, treat that like a
DELETE. Other clients will take that delete and reset the pref.
* This is a change from defaults, but it's a sane one. Here's why:
When a client changes a pref, we'll sync to the server, and other
clients will have that pave over. If another client changes the pref,
it'll then propagate cleanly. The only issue is the race condition
between two changes to non-default values for the same pref in the
same sync window. We could mitigate that by including a timestamp for
the change in the record and do resolution that way (last change
wins). Probably in practice not a big deal.
-- Mike
Anant Narayanan
On Sep 5, 2009, at 9:07 PM, Mike Connor wrote:
> My first question is "why are we syncing default values at all?"
> Especially since some platforms have different default values. I
> would think WBO-per-non-default-value would be reasonable.
I agree that we shouldn't be syncing default values at all. That's
just an artifact of the way in which the engine was initially written
-- I'll add a check to see if the local value is default, and if yes,
not include it in the upload record (or issue a DELETE if the pref was
changed to a default value).
However, I don't think one-WBO-per-non-default-value is reasonable,
the overhead would be quite a bit (especially for preferences that are
just boolean values). We thought grouping preferences by branch prefix
into a single WBO seemed like a good compromise between having several
tiny WBOs and one giant WBO. Are there any compelling reasons for not
doing this?
> How about something like this:
>
> - On download, if local is default, apply the remote value.
> - If local is a different non-default value, apply the remote value.*
> - If the local value changes to the default, treat that like a
> DELETE. Other clients will take that delete and reset the pref.
This sounds good to me. I'll add a section to the WEP that details
this and a policy for conflict resolution. IIRC, timestamps are
automatically added to a WBO by the server but if not we could
certainly have one and have the latest pref win.
Thanks for your comments!
Regards,
Anant
Mike Connor
On 6-Sep-09, at 1:17 AM, Anant Narayanan wrote:
>
> Hi Mike,
>
> On Sep 5, 2009, at 9:07 PM, Mike Connor wrote:
>> My first question is "why are we syncing default values at all?"
>> Especially since some platforms have different default values. I
>> would think WBO-per-non-default-value would be reasonable.
>
> I agree that we shouldn't be syncing default values at all. That's
> just an artifact of the way in which the engine was initially written
> -- I'll add a check to see if the local value is default, and if
> yes,
> not include it in the upload record (or issue a DELETE if the pref was
> changed to a default value).
>
> However, I don't think one-WBO-per-non-default-value is reasonable,
> the overhead would be quite a bit (especially for preferences that are
> just boolean values). We thought grouping preferences by branch prefix
> into a single WBO seemed like a good compromise between having several
> tiny WBOs and one giant WBO. Are there any compelling reasons for not
> doing this?
So we're going to create a record if anything in the prefbranch is non-
default, and propagate that value to all other clients? How do we
deal with different-OS defaults? If foo.bar is true on Windows, and
false on Linux, and I set to true on Linux, that value gets synced up,
then applied to the Windows box. But the pref API optimizes same-as-
default away, so if I change a related pref in the same prefbranch,
the record gets overwritten without the non-default pref from the
Linux box, unless I'm missing something.
We're not syncing that many records, and most prefs will be default
and thus not synced, so I don't know if the complexity for record-per-
branch is a win or not.
-- Mike
Edward Lee
> So we're going to create a record if anything in the prefbranch is non-
> default, and propagate that value to all other clients?
implement this, there's strange situations that come up. If you only
send changed values in a super-record that contains a branch, how do
you handle different mixes of modified/default. What about multiple
records that have overlapping sub-branches? How do you send a "delete
record" for just a part of a branch?
Additionally, there's issues of assigning GUIDs to these records where
the patch just uses the branch name, which could expose the value
being synced. (A record for a boolean must mean it's the non-default
value.)
Do clients need to know what preferences are being synced? I would
assume somebody adding a pref to be synced would expect other clients
to sync back changes to that pref.
Ed
Anant Narayanan
On Sep 22, 2009, at 5:31 AM, Mike Connor wrote:
> So we're going to create a record if anything in the prefbranch is
> non-
> default, and propagate that value to all other clients? How do we
> deal with different-OS defaults? If foo.bar is true on Windows, and
> false on Linux, and I set to true on Linux, that value gets synced up,
> then applied to the Windows box. But the pref API optimizes same-as-
> default away, so if I change a related pref in the same prefbranch,
> the record gets overwritten without the non-default pref from the
> Linux box, unless I'm missing something.
This is correct, and certainly an issue, but only when the default
value of a particular OS happens to be the non-default value set by
the user at another OS. However, this problem would occur even if we
create one WBO per non-default pref value, as the Windows box would
issue a DELETE for that WBO upon realizing that the applied value was
it's local default.
In order to solve this correctly, we need OS information in the WBO.
If a preference has different default values in two OSes, it is
unlikely that it is a preference to be synchronized, thus we shouldn't
be optimizing for this edge case (in my opinion).
> We're not syncing that many records, and most prefs will be default
> and thus not synced, so I don't know if the complexity for record-per-
> branch is a win or not.
I'm also inclined towards using just one WBO per pref value, that will
make a lot of things simpler. IIRC, Dan didn't like the idea because
of all the extra metadata per WBO - for example we'd be storing
approximately 240 bytes of metadata for just 4 or 5 bytes of a boolean
pref value. I think it's an OK compromise given that we aren't limited
by bandwidth (maybe on Fennec we are?). Dan?
Regards,
Anant
Anant Narayanan
On Sep 22, 2009, at 9:26 AM, Edward Lee wrote:
> I agree that there's weirdness and looking at the proposed patch to
> implement this, there's strange situations that come up. If you only
> send changed values in a super-record that contains a branch, how do
> you handle different mixes of modified/default. What about multiple
> records that have overlapping sub-branches? How do you send a "delete
> record" for just a part of a branch?
We carefully set the default preference sync parameters to ensure that
you would never have two records that have overlapping sub-branches,
and I think our strategy should be to maintain that. If a user goes
and modifies the sync preferences to allow this to happen, then he's
on his own.
A "delete record" for just a part of a branch is equivalent to not
including all those sub-parts of the branch in the WBO at the next
sync. All other client would apply only the values present in the WBO
and reset everything else in that branch to their default.
> Additionally, there's issues of assigning GUIDs to these records where
> the patch just uses the branch name, which could expose the value
> being synced. (A record for a boolean must mean it's the non-default
> value.)
Yes, this is a known privacy issue. I proposed a MD5 hash of the pref
branch + password (or passphrase) to be the GUID for the WBO, but Dan
didn't like that idea because it's a lot of processing (and we didn't
have very great results with this scheme for form sync).
> Do clients need to know what preferences are being synced? I would
> assume somebody adding a pref to be synced would expect other clients
> to sync back changes to that pref.
This is easily achieved by synchronizing the
extensions.weave.preferences.sync branch.
Regards,
Anant
P.S. Ed, thanks for your comments on the bug, I shall respond to them
there as well.