After restoring and reorganizing my bookmark tree back in january and moving the items in the unsorted bookmarks back to their real destination, I forced the tree to the server, and then back to all my machines. Sync 1.6.2 (I'm running FF 3.6.13) was supposed to fix most serious problems, but I also bumped the number of backup copies to keep to some large number, just in case.
Well, that happens to have been a good idea: I just found out that that my bookmarks tree has been severy destroyed again; Duplicate folders, duplicate items in folders, and something like 25% of my bookmarks (both single items and complete folders) have again landed in the unsorted bookmarks folder. Worse, a number of items and folders seem to have completely vanished, and a number of folders that were heavily populated are now empty...
I should have enough backup copies to recover, but I will have lost any bookmark that I stored between end of january and today <sigh>... and I'm going to bump the number of backup copies again.
So it might not be a good idea to assume that 1.6.2 fixed the problem and is now safe...
> Well, that happens to have been a good idea: > I just found out that that my bookmarks tree has been severy destroyed again; Duplicate folders, duplicate items in folders, and something like 25% of my bookmarks (both single items and complete folders) have again landed in the unsorted bookmarks folder. > Worse, a number of items and folders seem to have completely vanished, and a number of folders that were heavily populated are now empty...
> I should have enough backup copies to recover, but I will have lost any bookmark that I stored between end of january and today <sigh>... and I'm going to bump the number of backup copies again.
Sorry to hear that. To clarify:
* You're not running any other bookmark-related add-ons * You're running 1.6.2 on every machine * You restored from a backup after installing 1.6.2 * You did "Replace other machines" on the restore machine, and "replace this machine" on every other
I can't think of anything that Sync does that would cause bookmarks or folders to be deleted without explicit user action — that is, you deleting them on another machine. Any problem that would cause an improper removal (e.g., in dupe handling) would throw an exception first.
Are there any errors in your about:sync-log or about:sync-log.1 ?
If the above four bullet points are true, please file a bug:
> After restoring and reorganizing my bookmark tree back in january and moving
> the items in the unsorted bookmarks back to their real destination, I forced
> the tree to the server, and then back to all my machines. Sync 1.6.2 (I'm
> running FF 3.6.13) was supposed to fix most serious problems, but I also
> bumped the number of backup copies to keep to some large number, just in
> case.
> Well, that happens to have been a good idea:
> I just found out that that my bookmarks tree has been severy destroyed
> again; Duplicate folders, duplicate items in folders, and something like 25%
> of my bookmarks (both single items and complete folders) have again landed
> in the unsorted bookmarks folder.
> Worse, a number of items and folders seem to have completely vanished, and a
> number of folders that were heavily populated are now empty...
> I should have enough backup copies to recover, but I will have lost any
> bookmark that I stored between end of january and today <sigh>... and I'm
> going to bump the number of backup copies again.
> So it might not be a good idea to assume that 1.6.2 fixed the problem and is
> now safe...
> Sync never was and I think never will be safe....they simply don't > seem to be able to get it right...
Funny thing is, I've been running sync on several machines now since the 1.0 days and I haven't had many problems at all. In fact, the problems I had initially were due to my own misconfiguration problems on my server. Since I fixed that problem, sync has been working wonderfully.
- --------------------------- Jason 'XenoPhage' Frisvold xenoph...@godshell.com - --------------------------- "Any sufficiently advanced magic is indistinguishable from technology." - - Niven's Inverse of Clarke's Third Law
Ok, maybe you didn't have problems but keep in mind that: 1) You -
doesn't mean others too 2) You run your own server, while most people
use Mozilla infrastructure.
Also, me and many other people have problems with Sync, while didn't
have such problems when using other applications such as xmarks.
On 20 Feb, 04:16, Jason 'XenoPhage' Frisvold <xenoph...@godshell.com>
wrote:
> > Sync never was and I think never will be safe....they simply don't
> > seem to be able to get it right...
> Funny thing is, I've been running sync on several machines now since the 1.0 days and I haven't had many problems at all. In fact, the problems I had initially were due to my own misconfiguration problems on my server. Since I fixed that problem, sync has been working wonderfully.
> - ---------------------------
> Jason 'XenoPhage' Frisvold
> xenoph...@godshell.com
> - ---------------------------
> "Any sufficiently advanced magic is indistinguishable from technology."
> - - Niven's Inverse of Clarke's Third Law
> Ok, maybe you didn't have problems but keep in mind that: 1) You - > doesn't mean others too 2) You run your own server, while most people > use Mozilla infrastructure. > Also, me and many other people have problems with Sync, while didn't > have such problems when using other applications such as xmarks.
Yup. That about sums it up. It gets tiring seeing complaints about how bad sync is, how they can't get it right, how xmarks is better, etc. My point is, if you don't like how it works you have a few options. Don't use it, fix it yourself (the source code is there), or just go back to xmarks.
Now, if you're going to provide useful feedback that can be used to identify and resolve these bugs, that's great and I strongly encourage it. Providing as much detail as possible is a really good starting point. That would include the sync-log, what addons you have installed, and a detailed description of the problem.
- --------------------------- Jason 'XenoPhage' Frisvold xenoph...@godshell.com - --------------------------- "Any sufficiently advanced magic is indistinguishable from technology." - - Niven's Inverse of Clarke's Third Law
> Ok, maybe you didn't have problems but keep in mind that: 1) You - > doesn't mean others too 2) You run your own server, while most people > use Mozilla infrastructure.
I (as you might imagine) have put my profile through the absolute wringer with Sync versions, development and stable, on Mozilla's infrastructure. I've never experienced any problems with bookmarks (or anything else, for that matter).
None of the reported problems have any dependency on the server; the server is really just a dumb object store.
There's no evidence to suggest that Sync is intrinsically unreliable, or that more than a minority of users have problems.
That doesn't mean that we don't want to make those users happy, but I feel it's important to bring some balance to the discussion.
> Now, if you're going to provide useful feedback that can be used to identify and resolve these bugs, that's great and I strongly encourage it. Providing as much detail as possible is a really good starting point. That would include the sync-log, what addons you have installed, and a detailed description of the problem.
Add-ons are an important point. At least one of the "ZOMG my sync is broken!!!1!" reports was traced back to a malformed login item... which was created by the Xmarks add-on. I expect the same is true for bookmarks.
Well, I am not saying that add-ons are not a problem, they can easily
be.
However, I've seen a few threads here on google group where some
mozilla devs said that problems were caused server side. So I don't
agree that the sync server is completely out of this.
Secondly, when I had problems, I provided sync-logs when possible, but
nobody asked me to provide installed add-ons, other than asking
whether I had sync addon installed or not on firefox 4.
What I'm complaining about is not the fact that there are problems
here or there but rather that my bookmarks collection (as well as
other users too) have been completely messed up far too many times.
Also, nobody said "warning sync can wreck havoc to your bookmarks" as
far as I remember, correct me if I'm wrong.
And lastly, if the server is really just a dumb object store, then why
does it mess up my collection by creating duplicates, moving bookmarks
out of their folders etc?
On 20 Feb, 21:49, Richard Newman <holyg...@gmail.com> wrote:
> > Now, if you're going to provide useful feedback that can be used to identify and resolve these bugs, that's great and I strongly encourage it. Providing as much detail as possible is a really good starting point. That would include the sync-log, what addons you have installed, and a detailed description of the problem.
> Add-ons are an important point. At least one of the "ZOMG my sync is broken!!!1!" reports was traced back to a malformed login item... which was created by the Xmarks add-on. I expect the same is true for bookmarks.
> However, I've seen a few threads here on google group where some > mozilla devs said that problems were caused server side. So I don't > agree that the sync server is completely out of this.
FYI, I am a Mozilla dev.
I can think of one issue that was server-related, and it didn't affect bookmarks.
> And lastly, if the server is really just a dumb object store, then why > does it mess up my collection by creating duplicates, moving bookmarks > out of their folders etc?
The server has nothing to do with that.
Clients use the server to store records. The clients apply and generate those records to share changes. Blaming the server would be like blaming the bookstore when you re-buy a book you already have!
Here's a little journey into the details of this problem, if you're interested.
The code that formed the original version of the bookmarks sync code was slightly fragile, and its error handling turned out to be flawed. It worked fine for a very long time, because conditions weren't right to expose the problems -- in fact, for the vast majority of situations (and our QA folks have a lot of them) we couldn't even force it to break.
(This, incidentally, describes most of the software -- and hardware! -- you use. Stable, reliable software is usually stable and reliable because a piece of fragile, flawed software got fixed over time, and any remaining bugs don't occur in practice. Sometimes "time" is as much as 25 years[1], so I think we're doing pretty well!)
However, throw into the mix some recent changes in the Firefox core that made some dubious operations (such as adding invalid password records) fail, the wide variety of profiles, add-ons, and old data "in the wild", and a big Sync version bump that required a full sync, and suddenly errors start to occur in applying bad old records. Those errors caused sync to abort before finishing all of the reorganization it needs to do at the end of a sync... and dupes or re-ordered bookmarks, or just incomplete syncs, were the result.
The flaws in this old code are why there was no point rolling back any Sync releases in response to reports on the mailing list -- the flaws in the bookmarks engine have been quietly waiting in the background for two years! Any rollback would simply lose the benefit of recent changes.
Suffice to say that a large part of the work that Philipp and I have done for 1.6.2 and 1.6.3 was to address these issues, and similar mismatches in other engines:
Forms: Bug 597400: return null if GUID lookup fails in forms engine.
History: Bug 606353 - History sync: use mozIAsyncHistory::updatePlaces. Bug 621594 - History sync: failure when title is null. Bug 623375 - History sync: failure on javascript: URLs.
Passwords: Bug 609380: password sync: don't apply records with both httpRealm and formSubmitURL.
Bookmarks: Bug 610501: better rewriting of tag search smart bookmarks. Bug 610501: dupe Smart Bookmarks based on anno. Bug 610501: handle smart bookmarks correctly in Sync. Bug 618403 - Orphan reparenting too aggressive, can lead to double bookmarks on storage version upgrade. Bug 621489 - Make sure to call _orderChildren even if _processIncoming fails. Bug 623418 - Bookmark sync: don't record children in annotation. Bug 626796: Bookmark sync: restore from backup should trigger reupload. Bug 627490: Bookmark sync: don't cache places IDs. Bug 627497: remove annotations from copied bookmarks. Bug 628788: copying a bookmark should yield a new Sync GUID. Bug 634401 - Use nsINavBookmarksService::moveItem rather than setItemIndex in Sync.
General: Bug 615284 - Download chunking needs to be more resilient against app shutdowns. Bug 622762 - Add batch API for incoming records, deal with exceptions in SyncEngine's recordHandler.
tl;dr:
Philipp and I are fairly new to the party, and so we didn't write much of this code, and it was only a recent confluence of events that exposed problems in it. Once we had received enough detailed reports to understand the issues, we trawled through it fixing as many problems as we could find.
Sync should now skip over bad records, treat bookmark reordering with much greater care, and in general behave much more sanely. Philipp and I are much happier with the state of the code now, and we hope you will be, too.
> And lastly, if the server is really just a dumb object store, then why > does it mess up my collection by creating duplicates, moving bookmarks > out of their folders etc?
Remember, the server does not see your bookmarks/folders, etc. It get a bunch of encrypted data, stores it, then returns it on demand. Moving a bookmark would take banks of computers working feverishly to crack the encryption, and that sounds like way more hassle to implement than it would be worth!
This isn't to say the server can't cause problems - it can go down, hang or be otherwise unavailable, but munging the data would simply produce unreadable records, not moved or duplicate items.
On Feb 21, 2011, at 3:25 PM, Richard Newman wrote:
> Here's a little journey into the details of this problem, if you're interested.
<snip>
> Philipp and I are fairly new to the party, and so we didn't write much of this code, and it was only a recent confluence of events that exposed problems in it. Once we had received enough detailed reports to understand the issues, we trawled through it fixing as many problems as we could find.
> Sync should now skip over bad records, treat bookmark reordering with much greater care, and in general behave much more sanely. Philipp and I are much happier with the state of the code now, and we hope you will be, too.
Well done. And a very nice writeup.
- --------------------------- Jason 'XenoPhage' Frisvold xenoph...@godshell.com - --------------------------- "Any sufficiently advanced magic is indistinguishable from technology." - - Niven's Inverse of Clarke's Third Law
Ok, so what moves/deletes or otherwise messes up my bookmarks? Me
certainly not, because I only open them when I need to open them. If
the server is not the cause of the problem then it has to be the
client, right?
On 22 Feb, 18:16, Toby Elliott <telli...@mozilla.com> wrote:
> > And lastly, if the server is really just a dumb object store, then why
> > does it mess up my collection by creating duplicates, moving bookmarks
> > out of their folders etc?
> Remember, the server does not see your bookmarks/folders, etc. It get a bunch of encrypted data, stores it, then returns it on demand. Moving a bookmark would take banks of computers working feverishly to crack the encryption, and that sounds like way more hassle to implement than it would be worth!
> This isn't to say the server can't cause problems - it can go down, hang or be otherwise unavailable, but munging the data would simply produce unreadable records, not moved or duplicate items.
> Ok, so what moves/deletes or otherwise messes up my bookmarks? Me > certainly not, because I only open them when I need to open them. If > the server is not the cause of the problem then it has to be the > client, right?
Correct.
The client is responsible for all Sync functionality, both features and bugs!
OK. I'm one of those "minority of users" where sync doesn't work anymore ("*Sync encountered an error while syncing: Unknown error. Sync will retry this action.*"), and I didn't hear about the "restore bookmarks to repair the bookmarks that were broken by Sync" solution until more than ten days after the problem started - so *my bookmark backups are all broken by this sync bug*.
Is it the case that my bookmarks, history, passwords, and/or forms database(s) is/are *broken beyond repair* by sync? If no, *how* can I fix my data?
> OK. I'm one of those "minority of users" where sync doesn't work anymore ("Sync encountered an error while syncing: Unknown error. Sync will retry this action.")
Have you actually examined your bookmarks to see whether you've suffered any duplication or reordering, or are you just alarmed by the error message?
Most likely you have a single bad record (e.g., a blank password) from an earlier version of Firefox that's failing to be applied. If you post your about:sync-log (or, even better, file a bug with it attached), we can take care of it.
This error message doesn't necessarily indicate any kind of damage to your local data. It's exactly what it says: there was an unexpected error, and Sync will try again.
> Is it the case that my bookmarks, history, passwords, and/or forms database(s) is/are broken beyond repair by sync? If no, how can I fix my data?
They shouldn't be.
Firstly, there should be nothing wrong with your history, passwords, or forms. The worst that could happen with those sync engines is that a bad record aborts Sync part-way through, and so some changes from other machines haven't been applied whilst you've been seeing this error. You can either wait for 1.6.3, which will skip over these bad records, or file a bug and we can tell you how to work around it.
Secondly, your bookmarks (in the worst case) should only have been duplicated or reordered, not broken beyond repair. Fixing that is a simple — if time-consuming — process of rearranging your bookmarks in the browser.
On Feb 16 10:11AM -0800, Richard Newman <holy...@gmail.com> wrote: * You're not running any other bookmark-related add-ons * You're running 1.6.2 on every machine * You restored from a backup after installing 1.6.2 * You did "Replace other machines" on the restore machine, and "replace this machine" on every other
Yes, Yes, Yes, and No. About the final No: I didn't do "replace this machine" on the others because 1) I expected Sync to eventually do it on the other machines (and it actually did). 2) My usual 5 machines are not located in the same physical location.
Now I *might* have found a corruption scenario. At least, I could experimentally reproduce something that looks very much like it. I only did so after bumping my max_backups to 40 on all machines and restarting ff.
If I abruptly stop Firefox while it is importing data from the Mozilla sync server, it seems that I can end up with major losses of bookmarks, folders, and/or even the whole contents or the bookmarks toolbar. Those end up propagated to the sync server, and then sent back to the other machines in my pool. This "abrupt stop" might happen in a few circumstances, such as
1) Linux shutdown while Firefox is active: for instance, Firefox is minimized on another desktop space and I shutdown forgetting to exit from it. The Linux shutdown most of the time just shuts down in less than 2 seconds (unlike the Win one that leaves Firefox the opportunity to delay the shutdown and end an ongoing sync) 2) Firefox (or something in Firefox, possibly sync) hangs, and I have to kill FF.
If I then restart firefox and save some new bookmark soon enough, this will eventually trigger a resync and often result in bookmarks corruption.
It seems that when it merges data, Sync does this directly into the live set of bookmarks. I noticed that during my testing when my bookmark toolbar folder go deleted, and I restored it from a backup: I saw my bookmarks toolbar slowly come up during a merge, one bookmark at a time, spread over a rather lengthy period. It seems that if this bookmark update process is badly interrupted, the state of the whole set of bookmarks is undefined. I would guess a number of bookmarks and bookmark folders are caught somewhere in buffers and end up never being written back then.
If this were the case, I would suggest that Sync should write to a work file first, close and rename that file as ".new" at successful completion, then commit all modifications by cascaded file renaming (delete of any old backup file / rename of the current bookmark file to the backup one / rename of the new file to the current bookmarks file). This would reduce the dangerous failure time window to an insignificant duration. Furthermore, when Firefox starts, if the new bookmark file is present, this would indicate that the rename cascade didn't complete (and thus could be restarted). This, plus the (presumably independent mechanism) keeping of X max_backups should hopefully insure against bookmarks corruption and loss.
I agree that the fact only a few users of Sync complain (bitterly) about bookmarks losses and corruption is a subject for some thinking. OTOH, it also seems that those who complain rely heavily on their bookmarks in their day to day work (I'm definitely one of those). As a matter of consequence, - they are the most likely to add new bookmarks and modify their bookmark files in various ways, all along the day, and thus, to trigger numerous merge operations (presumably on several machines), - they are the most likely to maintain rather big bookmark files (my backups are something like a 3.5MB .json file, that is, when Sync doesn't trim them down severely ;-) ) - and those folks are also the most likely to notice bookmark losses, since they use them so much (unfortunately, handling very large bookmark trees also means that the loss of some large branch inside a folder might be unnoticed for days or weeks).
Finally, until this nasty problem is fixed forever, I miss any diagnostic / repair tool that would allow me to - easily navigate the bookmark files, either the "live" one or the backup files, - show some bookmarks statistics (number of bookmarks, number of folders) - compare (diff) bookmark trees and/or subtrees, - read-only navigate the bookmarks backups (to locate the "best" backup to restart from) - possibly allow the side-by-side display of two bookmarks files with the ability to move bookmarks and/or folders from one side to the other, so that some recent bookmarks could be recovered and moved to an older bookmarks file to create a new one with minimal losses.
If anyone had anything close to this in their toolbox, it could be invaluable tool to understand (and limit) losses in the interim period...
On Wednesday, February 23, 2011 10:38:32 PM UTC+1, Richard Newman wrote:
> Have you actually examined your bookmarks to see whether you've suffered > any duplication or reordering,
It used to reorder my bookmarks. Now I'm not sure if it's not reordering or if it's not syncing.
> or are you just alarmed by the error message?
The error message comes up *every* time. If I click it away (via X on left), it comes back the next time I load Firefox. If I click "Sync Now", the error comes back after a few seconds. I consider that very concerning.
> If I then restart firefox and save some new bookmark soon enough, this will eventually trigger a resync and often result in bookmarks corruption.
I think you might be confusing cause and effect, here.
When you start Firefox, it will typically sync after a short interval, and periodically thereafter. Modifying your bookmarks prior to a sync (even a first sync) should not be a problem.
That said, I'd be very happy if you could provide clear, reproducible steps to repro this — starting with a fresh profile, instructions to get a failure that we can study.
> It seems that when it merges data, Sync does this directly into the live set of bookmarks.
Correct.
> I noticed that during my testing when my bookmark toolbar folder go deleted, and I restored it from a backup: I saw my bookmarks toolbar slowly come up during a merge, one bookmark at a time, spread over a rather lengthy period.
If by "restored it from a backup" you mean "I loaded my bookmarks from a file", then that's a regular Firefox operation, and not much to do with Sync. You say "merge", though, which implies you're treating a sync as a restore.
> It seems that if this bookmark update process is badly interrupted, the state of the whole set of bookmarks is undefined. I would guess a number of bookmarks and bookmark folders are caught somewhere in buffers and end up never being written back then.
That depends what you mean by "buffers". The Firefox bookmark store is a transactional relational database: sqlite.
Sync works directly with Firefox's Places subsystem. When it pulls down a stream of records, Sync decrypts them and applies them by calling Places API calls. "Applying" them means attempting to find an equivalent local record to update, or creating one.
If you kill Firefox during this process, you'll interrupt the additions to the database, and Sync won't have updated its last sync time to cover the records that were being applied at time of death. That means it'll start the sync over from the last tracked point, which is fine — records that are applied twice will just be reconciled.
If you kill Firefox mid-way through a database write, it's conceivable that database corruption could occur. We can't do any better than the guarantees offered by sqlite. This should be very obvious, though; you'll see console and Sync log messages about a corrupt Places DB.
The only time this process is a little different is if you reset the client and replace with the server data. In that case we drop the Places database first, so you always get a clean slate.
It's never true that interrupting Sync results in an undefined set of bookmarks, unless you catch it in the reset phase of "reset from server" — each individual change (such as a change of title) is delegated to Places and occurs as an individual database operation. After all, every time you go offline, or Sync encounters a problem, that's an interrupted sync.
> If this were the case, I would suggest that Sync should write to a work file first, close and rename that file as ".new" at successful completion, then commit all modifications by cascaded file renaming (delete of any old backup file / rename of the current bookmark file to the backup one / rename of the new file to the current bookmarks file). This would reduce the dangerous failure time window to an insignificant duration. Furthermore, when Firefox starts, if the new bookmark file is present, this would indicate that the rename cascade didn't complete (and thus could be restarted).
I'm afraid that's not really feasible. The whole point of Sync is to do incremental updates of changes. We make a backup of your bookmarks prior to each sync, and then we apply changes to your bookmarks in a streaming fashion.
Bookmarks are not stored as files which we can manipulate. The backup system solves this problem quite neatly.
> Finally, until this nasty problem is fixed forever, I miss any diagnostic / repair tool that would allow me to > - easily navigate the bookmark files, either the "live" one or the backup files, > - show some bookmarks statistics (number of bookmarks, number of folders) > - compare (diff) bookmark trees and/or subtrees, > - read-only navigate the bookmarks backups (to locate the "best" backup to restart from) > - possibly allow the side-by-side display of two bookmarks files with the ability to move bookmarks and/or folders from one side to the other, so that some recent bookmarks could be recovered and moved to an older bookmarks file to create a new one with minimal losses.
> If anyone had anything close to this in their toolbox, it could be invaluable tool to understand (and limit) losses in the interim period...
> Most likely you have a single bad record (e.g., a blank password) from an earlier version of Firefox that's failing to be applied.
> I went through all my passwords (Options / Security / Saved Passwords), and deleted any "weird" (or blank) passwords. It didn't fix the error.
Did you check on your other machines, and then sync those machines? Cleaning the machine to which you're trying to sync won't help; you need to make sure the record isn't on the server.
There are also fields that you can't necessarily see, so you're unlikely to be able to fix them. There'll be an entry in the sync log.
> If you post your about:sync-log (or, even better, file a bug with it attached), we can take care of it.
> Done. I filed bug 637258.
I moved it to the right place and left an action item for you.
>> Is it the case that my bookmarks, history, passwords, and/or forms database(s) is/are broken beyond repair by sync? If no, how can I fix my data?
> They shouldn't be.
> Then why does sync bring up this error dialog ever time?
"Throws an error" and "broken beyond repair" are two very different things!
In particular, Sync 1.6.3 and up will skip over broken records, allowing sync to complete, and then throw the error at the end to make sure you know there was a problem. Your data might be syncing fine, but for a few bad records…
On Monday, February 28, 2011 2:59:02 PM UTC+1, Richard Newman wrote:
> Did you check on your other machines, and then sync those machines? > Cleaning the machine to which you're trying to sync won't help; you need to > make sure the record isn't on the server.
I cleansed the machine *from* which I synced, of course (Reset Sync / Replace all data on *other* machines).
I moved it to the right place and left an action item for you.
On Monday, February 28, 2011 2:38:57 PM UTC+1, Richard Newman wrote:
> The only time this process is a little different is if you reset the client > and replace with the server data. In that case we drop the Places database > first, so you always get a clean slate.
So the fix for bug 637258<http://bugzilla.mozilla.org/show_bug.cgi?id=637258>might be to a Reset Sync and *replace the local data with the server data*? (And accept that the server data might be a bit outdated (if the error means that "something" wasn't being synced).
On Feb 28, 11:05 am, Peter Lairo <pe...@lairo.com> wrote:
> On Monday, February 28, 2011 2:38:57 PM UTC+1, Richard Newman wrote:
> > The only time this process is a little different is if you reset the client
> > and replace with the server data. In that case we drop the Places database
> > first, so you always get a clean slate.
> So the fix for bug 637258<http://bugzilla.mozilla.org/show_bug.cgi?id=637258>might be to a Reset Sync and
> *replace the local data with the server data*? (And accept that the server
> data might be a bit outdated (if the error means that "something" wasn't
> being synced).
That's the workaround. The fix is bug 629463 (better deal with busted
records)
>> I noticed that during my testing when my bookmark toolbar folder >> go deleted, and I restored it from a backup: I saw my bookmarks >> toolbar slowly come up during a merge, one bookmark at a time, >> spread over a rather lengthy period. > If by "restored it from a backup" you mean "I loaded my > bookmarks from a file", then that's a regular Firefox operation, and not > much to do with Sync. You say "merge", though, which implies you're > treating a sync as a restore.
Sorry, what I wrote was very confusing.
I ended up with a garbled bookmark file, with a good third of the bookmarks definitly lost, including the bookmarks toolbar. Sync promptly pushed it everywhere, so all my machine ended up with an empty toolbar.
Then I restored through the most recent valid copy I had on all machines, and pushed the restored bookmarks through a "reset"/"Replace other machines".
A few minutes later, I saw the bookmarks toolbar slowly reappearing during the next merge. It's at this point that I realized that the restoring of the bookmarks was done "live" in the firefox bookmarks file.
What I recreated is this:
I can recreate (not everytime, but I could at least once) a case when - I crash firefox during a significant merge, - I restart firefox, - the local bookmarks are garbled, - At the next sync, the garbled file is uploaded to the sync server, - a few hours later, the bookmarks on all my machines are garbled the same way.
About the fact of creating a new bookmark before the initial sync: I mentioned that without being sure whether it was meaningful, but OTOH, I noticed that it seems bookmarks backups are only created on days when I actually create bookmarks (or modify the bookmarks on one of the sync'd machines). If this is so, the code path followed is not exactly the same -- and this was the scenario when I successfully corrupted my bookmarks file.
> I'm afraid that's not really feasible. The whole point of Sync is to > do incremental updates of changes. We make a backup of your > bookmarks prior to each sync, and then we apply changes to your > bookmarks in a streaming fashion.
And I'm afraid that's part of the problem: If a crash or abrupt shutdown occurs during this operation of streaming update, - the local bookmarks might end up badly screwed up, - the user will not necessarily notice it, - Next time FF is restarted, the crashed bookmark file will eventually be uploaded to the sync server and then propagated to all the machines of the gang.
Now as I said, this is just a guess. But I have been able to reproduce experimentally.
