[AddOn] file-backups - "TiddlyWiki File Save and Backup" V0.3.5 small bugfix release

[PMario]

Hi Folks,

I did just release the new Version 0.3.5 of the file-backups AddOn for firefox. It's a small bugfix version.

Fixed

• External links shouldn't have duplicate detection ... issue #16

Learn more at: https://github.com/pmario/file-backups/releases/tag/V0.3.5

If you have any problems, please let me know.

The goal was, to release a signed-beta and self-publish it at github. ... But it seems I only can have one thing. Self-publishing or Mozilla Store :(.

I will try to find out, how to create a "beta-channel" for those who want to help me testing.

have fun!
mario

[PMario]

Original intro posts with much more info:

https://groups.google.com/forum/#!searchin/tiddlywiki/file-backups%7Csort:date/tiddlywiki/iAx4wT_s9d0/XNOzwZx7AQAJ
https://groups.google.com/forum/#!searchin/tiddlywiki/file-backups|sort:date/tiddlywiki/h0k7z62N79Y/uhfU32RXBQAJ

-m

[Alfonso Arciniega]

Thanks, Mario,

The https://github.com/pmario/file-backups/releases/tag/V0.3.5 redirects you to the German version in FF addons, not the English one. You may want to change a part of the link from "de" to "en-US".

Regards,

Alfonso

[PMario]

On Thursday, March 8, 2018 at 12:10:34 AM UTC+1, Alfonso Arciniega wrote:

The https://github.com/pmario/file-backups/releases/tag/V0.3.5 redirects you to the German version in FF addons, not the English one. You may want to change a part of the link from "de" to "en-US".

Thx for the info. I didn't see a big difference, but I changed it :)

-m

[Zyb]

Hey Mario,

With the new version of the add-on (I presume because nothing else has changed since yesterday), my TWs won’t save and won’t backup anymore. Windows 7, FF 58.0.2. Hope you can fix the fix quickly. :)

All best,
Zyb

[PMario]

On Thursday, March 8, 2018 at 4:08:59 PM UTC+1, Zyb wrote:

With the new version of the add-on (I presume because nothing else has changed since yesterday), my TWs won’t save and won’t backup anymore. Windows 7, FF 58.0.2. Hope you can fix the fix quickly. :)

Hi,
Did you restart your browser?
Which TW version 2 or 5?
What did you do?
-m

[Zyb]

What do you mean ‘restart my browser’? I started it up this morning, did some work (TW5) and then noticed that I couldn’t save it. Went to the forum and immediately saw your update notice (thanks for posting those, btw). My addon settings are on automatic update, obviously.

 Are you saying that the addon was updated not on browser startup but at an arbitrary point in between? How could I have even noticed that? Why would it render the wikis unsavable until restart (which means losing the work already done)? That’s not something that should happen, ever, I think.

Best,
Zyb

[Ton Gerner]

Hi Zyb, Mario,

Yesterday I did notice the same.
At a certain moment I could not save anymore (normally auto saving but got the download message now), saw the announcement of the new version of the file-backups add-on, checked my Firefox and saw the add-on had been updated autonmatically.
After a restart of my browser everything worked again.

But, lateron I noticed a few 'not saved' tiddlers in my TW: for sure I edited some tiddlers but the changes had not been saved!?!  Strange.

Cheers,

Ton

[Thomas Elmiger]

Confirming: After the update a browser restart is necessary, otherwise the ability to save is lost. (Mac, FF Developer Edition)

Maybe the plugin could spit out a warning?

This is why I switched off auto-updates in the plugin settings some time ago. Nevertheless I forgot to restart and had to export a JSON file from the advanced search for the last changed tiddlers ...

Thank you anyway for this great tool!
Thomas

[Zyb]

Yeah, I switched off auto-updates now ... but ... I mean, I accept that a browser restart is necessary to load the new version of an addon -- but why does the mere update process render the previous version non-functional until the restart? Is that indeed a usual thing happening? I can’t remember having any such problem with another FF addon before.

[PMario]

On Thursday, March 8, 2018 at 8:08:51 PM UTC+1, Thomas Elmiger wrote:

Confirming: After the update a browser restart is necessary, otherwise the ability to save is lost. (Mac, FF Developer Edition)

I don't think, it's a browser restart. IMO reloading the tab is enough. ... But I have to have a closer look, if we can detect, a plugin update, while the browser is running. ..
 

Maybe the plugin could spit out a warning?

I don't know, if that's possible. As I wrote, I'll have a look, if we can detect it.
Every mechanism, that allows plugins to directly interact with the user can also be used to add-spam them. So the extension API is heavily restricted. We can't do what ever we want.
 

This is why I switched off auto-updates in the plugin settings some time ago. Nevertheless I forgot to restart and had to export a JSON file from the advanced search for the last changed tiddlers ...

The problem is, that testing plugin updates, can only done if you update the plugin. ... And the mechanism, how mozilla store updates happen is not always clear.

The way I'd like, that updates are done is:

 - Give the developer the possibility to notify the user, that there is a new update
 - Then the user should decide, when the new addOn is installed.
 - Everyone would be happy.

 - BUT this notification mechanism can be used to spam users with adds .... So ?!

-mario

[PMario]

On Thursday, March 8, 2018 at 6:42:24 PM UTC+1, Zyb wrote:

 Are you saying that the addon was updated not on browser startup but at an arbitrary point in between?

I think, it's a browser background task. ... but there is no description how updates happen. .. At least I haven't found one ...
 

How could I have even noticed that?

I will have a look, if there is a possibility to notify users.
 

Why would it render the wikis unsavable until restart (which means losing the work already done)?

As I wrote in the other post, it may be needed to reload the tab after a plugin update. .. I'll have a look, if a "running tab" can detect plugin updates. ATM I don't know, if there is any signal from the browser and if we can reliably detect it. 
 

That’s not something that should happen, ever, I think.

You are right. ... But there are always edge cases, a developer didn't think about. ..

Testing is hard. As I wrote, to test updates you have to make updates.

I can test the plugin with windows 10 and an ubuntu system.
I don't have the possibility to test with macOS. ... So I'm 100% dependent on user feedback. .. If no feedback happens, .. nothing happens.

Mozilla is switching off the "beta channel". That's what I had to find out, as I started to have a look, how I can use it.

As I wrote in my first post, the plugin should have not been listed and it should have not been auto-updated.

At the moment, I try to find out, how to create a signed beta-version, that can be published at github, without the mozilla store auto-update thing.
The problem then is. ... What happens if users switch from beta to stable ... That's a new mechanism, where things can go wrong. ... AND we have to test it.

-mario

[Zyb]

I have the greatest respect for your work, Mario, and am eternally greatful for your addon. If my posts sounded a little testy, I apologize, but I am quite angry, actually. Not at you personally of course, nor at Jeremy or anyone, but at a situation where I sit here replicating hours of lost work in dozens of tiddlers and thinking about the depressing fact that, in 2018 for crying out loud, the one piece of software that enriched my personal and professional life more than any other tool and which I think is the greatest thing since sliced bread, fails to save my work when I need it saved. Yesterday, I thought -- as I regularly do ever since the FF update last year -- about ditching TW and reverting to the half dozen or so applications which I used to use for stuff that I now do in TW. Because they save my work without, you know, an addon with all the crappy dependencies and unknowns and (your phrasing is ironic really) “edge cases”. Sure, that would feel as bad as reverting to ink and paper, or to chiseling letters on marble slabs, but hey -- my data is safer than in TW. I don’t know what you guys use TW for, but I do professional work with it, precious work not only in terms of invested time, I think with it, I produce scholarship. And TW is an integral part of why I love my job, because it elevates me in my professionalism. But that also means I depend on TW, I need it working. This is why it totally horrifies me when the one single thing the only function of which it is to keep my work safe -- fails. And it totally deflates my faith in TW that there even is a situation where a third party addon has to fulfil that function (yes, I know, this situation has been around since TW2 -- and if I could go back in time knowing how fundamentally broken TW’s dependency on a browser is when it needs a hack to be able to save it, I’d delete the day I stumbled upon TW and had my mind blown just to save myself the heartache of ditching it many years later). You know, I used to recommend TW to colleagues and friends, I created a customized TW2 empty.html to make it easier for them -- not only don’t I do that anymore, I cringe with awkwardness at my own enthusiasm. They all stopped using it, “Zyb, this thing you gave me doesn’t work any more, don’t know what happened.” and I stopped trying to fix it for them, I shrug and say “Yeah, I’m sorry.”, because TW’s problem seems to be unfixable.

Phew. Sorry for the rant. I guess I needed that if none of you guys do. Yes, I’m bitter. Please, also, if you’re tempted: refrain from giving me ‘good advice’. I did look into it: TiddlyDesktop is unusable and don’t even get me started on that server thingy.

Zyb

[PMario]

Hi Zyb,

I'm sorry that you lost work.

On Friday, March 9, 2018 at 11:34:33 AM UTC+1, Zyb wrote:
...

Because they save my work without, you know, an addon with all the crappy dependencies and unknowns and (your phrasing is ironic really) “edge cases”.

I didn't mean it ironic. I meant it as in: https://en.wikipedia.org/wiki/Edge_case 

You wrote:

That’s not something that should happen, ever, I think.

 

I wrote:

You are right. ... But there are always edge cases, a developer didn't think about. ..

I just wanted to say, that I didn't think about this case. ... and I didn't test it. ...
There is no code in the plugin, to deal with this case.

Now that I know, there is a critical problem, I can have a closer look and probably fix it.

-mario

[@TiddlyTweeter]

Ciao Zyb & PMario

Given that TiddlyWiki runs in a web browser and there are huge chains of complication that can lead to issues, IMO, especially for local TW, one should always also run redundant backup too. The more valuable the data the more important that is. Its not difficult.

FWIW in my practice I like TW options for backups/saving like PMario made. But NOT specifically for my final backup--only to give me a standard place to find the latest revisions.

I backup every changed TiddlyWiki via specialist backup software that auto-detects changes.

PMario's problem is he's writing a piece of software into a complex environment to bridge between a TiddlyWiki "page" and saving in a fashion that helps users. Sometimes that bridge fails.

On your, Zyb, utter frustration with the complexities and vast numbers of variations on TW, and your colleagues antipathy to getting into complications, I am in sympathy with you.

IMO, for a proper, reliable, standard, out-of-the-box usage for TW, where you don't have to deal with platforms & 17 configurations, or worry about backup, its probably gonna happen eventually in Cloud Computing.

Best wishes
Josiah

[Mark S.]

I can feel your pain, since I've lost work from time to time.

Pretty much all open source is really in some kind of beta. Even professional tools, like Java, make you agree when you use their product that you won't use it for anything mission-critical. The reason it's always in beta is that developers working for free don't have a team of quality control experts working hard to break the software.

So one thing you can do is to put a safety net under your work. Looking around, it appears that DropBox will keep a version history on all files that it is synching for 30 days on free accounts. So in your case, you could have recovered your file out of the DB version history. GoogleDrive looks to also have versions for some unspecified time (haven't tried it with GD though).

People will put up with all sorts of complexity once they see the power of it. Purchasing, setting up, securing, and initializing  a computer (or device) isn't easy, yet there are about 2 billion working computers in the world and 7 billion mobile devices.

Good luck!
-- Mark

[Thomas Elmiger]

Hey people,

I feel with Zyb too, data loss is always a disaster or a disappointment at least. BUT in this case it was probably not necessary!!!

As described in my last post, there is a workaround for even this scenario when Mario‘s save and backup (it covers that too!) refuses to save: export JSON.

So: Know your tools, backup and you will be as safe as possible in these days.

All the best,
Thomas

[Zyb]

Thanks for the sympathy. ;-) To the people suggesting backups: You’re aware, right? that we are talking about unsaved (and unsavable), ‘dirty’ files here, for which the backup process (simultaneous to the save) has already failed and cannot be fixed without restarting the browser, thus closing the dirty TW file, thus resulting in the data loss I’m talking about? That is why Dropbox won’t help (I suspect), either, because it only uploads edited files, but a dirty TW will not be detected as edited. I have plenty of backups right up until FF updated Mario’s addon and rendered it nonfunctional in the process.

Before I even read up on what ‘export JSON’ means, Thomas: Does that apply to a dirty, unsavable TW file? (Btw, I use your excellent TextStretch/Footnotes on one of my pages, thanks for that!)

All best,
Zyb

[Jed Carty]

Yes, the solution Thomas suggested is one of the last result methods that always works available when you know that you can't save your file in another way but have the wiki open.

[wjam]

export to json is done using advanced search

goto filter tab,

enter the filtertext below

[!is[system]!sort[modified]limit[25]]

Check the list of tiddlers.

adjust the number "25" to make sure you found all your recently modified tiddlers

Then press the bucket with the up arrow [_^_] which appears on the right

This will ask for a location to store the tiddlers into a file called tiddler.json on your local drive. Press save.

the tiddlers.json file can be imported (tools in sidebar) or drag and drop file on the top line of the story river of anather tw . (Press Esc if you change your mind ;-). You can (de)select specific tiddlers. Then press import.

Similar In the viewtoolbar of a tiddler you can export the current tiddler.

Njoy
Kr wjam

[@TiddlyTweeter]

Ciao Zyb

To the people suggesting backups: You’re aware, right? that we are talking about unsaved (and unsavable), ‘dirty’ files here, for which the backup process (simultaneous to the save) has already failed

ABSOLUTELY. That is why I run redundant backup that checks for changes every 5 minutes. External backup is simply the reliable method with the least hassle.

The other writers who are talking about JSON scenarios are correct, but overlooking breakdown scenarios.

Best wishes
Josiah

[Mark S.]

I'm thinking this is so useful, it should be added to the list of saving techniques under "Emergency Export Save". I hope you don't mind if I borrow your instructions...

Thanks!
Mark

[Mark S.]

Zyb is saying (I think) that the failure occurred silently. That is, it didn't announce that it wasn't saving. So your method would fail just like the DropBox method, because no new edits were being saved to disk.

Maybe TW needs to make a bigger fuss when a save fails so that the user knows to stop working. It already does this when running on node.js.

-- Mark

[@TiddlyTweeter]

Mark,

That is one of the reasons why I use external backup.

It ALERTS me on a real save of new. I get a small notification on screen indicating a new save was found and backed. That way I know if there is a problem when I thought I saved when I don't see that message. Most auto-backup tools can do that.

Josiah

[Mark S.]

But you said it runs every 5 minutes. So you kind of have to be aware that you haven't seen an announcement in the last 5 minutes?

Better to just get a warning from TW that something is amiss.

-- Mark

[TonyM]

Mark,

Yes do that. Actually a list of methods would not hurt under an emergency save document.

Due to FireFox Plugins occurring in the background has lost a lot of work for me in the past, especially with TWC. I now maintain autosave on every tiddler edit. I achieve this by using multiple structured Wikis so that the save time is not interrupting.

The problem of loosing work is made worse by my abhorrence of re-work and with tiddlywiki running but not saving in your face you can be looking at a zombie, it has all your work in it but you can't save it. Sometimes I have taken screen shots of tiddlers content  in the recent list as a last resort, I have more often copy and pasted the content of all edited tiddlers into a text field for recreation.

Not withstanding the above every platform has risks, In many ways it is much easier to set up counter measures with tiddlywiki than most other solutions.

Regards

Tony

[Zyb]

Mark: Yes, that is what I’m saying. And it’s worse: I let TW save on tiddler edit and the little popup box in the upper right corner that says ‘saved wiki’ does appear to signal the save when the tiddler is changed from draft to normal. So TW is lying about its status in the user notification -- the only signal for the unsaved status is the ‘save changes’ icon in the menu which doesn’t change colour.

Of course it would help if TW detected a failed save attempt to spit out a proper warning.

[Zyb]

Thanks, wjam, for this method, just tested it and it works nicely! That would have saved my ass two days ago ...

[@TiddlyTweeter]

Mark S. wrote:

But you said it runs every 5 minutes. So you kind of have to be aware that you haven't seen an announcement in the last 5 minutes?

No. Not quite. Only be aware that I "saved" a TW and didn't see a message pop up a few minutes later (i.e. the 5-minute backup only runs when there is change). Its just another one of numerous messages the OS pops up I keep an eye on.
 

Better to just get a warning from TW that something is amiss.

But THAT is the point. It isn't in this case :-).

... I guess behind my thinking is that (1) I trust TW; (2) I don't ultimately trust handing things over to the browser--mainly because of the complexities involved and the variations of platform etc. I'm very impressed with what people like PMario squeeze from it. But I don't want them lumbered with having to guarantee success 24/7 in a fluxing development context.

Far better to be REDUNDANTLY SAFE and use normal backup too.

Best wishes
Josiah 

[Jon]

Hi, 

Just wanted to thank PMario again for this backup system which I've used flawlessly since it was introduced.

This morning, I went to open my tiddlywiki and was horrified to discover the file had simply disappeared! (and not in deleted items)

I was using it yesterday so something must have happened when closing the browser which I had to do several times as it was freezing (using the task manager).

I do make my own backups, but the last one was over a week ago. 

Fortunately, I was able to pull out a very recent backup from Mario's automated backups and now everything is restored.

So thanks again

Jon

[PMario]

Hi Jon

On Sunday, September 2, 2018 at 8:58:50 AM UTC+2, Jon wrote:

....

I do make my own backups, but the last one was over a week ago. 

Yea, from time to time it's good to make backups from the backups :)

 

Fortunately, I was able to pull out a very recent backup from Mario's automated backups and now everything is restored.

I'm happy, that it worked out for you!

 

So thanks again

Jon

You are very welcome. Thank you for your feedback!

have fun!

mario