Another Alternate Backup Scheme

[kthejoker]

I noticed while using a bit of trial and error to make my new
TiddlyWiki's stylesheet my own that I ended up with 16 (!) megabytes of
Wiki backups in my directory. That seemed a bit excessive, but as
Jeremy pointed out in July, having only one backup file is just another
extreme.

I propose a compromise. This is my own hack, and I haven't studied the
TiddlyWiki code all that hard, so there may be a slicker implementation
than my own.

Step 1) I added two lines to the end of the config.options array of JS
vars:

txtMaxBackups: "10",
txtCurrentBackup: "1"

Step 2) I altered the getBackupPath JS function as follows:

function getBackupPath(localPath)
{
var backSlash = true;
var dirPathPos = localPath.lastIndexOf("\\");
if(dirPathPos == -1)
{
dirPathPos = localPath.lastIndexOf("/");
backSlash = false;
}
var backupFolder = config.options.txtBackupFolder;
if(!backupFolder || backupFolder == "")
backupFolder = ".";
var backupPath = localPath.substr(0,dirPathPos) + (backSlash ? "\\" :
"/") + backupFolder + localPath.substr(dirPathPos);
backupNum = config.options["txtCurrentBackup"];
if (backupNum++>config.options["txtMaxBackups"]) backupNum=1;
backupPath = backupPath.substr(0,backupPath.lastIndexOf(".")) + "-" +
backupNum + ".html";
config.options["txtCurrentBackup"]=backupNum;
saveOptionCookie("txtCurrentBackup");
return backupPath;
}

Which basically replaces the date backup system with a system that
incorporates a rotating set of backup files, limited by the
txtMaxBackups config.option variable set in Step 1.

Step 3) Finally, as a cosmetic bonus, I added the maximum backups
option to the AdvancedOptions tiddler for easy manipulation:

Maximum number of backups: <<option txtMaxBackups>>

Anyway, this seems to work for me, so I thought I would share it with
the group. Enjoy.

Kyle Hale
kthejoker at google mail

[Ken Girard]

This is great!
You need to see about setting it up as a plugin so that it works even
after core updates.

There is also another backup alterative listed at:
http://tiddlywikitips.com/#%5B%5BTip%20%2302%3A%20Alternate%20Backup%20method%5D%5D

which leaves you with a single file called 'backup.html'

Ken Girard

[Paul Petterson]

Cool stuff -   One thing I noticed:

 

if (backupNum++>config.options["txtMaxBackups"]) backupNum=1;
 

will increment backupNum after comparing it, so you'll end up with a number 1 greater than your txtMaxBackups setting.
 

if (++backupNum>config.options["txtMaxBackups"]) backupNum=1;
 

will increment before the compare...

 

HTH, & Keep up the great work! :)

Paul

 

On 3/3/06, kthejoker <kthe...@gmail.com> wrote:

I noticed while using a bit of trial and error to make my new
TiddlyWiki's stylesheet my own that I ended up with 16 (!) megabytes of
Wiki backups in my directory. That seemed a bit excessive, but as
Jeremy pointed out in July, having only one backup file is just another
extreme.

I propose a compromise. This is my own hack, and I haven't studied the
TiddlyWiki code all that hard, so there may be a slicker implementation
than my own.

Step 1) I added two lines to the end of the config.options array of JS
vars:

       txtMaxBackups: "10",
       txtCurrentBackup: "1"

Step 2) I altered the getBackupPath JS function as follows:

function getBackupPath(localPath)
{
       var backSlash = true;

       var dirPathPos = localPath.lastIndexOf ("\\");

[Jim Barr]

Ken,

I tried to incorporate my modification (that you linked to above) into
a Plugin, but I got stumped so I never completed it. If some Plugin
Wizard would like to create a Plugin to do EITHER of these
alternatives, that would be great!

-Jim
http://TiddlyWikiTips.com

[kthejoker]

Hey, I didn't even know you could do ++var isntead of var++ in JS.
Learn something new every day.

I was looking at all of the plugin stuff, but I didn't see any of them
trying to edit the actual JS functions that are part of the core. Most
of them instead added new JS functionality (which is great, of course)

As far as I know, there's no way to edit actual JavaScript code using
JavaScript, which makes substituting the "-Num" backup naming style
over the "-datetime" naming style impossible.

Maybe I'm wrong, though, and someone here in the group can correct me.
Please do!

Kyle Hale

[Jim Barr]

This is one area where I believe that the Core TW function should be
analyzed and reworked to provide the capability for extending backup
functionality through Plugins. While I realize that Jeremy is typically
hesitent to make Core TW modifications (and I believe he should be) but
I think this is one area that should be considered for modification.

(Said like a true TW //user// not programmer!) ;-)

Agreements? Disagreements?

-Jim

[kthejoker]

Well, I'd tend to agree, but that's just a personal value statement. I
think the backup system as is is a bit excessive (and definitely puts a
toll on my thumb drive space from time to time.)

And, unlike most of the other core functions, it's the only one that
feels somewhat "arbitrary".

But again, that's just me.

Kyle Hale

[Rich Carrillo]

I agree that this might be a good addition to the core, giving people
another option to manage their backups.

You CAN make it into a plugin, though. Some other plugins "hijack" a
core function and totally redefine it. So you'd be totally overwriting
the core function getBackupPath with your own. It's a bit dangerous
though, since other Plugins might also be hijacking it to add other
functionality.

If you do plan to hijack it, you should retain the old default
behaviour in some way. Maybe if you let the max backup be set to 0 or
-1, it will invoke the default unlimited dated backup behaviour.

-Rich

[kthejoker]

Rich, can you show me a plugin that hijacks a core function? I'd be
glad to do all the programming myself, but having an example would be a
great place to start.

Kyle Hale

[kthejoker]

Actually, nevermind, I found one that did just that.

I have a plugin ready, but I don't have an online TiddlyWiki to post it
to. Anybody care to help me out and post a plugin to a web-ready TW?

Email me at my username at gmail, or I can post it here if that'd help,
though I see bad things can come from Google Group linebreaking.

And to Jim, this plugin will be nice because you can of course set the
maxBackups to 1, thus achieving your one backup goal.

Kyle Hale

[Jell]

Good to see this topic raised again. I added a quick mod to my TW code
so that it keeps one backup a day, which limits the size of the backup
folder. I have recently repeated the hack on TW2. I did this by
modifying the backup code to produce a backup with the format filename
YYYYMMDD.html. (rather than YYYYMMRRHHMM)

The effect is that each time you change something, the system saves a
backup with a filename based on the original filename and today's date.
Each new change overwrites the last one, so only one file is ever
created each day. At the end of the day you are left with a file with
the latest version at close of play that day. The next day it starts a
new file.

Working with Yatwa as a base, I modified the [[SaveChanges
replacement]] Tiddly, so that it reads:

.
.

backupPath = backupPath.substr(0,backupPath.lastIndexOf(".")) + "." +
(new Date()).convertToYYYYMMDD() + ".html";
.
.

and then modified the base code to define a new function:

// Convert a date to UTC YYYYMMDD string format Added by Jell
Date.prototype.convertToYYYYMMDD = function()
{
return(String.zeroPad(this.getFullYear(),4) +
String.zeroPad(this.getMonth()+1,2) +
String.zeroPad(this.getDate(),2));

}

I hope I explained that well enough! I am sure that there are ways of
doing that with a plugin, but I am really a TW user rather than TW
hacker. If someone turns it into a plugin, please let me know!

Jell
TiddlyFiddler

[Daniel Baird]

That's definitely a step toward a better backup arrangement.

I've expounded (or some would say, blathered..) before on what I reckon the perfect backup plugin should be like, I'll post it here:

I'd like a really good backup plugin that lets me put backups into a subdir
and then set a bunch of time scales and a different backup rotation strategy
for each time scale.

For example, I'd probably want:
- backup for every save for the last 30 minutes
- one backup for every 10 minutes (or longer if there was no save during the
10 mins), from 30 minutes ago to about 2 hours ago
- one backup for every hour, from 2 hours ago to maybe 24 hours ago
- one backup for every day, from 24 hours ago to about 2 weeks ago
- one backup for every week, from 2 weeks ago to about 2 months ago
- one backup for every month, from two months ago til forever.

I've only just acquired this HP38G geekulator, but it tells me that if I've
been using TW for a year, I'd have 61 backups, plus the number of saves in
the last 30 mins (maybe another 60..). Nice manageable number hey.. and it
only approaches infinity very slowly, which is an advantage over my present
backup strategy :)

I know there's a few backup plugins out there, but I think none of them do
this kind of thing.. any one else want what I do?

;Daniel

I hope I explained that well enough!  I am sure that there are ways of (TiddlyW;nks! :: Whiteboard Koala :: Blog :: Things That Suck)
[[My webhost uptime is ~ 92%.. if no answer pls call again later!]]

[Jeremy Ruston]

> I've expounded (or some would say, blathered..) before on what I reckon the
> perfect backup plugin should be like, I'll post it here:

After an earlier discussion about this topic I added a hook that
enables plugin writers to adapt the backup file naming convention,
including being able to do things like rotate earlier backup files.
The function to override is getBackupPath(localPath).

As I mentioned back then, one of the rather cautious constraints that
I originally chose for the backup code was to avoid ever *deleting*
files, just to reduce the potential impact of any bugs.

Cheers

Jeremy

--
Jeremy Ruston
mailto:jer...@osmosoft.com
http://www.tiddlywiki.com

[kthejoker]

Well, I'll just my post my entire plugin tiddler's content here. If
Google Groups breaks it, I'll do my best to annotate it.

********************* PLUG IN TIDDLER COMMENCES ******************

/***
|''Name:''|AlternateBackupPlugin|
|''Version:''|1.0.1 (3-Mar-2006)|
|''Source:''||
|''Author:''|KyleHale|
|''Type:''|Plugin|
!Description
Hijacks core backupPath function, replacing the datetime naming system
with a rotating set of backups, whose number is limited by the user.

!Issues/Todos
* Possible to insert an option into the AdvancedOptions Tiddler? That'd
make editing the number of backups that much easier.

!Revision History
* 1.0.1 (3-Mar-2006)
** Wrote code, totally stole [[Simon|SimonBaird]]'s documentation
system

!Code
***/
//{{{

config.options.txtMaxBackups="10";
config.options.txtCurrentBackup="1";

getBackupPath = function(localPath)

{
var backSlash = true;
var dirPathPos = localPath.lastIndexOf("\\");
if(dirPathPos == -1)
{
dirPathPos = localPath.lastIndexOf("/");
backSlash = false;
}
var backupFolder = config.options.txtBackupFolder;
if(!backupFolder || backupFolder == "")
backupFolder = ".";
var backupPath = localPath.substr(0,dirPathPos) + (backSlash ? "\\" :
"/") + backupFolder + localPath.substr(dirPathPos);
backupNum = config.options["txtCurrentBackup"];

backupNum++;
if (backupNum>config.options["txtMaxBackups"]) {
backupNum=1;

}
backupPath = backupPath.substr(0,backupPath.lastIndexOf(".")) + "." +

backupNum + ".html";
config.options["txtCurrentBackup"]=backupNum;
saveOptionCookie("txtCurrentBackup");
return backupPath;
}

//}}}

[Daniel Baird]

Hmm.. I like the "no delete" policy..  I think it's possible to keep that rule and just use overwriting to implement a variable-interval backup like I want.

A first pass at a solution:
/backup/recent/ could have backup1, backup2 etc starting from 1 each day
/backup/daily/ could have backupMonday.html, backupTuesday.html and while it's Monday, keep replacing backupMonday.html whenever an autosave is done
/backup/weekly/ could have week1.html, week2.html etc and while it's the first week of the month, keep overwriting week1.html with the autosave
/backup/monthly/ could have Jan.html, Feb.html etc, and while it's January, keep overwriting Jan.html

I guess the problem with this would be that every save is making *four* backups (five  if you have a /backup/hourly, six if you have a /backup/yearly ..).

Hmm.. actually it would be better if the Monday.html wasn't a backup of the *last* thing you did on Monday, but the first -- that way, if it's Monday, autosave just needs to check Monday.html to see if it was created today, and if it was, leave it alone (and if it wasn't, save a backup there).  Then most saves each day are just saving a single file.

Wotcha all think?


;Daniel

On 06/03/06, Jeremy Ruston <jeremy...@gmail.com> wrote:

> I've expounded (or some would say, blathered..) before on what I reckon the
> perfect backup plugin should be like, I'll post it here:

After an earlier discussion about this topic I added a hook that
enables plugin writers to adapt the backup file naming convention,
including being able to do things like rotate earlier backup files.
The function to override is getBackupPath(localPath).

As I mentioned back then, one of the rather cautious constraints that
I originally chose for the backup code was to avoid ever *deleting*

files, just to reduce the potential impact of any bugs. (TiddlyW;nks! :: Whiteboard Koala :: Blog :: Things That Suck)

[Simon Baird]

On 06/03/06, Daniel Baird <danie...@gmail.com> wrote:

Wotcha all think?
[ ... ultimate backup method snipped ... ]

 

 

I think hurry up and write the darn plugin...  You spent more time describing it than it would take to code...  ;) ;)

 

 

 

Simon.

[Daniel Baird]

On 06/03/06, Simon Baird <simon...@gmail.com> wrote:

 

I think hurry up and write the darn plugin...  You spent more time describing it than it would take to code...  ;) ;)

That's true, but emailing *looks* like working, whereas swapping to and from a tiddlywiki browser window looks like goofing off.

;D

--
Daniel Baird
http://danielbaird.com (TiddlyW;nks! :: Whiteboard Koala :: Blog :: Things That Suck)

[fishzle]

That sounds pretty complicated.

Surely, surely, the better approach would be to store diffs between
each TW save? Then you would look at the version for March 1, and you
would see that you added "Buy potatoes, buy purified water", and you
would see that you modified your entry for "Have big party", and you
would see that you deleted "4 bottles of homebrew" from your
"DrinksCabinet" tiddler.

Diffs are the way to go. You can go back as far as you want without
using up lots of disk space. Once in a while, if you wanted to. You
would delete the older revs - perhaps by deleting the file, and then
doing a "save changes". Much like how you do an upgrade to TW.

Changing TW so that it supports RCS/SCCS style operations might be a
bit of work though. (As a single user tool, it doesn't need the complex
locking/sync stuff in CVS/Subversion). But perhaps a small subset...

Francis

[Daniel Baird]

Diffs would be nice, and you're right, it's usually the changes that I actually want to see when I'm looking at a backup.

However, although the algorithm sounds detailed (i should have posted it to TiddlyWikiDev anyway), I don't think it'd be that complicated to use -- if you've deleted something but you know it was there on Friday, go and open /backups/daily/friday.html and it will be there.  In fact there doesn't even need to be subfolders called  daily, weekly etc, the backup files just need unique names.

The advantage over diffs stored in the core TW is that if you completely screw up your TW so it's unusable, or your test plugin accidentally delete the contents of the store div, you have backups in other files.

I guess the ideal arrangement (for me) would be a tool that compared the various backup files and told me what had changed.. mm, sweet.  I guess a server backend might be nice for diffs too.

;D

--
Daniel Baird
http://danielbaird.com (TiddlyW;nks! :: Whiteboard Koala :: Blog :: Things That Suck)

[Jeremy Ruston]

> Surely, surely, the better approach would be to store diffs between
> each TW save? Then you would look at the version for March 1, and you
> would see that you added "Buy potatoes, buy purified water", and you
> would see that you modified your entry for "Have big party", and you
> would see that you deleted "4 bottles of homebrew" from your
> "DrinksCabinet" tiddler.

I'd thought about one quite nice-seeming way to do incremental backups...

The idea would be that each time a tiddler gets deleted or modified,
to write a file consisting just of the text of that tiddler. The
filename would consist of the name of the tiddler with a timestamp
appended. (Clearly it could be extended to backup multiple tiddlers
into a single file, which might help get around the horrendous
inefficiencies that most operating systems have with very small
files).

The issue that prevented me from following up this approach is that
it's (as far as I can tell) impossible to determine at run time what
character set is legal in filenames. In other words, there could
easily be characters in the name of the tiddler that cannot be safely
used in filenames.

I can think of a few hacks to get around the problem but they tend to
impact usability (eg by leading to filenames that are not easy to
read) or not be particularly reliable (eg hard coding ranges of 'safe'
characters).

None of this would matter, of course, in the old-fashioned ASCII world
but now we've got TiddlyWiki users working in lots of non-Latin
languages, and it's not acceptable for the core to implement something
that's not pretty much universal.

Cheers

Jeremy.

[kthejoker]

As an addendum, the first two lines of the actual code should be
changed to say:

if (!config.options.txtMaxBackups) config.options.txtMaxBackups="10";
if (!config.options.txtCurrentBackup)
config.options.txtCurrentBackup="1";

Otherwise these two variables just reset with every refresh of the
Wiki. Should've seen that coming.