Wiki data safety

[TPJ]

Hello!

I've been using WikidPad recently and I have some ideas to enhance it.
I don't want to write about them here, though. My primary concern is
data safety. I have experienced problems with data exporting many
times. Perhaps it's due to the fact I use the latest development
version, and perharps it's due to some problems with my operating
system (I use WikidPad on both Ubuntu and Arch, and I think I had
problems on Ubuntu only). Nevertheless, I think I have some solution
to such a problems, which might arise at any time and in any place.

I'm thinking about something that could be named a "safe mode of
work". The idea is simple: two different databases should be used for
managing our wiki data. Currently only one database is in use, and
thus when it fails (for some reasons), out data might be damaged or
lost. (By the way, that's exactly my case.) In this "safe mode" the
data would be stored in two different databases. It means that each
save operation would have to store data in two databases, instead of
just one. Moreover, after such a save of data its integrity would be
checked (both databases should be equivalent to each other).

Of course, each load should also perform such a data integrity check.

If such a two different databases would be used, the second
(additional, extra) one should be in a different format. Using the
same database engine would be a mistake (since the engine could be
broken itself). I think that the second database could simply be a
bunch of *.wiki files, stored somewhere.

I can think of two different critical situation that might happen:

1) We can achieve data from both databases, but one of the databases
gives us wrong data. In this case user can simply choose to use the
right data (both data are presented to user, and he has to made the
choice).

Of course, the user should be informed that there is some problem with
one of the databases, thus working in the "safe mode" is no longer
safe.

2) We can't achieve any data from one of the databases, so we have to
use data from the second one. Just as in the first case, the user
should be informed about it.

Of course, working in this proposed "safe mode" could have some impact
on performance, so it should be optional. In another words: let the
user decides, if he wants to work in this "safe mode" or not.


All right, I have described my idea, so now a few words on possible
implementation. It doesn't seem to be complicated (because of using
already implemented features), so I think it could be easily done.
Some GUI work might be required to implement new dialogs (used when
user has to make a decision of which data to use).


The final question is: what do you (WikidPad developers and users)
think about it?


Best regards,

Tomek

[Christian Ziemski]

On 03.09.2008 10:09 TPJ wrote:

> I've been using WikidPad recently and I have some ideas to enhance it.
> I don't want to write about them here, though. My primary concern is
> data safety. I have experienced problems with data exporting many
> times. Perhaps it's due to the fact I use the latest development
> version, and perharps it's due to some problems with my operating
> system (I use WikidPad on both Ubuntu and Arch, and I think I had
> problems on Ubuntu only). Nevertheless, I think I have some solution
> to such a problems, which might arise at any time and in any place.
>
> I'm thinking about something that could be named a "safe mode of
> work". The idea is simple: two different databases should be used for
> managing our wiki data. Currently only one database is in use, and
> thus when it fails (for some reasons), out data might be damaged or
> lost. (By the way, that's exactly my case.) In this "safe mode" the
> data would be stored in two different databases. It means that each
> save operation would have to store data in two databases, instead of
> just one. Moreover, after such a save of data its integrity would be
> checked (both databases should be equivalent to each other).
>

> [...]

I think it's more important to make the current implementation (with one
database!) as reliable as possible.

Data redundancy on application level isn't a solution here.

If there are issues with loss of data that has to be checked and fixed.

Just my $0.02

Christian

[Helge Joao]

I never had a problem with data safety in WikidPad.

It seems VERY robust to me: I am even using my Wikis synchronized to
different computers (with http://mydropbox.com) and I am often working
with a database while it is opened from two of my computers. From time
to time I rebuild the database to have the pages created on the a
different computer inside WikidPad's search.

I use an original_sqlite database.

I am using (most of the times) the newest development version of
WikidPad, since WikidPad 1.8.

:) Helge J

[Poojan Wagh]

http://www.getdropbox.com/

On Sep 3, 3:22 pm, Helge Joao <HelgeHe...@gmail.com> wrote:
> I never had a problem with data safety in WikidPad.
>
> It seems VERY robust to me: I am even using my Wikis synchronized to

> different computers (withhttp://mydropbox.com) and I am often working

[Poojan Wagh]

BTW: I've also heard good things about Winodws Live Mesh:
https://www.mesh.com/Welcome/Welcome.aspx which gives you 2GB for
free. I believe it's windows-only, though.

On Sep 3, 3:22 pm, Helge Joao <HelgeHe...@gmail.com> wrote:

> I never had a problem with data safety in WikidPad.
>
> It seems VERY robust to me: I am even using my Wikis synchronized to

> different computers (withhttp://mydropbox.com) and I am often working

[Poojan Wagh]

Finally, on the notion of syncing/backing up WikidPad data. I've been
using the distributed version control system git to sync all my
WikidPad data between computers. This is a great solution if you don't
have network access between computers (which is almost the case at my
work). Here's an excerpt from my blog entry
https://www.6waghs.net/poojanblog/wikidpad-on-multiple-computers-synced-with-git/
:

I personally like the “Original SQLite” format for storage. This
internal database has the following properties:

The data for your wiki is stored in plain text under the data
directory of the directory your Wiki config is stored in. There is
one
file per wiki word. The database to index the wiki is stored in
the
file “wikiovw.sli”.

Which is great, because I get all the features of SQLite search/
indexing, but all my pages are in a text format.

The great thing about this setup is that I can use version-control to
keep them in sync. I originally thought about using SubVersion.
However, I decided to go with a distributed version control system
instead. I took a hard look at both Mercurial and Git. I decided on
Git since it does implicit renames rather than explicit renames. This
means that I don’t have to write any hooks in WikidPad for when there
is a rename/copy of a page. Of course, I could do without this feature
in Mercurial, and it would still work. It’s just nice that Git does
provide the benefit of diff-based storage without the need for
explicit copies/renames/moves.

So, I keep two WikidPad notebooks going: a Work one, and a Personal
one. The Personal one is on all the computers at home (Windows Laptop,
Windows Desktop, FreeBSD server). It is also on a USB thumb drive. One
of the features of git is that the entire repository comes along with
a working copy. Let’s say I’m editing a WikidPad page at home that I
already edited on another computer. I don’t have to worry about
syncing. I can check in both changesets and merge later. This is a
feature that Mercurial also has.

That’s the great thing about text files: they work very well with
version-control systems.

P.S. I'm top-posting because that seems to be the default on Google
Groups.

On Sep 3, 3:22 pm, Helge Joao <HelgeHe...@gmail.com> wrote:

> I never had a problem with data safety in WikidPad.
>
> It seems VERY robust to me: I am even using my Wikis synchronized to

> different computers (withhttp://mydropbox.com) and I am often working

[TPJ]

Christian, although I see your point, I doubt that it's possible to
make any implementation reliable *enough*. My solution is based on
assumption, that the probability of two different databases crashing
in the same time is almost negligible.

[Michael Butscher]

Like Christian, I think there are not many scenarios where this would be
helpful. Only if code particular to one database type fails it would
help to have two databases, but it is much more probable that parts of
the main code fail which are shared for both databases (simply because
this is much more code than the database handling one).

Michael